I get very confused by temperatures, and agree with your point, I have been walking the dog in -1 in the Uk big coat and been chilled to the bone.
While on a ski holiday a few years back the temperature dropped to -26 one night, I had casual clothes on and it didn’t seem as bad