I'm in Winnipeg. I saw an old buddy today grabbing some beer wearing shorts and a t shirt. He walked past the cars when we left, I assume he was walking home.
It was -4c today. My kid took his gloves and jacket off at the park. He's 2. We were there for over an hour.
It's not really cold around 0c. It's definitely manageable if you're moving around.
I was on holiday at Disneyland Paris at 14 in December or January one year and it was -14/15 °C and it was the coldest place I've been and it definitely felt like it. I'm from Ireland for reference so winters are pretty mild here.
Depending on where you live, going below freezing, even down towards 0F (-18C), is common. I'd say things don't go into "too cold to go out" territory until the single digits for me.