It must confuse English learners to hear phrases like, "I'm home", instead of "I am at home." We don't say I'm school, or I'm post office.

Yes it does. I think it's that way because it's in locative case even though it doesn't make the word itself look any different. English sort of has cases and doesn't.

It works similarly in Latin. You don't say ad domum. You only say domum.