I spent 30+ years thinking that a pony was a baby horse rather than a smaller type of horse. You know how cats have kittens and dogs have puppies? Well I thought horses had ponies.
Even all the times that Lisa Simpson wanted a pony, I just thought it was similar to how a kid might want a puppy.