Least unreasonable childhood belief. There are horses. There are animals with horns. Why not a horse with a horn?
I think I actually recall a radio show where a guy thought unicorns were real up until his mid 20s. Not that he thought they were magic or anything, just that there was a horse-like animal with one horn that actually existed and was probably endangered or something.