It's honestly funny because I learned the concepts in the math classes a lot better as a result of this - it took a very thorough understanding of how to use a concept to write a program to solve it for you.
My experience almost exactly. I built the interest by making/hacking TI-83 games, then made math class programs which i never really used because i had learned the material. It was fun, eye opening, and paved a path to my career!