I always assumed it had more to do with him smuggling goods into federation colonies or some such. He's a very good pilot because he knows how to slip between the cracks of sensor fields and how to use unconventional maneuvers to throw off anyone chasing him. Sounds like the behaviour of a smuggler to me.
Also, I'm pretty sure the pitch for Tom Paris was basically "Han Solo but Star Trek".
lol yeah I never really understood why they tried to sell Neelix as this tough, no-nonsense scrapper when they hired this guy to play the role. Star Trek has some very questionable casting sometimes. Not that Neelix isn't great, but he definitely isn't a "tough guy".
On the other hand, if they made the character correctly, that guy being a real ass-kicking chaotic-good smuggler that faced armies and got our laughing would be awesome.