Of course it was about slavery. Aka money. They were intrinsically linked, and outlawing slavery would have trashed their largely cash crop economy while the North's industrialization kept right on chugging away. The racism was more complicated but primarily served to keep the poor - free - white citizenry from realizing that the rich elite were the real enemy. Just like today, really.
Which is doubly asinine on their part when we remember that was the time industrial agriculture machines were invented, like the cotton gin, that made slavery largely unnecessary.
They just wanted to have victims to feel superior to.
If you're making $30/hr now, and you could join a trade union and make $65/hr as a journeyman, you'd do it, no question. But if you first have to take a 5 year apprenticeship where you're only making $15/hr, that's a hard sell. Eventually it will work out, but you're making good enough money now and you don't want to go through 5 years of hardship.