Apart from the confederate flag itself, this show was pretty much anarchist. They spent every episode humiliating the cops and breaking any unrighteous law they could. The show treated the flag as set dressing.
They also came from a family that canonically resisted the Union during the civil war. And there's very few black people in the show whatsoever. So.
I know the hate symbol has always been a hate symbol, but if there's any show where you could say "it was a product of its time" (the 70's, btw) I think it's this one.
There's a whole episode of The Cleveland Show (as in Cleveland Brown, the black character from Family Guy) where Cleveland gets upset at his neighbor friend for flying a Confederate flag on his house and finally tries to get rid of it. When he fails, he confronts the hick neighbor and calls him a racist. The neighbor doesn't understand why Cleveland is upset and when Cleveland points to the flag the neighbor says, "What, my Dukes of Hazard flag?"
Cleveland immediately realizes he approached the problem from the completely wrong angle and drops the matter indefinitely. They continue to be friends.
Plus the black people that do appear in the show are always equals to the Hazard Boys. They never depict the Hazard Boys being anything but accepting of everyone except Boss Hog and his law dawgs.
And they were always wary of going into the next country because their sheriff actually had his shit together and would bring the pain to the Duke boys. That sheriff happens to be black.
I was never into it or pretty much anything country-related, being an Airwolf, Mission Impossible (the reboot), and MacGuyver kinda kid, but Dad liked it, and explained it to me pretty much the way you did.
Back in those times, people didn't really knew what it meant, so it got used as a "regular rebel flag", then white supremacists claimed they just flying it for "heritage" and "rebel" reasons...