A who's who of Cardassian officers and Bajoran collaborators (Darhe'el, for example) were persona non grata at best, and arrested to face charges of war crimes at worst. I can imagine that, if there were no specific incidents which could be linked to Gul Dukat, perhaps he and other Cardassian officials would be tolerated... But as he was the head of the occupation, I'm not sure this makes sense.
Is the best explanation that this is merely a matter of convenience to normalize relations between Bajor and Cardassia, or is there a plausible justification for his semi-frequent visits to DS9 and/or Bajor in the early seasons? What real historical examples are analogous to his relationship with the Bajoran (provisional) government?
The political situation in the region is incredibly complicated - if Bajor is interested in prosecuting, they risk a renewed conflict with Cardassia, which neither they nor the Federation particularly want. On top of that, many Bajorans are skeptical of the Federation and its intentions, at least in the early days.
Plus...did Dukat ever "visit" DS9 (as in, physically board the station) without explicit authorization from the command staff?
Yeah, at least in the beginning, Dukat only ever lurked nearby, menacingly. The first time he set foot on the station was to help Starfleet deal with what turned out to be the beginnings of the Maquis, during a conflict in the DMZ. That was also incredibly complicated. (Edit: and everyone was pretty pissed that he was the one Cardassia sent).
Everything on DS9 was incredibly complicated. That’s why I liked it so much. 
Staying in one spot was an asset, it helped justify recurring guests and villains if everyone’s in the same neighborhood. There’s no way the Kazon should have been bothering Voyager after Season 1 if they had been making a bee line for the Alpha Quadrant.
I could think a good scenario where small fleet of Kazon (or other pirate-like enemies) pursue Voyager far beyond where they first encountered, nipping at their heels and causing trouble sporadically... but yeah there's no reason Voyager should have been always in the heart of Kazon territory.
Add to that the fact that they weren't a very compelling bad guy in the first place and, well, we get what we got.
Maybe that's part of why the Borg never bothered with the Federation much? They weren't actually worthy of assimilation until they started developing drone separation tech.
One of my favorite fun facts to throw at people who dismissed DS9 for that reason is that TNG actually has more bottle episodes (where the ship in effectively "goes nowhere") than DS9.
I wish my dad would understand this. Trekky through and through EXCEPT he refuses to watch DS9 because in his view it CAN'T be startrek. Actually insane.
Technically he did in "Civil Defense" when they accidentally triggered the counter-insurgency subroutines on the station. Although he himself ended up getting trapped on the station that was about to blow up so he was stuck helping them disable it.
There was also the Federation-Cardassian peace treaty which preceded it, in which the “Cardassian punks fuck off” bit was negotiated and DMZ established.
Bjorans like to think that it was the gumption of the resistance that got the Cardassians to leave, and the Federation likes to let them think that (because telling them otherwise might piss the Bajorans off, and they deserve a win), but it was the Federation-Cardassian peace treaty that really got them to leave. That’s why any Cardassian you ask always says that they left for “political“ reasons. 
The Federation also has no party in the fight. It's not implausible that if Bajor were to kick off a new conflict with Cardassia over DS9, they might just leave to avoid intervening in the conflict, and then the Bajorans would be in a pickle.