I am personally someone who is annoyed by NDT. But I am not convinced his actions are net negative, and think the opposite is more likely. Folks who find him insufferable and also don't develop a passion for science probably aren't dissuaded from entering the field as a whole because of this one annoying guy. However, kids who find one season of Cosmos interesting or only see 5 of his tweets may actually develop a love for science before realizing that NDT is annoying. He'd be a much better role model, reach more people, and make the science community look better if he was better, but that doesn't make him bad when there are literally anti-science advocates out there. We need all the celebrity scientists we can get as long as they aren't actively doing harm in my opinion.
What do you find annoying about him? I think he's absolutely fine, though I don't follow him too closely, so I'm probably closer to that kid who watched a season of Cosmos and saw a handful of tweets than anything.
I looked a bit into him, and it looks like, as with many (most?) celebrities, there are sexual misconduct allegations. I don't know the details of any of those cases, so I can't really tell if it's legitimate, if he's largely being targeted due to his celebrity, or a mixture of both. Regardless, I don't think that matters at all when discussing his work on TV, running a planetarium, or doing science outreach programs. Nobody is perfect, so we should seek to separate the message from the messenger.
I'm much more interested in the "ragebait debates" thing you mentioned, but unfortunately his Wikipedia article doesn't mention "controversies" or whatever. That said, this has a lot more relevance to his work as a public science figure.
Honestly just look up "Neil Degrasse Tyson annoying" and I am sure you'll find plenty of examples and summaries. But it honestly boils down to his delivery and projected self image.
He's perfect for kids, i guess, but he's always telling the simplest things with the attitude of the smartest man alive, and that's what irritates me. I understand the good he's doing aind kinda appreciate it, it's just that he's every-fucking-where.
edit: omfg, ater reading the comments here i reconsider. Fuck NDT.