A lot of the punchlines are "ha ha, aren't they weird?", which hits totally different if you're the sort of person who the character is an obvious caricature of. Doubly so if the weirdness in question is associated with some form of neurodivergence. For some people - myself included - a lot of the laughs in that show feel very much like they're laughing at you, not with you. It's only funny if you identify more with the people on the other side of the joke.
That's an interesting take. I'm sure they didn't mean to hurt ND people, even if they did end up doing that. I loved the jokes despite identifying with most of the "weird" characters