My biggest (only real) gripe with it is the "sit by and watch a civilisation die from something we could prevent inside five minutes without ever being noticed" shtick.
The Prime Directive is not a bad idea when it exists to minimise harm.
When it gets turned into a pseudo-religious dogma, where it is considered better to allow a culture to be extinguished than to risk contaminating it, that's when there are problems for me.
Zero tolerance policies ensure injustice in outlier cases. Yes, it’s unethical to interfere in a civilization’s development 99.9% of the time, but there are always exceptions. Ignoring outliers is pretending your system is above the fundamental laws of the universe.
My theory -- based on us seeing numerous violations of the Prime Directive from main characters -- is that the interpretation of the rule is "you better be willing to risk your career if you break this," not "your career is 100% over if you break this."
It's a heuristic, and a good one, but there seems to be in-universe exceptions for exceptional cases.