It's mostly population density and specialization. You don't have time to think when you're doing everything yourself. The biggest advances come when we're able to fund the best and brightest to basically do nothing but think.
After getting into writing some hard science fiction futurism, I find it much more interesting that we have so very little perspective about where we exist within the present. Our technology is crap, we're poor as fuck, there is enormous wealth that dwarfs all the wealth on Earth and a whole lot of it is quite accessible if we tried, while we haven't even scratched the surface of the technology available within biology. Our medicine and healthcare practices are primarily based on anecdotal or correlative nonsense, low sigma test results, and cherry picked terrible science. Many of us here, myself included, are outliers that the present healthcare system fails to help. We have it better than some people in history, but worse than others. It feels like our culture has this mindset like we are the end game; no vision of the future. The only stories told are those of dystopianism. We should change that.
You're so right about healthcare. The only people who have faith in the healthcare industry have clearly never interacted with it. From the politicized researchers to the patient-facing morons it's all mostly shit all the way down.
But isn't that what genres like cyberpunk do? Technological progress (A(G)I, biotech, body modifications, true VR, you name it), but society is even shittier than now?
Sure, it is to some degree a cautious tale, but I feel there are quite a lot of near-future hard-ish scifi visions around
What we need is near-future hard-ish sci-fi visions that view the world positively or at least as capable of change. A lot more Star Trek TNG than expanse.