Infinite growth is an absolutely insane bar to set for the economy.
The lowered birthrates are because we're getting ground into dust - my engineering team of twenty millennials has two folks with kids and two folks who openly plan on having kids... we're aging out of the window and it's not that we're trying and failing - most of us just don't want a fucking family. We're too fucking busy already.