Why are there so many different AMD/Intel CPUs?

For years now, I do not buy/create assemble a new computer, because I am totally overwhelmed by the options available to me.

If we agree there is 'The Paradox of Choice', it seems to make sense to have a much more limited choice between CPU models from a consumer point of view. For example, have for each year an entry, business and a pro model, add extreme for gamer and have each of these models have a version with a beefy integrated CPU.

But it seems also a good idea for the manufacturers: They have to design, test and build each of their models, create advertisement etc., like configuring their assembly lines alone costs money. Further, compilers have to generate code for a specific architecture, which means that all my software I didn't compile myself ends up using an instruction set of the lowest common CPU, not utilizing whatever I bought fully.

Apple (not a fan ;-)) shows IMHO how it is done with their Apple Silicon: Basically even I understand which CPU choice would be the right one for me. The Steam Deck is IMHO another success story: As reference hardware I know easily if I can play a game, and it is easy to know if my hardware is faster than a Steam Deck. Compare that to games with hardware requirements like 'AMD TI 5800 8GB RAM' (made up model) which makes my life miserable.

What I am looking for is fact based knowledge:

• Why does it make (commercial) sense for AMD/Intel to create so many models?
• What are their incentives?
• What would happen, if they would reduce the amount of different CPUs they offer? (Is there historical knowledge?)