Because Digimon was a tamagochi clone first and everything else second. There was never a Digimon game formula or a Digimon card game formula or a Digimon anime formula. They just made random media to sell toys without any care for consistency. That lack of consistency made it have less general appeal.
Even today it's hard to introduce people to Digimon because there really is no right starting point, it's all different media and how much you like each one depends on your personal tastes.
Pokémon had it all consistent from day one. The tcg, the games, the anime, it's basically all the same even today. You can introduce someone to Pokémon with any media because it's all so consistently transversal.
I never liked that they went back to weaker forms, as a kid that inherently made me feel like Pokemon were better because they stayed evolved. Also, I did think the evolutions were very random as a kid and didn't understand why their lines weren't more congruent like Pokemon.
Honestly there were some good episodes based on pokemon changing as they evolve and whether that's something they and their trainer actually want. Digimon Tamers played with it a little bit but not for long.
Also yeah, the evolution lines are just so fucked. I remember trying out the card game ages ago and being so confused by how different the evolution lines were from the anime.
I read an article that said that the Ghost Game show was intentionally built up as a villain of the week show, with a minimal overarching plot, because the creators were aware of how successful tiktok and short form media was.
Digimon had (and still does, though they're way less frequent than Pokemon games) fantastic games. Digimon world 3 (while grindy) was one of my favorites growing up.
90s kids just need to come to terms with the fact that Pokemon is for children. Pokemon is the highest grossing franchise in the world, by a lot. They've made almost twice as much as Starwars. They have all the resources to create any kind of game they want and they are specifically choosing to produce this quality of game because they know it sells. If you enjoy them that's great, but I don't fit their target demographic anymore so I'm out.
I'm on your side mate, I know all this and honestly don't care about Pokemon nowadays either. I just want to play my generations three to five in peace lmao