a wholly new engine would almost certainly break mods or atleast make them harder to make as janky as creation engine can be it's the best engine for modding there is and bethesda games absolutely need modding and not just cause the glitches
A new engine would just have to have a new mod API. Plenty of engines have mod APIs. Nothing's really stopping them, but they really love driving creation engine onward for some reason.
bethesda is not making a whole new engine from scratch that just isn't happening if they switched engine it'd probably just be to a licensed engine like unreal and that sucks for mod support the reality isn't creation engine vs a from scratch in house engine that supports modding just as well but with less jank it's creation engine vs unreal or something else maybe but not in house and there's no other engine out there that currently exists that's as good for modding
They don't need to make one; they can use one of the many engines that already exist and can do everything their games try to do, but with far less jank. Unless they somehow manage to insert it in regardless, which I would not put past them.
No? What I mean is they have a massive more casual audience that doesn't play a lot of games to compare it to and this is true. Yes most people who regularly play games can enjoy Bethesda games too but they don't owe their success to that audience cause if they did they would stop selling reskinned skyrim.
Or they can keep using the same engine with the same issues because gamers will definitely buy their next title en-masse despite the previously mentioned issues. Eg. Starfield
They tend to do that by tacking on new jank without removing (much) of the old stuff, though, presumably because they have base assets and scripts that they're constantly re-using. Or, differently put: As long as Papyrus will still be in the thing I seriously doubt they're giving any thought to technical debt. Already in Skyrim people rather used the UI to script stuff (because that's Flash and ActionScript is at least remotely sane and fast) but ultimately it's SKSE (that is, native dlls) for anything that isn't a lag fest.
It's not so much that CreationEngine is easy to mod, it's that it's what a gigantic community of modders are used to and have developed tooling for (you can get by with little to no use of CreationKit which is an abomination all on its own). Stockholm Syndrome at its finest or we'd have seen much more content for RedEngine which is far technically superior (and yet CDPR is abandoning it for Unreal).