Is it possible to install my own OS on a "smart" TV? Is that a thing?

Those companies paid to have their buttons on the remote. Your TV manufacturer is not going to threaten their sponsorship deal by letting you use those buttons for anything else.

you can! (at least on googles android tv, not sure about amazon's bastardized version) I use an app called button remaster, available from the play store, to switch my chromecasts youtube button to smart tube and netflix button to stremio

I don't know how remotes work, physically, but they gotta send some sort of code to the TV. If your TV's OS can intercept that message it can choose its own response mapped to whatever you want to happen. Something akin to remapping keys in your keyboard.

I would assume, anyway. I could be wrong.