For me (I use Kavita) it’s because I want to be able to just pick up whatever device is in front of me at the moment and pick up the book where I last left off even if it was on another device
I do the same thing. I’ve tried Kavita and Audiobookshelf and ended up just keeping the books on a network share and then accessing them through Calibre. I am sideloading to a Kindle though.