E2E is complicated, if you self-host for a group, having TLS and encrypting data at rest (storage) may be enough. Get a threat model. That being said, I would recommend snikket.org which is a superset of extensions over XMPP which is the open source IM that was the base of almost every app out there. Matrix and Rocket are both alright too. Depends too on your resources, synapse requires too much RAM (or so I heard)
Yes, XMPP with proper TLS on the server side and Conversations or one of its forks (preferably fetched from F-Droid) using OMEMO encryption should be good enough.
If you are brave or paranoid, give Tox a try: https://tox.chat/