Question about self hosting a mail server at home - if my internet goes out at home, all mail sent to me during the outage would be undeliverable, right?
For a paid host, what would be a reliable and cheap option?
You usually can't fully self-host a mail server (like literally in your house) because most residential ISPs block port 25 to prevent spam/abuse. You can use a third party relay service like what Helm offered, but most people "self host" via a cloud provider or a web host.
If your email server is offline then mail won't be delivered. The sender's server will retry as other have noted, but how often and how many times varies. Hosting at home, even when possible, is not a great idea in my opinion.
There are endless options for hosting out there, I don't want to name names since I only really have experience with one.