I've tried MA, and it worked well, but I don't have smart speakers. Tried to cast it to a tablet where I installed a DLNA enabled app, but was very unreliable.
I have a stereo hooked up to my server via optical and use MPD, just haven't hooked it into HA.
I was considering a similar setup with some small Pi devices around the house and streaming via PulseAudio either to attached amps or just bluey speakers