Does it mean nobody knows what you're saying?
Doesn't mean nobody knows that you're talking?
Doesn't mean nobody can tell two people have engaged in a conversation?
In addition to direct observations, you can make indirect inferences from many of those characteristics.
If I can observe your peer to peer traffic I know who's talking to who.
If I can observe your network, onion routing layer, I can determine who is talking to who with high probability
If I can see network traffic at all, I can determine who our members of a group, if the group messages are delivered simultaneously.
Though I think everyone assumed Google required a warrant.
Even without Google, Apple's participation and push notifications - signal still has the same capability. Simply because they've created a centralized architecture.
If you allow for servers that can't read your messages (Tor nodes and such), "serverless" messaging is quite possible. All the layers of encryption and redirection aren't great for latency, but there's no reason two phones can't be connected over Tor/Veilid.
The problem in practice, I think, is notifications. To receive notifications, you need to be online all the time. To be available on Tor all the time may help deanonimze you so you also need to shake up your connections every now and then, which requires some CPU heavy recalculations and key exchange from the network as connections are reestablished.