That is because I had Matrix and briar confused. Matrix is the one with no forward secrecy or whatever it was called. If someone gets your Matrix key, they have all your previous messages.
No they don't, matrix has partial forward security, it does rotate the keys for messages, not for every message, but it does rotate them for all users in an encrypted room if a user has a new login so that session can't read past messages until it gets keys shared by verifying with another session, same for when a user joins the room and maybe leaves the room