YSK: Your Lemmy activities (e.g. downvotes) are far from private
Edit: obligatory explanation (thanks mods for squaring me away)...
What you see via the UI isn't "all that exists". Unlike Reddit, where everything is a black box, there are a lot more eyeballs who can see "under the hood". Any instance admin, proper or rogue, gets a ton of information that users won't normally see. The attached example demonstrates that while users will only see upvote/downvote tallies, admins can see who actually performed those actions.
Edit: To clarify, not just YOUR instance admin gets this info. This is ANY instance admin across the Fediverse.
I would hope this would be obvious to anyone. If your client can highlight which posts you have upvoted in the web and app UI then the fact that your user specifically upvoted that post must be recoverable from the instance server and thus must be recoverable by the instance admins. I would not expect anything different.
Only the server that you connect to so it can tell you what you voted for. It doesn't have to share who it was linked to. So voting profiles can be separated from posting as it doesn't need to be known.
Except it is not just your instance and the instance hosting the post (which would make sense). It is every instance in the federation. So if I want to see how you vote, I just need to pay few bucks for hosting and make my own instance.