YSK: Your Lemmy activities (e.g. downvotes) are far from private
Edit: obligatory explanation (thanks mods for squaring me away)...
What you see via the UI isn't "all that exists". Unlike Reddit, where everything is a black box, there are a lot more eyeballs who can see "under the hood". Any instance admin, proper or rogue, gets a ton of information that users won't normally see. The attached example demonstrates that while users will only see upvote/downvote tallies, admins can see who actually performed those actions.
Edit: To clarify, not just YOUR instance admin gets this info. This is ANY instance admin across the Fediverse.
Unless I'm missing something, in terms of federated data - no. Your local instance admin could check what posts are being read by you (Lemmy tracks this, but also could be done from the web-server level even if it didn't).
I suspect the reason why identity for likes/dislikes are attached is because that is just how the standard for ActivityStreams currently works. This makes more sense on platforms like Mastodon (or other micro-blogging services) because that's how when someone "likes" your post (which is what an upvote on Lemmy is), the author gets the "@username liked your post!" notification. Upvoting a Mastodon post from Lemmy for example should* send this notification to the author, and that's the only way that it could really work (other than a generic lemmy.world liked your post notification).
"Should" because I have yet to test it myself, this is just based off what I've gathered from the ActivityPub/ActivityStreams spec.