Rust @programming.dev erlend_sh @lemmyrs.org 10 mo. ago

Transitioning /r/rust to the Threadiverse

blog.erlend.sh Transitioning /r/rust to the Threadiverse

Three months ago I submitted a post to the Rust sub-reddit called 'Building a better /r/rust together'. It quickly rose to the top and ga...

You're viewing a single thread.

37 comments

The problem of which instance to host a community on is a big problem for Lemmy. Grouping is an interesting idea but it causes problems as now there are different mods and admins that control subsets of the community.

Picking a single "winner" and letting the others wither seems like the right approach and will probably happen naturally but if the original instance ever shuts down or struggles under the load you will have a mess to migrate to a new instance.

If Lemmy communities were decentralized it would make a huge difference. You could just have a single community but it could survive instances coming and going (as well as many other performance and resiliency benefits). But that would be a huge change to the underlying implementation of communities.
- The problem of which instance to host a community on is a big problem for Lemmy.
  
  Seeing the same content posted six times in six communities is a problem. It pollutes the feeds, it fragments the conversation, and prevents the natural death of low-traffic communities.
  
  Avoiding showing identical posts to a user separately seems like a very easy problem to solve.
- I think it probably makes sense to host similarly moderated content together.
  
  programming content being grouped together makes sense because it’s moderated to a similar extent between languages and communities.
  
  For example discussions/posts on rust programming and porn are moderated very differently, so they should be on different instances.
  
  I disagree. Grouping communities makes sense if they need the same admin policy. Admin policy sets a floor on what mods are required to police when it comes to things like hate speech and threats, but mods in a programming community aren't going to allow that shit anyway, so admin policy is irrelevant.
  
  I think we agree, and I didn’t know “moderated similarly” was referring to admin policy. Basically you want the people at the top to be generally aligned with the views of the community
- Yup, I'm actually interested in working on a truly decentralized lemmy. Basically, I want to use IPFS or Iroh to have it completely decentralized across all users' machines.
  
  However, the big problem remains the same: how does moderation work? With lemmy, you can always go to the instance admin if your mods suck, but if it's completely decentralized, you need some other mechanism. Also with lemmy, if a community starts sucking, there's usually enough redundancy that you can just go to another major instance and find a similar community, but if it's decentralized, I think you'll have the Reddit problem where you'll essentially have to get a large chunk of the community to move if there's an issue.
  
  So I'm not convinced that decentralization is the right way to go, at least until the problem of moderation is resolved. Maybe I'll try building a decentralized instance, which is largely intended to solve the issue of scaling, but I don't think a decentralized platform would be a good replacement for lemmy.
  
  I would also take a look at Matrix. Its rooms are decentralized (although other parts like users are not). It would be an interesting backing protocol for something like Lemmy. You could probably use a model where a community is a room, then posts are references to rooms posted into that room. Comments on posts then occur in the post room.
  
  how does moderation work?
  
  I think it works the same. As part of the community configuration you have a list of mods. Those mods can then post messages such as "ban this user" or "remove this post". If these messages are sent by a mod then they are applied by those in the community.
  
  With lemmy, you can always go to the instance admin if your mods suck, but if it’s completely decentralized, you need some other mechanism.
  
  I think removing the instance admin is actually a feature. IMHO the community is the mods to some degree. If the mods suck then you can either ask them nicely to transition to a new set of mods or you just start a new community. Basically I don't think it is valuable to add a power above mods, because then what if the instance admins suck? I think part of moving communities away from instances would be removing the power of instance admins. They can still ban communities from their instance, but they have no power over the community itself.
  
  if a community starts sucking, [...] you can just go to another major instance and find a similar community, but if it’s decentralized, [...] you’ll essentially have to get a large chunk of the community to move...
  
  I think this is a feature. It is basically democracy for communities. If the community is great then it will gather lots of people and be popular. If it starts sucking then people will leave and find alternatives. It is basically democracy. If there is one "Rust" community with fantastic mods that everyone loves then I don't see it as an issue that there aren't alternatives. But if they start upsetting some subset of the communities you will see alternative communities form and if the majority of people prefer the alternative then it will supplant the "main" community as the new biggest one.
  
  rooms are decentralized
  
  Are they? Or are they federated? I was under the impression that it worked like the rest of Matrix, which operates on ActivityPub, which is a federated network.
  
  I know encryption is run locally, but do the posts also only exist among users' devices? Or is storage centralized/federated?
  
  When I say decentralized, I mean something like BitTorrent, not Lemmy, Matrix, or Mastodon where there are admins managing instances where data is copied.
  
  As part of the community configuration you have a list of mods
  
  Sure, and what happens if one of those mods' accounts gets hacked? Can these attackers add a bunch of other mods to "take over" the community? Can they remove existing mods?
  
  what if the instance admins suck?
  
  Then use another instance. The admin changing significantly isn't all that likely since they have a financial stake in keeping the instance running to their satisfaction. It costs money to run an instance, it doesn't cost money to mod a community.
  
  then people will leave and find alternatives
  
  Sure, and that exists with lemmy as well. But one nice feature lemmy has is namespacing, where you can have multiple similar communities with the same name on different instances, and they can serve as fallbacks to others. Names matter, and they matter a lot more than I think they should, but they do matter.
  
  If !rust@programming.dev sucks, I can move to !rust@lemmy.ml or !rust@lemmy.world or whatever. On Reddit, if /r/rust sucks, I'd move to /r/rust_programming, /r/rust_official, or /r/rust_dev or something (not sure if any of those exist), and it's not exactly clear which I should pick, and new users will likely still flock to /r/rust and blame the Rust community for the online community with an official-sounding name sucking.
  
  There's no easy solution here, but I think namespacing does help, as does having the option of appealing to an admin if a high level mod account is compromised or something.
  
  I think the solution has something to do with users voting and being able to override mod actions, but I'm not sure how to structure that in a way that's unlikely to get manipulated. Perhaps requiring new mods to be approved by a majority of existing mods is an acceptable solution, which should reduce the risk of a hostile takeover. But I'm not a security expert, so I'm not sure what I may be missing.
  
  Regardless, I'm going to work on this decentralized lemmy idea as a hobby at least as long as it holds my interest. I sincerely hope someone beats me to it.
  
  the rest of Matrix, which operates on ActivityPub
  
  Matrix does not use ActivityPub, it has its own protocol.
  
  Sure, and what happens if one of those mods’ accounts gets hacked? Can these attackers add a bunch of other mods to “take over” the community?
  
  Yes. Or it depends on permissions. There have been some ideas for voting-based controls over things like adding and removing mods.
  
  The thing is that you can extend this to what if the instance admin account gets hacked? It seem better to have one point of failure (the mods) rather than two (mods + admins).
  
  what if the instance admins suck?
  
  Then use another instance.
  
  You can't do this if the community is tied to one instance. This is my point, right now you need to trust the instance admins, and sort of the mods. I think it would be better to just have mods. This means that there is a clearly defined source of control over the room rather than two levels.
  
  Names matter, and they matter a lot more than I think they should, but they do matter.
  
  You can still have names like this. The way Matrix does it is actually pretty clever. There is an internal ID for a room which should be mostly invisible to the user. But there are also local aliases. So for example you can have #rust:matrix.org. But if the matrix.org admins decide that that room is not a good target they can update the alias to a different room. These names are controlled by the instances but the room itself is not. I think overall this is a good strategy, trying to have one centralized directory is a recipe for fighting, disagreement and bad results. Pushing the directory responsibilities away from the core protocol makes a lot of sense to me.
  
  I don't think it is a perfect system but it works fairly well. Plus at the end of the day most users are going to come from whatever rust-lang.org points to. Or what they find in their search engine.
  
  Matrix does not use ActivityPub, it has its own protocol.
  
  Ah, I was misinformed. I'll have to read up more on it, because some less-than-authoritative sites claim that it is decentralized (as in, no main/client relationship like lemmy, but instead equal peers), but I'll want to verify that. Matrix claims to be decentralized, but I've seen a lot of misunderstanding on what "decentralized" actually means.
  
  If you have any good, approachable resources, I'm interested.
  
  what if the instance admin account gets hacked?
  
  The admin would probably just reset the instance from a backup. Or access the DB directly to fix things.
  
  A mod doesn't have that control, they only get the permissions granted to them.
  
  You can’t do this if the community is tied to one instance
  
  Right, which is why the Reddit model sucks. But with Lemmy, you can have multiple communities with the same name covering the same content. An enterprising user could even automatically cross-post everything between the communities with a bot, so it could very much operate as a "hot spare." I personally don't even know where most of the communities I engage with are hosted, nor do I particularly care because admins tend to only get involved when asked.
  
  So for example you can have #rust:matrix.org. But if the matrix.org admins decide that that room is not a good target they can update the alias to a different room
  
  I definitely need to read up more on Matrix then. It seems I made a lot of assumptions that just don't hold.
  
  most users are going to come from whatever rust-lang.org points to. Or what they find in their search engine.
  
  Unfortunately, most projects don't include all popular communities. Unless I missed it, rust-lang.org only mentions their own forums, Discord, and Zulip, yet /r/rust has a very significant number users engaging, and that's where I primarily engage as well, so I wouldn't be surprised if many users just assume /r/rust is an official channel (it's not).
  
  So are most users using the Rust forums and those two chat systems? The Rust forums claim ~10k users, whereas /r/rust has ~250k subscribed users when I checked today.
  
  If I search "Rust lemmy," I get !rust@lemmy.ml, and this instance doesn't show up at all in the first 2 pages of DuckDuckGo or Google results. It might get there eventually, but it's a problem any non-centralized platform is going to have, especially if there isn't a common word used in domain names. So if something non-centralized is going to have a shot at all, it needs to be blessed by the project, and even that may not be enough.
  
  I'd like to see truly decentralized platforms work, but I'm just not confident that we've really ironed out enough of the issues to really be successful. But I'm going to try anyway.
  
  claim that it is decentralized
  
  Yeah, the truth is a bit more complicated.
  
  Some objects not not decentralized. For example users are only federated. (there are some ideas brewing to improve this but right now they are not).
  
  It is decentralized between "homeservers" (sort of like instances) not end users. But this is really just an optimization and good for privacy. They do have prototypes of running a homeserver on your phone along with the client. So most clients only talk to their homeserver. But your instance works with federated and decentralized objects (depending on the type of object).
  
  But I think the key thing here is that rooms are decentralized and can survive sever losses. If I create a room on matrix.org and matrix.org disappears it can still be fully used by other servers. (As long as at least one other server was participating when matrix.org went offline, much like you need 1 seeder to keep a torrent alive.) Of course if all admins were accounts on matrix.org then the room will no longer have any admins, so it may be crippled, but if there were admins on multiple servers then the room can effectively live forever and survive any number of homeserver losses.
  
  This is very different from federated protocols like ActivityPub as you can probably tell where the death of an instance will kill any communities on it.
  
  And that's certainly something I'm worried about. If the admin of programming-dev gets tired of running the instance, it could just disappear.
  
  And I'm happy with using servers as an optimization, but IMO it should always allow users to host at least a portion of it themselves without a server. As in, everybody becomes a seeder. That way, if an instance gets hammered with a DDOS or something, the network can still survive by routing around it.
  
  Mod actions could be cryptographically signed using a private keys, and the public keys of the mods would be part of each community's metadata, updated in a way that establishes a chain of custody so only existing mods can add new mods. Each instance would independently verify that mod actions come from a legitimate mod. (I think I basically just described an implementation of NTFs representing mod privileges, BTW.)
  
  only existing mods can add new mods
  
  I prefer it to have multiple mods, ideally a majority. That way, you can't have one mod "go rogue" and add a bunch of alts or whatever, which also means a mod account getting compromised can't "go rogue."
  
  I'm less concerned about mod actions like deleting posts, banning users, etc, since mod actions should always be able to be rolled back since most decentralized systems use immutable data (so a mod action is merely data that instructs clients to ignore or prefer certain other data). However, I don't want a situation where mods become powerless because one of their accounts got compromised.
  
  I'm not concerned here about the rules for how mods are added or removed, just the technical implementation. It's easy enough to require a majority for decisions like that.
  
  There has to be a way to establish with certainty that a user taking mod actions is actually a mod. The fact that you can revert changes in a git repo doesn't make it ok for people to commit without permission, and mod actions are the same. Just allowing unauthorized users to perform mod actions would allow them to fuck up communities faster than the real mods could undo the damage.
  
  Yeah, I agree. All mod actions should be signed by their cryptographic key, and moderator cryptographic keys should probably be separate and stronger than regular user cryptographic keys. Mod actions should be rare enough that this isn't a burden to verify.
  
  One thing I'm not as sure about is if the data is decentralized as well, users would potentially be liable for illegal content. I suppose there could be a system that moderator-removed content gets removed from all regular users' devices, so maybe that's good enough. But then that makes auditing mod actions difficult, since the original data could be much harder to get.
  
  A lot of these problems aren't really technical, but rather UX when designing for a fully decentralized system.
  
  That sounds like a blockchain with signature verification against a previously established and acknowledged set of keys as consensus mechanism. Pretty reasonable, as far as use cases go.
  
  However, it doesn't solve the issue of disagreements and community splitting. If one part of the mod team decides to add another mod, but the rest doesn't, what's to prevent that part from splitting off and continuing their own version of the moderation chain? How is abuse of power handled? And in case of a split, how are community members informed?
  
  Don't get me wrong, I'm not saying it's a poor idea, I'm just saying that it won't solve the issues of community splits, and I'm not sure anything ever can.
  
  I wasn't trying to solve that particular problem, on the assumption that it has already been solved and the same solution can be adapted to the implement I proposed. Someone else who replied to me suggested something like requiring majority approval to add or remove a mod.
  
  Another possibility is for the creator of a community to be a super mod, who can add or remove regular mods, or transfer their super mod status to someone else. That scheme could easily be generalized to allow multiple super mods, or to include a whole hierarchy of mods for large communities.

You've viewed 37 comments.