What can we do to make Lemmy more resilient
What can we do to make Lemmy more resilient
What would be the best thing to do to help keep lemmy.world up? Donate? Servers? What is the best way to help?
Best I can do is upload memes and add to the load.
That's actually not a terrible idea. Lemmy really needs content. It doesn't necessarily matter what that content is, it Is just really starving for activity in general. So anything that you post is a huge help.
Lemmy needs content. Lemmy.world needs resources or even just for people to spread out into other instances. It's the fediverse after all. You don't even need to have or use Lemmy to enjoy Lemmy content.
Move to a different instance to spread the load. You can see all the same content from any other instance, but the experience will be a lot better with less lag.
Lemmy is not the same as lemmy.world.
The problem with lemmy.ml and lemmy.world is that they are just too popular and instances don't scale well.
I don't understand how a million tiny instances is supposed to scale better than a few big instances.
Caching all the data from another instance is overhead. If you're not serving that to enough people, your instance is going to create more traffic than it reduces.
1-10 person instances can't possibly help. Maybe 10,000 users on an instance is valuable for scaling.
I would indeed say 1000-5000 users instance should be the soft spot. Having a look at https://lemmy.fediverse.observer/list and filter by 1m (monthly active users) shows that 27k are on Lemmy.world, while Lemmy.ml is second with only 3,8k.
A healthier solution would be to have all the small instances (imagine the 25 biggest, so up to Lemmy.zip) to gain users, so that LW would be less critical
There are certain things that are memory intensive and CPU intensive. If you have 10k on one server doing that it really adds up. However having them across a wide range of smaller servers, its not such a big deal.
As a user, you literally lose out on nothing not being on lemmy.world. You can partake in all the same conversations, communities and everything. In fact when lemmy.world is down, you can still see everything and when it comes back up, your posts will synchronize. There's genuinely no upside to being on lemmy.world. That's the way the system was designed.
Each instance only downloads relieve content to its users. An instance with a popular community will have to handle all the posts made to that community, which will still be much smaller than all of them. While the overall load might be higher, the load on any given instance will be lower.
Images, by far the biggest bandwidth user, are directly transfered from posting instance to client and are not federated. Having more instances will spread out that load very effectively.
You can see all the same content from any other instance
Nah. I moved from ml to world to .ca, and .ca is the best. I didn't realize how much content ml and world defederated from. I love it on .ca
I moved from lemmy.ml to unilemmy who specifically don't defederate from anyone. So they treat people as adults who can decide for themselves what they want to see. If you don't like something just block it .. simple.
Helpful trick, if you go to [any instance url]/instances (for example https://discuss.tchncs.de/instances ) it will show all the instances it is federated and defederated with. (Use your browser's find in page feature to scroll down to the "blocked instances" section)
Spread out to one of the other 1300 instances maybe?
Fediverse means federated. Not one instance.
Donating to lemmy.world because it has become too big seems like a self inflicted problem.
Honestly, everyone not joining lemmy.world would help. Also, not confusing Lemmy and lemmy.world as the same thing. If you have an alt account on another server, consider making it your permanent account. It's the fediverse. You don't need to be on lemmy.world to see lemmy.world.
Honestly having a list of what makes instances different would help. I have two lemmy accounts right now and figuring out which instance to pick after Limmy.world is tough.
And I only want maximum federation. I'm willing to do my own moderation and curation, just not hosting. Though that may come later if needed.
Imo, the main difference between instances are the admin's attitudes.
Beehaw.com wants to be walled off for curation, lemmy.ml is a general server for tankies that don't want to troll and serves as a test bed for the devs, lemmy.world wants to be a general hub, sh.itjust.works also wants to be a general hub that puts more value on letting people manage their own content
Yeah, I originally made an account in like early June in Lemmy.world then I made this account recently to not add to the load. Haven’t touch other account ever since.
Two things I can see, needs to be on scalable infrastructure rather than a few hosts running docker-compose. Needs to have support for in-memory key/value stores for caching. Either of these would probably help out a bit. Donate to the developers or instance maintainer and either could happen.
It will also help to evolve towards some form of immutable governance for the instances. By this I mean an instance should be more than the individual admin(s). If such an individual was to tire off, get distracted etc, the instance does not suffer the same fate. Technical federation is one thing. Federated governance is a whole different issue. I am not advocating for formal organizations (but those would help in some cases), but rather a clear provision for instance-continuity beyond the current admins.
Me and a handful of friends started a formally registered non profit in germany. Not internet related but art related stuff and it was surprisingly easy, fast and even more surprising the regulations and requirements actually make sense. That is the way to go to secure that no admin ever goes nuts and takes an entire instance with them lmao.
I‘m fairly certain that similar organizations exist in most countries snd the process should be relatively similar.
I don't see this being a thing. In many cases, the admins own and pay for it. If they stop, it's not really like they'll just keep paying for it and have someone else run it.
Fantastic. Where do I do any of that?
Digging around... Here's to donate to the .world federation. https://www.patreon.com/mastodonworld?utm_campaign=creatorshare_fan
Easy migration. Migrating account post history or migrating community from one to another easily
I've heard that post migration is a big task, every instance has to re-appoint the posts to the new one. I'd love to have the feature though.
It'd probably also help comply with GDPR removal requests
If I create an account on a small instance and that instance shuts down, will I not lose my account and post history?
Yes. This is one of the risks picking a small instance. On the other hand, all instances were small once. Most still are. And you should help them grow instead of sitting on the largest one.
Instances that grow are much less likely to shut down. I only know a few instances that have shut down actually, so I don't think it's a large risk. We wouldn't have 1300 instances if they would shut down often.
I know it's only been a few months since Lemmy got a lot of new users. So we will see how it plays out. But for a healthy Lemmy ecosystem, there should be hundreds of active instances with lots of people.
I've been creating accounts on other servers as well so that I can protect the username and also have a certain amount of per acct specializations.
Permanently Deleted
The cost quickly adds up for small underfunded projects though. You also need to factor in how the application does scale. What kind of ingress/load balancing is required. What kind of stateful storage is required. Network policy. Resource monitoring. Config management, CI/CD pipeline. I setup a basic cluster on gke to start but haven’t gotten around to building these just yet. I’ve got a goal of attempting the most performant, scalable, and cheapest instance out there.
That’s the easy part. Next part is identifying which app endpoints are the most intensive and splitting them off into separate deployments.
The hard part is scaling the database, as I’ve only seen one tool that can autoshard effectively, but it’s for MySQL, not Postgres.
Ribbit
congrats! This doesn't answer the question at all.
Have any instances considered taking money for running advertisements? I don't mean the type where they are shoved between posts or following you down the page with flashing animated gifs, but subtle banners that may appear at the top or the side of the page.
