Issue Tracker
- My Perfect Social Media Platform: A Mix of Lemmy and Image Board with Customizable Feed Algorithms
Have you ever wished for a social media platform that combines the best features of Lemmy and image boards with customizable feed algorithms? I know I have. Here's what my perfect social media platform would look like:
-
A mix of Lemmy and image board: My ideal platform would allow users to post both text-based content and images, using tags instead of communities. With a view like Lemmy[^1], and another grid-view like an image board[^2].
-
User curation: Unlike Lemmy, where images are difficult to find again due to the lack of tags , my ideal platform would have well-curated images with tags for easy searching[^2].
-
Advanced search could also be implemented, as suggested in the Lemmy issue #3788.
-
User trust levels and community moderation: A hierarchical trust level system, similar to Discourse’s trust levels[^3], could distribute the responsibility among users and reduce the burden on admins. Trust levels would be assigned for each community based on user activity and voting affinity with the admin, allowing admins to shape their instance according to their preferences without micromanaging every aspect of the community. This idea is also discussed in the Lemmy issue #3548.
-
Customizable feed algorithms: One of the best things about Lemmy is that users can choose their own algorithm for their home feed[^1]. My ideal platform would take this a step further by allowing users to customize their feed algorithms like in Bluesky[^4].
-
Machine learning algorithms: To make the feed even more personalized, my ideal platform would use machine learning algorithms to suggest posts to users based on their activity on the platform[^5]. For example, if a user frequently upvotes posts about cats, the platform would suggest more cat-related posts to that user.
-
One-size-fits-all image format: Image boards are known for their simple, one-size-fits-all image format[^2]. My ideal platform would adopt this format to make it easy for users to share images without worrying about formatting issues.
[^1]: Lemmy [^2]: Imageboard [^3]: Understanding Discourse Trust Levels [^4]: Bluesky custom feeds and algorithms [^5]: How to implement personalized feed ranking
-
- Viewing lemmy posts by all tends to be dominated by a few communities
cross-posted from: https://lemmy.world/post/1723295
> I've noticed that there are a few communities that tend to dominate when viewing all. Some days it gets to where looking at all isn't very different than just looking at Memes@lemmy.ml or 196@lemmy.blahaj.zone. > > > Before someone says "you can just block communities you don't want to see," it's not that I never want to see them, it's that I want to be able to have a view that shows me what is new and popular in a wide variety of communities. I appreciate seeing a few good memes in my feed. The problem is when that's all I see. Changing the sort from active to hot or top x days doesn't have much effect on which communities dominate, so that isn't the solution either. > > > "You can just subscribe to communities you like". True, but that has the effect of narrowing what I see. I'd like a view that showed me new things I never thought to subscribe to. > > > Lemmy devs - if you are reading this - it would be nice to have a feed that limited the number of posts showing up from any particular community. It could be a simple cutoff of 2 or 3 posts, or maybe some sort of weighting function to cause additional posts from the same community to appear lower in the sort order for that feed. > > > I'd love to hear what devs and other users think about this. > > Edit: To everyone saying "just sort be new" - yes, that has its uses, but it only solves part of the problem. I'd like a feed that shows me what is new and popular, but from more than just one or two communities.
- [BE] [Bug]: User deleting their account causes database locking #3649github.com [Bug]: User deleting their account causes database locking · Issue #3649 · LemmyNet/lemmy
Requirements Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a single bug? Do not put multipl...
Requirements
- [X] Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a single bug? Do not put multiple bugs in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Summary
We have a user with a few hundred comments who has crashed the site twice today while trying to delete his account.
This query ends up running for a long time and locks subsequent updates to
comment
:UPDATE "comment" SET "content" = $1, "deleted" = $2, "updated" = $3 WHERE ("comment"."creator_id" = $4) RETURNING "comment"."id", "comment"."creator_id", "comment"."post_id", "comment"."content", "comment"."removed", "comment"."published", "comment"."updated", "comment"."deleted", "comment"."ap_id", "comment"."local", "comment"."path", "comment"."distinguished", "comment"."language_id"
This was running for 8 minutes before I killed it. The user in question has 352 comments and 3073 entries in comment_like. This doesn't seem like such a large amount that there should be significant impact from a user deletion.
Steps to Reproduce
I haven't been able to reproduce this with a test user, so far only this one external user keeps causing it on our site.
I've had to disable the /api/v3/user/delete_account URL for now.
Technical Details
Logs are too noisy but this is triggered by a post to /api/v3/user/delete_account from Jerboa
Version
0.18.2
Lemmy Instance URL
lemmy.ca
- [BE] [Bug]: Remote instance RSS returns "record not found" #3650github.com [Bug]: Remote instance RSS returns "record not found" · Issue #3650 · LemmyNet/lemmy
Requirements Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a single bug? Do not put multipl...
Requirements
- [X] Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a single bug? Do not put multiple bugs in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Summary
When on the page of a user on a remote instance, the linked RSS feed returns "Record not found"
Steps to Reproduce
- Go to the page of a remote user, on a local instance (https://lemmy.world/u/dessalines@lemmy.ml)
- Click the RSS button
- The linked page returns "Record not found" (https://lemmy.world/feeds/u/dessalines@lemmy.ml.xml)
Technical Details
For example, https://lemmy.world/u/dessalines@lemmy.ml links to https://lemmy.world/feeds/u/dessalines@lemmy.ml.xml and returns "Record not found"
The page of a local user correctly returns an RSS feed (https://lemmy.ml/u/dessalines links to https://lemmy.ml/feeds/u/dessalines.xml which returns an RSS record)
The RSS feed for a remote community on a local instance returns the same "Record not found". (https://lemmy.world/feeds/c/fediverse@lemmy.ml.xml)
See lemmy-ui issue (https://github.com/LemmyNet/lemmy-ui/issues/1954) for a related user interface bug.
Version
BE 0.18.2
Lemmy Instance URL
lemmy.ml, lemmy.world, lemmy.ca, etc
- [BE] prepare-drone-federation-test.sh has some more echo output and note a… #3651github.com prepare-drone-federation-test.sh has some more echo output and note a… by RocketDerp · Pull Request #3651 · LemmyNet/lemmy
…bout the LEMMY_DATABASE_URL format
…bout the LEMMY_DATABASE_URL format
- [BE] [Bug]: Removed comments are publically visible through the api. #3652github.com [Bug]: Removed comments are publically visible through the api. · Issue #3652 · LemmyNet/lemmy
Requirements Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a single bug? Do not put multipl...
Requirements
- [X] Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a single bug? Do not put multiple bugs in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Summary
When calling api/v3/comment/list you get a list of all comments on that post. Including any removed ones. The only change removed ones have is that the "removed" field is set to true. This is a massive problem because it delegates the obscuring of removed content to the front end. You can view these despite not being being logged in. When I used a mobile app that didn't take into account this removed flag, I was met with some disgusting NSFL imagery I'd rather not have seen.
I strongly recommend obscuring the content of removed comments from the API if the user is not logged in or if the user is not a mod of the community/ not an instance owner. I understand the need to keep this information in the case of reversing moderation decisions and the modlog, but there is zero reason for non-mods and non-admins to have access to it in the normal endpoints. Furthermore, the baton should not be passed to front end developers either. The source of truth should be the backend, and the backend should enforce it.
Finally this should happen with other places where comments are listed and posts are viewed. Comments deleted by the user should not be visible to anyone in the API besides the user and maybe mods/admins. I'm not sure what other endpoints it'll apply to, but in my opinion this is paramount.
Steps to Reproduce
- Create a post in a community you moderate
- Create a comment on that post
- Remove that comment
- Open up dev tools
- Go to that post again
- Look at the http response.
The JSON response has all identifying info removed, and the "removed" flag circled.
Technical Details
n/a
Version
0.18.2
Lemmy Instance URL
No response
- [BE] Denormalize community_id into post_aggregates for a 1000x speed-up when loading posts #3653github.com Denormalize community_id into post_aggregates for a 1000x speed-up when loading posts by sunaurus · Pull Request #3653 · LemmyNet/lemmy
Credit to @phiresky for this idea, originally posted in comments of #2994 This PR adds community_id to post_aggregates (& a new index on post_aggregates) to enable joining community directly to pos...
Credit to @phiresky for this idea, originally posted in comments of #2994
This PR adds
community_id
topost_aggregates
(& a new index onpost_aggregates
) to enable joiningcommunity
directly topost_aggregates
when querying posts.On lemm.ee, this optimization speeds up the query for front page of subscribed posts ~1000x, from several seconds to to just milliseconds. You can check a before/after of query plans here: https://gist.github.com/sunaurus/856e03165bb0c0010505afeebde45230
- [BE] Change post/comment URLs to include the community #3654github.com Change post/comment URLs to include the community · Issue #3654 · LemmyNet/lemmy
Requirements Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a feature request? Do not p...
Requirements
- [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a feature request? Do not put multiple feature requests in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Is your proposal related to a problem?
at first glance, with lemmy URLs I can't tell the context of a URL off the bat. (for example, which community it is in, which post a comment is a part of).
with the URLs for any community post just being {lemmy domain}/post/{post id} and the URLs for any comment being {lemmy domain}/comment/{comment id}
Describe the solution you'd like.
Changing these URLs to include:
- the community that they were posted in
- the post that they are a comment on
The result would be:
- Posts looking like \{lemmy domain\}/c/\{community name\}/post/\{post id\}
- Comments looking like \{lemmy domain\}/c/\{community name\}/post/\{post id\}/comment/\{comment id\}
Describe alternatives you've considered.
I don't have other ideas for URL formats. This seemed the neatest to me.
Additional context
No response
- [BE] Include support for handling image limits in the API #3655github.com Include support for handling image limits in the API · Issue #3655 · LemmyNet/lemmy
Requirements Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a feature request? Do not p...
Requirements
- [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a feature request? Do not put multiple feature requests in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Is your proposal related to a problem?
Some admins want to manage the amount of storage needed for image uploads and so have instituted limits on image sizes. Examples include lemmy.ee at 100kB (https://lemm.ee/post/25065) and beehaw.org at 4000x4000 (https://github.com/LemmyNet/lemmy/issues/3473#issuecomment-1620520547).
There does not seem to be a way to set, enforce or expose these rules via the API, so these appear to be implemented in NGINX, giving a 413 Payload Too Large error in the case of lemm.ee.
As a result, when a client app is attempting to upload an offending image, all it can currently tell the user is that the image is somehow too large and that they must find out for themself what the rules are.
Describe the solution you'd like.
The ideal would be that the instance handles the required resizing for the user.
If this is not seen as an attractive approach, it would then be extremely helpful for client apps if this information could be exposed via the API, so that they could query this information before attempting an upload and automatically handle the resizing to the instance's requirements on the user's behalf.
My initial inclination is that the bounding-box limit would generally be easier for developers to work with.
Describe alternatives you've considered.
The lemm.ee link above suggests that users should use other image hosts if their files are too large, but this is not an attractive option for app developers or end users, as if either requires the developers to make a choice of third-party host on behalf of their users (which some will doubtless have reasons for disliking) or it requires the users to make choices or take actions which they may not understand. Most users in most cases would rather just see their images resized, perhaps with a note in the app to inform them that this had happened in case they wanted to deal with this differently.
Additional context
No response
- [BE] Federation tests replication round1 - demonstrate absent replication of comment deletes #3657github.com Federation tests replication round1 - demonstrate absent replication of comment deletes by RocketDerp · Pull Request #3657 · LemmyNet/lemmy
This is an urgent test addition to highlight the problem with comment deletes not replicating when a remote-server creates the comment, the home server has no code to replicate delete of comment to...
This is an urgent test addition to highlight the problem with comment deletes not replicating when a remote-server creates the comment, the home server has no code to replicate delete of comment to all the downstream subscribe servers. Gamma serves as an example of the downstream servers subscribed who are not getting the delete in 0.18.2 version.
The intention here is to put more developer eyes on https://github.com/LemmyNet/lemmy/issues/3625
- [BE] "Pay it forward" Gifting #3658github.com "Pay it forward" Gifting · Issue #3658 · LemmyNet/lemmy
Requirements Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a feature request? Do not p...
Requirements
- [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a feature request? Do not put multiple feature requests in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Is your proposal related to a problem?
Just a lack of the ability to show silent appreciation
Describe the solution you'd like.
Lemmy is not a commercial project, nor should it be. Gifts as designed by other social media are a toxic money dump that prey on users' good will. Gifts to other users should not be "purchased" from some central authority (though a case could be made that server maintainers could be the distributors of gifts, and at that point it's up to them?).
I think theres a much more wholesome way to actually have a "gift economy" - if a post or comment of mine receives a "gift" from someone, that gift goes into my inventory. The only use for that gift in my inventory is to gift it to another post.
Describe alternatives you've considered.
more or less mused upon below
Additional context
I think this can create a positive feedback loop where users can show their appreciation to each other and feel motivated to do so
But there are a couple of issues:
- Where do gifts come from? Does every user just have one of each by default? Do servers distribute them based on their own rules? Are they earned via engagement?
- If servers determine this, this could help differentiate communities from one another, but also increase "choice paralysis" when choosing a server.
- This could also allow servers to opt out of gifting entirely if they choose to keep things simpler
- If servers determine this, this could help differentiate communities from one another, but also increase "choice paralysis" when choosing a server.
- What of people who don't give back? They become gift leeches and all the gifts from the community disappear into black holes.
- Auto-regen periods defined by the server? I.e. after one week, if a user has no "thank you" gift, they will be granted one in their inventory
- What would the gifts be, and who decides?
- Do servers decide? Does lemmy only support certain gifts? If servers decide, how do we limit or support gifting between users of X server on posts of Y servers?
- [BE] [Bug]: Pagination with "Show Read Posts" disabled leads to skipped pages of content #3659github.com [Bug]: Pagination with "Show Read Posts" disabled leads to skipped pages of content · Issue #3659 · LemmyNet/lemmy
Requirements Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a single bug? Do not put multipl...
Requirements
- [X] Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a single bug? Do not put multiple bugs in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Summary
If you browse with show read posts enabled, and read every post on page 1 of content, when you navigate to page 2, it will show you page 2 of "unread" content, and you need to go back to page one.
Steps to Reproduce
- Disable "Show Read Posts"
- Browse a community
- I suggest a community with consistent content but a relatively static sort - e.g. !spaceporn@lemmy.fmhy.ml w/TopMonth
- "Read" each post on the 1st page (e.g. upvote all of them)
- Pay attention to top couple of posts on page
- Notice vote counts at bottom of page
- Browse to Page 2
- Notice large drop in post vote count between end of page 1 and page 2
- Pay attention to top couple of posts on page
- Browse back to page 1
- Notice vote counts at top of page and bottom of page aligns between the end of the original page 1 and the viewed page "2"
- Notice Page 1 content is different from original page 1 content
Technical Details
I believe this is a pagination issue when constructing the offsets used for pagination, there may need to be a mechanism to deduct or track the state of read posts.
Version
BE: 18.
Lemmy Instance URL
lemmy.fmhy.ml
- How can we improve Lemmy’s SEO so we can google “(question) lemmy” instead of relying on “(question) reddit”
cross-posted from: https://merv.news/post/26663
> most people i know use google by searching whatever question they have and including the word “reddit” at the end to find reddit threads since it currently has the most useful information. > > As Lemmy gets more and more filled with useful threads and reviews it would be great if we can collectively improve Lemmy’s SEO so just including the word lemmy in a search will show lemmy threads related to the search. > > The obscure tlds used in lemmy servers don’t help and lemmy.com currently redirects to lemm.ee. Is there a way we can improve the SEO of all instances or have lemmy.com be a aggregator of threads from many Lemmy servers?
- Lemmy should have a way to browse by domainprogramming.dev Lemmy should have a way to browse by domain - programming.dev
I miss this function from reddit. I used it often to find if a post has already been submitted. Also, it was useful to see what else was posted from this domain. I hope some day this will come to Lemmy. Examples: 1. https://www.reddit.com/domain/hillelwayne.com/ [https://www.reddit.com/domain/hillel...
I miss this function from reddit. I used it often to find if a post has already been submitted. Also, it was useful to see what else was posted from this domain. I hope some day this will come to Lemmy.
Examples:
- https://www.reddit.com/domain/hillelwayne.com/
- https://www.reddit.com/domain/hillelwayne.com/top/?sort=top&t=all
- [BE] [Bug]: Users have no way to remove abusive messages from inbox without admin intervention #3629github.com [Bug]: Users have no way to remove abusive messages from inbox without admin intervention · Issue #3629 · LemmyNet/lemmy
Requirements Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a single bug? Do not put multipl...
Requirements
- [X] Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a single bug? Do not put multiple bugs in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Summary
If you receive an abusive DM there is no way to remove it from your inbox without admin help.
Steps to Reproduce
Ideally a person would be able to delete all private messages in their inbox, regardless of whether they created them or not.
But if this is not possible, blocking the abuse account should hide all DMs from them.
Reproduction:
- Person B send messages to person A
- Person A block person B
- See person B messages still show up in inbox of person A
- Observe person A also has no way to delete person B's messages. Their stuck it person A's inbox forever unless an admin intervenes.
Technical Details
N/A
Version
0.18.2
Lemmy Instance URL
No response
- [BE] different file types #3630github.com different file types · Issue #3630 · LemmyNet/lemmy
Requirements Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a feature request? Do not p...
Requirements
- [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a feature request? Do not put multiple feature requests in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Is your proposal related to a problem?
allow for .mp3 .ogg etc. uploads
Describe the solution you'd like.
uploading audio files
Describe alternatives you've considered.
n/a
Additional context
n/a
- [BE] New notification category for new posts in communities you moderate #3631github.com New notification category for new posts in communities you moderate · Issue #3631 · LemmyNet/lemmy
Requirements Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a feature request? Do not p...
Requirements
- [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a feature request? Do not put multiple feature requests in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Is your proposal related to a problem?
As a moderator I need to know when there are new posts in the communities I moderate so I can fulfill my role.
https://lemmy.world/post/1320681
Describe the solution you'd like.
A new option in settings to receive a notification for every new post only in the communities I moderate. Current notifications for new posts
Describe alternatives you've considered.
Creating a second account subscribed only to the communities I moderate and enable new post notification on that account.
https://lemmy.world/comment/1323243
Additional context
No response
- [BE] Better video URL resolution and embedding #3633github.com Better video URL resolution and embedding · Issue #3633 · LemmyNet/lemmy
Requirements Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a feature request? Do not p...
Requirements
- [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a feature request? Do not put multiple feature requests in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Is your proposal related to a problem?
Many videos from external sites fail to properly embed on Lemmy clients. Video detection from a URL is based on opengraph tags (specifically
og:video
), which a lot of sites intentionally don't send, and should be made more intelligent to ensure better content sharing. This is a major sore point for Lemmy at the moment.Describe the solution you'd like.
To facilitate better video embedding, the
fetch_site_metadata
function needs to be improved - this way, no UI changes will be required. One easy and cheap way to do this might be to use something likeyt-dlp
, which can extract direct video URLs from indirect URLs. There are Rust wrappers available for the library.For example, consider the imgur URL
https://imgur.com/gallery/hOPRxdR
. From this, yt-dlp is able to extract the direct MP4:~ ❯ yt-dlp https://imgur.com/gallery/hOPRxdR -g https://i.imgur.com/txMlHj7.mp4
This process should fail fast. If any error is encountered, simply ditch the approach and fall back to opengraph.
Considerations
- This can be too aggressive: for example, YouTube and
v.reddit
links can be resolved, but they arem3u8
streams that can't be played by the UI in a<video>
tag. Only three formats are supported:mp4
,webm
, andogg
. This can be tuned in the command:~ ❯ yt-dlp https://imgur.com/gallery/hOPRxdR -g -f 'best[ext=webm]/best[ext=mp4]/best[ext=ogg]' https://i.imgur.com/txMlHj7.mp4
- Error handling is required in case a URL can't be selected:
~ ❯ yt-dlp https://v.redd.it/s2426qje27cb1 -g -f 'best[ext=webm]/best[ext=mp4]/best[ext=ogg]' ERROR: [Reddit] s2426qje27cb1: Requested format is not available. Use --list-formats for a list of available formats
This is a non-zero exit code, so the normal fail-fast behavior might be enough.
Describe alternatives you've considered.
I considered using
yt-dlp
to also do the downloading of a video topictrs
, but that seems like unnecessary storage use. I'm sure there are alternative solutions out there.Additional context
Sample Rust code (I am not a Rust programmer so don't judge)
```rust fn run_ytdl(input: &str) -> Result<YoutubeDlOutput, youtube_dl::Error> { YoutubeDl::new(input) .download(false) .format("best[ext=webm]/best[ext=mp4]/best[ext=ogg]") .socket_timeout("10") .run() }
fn extract_direct_url(input: &str) -> Option<String> { let output = run_ytdl(input).ok()?;
match output { YoutubeDlOutput::Playlist(playlist) => playlist.entries?.get(0)?.clone().url, YoutubeDlOutput::SingleVideo(video) => video.url } } ```
- [BE] Don't panic when scheduled tasks can't connect to database #3634github.com Don't panic when scheduled tasks can't connect to database by sunaurus · Pull Request #3634 · LemmyNet/lemmy
If a database is temporarily unavailable at the start of a scheduled task, the resulting panic will permanently crash the scheduled tasks thread. This PR replaces the panic with an error log.
If a database is temporarily unavailable at the start of a scheduled task, the resulting panic will permanently crash the scheduled tasks thread. This PR replaces the panic with an error log.
- [BE] Sort type which favors recent posts with active discussions #3635github.com Sort type which favors recent posts with active discussions · Issue #3635 · LemmyNet/lemmy
Requirements Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a feature request? Do not p...
Requirements
- [x] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a feature request? Do not put multiple feature requests in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Is your proposal related to a problem?
When I open Lemmy as an average user, I want to see posts that:
- have active discussions
- are from within the last day
- are highly upvoted
However, the
Hot
sort type shows me extremely recent posts with few votes or comments, andActive
mostly shows me posts that are between one and two days old.Describe the solution you'd like.
Create additional sort types:
Active Six Hours
Active Twelve Hours
Active Day
Active Two Days
With
Active Six Hours
, the latest comment time no longer updates after six hours past the post published time. WithActive Twelve Hours
, the latest comment time no longer updates past twelve hours after the post published time, and so on an so forth.Active Two Days
would be the current default behavior of theActive
sorting algorithm.Describe alternatives you've considered.
- Set the default sorting algorithm to
Hot
- Adjust the
Gravity
value until the average age of posts on the front page of lemmy.ml is twelve hours.
Additional context
Posts on the topic:
- [BE] Auth overhaul (access tokens, refresh tokens, api tokens) #3636github.com Auth overhaul (access tokens, refresh tokens, api tokens) by sunaurus · Pull Request #3636 · LemmyNet/lemmy
This PR is not complete (missing items detailed below), but I am submitting it already as a draft to get some early feedback. Please check the description below before checking code - I would reall...
This PR is not complete (missing items detailed below), but I am submitting it already as a draft to get some early feedback. Please check the description below before checking code - I would really appreciate feedback on the overall design which is included in the description. But comments on the partially complete code are of course welcome as well.
----
Introduction
This PR contains an overhaul of Lemmy authentication. It introduces three new authentication tokens: access tokens, refresh tokens, and api tokens (more details below).
The changes are intended to be backwards compatible - the existing /login endpoint will become deprecated but will remain operational until we are ready to remove it in a future version.
What is wrong with our current authentication?
- Auth tokens never expire: https://github.com/LemmyNet/lemmy/issues/3364
- Auth sessions can't be revoked by users
- There is no support for httpOnly cookie based auth: https://github.com/LemmyNet/lemmy-ui/issues/1252
- There is no support for api token based auth - all 3rd party apps require user passwords
- All auth tokens have full access to everything, their scope can't be limited
This PR contains intends to solve all these issues.
Proposed solution
This PR proposes to replace the existing auth token with 3 new types of tokens:
Access token
This token can be acquired with either a refresh token or an API token.
The new access token is intended to be a backwards compatible drop-in replacement for the existing auth token, with a few key differences:
- It expires within 5 minutes (so even if it leaks, it can only be abused within 5 minutes of the leak)
- It contains a
method
claim, which can be used later to limit certain activities to specific methods (for example, disallow password changes if the access token was obtained via an API token)
Refresh token
This token can be acquired using username + password (+ 2fa).
It lives in a secure httpOnly cookie (can't be read from browser js), which is limited only to the /api/v3/get_access_token path.
This is intended only for trusted web interfaces (such as lemmy-ui) and can be used to create access tokens with full access to the user. Each refresh token can be considered a separate "session". Each token records its last use time, as well as last use ip address - these values can be displayed to users in some new security UI so they get an overview of their active sessions. Each refresh token expires 2 weeks after it was last used, or when revoked manually by a user.
API token
This token must be manually created by users with a specific label and expiry date.
This is intended for 3rd party apps to avoid users from entering their passwords directly into untrusted code. The api token can be used similarly to refresh tokens to request access tokens, but the created access tokens would have limited access. Each API token will also record their last use time as well as last use ip address. API tokens expire after their user defined expiry date, or when revoked manually.
----
To summarize the general flow:
- Acquire either a refresh token (if trusted web ui) or an API token (if 3rd party app)
- Request access token using the token from step 1
- Make all API requests with access token from step 2
- If access token is close to expiry (or last request failed due to token), get a new access token (and retry last request)
- If getting access token fails due to a token error, assume the (refresh or api) token has expired and go back to step 1
Rollout plan
- Release the new logic in a minor Lemmy version
- Add a migration guide to release notes to allow app developers to migrate to the new APIs
- Update Lemmy-ui to use the new endpoints
- After some time has passed, remove the old /login endpoint in a backwards-incompatible Lemmy update
TODO in this PR
- Add refresh token list & revoke endpoints
- Add api token create & list & revoke endpoints
- Disallow some actions (new api token creation + password change + reading user e-mail?) when access token method is
Api
- Add some tests
TODO in future PRs
- Switch lemmy-ui to use new authentication
- Add security page to lemmy-ui, where users can see and revoke their sessions (refresh tokens), as well as see/revoke/create API tokens
- Add method for 3rd party apps to redirect users to an API token creation page (with a potential return_url to automatically get back to the app with the created token)
- [Closed][BE] Remove TypedBuilder from db_views and db_views_actor #3637github.com Remove TypedBuilder from db_views and db_views_actor by dullbananas · Pull Request #3637 · LemmyNet/lemmy
When re-running the first cargo clippy command in fix-clippy.sh, the build time of db_views is now 311.1s instead of 1281.9s Helps with #3610
When re-running the first cargo clippy command in fix-clippy.sh, the build time of db_views is now 311.1s instead of 1281.9s
Helps with #3610
- [BE] Opt-in remote security takedown feature #3638github.com Opt-in remote security takedown feature · Issue #3638 · LemmyNet/lemmy
Requirements Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a feature request? Do not p...
Requirements
- [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a feature request? Do not put multiple feature requests in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Is your proposal related to a problem?
hacking?
Describe the solution you'd like.
Pitch Jerboa should have an opt-in feature for servers that lets them be temporarily taken down remotely for their own safety by the Jerboa non-profit in case of an extreme security vulnerability requiring an update
Motivation This would protect the servers of those who opt-in along with the users on that server. As it is an opt-in, no one will be left annoyed. All users who join the server alongside the owner would be aware of a potential takedown at any moment and would know of its importance and how it is solely for their safety. Servers can freely opt-in or out.
Describe alternatives you've considered.
sending out a please update notice?
Additional context
mastodon got lucky that mozilla paid pentesters who reported it and it got patched before it coukd be exploited, lemmy migjt not be so fortjnate
- [BE] [Bug]: ActivePub federation incoming "Page" is not properly parsed from RawAnnouncableActivities into a AnnouncableActivities:Page, code logic to detect "Page" is not being reached #3639github.com [Bug]: ActivePub federation incoming "Page" is not properly parsed from RawAnnouncableActivities into a AnnouncableActivities:Page, code logic to detect "Page" is not being reached · Issue #3639 · LemmyNet/lemmy
Requirements Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a single bug? Do not put multipl...
Requirements
- [X] Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a single bug? Do not put multiple bugs in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Summary
lemmy_server code fails to convert RawAnnouncableActivities into a AnnouncableActivities:Page before reaching logic to reject Page.
https://github.com/LemmyNet/lemmy/blob/e9e76549a88cfbdab36f00d302cceabcaaa24f4c/crates/apub/src/activities/community/announce.rs#L47
let activity: AnnouncableActivities = self.clone().try_into()?; // This is only for sending, not receiving so we reject it. if let AnnouncableActivities::Page(_) = activity { return Err(LemmyErrorType::CannotReceivePage)?; }
The code errors on the first line with
data did not match any variant of untagged enum AnnouncableActivities
, not reaching the second line of code that would returnErr(LemmyErrorType::CannotReceivePage)
.Steps to Reproduce
- Subscribe a lemmy instance to various bots mentioned below
- Look in server error logs for
data did not match any variant of untagged enum AnnouncableActivities
- capture the raw incoming data to analyze
Example of incoming raw data causing this match problem:
RawAnnouncableActivities { id: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmit.online")), port: None, path: "/activities/announce/f513b770-c440-48f3-b0f0-21317b9e85b7", query: None, fragment: None }, actor: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmit.online")), port: None, path: "/c/comics", query: None, fragment: None }, other: {"to": Array [String("https://www.w3.org/ns/activitystreams#Public")], "object": Object {"id": String("https://lemmit.online/post/201322"), "actor": String("https://lemmit.online/u/bot"), "type": String("Page"), "attributedTo": String("https://lemmit.online/u/bot"), "to": Array [String("https://lemmit.online/c/comics"), String("https://www.w3.org/ns/activitystreams#Public")], "name": String("I guess my son qualifies as my pet as well."), "cc": Array [], "content": String("<h5>This is an automated archive made by the <a href=\"https://lemmit.online/post/14692\">Lemmit Bot</a>.</h5>\n<p>The original was posted on <a href=\"https://old.reddit.com/r/comics/comments/151el8i/i_guess_my_son_qualifies_as_my_pet_as_well/\">/r/comics</a> by <a href=\"https://old.reddit.com/u/AlloyComics\">/u/AlloyComics</a> on 2023-07-16 21:08:16.</p>\n"), "mediaType": String("text/html"), "source": Object {"content": String("##### This is an automated archive made by the [Lemmit Bot](https://lemmit.online/post/14692).\nThe original was posted on [/r/comics](https://old.reddit.com/r/comics/comments/151el8i/i_guess_my_son_qualifies_as_my_pet_as_well/) by [/u/AlloyComics](https://old.reddit.com/u/AlloyComics) on 2023-07-16 21:08:16.\n"), "mediaType": String("text/markdown")}, "attachment": Array [Object {"href": String("https://i.redd.it/q53smvggldcb1.png"), "type": String("Link")}], "commentsEnabled": Bool(true), "sensitive": Bool(false), "published": String("2023-07-17T01:58:34.158916+00:00"), "language": Object {"identifier": String("en"), "name": String("English")}, "audience": String("https://lemmit.online/c/comics")}, "cc": Array [String("https://lemmit.online/c/comics/followers")], "type": String("Announce")} }
Technical Details
It seems bods are generating these type: Page
https://lemmy.world/u/MatchThreadBot https://lemmit.online/u/bot
Example of a post that comes as type: Page and causes this problem: https://lemmit.online/post/201322
Version
BE: 0.18.2
Lemmy Instance URL
No response
- [BE] Disable unsolicited private or direct messages #3640github.com Disable unsolicited private or direct messages · Issue #3640 · LemmyNet/lemmy
Requirements Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a feature request? Do not p...
Requirements
- [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a feature request? Do not put multiple feature requests in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Is your proposal related to a problem?
Users can receive unsolicited private or direct messages (PM or DM) in their inbox.
Although a feature like OM or DM can be valuable to some, it can also lead to unwanted, abusive, commercial or otherwise unsolicited messages without the user being able to block them before they are received.
Related issue (deleting unwanted PM or DM):
https://github.com/LemmyNet/lemmy/issues/3629
Describe the solution you'd like.
Add a checkbox in settings where a user can disable a PM or DM being received
Describe alternatives you've considered.
Add the line "No DM" in my profile
Additional context
No response
- [BE] arm64 Docker builds #3641github.com arm64 Docker builds · Issue #3641 · LemmyNet/lemmy
Requirements Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a feature request? Do not p...
Requirements
- [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a feature request? Do not put multiple feature requests in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Is your proposal related to a problem?
I have just heared about the XSS volunerability and wanted to update my instance ASAP, but was left realizing that since 0.17.3, there has not been any new arm64 builds submitted to Docker Hub at all.
So just in case, I checked the docs to see if I might just be using the wrong containers, but no - all the ones I use.
Describe the solution you'd like.
I would like to see arm64 builds return. Even if they are only found on
:latest
, that'd still be enough for me - I use Watchtower to autp-update my containers anyway.Describe alternatives you've considered.
Provide arm-tagged images like older versions (i.e.
0.17.3-arm64-linux
).Additional context
No response
- [BE] Balance Instance Scores Based on Monthly Active Users #3642github.com Balance Instance Scores Based on Monthly Active Users · Issue #3642 · LemmyNet/lemmy
Requirements Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a feature request? Do not p...
Requirements
- [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a feature request? Do not put multiple feature requests in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Is your proposal related to a problem?
Yes, the proposal is related to the problem of underrepresentation of smaller instances on the main feed in Lemmy, as mentioned in issue #1026.
Describe the solution you'd like.
The solution proposed is to balance instance scores based on monthly active users instead of the size of the community. By considering the monthly active users, smaller instances will have a fair chance of being represented on the main feed, ensuring a more diverse representation of instances.
Describe alternatives you've considered.
The approach considered on issue #1026 can be particularly important in the subscribed feed, where communities should be equally represented. Or even better allow for weighted community subscription as proposed in issue #3518. This would ensure that you see more from communities that you like more.
Additional context
Balancing communities would dilute the uniqueness and individual themes of each instance. Instead of balancing communities, balancing instance scores based on monthly active users would accomplish the goal of achieving a more inclusive and diverse experience for users while still maintaining the individuality of each instance.
Considering the possibility of Meta making a Reddit alternative with millions of active users, the feed of every instance would be dominated by the communities of that instance. Implementing a balanced scoring system based on monthly active users would help prevent this domination and promote a more diverse and inclusive experience for users on Lemmy.
- [UI] Server error after updating to 0.18.0 #1530github.com Server error after updating to 0.18.0 · Issue #1530 · LemmyNet/lemmy-ui
Requirements This is a bug report, and if not, please post to https://lemmy.ml/c/lemmy_support instead. Please check to see if this issue already exists. It's a single bug. Do not report multiple b...
Requirements
- [X] This is a bug report, and if not, please post to https://lemmy.ml/c/lemmy_support instead.
- [X] Please check to see if this issue already exists.
- [X] It's a single bug. Do not report multiple bugs in one issue.
- [X] It's a frontend issue, not a backend issue; Otherwise please create an issue on the backend repo instead.
Summary
Docker setup. After updating to the 0.18.0 images for both lemmy-ui and lemmy backend, the lemmy-ui logs show a pictrs error, and the main site returns a "Server error".
FetchError: request to https://SITE_URL_REDACTED/pictrs/image/a29da3fc-b6ce-4e59-82b0-1a9c94f8faed.webp failed, reason: connect ECONNREFUSED 127.0.1.1:443 at ClientRequest.<anonymous> (/app/node_modules/node-fetch/lib/index.js:1505:11) at ClientRequest.emit (node:events:511:28) at TLSSocket.socketErrorListener (node:_http_client:495:9) at TLSSocket.emit (node:events:511:28) at emitErrorNT (node:internal/streams/destroy:151:8) at emitErrorCloseNT (node:internal/streams/destroy:116:3) at process.processTicksAndRejections (node:internal/process/task_queues:82:21) { type: 'system', errno: 'ECONNREFUSED', code: 'ECONNREFUSED' }
Steps to Reproduce
- Update docker-compose.yml to 0.18.0
- docker-compose down
- docker-compose up
- access site
Technical Details
Docker setup. After updating to the 0.18.0 images for both lemmy-ui and lemmy backend, the lemmy-ui logs show a pictrs error, and the main site returns a "Server error".
FetchError: request to https://SITE_URL_REDACTED/pictrs/image/a29da3fc-b6ce-4e59-82b0-1a9c94f8faed.webp failed, reason: connect ECONNREFUSED 127.0.1.1:443 at ClientRequest.<anonymous> (/app/node_modules/node-fetch/lib/index.js:1505:11) at ClientRequest.emit (node:events:511:28) at TLSSocket.socketErrorListener (node:_http_client:495:9) at TLSSocket.emit (node:events:511:28) at emitErrorNT (node:internal/streams/destroy:151:8) at emitErrorCloseNT (node:internal/streams/destroy:116:3) at process.processTicksAndRejections (node:internal/process/task_queues:82:21) { type: 'system', errno: 'ECONNREFUSED', code: 'ECONNREFUSED' }
Lemmy Instance Version
0.18.0
Lemmy Instance URL
No response
- [UI] Open links in a new tab setting in UI #1546github.com Open links in a new tab setting in UI by Dogeek · Pull Request #1546 · LemmyNet/lemmy-ui
related to LemmyNet/lemmy#3318
related to https://github.com/LemmyNet/lemmy/pull/3318
- [Closed][UI] Allow sorting `/community/list` output by activity/subscriber count/etc. #1941github.com Allow sorting `/community/list` output by activity/subscriber count/etc. · Issue #1941 · LemmyNet/lemmy-ui
Requirements Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support Did you check to see if this issue already exists? Is this only a feature request? Do not p...
Requirements
- [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support
- [X] Did you check to see if this issue already exists?
- [X] Is this only a feature request? Do not put multiple feature requests in one issue.
- [X] Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.
Is your proposal related to a problem?
In the web UI, when visiting the
/communities
page, I would like the ability to sort by clicking the column headers. I believe this may be a backend issue, though, as I can't seem to supply any sort values in the API call to/community/list
that have the effect I want.I have never written any Rust before, so feel free to tell me I have no idea what I'm talking about, but it would appear that the
sort
URL parameter just gets passed straight to the query builder. This request type is defined here as aSortType
, which is defined as an enum here. The values in that enum do not correspond to much of the data in thecounts
section of the/community/list
response, making me think the sorting is ultimately ignored.Furthermore, if I query the server with
&sort=Hot
one might think that the results would be sorted by theirhot_rank
, but this does not appear to be the case.Again, if I'm off base here, please point me in the right direction.
Describe the solution you'd like.
Seems the solution is to implement listing communities sorted by various count values in the API, then to expose the new sorting features through the lemmy-ui project.
Describe alternatives you've considered.
I have used the tools at https://lemmyverse.net/ to get to the data I want. There I can filter down to my instance, view its communities, and sort by things like subscriber count and activity and so on.
Additional context
No response
- [UI] Potential use of automatic image resolution for tracking pixel #1945github.com Potential use of automatic image resolution for tracking pixel · Issue #1945 · LemmyNet/lemmy-ui
Requirements This is a bug report, and if not, please post to https://lemmy.ml/c/lemmy_support instead. Please check to see if this issue already exists. It's a single bug. Do not report multiple b...
Requirements
- [X] This is a bug report, and if not, please post to https://lemmy.ml/c/lemmy_support instead.
- [X] Please check to see if this issue already exists.
- [X] It's a single bug. Do not report multiple bugs in one issue.
- [X] It's a frontend issue, not a backend issue; Otherwise please create an issue on the backend repo instead.
Summary
It's come to my attention (via a mastodon post) that due to Lemmy's automatic loading of image links, that this can pose a privacy issue due to the use "tracking pixels".
See here: https://raddle.me/f/fediverse/166674/lemmy-is-so-much-like-email-it-even-brought-back-spy-tracker
It may be worth looking into solutions to limit this as much as possible.
Steps to Reproduce
- Add an image link to an image on a web server that keeps track of all requests for that image
- Lemmy will auto load and display this image.
- Everyone who comes across said image will have their connection and browser details captured.
Technical Details
This will happen on any web browser on any device.
Lemmy Instance Version
0.18
Lemmy Instance URL
No response