Skip Navigation
Remote View
Lemmy Support @lemmy.ml
maltfield @monero.town

How to purge an image on Lemmy? (GDPR Data Erasure)

Given a URL to an image on my lemmy instance, how can I (as an admin) permanently delete the image (and all cache/variants of the image)?

I operate a lemmy instance server. One of our users just submitted a GDPR Data Erasure request for an image. The image is orphaned, so it is not tied to any post or comment. We have a URL to the image only.

Images in lemmy are handled by the pict-rs service, which is itself distinct from lemmy. As stated in the lemmy documentation, there is a way to purge posts and comments, but there appears to be no way to purge a given image in lemmy through the WUI or lemmy API.

How can I entirely purge the image from my lemmy instance, given only the URL to the image?

6 comments

  • If it's not tied to any post or comment, is it still the user's data and subject to the GDPR? What if I were to create a post that linked to that image? Would it become my image?

  • You can call the delete endpoint with the pictrs token your lemmy instance is using to talk to pictrs.

    It's quite well documented: https://crates.io/crates/pict-rs

    • Thanks, but I'm asking because I didn't find the reference documentation especially helpful.

      It says I need the "delete token" or "alias". How do I get that for a given URL?

      I'm looking for an example that describes how to construct the commands for the API calls knowing only the URL of the image.

  • This is a big problem. At the time of writing:

    1. Users cannot delete their images on Lemmy
    2. If a user deletes their account, their images don't get deleted
    3. There is no WUI for admins to delete images on Lemmy
    4. It is very difficult for admins to find & delete images on Lemmy (via the CLI)
    5. The Lemmy team didn't bother documenting how admins can delete images on Lemmy

    How to purge images in Lemmy

    pict-rs is a third-party simple image hosting service that runs along-side Lemmy for instances that allow users to upload media.

    At the time of writing, there is no WUI for admins to find and delete images. You have to manually query the pict-rs database and execute an API call from the command-line. Worse: Lemmy has no documentation telling instance admins how to delete images 🤦

    For the purposes of this example, lets assume youre trying to delete the following image 

     undefined
        
https://monero.town/pictrs/image/001665df-3b25-415f-8a59-3d836bb68dd1.webp

    There are two API endpoints in pict-rs that can be used to delete an image

    Method One: /image/delete/{delete_token}/{alias}

    This API call is publicly-accessible, but it first requires you to obtain the images delete_token

    The delete_token is first returned by Lemmy when POSTing to the /pictrs/image endpoint 

     undefined
        
{
   "msg":"ok",
   "files":[
      {
         "file":"001665df-3b25-415f-8a59-3d836bb68dd1.webp",
         "delete_token":"d88b7f32-a56f-4679-bd93-4f334764d381"
      }
   ]
}

    Two pieces of information are returned here:

    1. file (aka the alias) is the server filename of the uploaded image
    2. delete_token is the token needed to delete the image

    Of course, if you didnt capture this images delete_token at upload-time, then you must fetch it from the postgres DB.

    First, open a shell on your running postgres container. If you installed Lemmy with docker compose, use docker compose ps to get the SERVICE name of your postgres host, and then enter it with docker exec 

     undefined
        
docker compose ps --format "table {{.Service}}\t{{.Image}}\t{{.Name}}"
docker compose exec <docker_service_name> /bin/bash

    For example: 

     undefined
        
user@host:/home/user/lemmy# docker compose ps --format "table {{.Service}}\t{{.Image}}\t{{.Name}}"
SERVICE    IMAGE                            NAME
lemmy      dessalines/lemmy:0.19.3          lemmy-lemmy-1
lemmy-ui   dessalines/lemmy-ui:0.19.3       lemmy-lemmy-ui-1
pictrs     docker.io/asonix/pictrs:0.5.4    lemmy-pictrs-1
postfix    docker.io/mwader/postfix-relay   lemmy-postfix-1
postgres   docker.io/postgres:15-alpine     lemmy-postgres-1
proxy      docker.io/library/nginx          lemmy-proxy-1
user@host:/home/user/lemmy# 

user@host:/home/user/lemmy# docker compose exec postgres /bin/bash
postgres:/#

    Connect to the database as the lemmy user 

     undefined
        
psql -U lemmy

    For example 

     undefined
        
postgres:/# psql -U lemmy
psql (15.5)
Type "help" for help.

lemmy=#

    Query for the image by the alias (the filename) 

     undefined
        
select * from image_upload where pictrs_alias = '<image_filename>';

    For example 

     undefined
        
lemmy=# select * from image_upload where pictrs_alias = '001665df-3b25-415f-8a59-3d836bb68dd1.webp';
 local_user_id | pictrs_alias | pictrs_delete_token | published 
---------------+--------------+---------------------+-----------
1149 | 001665df-3b25-415f-8a59-3d836bb68dd1.webp | d88b7f32-a56f-4679-bd93-4f334764d381 | 2024-02-07 11:10:17.158741+00
(1 row)

lemmy=#

    Now, take the pictrs_delete_token from the above output, and use it to delete the image.

    The following command should be able to be run on any computer connected to the internet. 

     undefined
        
curl -i "https://<instance_domain>/pictrs/image/delete/<pictrs_delete_token>/<image_filename>"

    For example: 

     undefined
        
user@disp9140:~$ curl -i "https://monero.town/pictrs/image/delete/d88b7f32-a56f-4679-bd93-4f334764d381/001665df-3b25-415f-8a59-3d836bb68dd1.webp"

HTTP/2 204 No Content
server: nginx
date: Fri, 09 Feb 2024 15:37:48 GMT
vary: Origin, Access-Control-Request-Method, Access-Control-Request-Headers
cache-control: private
referrer-policy: same-origin
x-content-type-options: nosniff
x-frame-options: DENY
x-xss-protection: 1; mode=block
X-Firefox-Spdy: h2
user@disp9140:~$

    ⓘ Note: If you get an incorrect_login error, then try a logging into the instance in your web browser and then b pasting the https://<instance_domain>/pictrs/image/delete/<pictrs_delete_token>/<image_filename> URL into your web browser.

    The image should be deleted.

    Method Two: /internal/purge?alias={alias}

    Alternatively, you could execute the deletion directly inside the pictrs container. This eliminates the need to fetch the delete_token.

    First, open a shell on your running pictrs container. If you installed Lemmy with docker compose, use docker compose ps to get the SERVICE name of your postgres host, and then enter it with docker exec 

     undefined
        
docker compose ps --format "table {{.Service}}\t{{.Image}}\t{{.Name}}"
docker compose exec <docker_service_name> /bin/sh

    For example: 

     undefined
        
user@host:/home/user/lemmy# docker compose ps --format "table {{.Service}}\t{{.Image}}\t{{.Name}}"
SERVICE    IMAGE                            NAME
lemmy      dessalines/lemmy:0.19.3          lemmy-lemmy-1
lemmy-ui   dessalines/lemmy-ui:0.19.3       lemmy-lemmy-ui-1
pictrs     docker.io/asonix/pictrs:0.5.4    lemmy-pictrs-1
postfix    docker.io/mwader/postfix-relay   lemmy-postfix-1
postgres   docker.io/postgres:15-alpine     lemmy-postgres-1
proxy      docker.io/library/nginx          lemmy-proxy-1
user@host:/home/user/lemmy# 

user@host:/home/user/lemmy# docker compose exec pictrs /bin/sh
~ $

    Execute the following command inside the pictrs container. 

     undefined
        
wget --server-response --post-data "" --header "X-Api-Token: ${PICTRS__SERVER__API_KEY}" "http://127.0.0.1:8080/internal/purge?alias=<image_filename>"

    For example: 

     undefined
        
~ $ wget --server-response --post-data "" --header "X-Api-Token: ${PICTRS__SERVER__API_KEY}" "http://127.0.0.1:8080/internal/purge?alias=001665df-3b25-415f-8a59-3d836bb68dd1.webp"
Connecting to 127.0.0.1:8080 (127.0.0.1:8080)
HTTP/1.1 200 OK
content-length: 67
connection: close
content-type: application/json
date: Wed, 14 Feb 2024 12:56:24 GMT

saving to 'purge?alias=001665df-3b25-415f-8a59-3d836bb68dd1.webp'
purge?alias=001665df 100% |*****************************************************************************************************************************************************************************************************************************| 67 0:00:00 ETA
'purge?alias=001665df-3b25-415f-8a59-3d836bb68dd1.webp' saved

~ $

    ⓘ Note: Theres an error in the pict-rs reference documentation. It says you can POST to /internal/delete, but that just returns 404 Not Found.

    The image should be deleted

    Further Reading

    Unfortunately, it seems that the Lemmy develoeprs are not taking these moral and legal (GDPR) risks seriously (they said it may take years before they address them), and they threatened to ban me for trying to highlight the severity of this risk, get them to tag GDPR-related bugs, and to prioritize them.

    If GDPR-compliance is important to you on the fediverse, then please provide feedback to the Lemmy developers in the GitHub links above.

    Attribution

    This comment was copied from the following article: Nightmare on Lemmy Street (A Fediverse GDPR Horror Story)

    Nightmare on Lemmy Street (A Fediverse GDPR Horror Story)
6 comments