[Done] Migrating images to object storage, images will be inaccessible for a bit
[Done] Migrating images to object storage, images will be inaccessible for a bit
I am performing a migration that moves the images files from disk storage to object storage. During this migration the image back-end needs to be turned off, and so images will not be accessible.
It should still be possible to comment.
I am not sure how long this will take.
This is part of a few optimizations that I plan to do before I upgrade the instance to version 0.19.4.
EDIT: Update, June 17 06:08 UTC
The transfer is working, but, after 12 hours, only ~50 GB have been transferred to Contabo's object storage (rate of about 1.16 MB/s). This is a lot slower than I imagined, and, if this rate continues, it will take about 3 more days to transfer the remaining 300 GB.
I am not sure if this slow rate is to be expected for moving files to object storage, or if something is wrong. If any of you has experience with this I would appreciate some tips! I have also asked in the Matrix chat in case someone there can provide me some feedback as well.
EDIT: Update, June 18, 07:44 UTC
Migration is done!
272.16 GB of image data has been transferred to object storage. This took about 37 hours. This means that transfer occurred at a rate of 2 MB/s. From other admin's experience, this transfer was quite slow. I am making use of Contabo as my object storage provider, and I suspect that the speed of the transfer was limited on their end. Another admin reported that they used backblaze b2 as the object storage provider and they experienced much faster transfers. All of this should have no influence on how the site performs from the end-user's point of view, I am documenting this for other admins who learn from other's experiences by searching through lemmy.
Object storage is a much cheaper way to save files in the cloud. Before this, I have stored the image files into the same SSD disk where the instance lives (rented from serverspace.io). The cost of this disk space goes up as the disk space needs increase. At the current disk size of 400 GB renting that space was costing $36/month and increased $0.9/mo for every 10 GB.
Object storage is a specialized way of saving and serving files from the cloud. Object storage is much cheaper, at €6 for 500 GB (from Contabo), and so it is a much more scalable way of storing the instance's files. The only reason I did not do this earlier was because I was not sure about how to do it and kept pushing it back. But now, it's done! I would recommend other admins to do this as early as possible, as migrating many files is more of a hassle than migrating a few.
Istill need to do a few more tasks (moving the instance to a new server with a smaller disk, upgrading the postgres database to be compatible with 0.19.4, changing the pict-rs database format from sled to postgres, and then upgrading to 0.19.4). But first I will let the instance run for a few days with object storage to make sure nothing is breaking.
Thank you for your patience :D
You're viewing a single thread.
Late to the party, but I hope that the transfer goes well!
Thanks!
Almost a day and only about 50% transferred. I was expecting it to be much faster... I just hope once it finishes it all works smoothly 😄
On a lighter note, it would take far longer if you delayed it! Just imagine some years from now, the amount of data being transferred!
Exactly! It is something that I wanted to do over a year ago, but I was not sure how... Pretty much all my knowledge about servers comes from setting up and trying not to break a lemmy instance, so this was another lesson that I was a bit scared to learn 😂 It turns out that it was very simple as the pict-rs (Lemmy's image backend) developer made a robust migration tool that automates this process. But since I don't know that beforehand, I need to triple check backups and read the instructions several times in preparation so even if it is easy and the hard work has been done by others it still requires some focus.