Skip Navigation

technology @hexbear.net TankieTanuki [he/him] @hexbear.net 3 mo. ago

How to temporarily increase webserver storage by 1-2 TB?

Your answer could potentially help me un-fuck my website!

Edit: The total capacity is currently 160 GB, so no amount of pruning will free up 1 TB. I need to mount an external share or something.

Edit2: I'm trying this but the command isn't working despite zero exit status.

34 comments

Do you have your own physical server or are renting one?
- Renting a VPS in the Netherlands.
  
  You may be able to just buy more capacity from the vendor by writing to their support. It's all containers
regarding your cgnat idea you can try to work with an ssh tunnel between your PC and the VPS to bridge them and do what you were planning to by mounting
I have been in similar situations before and there is no easy answer

I have no idea how Peertube works but if you're on Linux you might be able to turn on filesystem compression.... but if the data you're storing is already compressed (like video) that probably won't help very much

You could try sshfs like you mentioned in your post but it might be unreliable :( , maybe you could try NFS or even mount remote storage over Plan 9 protocol
- To mount an sshfs share I think I need to shell into my PC from the webserver (not the other way around) and I can't do that.
  
  Unlimited destruction upon NAT
  
  Do both hosts have internet connection via IPv6? You could try this over IPv6, usually no NAT, but your ISP might do some kind of firewalling
  
  You could also probably put up a Wireguard (VPN) connection between them since the webserver is not behind NAT
  
  Or also "forward a port" if you have access to the NATing router between your PC and webserver
I don't know what peertube can do but can you move stuff to /dev/null as a storage device?
- There's a cloud option if you don't want to self host: https://devnull-as-a-service.com/
  
  lol reminds me of the odd/even API.
- That's a neat idea, but when peertube goes to put the files back into remote storage it would fail to find them locally.
You probably have some archival shit under /var/log that you can prune. I will usually go level-by-level running du -sh {PARENT_PATH}/* to narrow things down one level at a time until I find the offender. Check under /opt and user home dirs, too. Otherwise, you're probably stuck adding another disk or pushing data out to an S3 bucket.
- Can I make a BackBlaze bucket and mount that on the filesystem?
  
  Depends on if you can free up enough space on / to install any packages you may need for the filesystem drivers. I've never had to do it either way; if you have a local rig that you can test with, that might be a start.
Alternate solution: use atop sorting by fs access and kill the process doing it.
Stop the service that’s doing the thing causing you a problem.

Fix the thing you screwed up

Restart the fixed service that was causing the problem

What could possibly go wrong?

In this case, and I’m making wild assumptions because I don’t have a peertube instance to play with, stop peertube, connect your local system up to it with a vpn to traverse your nat, mount some volume in the local system as the remote systems target for the file system move, then start peertube back up and see if it starts filling your local drive with some data.
- When I made the wrong command, the application registered hundreds of move jobs in its database. The developer said there is no way to cancel the jobs. If I restart peertube.service then it will notice the incomplete jobs and immediately resume them. I think I'm going to use btrfs next time so I can take a system snapshot before executing risky server commands.
  
  connect your local system up to it with a vpn to traverse your nat
  
  I need a little more detail on this. Here's my takeaway:
  
  Host an (e.g.) OpenVPN server on the webserver, after creating the proper certificate infrastructure.
  
  Connect as client from home PC.
  
  VPN Success. Now I can access each other on the local subnet that starts with a 10. or something.
  
  Mount home PC's NFS share on the webserver.
  
  Yeah that’s what I was thinking. Standing up openvpn especially is pretty well documented.
So if I'm reading this right, the issue is going to be that when you revive peertube, it's going to resume trying to move stuff? Is it running in a container or directly on that machine?
- If I'm understanding this correctly, and keep in mind I know nothing of peertube, I'm just an old coder tearing through their source, you could break your config. Configure it not to use object storage and restart. If can't find the storage it's trying to move stuff from, it can't move that stuff.
  
  OBJECT_STORAGE: { ENABLED: config.get<boolean>('object_storage.enabled'),
  
  Or make the target folder/file read only

You've viewed 34 comments.