It's fine until you run out of disk space

- Once we have super fast reliable internet we'll likely have the whole computer as a service. We'll just have access terminals basically and a subscription with a login, except for the nerds who want their own physical machine.
- It'll never be fast enough. An SSD is orders of magnitude slower than RAM, which is orders of magnitude slower than cache. Internet speed is orders of magnitude slower than the slowest of hard drives, which is still way too slow to be used for anything that needs memory relatively soon.
- You can do it today, just put your swapfile on sshfs and you're done.
- So I could download more RAM?
It will crash as soon as it needs to touch the swap due to the relatively insane latency difference.
- So use a small area in memory as cache
Imagine doing this on a dial-up 56K modem
- A:SPICYMEMESMODEMSOUND.WAV
wait, didn't some tech youtubers like LTT try using cloud storage as swap/RAM? afaik they failed because of latency
- This guy used ICMP data payload as a hard drive. It kinda worked.
- Afaik they used it as redundant off-site backup
I feel like this might be a giant gaping security risk.
- So is pretty much all of the cloud services the average user already subscribes to. People still use them though.
- Obviously you should set up device mapper to encrypt the gdrive device then put the swap on the encrypted mapper device.
Oh wow, I didn't even know Gdrive offered a 1 petabyte option 😂
- They don't to my knowledge, I believe that's mounted through rclone which just usually sets the filesystem size to 1PB so that it doesn't have to try to query what the actual limit is for the various providers (and your specific plan).
- Once upon a time, Google offered unlimited drive storage as part of some GSuite tiers. They stopped offering it a while ago and have kicked most/all legacy users off of it in the past few months. It was glorious while it lasted 😢
- Yea where do you get that? I can’t see anything on their pricing page, only goes up to 2tb
Even better:
Free cloud storage that doesn't require an account and provides no limit to the volume of data stored
https://github.com/yarrick/pingfs
The image doesn't load.
- I posted that 10 months ago.
  That being said, it seems to still work for me.

Protip: Put swapfile on ramdisk for highest speed

Unironically that's how zram works
- Don't do boy zram dirty, it has a ton of utility when you have ample spare compute and limited RAM.
- Doesn't it compress the contents that it's storing to help kind of get the best of both worlds?
  You get faster storage because it's in ram still, but with it being compressed there's also "more" available?
  I could be completely mistaken though

Hopefully that swap is on an SSD, otherwise that query may not ever finish lol
\ Once you're deep into swap, things can get so slow that there's no recovering from it.

WHAT FUCKING QUERY ARE YOU RUNNING TO USE UP THAT MUCH MEMORY DAMN

In a database course I took, the teacher told a story about a company that would take three days to insert a single order. Thing was, they were the sort of company that took in one or two orders every year. When it's your whole revenue on the line, you want to make sure everything is correct. The relations in that database were checked to hell and back, and they didn't care if it took a week.
Though that would have been in the 90s, so it'd go a lot faster now.
- What did they produce? Cruiseships?
A very very badly written one no doubt..
- Why stop at just one full table scan?

Exactly how I plan to deploy LLMs on my desktop 😹

You should be able to fit a model like LLaMa2 in 64GB RAM, but output will be pretty slow if it's CPU-only. GPUs are a lot faster but you'd need at least 48GB of VRAM, for example two 3090s.
- Amazon had some promotion in the summer and they had a cheap 3060 so I grabbed that and for Stable Diffusion it was more than enough, so I thought oh... I'll try out llama as well. After 2 days of dicking around, trying to load a whack of models, I spent a couple bucks and spooled up a runpod instance. It was more affordable then I thought, definitely cheaper than buying another video card.
- *laughs in top of the line 2012 hardware 😭
I need it just for the initial load on transformers based models to then run them in 8 bit. It is ideal for that situation
- That does make a lot of sense
Same. I'm patient

I'd be lying if I said I hadn't done something similar before.

Wrote my master thesis this way - didn't have enough ram or knowledge, but plenty of time on the lab machine, so I let it do its thing over night.
Sorry, lab machine ssd.
- It gave its life for academic achievement, there is no finer death for hardware. o7

You really need to index your tables. This has all the hallways of a Cartesian cross product.

I dunno why I didn't realize you can add more swap to a system while running. Nice trick for a dire emergency.

Even better, you can swapoff swap too!
It's Linux it's made by people with a brain.

Does the OOM killer actually work for anyone? In every linux system I've used, if I run out of memory, the system simply freezes.

Absolutely can and will take action. Doesn't always kill the right process (sometimes it kills big database engines for the crime of existing), but usually gives me enough headroom to SSH back in and fix it myself.
- I have limited experience with Linux, but why is it that when my system locks up, SSH still tends to work and let me fix things remotely? Like, if the system isn't locked up, let me fix it right here and now and give me back control, if it is locked up, how is SSH working to help me?
Yes. If you have swap the system will crawl to a halt before the process is killed though, SSDs are like a thousand times slower than RAM. Swapoff and allocate a ton of memory to see it in action.
- Nvme PCIe 4 SSDs are quite fast now tho, you can get between DDR1 and DDR2 speeds from a modern SSDs. This is why Apple are using their SSDs as swap quite aggressively. I'm using a MacBook Pro with 16 GBs of RAM and my swap usage regularly goes past 20 GBs and I didn't experience any slowdown during work.
It never kicks in for me when it should, but I figured out I can force trigger it manually with the magic SysRq key (Alt+SysRq+F, needs to be enabled first), which instantly recovers my system when it starts freezing from memory pressure.
- Alt+SysRq+F, needs to be enabled first
  Do note that this opens up a security hole. Since this can kill any app at random and is not interceptable, if you leave your PC in a public place, someone could come up and press this combo a few times. Chances are, it'll kill whatever the locking app you're using.
Oh yes. I've had massive compiles (well linking) which failed because of the OOM killer, and I did exactly the same, massive swap so it will just keep going. So what if it's using disk as RAM and unusable for a few hours in the middle of the night, at least it finishes!
Yeah, default Ubuntu LTS webserver kicked the mysqld on a stupid query (but it worked on dev - all developers, someday) not too long ago...
it does for me, usually by killing my session and throwing me back to the login screen

Slow SSD issue. RAM is for chumps.

Poor man's Optane

Swap thrashing goes brrrrrrrrrrrrr

Yeah it works surprisingly well. I installed Gentoo on a 2005 era laptop a few years ago and had to keep adding zswap until Rust could compile for Firefox. Iirc it took about 12G of zswap to get it working, but it wasn’t too bad overall.

Just download more RAM.

Jokes on you I used to have a 128gb ssd just for swap in my laptop

Shoot and I thought my 30 second SQL queries were a problem

Wait until you find about S3 and s3fs.

I don't want to see the EXPLAIN for that query. This person really needs to learn more about sql, I'd wager.