Say, if you compress some data using these LLMs, how hard it is to decompress the data again without access to the LLM used to perform the compression? Is the compression "algorithm" used by the LLM will be the same for all runs (which means you probably can reverse engineer it to created a decompressor program), or will it be different every time it compress new data?
I mean, having to download a huge LLM to decompress some data, which probably also requires GPU with big VRAM, seems a bit much.
Skimming through the linked paper, I noticed this:
Scaling beyond a certain point will deteriorate the compression performance since the model parameters need to be accounted for in the compressed output.
So it sounds like the model parameters needed to decompress the file are included in the file itself.
So, you'll have to use the same LLM to decompress the data? For example, if your friend send you an archive compressed with this LLM, then you won't be able to decompress it without downloading the same LLM?
It probably is more like the LLM is able to „pack the truck much more efficiently“ and decompression should be the same.
But I agree that the likely use-case of uploading all your files to the cloud, having it compress your files, and downloading the result which is a few kb smaller isn’t really practical time efficient or even needed at all.
Correct me if I'm wrong, but don't algorithms like Huffman or even Shannon-Fano code with blocks already pack the files as efficiently as possible? It's impossible to compress a file beyond it's entropy, and those algorithms get pretty damn close to it.
This is totally going to turn into another JBIG2 lossy compression clusterfuck isn't it...
For those who are unfamiliar, JBIG2 is a compression standard that has a dubious reputation for replacing characters incorrectly in scanned documents (so 6 could become an 8, for example) leading to potentially serious issues when scanning things like medical and legal documents, construction blueprints, etc.
It'll be interesting to see if this gets used in places where the cost of dedicated hardware out ways the bandwidth available. Video calls to Antarctica, shipping vessels, airplanes, space, etc. At least that's something that comes to mind. Could also see a next interation of CDNs using it, if the numbers check out.