Let's talk about the curious and ingenious DriveSpace, an MS-DOS program promising to double the available disk space.

Remember, this was way before Windows.

it’s 1994

Windows 3.1 came out in 1992. Yes you still were in DOS a lot back then, but it was hardly “way before windows”.

Speaking of just hunting for random stuff in DOS, my personal favorite “what’s this?” in DOS was Gorillas.bas https://en.wikipedia.org/wiki/Gorillas_(video_game)?wprov=sfti1

I agree, but Windows 3.x was more a shell on top of MSDOS and had a more niche market, and windows 95 didn't get most popular OS until late 1998. So for a lot of people that was way before windows. Also tech went a lot faster back then. Updates to an old system isn't as important if it's not connected to the world of online hackers.
- Windows 95 was also a shell on top of MSDOS. Windows NT wasn't running on top of DOS, but it was primarily for business use until Windows XP.
Permanently Deleted
I'm pretty sure I still have Gorillas on my HDD, carried with me from my families 286.
Yeah, Gorillas was awesome ... came with MSDos 6 IIRC
Gorillas, Remline and Nibbles were so instructional to a curious child with no internet!

Drivespace was what enabled me to play Baldurs Gate 1 back in the day. My specs back then:

32MB RAM
Pentium166 MMX
500ish MB drive (My 2GB went bust, so I used an old spare drive for quite a while)
16X CDROM
2X CD Burner... yarr, that made me a lot of money
3dFX Voodoo2 8MB coupled with an ATI Rage Pro
Soundblaster Live

wooo, look at mr high end over here :-)

640kb RAM 2 x 512 floppies CGA green screen / monochrome
- That wasn't my first PC, just the one on which I played BG1 and used drivespace. My 1st PC was more like the generation after yours. 386 of some sort, 2x 3.5" floppies, and a whooping 43MB harddisk.
Permanently Deleted

Between this and RAMDOUBLER my PC was ~~unstable~~ unstoppable! ;-)

And then the Internet happened and we could freely download as much ram as we wanted to!
- I remember there was one program that claimed it would update the microcode on your CPU to allow you to basically update your CPU for free to a newer type of processor, for example making your 486 operate like a pentium pro.
  
  Entirely fake, but given the miracles of the time it seemed plausible.

I used the proprietary predecessor, Stacker.

It was pretty magical. It turned my 40 MB hard drive into a (seemingly) 80 MB hard drive.

I don't remember there being a significant performance penalty, because it was presumably overshadowed by the relatively (compared to processor speed) slow disk speeds.

Both times I used either one of them, it crashed in the middle of compressing my hard drive and I had to reformat. But they worked fine the 2nd try!
"I don't remember a performance penalty, because everything was so slow it didn't matter"
Stacker was the one I used also. It really did work

These days, modern filesystem like ZFS has compression and data deduplication (identical data only stored once) support, as well as other useful features such as snapshots and copy-on-write.

The idea is still around! Apple’s APFS file system (and HFS+in its later days) support sort-of transparent compression, and on all its platforms most system files - the ones that don’t change much - are compressed to save space for user files. There’s surprisingly little documentation about this.

There’s a third party tool you can use to compress files yourself: https://github.com/RJVB/afsctool

It looks like the technical details are in this pdf: https://developer.apple.com/support/downloads/Apple-File-System-Reference.pdf

Btrfs has compression as well. It compressed my root partition to a third of it's size. It helps out with some games as well, but they usually are not as compressible. The performance impact is pretty minimal as long as you don't set the compression level excessively high.

This was the nerdiest "Choose your adventure" rpg but it was a fun read.

We used a similar program for Windows 3.11, "doublestack" or something. It did work. It did make it a lot slower. We used it on one of the drives.

Laughs in PKUNZIP.EXE

wow, that was a blast from the past... thank you

Stacker, then MS ripped off Stacker and made Doublespace, got sued and changed the compression algorithm and renamed it DriveSpace.

Couldn’t use DoubleSpace or Stacker with Windows 3.X, there was no 32bit driver so disk access was horrendously slow. Windows95 was needed to use DriveSpace with full driver support, but it was still slow and by that time hard drives had caught up with the growing size of the OS and applications somewhat and live disk compression lost popularity, particularly with the way DriveSpace did it. Storing your entire drive as a single giant file backed by FAT32 was a terrible idea and prone to corruption.

When NTFS came around and introduced transparent file compression, that pretty much ended DriveSpace style compression. All modern FS now include some kind of compression, NTFS, APFS, BTRFS, ZFS. Even HFS+ had some ability to compress similar to APFS, but wasn’t very well known.

Which came first, DriveSpace or Norton Speed Disk? I thought Norton was first.

Different tools. Speed disk was a disk defragmenter, DriveSpace was whole disk compression. The Norton tool you’d have used a lot if you used DriveSpace was Norton Disk Doctor.
- And as I recall, Norton had all the tools long before MS-DOS included them by default. It was sort of a dick move by Microsoft, the sort of thing they're famous for now.
Didn't speed disk do an optimised defragment operation, putting OS files at the start of the drive etc
I believe so yes

You wouldn't download a hard drive.

on my old Windows 95 laptop I used the drive compression to create a partition to put some games on it, worked pretty well

But look at that estimation screen! Again, rant all you wish, Microsoft knew how to handle a long running task even back in MS-DOS days. In this case, it’s estimated at 46 minutes. Great!

Meanwhile, today it's often just "beachball!". It's become a bit of a lost art.

Cool, now do Stacker.

This was a double edged sword. For a while I wanted to play with Windows 95, and my hard drive wasn't large enough. So what I did is I'd run drivespace on dos 6.22 which would double the size of the drive reported and let me install windows 95.

Big problem is that this is prior to journaling filesystems, and Windows 95 was buggy as hell. So windows 95 would crash, it would damage the File Allocation Table, the drivespace file would get corrupted, and you'd have to reinstall windows from scratch all over again.

Really frustrating era of computing, but on the other hand, something like drivespace made the impossible possible even if it was flawed, and many such technologies were coming out that were like that. Video game console emulation in the late 90s was another such thing that was like "What? This shouldn't be possible....should it?", as well as stuff like downloading video or audio, or even voice chat over a modem which is sort of insane when you think about it.

So a lot of stuff was frustrating and broken, but also miraculous and impressive. Really interesting time to be in love with computers as a hobby.