That's certainly a big part of it. When one needs to buy a metric crap load of CPUs, one tends to shop outside the popular defaults.
Another big reason, historically, is that Supercomputers didn't typically have any kind of non-command-line way to interact with them, and Windows needed it.
Until PowerShell and Windows 8, there were still substantial configuration options in Windows that were 100% managed by graphical packages. They could be changed by direct file edits and registry editing, but it added a lot of risk. All of the "did I make a mistake" tools were graphical and so unavailable from command line.
So any version of Windows stripped down enough to run on any super-computer cluster was going to be missing a lot of features, until around 2006.
Since Linux and Unix started as command line operating systems, both already had plenty fully featured options for Supercomputing.
Almost, the default boot drive is C:, everything gets mapped after that. So if you have a second HDD at D: and a disk reader at E:, any USBs you plug in would go to F:.