If you're in a major city theres likely a recycling centre just for old office machines. You can snag them dirt cheap, but with no Harddrive. Theyre a bit dated, but will work great as a server.
In a similar vein is to look for government auctions in town. I've got a major public university in my city, and it maintains a permanent auction warehouse. Like once a month they sell all kinds of stuff, from mini fridges to laptops by the pallet.
My city has a couple mom-and-pop type businesses doing it, I'd hazard a guess it's similar elsewhere - never heard of any 'big name' outfits doing it on any real scale.