9h ago

Is there a network to retrieve files by their hash?

inb4: IPFS doesn't work, unfortunately as you cannot provide the hash of an arbitrarily large file and retrieve it from the network. IPFS content IDs (CID) are a hash of the tree of chunks. Changes to chunk size can also change the hash!

Basically, I'd like to take the SHA256, SHA3, blake2, md5, of a file and either retrieve it from a network or get a list of sources for that file. Does something like that exist already or will I have to build it?

You're viewing a single thread.

16 comments

you have to fucking hope no one figures out how to backwards engineer the algorithm you choose
- Why?
  
  If two files have the same hash, you may receive the file you request by hash, or you may receive a different, possibly malicious file.
  https://en.m.wikipedia.org/wiki/Collision_attack
  Strong cryptographic hashes are resistant to such attacks, but md5 is relatively weak.
  
  Absolutely. An example of a malicious collision would be to request the file with the SHA-1 of 38762cf7f55934b34d179ae6a4c80cadccbb7f0a. But... there's two of them here.
  MD5 is so broken that its former status as a cryptographic hash function has been stripped. And efforts are underway to replace SHA-1 where it's used, since although it takes some prerequisites to intentionally create a SHA-1 collision today, it's worth remembering that "attacks always get better, they never get worse".
- I'm not sure what your concern is. I'd basically like to call a function retrieveFile(fileHash) and get bytes back. Or call retrieveFileLocations(fileHash) and get URIs back to where the file can be downloaded. Also, it'll be opensource, so nothing to reverse engineer.

16 comments