Deferred hashing

[smiklosovic]

Defer hashing of files until they are really needed to be uploaded, otherwise just skip. This will make backups way faster on frequent backups with large sstables.

[smiklosovic]

it is a little bit more complicated than it seems, the problem is that we are saving hashes in manifest we upload. When we take a backup and we evaluate that file is already there, we do not hash, but then we will upload manifest of a snapshot with no hash either. Then it will be missing when we restore it.

The solution is to have some kind of a "register" where all hashes are present so we will try to get a hash by not computing it but by looking into existing list of hashes.

I am not completely sure about the architecture of that, naive solution would be to go over all existing manifests and try to match file we go to upload with existing one somewhere in some manifest just for the sake of getting a hash of it (because it had to be hashed at least once) but I do not think like this is robust enough.