What changes a hash value?

It's common knowledge that making even a minor change to an electronic file (deleting a period or adding a single letter to a Word document) will generate an entirely different MD5 or SHA-1 hash value for the file. However it's less well known exactly what actions till cause the hash value of the file to change. Craig Ball explored this issue in his blog, Ball in Your Court.

It should be readily apparent that changing the system metadata of a file, will not alter the hash value. Hash values are calculated only on the basis of the contents of the file. Changes to the file metadata will produce new hash values. You can be certain that a file's hash value will not change if it is opened from read only media.

According to Ball here's what will and will not cause a new hash value to be generated:

HASH VALUE STAYS THE SAME

1. Changing the file extension

2. Emailing a file as an attachment and then receiving it back in another email.

3. Renaming a file without opening it.

HASH VALUE CHANGES

1. Copying the text of a Word document into a new Word document.

2. Printing out a Word document [the last printed date is a meta data field].

3. Opening a file and then renaming it.

md5deep to compare hash values from two data sets

Meridian Discovery has a great step by step guide on how to use the md5deep utility to compare hash values in two sets of files. See the instructions on this web page: http://www.meridiandiscovery.com/how-to/validating-copy-results-using-md5deep/ . Meridian's instructions are excellent, but I'll provide a quick summary of them:

1. We begin by downloading md5deep from this site. , and copying the md5deep.exe file to this folder, C:\Windows\System32 , assuming you're running Windows 7. On the github site, click on the releases tab and then choose md5deep-4.4.zip. If you have a 64 bit OS, use md5deep64.exe instead of md5deep.exe

2. Press Ctrl + SHIFT, plus right click in a folder which contains your original set of files. Enter the command

md5deep -rel * > “C:\Temp\InputHashes.md5”

OR if you're using a 64 bit OS:

md5deep64 -rel * > “C:\Temp\InputHashes.md5”

3. Then presss Ctrl + SHIFT, plus right click inside a folder which contains the copy of the original set of files. As noted on the Meridian site, you enter this command:

md5deep -rel * > “C:\Temp\OutputHashes.md5”

The two files that are generated by these commands are simply lists of MD5 hash values on the left, with the corresponding file names on the right.

4. Now, as explained by Meridian, you enter two commands to generate hash values for the files which contain the lists of hash values, and put the two new hash values in a single text file. If there is any difference, it's a sign that the two data sets are not identical.

md5deep “C:\Temp\InputHashes.md5” > “C:\Temp\Comparison.txt”

md5deep “C:\Temp\OutputHashes.md5” >> “C:\Temp\Comparison.txt”

FIPS standards

The National Institute of Standards and Technology (NIST) publishes Federal Information Processing Standards (FIPS) for encryption standards and digital signatures. You can find specs for secure hash standards in this FIPS guide on the NIST site. The guide serves as a blueprint for the generation of SHA-1 and SHA-256 hash values.