On October 2nd NIST announced that they picked the Keccak algorithm as the winner of the SHA-3 competition. Thus, the cryptographic hash function is prepared for future use in a wide range of applications. Keccak was created by Guido Bertoni, Joan Daemen, and Gilles Van Assche of STMicroelectronics and Michaël Peeters of NXP Semiconductors.
A cryptographic hash function is an algorithm that takes a message (usually a string of text or a binary file) and calculates a fixed size digest, which is usually represented as a hexadecimal string. Hash functions are designed in a way that the same input will generate the same output, but starting with just the output, it is not possible to calculate the input other than doing a brute force check which involves trying all possible inputs until you come up with a suitable candidate. Any hash function that violates this property, i.e. it is possible to come up with the input belonging to the hash faster than with a brute force attack is considered broken by cryptographic researchers. There are also other desired properties which are explained in the linked Wikipedia article.
Hash functions are commonly used for password verification, file identification, and integrity verification. Without going too much into technical details, suffice it to say that the security properties of hash functions are a major building block of certificate based security infrastructure and digital signatures verifying the authenticity of digital documents.
From time to time advances in cryptographic research lead to previous hash functions being broken. A famous example of a broken hash function would be MD5, which is still widely used for less critical applications. In general, use of known broken hash functions should be avoided when more secure alternatives are available.
In the last decade some critical advances in cryptographic research raised fears that soon the currently employed SHA-2 family of hash functions would be broken. In order to come up with a new cryptographic hash function as an alternative, the NIST started a competition back in 2007. The winning algorithm of the competition would be designated as a new hash primitive to be standardized for widespread use.
The competition initially had 64 entries, 51 of which made it to the first round (meaning they fulfilled the basic application guidelines by NIST) and 14 algorithms made it to the second round (the rest either got broken or had other design flaws). Of these, 5 finalists were selected based on a number of criteria. Finally, Keccak was selected as the winner.
It should be noted that the other entrants did not show clear disadvantages compared to the winner, but exhibited some theoretical similarities with previous hash functions while Keccak employed quite a few novel design elements. The rationale behind this is that in the event of SHA-2 being broken, it should be unlikely that the same approach would break SHA-3 as well.
?Keccak has the added advantage of not being vulnerable in the same ways SHA-2 might be,? says NIST computer security expert Tim Polk. ?An attack that could work on SHA-2 most likely would not work on Keccak because the two algorithms are designed so differently.?
Security researcher Bruce Schneier applauds the selection in his blog. Schneier also participated in the competition with the Skein algorithm he developed with a team of experienced cryptographers and made it to the final round. Last week he explained that even if the competition ended without a winner, the world would not be in danger, as there currently is no need for a new hash function. The currently standardized SHA-2 family is considered secure.
NIST continues to suggest using SHA-2 as it is considered secure at the time of writing. However, the availability of SHA-3 gives implementers an additional choice and serves as insurance in the event of SHA-2 being broken. The underlying algorithm of SHA-3 should lend itself to be easy to implement in embedded systems, which might be another advantage.