Analysis, Event, Software Programs, Space and Science, Supercomputing Frontiers 2015, VR World

Supercomputing Frontiers 2015: The 101×10^2 Problem. Solution: Unums

As the processing power of the world’s fastest high-performance computers gets faster and faster, we eventually need to think about an era after the processing speed arms race, argued John Gustafson at the Supercomputing Frontiers 2015 conference in Singapore on Tuesday.

Gustafson said that the big challenge for the future of HPC is not necessarily faster processors, but more accurate processors. Using the metaphor of HPC doesn’t need a faster horse, but rather start thinking about a world “post-horse” era, Gustafson proposed moving beyond floating point rounding numbers — which he described as having become sloppy — to something called the universal number or “Unum”.

Unum, as Gustafson first proposed in his book The End of Error, is a new way to represent numbers that’s more accurate than the floating point estimate found in the IEEE 754 standard which Gustafson hopes it would ultimately replace. The IEEE 754 standard is based on the 101×10^2 floating point (which adds up to 64-bits) first introduced by L. Torres y Quevedo in Madrid in 1914.

Unums, which are 29-bits, contain metadata that allows for a longer and more in-depth answer rather than the rounding that floating point integers contain — including the overflow and underflow that goes along with it. Unums also obey algebraic laws and are safe to parallelize. With the mathematically complex problems that comes with the parallelism and sheer power found in modern HPC clusters, complex physics equations are reduced to mere “guesswork”.

The use of rounding can lead to disastrous results. Gustafson gave the example of how during the first Gulf War the 24-bit integer clock used in the Patriot missile batteries miscalculated the approach of a Scud missile by 0.34 seconds — killing 28 and injuring 100. The reason why the missile launched late is because of integer crowding. This came from the inaccuracy of the computer’s system clock due to it multiplying the time from milliseconds to seconds by multiplying 1/10. The 1/10 value was chopped after 24 decimal points. As the system had been on for 100 hours, the continued decimal chopping made the system continually less accurate. When dealing with missiles that travel hundreds of meters per second, this inaccuracy is unacceptable.

The other advantage of Unums is that due to their shorter float size, they take less external memory bandwidth to process. For a data center the largest single line item is its power bill. If power can be saved for the lowest level, for things like RAM calls, this would add up substantially in a massive data center. The US Department of Energy wants vendors to be able to produce an exascale system by 2019-2020 that uses less than 20 MW and to do this power savings has to happen everywhere.

Gustafson said that the next steps to get Unums to go “mainstream” is to convert the Mathematica C library into Unums. After that a strictly Unum compatible FPGA will need to be created. These are the first steps to the long road to a fully Unum compatible CPU.

For more on Unums, Gustafson’s book The End of Errors is worth a read.

 

 

  • David Gustafson

    Great writeup – well stated summary in a very digestible article. One small tweak is that unums are not necessarily exactly 29 bits, but the meta-data of unums allows for a flexible ‘environment’ of fraction and exponent bits. With a standard environment 29 bits is reasonable for most operations. … not completely dissimilar to quad/double/half precision IEEE floats.

  • John Gustafson

    And a correction to the “101×10^2 floating point (which adds up to 64-bits)” part: That should have been “1.01×10^2” or 101, which is how many years ago floating point was invented. It has nothing to do with 64 bits; it’s just that 64-bit floats now are the dominant size used in high-performance computing (HPC).

    The reference to “faster horse” was a quotation by Henry Ford soon after he had introduced the first commercially successful automobile: “If I had asked my customers what they wanted, they would have asked for a faster horse.” The HPC community asking for faster floating point, with all of its rounding errors and other drawbacks, is a lot like asking for a faster horse instead of asking what comes after horses as a means of transportation.

    The Mathematica notebook is written, of course, in Mathematica; it needs to be ported to C to make it practical to try the format on larger applications to see if it is ready to go mainstream.

    Many thanks to Sam Reynolds for this writeup!