Very fast and exact accumulation of products

Abstract

The IFIP Working Group on Numerical Software and other scientists repeatedly requested that a future arithmetic standard should consider and specify an exact dot product (EDP) [The IFIP WG—IEEE 754R letter, dated September 4 (2007), The IFIP WG—IEEE P1788 letter, dated September 9 (2009)]. On 18 November 2009 the IEEE standards committee P1788 on interval arithmetic accepted a motion [Kulisch and Snyder (The exact dot product as basic tool for long interval arithmetic, passed on Nov 18, 2009 as official IEEE P1788 document)] for including the EDP into a future interval arithmetic standard. Actually the simplest and fastest way for computing a dot product is to compute it exactly. By pipelining, it can be computed in the time the processor needs to read the data, i.e., it comes with utmost speed. A hardware implementation of the EDP exceeds any approximate computation of the dot product in software by several orders of magnitude. By a sample illustration the paper informally specifies the implementation of the EDP on computers. While [Kulisch and Snyder (The exact dot product as basic tool for long interval arithmetic, passed on Nov 18, 2009 as official IEEE P1788 document)] defines what has to be provided, how to embed the EDP into the new standard IEEE 754, [IEEE Floating-Point Arithmetic Standard 754 (2008)] and how exceptions like NaN are to be dealt with, this article illustrates how the EDP can be implemented on computers. There is indeed no simpler way of accumulating a dot product. Any method that just computes an approximation also has to consider the relative values of the summands. This results in a more complicated method. The hardware needed for the EDP is comparable to that for a fast multiplier by an adder tree, accepted years ago and now standard technology in every modern processor. The EDP brings the same speedup for accumulations at comparable costs. In Numerical Analysis the dot product is ubiquitous. It is not merely a fundamental operation in all vector and matrix spaces. It is the EDP which makes residual correction effective. This has a direct and positive influence on all iterative solvers of systems of equations. The EDP is essential for fast long real and long interval arithmetic, as well as for assessing and managing uncertainty in computing. By operator overloading variable precision interval arithmetic is very easy to use. With it the result of every arithmetic expression can be guaranteed to a number of correct digits.