Design of a Residue Number System Based Linear System Solver in Hardware

Abstract

This paper is focused on error-free solution of dense linear systems using residual arithmetic in hardware. The designed Modular System uses hardware identical Residual Processors (RP)s for solving independent systems of linear congruences and combines their solutions into the solution of the given linear system. This approach uses the residue number system which is based on the Chinese remainder theorem. In order to efficiently exploit parallel processing and cooperation of the individual components, a hardware architecture of the Modular System with several RPs is designed. In order to verify the proposed architecture, a Xilinx FPGA with a MicroBlaze processor was used. Experimental results are obtained for an evaluation FPGA board with Virtex 6. Results from implementation serve for subsequent theoretical analysis of the system performance for various linear system sizes and further improvement of the system. The proposed system can be useful as a special hardware peripheral or a part of an embedded system for solving large nonsingular systems of linear equations with integer, rational or floating-point coefficients with arbitrary precision.

Appendix A: Examples

Let us have a system of 3 linear equations given by the augmented matrix:

$$\left( \begin{array}{rrrr} 3 & -1 & 2 & 1 \\ 1.5 & 3 & -2 & -1 \\ 0.5 & -1 & 1.5 & 2 \end{array} \right) $$

First, we convert the system to an equivalent system with integer coefficients. In this case, it suffices to multiply rows 2 and 3 by 2. The solution to this system is the same as before. This matrix is the input to Method 1, and thereby also to Algorithm 1:

$$(\mathbf{A|b})=\left( \begin{array}{rrrr} 3 & -1 & 2 & 1 \\ 3 & 6 & -4 & -2 \\ 1 & -2 & 3 & 4 \end{array} \right) $$

Let us now choose the set of moduli (m ₁,m ₂,m ₃)=(5,7,11) for the RNS representation. For example, the element a _2,2=6 has the RNS representation (1,6,6) modulo (5,7,11). This representation is unique modulo $M = {\prod }_{i=1}^{r}m_{i} = 5\times 7\times 11 = 385$.

Then, we compute the residues of (A | b) mod m ₁,m ₂, m ₃, which gives the augmented matrices of three systems of linear congruences (SLCs). This operation denoted in Algorithm 1 as well as in Method 1 as Input conversion.

$$(\mathbf{A|b})\bmod 5=\left( \begin{array}{cccc} 3 & 4 & 2 & 1 \\ 3 & 1 & 1 & 3 \\ 1 & 3 & 3 & 4 \end{array} \right)$$

$$(\mathbf{A|b})\bmod 7=\left( \begin{array}{cccc} 3 & 6 & 2 & 1 \\ 3 & 6 & 3 & 5 \\ 1 & 5 & 3 & 4 \end{array} \right)$$

$$(\mathbf{A|b})\bmod 11=\left( \begin{array}{cccc} 3 & 10 & 2 & 1 \\ 3 & 6 & 7 & 9 \\ 1 & 9 & 3 & 4 \end{array} \right) $$

Next, we will solve the individual SLCs in their respective moduli (SLC Solving). We will show the process for m ₂=7.

In the first elimination step, a pivot is found $a_{1,1}^{(0)}=3$. Its row index is stored in c ₁=1, and the determinant intermediate value is d ₂=3.

The pivot’s inverse² is 3⁻¹ mod 7=5 and the pivot’s row is multiplied with this inverse: $a_{1,j}^{(1)}=[a_{1,j+1}^{(0)} \times 3] \bmod 7$. The first row is thereby shifted one element to the left (no information is lost, the discarded value would be a constant 1).

$$a_{i,j}^{(0)}= \left( \begin{array}{cccc} \mathbf{3} & 6 & 2 & 1 \\ 3 & 6 & 3 & 5 \\ 1 & 5 & 3 & 4 \end{array} \right) \rightarrow \left( \begin{array}{cccc} 2 & 3 & 5 & \cdot \\ 3 & 6 & 3 & 5 \\ 1 & 5 & 3 & 4 \end{array} \right) $$

Further in this elimination step, all other rows (l=2,3) are reduced using the adjusted pivot’s row: $a_{l,j}^{(1)}= [a_{l,j+1}^{(0)} - (a_{l,1}^{(0)}\, a_{1,j}^{(1)})] \bmod 7$. Again, the elements are by this process shifted to the left, discarding unnecessary zeros.

$$a_{i,j}^{(1)}=\left( \begin{array}{cccc} 2 & 3 & 5 & \cdot \\ 0 & 1 & 4 & \cdot \\ \mathbf{3} & 0 & 6 & \cdot \end{array} \right) $$

In the second elimination step, a pivot is found $a_{3,1}^{(1)}=3$. Its row index is stored in c ₂=3, and the determinant intermediate value is d ₂=3×3 mod 7=2. The pivot’s inverse is 3⁻¹ mod 7=5 and the pivot’s row is multiplied with this inverse: $a_{3,j}^{(2)}=[a_{3,j+1}^{(1)} \times 5] \bmod 7$.

$$\left( \begin{array}{cccc} 2 & 3 & 5 & \cdot\\ 0 & 1 & 4 & \cdot\\ 0 & 2 &\cdot& \cdot \end{array} \right) $$

Then, all other rows (l=1,2) are reduced using the adjusted pivot’s row: $a_{l,j}^{(2)}= [a_{l,j+1}^{(1)} - (a_{l,1}^{(1)}\, a_{3,j}^{(2)})] \bmod 7$. Again, the elements are by this process shifted to the left, discarding unnecessary zeros.

$$a_{i,j}^{(2)}=\left( \begin{array}{cccc} 3 & 1 & \cdot & \cdot \\ \mathbf{1} & 4 & \cdot & \cdot \\ 0 & 2 & \cdot & \cdot \end{array} \right) $$

In the third and final elimination step, a pivot is found $a_{2,1}^{(2)}=1$. Its row index is stored in c ₃=2, and the determinant intermediate value is d ₂=2×1 mod 7=2. The pivot’s inverse is 1⁻¹ mod 7=1 and the pivot’s row is multiplied with this inverse: $a_{2,j}^{(3)}=[a_{2,j+1}^{(2)} \times 1] \bmod 7$.

$$\left( \begin{array}{cccc} 3 & 1 & \cdot & \cdot \\ 4 & \cdot & \cdot & \cdot \\ 0 & 2 & \cdot & \cdot \end{array} \right) $$

The other rows (l=1,3) are reduced as in the previous steps: $a_{l,j}^{(3)}= [a_{l,j+1}^{(2)} - (a_{l,1}^{(2)}\, a_{2,j}^{(3)})] \bmod 7$

$$a_{i,j}^{(3)}= \left( \begin{array}{cccc} 3 & \cdot & \cdot & \cdot \\ 4 & \cdot & \cdot & \cdot \\ 2 & \cdot & \cdot & \cdot \end{array} \right) $$

The solution y ₂ is now in the first column, but not in the correct order, since the pivots were found in the order of c=(1,3,2). Therefore, the solution is reordered to get y ₂=(3,2,4)^T.

For the same reason, the sign of the computed determinant must be corrected. In the index vector c, an odd number of pair swaps is needed to create the ordered sequence (1,2,3). Therefore, sgn(c)=−1 and the determinant’s sign is corrected d ₂=2×(−1) mod 7=5.

The solution is then multiplied by the determinant to get z ₂ = y ₂ d ₂ mod 7=(3,2,4)^T×5 mod 7=(1,3,6)^T.

Similarly, the solutions and determinants in the other moduli are computed, giving z ₁=(0,0,2)^T, d ₁=4 and z ₃=(2,1,7)^T, d ₃=8.

Next, we will perform the Output conversion, the third stage of Method 1. First, the intermediate results are written in the form of $\textbf {t}_{k}= \left [ \begin {array}{c} \textbf {z}_{k} \\ d_{k} \end {array} \right ]$

$$[\textbf{t}_{1}, \textbf{t}_{2}, \textbf{t}_{3}]= \left[ \begin{array}{ccc} 0 & 1 & 2 \\ 0 & 3 & 1 \\ 2 & 6 & 7 \\ 4 & 5 & 8 \end{array} \right] $$

The values t ₁,t ₂,t ₃ are now converted according to Algorithm 2. First, we will show the conversion of the determinant d, whose RNS digits are the last elements of t _k , i.e. $d=\det \mathbf {A}$ has the RNS representation (d ₁,d ₂,d ₃)=(4,5,8) modulo (5,7,11).

During the computation of Algorithm 2, mixed-radix digits (4,3,0) are computed, shown in boxes in the table above. (In fact, the first digit 4 is just taken from t ₁). The mixed-radix digit weights are (1,m ₁,m ₁ m ₂)=(1,5,5×7). The value of the determinant is thus d=4×1+3×5+0×5×7=19. Because $d<\frac {M}{2}$, i.e. $19<\frac {385}{2}$, it is positive.

A negative number is converted the same way. For example, the first element of z has the RNS represenatation (0,1,2):

The mixed-radix representation is thus (0,3,10), and the value is 0+3×5+10×5×7=365, which is more than $\frac {M}{2}$, therefore it is negative. The correct value can be computed by subtracting M, yielding 365−385=−20.

The same process is applied on all members of [t ₁,t ₂,t ₃], getting (z,d)^T=(−20,45,62,19)^T.

The final step is to get the solution vector $\mathbf {x}=\frac {\mathbf {z}}{d}=(-\frac {20}{19},\frac {45}{19},\frac {62}{19})^{\mathsf {T}}$. We can now verify that x is indeed the solution of the SLE (A | b):

$$\left( \begin{array}{rrr} 3 & -1 & 2 \\ 3 & 6 & -4 \\ 1 & -2 & 3 \end{array} \right).\left( \begin{array}{r} -\frac{20}{19} \\ \frac{45}{19} \\ \frac{62}{19} \end{array} \right)=\left( \begin{array}{r} 1 \\ -2 \\ 4 \end{array} \right) $$