Digital hardware realization of a novel adaptive ink drop spread operator and its application in modeling and classification and on-chip training

Abstract

In artificial intelligence (AI), proposing an efficient algorithm with an appropriate hardware implementation has always been a challenge because of the well-accepted fact that AI hardware implementations should ideally be comparable to biological systems in terms of hardware area. Active learning method (ALM) is a fuzzy learning algorithm inspired by human brain computations. Unlike traditional algorithms, which employ complicated computations, ALM tries to model human brain computations using qualitative and behavioral descriptions of the problem. The main computational engine in ALM is the ink drop spread (IDS) operator, but this operator imposes high memory requirements and computational costs, making the ALM algorithm and its hardware implementation unsuitable for some of the applications. This paper proposes an adaptive alternative method for implementing the IDS operator; a method which results in a marked reduction in the algorithm’s computational complexity and in the amount of memory required and hardware. To check its validity and performance, the method was used to carry out modeling and pattern classification tasks. This paper used challenging and real-world datasets and compared with well-known algorithms (adaptive neuro-fuzzy inference system and multi-layer perceptron) in software simulation and hardware implementation. Compared to traditional implementations of the ALM algorithm and other learning algorithms, the proposed FPGA implementation offers higher speed, less hardware, and improved performance, thus facilitating real-time application. Our ultimate goal in this paper was to present a hardware implementation with an on-chip training that allows it to adapt to its environment without dependency on the host system (on-chip learning).

Appendix A: Proof of convergence

As it has also been discussed, the training mode in the original ALM algorithm is batch-Mode, and in NAALM is Sample-Mode. In order to prove the convergence of ${{\text{v}}_{{\text{NP}}}}$ vector, first, we show the change in one element of this vector for training samples:

$$v_{{NPK_{I} + 1}}^{I} = v_{{NP0}}^{I} + \omega \times \mathop \sum \limits_{{t = 0}}^{{K_{I} }} e^{{ - \frac{{\left( {I - x_{t} } \right)^{2} }}{{2\sigma ^{2} }}}} \times (y_{t} ~ - ~v_{{NP_{t} }}^{{x_{t} }} ),$$

(23)

where, $({{\text{x}}_{\text{t}}},{{\text{y}}_{\text{t}}})$ is training sample, ${\text{I}}$ is the index of the vector for updating, ${{\text{K}}_I}$ is the total number of training samples in element $I$, $\omega$ is learning rate, ${{\text{x}}_{\text{t}}}$ is the input that vary between one and the quantization level of $X$ axis (${{\text{L}}_x}$). For each training sample associated with each quantization level, following equation holds:

$${k_1}+{k_2}+ \ldots +{k_{{L_x}}}=K$$

(24)

$K$ is the total number of training samples (number of iterations) in the IDS plane. For the benefit of simplicity and less computations, if ${{\text{x}}_{\text{t}}}={\text{I}}$, the previous equation can be simplified:

$$v_{{NPK_{I} + 1}}^{I} = v_{{NP0}}^{I} + \omega \times \mathop \sum \limits_{{t = 0}}^{{K_{I} }} ~(y_{t} - v_{{NP_{t} }}^{I} ).$$

(25)

If we expand the summation, a recursive equation can be obtained as follows:

$$v_{{NPK_{I} + 1}}^{I} = (1 - \omega )^{{k_{I} + 1}} \times v_{{NP0}}^{I} + \omega \times \mathop \sum \limits_{{i = 0}}^{{K_{I} }} ~(1 - \omega )^{{k_{I} - i}} \times y_{i} .$$

(26)

If ${{\text{K}}_{\text{I}}} \gg 1,$ in the previous equation, the first term of the equation tends to zero, and by assuming ${\text{m}}={{\text{k}}_{\text{I}}} - {\text{i}},$ following equation can be obtained:

$$v_{{NPK_{I} + 1}}^{I} = \omega \times \mathop \sum \limits_{{i = 0}}^{m} ~(1 - \omega )^{m} \times y_{{k_{I} - m}} .$$

(27)

This equation shows that the final output of the algorithm converges to the weighted sum of outputs. With regard to forgetting property introduced in the proposed method (NAALM), the impact of training samples that were shown later in the training phase is higher than that of training samples that were shown to the algorithm earlier. It should be mentioned that because of the fact that NAALM algorithm partition the inputs domains. Therefore, for the mentioned recursive algorithm, the standard deviation of $v_{{NP}}^{~}$ is small and because the computations are conducted in the fuzzy space (space with uncertainty), the lack of equivalency (the difference between Batch-Mode and Sample-Mode in training phase) do not significantly affect the final result.

With regard to assumptions we made about the convergence of describing vectors, the value of degree of Belief would be:

$$v_{{BLK_{I} + 1}}^{I} = v_{{BL0}}^{I} + {{\upalpha }}.$$

(28)

If we expand the summation, a recursive equation can be obtained as follows:

$$v_{{BLK_{I} + 1}}^{I} = v_{{BL0}}^{I} + \mathop \sum \limits_{{i = 0}}^{{K_{I} }} {{\upalpha }}.$$

(29)

If ${{\text{K}}_{\text{I}}} \gg 1,{\text{~}}$in the previous equation, following equation can be obtained:

$$v_{{BLK_{I} + 1}}^{I} = v_{{BL0}}^{I} + (K_{I} + 1) \times {{\upalpha }}.$$

(30)

This equation shows that as the number of training samples (or the number of iterations) increases, the value of Belief increases, and eventually our belief to the occurrence of that event converge to constant value (${K_{\text{I}}}<\infty \,~{\text{and~}}\,{{\varvec{\upalpha}}}<1$).