Fuzzy Min–Max Neural Network for Learning a Classifier with Symmetric Margin

Abstract

A fuzzy min–max neural network with symmetric margin (FMNWSM) is proposed in this paper. Therefore, its probability of misclassification is lower than traditional fuzzy min–max neural networks if both training and test samples are from identical probability distribution. Meanwhile, data is classified with symmetric margin by the use of a non-linear program which is solved analytically. In other words, to decrease learning time, no numerical optimization algorithm is used to solve the non-linear program. Only hyperbox expansion is performed in training phase of FMNWSM. On the contrary, in training phase of traditional fuzzy min–max neural networks, another process also is performed for each overlapped region such as (a) contraction process or (b) creating an especial node. Therefore, learning time of FMNWSM is less than that of traditional fuzzy min–max neural networks and since FMNWSM does not create any special node for overlapped regions, the space complexity of FMNWSM is better than those that create an especial node for an overlapped region. It is shown also that the test time complexity of FMNWSM is much better than that of traditional fuzzy min–max neural networks because of the use of a simpler activation function in its hyperbox node. Finally, the proposed fuzzy min–max neural networks, namely FMNWSM, is compared with some of traditional fuzzy min–max neural networks (i.e. FMNN, GFMN, FMCN and DCFMN) empirically (by using some real datasets) and also analytically to show the superiority of FMNWSM.

Appendix

Proposition 1

The behavior of FMNWSM is equivalent to the behavior of the model M1).

Proof

FMNWSM assigns applied input data $A$ to the class of hyperbox $B_q$, where

$$\begin{aligned} q=\hbox {arg }\hbox {max}_l \left\{ {\min \nolimits _j \left\{ {s_{lj} -\left| {a_j -m_{lj} } \right| } \right\} } \right\} , \end{aligned}$$

(6)

If we prove that $\hbox {arg }\hbox {min}_l \left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_l } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} =\hbox {arg }\hbox {max}_l \left\{ {\min _j \left\{ {s_{lj} -\left| {a_j -m_{lj} } \right| } \right\} } \right\} =q$ (where $\left| . \right|$ is absolute function and $||.||_\infty$ is infinite-norm), it can be said that the behavior of FMNWSM is equivalent to the behavior of the model (M1) and the proof is complete.

(a) A is in non-overlapped regions:

When $A$ is in non-overlapped regions, it is either out of every hyperboxes or in non-overlapped region of a hyperbox.

(a.1) A is out of every hyperboxes:

The proposed neural network FMNWSM assigns $A$ to the class of hyperbox $B_q$, where $q=\hbox {arg }\hbox {max}_l \left\{ {\min _j \left\{ {s_{lj} -\left| {a_j -m_{lj} } \right| } \right\} } \right\}$. If we prove that $\min _j \left\{ {s_{lj} -\left| {a_j -m_{lj} } \right| } \right\} =-\min _{\mathrm{Z}\in B_l } \left\{ {||A-Z||_\infty } \right\}$, then it can be said that FMNWSM assigns $A$ to the class of hyperbox $B_q$, where $q=\hbox {arg }\hbox {max}_l \left\{ {-\min _{\mathrm{Z}\in B_l } \left\{ {d\left( {A,Z} \right) } \right\} } \right\} =\hbox {arg }\hbox {min}_l \left\{ {\min _{\mathrm{Z}\in B_l } \left\{ {d\left( {A,Z} \right) } \right\} } \right\}$, and $d\left( {A,Z} \right) =||A-Z||_\infty$. Therefore, the behavior of the proposed neural network FMNWSM for such applied input data is equivalent to the model (M1). Thus, it suffices to prove that $-\min _{\mathrm{Z}\in B_l } \left\{ {||A-Z||_\infty } \right\} =\min _j \left\{ {s_{lj} -\left| {a_j -m_{lj} } \right| } \right\} ,$ or equivalently $\min _{\mathrm{Z}\in B_l } \left\{ {||A-Z||_\infty } \right\} =\max _j \left\{ {\left| {a_j -m_{lj} } \right| -s_{lj} } \right\} .$

We have

$$\begin{aligned} \min \nolimits _{\mathrm{Z}\in B_l } \left\{ {||A-Z||_\infty } \right\} =\min \nolimits _{\mathrm{Z}\in B_l } \left\{ {\max \nolimits _j \left\{ {\left| {a_j -z_j } \right| } \right\} } \right\} =\max \nolimits _j \left\{ {\min \nolimits _{z_{j} \in B_{lj} } \left\{ {\left| {a_j -z_j } \right| } \right\} } \right\} .\nonumber \\ \end{aligned}$$

(7)

If $a_j$ is out of the $B_{lj}$, $\min _{z_j \in B_{lj} } \left\{ {\left| {a_j -z_j } \right| } \right\} =\left| {a_j -m_{lj} } \right| -s_{lj}$. If $a_j$ is in the $B_{lj}$, $\min _{z_j \in B_{lj} } \left\{ {\left| {a_j -z_{lj} } \right| } \right\} =0 \ge \left| {a_j -m_{lj} } \right| -s_{lj}$. Moreover, since we supposed that $A$ is out of every hyperboxes, there is at-least one $j$ such that $a_j$ be out of the $B_{lj}$ (for each $l)$, and we have for such $j$, $\left| {a_j -m_{lj} } \right| -s_{lj} >0$. Therefore, from Eq. (7), it can be stated that $\min _{\mathrm{Z}\in B_l } \left\{ {||A-Z||_\infty } \right\} =\max _j \left\{ {\left| {a_j -m_{lj} } \right| -s_{lj} } \right\}$.

(a.2) A is in non-overlapped region of a hyperbox, i.e. $B_q:$

We have

$$\begin{aligned} -\min \nolimits _{\mathrm{Z}\in B_q } \left\{ {||A-Z||_\infty } \right\} =0\le \min \nolimits _j \left\{ {s_{qj} -\left| {a_j -m_{qj} } \right| } \right\} . \end{aligned}$$

(8)

Since we supposed that $A$ is in non-overlapped regions, for each $i\ne q$, $A$ is out of $B_i$. Therefore, from part (a.1) of this proof, for each $i\ne q$:

$$\begin{aligned} -\min \nolimits _{\mathrm{Z}\in B_i } \left\{ {||A-Z||_\infty } \right\} =\min \nolimits _j \left\{ {s_{ij} -\left| {a_j -m_{ij} } \right| } \right\} <0. \end{aligned}$$

(9)

Thus, from Eqs. (8) and (9), $\hbox {arg }\hbox {max}_l \left\{ {\min \nolimits _j \left\{ {s_{lj} -\left| {a_j -m_{lj} } \right| } \right\} } \right\} =\hbox {arg }\hbox {max}_l \left\{ {-\min _{\mathrm{Z}\in B_l } \left\{ {||A-Z||_\infty } \right\} } \right\} = \hbox {arg }\hbox {min}_l \left\{ {\min _{\mathrm{Z}\in B_l } \left\{ {d\left( {A,Z} \right) } \right\} } \right\} =q$.

Therefore, since the proposed neural network FMNWSM, assigns $A$ to the class of hyperbox $B_q$, where $q=\hbox {arg }\hbox {max}_l \left\{ {\min _j \left\{ {s_{lj} -\left| {a_j -m_{lj} } \right| } \right\} } \right\}$, it can be said that FMNWSM assigns $A$ to the class of hyperbox $B_q$, where $q=\hbox {arg }\hbox {min}_l \left\{ {\min _{\mathrm{Z}\in B_l } \left\{ {d\left( {A,Z} \right) } \right\} } \right\}$, and $d\left( {A,Z} \right) =||A-Z||_\infty$.

Therefore, from part (a.1) and (a.2) of this proof, if $A$ is in non-overlapped regions, FMNWSM assigns it to the class of hyperbox $B_q$, where $q=\hbox {arg }\hbox {min}_l \left\{ {\min _{\mathrm{Z}\in B_l } \left\{ {d\left( {A,Z} \right) } \right\} } \right\}$ and $d\left( {A,Z} \right) =||A-Z||_\infty$. Since $A$ is in non-overlapped regions, $\min _{\mathrm{Z}\in B_l } \left\{ {d\left( {A,Z} \right) } \right\} =\min _{\mathrm{Z}\in \hbox {closure}\left( {SR_l } \right) } \left\{ {d\left( {A,Z} \right) } \right\}$. Therefore, $q=\hbox {arg }\hbox {min}_l \left\{ {\min _{\mathrm{Z}\in \hbox {closure}\left( {SR_l } \right) } \left\{ {d\left( {A,Z} \right) } \right\} } \right\}$.

b) A is in an overlapped region, i.e. $\bigcap \nolimits _{l=d}^h B_l \quad \left( {1\le d<h\le n} \right) :$

From proposition 2, if $A$ is in the $B_l$, or in other words, if $A\cap B_l \ne \emptyset ,$

$$\begin{aligned} \min \nolimits _j \left\{ {s_{lj} -\left| {a_j -m_{lj} } \right| } \right\} =\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_l } \right) } \left\{ {||A-Z||_\infty } \right\} \ge 0, \end{aligned}$$

(10)

where $\bar{b}_l =\left\{ {Z\left| {Z\notin B_l } \right. } \right\}$. Moreover, For each $l$, if $A$ is out of $B_l$ or $A\mathop \cap \nolimits ^ B_l =\emptyset$,

$$\begin{aligned} \min \nolimits _j \left\{ {s_{lj} -\left| {a_j -m_{lj} } \right| } \right\} <0. \end{aligned}$$

(11)

From Eqs. (10) and (11), Eq. (6) can be restated as follows:

$$\begin{aligned} q&= \hbox {arg }\hbox {max}_{l=d,d+1,\ldots ,h} \left\{ {\min \nolimits _j \left\{ {s_{lj} -\left| {a_j -m_{lj} } \right| } \right\} } \right\} \nonumber \\&= \hbox {arg }\hbox {max}_{l=d,d+1,\ldots ,h} \left\{ \min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( \bar{b}_l \right) } \left\{ {||A-Z||_\infty } \right\} \right\} . \end{aligned}$$

(12)

b.1) h-d=1, namely the overlapped region $\bigcap \nolimits _{l=d}^h B_l$ belongs to only two hyperboxes $B_d$ and $B_h$

Therefore, Eq. (12) can be restated as follows:

$$\begin{aligned} q=\hbox {arg }\hbox {max}_{l=d,h} \left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_l} \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} . \end{aligned}$$

(13)

Without loss of generality, let $\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_d } \right) } \left\{ {||A-Z||_\infty } \right\} >\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_h } \right) } \left\{ {||A-Z||_\infty } \right\}$. Since $B_d \cap B_h \cap closure\left( {\bar{b}_d } \right) =B_d \cap B_h \cap closure\left( {SR_h } \right)$, and $B_d \cap B_h \cap closure\left( {\bar{b}_h } \right) =B_d \cap B_h \cap closure\left( {SR_d } \right)$, and $A$ was supposed to be in the overlapped region $B_d \cap B_h$, $\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {SR_h } \right) } \left\{ \right. ||A-Z \infty >\min {\hbox {Z}\in \hbox {closure} SRd A-Z \infty }$. Therefore, $max_{l=d,h} \left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_l } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} =min_{l=d,h} \left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {SR_l } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\}$. Thus, Eq. (13) can be restated as $q=\hbox {arg }\hbox {min}_{l=d,h} \left\{ {\min \nolimits _{\mathrm{Z} \in \mathrm{closure}\left( {SR_l } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\}$, and since for each $o\ne d$ and $h$, $min_{l=d,h} \left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {SR_l } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} <\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {SR_o } \right) } \left\{ {||A-Z||_\infty } \right\}$, It can be stated that $\hbox {arg }\hbox {min}_{l=d,h} \left\{ {\min \nolimits _{\mathrm{Z} \in \mathrm{closure}\left( {SR_l } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} =\hbox {arg }\hbox {min}_l \left\{ {\min \nolimits _{\mathrm{Z} \in \mathrm{closure}\left( {SR_l } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} .$ Therefore, $q=\hbox {arg }\hbox {min}_l \left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {SR_l } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\}$.

b-2) h-d>1, namely the overlapped region $\bigcap \nolimits _{l=d}^h B_l$ belongs to more than two hyperboxes

For simplicity and without loss of generality of proof, suppose that $h-d=2$, namely the overlapped region $\bigcap \nolimits _{l=d}^h B_l$ belongs to three hyperboxes (See Fig. 17). Let $d$, $e$ and $h$ be the indices of these three hyperboxes. We have for each $A$ in this overlapped region:

$$\begin{aligned}&max\left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_d } \right) } \left\{ {||A-Z||_\infty } \right\} ,} \right. \left. \min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_e } \right) } \left\{ {||A-Z||_\infty } \right\} ,\right. \\&\quad \left. \min \nolimits _{\mathrm{Z} \in \mathrm{closure}\left( {\bar{b}_h } \right) } \left\{ {||A-Z||_\infty } \right\} \right\} \\&\quad = max\left\{ {max\left\{ {\left. {\min \nolimits _{\mathrm{Z} \in \mathrm{closure}\left( {\bar{b}_d } \right) } \left\{ {||A-Z||_\infty } \right\} ,\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_e } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} ,} \right. } \right. \\&\quad \left. {\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {\bar{b}_h } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} . \end{aligned}$$

Without loss of generality of proof, let

$$\begin{aligned}&\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{\bar{b}}}_d\right) } \left\{ {||A-Z||_\infty } \right\} \nonumber \\&\quad = max\left\{ {\min \nolimits _{\mathrm{Z} \in \mathrm{closure}\left( \bar{b}_d\right) } \left\{ {||A-Z||_\infty } \right\} } \right. , \left. {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_e}\right) } \left\{ {||A-Z||_\infty } \right\} } \right\} . \end{aligned}$$

(14)

Therefore,

$$\begin{aligned}&max\left\{ max\left\{ {\left. {\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {\bar{b}_d } \right) } \left\{ {||A-Z||_\infty } \right\} ,\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_e } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} ,} \right. \left. {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_h } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} \right. \nonumber \\&\quad =max\left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_d } \right) } \left\{ {||A-Z||_\infty } \right\} , \min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_h } \right) } \left\{ {||A-Z||_\infty } \right\} } \right. . \end{aligned}$$

(15)

Similar to part (b.1) of this proof,

$$\begin{aligned}&max\left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_d } \right) } \left\{ {||A-Z||_\infty } \right\} } \right. ,\left. { \min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_h } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} \nonumber \\&\quad = min\left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {SR_d } \right) } \left\{ {||A-Z||_\infty } \right\} } \right. ,\left. {\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_h } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} . \end{aligned}$$

(16)

From Eq. (14), $\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b} _d } \right) } \left\{ {||A-Z||_\infty } \right\} \ge \min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_e } \right) } \left\{ {||A-Z||_\infty } \right\}$. Therefore, since $B_d \cap B_e \cap \hbox {closure}\left( {\bar{b}_d } \right) =B_d \cap B_e \cap \hbox {closure}\left( {SR_e } \right) , B_d \cap B_e \cap \hbox {closure}\left( {\bar{b}_e } \right) =B_d \cap B_e \cap \hbox {closure}\left( {SR_d } \right)$, and $A$ is supposed to be in $B_d \cap B_e , \min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_e } \right) } \left\{ {||A-Z||_\infty } \right\} \ge \min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_d } \right) } \left\{ {||A-Z||_\infty } \right\}$. Therefore,

$$\begin{aligned}&min\left\{ {min_{\mathrm{Z}\in \mathrm{closure}\left( {SR_d } \right) } \left\{ {||A-Z||_\infty } \right\} } \right. , \left. {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {SR_h } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} \nonumber \\&\quad = min\left\{ {\hbox {min}\left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_d } \right) } \left\{ {||A-Z||_\infty } \right\} } \right. } \right. ,\left. {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {SR_e } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} ,\left. {\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_h } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} \nonumber \\&\quad = min\left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_d } \right) } \left\{ {||A-Z||_\infty } \right\} ,\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {SR_e } \right) } \left\{ {||A-Z||_\infty } \right\} ,} \right. \left. {\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_h } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} . \end{aligned}$$

(17)

Thus, from Eqs. (15), (16) and (17),

$$\begin{aligned}&max\left\{ \min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_d } \right) } \left\{ {||A-Z||_\infty } \right\} ,\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {\bar{b}_e } \right) } \left\{ {||A-Z||_\infty } \right\} ,\left. {\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {\bar{b}_h } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\} \right. \\&\quad = min\left\{ \min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_d } \right) } \left\{ {||A-Z||_\infty } \right\} , \min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_e } \right) } \left\{ {||A-Z||_\infty } \right\} , \min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_h } \right) } \left\{ {||A-Z||_\infty } \right\} \right\} . \end{aligned}$$

Therefore, similar to part (b.1) of this proof, we can write $q=\hbox {arg }\hbox {min}_l \left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_l } \right) } \left\{ {||A-Z||_\infty } \right\} } \right\}$.

Fig. 17

An overlapped region which belongs to three hyperboxes

Finally, based on part (a) and (b) of this proof, FMNWSM assigns each $A$ to the class of hyperbox $B_q$, where

$$\begin{aligned} q=\hbox {arg }\hbox {min}_l \left\{ \min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_l } \right) } \left\{ d\left( {A,Z} \right) \right\} \right\} , \hbox {and } d\left( {A,Z} \right) =||A-Z||_\infty . \end{aligned}$$

$\square$

Proposition 2

If $A \cap B_l \ne \emptyset$ ($A$ is in hyperbox $B_l$), $\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {\bar{b}_l } \right) } \left\{ {||A-Z||_\infty } \right\} =\min _j \left\{ {s_{lj} -\left| {a_j -m_{lj} } \right| } \right\}$.

Proof

Let $D=\arg \min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_l } \right) } \left\{ {||A-Z||_\infty } \right\}$, where $D=\left( {d_1 ,d_2 ,\ldots ,d_n } \right) ^{{\prime }}$. The line $AD$ is orthogonal to a surface of $B_l$, or in other words, it is parallel to $n-1$ axes and orthogonal to one remaining axis. If $AD$ is parallel to $t$-th axis, for each $i\ne t$, $a_i -d_i =0$. Therefore, $\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {\bar{b}_l } \right) } \left\{ {||A-Z||_\infty } \right\} =A-D_\infty =\max _j \left\{ {\left| {a_j -d_j } \right| } \right\} =\left| {a_t -d_t } \right| =\hbox {min}_j \left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_{lj} } \right) } \left\{ {\left| {a_j -z} \right| } \right\} } \right\}$. Meanwhile $,\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {\bar{b}_{lj} } \right) } \left\{ \left| {a_j\!-\!z} \!=\!slj\!-\!aj\!-\!mlj\right. \right.$. Therefore, $\min \mathrm{Z}\in \mathrm{closure} bl A-Z\infty =\min j slj-aj-mlj$. $\square$

Proposition 3

The model (M1) classifies $SR_i \left( {i=1,2,\ldots ,m} \right)$ with symmetric margin.

Proof

Suppose that the hyperboxes have no overlapping. Let class $\left( {-1} \right) \equiv$ classes $\left( {1,2,\ldots ,q-1,q+1,\ldots ,k} \right)$ and class $\left( {+1} \right) \equiv$ class $\left( q \right)$. Then, the model (M1) can be restated as follows:

$$\begin{aligned} {\begin{array}{ll} {\hbox {find }f\left( . \right) } \\ {\hbox {s}.\hbox {t}.f\left( A \right) =\left\{ {{\begin{array}{ll} {+1}&{} {\min \nolimits _{SR_l \in class+1} \left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_l } \right) } \left\{ {d\left( {A,Z} \right) } \right\} } \right\} <\min \nolimits _{SR_l \in class-1} \left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_l } \right) } \left\{ {d\left( {A,Z} \right) } \right\} } \right\} ,} \\ {-1}&{} {\min \nolimits _{SR_l \in class+1} \left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {SR_l } \right) } \left\{ {d\left( {A,Z} \right) } \right\} } \right\} >\min \nolimits _{SR_l \in class-1} \left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {SR_l } \right) } \left\{ {d\left( {A,Z} \right) } \right\} } \right\} .} \\ \end{array} }} \right. } \\ \end{array} } \end{aligned}$$

In other words, test data $A$ is assigned to

$$\begin{aligned} f\left( A \right) =\hbox {sign}\left( {\min \nolimits _{SR_l \in class-1} \left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {SR_l } \right) } \left\{ {d\left( {A,Z} \right) } \right\} } \right\} -} \right. \left. {\min \nolimits _{SR_l \in class+1} \left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_l } \right) } \left\{ {d\left( {A,Z} \right) } \right\} } \right\} } \right) . \end{aligned}$$

Therefore, when $\min _{SR_l \in class-1} \left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure} \left( {SR_l } \right) } \left\{ {d\left( {A,Z} \right) } \right\} } \right\} =\min _{SR_l \in class+1} \left\{ {\min \nolimits _{\mathrm{Z}\in \mathrm{closure}\left( {SR_l } \right) } \left\{ {d\left( {A,Z} \right) } \right\} } \right\} , \quad f\left( A \right) =0,$ and $A$ is a point of the learned classifier. Thus, the distance from each point of the learned classifier to the nearest hyperboxes of class +1 is equal to the distance from that point of the learned classifier to the nearest hyperboxes of class -1. In other words, the $f\left( . \right)$ has a symmetric margin. $\square$