Evaluating different methods of microarray data normalization

Abstract

Background

With the development of DNA hybridization microarray technologies, nowadays it is possible to simultaneously assess the expression levels of thousands to tens of thousands of genes. Quantitative comparison of microarrays uncovers distinct patterns of gene expression, which define different cellular phenotypes or cellular responses to drugs. Due to technical biases, normalization of the intensity levels is a pre-requisite to performing further statistical analyses. Therefore, choosing a suitable approach for normalization can be critical, deserving judicious consideration.

Results

Here, we considered three commonly used normalization approaches, namely: Loess, Splines and Wavelets, and two non-parametric regression methods, which have yet to be used for normalization, namely, the Kernel smoothing and Support Vector Regression. The results obtained were compared using artificial microarray data and benchmark studies. The results indicate that the Support Vector Regression is the most robust to outliers and that Kernel is the worst normalization technique, while no practical differences were observed between Loess, Splines and Wavelets.

Conclusion

In face of our results, the Support Vector Regression is favored for microarray normalization due to its superiority when compared to the other methods for its robustness in estimating the normalization curve.

Background

DNA microarray technology is a powerful approach for genomic research, playing an increasingly important role in biomedical research. This technology yields simultaneous measurement of gene expression levels of thousands of genes, allowing the analysis of differential gene expression patterns under different conditions such as disease (pathological) states or treatment with different chemotherapeutic drugs. Due to small differences in RNA quantities and fluctuations generated by the technique, the intensity levels may vary from one replicate to the other due to effects which are unrelated to the genes, requiring data normalization before they can be compared.

Therefore, normalization is an important step for microarray data analysis. The purpose of data normalization is to minimize the effects caused by technical variations and, as a result, allow the data to be comparable in order to find actual biological changes. Several normalization approaches have been proposed, most of which derive from studies using two-color spotted microarrays. Some authors proposed normalization of the hybridization intensity ratios; others use global, linear methods, while others use local, non-linear methods. Several authors suggested using the spike-in controls, housekeeping genes, or invariant genes [1–7].

Recently, some authors suggested the use of non-linear normalization methods [8–10] which are believed to be superior to the above mentioned approaches. The locally weighed regression Lowess procedure [11] has been widely used for this purpose and implemented by several microarray analysis software packages [12, 13], but similar methods are suggested such as Splines [14, 15] and Wavelets [16].

Here, we compare three different well-known microarray data normalization methods, namely: Loess Regression (LR), Splines Smoothing (SS) and Wavelets Smoothing (WS). In addition, we propose two different normalization approaches, called Kernel Regression (KR) [17, 18] and Support Vector Regression (SVR) [19], which, to the best of our knowledge, have yet to be applied for microarray normalization. In order to assess the most appropriate normalization technique, benchmark studies were carried out using data derived from CodeLink™ mouse microarray experiments [20], generated at our Cell and Molecular Biology Laboratory (Chemistry Institute, University of São Paulo).

Results

We sought to highlight the performance of five different methods of microarray normalization, namely: Loess, Splines, Wavelets, Kernel and Support Vector Regression in a simulated microarray and in an actual CodeLink™ microarray platform, which comprised ten thousand mouse genes. Although we have focused on the use of simulated two-color cDNA microarray data analysis, our discussions are also applicable to the single-color oligonucleotide microarrays.

The artificial microarrays composed by ten thousand spots were generated using the model proposed by Balagurunathan et al. (2002) [21]. The parameters used were: ($a_0^1$ = 0, $a_1^1$ = 100^1/0.7, $a_2^1$ = -0.7, $a_3^1$ = 1) and ($a_0^2$ = 0, $a_1^2$ = 100^1/0.9, $a_2^2$ = -0.9, $a_3^2$ = 1) for sinusoid shape, ($a_0^1$ = 0, $a_1^1$ = 500 $a_2^1$ = -1, $a_3^1$ = 1) and ($a_0^2$ = 0, $a_1^2$ = 10, $a_2^2$ = -1, $a_3^2$ = 1) for banana shape and, ($a_0^1$ = 0, $a_1^1$ = 10, $a_2^1$ = -1, $a_3^1$ = 1) and ($a_0^2$ = 0, $a_1^2$ = 100^1/0.7, $a_2^2$ = -0.7, $a_3^2$ = 1) for mixed shape. Gene expression was generated by an exponential distribution with parameter λ = 1/3000 and the outliers were generated by a Beta distribution with parameters B(1.7,4.8). For more details, see Balagurunathan et al. (2002).

The smoothing parameters used in each dataset are described in Table 1. For SVR, we tested a range of values and, as a result, we selected ε = 0.01 and C = 4 as the most adequate one. It is important to highlight that the parameters are arbitrary; therefore, we chose the optimum parameters for each method, i.e., the one which resulted in the lowest mean square error. In Figure 1 are described the mean square errors for each normalization method applied to three different simulated microarrays with no outliers.

Table 1 Smoothing parameters used for each microarray dataset. For Loess, it is the span value, for Splines and Wavelets it is the number of functions, for Kernel and SVR it is the maximum value minus the minimum value multiplied by the number described in the table.

Figure 1

The minimum mean square error for three different simulated microarray datasets. From left to right: 1) sinusoid shape; 2) banana shape; 3) mix shape. The Kernel regression was not included in this figure because its MSE is 10³ orders of magnitude greater than the other normalization methods.

In order to compare the perturbation caused by the presence of outliers and the robustness of each normalization method, we randomly inserted 5, 10, 15, 20 and 40% of outliers (genes which display very high differential expression) at three different expression levels (low, medium, high), and the respective mean square errors between the regression curve and the actual curve (the function from which the microarray was generated) was calculated. This step was repeated 100 times to estimate the average sum of the squared errors and their variance. The Wilcoxon and the Kolmogorov-Smirnov tests were performed in order to determine whether the five regression methods differ from one another in any significant manner.

A high performance normalization technique should yield unbiased corrections and corrections with the smallest standard deviation.

Comparison of the results presented in Table 2, 3 and 4 shows no important difference between LR, SS and WS. Although the non-parametric KR method has been successfully applied in econometrics data analysis [22], it displayed a poor performance for microarray normalization, probably because it is highly sensitive to outliers [23].

Table 2 The mean square errors of estimated gene expression levels for simulated cDNA microarray data with differentially expressed genes inserted under the low expression levels condition.

Table 3 The mean square errors of estimated gene expression levels for simulated cDNA microarray data with differentially expressed genes inserted under the medium expression levels condition.

Table 4 The mean square errors of estimated gene expression levels for simulated cDNA microarray data with differentially expressed genes inserted under the high expression levels conditions.

Upon analyzing Table 2, it is possible to observe, in the case of sinusoid shape, that when outliers are inserted in regions of low gene expression, SVR, WS, SS, LR and KR, in this order, have the lowest to the highest mean square error, being statistically different (p value < 0.001) from one another. For the banana and mixed shapes, LR and SS presented a lower MSE than WS. In Table 3, it is interesting to note that when outliers are inserted in regions of medium gene expression, i.e., high density of genes, the order of performance remains the same as in Table 2 and SVR displays a mean square error which is significantly different from the others (p value < 0.001). LR and SS showed no significant difference (p value > 0.05) and KR is significantly worse than the other methods (p value < 0.001). In Table 4, the outliers are inserted in a high gene expression region. Once more, the trend is maintained, namely, KR is the most affected by outliers (p value < 0.001) and no differences between SS and WS (p value > 0.05) were observed for the sinusoid shape. For the other two shapes, LR and SS were better than WS (p value < 0.001).

In all three cases (outliers at low, medium and high gene expression), SVR is the affected by outliers (p value < 0.001), independently the microarray's shape. In addition, SVR yields the smallest standard deviation, followed by LR, SS, WS, with KR displaying the largest deviation. In addition, the five methods were applied to actual microarray data, with outliers inserted artificially, and the results were the same when compared to those obtained from artificial microarray experiments.

In Figure 2, we illustrate the performance of the five normalization methods applied to actual microarray data, without the insertion of artificial outliers. A small difference could be observed in the normalization curves in which the genes displayed low and high expression, due to the low quantity of genes and the high variance.

Figure 2

Fitted normalization curves for actual cDNA microarray data using the five different normalization methods (Loess, Splines, Wavelets, Kernel, SVR).

Discussion

By analyzing the extent to which the outliers could disturb the regression curve, we observed that KR is more highly sensitive to outliers than LR, SS and WS in all three cases (outliers in low, medium and high expression). In all three cases, SVR is shown to be the least affected.

The superior performance of Splines, when compared to KR, may be explained by the degree of smoothing, which varies according to the density of points, differently from KR, which has a fixed window size. Wavelet also has a slightly better performance than KR, probably due to multi-resolution properties. In general, SS and WS presented similar performance when we compared the median of the mean square error using the Wilcoxon test. However, when we used the Kolmogorov-Smirnov test, they presented a statistically significant difference (p value < 0.001). SS and WS constitute somewhat better normalization techniques than LR when we analyzed the sinusoid shape, but, for the other two shapes, LR is better than SS and WS. For practical purposes, the differences between them in terms of disturbance by outliers are too small to be of any concern.

The SVR method is shown to be very robust to outliers presented at different gene expression levels, becoming the best normalization technique to identify actual differentially expressed genes.

One well-known problem in identifying differentially expressed genes is normalizing genes displaying low expression levels, due to the low quantity of the corresponding transcripts and the high spot intensity variance. An equivalent problem occurs with genes presenting very high expression levels due to the low frequency of these genes. Once more, under these conditions, the SVR method is shown to be better than other currently used methods.

We performed the same tests for five other pairs of CodeLink™ microarrays and the results obtained were the same: the SVR is the most robust to outliers and the KR method is the worst method, being highly sensitive to differentially expressed genes and yielding poor regression curves.

Other methods, which are also robust to outliers and are based on a new regression method called two-way semi-linear model [24–27] have also been applied for microarray data normalization. This new approach developed in the last few years, deserves further studies, which we are planning to undertake in the future.

Conclusion

We have proposed a new approach to normalize microarray data and tested this SVR method by benchmark studies and by several simulations. The results obtained with SVR were superior than those obtained with some widely used normalization techniques such as LR, SS and WS. SVR is shown to be more robust to outliers even at very low and very high gene expression levels, being useful to identify differentially expressed genes. Even tested in different microarray shapes, SVR was superior to the other methods, while LR, SS and WS presented similar performances. Therefore, we have demonstrated that SVR is feasible and very promising for microarray data normalization.

Methods

Simulation

The program which generates the artificial microarray and the analyses were implemented in R, a language for statistical computing [28]. This script may be downloaded at: [29].

CodeLink™ microarray

Cell lysis and RNA extraction

Cell cultures were lysed with guanidine isocyanate and RNA was purified by of the cell lysates on a cesium chloride cushion (Chirgwin el al, 1979). Absorbance ratio at 260/280 nm was used to assess the RNA purity, a ratio of 1.8 – 2.0 indicating adequate purity.

Labeling and purification of targets

RNA samples were prepared and processed according to protocols supplied by the manufacturer (Amersham Biosciences). Briefly, cDNAs were synthesized from purified RNA (2 μg) and control bacterial mRNAs. Samples were purified using the QIAquick Spin kit (Qiagen) and concentrated by SpeedVac. Concentrated pellets were used in a biotinylated-UTP based cRNA synthesis using the CodeLink™ Expression Assay Reagent Kit (Amersham). Labeled cRNAs were purified using the RNeasy kit (Qiagen) and fragmented with supplied solution at 94°C for 20 min.

Hybridization and washing of arrays

Fragmented biotin-labeled cRNAs (10 μg) were incubated with CodeLink™ bioarrays and shaken (300 rpm) for 20 h. The bioarrays were then washed and incubated with Cy5-Streptavidin (30 min). Scanning of the bioarrays was performed in a GenePix 4000 B Array Scanner (Axon Instruments) and the data were collected using the CodeLink™ System Software (Amersham), which provided the raw data and invalidated data from irregular spots.

Loess regression

Consider we have n measurements, for each of which the response expected is y _i and let x _i be the predictor, where x is the log intensity of one microarray and y is the log intensity of the other one, in case we are analyzing a single-color microarrays. Whether the microarray is a two-color platform, x is the log of one dye intensity and y is the log of the other dye intensity.

In this model, they are supposed to be related by

y _i = g(x _i ) + ε _i     (1)

where g is the regression function and ε _i is a random error. The idea of local regression is that near x = x₀, the regression function g(x) can be locally approximated by the value of a function in some specified parametric class. Such a local approximation is obtained by fitting a regression surface to the data points within a chosen neighborhood of the point x₀.

In this method, weighed least squares are used to fit linear or quadratic functions of the predictors at the centers of the neighborhoods. The radius of each neighborhood is chosen so that the neighborhood contains a specified percentage of the data points. The fraction of the data, called the smoothing parameter, in each local neighborhood is weighted by a smooth decreasing function of their distance from the center of the neighborhood [30].

B-Splines smoothing

Due to its simple structure and good approximation properties, polynomials are widely used in practice for approximating functions [31, 32]. Let x and y as defined above and

$${(x-y)}_{+}^0=\{\begin{array}{c}1,x\ge y\\ 0,x<y\end{array}\left(2\right)$$

and

$${(x-y)}_{+}^{m-1}=\{\begin{array}{l}{(x-y)}^{m-1},x\ge y,m>1\hfill \\ 0,x<y\hfill \end{array}(3)$$

Therefore, let

... ≤ y_-1 ≤ y₀ ≤ y₁ ≤ y₂ ≤ ...     (4)

be a sequence of real numbers. Given integers i and m > 0, we define

$$Q_i^m(x)=\{\begin{array}{l}{(-1)}^m[y_i,\mathrm{...},y_{i+m}]{(x-y)}_{+}^{m-1},if{y}_i<y_{i+m}\hfill \\ 0,otherwise\hfill \end{array}(5)$$

for all real x. We call $Q_i^m$ the m th order B-Spline associated with the knots y _i ,..., y_{i + m}.

For m = 1, the B-Spline associated with y _i <y_{i + 1}is particularly simple. It is the piecewise constant function

$$Q_i^1(x)=\{\begin{array}{l}\frac{1}{y_{i+1}-y_i},y_i\le x\le y_{i+1}\hfill \\ 0,otherwise.\hfill \end{array}(6)$$

In our analysis, we applied the cubic Splines, i.e., Splines of order 3.

We can also give explicit formulate for $Q_i^m$ in case either y _i or y_{i + m}is a knot of multiplicity m.

Wavelet smoothing

The Wavelet transform is a relatively new approach and has some similarities with the Fourier transform. Wavelets differ from Fourier methods in that they allow the localization of a signal in both time and frequency. In the wavelet theory, a function is represented by an infinite series expansion in terms of dilated and translated version of a basic function ψ called the "mother' Wavelet. A Wavelet transformation leads to an additive decomposition of a signal into a series of different components describing smooth and rough features of the signal.

The term Wavelets means small curves, therefore, they are oscillations that rapidly decay. As the B-Splines functions system, the Wavelets functions ψ(t) can be used to generate a function basis for certain spaces [33]. An ortonormal basis can be generated by dyadic dilations and translations of a mother Wavelet ψ(t), by

ψ _{j, k} (t) = 2^j/2ψ (2^jj - k),     j, k ∈ Z     (7)

Wavelets are functions which satisfy the following properties:

• i)
  $${\displaystyle \underset{-\infty }{\overset{\infty }{\int }}\psi }(t)$$

  (1)

  dt = 0     (8).

ii) ${\displaystyle \underset{-\infty }{\overset{\infty }{\int }}\psi \left(t\right)}$dt < ∞     (9).

iii) ${\displaystyle \underset{-\infty }{\overset{\infty }{\int }}\frac{{\left|\Psi (\omega )\right|}^{2}d\omega }{\left|\omega \right|}}=0$, where the function Ψ(ω) is the Fourier transform of ψ(t)     (10).

iv) ${\displaystyle \underset{-\infty }{\overset{\infty }{\int }}{t}^j\psi (t)dt=0}$,     j = 0,1,..., r - 1     for r ≥ 1 and ${\displaystyle \underset{-\infty }{\overset{\infty }{\int }}\left|t^r\psi (t)\right|}dt<\infty$     (11).

An important result is that any function f (t) with ${\displaystyle \underset{-\infty }{\overset{\infty }{\int }}{f}^2(t)<\infty }$ can be expanded as

$$f(t)={\displaystyle \sum _{j=-\infty }^{\infty }{\displaystyle \sum _{k=-\infty }^{\infty }{c}_{j,k}}{\psi }_{j,k}}(t)(12).$$

In other words, any function f (t) can be represented by a linear combination of functions ψ _{j, k} (t). The smoothing procedure can be carried out by an approximation, choosing a maximum resolution J (t) for j =1,2,..., J (t) and k = 1,2,..., 2^j-1. Here, we considered the Mexican hat Wavelet [34] defined by

$$\psi (t)=(1-t^2)\exp \left(\frac{-t^2}{2}\right)(13)$$

rather than other functions such as Morlet or Shannon since they do not have an analytic formula.

The C _jk coefficients are estimated via an ordinary least square regression. An important feature in the wavelets representation is that it allows the description of functions belonging to both Sobolev and Besov spaces [35].

Kernel regression

KR is one class of modeling methods that belongs to the smoothing methods family. It is part of the non-parametric regression methods. KR allows basing the prediction of a value on passed observations, and weighing the impact of past observations depending on how similar they are, compared to the current values of the explanatory variables.

The KR is one of the most widely used procedures in non-parametric curve estimation. Nadaraya (1964) and Watson (1964) proposed an estimator for the curve g given by

$$g_h(x)=\frac{{\displaystyle \sum _{i=1}^n{K}_h}(x-X_i)Y_i}{{\displaystyle \sum _{j=1}^n{K}_h}(x-X_j)}(14)$$

In our datasets, we used the Gaussian Kernel because it is symmetric and centralized in the mean.

In addition to being easy to compute, the Nadaraya-Watson estimator g _h (x) is consistent. When h → 0 the estimated curve presents a large variability and when nh → ∞, we obtain an overly smooth curve [36]. The bandwidth h controls the smoothness degree of the estimated curve. It is easy to observe that this KR estimator is just a weighted sum of the observed responses Y _i . The denominator ensures that the weights sum up to 1.

Support Vector Regression

SVR generalized algorithm is a non-linear regression from the Generalized Portrait algorithm developed in Russia by Vapnik and Lerner (1963) [37] and Vapnik and Chervonenkis (1964) [38]. It is based upon the statistical learning theory which has been developed by Vapnik and Chervonenkis (1974) [39]. In Bioinformatics, and, more specifically, in microarray data analysis, to the best of our knowledge, this algorithm has previously been used only once, by Hisanori et al. (2004), to extract relations between promoter sequences and strengths [40]. Here, we propose the use of SVR to normalize microarray data.

Let {(x₁, y₁),..., (x₁, y₁)} ⊂ R × R be the gene expression data derived from microarray experiments, where x is the log intensity of one microarray and y is the log intensity of the other one, in case we are analyzing a single-color microarrays. When the microarray is a two-color platform, x is the log of one dye intensity and y is the log of the other dye intensity. In ε-SVR [41], the goal is to obtain a function f (x) that has at the most ε deviation from the y _i for all the data, and is as flat as possible.

In the case of linear functions f :

f (x) = (w^tx) + b with w ∈ Rⁿ, b ∈ R     (15)

Flatness in (15) means

Minimize $\frac{1}{2}{\Vert w\Vert }^2$

$$\text{Constrained to }\{\begin{array}{l}{y}_i-\left(w^t{x}_i\right)-b\le \epsilon \hfill \\ \left(w^t{x}_i\right)+b-y_i\le \epsilon \hfill \end{array}(16)$$

In (16) there is a function f which, with ε precision, approximates all pairs (x_i, y _i ). But there are cases where it is necessary to allow for some errors. To solve this problem, one can introduce slack variables ξ _i , ${\xi }_i^{\ast }$ to deal with unfeasible constraints of the optimization problem (16) arriving at the formulation stated in [41]

Minimize $\frac{1}{2}{\Vert w\Vert }^2$ + C ∑(ξ _i + ${\xi }_i^{\ast }$)

$$\text{Constrained to }\{\begin{array}{l}{y}_i-\left(w^t{x}_i\right)-b\le \epsilon +{\xi }_i\hfill \\ \left(w^t{x}_i\right)+b-y_i\le \epsilon +{\xi }_i^{*}\hfill \\ {\xi }_i,{\xi }_i^{*}\ge 0\hfill \end{array}(17)$$

where the constant C > 0 is the trade-off between the amount up to which deviations larger than ε are tolerated, maintaining the flatness of f. This corresponds to dealing with the ε-insensitive loss function |ξ| _ε :

$${\left|\xi \right|}_{\epsilon }:=\{\begin{array}{l}if\left|\xi \right|\le \epsilon \hfill \\ \left|\xi \right|-\epsilon otherwise\hfill \end{array}(18)$$

It is necessary to construct a Lagrange function from the primal objective function and the corresponding constraints by introducing a dual set of variables. According to Mangasarian (1969) [42], McCormick (1983) [43], and Vanderbei (1997) [44] it follows that:

$$L:=\frac{1}{2}{\Vert w\Vert }^2+C{\displaystyle \sum _{i=1}^l\left({\xi }_i+{\xi }_i^{*}\right)-}{\displaystyle \sum _{i=1}^l\left({\eta }_i{\xi }_i+{\eta }_i^{*}{\xi }_i^{*}\right)}-{\displaystyle \sum _{i=1}^l{\alpha }_i}\left(\epsilon +{\xi }_i-y_i+\left(w^t{x}_i\right)+b\right)-{\displaystyle \sum {\alpha }_i^{*}}\left(\epsilon +{\xi }_i^{*}+y_i-\left(w^t{x}_i\right)-b\right)(19)$$

where L is the Lagrangian and η _i , ${\eta }_i^{*}$, α _i , ${\alpha }_i^{*}$ are Lagrange multipliers. Hence the dual variables in (19) have to satisfy

$${\alpha }_i^{(*)},{\eta }_i^{(*)}\ge 0(20)$$

Note that we refer to α _i and ${\alpha }_i^{*}$ as ${\alpha }_i^{(\ast )}$.

From the saddle point condition, the partial derivatives of L related to (w, b, ξ _i , ${\xi }_i^{\ast }$) have to vanish for optimality.

$${\partial }_{b}L={\displaystyle \sum _{i=1}^l\left({\alpha }_i^{*}-{\alpha }_i\right)=0(21)}$$

$${\partial }_{w}L=w-{\displaystyle \sum _{i=1}^l\left({\alpha }_i-{\alpha }_i^{*}\right)}{x}_i=0(22)$$

$${\partial }_{{\xi }_i}(*)L=C-{\alpha }_i^{(*)}-{\eta }_i^{(*)}=0(23)$$

From the substitution of (21), (22) and (23) into (19) we obtain a dual optimization problem.

$$\text{Maximize }\{\begin{array}{l}-\frac{1}{2}{\displaystyle \sum _{i,j=1}^l\left({\alpha }_i-{\alpha }_i^{*}\right)}\left({\alpha }_j-{\alpha }_j^{*}\right)\left(x_i^t{x}_j\right)\hfill \\ -\epsilon {\displaystyle \sum _{i=1}^l\left({\alpha }_i-{\alpha }_i^{*}\right)}+{\displaystyle \sum _{i=1}^l{y}_i}\left({\alpha }_i-{\alpha }_i^{*}\right)\hfill \end{array}(24)$$

Subject to ${\displaystyle \sum _{i=1}^l\left({\alpha }_i-{\alpha }_i^{*}\right)=0}$ and α _i , ${\alpha }_i^{*}$ ∈ [0, C]

Equation (22) can be rewritten as follows

$$w={\displaystyle \sum _{i=1}^l\left({\alpha }_i-{\alpha }_i^{*}\right)}{x}_i$$

(2)

, thus $f(x)={\displaystyle \sum _{i=1}^l\left({\alpha }_i-{\alpha }_i^{*}\right)\left(x^{t}x\right)}+b$     (25)

This is the Support Vector expansion, i.e., the description of w as a linear combination of x _i .

To compute b, it is necessary to use Karush-Kuhn-Tucker (KKT) conditions [45, 46]. These authors state that at the point of the solution the product between dual variables and constraints has to vanish.

α _i (ε + ξ _i - y _i + (w^tx _i ) + b) = 0     (26)

$${\alpha }_i^{*}$$

(3)

(ε + ${\xi }_i^{\ast }$ + y _i - (w^tx _i ) - b) = 0

and

(C - α _i )ξ _i = 0     (27)

(C - ${\alpha }_i^{*}$) ${\xi }_i^{\ast }$ = 0

From (26) and (27) it follows that:

• (i) Only samples (x _i , y _i ) with corresponding ${\partial }_{b}L={\displaystyle \sum _{i=1}^l\left({\alpha }_i^{*}-{\alpha }_i\right)=0(21)}$ = C lie outside the ε-insensitive tube;

• (ii) α _i ${\alpha }_i^{*}$ = 0

From (i) and (ii), it is possible to conclude that

ε - y _i + (w^tx _i ) + b ≥ 0 and ξ _i = 0 if α _i <C     (28)

ε - y _i + (w^tx _i ) + b ≤ 0     if α _i > 0     (29)

In conjunction with an analogous analysis on ${\alpha }_i^{*}$

max{-ε + y _i - (w^tx _i )|α _i <C or ${\alpha }_i^{*}$ > 0} ≤ b ≤ min{-ε + y _i - (w^tx _i )|α _i > 0 or ${\alpha }_i^{*}$ <C}     (30)

If some ${\alpha }_i^{*}$ ∈ (0, C) the inequalities become equalities.

To point out the sparsity of the SV expansion: from (26), the Lagrange multipliers may be nonzero only for |f (x _i ) - y _i | ≥ ε.

Therefore, we have a sparse expansion of w in terms of x _i [47].