Kernel-based data transformation model for nonlinear classification of symbolic data

Abstract

Symbolic data are usually composed of some categorical variables used to represent discrete entities in many real-world applications. Mining of symbolic data is more difficult than numerical data due to the lack of inherent geometric properties of this type of data. In this paper, we use two kinds of kernel learning methods to create a kernel estimation model and a nonlinear classification algorithm for symbolic data. By using the kernel smoothing method, we construct a squared-error consistent probability estimator for symbolic data, followed by a new data transformation model proposed to embed symbolic data into Euclidean space. Based on the model, the inner product and distance measure between symbolic data objects are reformulated, allowing a new Support Vector Machine (SVM), called SVM-S, to be defined for nonlinear classification on symbolic data using the Mercer kernel learning method. The experiment results show that SVM can be much more effective for symbolic data classification based on our proposed model and measures.

Appendices

Appendices

A Proof of Theorem 1

Since ${[I\left( \cdot \right) ]}^{2} = I\left( \cdot \right)$ and $\sum _{o \in O_{d}}^{}{[p(o)]} = 1 ,$ the expectation of $\hat{p}\left( o_{dl} \big | \lambda _{d} \right)$ can be obtained from Eq. (4):

$$\begin{aligned} \begin{array}{l} E\left( \hat{p}\left( o_{dl} \big | \lambda _{d} \right) \right) \\ \quad = E\left[\ell \left( X_{d},o_{dl},\lambda _{d} \right) \right]\\ \quad = \sum _{o \in O_{d}}^{}{[\frac{1}{{|O}_{d}|}\lambda _{d} + \left( 1 - \lambda _{d} \right) I\left( o = o_{dl} \right) ]p(o)}\\ \quad = \frac{\lambda _{d}}{{|O}_{d}|} + \left( 1 - \lambda _{d} \right) p\left( o_{dl} \right) \text { .} \end{array} \end{aligned}$$

So, the $\text {Bias}\left( \hat{p}\left( o_{dl} \big | \lambda _{d} \right) \right)$ and the$\text {Var}\left( \hat{p}\left( o_{dl} \big | \lambda _{d} \right) \right)$ can be computed as:

$$\begin{aligned}&[Bias\left( \hat{p}\left( o_{dl} \big | \lambda _{d} \right) \right) ]^{2}\\&= \left[\frac{\lambda _{d}}{{|O}_{d}|} - \lambda _{d}p\left( o_{dl} \right) \right]^{2}\\&= \lambda _{d}^{2}\left[{{|O}_{d}|}^{- 1} - p\left( o_{dl} \right) \right]^{2}, \end{aligned}$$

and

$$\begin{aligned}&\text {Var}\left( \hat{p}\left( o_{dl} \big | \lambda _{d} \right) \right) \nonumber \\&\quad = \frac{1}{N}\text {Var}\left[\ell \left( X_{d},o_{dl},\lambda _{d} \right) \right]\nonumber \\&\quad = \frac{1}{N}\left[E\left( \ell ^{2}\left( X_{d},o_{dl},\lambda _{d} \right) \right) - \left( E\left( \ell \left( X_{d},o_{dl},\lambda _{d} \right) \right) \right) ^{2} \right]\nonumber \\&\quad = \frac{1}{N}\left\{ \sum _{o \in O_{d}}^{}\left[\frac{\lambda _{d}}{{|O}_{d}|} + \left. (1 - \lambda _{d} \right. ) I\left( o = o_{dl} \right) \right]^{2} p(o)\right. \\&\qquad \left. - \left[\frac{\lambda _{d}}{{|O}_{d}|} + \left( 1 - \lambda _{d} \right) p\left( o_{dl} \right) \right]^{2} \right\} \nonumber \\&\quad = {\frac{1}{N}[\left. (1 - \lambda _{d} \right. )}^{2}p\left( o_{dl} \right) - \left. (1 - \lambda _{d} \right. )^{2}p^{2}\left( o_{dl} \right) ]\nonumber \\&\quad = \frac{\left. (1 - \lambda _{d} \right. )^{2}}{N}\left[p\left( o_{dl} \right) - p^{2}\left( o_{dl} \right) \right]\nonumber \end{aligned}$$

(15)

By combining the above two equalities, the theorem is proved.

B Proof of Theorem 2

For each $o_{dl}$ in Eq. (6), we have that

$$\begin{aligned}&E\left[{(\left( 1 - \lambda _{d} \right) f\left( o_{dl} \right) + \frac{\lambda _{d}}{{|O}_{d}|} - \ p{(o}_{dl}))}^{2} \right]\\&\quad =\left( 1 - \lambda _{d} \right) ^{2}E\left[f^{2}\left( o_{dl} \right) \right]\\&\qquad + 2\left[\frac{\lambda _{d} - \lambda _{d}^{2}}{{|O}_{d}|} + (\lambda _{d} - 1)p(o_{dl}))\right]\\&\quad E\left[f\left( o_{dl} \right) \right]+ [{p{(o}_{dl})]}^{2} - \frac{2\lambda _{d}}{{|O}_{d}|}p{(o}_{dl}) + \frac{\lambda _{d}^{2}}{{{|O}_{d}|}^{2}}. \end{aligned}$$

Base on the facts that $E\left[f(o_{dl}) \right]=p( o_{dl})$ and ${[I(\cdot )]}^{2} = I(\cdot )$, the above equality can be simplified as

$$\begin{aligned} \begin{array}{l} \left( 1 - \lambda _{d} \right) ^{2} \left( E\left[f^{2}\left( o_{dl} \right) \right]- (E{\left[f \left( o_{dl} \right) \right])}^{2} \right) \\ \qquad + \left( 1 - \lambda _{d} \right) ^{2}p^{2}{(o}_{dl}) + \ 2\left[\frac{\lambda _{d} - \lambda _{d}^{2}}{{|O}_{d}|} + (\lambda _{d} - 1)p{(o}_{dl})) \right]\\ \qquad p{(o}_{dl}) + p^{2}{(o}_{dl}) - \frac{2\lambda _{d}}{{|O}_{d}|}p{(o}_{dl}) + \frac{\lambda _{d}^{2}}{{{|O}_{d}|}^{2}}\\ \quad = \left( 1 - \lambda _{d} \right) ^{2}\frac{p{(o}_{dl})(1 - p{(o}_{dl}))}{N} + \lambda _{d}^{2} [{p(o_{dl})}]^{2}\\ \qquad - \frac{2\lambda _{d}^{2}}{|O_{d}|}p(o_{dl}) + \frac{\lambda _{d}^{2}}{{{|O}_{d}|}^{2}}\\ \quad = \left[\lambda _{d}^{2} - \frac{\left( 1 - \lambda _{d} \right) ^{2}}{N} \right]p^{2}{(o}_{dl}) + \left[\frac{\left( 1 - \lambda _{d} \right) ^{2}}{N} - \frac{2\lambda _{d}^{2}}{{|O}_{d}|} \right]p{(o}_{dl})\\ \qquad + \frac{\lambda _{d}^{2}}{{{|O}_{d}|}^{2}}. \end{array} \end{aligned}$$

Therefore, $\mathcal {L}\left( \lambda _{d} \right)$ can be computed as

$$\begin{aligned} \mathcal {L}\left( \lambda _{d} \right)= & {} \left[\lambda _{d}^{2} - \frac{\left( 1 - \lambda _{d} \right) ^{2}}{N} \right]\sum _{o_{dl} \in O_{d}}^{}{[{p{(o}_{dl})]}^{2}}\\&+ \frac{\left( 1 - \lambda _{d} \right) ^{2}}{N} - \frac{\lambda _{d}^{2}}{{|O}_{d}|} \\= & {} \left( 1 - \frac{1}{{|O}_{d}|} \right) \lambda _{d}^{2} + \left[\frac{\left( 1 - \lambda _{d} \right) ^{2}}{N} - \lambda _{d}^{2} \right]\sigma _{d}^{2}\text {\ .} \end{aligned}$$

Let $\frac{\partial \mathcal {L}\left( \lambda _{d} \right) }{\partial \lambda _{d}} = 0$, we get the optimal estimate of $\lambda _{d}$, and Eq. (7).