Clustering Methods

Abstract

What the reader should know to understand this chapter $\bullet$ Basic notions of calculus and linear algebra. $\bullet$ Basic notions of machine learning. $\bullet$ Programming skills to implement some computer projects proposed in the Problems section.

Problems

6.1

Implement batch K-Means and test it on Iris Data  [11] that can be downloaded at ftp.ics.uci.edu/pub/machine-learning-databases/iris. Plot the quantization error versus the number of iterations.

6.2

Can K-Means separate clusters nonlinearly separated using only two codevectors? And neural gas and SOM? Explain your answers.

6.3

Study experimentally (e.g., on Iris Data) how the initialization affects K-Means performances.

6.4

Suppose that the empirical quantization error $E(\mathcal {X})$ of a data set $\mathcal {X}=(\mathbf{x }_1,\dots ,\mathbf{x }_{\ell })$ assumes the following form:

$$ E(\mathcal {X}) = \frac{1}{2 \ell } \sum _{c=1}^K \sum _{x \in V_c } \left( G(\mathbf{x },\mathbf{x })-2G(\mathbf{x },\mathbf{w }_c)+G(\mathbf{w }_c,\mathbf{w }_c)\right) $$

where the function $G(\cdot )$ is $G(x,y)=\exp \left( -\frac{\Vert \mathbf{x }-\mathbf{y }\Vert ^2}{\sigma ^2}\right)$. Find the online K-Means learning rule, in this case.

6.5

Suppose that the empirical quantization error $E(\mathcal {X})$ of a data set $\mathcal {X}$ assumes the form of Exercise 4. Find the neural gas learning rule.

6.6

Implement K-Means online and test it on Wisconsin Breast Cancer Database [36] which can be dowloaded at ftp.ics.uci.edu/pub/machine-learning-databases/breast-cancer-wisconsin. Compare its performances with Batch K-Means’s ones. Use in both cases only two codevectors.

6.7

Use SOM-PAK on Wisconsin Breast Cancer Database. Divide the data in three parts. Train SOM on the first part of data (training set) changing number of codevectors and other neural network parameters (e.g. learning rate). Select the neural network configuration (best SOM) that has the best performance on the second part of data (validation set). Finally measure the best SOM performances on the third part of data (test set).

6.8

Using the function sammon of SOM-PAK visualize the codebook produced by best SOM (see Exercise 7).

6.9

Permute randomly Wisconsin Breast Cancer Database and repeat again the Exercise 7. Compare and discuss the results.

6.10

Implement neural gas and test it on Spam Data which can be dowloaded at ftp.ics.uci.edu/pub/machine-learning-databases/spam. Use only two codevectors.