Dimensionality Reduction

Abstract

We closed the previous chapter by introducing the "curse of dimensionality", which refers to the sparseness problem that typically affects models involving a very large number of variables, i.e. high dimensionality spaces. This problem is alleviated in practice by the use of dimensionality reduction techniques, which aim at reducing the sparseness of the data representation by projecting the original model into a new space of lower dimensionality. There exist several different approaches to dimensionality reduction, as well as it constitutes a very common practice in data mining applications. Indeed, almost every standard data mining method or procedure involves some sort of dimensionality reduction. In this chapter we will focus our attention on three basic methods for dimensionality reduction. First, in Sect. 9.1, vocabulary pruning and merging methods will be presented. Then, in Sect. 9.2, the linear transformation approach to dimensionality reduction will be introduced and discussed. In Sect. 9.3, the use of non-linear projection methods for dimensionality reduction will be described. Finally, some relevant references to other important and commonly used methods are provided in the Further Reading section at the end of the chapter.