Nonlinear Dimensionality Reduction

Methods of dimensionality reduction provide a way to understand and visualize the structure of complex data sets. Traditional methods like principal component analysis and classical metric multidimensional scaling suffer from being based on linear models. Until recently, very few methods were able to reduce the data dimensionality in a nonlinear way. However, since the late nineties, many new methods have been developed and nonlinear dimensionality reduction, also called manifold learning, has become a hot topic. New advances that account for this rapid growth are, e.g. the use of graphs to represent the manifold topology, and the use of new metrics like the geodesic distance. In addition, new optimization schemes, based on kernel techniques and spectral decomposition, have lead to spectral embedding, which encompasses many of the recently developed methods.

This book describes existing and advanced methods to reduce the dimensionality of numerical databases. For each method, the description starts from intuitive ideas, develops the necessary mathematical details, and ends by outlining the algorithmic implementation. Methods are compared with each other with the help of different illustrative examples.

The purpose of the book is to summarize clear facts and ideas about well-known methods as well as recent developments in the topic of nonlinear dimensionality reduction. With this goal in mind, methods are all described from a unifying point of view, in order to highlight their respective strengths and shortcomings.

The book is primarily intended for statisticians, computer scientists and data analysts. It is also accessible to other practitioners having a basic background in statistics and/or computational learning, like psychologists (in psychometry) and economists.

• Principal component analysis
• algorithms
• computer
• database
• learning
• multidimensional scaling
• optimization
• statistics
• topology

From the reviews:

"This beautifully produced book covers various innovative topics in nonlinear dimensionality reduction, such as Isomap, locally linear embedding, and Laplacian eigenmaps, etc. Those topics are usually not covered by existing texts on multivariate statistical techniques. Moreover, the text offers an excellent overview of the concept of intrinsic dimension. Special attention is devoted to the topic of estimation of the intrinsic dimension, which has been previously overlooked by many researchers.… A strong feature of the book is the style of presentation. The book is clearly written, …A large number of examples and graphical displays in color help the reader in understanding the ideas. For each method discussed, the authors do a credible job by starting from motivating examples and intuitive ideas, introducing rigorous mathematical notation without being cumbersome, and ending with discussion and conclusion remarks. All in all, this is an interesting book, and I would recommend this text to those researchers who want to learn quickly about this new field of manifold learning. This book will serve as a useful and necessary resource to several advanced statistics courses in machine learning and data mining.… In addition, the Matlab and R packages will surely enhance the learning resources and increase the accessibility of this book to data analysts. " (Haonan Wang, Biometrics, June 2009, 65)

"The book by Lee and Verleysen presents a comprehensive summary of the state-of-the-art of the field in a very accessible manner. It is the only book I know that offers such a thorough and systematic account of this interesting and important area of research. … Reading the book is quite enjoyable … ." (Lasse Holmström, International Statistical Reviews, Vol. 76 (2), 2008)

"The book provides an effective guide for selecting the right method and understanding potential pitfalls and limitations of themany alternative methods. … All in all, Nonlinear Dimensionality Reduction may serve two groups of readers differently. To the reader already immersed in the field it is a convenient compilation of a wide variety of algorithms with references to further resources. To students or professionals in areas outside of machine learning or statistics … it can be highly recommended as an introduction." (Kilian Q. Weinberger, Journal of the American Statistical Association, Vol. 104 (485), March, 2009)

• Molecular Imaging and Experimental Radiotherapy, Université catholique de Louvain, B-1200 Bruxelles, Belgium

  John A. Lee

• Machine Learning Group - DICE, Université catholique de Louvain, Belgium

  Michel Verleysen

Policies and ethics