Dimensionality Reduction Hybridizations with Multi-dimensional Scaling

Abstract

Dimensionality reduction is the task of mapping high-dimensional patterns to low-dimensional spaces while maintaining important information. In this paper, we introduce a hybrid dimensionality reduction method that is based on the weighted average of the normalized distance matrices of two or more embeddings. Multi-dimensional scaling embeds the weighted average distance matrix in a low-dimensional space. The approach allows the hybridization of arbitrary point-wise embeddings. Instances of the hybrid algorithm template use principal component analysis, multi-dimensional scaling, and locally linear embedding. The variants are experimentally compared using three dimensionality reduction measures, i.e., the Shepard-Kruskal scaling, a co-ranking matrix measure, and the nearest neighbor regression error in presence of label information. The results show that the hybrid approaches outperform their native pendants in the majority of the experiments.

Appendix A: Benchmark Problems

The experiments in this paper are based on the following data sets:

• The Swiss Roll is a simple artificial data set with $d=3$ that allows a visualization of neighborhoods with colored patterns and label information based on pattern colors from scikit-learn [11].

• The Housing data set, also known as California Housing from the StatLib repository [14] comprises 20,640 8-dimensional patterns and one label.

• Friedman is the high-dimensional regression problem Friedman #1 generated with scikit-learn [11] with $d=500$.