Skip to main content

Random Projections for SVM Ensembles

  • Conference paper
  • 1182 Accesses

Part of the Lecture Notes in Computer Science book series (LNAI,volume 6097)


Data projections have been used extensively to reduce input space dimensionality. Such reduction is useful to get faster results, and sometimes can help to discard unnecessary or noisy input dimensions. Random Projections (RP) can be computed faster than other methods as for example Principal Component Analysis (PCA). This paper presents an experimental study over 62 UCI datasets of three types of RPs taking into account the size of the projected space and using linear SVMs as base classifiers. We also combined random projections with sparse matrix strategy used by Rotation Forests, which is a method based in projections too. Results shows that Random Projections use to be better than using PCA for SVMs ensembles.


  • Ensembles
  • Random Projections
  • Rotation Forests
  • Diversity
  • Kappa-Error Relative Movement diagrams

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Achlioptas, D.: Database-friendly random projections: Johnson-Lindenstrauss with binary coins. J. Comput. Syst. Sci. 66(4), 671–687 (2003)

    CrossRef  MathSciNet  MATH  Google Scholar 

  2. Fradkin, D., Madigan, D.: Experiments with random projections for machine learning. In: KDD 2003: Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 517–522. ACM, New York (2003)

    Google Scholar 

  3. Vapnik, V.N.: The Nature of Statistical Learning Theory (Information Science and Statistics). Springer, Heidelberg (1999)

    Google Scholar 

  4. Rodríguez, J.J., Kuncheva, L.I., Alonso, C.J.: Rotation forest: A new classifier ensemble method. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(10), 1619–1630 (2006)

    CrossRef  Google Scholar 

  5. Kuncheva, L.I., Rodríguez, J.J.: An experimental study on rotation forest ensembles. In: Haindl, M., Kittler, J., Roli, F. (eds.) MCS 2007. LNCS, vol. 4472, pp. 459–468. Springer, Heidelberg (2007)

    CrossRef  Google Scholar 

  6. Johnson, W., Lindenstrauss, J.: Extensions of Lipschitz maps into a Hilbert space. In: Conference in Modern Analysis and Probability. Yale University, New Haven and London (1982); Vol. 26 of Contemporary Mathematics, AMS, 189–206 (1984)

    Google Scholar 

  7. Ho, T.K.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)

    CrossRef  Google Scholar 

  8. Witten, I., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005),

    MATH  Google Scholar 

  9. Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: Liblinear: A library for large linear classification. Journal of Machine Learning Research 9, 1871–1874 (2008)

    MATH  Google Scholar 

  10. Asuncion, A., Newman, D.: UCI machine learning repository (2007),

  11. Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)

    MATH  Google Scholar 

  12. Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55(1), 119–139 (1997)

    CrossRef  MathSciNet  MATH  Google Scholar 

  13. Webb, G.I.: Multiboosting: A technique for combining boosting and wagging. Machine Learning 40(2) (2000)

    Google Scholar 

  14. Demšar, J.: Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1–30 (2006)

    MathSciNet  MATH  Google Scholar 

  15. Margineantu, D.D., Dietterich, T.G.: Pruning adaptive boosting. In: Proc. 14th International Conference on Machine Learning, pp. 211–218. Morgan Kaufmann, San Francisco (1997)

    Google Scholar 

  16. Maudes, J., Rodríguez, J.J., García-Osorio, C.: Disturbing neighbors diversity for decision forests. In: Okun, O., Valentini, G. (eds.) Workshop on Supervised and Unsupervised Ensemble Methods and their Applications, SUEMA 2008, pp. 67–71 (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations


Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Maudes, J., Rodríguez, J.J., García-Osorio, C., Pardo, C. (2010). Random Projections for SVM Ensembles. In: García-Pedrajas, N., Herrera, F., Fyfe, C., Benítez, J.M., Ali, M. (eds) Trends in Applied Intelligent Systems. IEA/AIE 2010. Lecture Notes in Computer Science(), vol 6097. Springer, Berlin, Heidelberg.

Download citation

  • DOI:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-13024-3

  • Online ISBN: 978-3-642-13025-0

  • eBook Packages: Computer ScienceComputer Science (R0)