Skip to main content
SpringerLink
Log in

Substructural Surrogates for Learning Decomposable Classification Problems

  • Conference paper
Learning Classifier Systems (IWLCS 2006, IWLCS 2007)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 4998))

Abstract

This paper presents a learning methodology based on a substructural classification model to solve decomposable classification problems. The proposed method consists of three important components: (1) a structural model, which represents salient interactions between attributes for a given data, (2) a surrogate model, which provides a functional approximation of the output as a function of attributes, and (3) a classification model, which predicts the class for new inputs. The structural model is used to infer the functional form of the surrogate. Its coefficients are estimated using linear regression methods. The classification model uses a maximally-accurate, least-complex surrogate to predict the output for given inputs. The structural model that yields an optimal classification model is searched using an iterative greedy search heuristic. Results show that the proposed method successfully detects the interacting variables in hierarchical problems, groups them in linkages groups, and builds maximally accurate classification models. The initial results on non-trivial hierarchical test problems indicate that the proposed method holds promise and also shed light on several improvements to enhance the capabilities of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Baluja, S.: Incorporating a priori Knowledge in Probabilistic-Model Based Optimization. In: Pelikan, M., Sastry, K., Cantú-Paz, E. (eds.) Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications ch. 9, pp. 205–219. Springer, Berlin (2006)

    Chapter  Google Scholar 

  2. Bernadó-Mansilla, E., Garrell, J.M.: Accuracy-Based Learning Classifier Systems: Models, Analysis and Applications to Classification Tasks. Evolutionary Computation 11(3), 209–238 (2003)

    Article  Google Scholar 

  3. Butz, M.V.: Rule-Based Evolutionary Online Learning Systems: A Principled Approach to LCS Analysis and Design. In: Studies in Fuzziness and Soft Computing, vol. 109. Springer, Heidelberg (2006)

    Google Scholar 

  4. Butz, M.V., Pelikan, M., Llorà, X., Goldberg, D.E.: Automated Global Structure Extraction for Effective Local Building Block Processing in XCS. Evolutionary Computation 14(3), 345–380 (2006)

    Article  Google Scholar 

  5. Drapper, N.R., Smith, H.: Applied Regression Analysis. John Wiley & Sons, New York (1966)

    Google Scholar 

  6. Gibson, J.J.: The Ecological Approach to Visual Perception. Lawrence Erlbaum Associates, Mahwah (1979)

    Google Scholar 

  7. Goldberg, D.E.: Genetic Algorithms in Search, Optimization & Machine Learning, 1st edn. Addison Wesley, Reading (1989)

    MATH  Google Scholar 

  8. Goldberg, D.E.: The Design of Innovation: Lessons from and for Competent Genetic Algorithms, 1st edn. Kluwer Academic Publishers, Dordrecht (2002)

    Book  MATH  Google Scholar 

  9. Harik, G.: Linkage Learning via Probabilistic Modeling in the ECGA. Technical report. University of Illinois at Urbana-Champaign, Urbana, IL (January 1999) (IlliGAL Report No. 99010)

    Google Scholar 

  10. Harik, G.R., Lobo, F.G., Sastry, K.: Linkage Learning via Probabilistic Modeling in the ECGA. In: Pelikan, M., Sastry, K., Cantú-Paz, E. (eds.) Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications ch. 3, pp. 39–61. Springer, Berlin (2006) (Also IlliGAL Report No. 99010)

    Chapter  Google Scholar 

  11. Holland, J.H.: Adaptation in Natural and Artificial Systems. The University of Michigan Press (1975)

    Google Scholar 

  12. De Jong, K.A., Spears, W.M.: Learning Concept Classification Rules Using Genetic Algorithms. In: Proceedings of the International Joint Conference on Artificial Intelligence, Sidney, Australia, pp. 651–656 (1991)

    Google Scholar 

  13. Keerthi, S.S., Lin, C.J.: Asymptotic Behaviors of Support Vector Machines with Gaussian Kernel. Neural Computation 15(7), 1667–1689 (2003)

    Article  MATH  Google Scholar 

  14. Korst, J., Aarts, E.: Simulated Annealing and Boltzmann Machines. Wiley-Interscience, New York (1997)

    MATH  Google Scholar 

  15. Kovacs, T.: Deletion Schemes for Classifier Systems. In: GECCO 1999: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 329–336. Morgan Kaufmann, San Francisco (1999)

    Google Scholar 

  16. Llorà, X., Sastry, K., Goldberg, D.E., de la Ossa, L.: The χ-ary extended compact classifier system: Linkage learning in Pittsburgh LCS. In: Proceedings of the 2006 Genetic and Evolutionary Computation Conference Workshop Program. ACM Press, Berlin (2006) (Also IlliGAL Report No. 2006015)

    Google Scholar 

  17. Llorà, X., Sastry, K., Yu, T.-L., Goldberg, D.E.: Do not match, inherit: Fitness surrogates for genetics-based machine learning. In: Proceedings of the 2007 Genetic and Evolutionary Computation Conference, vol. 2, pp. 1798–1805 (2007)

    Google Scholar 

  18. Pelikan, M.: Hierarchical Bayesian Optimization Algorithm: Toward a new Generation of Evolutionary Algorithms. Springer, Berlin (2005)

    Book  MATH  Google Scholar 

  19. Pelikan, M., Sastry, K.: Fitness inheritance in the Bayesian optimization algorithm. In: Proceedings of the 2004 Genetic and Evolutionary Computation Conference, vol. 2, pp. 48–59 (2004) (Also IlliGAL Report No. 2004009)

    Google Scholar 

  20. Pelikan, M., Sastry, K., Cantú-Paz, E. (eds.): Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications. Studies in Computational Intelligence, vol. 33. Springer, Heidelberg (2006)

    MATH  Google Scholar 

  21. Platt, J.: Fast Training of Support Vector Machines using Sequential Minimal Optimization. In: Advances in Kernel Methods - Support Vector Learning, pp. 557–563. MIT Press, Cambridge (1998)

    Google Scholar 

  22. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, San Mateo (1995)

    Google Scholar 

  23. Rao, C.R., Toutenburg, H.: Linear Models: Least Squares and Alternatives. Springer, Berlin (1999)

    MATH  Google Scholar 

  24. Recktenwald, G.: Numerical Methods with MATLAB: Implementations and Applications. Prentice Hall, Englewood Cliffs (2000)

    Google Scholar 

  25. Sastry, K., Goldberg, D.E.: Probabilistic Model Building and Competent Genetic Programming. In: Riolo, R.L., Worzel, B. (eds.) Genetic Programming Theory and Practise, ch. 13, pp. 205–220. Kluwer, Dordrecht (2003)

    Chapter  Google Scholar 

  26. Sastry, K., Lima, C.F., Goldberg, D.E.: Evaluation Relaxation Using Substructural Information and Linear Estimation. In: GECCO 2006: Proceedings of the 8th annual Conference on Genetic and Evolutionary Computation, pp. 419–426. ACM Press, New York (2006)

    Google Scholar 

  27. Sastry, K., Pelikan, M., Goldberg, D.E.: Efficiency enhancement of genetic algorithms via building-block-wise fitness estimation. In: Proceedings of the IEEE International Conference on Evolutionary Computation, pp. 720–727 (2004) (Also IlliGAL Report No. 2004010)

    Google Scholar 

  28. Simon, H.A.: Sciences of the Artificial. MIT Press, Cambridge (1969)

    Google Scholar 

  29. Dietterich, T.G.: Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms. Neural Comp. 10(7), 1895–1924 (1998)

    Article  Google Scholar 

  30. Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)

    Book  MATH  Google Scholar 

  31. Wilson, S.W.: Quasi-Darwinian Learning in a Classifier System. In: 4th IWML, pp. 59–65. Morgan Kaufmann, San Francisco (1987)

    Google Scholar 

  32. Wilson, S.W.: Classifier Fitness Based on Accuracy. Evolutionary Computation 3(2), 149–175 (1995)

    Article  Google Scholar 

  33. Wilson, S.W.: Generalization in the XCS Classifier System. In: 3rd Annual Conf. on Genetic Programming, pp. 665–674. Morgan Kaufmann, San Francisco (1998)

    Google Scholar 

  34. Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)

    MATH  Google Scholar 

  35. Yu, T.-L.: A matrix approach for finding extrema: Problems with modularity, hierarchy, and overlap. PhD thesis, University of Illinois at Urbana-Champaign, Urbana, IL (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Grup de Recerca en Sistemes Intel·ligents, Enginyeria i Arquitectura La Salle, Universitat Ramon Llull, Quatre Camins 2, 08022, Barcelona, Spain

    Albert Orriols-Puig & Ester Bernadó-Mansilla

  2. Illinois Genetic Algorithms Laboratory, Department of Industrial and Enterprise Systems Engineering, University of Illinois, Urbana-Champaign, USA

    Albert Orriols-Puig, Kumara Sastry & David E. Goldberg

Authors
  1. Albert Orriols-Puig
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kumara Sastry
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David E. Goldberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ester Bernadó-Mansilla
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Nottingham, School of Computer Science, ASAP research group, Jubilee Campus, Nottingham, NG8 1BB, and Multidisciplinary Centre for Integrative Biology, School of Biosciences, Sutton Bonington, LE12 5RD, UK

    Jaume Bacardit

  2. Enginyeria i Arquitectura La Salle, Gruß de Recerca en Sistemes Intel.ligents, Quatre Camins 2, Universitat Ramon Llull, 08022, Barcelona, Spain

    Ester Bernadó-Mansilla

  3. Department of Psychology, University of Würzburg, Röntgenring 11, 97070, Würzburg, Germany

    Martin V. Butz

  4. Department of Computer Science, University of Bristol, Merchant Venturers Building, Woodland Road, BS8 1UB, Bristol, UK

    Tim Kovacs

  5. Department of Industrial and Enterprise Systems Engineering, Illinois Genetic Algorithms Lab (IlliGAL), University of Illinois at Urbana-Champaign, 104 S. Mathews Avenue, 61801-2996, Urbana, IL, USA

    Xavier Llorà

  6. Tokyo Institute of Technology, 152-8550, tokyo, Japan

    Keiki Takadama

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Orriols-Puig, A., Sastry, K., Goldberg, D.E., Bernadó-Mansilla, E. (2008). Substructural Surrogates for Learning Decomposable Classification Problems. In: Bacardit, J., Bernadó-Mansilla, E., Butz, M.V., Kovacs, T., Llorà, X., Takadama, K. (eds) Learning Classifier Systems. IWLCS IWLCS 2006 2007. Lecture Notes in Computer Science(), vol 4998. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88138-4_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-88138-4_14

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-88137-7

  • Online ISBN: 978-3-540-88138-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation