Optimizing Classification Ensembles via a Genetic Algorithm for a Web-Based Educational System

  • Behrouz Minaei-Bidgoli
  • Gerd Kortemeyer
  • William F. Punch
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3138)


Classification fusion combines multiple classifications of data into a single classification solution of greater accuracy. Feature extraction aims to reduce the computational cost of feature measurement, increase classifier efficiency, and allow greater classification accuracy based on the process of deriving new features from the original features. This paper represents an approach for classifying students in order to predict their final grades based on features extracted from logged data in an educational web-based system. A combination of multiple classifiers leads to a significant improvement in classification performance. By weighing feature vectors representing feature importance using a Genetic Algorithm (GA) we can optimize the prediction accuracy and obtain a marked improvement over raw classification. We further show that when the number of features is few, feature weighting and transformation into a new space works efficiently compared to the feature subset selection. This approach is easily adaptable to different types of courses, different population sizes, and allows for different features to be analyzed.


Genetic Algorithm Feature Selection Multiple Classifier Classification Fusion Simple Genetic Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Raymer, M.L., Punch, W.F., Goodman, E.D., Kuhn, L.A., Jain, A.K.: Dimensionality Reduction Using Genetic Algorithms. IEEE Transactions on Evolutionary Computation 4, 164–171 (2000)CrossRefGoogle Scholar
  2. 2.
    Jain, A.K., Zongker, D.: Feature Selection: Evaluation, Application, and Small Sample Performance. IEEE Transaction on Pattern Analysis and Machine Intelligence 19(2) (February 1997)Google Scholar
  3. 3.
    De Jong, K.A., Spears, W.M., Gordon, D.F.: Using genetic algorithms for concept learning. Machine Learning 13, 161–188 (1993)CrossRefGoogle Scholar
  4. 4.
    Bandyopadhyay, S., Muthy, C.A.: Pattern Classification Using Genetic Algorithms. Pattern Recognition Letters 16, 801–808 (1995)CrossRefGoogle Scholar
  5. 5.
    Bala, J., De Jong, K., Huang, J., Vafaie, H., Wechsler, H.: Using learning to facilitate the evolution of features for recognizing visual concepts. Evolutionary Computation 4(3) Special Issue on Evolution, Learning, and Instinct: 100 years of the Baldwin Effect (1997)Google Scholar
  6. 6.
    Guerra-Salcedo, C., Whitley, D.: Feature Selection mechanisms for ensemble creation: a genetic search perspective. In: Freitas, A. (ed.) Data Mining with Evolutionary Algorithms: Research Directions – Papers from the AAAI Workshop, pp. 13–17, Technical Report WS-99-06. AAAI Press, Menlo Park (1999)Google Scholar
  7. 7.
    Vafaie, H., De Jong, K.: Robust feature Selection algorithms. In: Proceeding of IEEE International Conference on Tools with AI, Boston, Mass., USA, November 1993, pp. 356–363 (1993)Google Scholar
  8. 8.
    Martin-Bautista, M.J., Vila, M.A.: A survey of genetic feature selection in mining issues. In: Proceeding Congress on Evolutionary Computation (CEC 1999), Washington D.C, July 1999, pp. 1314–1321 (1999)Google Scholar
  9. 9.
    Pei, M., Goodman, E.D., Punch, W.F.: Pattern Discovery from Data Using Genetic Algorithms. In: Proceeding of 1st Pacific-Asia Conference Knowledge Discovery & Data Mining, PAKDD 1997 (1997)Google Scholar
  10. 10.
    Punch, W.F., Pei, M., Chia-Shun, L., Goodman, E.D., Hovland, P., Enbody, R.: Further research on Feature Selection and Classification Using Genetic Algorithms. In: 5th International Conference on Genetic Algorithm, Champaign IL, pp. 557–564 (1993)Google Scholar
  11. 11.
    Kuncheva, L.I., Jain, L.C.: Designing Classifier Fusion Systems by Genetic Algorithms. IEEE Transaction on Evolutionary Computation 33, 351–373 (2000)Google Scholar
  12. 12.
    Skalak, D.B.: Using a Genetic Algorithm to Learn Prototypes for Case Retrieval an Classification. In: Proceeding of the AAAI 1993 Case-Based Reasoning Workshop, Washigton, D.C, pp. 64–69. American Association for Artificial Intelligence, Menlo Park (1994)Google Scholar
  13. 13.
    John, G.H., Kohavi, R., Pfleger, K.: Irrelevant Features and the Subset Selection Problem. In: Proceedings of the Eleventh International Conference of Machine Learning, pp. 121–129. Morgan Kaufmann Publishers, San Francisco (1994)Google Scholar
  14. 14.
    Goldberg, D.E.: Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Reading (1989)zbMATHGoogle Scholar
  15. 15.
    Muhlenbein, Schlierkamp-Voosen, D.: Predictive Models for the Breeder Genetic Algorithm: I. Continuous Parameter Optimization. Evolutionary Computation 1(1), 25–49 (1993)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Behrouz Minaei-Bidgoli
    • 1
  • Gerd Kortemeyer
    • 2
  • William F. Punch
    • 1
  1. 1.Genetic Algorithms Research and Applications Group (GARAGe), Department of Computer Science & EngineeringMichigan State UniversityEast LansingUSA
  2. 2.Division of Science and Math EducationMichigan State University, College of Natural Science, LITE labEast LansingUSA

Personalised recommendations