Sampling of Virtual Examples to Improve Classification Accuracy for Nominal Attribute Data

  • Yujung Lee
  • Jaeho Kang
  • Byoungho Kang
  • Kwang Ryel Ryu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4259)


This paper presents a method of using virtual examples to improve the classification accuracy for data with nominal attributes. Most of the previous researches on virtual examples focused on data with numeric attributes, and they used domain-specific knowledge to generate useful virtual examples for a particularly targeted learning algorithm. Instead of using domain-specific knowledge, our method samples virtual examples from a naïve Bayesian network constructed from the given training set. A sampled example is considered useful if it contributes to the increment of the network’s conditional likelihood when added to the training set. A set of useful virtual examples can be collected by repeating this process of sampling followed by evaluation. Experiments have shown that the virtual examples collected this way can help various learning algorithms to derive classifiers of improved accuracy.


Classification Accuracy Bayesian Network Statistical Significance Test Nominal Attribute Conditional Likelihood 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)MATHMathSciNetGoogle Scholar
  2. 2.
    Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Proceedings of the International Conference on Machine Learning, pp. 148–156 (1996)Google Scholar
  3. 3.
    Wolpert, D.H.: Stacked Generalization Neural Networks. vol. 5, pp. 241–259 (1992)Google Scholar
  4. 4.
    Aha, D.W.: Tolerating Noisy, Irrelevant, and Novel Attributes in Instance-based Learning Algorithms. International Journal of Man-Machine Studies 36(2), 267–287 (1992)CrossRefGoogle Scholar
  5. 5.
    Kohavi, R., Sahami, M.: Error-based and Entropy-based Discretization of Continuous Features. In: Proc. of the 2nd International Conference on Knowledge Discovery and Data Mining, pp. 114–119 (1996)Google Scholar
  6. 6.
    Pazzani, M.: Constructive induction of Cartesian product attributes. Information, Statistics and Induction in Science, 66–77 (1996)Google Scholar
  7. 7.
    Almuallim, H., Dietterich, T.G.: Learning With Many Irrelevant Features. In: Proc. of the 9th National Conference on Artificial Intelligence, pp. 547–552 (1991)Google Scholar
  8. 8.
    Nigam, K., McCallum, A., Thrun, S., Mitchell, T.: Text Classification from Labeled and Unlabeled Documents using EM. Machine Learning 39(2), 103–134 (1999)CrossRefGoogle Scholar
  9. 9.
    Cho, S., Cha, K.: Evolution of Neural Network Training Set through Addition of Virtual samples. In: Proc. of the 1996 IEEE International Conference on Evolutionary Computation, pp. 685–688 (1996)Google Scholar
  10. 10.
    Marcel, S.: Improving Face Verification using Symmetric Transformation. In: Proc. of IEICE Transactions on Information and Systems, vol. E81(1), pp. 124–135 (1993)Google Scholar
  11. 11.
    Wen, J., Zhao, J., Luo, S., Huang, H.: Face Recognition Method Based on Virtual Sample. In: Proc. of the ICII, vol. 3, pp. 557–562 (2001)Google Scholar
  12. 12.
    Thian, N.P.H., Marcel, S., Bengio, S.: Improving Face Authentication Using Virtual Samples. In: Proc. of the Acoustics, Speech, and Signal International Conference, vol. 3, pp. 233–236 (2003)Google Scholar
  13. 13.
    Burges, C., Scholkopf, B.: Improving the Accuracy and Speed of Support Vector Machines. In: Proc. of Advances in Neural Information Processing System, vol. 9(7) (1997)Google Scholar
  14. 14.
    Miyao, H., Maruyama, M., Nakano, Y., Hanamoi, T.: Off-line handwritten character recognition by SVM based on the virtual examples synthesized from on-line characters. In: Proc. of Eighth International Conference on Document Analysis and Recognition, vol. 1, pp. 494–498 (2005)Google Scholar
  15. 15.
    Lee, K.S., An, D.U.: Performance Improvement by a Virtual Documents Technique in Text Categorization. Korea Information Processing Society 11-B(4), 501–508 (2004)Google Scholar
  16. 16.
    Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Machine Learning 29, 131–163 (1997)MATHCrossRefGoogle Scholar
  17. 17.
    John, G., Langley, P.: Estimating Continuous Distributions in Bayesian Classifiers. In: Proc. of the Eleventh Conference on Uncertainty in Artificial Intelligence, pp. 338–345 (1995)Google Scholar
  18. 18.
    Duda, R.O., Hart, P.E.: Pattern Classification and scene analysis. Wiley, New York (1973)MATHGoogle Scholar
  19. 19.
    Grossman, D., Domingos, P.: Learning Bayesian Network Classifiers by Maximizing Conditional Likelihood. In: Proc. of the 21th International Conference on Machine Learning, pp. 361–368 (2004)Google Scholar
  20. 20.
    Greiner, R., Zhou, W.: Structural Extension to Logistic Regression: Discriminative parameter learning of belief net classifiers. In: Proc. of the 18th National Conference on Artificial Intelligence, pp. 167–173 (2002)Google Scholar
  21. 21.
    Burge, J., Lane, T.: Learning Class-Discriminative Dynamic Bayesian Networks. In: Proc. of the 22th International Conference on Machine Learning, pp. 97–104 (2005)Google Scholar
  22. 22.
    Roy, N., McCallum, A.: Toward optimal active learning through sampling estimation of error reduction. In: Proc. of the 18th International Conference on Machine Learning, pp. 441–448 (2004)Google Scholar
  23. 23.
    Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI Repository of machine learning databases. University of California, CA, Department of Information and Computer Science (1998),
  24. 24.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, San Francisco (1993)Google Scholar
  25. 25.
    Platt, I.J.: Fast Training of Support Vector Machines using Sequential Minimal Optimization. In: Schoelkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods - Support Vector Learning. MIT Press, Cambridge (1998)Google Scholar
  26. 26.
    Witten, I.H., Frank, E.: Data Mining-Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufman Publishers, San Francisco (1999)Google Scholar
  27. 27.
    Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network Classifiers. Machine Learning 29, 131–163 (1997)MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Yujung Lee
    • 1
  • Jaeho Kang
    • 1
  • Byoungho Kang
    • 1
  • Kwang Ryel Ryu
    • 1
  1. 1.Department of Computer EngineeringPusan National UniversityBusanKorea

Personalised recommendations