Abstract
This Letter presents an approach to using both labelled and unlabelled data to train a multilayer perceptron. The unlabelled data are iteratively pre-processed by a perceptron being trained to obtain the soft class label estimates. It is demonstrated that substantial gains in classification performance may be achieved from the use of the approach when the labelled data do not adequately represent the entire class distributions. The experimental investigations performed have shown that the approach proposed may be successfully used to train neural networks for learning different classification problems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baluja, S.: Probabilistic modeling for face orientation discrimination: Learning from labeled and unlabeled data. In: M. S. Kearns, S. A. Solla and D. A. Cohn (eds), Advances in Neural Information Processing Systems 11, MIT Press, 1999, pp. 854–860.
Bensaid, A.M. and Hall, L. O.: Partially Supervised Clustering for Image Segmentation, Pattern Recognition 29(5) (1996), 859–871.
Bishop, C.: Neural Networks for Pattern Recognition, Oxford University Press, 1995.
Castelli, V. and Cover, T. M.: On the exponential value of labeled samples, Pattern Recognition Letters 16 (1995), 105–111.
Cataltepe, Z. and Magdon-Ismail, M. I: Incorporating test inputs into learning, In:M. I. Jordan, M. J. Kearns and S. A. Solla (eds), Advances in Neural Information Processing Systems 10, MIT Press, 1998, pp. 437–443.
Dempster, A. P., Laird, N. M. and Rubin, D. B.: Maximum-likelihood from incomplete data via the EM algorithm, Journal of the Royal Statistical Society, Series B, 39 (1977), 1–38.
Foresee, F. D. and Hagan, M. T.: Gauss-Newton approximation to Bayesian learning, In: Proceedings of the IEEE International Joint Conference on Neural Networks, 1997, pp. 1930–1935.
Hagan, M. T. and Menhaj, M.: Training multilayer networks with the Marquardt algorithm, IEEE Trans Neural Networks 6(5) (1994), 989–993.
Intrator, N., Reisfeld, D. and Yeshurun, Y.: Face recognition using a hybrid supervised /unsupervised neural network, Pattern Recognition Letters 17 (1996), 67–76.
Kerhagias, A. and Petridis, V.: Time-series segmentation using predictive modular neural networks, Neural Computation 9(1997), 1691–1709.
Klir, G. S. and Yuan, B.: Fuzzy sets and Fuzzy Logic - Theory and applications, Prentice-Hall, Englewood Cliffs, 1995.
Lippman, R. P.: Pattern Classification Using Neural Networks, IEEE Communications Magazine 27 (1989), 47–64.
MacKay, D. J.: Bayesian interpolation, Neural Computation 4 (1992), 415–447.
Miller, D. J. and Uyar, H. S.: Combined learning and use for a mixture model equivalent to the RBF classifier, Neural Computation 10 (1998), 281–293.
Miller, D. J. and Uyar, H. S.: A mixture of experts classifier with learning based on both labelled and unlabelled data, In: M. C. Mozer, M. I. Jordan and T. Petsche (eds). Advances in Neural Information Processing Systems 9, MIT Press, 1997, pp. 571–577.
Ruck, D. W., Rogers, S. K., Kabrisky, M., Oxley, M. E. and Suter, B. W.: Themultilayer perceptron as an approximation to a Bayes optimal discriminant function, IEEE Trans. Neural Networks 1 (1990), 296–298.
Sarkar, M., Yegnanarayana, B. and Khemani, D.: Backpropagation learning algorithms for classification with fuzzy mean square error, Pattern Recognition Letters 19 (1998), 43–51.
Shashahani, B. and Landgrebe, D.: The Effect on Unlabelled Samples in Reducing the Small Sample Size Problem and Mitigating the Huges Phenomenon, IEEE Transactions on Geoscience and Remote Sensing 32 (1994), 1087–1095.
Shashahani, B. and Landgrebe, D.: On the asymptotic improvement of supervised learning by utilizing additional unlabeled samples; normal mixture density case, SPIE 1766 (1992), 143–155.
Towell, G.: Using unlabeled data for supervised learning, In: M. C. Mozer, M. I. Jordan, T. Petsche (eds), Advances in Neural Information Processing Systems 9, MIT Press, 1997, pp. 647–653.
Verikas, A., Malmqvist, K., Signahl, M. and Bergman, L.: Colour Classification by Neural Networks in Graphic Arts, Neural Computing & Applications 7(1998), 52–64.
Verikas, A., Malmqvist, K. and Bergman, L.: Colour Image Segmentation by Modular Neural Network, Pattern Recognition Letters 18 (1997), 173–185.
Verikas, A., Gelzinis, A. and Malmqvist, K.: Using labelled and unlabelled data to train a multilayer perceptron for colour classification in graphic arts, In: L. Imam, Y. Kodratoff, A. El-Dessouki and M. Ali (eds), Lecture Notes in Artificial Intelligence 1611, Multiple Approaches to Intelligent Systems, Springer-Verlag Heidelberg, 1999, pp. 550–559.
Verikas, A., Malmqvist, K., Bacauskiene, M. and Bergman, L.: Monitoring the De-Inking Process through Neural Network Based Colour Image Analysis, Neural Computing & Applications 9 (2000), 142–151.
Wan, E. A.: Neural network classification: A Bayesian interpretation, IEEE Trans Neural Networks 1 (1990), 303–305.
Yoon, S. Y. and Lee, S. Y.: Training algorithm with incomplete data for feed-forward neural networks, Neural Processing Letters 10 (1999), 171–179.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Verikas, A., Gelzinis, A. & Malmqvist, K. Using Unlabelled Data to Train a Multilayer Perceptron. Neural Processing Letters 14, 179–201 (2001). https://doi.org/10.1023/A:1012707515770
Issue Date:
DOI: https://doi.org/10.1023/A:1012707515770