Abstract
Rather than presenting a specific trick, this paper aims at providing a methodology for large scale, real-world classification tasks involving thousands of classes and millions of training patterns. Such problems arise in speech recognition, handwriting recognition and speaker or writer identification, just to name a few. Given the typically very large number of classes to be distinguished, many approaches focus on parametric methods to independently estimate class conditional likelihoods. In contrast, we demonstrate how the principles of modularity and hierarchy can be applied to directly estimate posterior class probabilities in a connectionist framework. Apart from offering better discrimination capability, we argue that a hierarchical classification scheme is crucial in tackling the above mentioned problems. Furthermore, we discuss training issues that have to be addressed when an almost infinite amount of training data is available.
Previously published in: Orr, G.B. and Müller, K.-R. (Eds.): LNCS 1524, ISBN 978-3-540-65311-0 (1998).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Baum, E.B., Haussler, D.: What Size Net Gives Valid Generalization? Neural Computation 1, 151–160 (1989)
Bishop Training, C.M.: with Noise is Equivalent to Tikhonov Regularization. Neural Computation 7(1), 108–116 (1995)
Bourlard, H., Morgan, N.: Connectionist Speech Recognition – A Hybrid Approach. Kluwer Academic Press (1994)
Bourlard, H., Morgan, N.: A Context Dependent Neural Network for Continuous Speech Recognition. In: IEEE Proc. Intl. Conf. on Acoustics, Speech and Signal Processing, San Francisco, CA, vol. 2, pp. 349–352 (1992)
Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth International Group, Belmont (1984)
Bridle, J.: Probabilistic Interpretation of Feed Forward Classification Network Outputs, with Relationships to Statistical Pattern Recognition. In: Fogelman-Soulie, F., Hérault, J. (eds.) Neurocomputing: Algorithms, Architectures, and Applications. Springer, New York (1990)
Duda, R., Hart, P.: Pattern Classification and Scene Analysis. John Wiley & Sons, Inc. (1973)
Finke, M., Fritsch, J., Geutner, P., Ries, K., Zeppenfeld, T.: The JanusRTk Switchboard/Callhome 1997 Evaluation System. In: Proceedings of LVCSR Hub5-e Workshop, Baltimore, Maryland, May 13-15 (1997)
Franco, H., Cohen, M., Morgan, N., Rumelhart, D., Abrash, V.: Context-Dependent Connectionist Probability Estimation in a Hybrid Hidden Markov Model – Neural Net Speech Recognition System. Computer Speech and Language 8(3), 211–222 (1994)
Fritsch, J., Finke, M.: ACID/HNN: Clustering Hierarchies of Neural Networks for Context-Dependent Connectionist Acoustic Modeling. In: Proceedings of International Conference on Acoustics, Speech and Signal Processing, Seattle, Wa (May 1998)
Fritsch, J.: ACID/HNN: A Framework for Hierarchical Connectionist Acoustic Modeling. In: Proceedings of IEEE Workshop on Automatic Speech Recognition and Understanding, Santa Barbara, Ca (December 1997)
Fritsch, J., Finke, M., Waibel, A.: Context-Dependent Hybrid HME/HMM Speech Recognition using Polyphone Clustering Decision Trees. In: Intl. Conf. on Acoustics, Speech and Signal Processing, Munich, Germany, vol. 3, p. 1759 (1997)
Fritsch, J.: Modular Neural Networks for Speech Recognition, Tech. Report CMU-CS-96-203, Carnegie Mellon University, Pittsburgh, PA (1996)
Hochberg, M.M., Cook, G.D., Renals, S.J., Robinson, A.J., Schechtman, R.S.: The 1994 ABBOT Hybrid Connectionist-HMM Large-Vocabulary Recognition System. In: Spoken Language Systems Technology Workshop, pp. 170–176. ARPA (January 1995)
Kershaw, D.J., Hochberg, M.M., Robinson, A.J.: Context-Dependent Classes in a Hybrid Recurrent Network-HMM Speech Recognition System, Tech. Rep. CUED/F-INFENG/TR217, Cambridge University Engineering Department, Cambridge, England (1995)
Merz, C.J., Murphy, P.M.: UCI Repository of Machine Learning Databases, University of California, Department of Information and Computer Science (1996), http://www.ics.uci.edu/~mlearn/MLRepository.html
Morgan, N., Bourlard, H.: Factoring Networks by a Statistical Method. Neural Computation 4(6), 835–838 (1992)
Morgan, N., Bourlard, H.: An Introduction to Hybrid HMM/Connectionist Continuous Speech Recognition Signal Processing Magazine, 25–42 (May 1995)
NIST, Conversational Speech Recognition Workshop, DARPA Hub-5E Evaluation, May 13-15, Baltimore, Maryland (1997)
Prechelt, L.: Proben1 – A Set of Neural Network Benchmark Problems and Benchmarking Rules. Technical Report 21/94, University of Karlsruhe, Germany (1994)
Quinlan, J.R.: Induction of Decision Trees. Machine Learn. 1, 81–106 (1986)
Rabiner, L.R.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE 77, 257–285 (1989)
Safavian, S.R., Landgrebe, D.: A Survey of Decision Tree Classifier Methodology. IEEE Transactions on Systems, Man and Cybernetics 21(3), 660–674 (1991)
Schürmann, J., Doster, W.: A Decision Theoretic Approach to Hierarchical Classifier Design. Pattern Recognition 17(3), 359–369 (1984)
Schürmann, J.: Pattern Classification: A Unified View of Statistical and Neural Approaches. John Wiley & Sons, Inc., New York (1996)
Tou, J.T., Ganzales, R.C.: Pattern Recognition Principles. Addison Wesley, Reading (1974)
Young, S.: Large Vocabulary Continuous Speech Recognition: a Review. CUED Technical Report, Cambridge University (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Fritsch, J., Finke, M. (2012). Applying Divide and Conquer to Large Scale Pattern Recognition Tasks. In: Montavon, G., Orr, G.B., Müller, KR. (eds) Neural Networks: Tricks of the Trade. Lecture Notes in Computer Science, vol 7700. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35289-8_20
Download citation
DOI: https://doi.org/10.1007/978-3-642-35289-8_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35288-1
Online ISBN: 978-3-642-35289-8
eBook Packages: Computer ScienceComputer Science (R0)