Objectives for Discriminative Training

  • Qi (Peter) LiEmail author
Part of the Signals and Communication Technology book series (SCT)


The first step in discriminative training is to define an objective function. In this chapter, the relations among a class of discriminative training objectives is derived and discovered through our theoretical analysis. The objectives selected for our discussion are the minimum classification error (MCE), maximum mutual information (MMI), minimum error rate (MER), and generalized minimum error rate (GMER). The author’s analysis shows that all these objectives can be related to both minimum error rates and maximum a posteriori probability. In theory, the MCE and GMER objectives are more general and flexible than the MMI and MER objectives, and MCE and GMER are beyond the Bayesian decision theory. The results and the analytical methods used in this chapter can help in judging and evaluating discriminative objectives, and in defining new objectives for different tasks and better performances. We note that although our discussions are based on the applications of speaker recognition, the analysis can be further extended to speech recognition tasks.


Sigmoid Function Equal Error Rate Speaker Recognition Posteriori Probability Speaker Veri 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bahl, L.R., Brown, P.F., de Souza, P.V., and Mercer, R.L.: Maximum mutual information estimation of hidden Markov model parameters for speech recogni- tion, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing (Tokyo), pp. 49–52 (1986)Google Scholar
  2. 2.
    Chou, W.:Discriminant-function-based minimum recognition error rate pattern-recognition approach to speech recognition, Proceedings of the IEEE, vol. 88, pp. 1201–1222, August 2000Google Scholar
  3. 3.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, Second Edition. Wiley, New York (2001)Google Scholar
  4. 4.
    Gopalakrishnan, P.S., Kanevsky, D., Nadas, A., Nahamoo, D.: An inequality for rational functions with applications to some statistical estimation problems. IEEE Trans. on Information Theory 37, 107–113 (1991)zbMATHCrossRefGoogle Scholar
  5. 5.
    Juang, B.-H., Chou, W., Lee, C.-H.: Minimum classification error rate methods for speech recognition. IEEE Trans. on Speech and Audio Process 5, 257–265 (1997)CrossRefGoogle Scholar
  6. 6.
    Juang, B.-H. Katagiri S.: Discriminative learning for minimum error clas- sification. IEEE Transactions on Signal Processing 40, 3043–3054 (1992)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Katagiri, S., Lee, C.-H., and Juang, B.-H., New discriminative algorithm based on the generalized probabilistic descent method, in Proceedings of IEEE Workshop on Neural Network for Signal Processing (Princeton), pp. 299–309, September 1991Google Scholar
  8. 8.
    Korkmazskiy, F. and Juang, B.-H., Discriminative adaptation for speaker verification, in Proceedings of Int. Conf. on Spoken Language Processing (Philadelphia), pp. 28–31 1996Google Scholar
  9. 9.
    Li, J., Yuan, M., and Lee, C.H., Soft margin estimation of hidden markov model parameters, in Proc. ICSLP, pp. 2422–2425 (2007)Google Scholar
  10. 10.
    Li, Q., Discovering relations among discriminative training objectives, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing (Montreal), p. 2004, May 2004Google Scholar
  11. 11.
    Li, Q. and Juang, B.-H., Fast discriminative training for sequential observations with application to speaker identification, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing (Hong Kong), April 2003Google Scholar
  12. 12.
    Li, Q. and Juang, B.-H., A new algorithm for fast discriminative training, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing (Orlando, FL), May 2002Google Scholar
  13. 13.
    Li, Q. Juang B.-H.: Study of a fast discriminative training algorithm for pattern recognition. IEEE Trans. on Neural Networks 17, 1212–1221 (2006)CrossRefGoogle Scholar
  14. 14.
    Ma, C. and Chang, E., Comparison of discriminative training methods for speaker verification, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing, pp. I-192–I-195, 2003Google Scholar
  15. 15.
    Nadas, A., Nahamoo, D., and Picheny, M. A., On a model-robust training method for speech recognition, IEEE Transactions on Acoust., Speech, Signal Processing, vol. 36, pp. 1432–1436, Sept. 1988Google Scholar
  16. 16.
    Normandin, Y., Cardin, R., and Mori, R. D., High-performance connected digit recognition using maximum mutual information estimation, IEEE Trans. on Speech and Audio Processing, vol. 2, pp. 299–311, April 1994Google Scholar
  17. 17.
    Reichl, W. and Ruske, G., Discriminant training for continuous speech recog- nition, in Proceedings of Eurospeech, 1995Google Scholar
  18. 18.
    Schluter, R. and Macherey,W., Comparison of discriminative training criteria, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing, pp. 493–497, 1998Google Scholar
  19. 19.
    Siohan, O., Rosenberg, A. E., and Parthasarathy, S., Speaker identification using minimum verification error training, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing (Seattle), pp. 109–112, May 1998Google Scholar
  20. 20.
    Siohan, O., Rosenberg, A., and Parthasarathy, S., Speaker identification using minimum classification error training, in Proc. IEEE Int. Conf. on Acoustic, Speech, and Signal Process, pp. 109–112, 1998Google Scholar
  21. 21.
    Vapnik, V.N.: The nature of statistical learning theory. Springer, NY (1995)zbMATHGoogle Scholar
  22. 22.
    Yin, Y. and Li, Q., Soft frame margin estimation of Gaussian mixture models for speaker recognition with sparse training data, in ICASSP 2011 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg  2012

Authors and Affiliations

  1. 1.Li Creative Technologies (LcT), IncFlorham ParkUSA

Personalised recommendations