Optimization Letters

, Volume 8, Issue 3, pp 823–839 | Cite as

On the optimization properties of the correntropic loss function in data analysis

  • Mujahid N. Syed
  • Panos M. Pardalos
  • Jose C. Principe
Original Paper


Similarity measures play a critical role in the solution quality of data analysis methods. Outliers or noise often taint the solution, hence, practical data analysis calls for robust measures. The correntropic loss function is a smooth and robust measure. In this paper, we present the properties of the correntropic loss function that can be utilized in optimization based data analysis methods.


Classification Correntropy Clustering Pseudoconvexity Invexity Regression and robust data analysis 



This research is partially supported by NSF.


  1. 1.
    Bazaraa, M.S., Sherali, H.D., Shetty, C.M.: Nonlinear Programming: Theory and Algorithms. Wiley, London (2006)CrossRefGoogle Scholar
  2. 2.
    Ben-Israel, A., Mond, B.: What is invexity. J. Aust. Math. Soc. Ser. B 28(1), 1–9 (1986)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Cambini, A., Martein, L.: Generalized Convexity and Optimization: Theory and Applications, vol. 616. Springer, Berlin (2008)Google Scholar
  4. 4.
    Craven, B.D.: Duality for generalized convex fractional programs. In: Schaible, S., Ziemba, W.T. (eds.) Generalized Concavity in Optimization and Economics, pp. 473–489. Academic Press, New York (1981)Google Scholar
  5. 5.
    Eddington, S.A.S.: Stellar Movements and the Structure of the Universe. Macmillan and Company, limited, London (1914)Google Scholar
  6. 6.
    Fisher, R.A., et al.: A mathematical examination of the methods of determining the accuracy of an observation by the mean error, and by the mean square error. Mon. Not. R. Astron. Soc. 80, 758–770 (1920)Google Scholar
  7. 7.
    Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., Stahel, W.A.: Robust Statistics: The Approach Based on Influence Functions, vol. 114. Wiley, London (2011)Google Scholar
  8. 8.
    Hanson, M.A.: On sufficiency of the kuhn-tucker conditions. J. Math. Anal. Appl. 80(2), 545–550 (1981)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    He, R., Zheng, W.S., Hu, B.G., Kong, X.W.: A regularized correntropy framework for robust pattern recognition. Neural. Comput. 23(8), 2074–2100 (2011)CrossRefzbMATHGoogle Scholar
  10. 10.
    Huber, P.J., Ronchetti, E.M.: Robust Statistics, Wiley Series in Probability and Statistics, New Jersey (2009)Google Scholar
  11. 11.
    Huber, P.J.: Robust Statistical Procedures. Number 27. SIAM, Philadelphia, USA (1997)Google Scholar
  12. 12.
    Khanh, P.Q.: Invex-convexlike functions and duality. J. Optim. Theory. Appl. 87(1), 141–165 (1995)CrossRefzbMATHMathSciNetGoogle Scholar
  13. 13.
    Liu, W., Pokharel, P.P., Principe, J.C.: Correntropy: a localized similarity measure. In: International Joint Conference on Neural Networks, 2006. IJCNN’06, pp. 4919–4924. IEEE (2006)Google Scholar
  14. 14.
    Liu, W., Pokharel, P.P., Principe, J.C.: Error entropy, correntropy and m-estimation. In: Proceedings of the 2006 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing, 2006, pp. 179–184. IEEE (2006)Google Scholar
  15. 15.
    Liu, W., Pokharel, P.P., Principe, J.C.: Correntropy: properties and applications in non-gaussian signal processing. Signal Process. IEEE Trans. 55(11), 5286–5298 (2007)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Mangasarian, O.L.: Pseudo-convex functions. J. Soc. Ind. Appl. Math. Ser. A Control 3(2), 281–290 (1965)CrossRefzbMATHMathSciNetGoogle Scholar
  17. 17.
    Mangasarian, O.L., Mangasarian, O.L., Mangasarian, O.L.: Nonlinear Programming. Society for Industrial and Applied Mathematics, Philadelphia, PA (1994)CrossRefzbMATHGoogle Scholar
  18. 18.
    Pardalos, P.M., Hansen, P.: Data Mining and Mathematical Programming, vol. 45. Amer Mathematical Society, Rhode Island, USA (2008)Google Scholar
  19. 19.
    Principe, J.C.: Information Theoretic Learning: Renyi’s Entropy and Kernel Perspectives. Springer, Berlin (2010)CrossRefGoogle Scholar
  20. 20.
    Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Risk tuning with generalized linear regression. Math. Oper. Res. 33(3), 712–729 (2008)CrossRefzbMATHMathSciNetGoogle Scholar
  21. 21.
    Santamaría, I., Pokharel, P.P., Principe, J.C.: Generalized correlation function: definition, properties, and application to blind equalization. Signal Process. IEEE Trans. 54(6), 2187–2197 (2006)CrossRefGoogle Scholar
  22. 22.
    Singh, A., Principe, J.C.: Using correntropy as a cost function in linear adaptive filters. In: International Joint Conference on Neural Networks, 2009. IJCNN 2009, pp. 2950–2955. IEEE (2009)Google Scholar
  23. 23.
    Singh, A., Principe, J.C.: A loss function for classification based on a robust similarity metric. In: The 2010 International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE (2010)Google Scholar
  24. 24.
    Syed, M.N., Principe, J.C., Pardalos, P.M.: Correntropy in data classification. In: Sorokin, A., Murphey, R., Thai, M.T., Pardalos, P.M. (eds.) Dynamics of Information Systems: Mathematical Foundations, pp. 81–117. Springer, Berlin (2012)Google Scholar
  25. 25.
    Tukey, J.W.: A survey of sampling from contaminated distributions. Contrib. Probab. Stat. 2, 448–485 (1960)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Mujahid N. Syed
    • 1
  • Panos M. Pardalos
    • 2
  • Jose C. Principe
    • 3
  1. 1.Department of Industrial and Systems EngineeringUniversity of FloridaGainesvilleUSA
  2. 2.Department of Industrial and Systems Engineering, and Biomedical EngineeringUniversity of FloridaGainesvilleUSA
  3. 3.Department of Electrical and Biomedical EngineeringUniversity of FloridaGainesvilleUSA

Personalised recommendations