Optimization Letters

, Volume 8, Issue 3, pp 823–839 | Cite as

On the optimization properties of the correntropic loss function in data analysis

  • Mujahid N. Syed
  • Panos M. Pardalos
  • Jose C. Principe
Original Paper

Abstract

Similarity measures play a critical role in the solution quality of data analysis methods. Outliers or noise often taint the solution, hence, practical data analysis calls for robust measures. The correntropic loss function is a smooth and robust measure. In this paper, we present the properties of the correntropic loss function that can be utilized in optimization based data analysis methods.

Keywords

Classification Correntropy Clustering Pseudoconvexity Invexity Regression and robust data analysis 

References

  1. 1.
    Bazaraa, M.S., Sherali, H.D., Shetty, C.M.: Nonlinear Programming: Theory and Algorithms. Wiley, London (2006)CrossRefGoogle Scholar
  2. 2.
    Ben-Israel, A., Mond, B.: What is invexity. J. Aust. Math. Soc. Ser. B 28(1), 1–9 (1986)CrossRefMATHMathSciNetGoogle Scholar
  3. 3.
    Cambini, A., Martein, L.: Generalized Convexity and Optimization: Theory and Applications, vol. 616. Springer, Berlin (2008)Google Scholar
  4. 4.
    Craven, B.D.: Duality for generalized convex fractional programs. In: Schaible, S., Ziemba, W.T. (eds.) Generalized Concavity in Optimization and Economics, pp. 473–489. Academic Press, New York (1981)Google Scholar
  5. 5.
    Eddington, S.A.S.: Stellar Movements and the Structure of the Universe. Macmillan and Company, limited, London (1914)Google Scholar
  6. 6.
    Fisher, R.A., et al.: A mathematical examination of the methods of determining the accuracy of an observation by the mean error, and by the mean square error. Mon. Not. R. Astron. Soc. 80, 758–770 (1920)Google Scholar
  7. 7.
    Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., Stahel, W.A.: Robust Statistics: The Approach Based on Influence Functions, vol. 114. Wiley, London (2011)Google Scholar
  8. 8.
    Hanson, M.A.: On sufficiency of the kuhn-tucker conditions. J. Math. Anal. Appl. 80(2), 545–550 (1981)CrossRefMATHMathSciNetGoogle Scholar
  9. 9.
    He, R., Zheng, W.S., Hu, B.G., Kong, X.W.: A regularized correntropy framework for robust pattern recognition. Neural. Comput. 23(8), 2074–2100 (2011)CrossRefMATHGoogle Scholar
  10. 10.
    Huber, P.J., Ronchetti, E.M.: Robust Statistics, Wiley Series in Probability and Statistics, New Jersey (2009)Google Scholar
  11. 11.
    Huber, P.J.: Robust Statistical Procedures. Number 27. SIAM, Philadelphia, USA (1997)Google Scholar
  12. 12.
    Khanh, P.Q.: Invex-convexlike functions and duality. J. Optim. Theory. Appl. 87(1), 141–165 (1995)CrossRefMATHMathSciNetGoogle Scholar
  13. 13.
    Liu, W., Pokharel, P.P., Principe, J.C.: Correntropy: a localized similarity measure. In: International Joint Conference on Neural Networks, 2006. IJCNN’06, pp. 4919–4924. IEEE (2006)Google Scholar
  14. 14.
    Liu, W., Pokharel, P.P., Principe, J.C.: Error entropy, correntropy and m-estimation. In: Proceedings of the 2006 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing, 2006, pp. 179–184. IEEE (2006)Google Scholar
  15. 15.
    Liu, W., Pokharel, P.P., Principe, J.C.: Correntropy: properties and applications in non-gaussian signal processing. Signal Process. IEEE Trans. 55(11), 5286–5298 (2007)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Mangasarian, O.L.: Pseudo-convex functions. J. Soc. Ind. Appl. Math. Ser. A Control 3(2), 281–290 (1965)CrossRefMATHMathSciNetGoogle Scholar
  17. 17.
    Mangasarian, O.L., Mangasarian, O.L., Mangasarian, O.L.: Nonlinear Programming. Society for Industrial and Applied Mathematics, Philadelphia, PA (1994)CrossRefMATHGoogle Scholar
  18. 18.
    Pardalos, P.M., Hansen, P.: Data Mining and Mathematical Programming, vol. 45. Amer Mathematical Society, Rhode Island, USA (2008)Google Scholar
  19. 19.
    Principe, J.C.: Information Theoretic Learning: Renyi’s Entropy and Kernel Perspectives. Springer, Berlin (2010)CrossRefGoogle Scholar
  20. 20.
    Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Risk tuning with generalized linear regression. Math. Oper. Res. 33(3), 712–729 (2008)CrossRefMATHMathSciNetGoogle Scholar
  21. 21.
    Santamaría, I., Pokharel, P.P., Principe, J.C.: Generalized correlation function: definition, properties, and application to blind equalization. Signal Process. IEEE Trans. 54(6), 2187–2197 (2006)CrossRefGoogle Scholar
  22. 22.
    Singh, A., Principe, J.C.: Using correntropy as a cost function in linear adaptive filters. In: International Joint Conference on Neural Networks, 2009. IJCNN 2009, pp. 2950–2955. IEEE (2009)Google Scholar
  23. 23.
    Singh, A., Principe, J.C.: A loss function for classification based on a robust similarity metric. In: The 2010 International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE (2010)Google Scholar
  24. 24.
    Syed, M.N., Principe, J.C., Pardalos, P.M.: Correntropy in data classification. In: Sorokin, A., Murphey, R., Thai, M.T., Pardalos, P.M. (eds.) Dynamics of Information Systems: Mathematical Foundations, pp. 81–117. Springer, Berlin (2012)Google Scholar
  25. 25.
    Tukey, J.W.: A survey of sampling from contaminated distributions. Contrib. Probab. Stat. 2, 448–485 (1960)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Mujahid N. Syed
    • 1
  • Panos M. Pardalos
    • 2
  • Jose C. Principe
    • 3
  1. 1.Department of Industrial and Systems EngineeringUniversity of FloridaGainesvilleUSA
  2. 2.Department of Industrial and Systems Engineering, and Biomedical EngineeringUniversity of FloridaGainesvilleUSA
  3. 3.Department of Electrical and Biomedical EngineeringUniversity of FloridaGainesvilleUSA

Personalised recommendations