Neural Computing and Applications

, Volume 25, Issue 7–8, pp 1913–1920 | Cite as

Image classification using local linear regression

  • Wankou YangEmail author
  • Karl Ricanek
  • Fumin Shen
Original Article


In the past several decades, classifier design has attracted much attention. Inspired by the locality preserving idea of manifold learning, here we give a local linear regression (LLR) classifier. The proposed classifier consists of three steps: first, search k nearest neighbors of a pointed sample from each special class, respectively; second, reconstruct the pointed sample using the k nearest neighbors from each special class, respectively; and third, classify the test sample according to the minimum reconstruction error. The experimental results on the ETH80 database, the CENPAMI handwritten number database and the FERET face image database demonstrate that LLR works well, leading to promising image classification performance.


Linear regression Manifold learning Locality Image classification 



This project is partly supported by NSF of China (61375001, 61273023), partly supported by the open fund of Key Laboratory of Measurement and partly supported by Control of Complex Systems of Engineering, Ministry of Education (No. MCCSE2013B01) and the Jiangsu Key Laboratory of Image and Video Understanding for Social Safety (Nanjing University of Science and Technology) (No. 30920130122006).


  1. 1.
    Kotsiantis SB (2007) Supervised machine learning: a review of classification techniques. Informatica 31:249–268zbMATHMathSciNetGoogle Scholar
  2. 2.
    Cover TM, Hart PE (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27CrossRefzbMATHGoogle Scholar
  3. 3.
    Yu K, Ji L, Zhang X (2002) Kernel nearest-neighbor algorithm. Neural Process Lett 15:147–156CrossRefzbMATHGoogle Scholar
  4. 4.
    Li SZ, Lu J (1999) Face recognition using the nearest feature line method. IEEE Trans Neural Netw 10(2):439–443CrossRefGoogle Scholar
  5. 5.
    Zheng W, Zhao L, Zou C (2004) Locally nearest neighbour classifiers for pattern recognition. Pattern Recognit 37(6):1307–1309CrossRefzbMATHGoogle Scholar
  6. 6.
    Lou Z, Jin Z (2006) Novel adaptive nearest neighbour classifiers based on hit-distance. In: Proceedings of the 18th international conference on pattern recognition (ICPR2006), vol. 3, pp. 87–90Google Scholar
  7. 7.
    Gao Q, Wang Z (2007) Center-based nearest neighbor classifier. Pattern Recognit 40(1):346–349CrossRefzbMATHGoogle Scholar
  8. 8.
    Shen F, Hasegawa O (2008) A fast nearest neighbor classifier based on self-organizing incremental neural network. Neural Netw 21(10):1537–1547CrossRefzbMATHGoogle Scholar
  9. 9.
    Mitani Y, Hamamoto Y (2006) A local mean-based nonparametric classifier. Pattern Recognit Lett 27(10):1151–1159CrossRefGoogle Scholar
  10. 10.
    Zhang C, Zhang J (2008) RotBoost: a technique for combining Rotation Forest and AdaBoost. Pattern Recognit Lett 29(10):1524–1536CrossRefGoogle Scholar
  11. 11.
    Zhou A, Zhu Z, Fan H (2012) A new semi-supervised PSVM classifier. Appl Math Comput 219(8):883–889MathSciNetGoogle Scholar
  12. 12.
    Ben X, Meng W, Yan R, Wang K (2013) Kernel coupled distance metric learning for gait recognition and face recognition. Neurocomputing 2013(120):577–589CrossRefGoogle Scholar
  13. 13.
    Yang J, Wright J, Huang T, Ma Y (2008) Image super-resolution as sparse representation of raw patches. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR 2008)Google Scholar
  14. 14.
    Rao S, Tron R, Vidal R, Ma Y (2008) Motion segmentation via robust subspace separation in the presence of outlying, incomplete, and corrupted trajectories. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR 2008)Google Scholar
  15. 15.
    Mairal J, Sapiro G, Elad M (2008) Learning multiscale sparse representations for image and video restoration. SIAM MMS 7(1):214–241CrossRefzbMATHMathSciNetGoogle Scholar
  16. 16.
    Wright J, Yang A, Sastry S, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227CrossRefGoogle Scholar
  17. 17.
    Wright J, Ma Y, Mairial J, Sapiro G, Huang T, Yan S (2010) Sparse representation for computer vision and pattern recognition. In Proceedings of IEEE, special issue on applications of compressive sensing & sparse representation, 98(6): 1031–1044Google Scholar
  18. 18.
    Rigamonti R, Brown MA, Lepetit V (2011) Are sparse representations really relevant for image classification? In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR2011)Google Scholar
  19. 19.
    Shi Q, Erisson A, van den Hengel A, Shen C (2011) Is face recognition really a compressive sensing problem? In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR 2011)Google Scholar
  20. 20.
    Zhang L, Yang M, Feng X (2011) Sparse representation or collaborative representation: which helps face recognition. In Proceedings of IEEE conference on computer vision (ICCV2011)Google Scholar
  21. 21.
    Yang W, Wang Z, Yin J, Sun C, Ricanek K (2013) Image classification using Kernel collaborative representation with regularized least square. Appl Math Comput 222(13–28):2013MathSciNetGoogle Scholar
  22. 22.
    Xu Y, Zhang D, Yang J, Yang J-Y (2011) A two-phase test sample sparse representation method for use with face recognition. IEEE Trans Circuits Syst Video Technol 21(9):1255–1262CrossRefGoogle Scholar
  23. 23.
    Qiao L, Chen S, Tan X (2010) Sparsity preserving projections with applications to face recognition. Pattern Recognit 43(1):331–341CrossRefzbMATHGoogle Scholar
  24. 24.
    Lai Z, Xu Y, Yang J, Zhang D (2013) Sparse tensor discriminant analysis. IEEE Trans Image Process 22(10):3904–3915CrossRefMathSciNetGoogle Scholar
  25. 25.
    Lai Z, Wong WK, Jin Z, Yang J, Xu Y (2012) Sparse approximation to the eigensubspace for discrimination. IEEE Trans Neural Netw Learn Syst 23(12):1948–1960CrossRefGoogle Scholar
  26. 26.
    Naseem I, Togneri R, Bennamoun M (2010) Linear regression for face recognition. IEEE Trans Pattern Anal Mach Intell 32(11):2106–2112CrossRefGoogle Scholar
  27. 27.
    Chai X, Shan S, Chen X, Gao W (2007) Locally linear regression for pose invariant face recognition. IEEE Trans Image Process 16(7):1716–1725CrossRefMathSciNetGoogle Scholar
  28. 28.
    Hastie T, Tibshirani R, Friedman J (2001) The elements of statistical learning; data mining, inference and prediction. Springer, New YorkzbMATHGoogle Scholar
  29. 29.
  30. 30.
    Leibe B, Schiele B (2003) Analyzing appearance and contour based methods for object categorization. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR 2003)Google Scholar
  31. 31.
  32. 32.
    Yang J, Zhang L, Yang J, Zhang D (2011) From classifiers to discriminators: a nearest neighbor rule induced discriminant analysis. Pattern Recognit 44(7):1387–1402CrossRefzbMATHGoogle Scholar
  33. 33.
    Liao SX, Pawlark M (1996) On image analysis by moments. IEEE Trans Pattern Anal Mach Intell 18(3):254–266CrossRefGoogle Scholar
  34. 34.
    Phillips PJ, Moon H, Rizvi SA, Rauss PJ (2000) The FERET evaluation methodology for face recognition algorithms. IEEE Trans Pattern Anal Mach Intell 22(10):1090–1104CrossRefGoogle Scholar
  35. 35.
    Phillips PJ (2004) The facial recognition technology (FERET) database:

Copyright information

© The Natural Computing Applications Forum 2014

Authors and Affiliations

  1. 1.School of AutomationSoutheast UniversityNanjingChina
  2. 2.Key Lab of Measurement and Control of Complex Systems of Engineering, Ministry of EducationSoutheast UniversityNanjingChina
  3. 3.Jiangsu Key Laboratory of Image and Video Understanding for Social SafetyNanjing University of Science and TechnologyNanjingChina
  4. 4.Face Aging Group, Department of Computer ScienceUNC WilmingtonWilmingtonUSA
  5. 5.School of Computer Science and EngineeringUniversity of Electronic Science and Technology of ChinaChengduChina

Personalised recommendations