Skip to main content
SpringerLink
Log in

Multiple-relations-constrained image classification with limited training samples via Pareto optimization

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

It is a significant challenge to classify images using limited training samples. To this end, we formulate image classification as a multi-task multi-view (MTMV) learning problem and propose a novel Pareto optimization-based method to find the solution. Specifically, we first build a multi-objective multiple-relations-constrained MTMV model called M4 to formulate the procedure of image classification. This model integrates comprehensive relations so that more knowledge can be used when classifying the images. We formulate the model as a multi-objective optimization problem, which addresses conflicts between the inconsistencies of each item in the model and the limitation on the number of relations. To generate the final classifier for each image classification task, an effective Pareto optimization-based algorithm Pareto-M4 is proposed. Pareto-M4 first generates the Pareto-optimal solutions using a novel multi-objective solver MOQPSO and then obtains the final classifiers from all the Pareto-optimal solutions using a recombination procedure. Experiments on various real-world image data sets demonstrate the effectiveness of the proposed image classification method when limited training samples are given.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Yu J, Rui Y, Tang YY, Tao D (2014) High-order distance-based multiview stochastic learning in image classification. IEEE Trans Cybern 44:2431–2442

    Article  Google Scholar 

  2. Xia J, Chanussot J, Du P, He X (2016) Rotation-based support vector machine ensemble in classification of hyperspectral data with limited training samples. IEEE Trans Geosci Remote Sens 54:1519–1531

    Article  Google Scholar 

  3. Zhao C, Gao X, Wang Y, Li J (2016) Efficient multiple-feature learning-based hyperspectral image classification with limited training samples. IEEE Trans Geosci Remote Sens 54:4052–4062

    Article  Google Scholar 

  4. Caruana R (1998) Multitask learning. In: Learning to learn. Springer, Berlin, pp 95–133

  5. Evgeniou T, Micchelli CA, Pontil M (2005) Learning multiple tasks with kernel methods. J Mach Learn Res 6:615–637

    MathSciNet  MATH  Google Scholar 

  6. He J, Lawrence R (2011) A graph-based framework for multi-task multi-view learning. In: Proceedings of the 28th International Conference on Machine Learning (ICML-11), pp 25–32

  7. Wang J, Wang Q, Peng J, Nie D, Zhao F, Kim M et al (2017) Multi-task diagnosis for autism spectrum disorders using multi-modality features: a multi-center study. Hum Brain Mapp 38:3081–3097

    Article  Google Scholar 

  8. Zhu X, Li X, Zhang S (2016) Block-row sparse multiview multilabel learning for image classification. IEEE Trans Cybern 46:450–461

    Article  Google Scholar 

  9. Guillaumin M, Verbeek J, Schmid C (2010) Multimodal semi-supervised learning for image classification. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR), pp 902–909

  10. Yan Y, Ricci E, Subramanian R, Lanz O, Sebe N (2013) No matter where you are: flexible graph-guided multi-task learning for multi-view head pose classification under target motion. In: Proceedings of the IEEE international conference on computer vision, pp 1177–1184

  11. Luo Y, Tao D, Geng B, Xu C, Maybank SJ (2013) Manifold regularized multitask learning for semi-supervised multilabel image classification. IEEE Trans Image Process 22:523–536

    Article  MathSciNet  Google Scholar 

  12. Yuan X-T, Liu X, Yan S (2012) Visual classification with multitask joint sparse representation. IEEE Trans Image Process 21:4349–4360

    Article  MathSciNet  Google Scholar 

  13. Torralba A, Murphy KP, Freeman WT (2007) Sharing visual features for multiclass and multiview object detection. IEEE Trans Pattern Anal Mach Intell 29(5):854–869

    Article  Google Scholar 

  14. Zhang J, Huan J (2012) Inductive multi-task learning with multiple view data. In: Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 543–551

  15. Jin X, Zhuang F, Wang S, He Q, Shi Z (2013) Shared structure learning for multiple tasks with multiple views. In: Joint European conference on machine learning and knowledge discovery in databases, pp 353–368

  16. Yang P, He J (2015) A graph-based hybrid framework for modeling complex heterogeneity. In: 2015 IEEE international conference on data mining (ICDM), pp 1081–1086

  17. Li B, Li J, Tang K, Yao X (2015) Many-objective evolutionary algorithms: a survey. ACM Comput Surv (CSUR) 48:13

    Article  Google Scholar 

  18. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6:182–197

    Article  Google Scholar 

  19. Erickson M, Mayer A, Horn J (2002) Multi-objective optimal design of groundwater remediation systems: application of the niched Pareto genetic algorithm (NPGA). Adv Water Resour 25:51–65

    Article  Google Scholar 

  20. Reyes-Sierra M, Coello CAC (2006) Multi-objective particle swarm optimizers: a survey of the state-of-the-art. Int J Comput Intell Res 2:287–308

    MathSciNet  Google Scholar 

  21. Nebro AJ, Durillo JJ, Garcia-Nieto J, Coello CC, Luna F, Alba E (2009) SMPSO: a new PSO-based metaheuristic for multi-objective optimization. In: IEEE symposium on computational intelligence in multi-criteria decision-making. mcdm’09, pp 66–73

  22. Zhang Q, Zhou A, Jin Y (2008) RM-MEDA: a regularity model-based multiobjective estimation of distribution algorithm. IEEE Trans Evol Comput 12:41–63

    Article  Google Scholar 

  23. Okabe T, Jin Y, Sendoff B, Olhofer M (2004) Voronoi-based estimation of distribution algorithm for multi-objective optimization. In: Congress on evolutionary computation. CEC2004, pp 1594–1601

  24. Elhossini A, Areibi S, Dony R (2010) Strength Pareto particle swarm optimization and hybrid EA-PSO for multi-objective optimization. Evol Comput 18:127–156

    Article  Google Scholar 

  25. Li B-B, Wang L (2007) A hybrid quantum-inspired genetic algorithm for multiobjective flow shop scheduling. IEEE Trans Syst Man Cybern Part B (Cybern) 37:576–591

    Article  Google Scholar 

  26. Abido MA (2010) Multiobjective particle swarm optimization with nondominated local and global sets. Nat Comput 9:747–766

    Article  MathSciNet  Google Scholar 

  27. Wang Y, Yang Y (2009) Particle swarm optimization with preference order ranking for multi-objective optimization. Inf Sci 179:1944–1959

    Article  MathSciNet  Google Scholar 

  28. Van Den Bergh F (2006) An analysis of particle swarm optimizers. University of Pretoria

  29. Solis FJ, Wets RJB (1981) Minimization by random search techniques. Math Oper Res 6:19–30

    Article  MathSciNet  Google Scholar 

  30. Sun J, Feng B, Xu W (2004) Particle swarm optimization with particles having quantum behavior. In: Congress on evolutionary computation

  31. Sun J, Wu X, Palade V, Fang W, Lai C-H, Xu W (2012) Convergence analysis and improvements of quantum-behaved particle swarm optimization. Inf Sci 193:81–103

    Article  MathSciNet  Google Scholar 

  32. Banka H, Dara S (2014) Hamming distance based binary PSO for feature selection and classification from high dimensional gene expression data. In: IWBBIO, pp 507–514

  33. Ding S, Lin L, Wang G, Chao H (2015) Deep feature learning with relative distance comparison for person re-identification. Pattern Recognit 48:2993–3003

    Article  Google Scholar 

  34. Sun J, Fang W, Wu X, Palade V, Xu W (2012) Quantum-behaved particle swarm optimization: analysis of individual particle behavior and parameter selection. Evol Comput 20:349–393

    Article  Google Scholar 

  35. dos Santos Coelho L, Alotto P (2008) Global optimization of electromagnetic devices using an exponential quantum-behaved particle swarm optimizer. IEEE Trans Magn 44:1074–1077

    Article  Google Scholar 

  36. Cai Y, Sun J, Wang J, Ding Y, Tian N, Liao X et al (2008) Optimizing the codon usage of synthetic gene with QPSO algorithm. J Theor Biol 254:123–127

    Article  MathSciNet  Google Scholar 

  37. Sun C, Lu S (2010) Short-term combined economic emission hydrothermal scheduling using improved quantum-behaved particle swarm optimization. Expert Syst Appl 37:4232–4241

    Article  Google Scholar 

  38. Chua T-S, Tang J, Hong R, Li H, Luo Z, Zheng Y (2009) NUS-WIDE: a real-world web image database from National University of Singapore. In: Proceedings of the ACM international conference on image and video retrieval, p 48

  39. Martino AD, Castellanos FX, Anderson J, Alaerts K, Assaf M, Behrmann et al (2012) The Autism Brain Imaging Data Exchange (ABIDE) consortium: open sharing of autism resting state fMRI data

  40. Chang C-C, Lin C-J (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2:27

    Google Scholar 

  41. Zhang D, Wang Y, Zhou L, Yuan H, Shen D, Alzheimer’s Disease Neuroimaging Initiative (2011) Multimodal classification of Alzheimer’s disease and mild cognitive impairment. Neuroimage 55:856–867

    Article  Google Scholar 

  42. Zhou J, Chen J, Ye J (2011) Malsar: multi-task learning via structural regularization. Arizona State University

  43. Sierra MR, Coello CAC (2005) Improving PSO-based multi-objective optimization using crowding, mutation and ∈-dominance. In: Evolutionary multi-criterion optimization, pp 505–519

  44. Zitzler E, Laumanns M, Thiele L (2001) SPEA2: improving the strength Pareto evolutionary algorithm

  45. Zhang Q, Li H (2007) MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans Evol Comput 11:712–731

    Article  Google Scholar 

  46. Durillo JJ, García-Nieto J, Nebro AJ, Coello CAC, Luna F, Alba E (2009) Multi-objective particle swarm optimizers: an experimental comparison. In: International conference on evolutionary multi-criterion optimization, pp 495–509

  47. Gao S, Chia L-T, Tsang IW-H, Ren Z (2014) Concurrent single-label image classification and annotation via efficient multi-layer group sparse coding. IEEE Trans Multimed 16:762–771

    Article  Google Scholar 

  48. Gao S, Chia L-T, Tsang IW-H (2011) Multi-layer group sparse coding—for concurrent image classification and annotation. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp 2809–2816

  49. Feng Y, Xiao J, Zhuang Y, Liu X (2012) Adaptive unsupervised multi-view feature selection for visual concept recognition. In: Asian conference on computer vision, pp 343–357

  50. Zhang H, Cao X, Ho JK, Chow TW (2017) Object-level video advertising: an optimization framework. IEEE Trans Ind Inf 13:520–531

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 61300151, the Natural Science Foundation of Jiangsu Province under Grant BK20130160, the Fundamental Research Funds for the Central Universities under Grant 30917013107, and the Ministry of Education in China Liberal Arts and Social Sciences Foundation under Grant 17YJA760022.

Author information

Authors and Affiliations

  1. School of Design Art and Media, Nanjing University of Science and Technology, Nanjing, 210094, China

    Di Zhou, Bin Jiang & Yajun Li

  2. School of Digital Media, Jiangnan University, Wuxi, 214122, China

    Jun Wang

Authors
  1. Di Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bin Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yajun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Wang.

Ethics declarations

Conflict of interest

We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, D., Wang, J., Jiang, B. et al. Multiple-relations-constrained image classification with limited training samples via Pareto optimization. Neural Comput & Applic 31, 6821–6842 (2019). https://doi.org/10.1007/s00521-018-3491-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-018-3491-4

Keywords

Search

Navigation