Abstract
The curriculum learning has shown immense potential in improving computer vision tasks. However, the drawback still exists when it comes to the multiclass classification problem, because of the nature of both data and model uncertainties. In this paper, we introduce a novel curriculum sampling strategy that takes into consideration uncertainty, confidence, score, and negative log-likelihood. We also suggest a novel method of grading the samples that have already been shown to be very successful. During the training period, curriculum learning is put into practice. After the preliminary training is finished, we use curriculum learning in our experimental setting. For this experiment, we used the CIFAR-10 dataset, and we were able to demonstrate the effectiveness of our approach by showing faster convergence, more accurate findings, and a strong deep learning model for image classification. We have demonstrated the use of NLL-based CL post-training on the same model to accomplish the indicated results, in contrast to the state of the art where curriculum learning is utilized before the model training. Problem statements involving multiclass object detection and segmentation can be addressed using the technique.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bengio Y, Louradour J, Collobert R, Weston J (2009) Curriculum learning. In: Proceedings of the 26th annual international conference on machine learning, June 2009, pp 41–48
Peng X, Li L, Wang FY (2019) Accelerating minibatch stochastic gradient descent using typicality sampling. IEEE Trans Neural Netw Learn Syst 31(11):4649–4659
Brewer E, Lin J, Kemper P, Hennin J, Runfola D (2021) Predicting road quality using high resolution satellite imagery: a transfer learning approach. PLoS ONE 16(7):e0253370
Hacohen G, Weinshall D (2019) On the power of curriculum learning in training deep networks. In: International conference on machine learning, May 2019. PMLR, pp 2535–2544
Liu F, Ge S, Wu X (2022) Competence-based multimodal curriculum learning for medical report generation. arXiv preprint arXiv:2206.14579
Jiang L, Meng D, Yu S-I, Lan Z, Shan S, Hauptmann A (2014) Self-paced learning with diversity. In: Advances in neural information processing systems, vol 27
Klink P, Yang H, D’Eramo C, Peters J, Pajarinen J (2022) Curriculum reinforcement learning via constrained optimal transport. In: International conference on machine learning. PMLR, pp 11341–11358
Penha G, Hauff C (2020) Curriculum learning strategies for IR. In: European conference on information retrieval, Apr 2020. Springer, Cham, pp 699–713
Zhou Y, Yang B, Wong DF, Wan Y, Chao LS (2020) Uncertainty-aware curriculum learning for neural machine translation. In: Proceedings of the 58th annual meeting of the association for computational linguistics, July 2020, pp 6934–6944
Wei Y, Liang X, Chen Y, Shen X, Cheng M-M, Feng J, Zhao Y, Yan S (2016) STC: a simple to complex framework for weakly-supervised semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(11):2314–2320
Kocmi T, Bojar O (2017) Curriculum learning and minibatch bucketing in neural machine translation. arXiv preprint arXiv:1707.09533
Chen X, Gupta A (2015) Webly supervised learning of convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 1431–1439
Guo S, Huang W, Zhang H, Zhuang C, Dong D, Scott MR, Huang D (2018) CurriculumNet: weakly supervised learning from large-scale web images. In: Proceedings of the European conference on computer vision (ECCV), pp 135–150
Braun S, Neil D, Liu S-C (2017) A curriculum learning method for improved noise robustness in automatic speech recognition. In: 2017 25th European signal processing conference (EUSIPCO). IEEE, pp 548–552
Ranjan S, Hansen JHL (2017) Curriculum learning based approaches for noise robust speaker recognition. IEEE/ACM Trans Audio Speech Lang Process 26(1):197–210
Gui L, Baltrušaitis T, Morency L-P (2017) Curriculum learning for facial expression recognition. In: 2017 12th IEEE international conference on automatic face & gesture recognition (FG 2017). IEEE, pp 505–511
Tang Y, Wang X, Harrison AP, Lu L, Xiao J, Summers RM (2018) Attention-guided curriculum learning for weakly supervised classification and localization of thoracic diseases on chest radiographs. In: International workshop on machine learning in medical imaging. Springer, Cham, pp 249–258
Soviany P, Ardei C, Ionescu RT, Leordeanu M (2020) Image difficulty curriculum for generative adversarial networks (CuGAN). In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 3463–3472
Tudor Ionescu R, Alexe B, Leordeanu M, Popescu M, Papadopoulos DP, Ferrari V (2016) How hard can it be? Estimating the difficulty of visual search in an image. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2157–2166
Lotter W, Sorensen G, Cox D (2017) A multi-scale CNN and curriculum learning strategy for mammogram classification. In: Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, Cham, pp 169–177
Park B, Cho Y, Lee G, Lee SM, Cho YH, Lee ES et al (2019) A curriculum learning strategy to enhance the accuracy of classification of various lesions in chest-PA X-ray screening for pulmonary abnormalities. Sci Rep 9(1):1–9
Wei J, Suriawinata A, Ren B, Liu X, Lisovsky M, Vaickus L et al (2021) Learn like a pathologist: curriculum learning by annotator agreement for histopathology image classification. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 2473–2483
Wang Y, Gan W, Yang J, Wu W, Yan J (2019) Dynamic curriculum learning for imbalanced data classification. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 5017–5026
Yu Q, Ikami D, Irie G, Aizawa K (2020) Multi-task curriculum framework for open-set semi-supervised learning. In: European conference on computer vision, Aug 2020. Springer, Cham, pp 438–454
Yao H, Zhu DL, Jiang B, Yu P (2019) Negative log likelihood ratio loss for deep neural network classification. In: Proceedings of the future technologies conference, Oct 2019. Springer, Cham, pp 276–282
Wu Z, Shen C, Van Den Hengel A (2019) Wider or deeper: revisiting the ResNet model for visual recognition. Pattern Recogn 90:119–133
Koonce B (2021) EfficientNet. In: Convolutional neural networks with swift for tensorflow. Apress, Berkeley, CA, pp 109–123
Orhan AE (2019) Robustness properties of Facebook’s ResNeXt WSL models. arXiv preprint arXiv:1907.07640
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Kar, I., Chatterjee, A.S., Mukhopadhyay, S., Singh, V. (2023). Toward More Robust Classifier: Negative Log-Likelihood Aware Curriculum Learning. In: Chaki, N., Devarakonda, N., Cortesi, A. (eds) Proceedings of International Conference on Computational Intelligence and Data Engineering. ICCIDE 2022. Lecture Notes on Data Engineering and Communications Technologies, vol 163. Springer, Singapore. https://doi.org/10.1007/978-981-99-0609-3_8
Download citation
DOI: https://doi.org/10.1007/978-981-99-0609-3_8
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-0608-6
Online ISBN: 978-981-99-0609-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)