Abstract
A supervised learning framework has been proposed for a situation in which each training data is provided with a complementary label that represents a class to which the pattern does not belong. In the existing literature, complementary-label learning has been studied independently from ordinary-label learning, which assumes that each training data is provided with a label representing the class to which the pattern belongs. However, providing a complementary label should be treated as equivalent to providing the rest of all labels as candidates of the one true class. In this paper, we focus on the fact that the loss functions for one-versus-all and pairwise classifications corresponding to ordinary-label learning and complementary-label learning satisfy additivity and duality, and provide a framework that directly bridges the existing supervised learning frameworks. We also show that the complementary labels generated from a probabilistic model assumed in the existing literature is equivalent to the ordinary labels generated from a mixture of ground-truth probabilistic model and uniform distribution. Based on this finding, the relationship between our work and the existing work can be naturally derived. Further, we derive the classification risk and error bound for any loss functions that satisfy additivity and duality.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability Statement
Data-generation process is indicated in “Dataset Generation”.
References
Cao Y, Xu Y. Multi-complementary and unlabeled learning for arbitrary losses and models. 2020.arXiv:2001.04243
Chapelle O, Schlkopf B, Zien A. Semi-supervised learning. 1st ed. Cambridge: The MIT Press; 2010.
Charoenphakdee N, Lee J, Sugiyama M. On symmetric losses for learning from corrupted labels. In: Proceedings of the 36th International Conference on Machine Learning, 2019; pp. 961–970.
Cid-sueiro J. Proper losses for learning from partial labels. Adv Neural Inf Process Syst. 2012;25:1565–73.
Cour T, Sapp B, Taskar B. Learning from partial labels. J Mach Learn Res. 2011;12:1501–36.
Feng L, Kaneko T, Han B, Niu G, An B, Sugiyama M. Learning with multiple complementary labels. In: Proceedings of the 37th International Conference on Machine Learning. 2020; pp. 3072–3081.
Ghosh A, Manwani N, Sastry P. Making risk minimization tolerant to label noise. Neurocomputing. 2015;160:93–107.
Grandvalet Y, Bengio Y. Semi-supervised learning by entropy minimization. Adv Neural Inf Process Syst. 2005;17:529–36.
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017; pp. 4700–4708.
Ishida T, Niu G, Hu W, Sugiyama M. Learning from complementary labels. Adv Neural Inf Process Syst. 2017;30:5639–49.
Ishida T, Niu G, Sugiyama M. Binary classification from positive-confidence data. Adv Neural Inf Process Syst. 2018;31:5917–28.
Ishida T, Niu G, Menon A, Sugiyama M. Complementary-label learning for arbitrary losses and models. In: Proceedings of the 37th International Conference on Machine Learning, 2020; pp. 2971–2980.
Kim Y, Yim J, Yun J, Kim J. NLNL: Negative learning for noisy labels. In: Proceedings of the IEEE International Conference on Computer Vision, 2019; pp. 101–110.
Kingma DP, Ba J. Adam: A method for stochastic optimization. In: Proceedings of the International Conference on Learning Representations, 2015.
Kipf TN, Welling M. Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations, 2017.
Laine S, Aila T. Temporal ensembling for semi-supervised learning. In: Proceedings of the International Conference on Learning Representations, 2017.
Ledoux M, Talagrand M. Probability in Banach spaces: isoperimetry and processes. Berlin: Springer Science & Business Media; 2013.
Luo J, Orabona F. Learning from candidate labeling sets. Adv Neural Inf Process Syst. 2010;23:1504–12.
Mann GS, McCallum A. Simple, robust, scalable semi-supervised learning via expectation regularization. In: Proceedings of the 24th International Conference on Machine Learning, 2007; pp. 593–600.
McDiarmid C. On the method of bounded differences. Surveys in Combinatorics. 1989;141:148–88.
Mohri M, Rostamizadeh A, Talwalkar A. Foundations of machine learning. Cambridge: MIT press; 2018.
Natarajan N, Dhillon IS, Ravikumar PK, Tewari A. Learning with noisy labels. Adv Neural Inf Process Syst. 2013;26:1196–204.
Niu G, Jitkrittum W, Dai B, Hachiya H, Sugiyama M. Squared-loss mutual information regularization: a novel information-theoretic approach to semi-supervised learning. In: Proceedings of the 30th International Conference on Machine Learning, 2013; pp. 10–18.
Raykar VC, Yu S, Zhao LH, Valadez GH, Florin C, Bogoni L, Moy L. Learning from crowds. J Mach Learn Res. 2010;11:1297–322.
Yu X, Liu T, Gong M, Tao D. Learning with biased complementary labels. In: Proceedings of the 15th European conference on computer vision, 2018; pp. 68–83.
Acknowledgements
This work was supported in part by the Japan Society for the Promotion of Science through Grants-in-Aid for Scientific Research (C) (20K11800).
Author information
Authors and Affiliations
Contributions
YK is in charge of conceptualization, methodology, validation, formal analysis, investigation, data curation, writing—original draft, and visualization. MU is in charge of conceptualization, methodology, formal analysis, investigation, writing—review and editing, supervision, and funding acquisition.
Corresponding author
Ethics declarations
Funding
This work was supported in part by the Japan Society for the Promotion of Science through Grants-in-Aid for Scientific Research (C) (20K11800).
Conflict of interest
The authors declare that they have no conflict of interest.
Code availability
Available at https://github.com/YasuhiroKatsura/ord-comp.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article is part of the topical collection “ACML 2020” guest edited by Masashi Sugiyama, Sinno Jialin Pan, Thanaruk Theeramunkong and Wray Buntine.
This article is the extended version of our own paper entitled “Bridging Ordinary-Label Learning and Complementary-Label Learning” presented in 2020 Asian Conference on Machine Learning.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Katsura, Y., Uchida, M. Candidate-Label Learning: A Generalization of Ordinary-Label Learning and Complementary-Label Learning. SN COMPUT. SCI. 2, 288 (2021). https://doi.org/10.1007/s42979-021-00681-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s42979-021-00681-x