Multi-labeler Analysis for Bi-class Problems Based on Soft-Margin Support Vector Machines
This work presents an approach to quantify the quality of panelist’s labeling by means of a soft-margin support vector machine formulation for a bi-class classifier, which is extended to multi-labeler analysis. This approach starts with a formulation of an objective function to determine a suitable hyperplane of decision for classification tasks. Then, this formulation is expressed in a soft-margin form by introducing some slack variables. Finally, we determine penalty factors for each panelist. To this end, a panelist’s effect term is incorporated in the primal soft-margin problem. Such problem is solved by deriving a dual formulation as a quadratic programming problem. For experiments, the well-known Iris database is employed by simulating multiple artificial labels. The obtained penalty factors are compared with standard supervised measures calculated from confusion matrix. The results show that penalty factors are related to the nature of data, allowing to properly quantify the concordance among panelists.
KeywordsBi-class classifier multi-labeler analysis quadratic programming support vector machines
Unable to display preview. Download preview PDF.
- 2.Dekel, O., Shamir, O.: Vox populi: Collecting high-quality labels from a crowd. In: Proceedings of the 22nd Annual Conference on Learning Theory (2009)Google Scholar
- 3.Smyth, P., Fayyad, U.M., Burl, M.C., Perona, P., Baldi, P.: Inferring ground truth from subjective labelling of venus images. In: NIPS 1994, pp. 1085–1092 (1994)Google Scholar
- 4.Crammer, K., Kearns, M., Wortman, J., Bartlett, P.: Learning from multiple sources. In: Advances in Neural Information Processing Systems, vol. 19 (2007)Google Scholar
- 5.Dekel, O., Shamir, O.: Good learners for evil teachers. In: ICML, p. 30 (2009)Google Scholar
- 6.Frank, A., Asuncion, A.: UCI machine learning repository (2010)Google Scholar