System Evaluation of Construction Methods for Multi-class Problems Using Binary Classifiers
Construction methods for multi-valued classification (multi-class) systems using binary classifiers are discussed and evaluated by a trade-off model for system evaluation based on rate-distortion theory. Suppose the multi-class systems consisted of \(M (\ge 3)\) categories and \(N (\ge M-1)\) binary classifiers, then they can be represented by a matrix W, where the matrix W is given by a table of M code words with length N, called a code word table. For a document classification task, the relationship between the probability of classification error \(P_e\) and the number of binary classifiers N for given M is investigated, and we show that our constructed systems satisfy desirable properties such as “Flexible”, and “Elastic”. In particular, modified Reed Muller codes perform well: they are shown to be “Effective elastic”. As a second application we consider a hand-written character recognition task, and we show that the desirable properties are also satisfied.
KeywordsMulti-valued classification Binary classifier Trade-off model ECOC Exhaustive code Error correcting codes
One of the authors S. H. would like to thank Professor Shin’ ichi Oishi of Waseda University for giving a chance to study this work. The research leading to this paper was partially supported by MEXT Kakenhi under Grant-in Aids for Scientific Research (B) No. 26282090 and (C) No. 16K00491.
- 2.Armano, G., Chira, C., Hatami, N.: Error-correcting output codes for multi-label text categorization. In: Proceedings of the Third Italian Information Retrieval Workshop, IIR 2012, Italy, pp. 26–37, January 2012Google Scholar
- 4.Goto, M., Kobayashi, M.: Introduction to Pattern Recognition and Machine Learning (in Japanese). Corona-Sha, Tokyo (2014)Google Scholar
- 5.Hirasawa, S., Inazumi, H.: A system evaluation model by using information theory. In: The 30th Joint National Meeting, ORSA/TIMS, MB35.3, Philadelphia, PE, USA, October 1990Google Scholar
- 6.Hirasawa, S., Inazumi, H.: A model for system evaluation based on information theory. In: The 2000 International Conference of Management Science and Decision Making. Tamkang University, Taipei, ROC (2000)Google Scholar
- 7.Hirasawa, S., Saito, T., Inazumi, H., Matsushima, T.: System evaluation of disk allocation methods for Cartesian product files by using error correcting codes. In: Proceedings of 2011 IEEE International Conference on Systems, Man, and Cybernetics, Anchorage, Alaska, USA, pp. 2443–2448, 9–12 October 2011Google Scholar
- 8.Inazumi, H.: Studies on the evaluations for information systems based on rate distortion theory. Dissertation of Dr. Eng, Waseda University, November 1989Google Scholar
- 10.Kumoi, G., Kobayashi, M., Goto, M., Hirasawa, S.: A consideration in code word composition in multi-level document classification by ECOC method (in Japanese). In: 15th Information Science and Technology Forum, FIT 2016, Toyama, 7–9 September 2016Google Scholar
- 11.Luo, Y., Najarian, K.: Employing decoding of specific error correcting codes as a new classification criterion in multiclass learning problems. In: Proceedings of 2010 International Conference on Pattern Recognition, pp. 4238–4241 (2010)Google Scholar
- 13.Yomiuri News Paper Articles 2000, Naigai Associates IncGoogle Scholar
- 14.UCI machine learning repository. http://www.trifields.jp/uci-machine-learning-repository-dataset s-956G