Combination of Linear Classifiers Using Score Function – Analysis of Possible Combination Strategies
In this work, we addressed the issue of combining linear classifiers using their score functions. The value of the scoring function depends on the distance from the decision boundary. Two score functions have been tested and four different combination strategies were investigated. During the experimental study, the proposed approach was applied to the heterogeneous ensemble and it was compared to two reference methods – majority voting and model averaging respectively. The comparison was made in terms of seven different quality criteria. The result shows that combination strategies based on simple average, and trimmed average are the best combination strategies of the geometrical combination.
KeywordsBinary classifiers Linear classifiers Geometrical space Potential function
This work was supported in part by the National Science Centre, Poland under the grant no. 2017/25/B/ST6/01750.
- 3.Burduk R, Walkowiak K (2015) Static classifier selection with interval weights of base classifiers. In: Asian conference on intelligent information and database systems. Springer, pp 494–502Google Scholar
- 12.Gurney K (1997) An introduction to neural networks. Taylor & Francis, London. https://doi.org/10.4324/9780203451519
- 14.Hall MA (1999) Correlation-based feature selection for machine learning. Ph.D. thesis, The University of WaikatoGoogle Scholar
- 19.Kuncheva LI (2004) Combining pattern classifiers: methods and algorithms, 1st edn. Wiley-InterscienceGoogle Scholar
- 21.Markiewicz A, Forczmański P (2015) Detection and classification of interesting parts in scanned documents by means of adaboost classification and low-level features verification. In: International conference on computer analysis of images and patterns. Springer, pp 529–540Google Scholar
- 24.Ponti MP Jr (2011) Combining classifiers: from the creation of ensembles to the decision fusion. In: 2011 24th SIBGRAPI conference on graphics, patterns and images tutorials (SIBGRAPI-T). IEEE, pp 1–10Google Scholar
- 28.Rejer I, Burduk R (2017) Classifier selection for motor imagery brain computer interface. In: IFIP international conference on computer information systems and industrial management. Springer, pp 122–130Google Scholar
- 31.Trawiński B, Lasota T, Kempa O, Telec Z, Kutrzyński M (2017) Comparison of ensemble learning models with expert algorithms designed for a property valuation system. In: International conference on computational collective intelligence. Springer, pp 317–327Google Scholar
- 32.Tulyakov S, Jaeger S, Govindaraju V, Doermann D (2008) Review of classifier combination methods. In: Machine learning in document analysis and recognition. Springer, pp 361–386Google Scholar