Bi-criteria optimization problems for decision rules
- 150 Downloads
We consider bi-criteria optimization problems for decision rules and rule systems relative to length and coverage. We study decision tables with many-valued decisions in which each row is associated with a set of decisions as well as single-valued decisions where each row has a single decision. Short rules are more understandable; rules covering more rows are more general. Both of these problems—minimization of length and maximization of coverage of rules are NP-hard. We create dynamic programming algorithms which can find the minimum length and the maximum coverage of rules, and can construct the set of Pareto optimal points for the corresponding bi-criteria optimization problem. This approach is applicable for medium-sized decision tables. However, the considered approach allows us to evaluate the quality of various heuristics for decision rule construction which are applicable for relatively big datasets. We can evaluate these heuristics from the point of view of (i) single-criterion—we can compare the length or coverage of rules constructed by heuristics; and (ii) bi-criteria—we can measure the distance of a point (length, coverage) corresponding to a heuristic from the set of Pareto optimal points. The presented results show that the best heuristics from the point of view of bi-criteria optimization are not always the best ones from the point of view of single-criterion optimization.
KeywordsDecision tables with many-valued decisions Systems of decision rules Dynamic programming Pareto optimal points Greedy heuristics
Research reported in this publication was supported by King Abdullah University of Science and Technology (KAUST). We are greatly indebted to the anonymous reviewer for useful comments and suggestions.
- Azad, M., Chikalov, I., & Moshkov, M. (2013). Optimization of decision rule complexity for decision tables with many-valued decisions. In IEEE international conference on systems, man, and cybernetics (pp. 444–448).Google Scholar
- Blockeel, H., Schietgat, L., Struyf, J., Džeroski, S., & Clare, A. (2006). Decision trees for hierarchical multilabel classification: A case study in functional genomics. In European conference on principles and practice of knowledge discovery in databases, Lecture notes in computer science (Vol. 4213, pp. 18–29).Google Scholar
- Bostrom, H. (1995). Covering vs divide-and-conquer for top-down induction of logic programs. In Proceedings of the 14th international joint conference on artificial intelligence (Vol. 2, pp. 1194–1200).Google Scholar
- Clark, P., & Niblett, T. (1989). The CN2 induction algorithm. Machine Learning, 3, 261–283.Google Scholar
- Cohen, W. W., & Singer, Y. (1999). A simple, fast, and effective rule learner. In Proceedings of the sixteenth national conference on artificial intelligence, American Association for Artificial Intelligence, AAAI ’99 (pp. 335–342).Google Scholar
- Lavrač, N., Fürnkranz, J., & Gamberger, D. (2010). Explicit feature construction and manipulation for covering rule learning algorithms. In J. Koronacki, Z. W. Raś, S. T. Wierzchoń & J. Kacprzyk (Eds.), Advances in machine learning. Studies in Computational Intelligence (Vol. 262, pp. 121–146). Berlin: Springer.CrossRefGoogle Scholar
- Lichman, M. (2013). UCI machine learning repository. http://archive.ics.uci.edu/ml. Accessed 10 Dec 2015.
- Michalski, S., & Pietrzykowski, J. (2007). iAQ: A program that discovers rules. In AAAI-07 AI Video Competition.Google Scholar
- Moshkov, M., & Chikalov, I. (2000). On algorithm for constructing of decision trees with minimal depth. Fundamenta Informaticae, 41(3), 295–299.Google Scholar
- Quinlan, J. R. (1993). C4.5: Programs for machine learning. Los Altos: Morgan Kaufmann.Google Scholar
- Rivest, R. L. (1987). Learning decision lists. Machine Learning, 2, 229–246.Google Scholar
- Wieczorkowska, A., Synak, P., Lewis, R. A., & Raś, Z. W. (2005). Extracting emotions from music data. In: Foundations of intelligent systems, Lecture notes in computer science (Vol. 3488, pp. 456–465). Berlin: Springer.Google Scholar
- Zielosko, B., Chikalov, I.,Moshkov,M., & Amin, T. (2014). Optimization of decision rules based on dynamic programming approach. In C. Faucher & L.C. Jain (Eds.), Innovations in intelligent machines-4. Studies in Computational Intelligence (Vol. 514, pp. 369–392). Cham: Springer.Google Scholar