Abstract
A number of experimental evaluations ofexplanation-based learning (EBL) have been reported in the literature on machine learning. A close examination of the design of these experiments revelas certain methodological problems that could affect the conclusions drawn from the experiments. This article analyzes some of the more common methodological difficulties, and illustrates them using selected previous studies.
Article PDF
Similar content being viewed by others
References
BancilhonF., & RamakrishnanR. (1988). Performance evaluation of data intensive logic programs. In J.Minker (Ed.),Deductive databases and logic programming. San Mateo, CA: Morgan Kaufmann Publishers.
DeJongG., & MooneyR. (1986). Explanation-based learning: An alternative view,Machine Learning,1, 145–176.
Eisenberg, G., & Rivest, R. (1990). On the sample complexity of PAC-learning using random and chosen examples.Proceedings of the Third Workshop on Computational Learning Theory. Rochester, NY.
Elkan, C. (1989). Conspiracy numbers and caching for searching and/or trees and theorem-proving.Proceedings of the Eleventh International Joint Conference on Artificial Intelligence (pp. 341–346). Detroit, MI.
ElkanC., & SegreA.M. (1989).Not the last word on EBL algorithms (Technical Report 89–1010). Ithaca, NY: Cornell University, Department of Computer Science.
Eskey, M., & Zweben, M. (1990). Learning search control for constraint-based scheduling.Proceedings of the National Conference on Artificial Intelligence (pp. 908–915). Boston, MA.
Etzioni, O. (1990). Why PRODIGY/EBL works.Proceedings of the National Conference on Artificial Intelligence (pp. 916–922). Boston, MA.
Greiner, R., & Likuski, J. (1989). Incorporating redundant learned rules: A preliminary formal analysis of EBL.Proceedings of the Eleventh International Joint Conference on Artificial Intelligence (pp. 744–749). Detroit, MI.
Knoblock, C.A. (1990). Learning abstraction hierarchies for problem solving.Proceedings of the National Conference on Artificial Intelligence (pp. 923–928). Boston, MA.
KorfR. (1985). Depth-first iterative deepening: An optimal admissible tree search.Artificial Intelligence,27, 97–109.
Markovitch, S., & Scott, P.D. (1988). The role of forgetting in learning.Proceedings of the Fifth International Machine Learning Conference (pp. 459–465). Ann Arbor, MI.
MintonS. (1990).Learning search control knowledge. Hingham, MA: Kluwer Academic Publishers.
MintonS. (1990). Quantitative results concerning the utility of explanation-based learning.Artificial Intelligence,42, 363–392.
MitchellT.M., UtgoffP.E., & BanerjiR. (1983). Learning by experimentation: Acquiring and refining problem-solving heuristics. In R.S.Michalski, J.G.Carbonell, and & T.M.Mitchell (Eds.),Machine learning: An artificial intelligence approach (Vol. 1). San Mateo, CA: Morgan Kaufmann Publishers.
MitchellT.M., KellerR.M., & Kedar-CabelliS.T. (1986). Explanation-based generalization: A unifying view. Machine Learning,1, 47–80.
Mooney, R. (1989). The effect of rule use on the utility of explanation-based learning.Proceedings of the Eleventh International Joint Conference on Artificial Intelligence (pp. 725–730). Detroit, MI.
MooneyR. (1990).A general explanation-based learning mechanism. San Mateo, CA: Morgan Kaufmann Publishers.
NewellA., ShawJ.C., & SimonH. (1963). Empirical explorations with the logic theory machine: A case study in heuristics. In E.Feigenbaum & J.Feldman (Eds.),Computers and Thought. New York, NY: McGraw-Hill.
O'Rorke, P. (1987). LT revisited: Experimental results of applying explanation-based learning to the logic ofPrincipia Mathematica. Proceedings of the Fourth International Machine Learning Workshop (pp. 148–159). Irvine, CA.
O'RorkeP. (1989). LT revisited: Explanation-based learning and the logic ofPrincipia Mathematica.Machine Learning,4, 117–160.
Saletore, V., & Kale, L.V. (1989). Obtaining first solutions faster in AND/OR parallel execution of logic programs.Proceedings of the North American Conference on Logic Programming (pp. 390–408). Cleveland, OH.
Segre, A.M. (1987). On the operationality/generality trade-off in explanation-based learning.Proceedings of the Tenth International Joint Conference on Artificial Intelligence (pp. 242–248). Milan, Italy.
SeereA.M. (1987).Machine learning of robot assembly plans. Norwell, MA: Kluwer Academic Publishers.
SegreA.M., ElkanC., & RussellA. (1990).On valid and invalid methodologies for experimental evaluations of EBL (Technical Report 90–1126). Ithaca, NY: Cornell University, Department of Computer Science.
ShavlikJ.W. (1990). Acquiring recursive and iterative concepts with explanation-based learning.Machine Learning,5, 39–70.
ShavlikJ.W. (1990).Extending explanation-based learning. San Mateo, CA: Morgan Kaufmann Publishers.
Subramanian, D., & Feldman, R. (1990). The utility of EBL in recursive domain theories.Proceedings of the National Conference on Artificial Intelligence (pp. 942–951). Boston, MA.
Tambe, M., & Newell, A. (1988). Some chunks are expensive.Proceedings of the Fifth International Machine Learning Conference (pp. 451–458). Ann Arbor, MI.
ValiantL.G. (1984). A theory of the learnable.Communications of the ACM.27, 1134–1142.
Warren, D.S., Ahamad, M., Debray, S.K., & Kale, L.V. (1984). Executing distributed-prolog programs on a broadcast network.Proceedings of the Institute of Electrical and Electronics Engineers Symposium on Logic Programming (pp. 12–21). Atlantic City, NJ.
WhiteheadA.N., & RussellB. (1913).Principia Mathematica. Cambridge, England: Cambridge University Press.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Segre, A., Elkan, C. & Russell, A. A critical look at experimental evaluations of EBL. Mach Learn 6, 183–195 (1991). https://doi.org/10.1007/BF00114163
Issue Date:
DOI: https://doi.org/10.1007/BF00114163