Abstract
The act of evaluation requires an expenditure of resources. In Part I of this paper, we present a simple decision tree model borrowed from operations research to provide a conceptual framework for considering whether or not to commit such resources. In Part II, once the evaluation is carried out, we address the problem of evaluating the evaluation as a vehicle for producing useful information to decisionmakers. Evaluation inputs, processes, and outcomes are defined and discussed within the context of comprehensive evaluation of evaluations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bernstein, I. N. and Freeman, H. E. (1975). Academic and Entrepreneurial Research: The Consequences of Diversity in Federal Evaluation Studies. New York: Russell Sage Foundation.
Chaiken, J., Cranbill, L., Holliday, L., Jaquette, D., Lawless, M. and Quade, E. (1975). “Criminal Justice Models: An Overview.” Santa Monica, CA: Rand Corporation Technical Report R-1859-DOJ.
Cook, T. D. (1978) “Utilization, knowledge-building, and institutionalization: Three criteria by which evaluation research can be evaluated,” (Introduction) in Cook, T. D. (ed.), Evaluation Studies Review Annual, Vol. 3. Beverly Hills: Sage.
Cook, T. D. and Gruder, C. L. (1978). “Metaevaluation research,” Evaluation Quarterly 2: 5–51.
Howard, R. A. (1966). “Decision Analysis: Applied Decision Theory,” in Hertz, D. B. and Melese, J. (eds.) Proceedings of the Fourth International Conference on Operations Research. New York: Wiley.
Glass, G. U. (1976). “Primary, secondary and meta-analysis of research,” Educational Researcher, 5 (10): 3–8.
Larson, R. C., Bier, V. M., Kaplan, E. H., Mattingly, C., Eckels, T. J., Rechman, N. and Berliner, L. S. (1979). “Interim Analysis of 200 Evaluations on Criminal Justice.” (LEAA Grant 78NI-AX-0007). Operations Research Center. Cambridge, MA.: Massachusetts Institute of Technology.
Larson, R. C. and Kaplan, E. H. (1981). “Decision oriented approaches to program evaluation,” New Directions for Program Evaluation, 10: 49–68.
Minnesota Systems Research, Inc. (1973). “Proposed Review Systems for the Evaluation of Technical Quality & R & D Projects in the Department of Health Education and Welfare.”
Rosenthal, R. (1978) “Combining Results of Independent Studies,” Psychological Bulletin 85, No. 1, 185–193.
Rossi, P. H. and Wright, S. R. (1977) “An assessment of theory, practice, and politics,” Evaluation Research 1: 5–52.
Stufflebeam, D. L. (1974). “Meta-Evaluation.” Michigan University Evaluation Center, Occasional Paper #3.
The Literary Digest, Vol. 121, No. 1, January 4, (1936). Funk & Wagnalls Co., 354–360 Fourth Avenue, New York.
Thompson, M. (1975). Evaluation for Decisions in Social Programmes. Westmead, England: Saxon House, D.C. Heath Ltd.
Thompson, M. (1982). Decision Analysis For Program Evaluation. Cambridge, MA: Ballinger Publishing Co.
United States General Accounting Office (1978a) “Status and Issues: Federal Program Evaluation.” (PAD-78-83). Washington, D.C.
United States General Accounting Office (1978b). “Assessing Social Program Impact Evaluations: A Checklist Approach.” (Exposure Draft) (PAD-79-2), Washington, D.C.
Weiss, C. H. (1977). Using Social Research in Public Policy Making. Lexington, MA.: Lexington Books.
Author information
Authors and Affiliations
Additional information
This work was supported by grant 80-IJ-CX-0048 from the National Institute of Justice of the U.S. Department of Justice to the Operations Research Center of the Massachusetts Institute of Technology. Points of view or opinions stated in this document are those of the authors and do not necessarily represent the official position or policies of the U.S. Department of Justice.
Rights and permissions
About this article
Cite this article
Larson, R.C., Berliner, L. On evaluating evaluations. Policy Sci 16, 147–163 (1983). https://doi.org/10.1007/BF00138348
Issue Date:
DOI: https://doi.org/10.1007/BF00138348