Kirkpatrick plus: Evaluation and continuous improvement with a community focus

  • Ryan Watkins
  • Doug Leigh
  • Rob Foshay
  • Roger Kaufman


For almost 40 years, Donald Kirkpatrick's framework for evaluation has been used as a basic model for the identification and targeting of training-specific interventions in business, government, military, and industry alike. By approaching evaluation from four different perspectives—reaction, learning, behavior, and results—the model has provided a solid basis for the examination of training's impact on the organization. Despite the current practice of measuring one's success according to the success of one's clients, proposed changes in the model have not been frequently adopted. It is therefore likely time for professionals to reevaluate the utility and responsiveness of the Kirk-patrick framework to meet the value-added requirements of today's organizations. This article identifies tools and concepts for being responsive to the new organizational realities not originally addressed by the Kirkpatrick model.


Current Practice Educational Technology Continuous Improvement Solid Basis Organizational Reality 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. American Society for Training and Development (ASTD). (no date). Percent of courses evaluated at Kirkpatrick levels [online]. Available: [March 30, 1998].Google Scholar
  2. Dick, W., & Carey, L. (1989).The systematic design of instruction (3rd ed.). Glenview, IL: Scott Foresman & Co.Google Scholar
  3. Drucker, P.F. (1993)Post-capitalist society. New York: HarperBusiness.Google Scholar
  4. Foshay, R. (in press). An examination of evaluation research.Google Scholar
  5. Gilbert, T., & Gilbert, M. (1989: January). Performance engineering: Making human productivity a science.Performance Improvement.Google Scholar
  6. Hall, J., Sprague, D., & Watkins, R. (1995; August).Florida's job training programs: What is the return on taxpayers' investment? Tallahassee, FL: Florida TaxWatch Incorporated.Google Scholar
  7. Kaufman, R. (1992).Strategic planning plus: An organizational guide. Newbury Park, CA: Sage (Revised).Google Scholar
  8. Kaufman, R. (May–June, 1997). Avoiding the “dumbing down” of human performance improvement.Performance Improvement.Google Scholar
  9. Kaufman, R. (1998).Strategic thinking: A guide to identifying and solving problems—Revised. Arlington, VA & Washington, DC: Jointly published by the American Society for Training and Development and the International Society for Performance Improvement.Google Scholar
  10. Kaufman, R., & Keller, J. (1994: Winter). Levels of evaluation: Beyond Kirkpatrick.Human Resources Quarterly, 5(4).Google Scholar
  11. Kaufman, R., Keller, J., & Watkins, R. (1995). What works and what doesn't: Evaluation beyond Kirkpatrick.Performance and Instruction 35(2), 8–12.Google Scholar
  12. Kaufman, R., & Watkins, R. (1996: Spring). Costs-consequences analysis.HRD Quarterly.Google Scholar
  13. Kaufman, R., Watkins, R., & Sims, L. (1997). Cost-consequences analysis: A case study.Performance Improvement Quarterly 10(2).Google Scholar
  14. Kaufman, R., Watkins, R., Triner, D., & Stith, M. (1998: summer). The changing corporate mind: Organizations, vision, missions, purposes, and indicators on the move toward societal payoffs.Performance Improvement Quarterly.Google Scholar
  15. Kirkpatrick, D.L. (1959a). Techniques for evaluating training programs.Journal of ASTD, 13(11), 3–9.Google Scholar
  16. Kirkpatrick, D.L. (1959b). Techniques for evaluating training programs: Part 2—Learning.Journal of ASTD, 13(12), 21–26.Google Scholar
  17. Kirkpatrick, D.L. (1960a). Techniques for evaluating training programs: Part 3—Behavior.Journal of ASTD, 14(1), 13–18.Google Scholar
  18. Kirkpatrick, D.L. (1960b). Techniques for evaluating training programs: Part 4—Results.Journal of ASTD, 14(2), 28–32.Google Scholar
  19. Kirkpatrick, D.L. (1994).Evaluating training programs: The four levels. San Francisco: Berret-Koehler.Google Scholar
  20. Kirkpatrick, D.L. (1996). Evaluation. In R.L. Craig, & L.R. Bittel (Eds.),Training & Development Handbook. American Society for Training and Development, New York: McGraw-Hill Book Co.Google Scholar
  21. Kirkpatrick, D.L. (1996: Spring). Invited reaction: Reaction to Holton article.Performance Improvement Quarterly, 7(1), 23–29.Google Scholar
  22. Muellner, A. (Reporter). (1998, March 18).Morning Edition. Upper Marlboro, Maryland: National Public Radio.Google Scholar
  23. Muir, M., Watkins, R., Kaufman, R., & Leigh, D. (June, 1998). Costs-consequences analysis: A primer.Performance Improvement.Google Scholar
  24. Pava, M.N., & Krausz, J. (1995).Corporate responsibility and financial performance: The paradox of social cost. Westport, CN: Quorum Books.Google Scholar
  25. Phillips, J. (1996). Measuring the Results of Training. In R.L. Craig, & L.R. Bittel (Eds.),Training & Development Handbook. American Society for Training and Development, New York: McGraw-Hill Book Co.Google Scholar
  26. Popcorn, F. (1991).The Popcorn report. New York: Doubleday.Google Scholar
  27. Rothwell, W.J., & Kazanas, H.C. (1992).Mastering the instructional design process: A systematic approach. San Francisco: Jossey-Bass Publishers.Google Scholar
  28. Toffler, A. (1990).Powershift: Knowledge, wealth and violence at the edge of the 21st century. New York: Bantam Books.Google Scholar

Copyright information

© the Association for Educational Communications and Technology 1998

Authors and Affiliations

  • Ryan Watkins
    • 1
  • Doug Leigh
    • 1
  • Rob Foshay
    • 1
  • Roger Kaufman
    • 2
  1. 1.Office for Needs Assessment and PlanningFlorida State University at TallahasseeUSA
  2. 2.TRO Learning, Inc.Hoffman Estates

Personalised recommendations