The Measurement of Transfer Using Return on Investment

Chapter

Abstract

Research into transfer of learning was originally focused on outcomes’ evaluation in terms of reaction, learning, behavior, and results. Outcomes’ evaluation, widely accepted by practitioners, is criticized by researchers seeking a more systemic approach for assessing the effectiveness of training interventions. As a result, the field of transfer research developed approaches more cognizant of context with a muted emphasis on outcomes. In turn, these approaches were criticized for their lack of tangible evidence of transfer of training. This chapter describes the development of evaluation from its early days up until its current evolution. The original outcomes’ model, the Four-Level Model by Donald Kirkpatrick, is described and its contributions and criticisms are discussed. Phillips’ return on investment approach (ROI) is also described. An account of how to measure the transfer of training using ROI is discussed. Finally, the implications of using ROI as a measure of transfer are considered.

References

  1. Alliger, G. M., & Janak, E. A. (1989). Kirkpatrick’s levels of training criteria: Thirty years later. Personnel Psychology, 42(2), 331–340.CrossRefGoogle Scholar
  2. Alliger, G. M., Tannenbaum, S. I., Bennett, W. Jr., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50(2), 341–358.CrossRefGoogle Scholar
  3. Aragón-Sanchéz, A., Barba-Aragón, I., & Sanz-Valle, R. (2003). Effects of training on business results. Journal of Human Resource Management, 14(6), 956–980.Google Scholar
  4. Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for future research. Personnel Psychology, 41(1), 63–105.CrossRefGoogle Scholar
  5. Bartel, A. P. (2000). Measuring the employer’s return on investments in training: Evidence from the literature. Industrial Relations, 39(3), 502–524.Google Scholar
  6. Bates, R. (2004). A critical analysis of evaluation practice: The Kirkpatrick model and the principle of beneficience. Evaluation and Program Planning, 27(3), 341–347.CrossRefGoogle Scholar
  7. Bates, R., & Holton, E. F. III (2004). Linking workplace literacy skills and transfer system perceptions. Human Resource Development Quarterly, 15(2), 153–170.CrossRefGoogle Scholar
  8. Bee, F., & Bee. R. (1997). Training needs analysis and evaluation. London: Institute of Personnel and Development.Google Scholar
  9. Bramley, P. (1991). Evaluating training effectiveness: Translating theory into practice. London: McGraw.Google Scholar
  10. Brinkerhoff, R. O. (1989). Achieving results from training. San Francisco: Jossey-Bass.Google Scholar
  11. Broad, M. L., & Newstrom, J. W. (1992). Transfer of training. Reading: Addison-Wesley.Google Scholar
  12. Burke, L. A., & Hutchins, H. M. (2007). Training transfer: An integrative literature review. Human Resource Development Review, 6(3), 263–296.CrossRefGoogle Scholar
  13. Burkett, H. (2005). ROI on a shoestring: Evaluation strategies for resource-constrained environments or ROI on a shoestring. Industrial & Commercial Training, 37(2), 97–105.CrossRefGoogle Scholar
  14. Cascio, W. F. (1987). Applied psychology in personnel management. Englewood Cliffs: Prentice-HallGoogle Scholar
  15. Diaz, M., & Sligo, J. (1997). How software process improvement helped Motorola. Software, IEEE, 14(5), 75–81.CrossRefGoogle Scholar
  16. Dixon, N. M. (1996). New routes to evaluation. Training and Development, 50(5), 82.Google Scholar
  17. Drimmer, A. (2002). Reframing the measurement debate: Moving beyond program analysis on the learning function. Washington, DC: Corporate Executive Board.Google Scholar
  18. Easterby-Smith, M. (1986). Evaluating management development, training and education. Aldershot: Gower.Google Scholar
  19. Gill, J. J. P. (1996). Research methods for managers. London: Paul Chapman.Google Scholar
  20. Goldstein, I. L. (1986). Training in organisation: Needs assessment, development and evaluation. Monterey: Brooks/Cole.Google Scholar
  21. Hamblin, A. C. (1974). Evaluation and control of training. New York: McGraw-Hill.Google Scholar
  22. Holton, E. F. III. (1996). The flawed four-level evaluation model. Human Resource Development Quarterly, 7(1), 5–21.CrossRefGoogle Scholar
  23. Holton, E. F. III., & Naquin, S. (2005). A critical analysis of HRD evaluation models from a decision-making perspective. Human Resource Development Quarterly, 16(2), 257–280.CrossRefGoogle Scholar
  24. Kaufman, R., & Keller, J. M. (1994). Levels of evaluation: Beyond Kirkpatrick. Human Resource Development Quarterly, 5(4), 371–380.CrossRefGoogle Scholar
  25. Kirkpatrick, D. L. (1959a). Techniques for evaluating training programs. Journal of ASTD, 13(11), 3–9.Google Scholar
  26. Kirkpatrick, D. L. (1959b). Techniques for evaluating training programs: Part 2—Learning. Journal of ASTD, 13(12), 21–26.Google Scholar
  27. Kirkpatrick, D. L. (1960a). Techniques for evaluating training programs: Part 3—Behaviour. Journal of ASTD, 14(1), 13–18.Google Scholar
  28. Kirkpatrick, D. L. (1960b). Techniques for evaluating training programs: Part 4—Results. Journal of ASTD, 14(2), 28–32.Google Scholar
  29. Kirkpatrick, D. L. (1994). Evaluating training programs. San Francisco: Berrett-Koehler.Google Scholar
  30. Kontoghiorghes, C. (2001). Factors affecting training effectiveness in the context of the introduction of new technology—A US case study, 248–260.Google Scholar
  31. Kontoghiorghes, C. (2002). Predicting motivation to learn and motivation to transfer learning back to the job in a service organization: A new systemic model for training effectiveness. Performance Improvement Quarterly, 15(3), 114–119.CrossRefGoogle Scholar
  32. Kontoghiorghes, C. (2004). Reconceptualizing the learning transfer conceptual framework: Empirical validation of a new systemic model. International Journal of Training and Development, 8(3), 210–221.CrossRefGoogle Scholar
  33. Matalonga, S., & Feliu, T. S. (2012). Calculating return on investment of training using process variation. IET Software, 6(2), 140–147.CrossRefGoogle Scholar
  34. Mathieu, J., Tannenbaum, S., & Salas, E. (1992). Influences of individual and situational characteristics on measures of training effectiveness. Academy of Management Journal, 35(4), 828–847.CrossRefGoogle Scholar
  35. McArdle, G. (2011). Instructional Design for Action Learning. New York: Amacom.Google Scholar
  36. Olsen, J. H., Jr. (1998). The evaluation and enhancement of training transfer. International Journal of Training and Development, 2(1), 61–75.CrossRefGoogle Scholar
  37. Paradise, A. (2007). State of the industry: ASTD’s annual review of trends in workplace learning and performance. Alexandria: ASTD.Google Scholar
  38. Phillips, J. J. (1995). Return on investment-Beyond the four levels. In E. Holton III (Ed.), Academy of HRD 1995 conference proceedings. Baton Rouge: Academy of HRDGoogle Scholar
  39. Phillips, J. J. (1996). Accountability in human resource management. Houston: Gulf.Google Scholar
  40. Phillips, J. J. (1999). HRD trends worldwide: Shared solutions to compete in a global economy. Houston: Gulf.Google Scholar
  41. Phillips, J. J. (2000). The corporate university: Measuring the impact of learning. Houston: American Productivity and Quality Center.Google Scholar
  42. Phillips, J. J. (2003). Return on investment in training and performance improvement programs. Burlington: Butterworth-Heinemann.Google Scholar
  43. Phillips, J. J. (2005). Communicating results to top executives. Chief Learning Officer, 4(4), 60–68.Google Scholar
  44. Phillips, P. P., & Phillips, J. J. (2001). Action: Measuring Return on Investment. Alexandria: American Society for Training and Development.Google Scholar
  45. Phillips, J. J., & Phillips, P. P. (2002). Technology’s return on investment. Advances in Human Resource Development, 4(4), 512–532.CrossRefGoogle Scholar
  46. Preskill, H. (1997). HRD evaluation as the catalyst for organizational learning. In E. Holton III (Ed.), Proceedings of the Academy of Human Resource Development. Baton Rouge: AHRD.Google Scholar
  47. Rowden, R. W. (2005). Exploring methods to evaluate the return-on-investment from training. Business Forum, 27(1), 31–36.Google Scholar
  48. Ruona, W., Leimbach, M., Holton, E. III., & Bates, R. (2002). The relationship between learner utility reactions and predicted learning transfer among trainees. International Journal of Training and Development, 6(4), 218–228.CrossRefGoogle Scholar
  49. Salas, E., & Cannon-Bowers, J. A. (2001). The science of training: A decade of progress. Annual Review of Psychology, 52(1), 471–499.CrossRefGoogle Scholar
  50. Shelton, S., & Alliger, G. M. (1993). Who’s afraid of level 4 evaluation? A practical approach. Training and Development Journal, 47(6), 43–46.Google Scholar
  51. Smidt, A., Balandin, S., Sigafoos, J., & Reed, V. A. (2009). The Kirkpatrick model: A useful tool for evaluating training outcomes. Journal of Intellectual & Developmental Disability, 34(3), 266–274.CrossRefGoogle Scholar
  52. Subramanian, K. S., Sinha, V., & Gupta, P. D. (2012). A study on return on investment of training programme in a government enterprise in India. Vikalpa: The Journal for Decision Makers, 37(1), 31–48.Google Scholar
  53. Tannenbaum, S. I., & Yukl, G. (1992). Training and development in work organizations. Annual Reveiw of Psychology, 43(3), 399–441.CrossRefGoogle Scholar
  54. Tracey, J. B., & Tews, M. J. (2005). Construct validity of a general training climate scale. Organizational Research Methods, 8(4), 353–374.CrossRefGoogle Scholar
  55. Twitchell, S., Holton, E. F. III., & . Trott Jr., J. R. (2000). Technical training evaluation practices in the United States. Performance Improvement Quarterly, 13(3), 84–110.CrossRefGoogle Scholar
  56. Tyler, R. W. (1949). Basic principles of curriculum design. Chicago: University of Chicago Press.Google Scholar
  57. Van Buren, M. E., & Erskine, W. (2002). The 2002 state of the Industry report. Alexandria: American Society for Training and Development.Google Scholar
  58. Van der Klink, M., Gielen, E., & Nauta, C. (2001). Supervisory support as a major condition to enhance transfer. International Journal of Training and Development, 5(1), 52–63.CrossRefGoogle Scholar
  59. Wang, G. G., & Spitzer, D. R. (2005). Human resource development measurement and evaluation: Looking back and moving forward. Advances in Developing Human Resources, 7(1), 5–15.CrossRefGoogle Scholar
  60. Wang, G. G., & Wang, J. (2005). Human resource development evaluation: Emerging market, barriers, and theory building. Advances in Developing Human Resources, 7(1), 22–36.CrossRefGoogle Scholar
  61. Wang, G. G., Dou, Z., & Li, N. (2002). A systems approach to measuring return on investment for HRD interventions. Human Resource Development Quarterly, 13(2), 203–224.CrossRefGoogle Scholar
  62. Warr, P., Allan, C., & Birdi, K. (1999). Predicting three levels of training outcome. Journal of Occupational and Organizational Psychology, 72(3), 351–375.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.National University of IrelandMaynoothIreland

Personalised recommendations