Skip to main content
SpringerLink
Log in

Customizable Tool for Online Training Evaluation

  • Living reference work entry
  • First Online:
Learning, Design, and Technology

  • 411 Accesses

Abstract

A proliferation of retail online training materials exists for many industries, but often the person in charge of choosing the most appropriate online training materials is neither a training expert nor versed in best practices associated with online training. To this end, it is critical that uninformed decision makers have access to an easy-to-use evaluation tool which allows for the assessment of strengths and weaknesses among multiple online training programs. Additionally, this tool must take into account the context of the training situation to ensure the chosen program is not only instructionally sound but also meets contextually specific training needs. This article describes the creation, testing, and application of the Customizable Tool for Online Training Evaluation (CTOTE), an evaluation instrument developed to help decision makers: (1) assess multiple online training programs against known best practices and (2) consider context-specific training needs via a weighting process. The three-step development process is explained including item selection and revision, determination of content validity and reliability, and the use of a Delphi panel to inform contextualized weighting. The instrument is then tested across multiple online training programs with results compared to an established online training evaluation instrument to illustrate the impact of the contextualized weighting. Lastly, the application of the instrument in a specific industry setting (food service) is presented to demonstrate the effectiveness of the instrument in this setting and to establish the potential of the CTOTE in helping uninformed decision makers assess multiple online training programs and make effective context-specific purchasing decisions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • Agariya, A. K., & Singh, D. (2012). e-Learning quality: Scale development and validation in Indian context. Knowledge Management & E-Learning: An International Journal (KM&EL), 4(4), 500–517. Retrieved from http://www.kmel-journal.org/ojs/index.php/online-publication/article/view/139/163

  • Alkhattabi, M., Neagu, D., & Cullen, A. (2010). Information quality framework for e-learning systems. Knowledge Management & E-Learning: An International Journal (KM&EL), 2(4), 340–362. Retrieved from http://www.kmel-journal.org/ojs/index.php/online-publication/article/view/21/62

  • Almanza, B. A., & Nesmith, M. S. (2004). Food safety certification regulations in the United States. Journal of Environmental Health, 66(9), 10–14.

    Google Scholar 

  • ATD Research. (2015). 2014 state of the industry report. Alexandria, VA: Association for Talent Development.

    Google Scholar 

  • Barker, N. K. (2004). The quality dilemma in online education. In T. Anderson & F. Elloumi (Eds.), Theory and practice of online learning (pp. 385–421). Retrieved from http://www.aupress.ca/index.php/books/120146

  • Barker, N. K. (2007). E-learning quality standards for consumer protection and consumer confidence: A Canadian case study in e-learning quality assurance. Journal of Educational Technology and Society, 10(2), 109–119. Retrieved from http://www.ifets.info/journals/10_2/10.pdf

  • Batalla-Busquets, J. M., & Pacheco-Bernal, C. (2013). On-the-job e-learning: Workers’ attitudes and perceptions. The International Review of Research in Open and Distributed Learning, 14(1), 40–64. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1304/2444

  • Becker, K., Fleming, J., & Keijsers, W. (2012). E-learning: Ageing workforce versus technology-savvy generation. Education + Training, 54(5), 385–400. 10.1108/00400911211244687.

    Google Scholar 

  • Caudill, J. G., & Reeves, B. (2014). Strategic management of workplace e-learning. In S. Hai-Jew (Ed.), Remote workforce training: Effective technologies and strategies (pp. 17–27). Hershey, PA: IGI Global.

    Chapter  Google Scholar 

  • Egan, M. B., Raats, M. M., Grubb, S. M., Eves, A., Lumbers, M. L., Dean, M. S., & Adams, M. R. (2007). A review of food safety and food hygiene training studies in the commercial sector. Food Control, 18, 1180–1190. doi:10.1016/j.foodcont.2006.08.001.

    Article  Google Scholar 

  • Frash, R., Jr., Binkley, M., Nelson, D., & Almanza, B. (2006). Transfer of training efficacy in US food safety accreditation. Journal of Culinary Science & Technology, 4(2–3), 7–38.

    Google Scholar 

  • Gillis, L. (2000). Quality standards for evaluating multimedia and online training. Whitby, ON: McGraw-Hill Ryerson.

    Google Scholar 

  • Grollman, W. K., & Cannon, D. (2003). eLearning: A better chalkboard. Financial Executive, 19(8), 45–47.

    Google Scholar 

  • Howton, J., Keiffer, E., Murphy, C. A., Sirsat, A. S., O’Bryan, C. A., Ricke, S. C., Neal, J. A. (2016). A comparison of online food safety training programs using the Customizable Tool for Online Training Evaluation. Food Control, 59, 82–87. doi:10.1016/j.foodcont.2015.04.034.

    Google Scholar 

  • Illinois Online Network. (2010). ION’s quality online course initiative. Retrieved from http://www.ion.uillinois.edu/initiatives/qoci/

  • Istrate, O. (2013). Criteria for e-learning programmes evaluation. Journal of Advanced Distributed Learning Technology, 1(1), 27–35. Retrieved from http://jadlet.com/index.php/jadlet/article/view/12/18

  • Jan, P. T., Lu, H. P., & Chou, T. C. (2012). The adoption of e-learning: An institutional theory perspective. Turkish Online Journal of Educational Technology-TOJET, 11(3), 326–343. Retrieved from http://www.tojet.net/articles/v11i3/11331.pdf

  • MarylandOnline. (2011). Quality Matters rubric standards 2011–2013 edition with assigned point values. Retrieved from http://www.qmprogram.org/rubric

  • McCue, T. J. (2014, October). Online learning industry poised for $107 billion in 2017. Forbes. Retrieved from http://www.forbes.com/sites/tjmccue/2014/08/27/online-learning-industry-poised-for-107-billion-in-2015/

  • Mosharraf, M., & Taghiyareh, F. (2013). Qualitative development of e-learning environments through a learner relationship management methodology. Knowledge Management & E-Learning: An International Journal (KM&EL), 5(1), 56–65. Retrieved from http://www.kmel-journal.org/ojs/index.php/online-publication/article/view/211/171

  • Murphy, C. A., Keiffer, E. A., Neal, J. A., & Crandall, P. G. (2013). A customizable evaluation instrument to facilitate comparisons of existing online training programs. Knowledge Management & E-Learning: An International Journal, 5(3), 251–268. Retrieved from http://www.kmel-journal.org/ojs/index.php/online-publication/article/view/242/186

  • Neal, J. A., Dawson, M., & Madera, J. M. (2011). Identifying food safety concerns when communication barriers exist. Journal of Food Science Education, 10, 36–44. doi:10.1111/j.1541-4329.2011.00128.x.

    Article  Google Scholar 

  • Neal, J. A., Murphy, C. A., Crandall, P. G., O’Bryan, C. A., Keiffer, E., & Ricke, S. C. (2011). Development of an evaluation tool for online food safety training programs. Journal of Food Science Education, 10(1), 9–12. Retrieved from http://onlinelibrary.wiley.com/doi/10.1111/j.1541-4329.2010.00112.x/epdf

  • Nunnaly, J. (1978). Psychometric theory. New York, NY: McGraw-Hill.

    Google Scholar 

  • Pappas, C. (2015, January). The top e-learning statistics and facts for 2015 you need to know. eLearning Industry. Retrieved from http://elearningindustry.com/elearning-statistics-and-facts-for-2015

  • Pisik, G. B. (1997). Is this course instructional sound? A guide to evaluating online training courses. Educational Technology, 37(4), 50–59.

    Google Scholar 

  • Pisik, G. B. (2004). Is this course instructionally sound? An updated guide to evaluating online training courses. Educational Technology, 44(1), 57–60. http://c.ymcdn.com/sites/aect.site-ym.com/resource/resmgr/Publication_PDFs/ET_Contents_Pages_2000-2015a.PDF

  • Santerre, C. R. (2005). X-train: Teaching professionals remotely. The Journal of Nutrition, 135(5), 1248–1252. Retrieved from http://jn.nutrition.org/content/135/5/1248.full

    Google Scholar 

  • Seufert, S. (2002). eLearning business models: Strategies, success factors, and best practice examples. In C. Wankel & R. DeFillippi (Eds.), Rethinking management education for the twenty-first century (pp. 109–132). Greenwich, CT: Information Age Publishing.

    Google Scholar 

  • Skillsoft. (2013). From boardroom to desktop: Skillsoft connects corporate strategy to the performance needs of individual learners with new Enterprise Learning Connection [Press Release]. Retrieved from http://www.skillsoft-usa.net/about/press_room/press_releases/may_02_06_elc.asp

  • Steele, J. (2015). Comparing computer-based employee training with traditional face-to-face methods. In D. Slykhuis & G. Marks (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2015 (pp. 504–507). Chesapeake, VA: Association for the Advancement of Computing in Education (AACE). Retrieved from http://www.editlib.org/p/150043/

  • Strother, J. B. (2002). An assessment of the effectiveness of e-learning in corporate training programs. The International Review of Research in Open and Distance Learning, 3(1). Retrieved from http://www.irrodl.org/index.php/irrodl/article/viewArticle/83/160Ubell

  • Training. (2014, November/December). 2014 training industry report. Training, 51(8), 16–29. Retrieved from http://www.trainingmag.com/sites/default/files/magazines/2014_11/2014-Industry-Report.pdf

  • Turner, R. C., & Carlson, L. (2003). Indexes of item-objective congruence for multidimensional items. International Journal of Testing, 3(2), 163–171. doi:10.1207/S15327574IJT0302_5.

    Article  Google Scholar 

  • Van Rooij, S. W. (2011). Instructional design and project management: Complementary or divergent? Educational Technology Research and Development, 59(1), 139–158.

    Article  Google Scholar 

  • Zaied, A. N. H. (2012). Multi-criteria evaluation approach for e-learning technologies: Selection criteria using AHP. International Journal on E-Learning, 11(4), 465–485.

    Google Scholar 

  • Zhang, D., & Nunamaker, J. F. (2003). Powering e-learning in the new millennium: An overview of e-learning and enabling technology. Information Systems Frontiers, 5(2), 207–218. doi:10.1023/A:1022609809036.

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported in part by an USDA National Integrated Food Safety Initiative Grant to the authors.

Author information

Authors and Affiliations

  1. Educational Technology, College of Education and Health Professions, Univeristy of Arkansas, 101 Peabody Hall, Fayetteville, AR, 72701, USA

    Cheryl A. Murphy

  2. Mathematical Sciences, J. William Fulbright College of Arts and Sciences, University of Arkansas, 311 Champion Hall, Fayetteville, AR, 72701, USA

    Elizabeth A. Keiffer

  3. Conrad N. Hilton College of Hotel and Restaurant Manegement, University of Houston, 229 C. N. Hilton Hotel and College, Houston, TX, 77204, USA

    Jack A. Neal

  4. H-E-B Grocery Company, LP, 636 S. Main, S, San Antonio, TX, 78204, USA

    Jessica Howton

Authors
  1. Cheryl A. Murphy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elizabeth A. Keiffer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jack A. Neal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jessica Howton
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cheryl A. Murphy .

Editor information

Editors and Affiliations

  1. College of Info, Ste G150, Univ of North Texas, Denton, Texas, USA

    Michael J Spector

  2. School of Education, Virginia Tech, Blacksburg, Virginia, USA

    Barbara B Lockee

  3. School of Education, Baker University, Overland Park, Kansas, USA

    Marcus D. Childress

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this entry

Cite this entry

Murphy, C.A., Keiffer, E.A., Neal, J.A., Howton, J. (2016). Customizable Tool for Online Training Evaluation. In: Spector, M., Lockee, B., Childress, M. (eds) Learning, Design, and Technology. Springer, Cham. https://doi.org/10.1007/978-3-319-17727-4_22-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-17727-4_22-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, Cham

  • Online ISBN: 978-3-319-17727-4

  • eBook Packages: Springer Reference EducationReference Module Humanities and Social SciencesReference Module Education

Publish with us

Policies and ethics

Search

Navigation