Using the “Indicators of Engaged Learning Online” Framework to Evaluate Online Course Quality
This article is a case study of the use of the Indicators of Engaged Learning Online (IELO) framework (See Appendix 1) as a guide to evaluate the quality of online courses. The framework lends itself well to measures of engagement, particularly, in terms of online course design because of its comprehensiveness. Six online courses were evaluated for quality in terms of engaged learning based on thirty indicators contained within the framework. Results ranged from a score of 21 to 71 out of a potential total score of 90. This 0–90 scale represented a continuum of passive to engaged learning. The purpose of the pilot study was to explore how the Indicators of Engaged Learning Online (IELO) framework could be used as a tool for evaluating the quality of online courses by instructional designers (IDs) and instructors. Insights into the practical use of the IELO framework and the need for improved guidelines for IDs and instructors as they assess the amount of student engagement designed in a course are provided. Recommendations for practice have implications for both aspects of engagement: how a course is designed and how it is delivered.
KeywordsInstructional design Online learning Quality online instruction Student engagement
Compliance with Ethical Standards
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study.
Conflict of Interest
Paula M Bigatel declares that she has no conflict of interest. Stephanie Edel-Malizia declares that she has no conflict of interest.
- Allen, I., & Seaman, J. (2015). Grade level: Tracking online education in the United States. Babson Survey Research Group.Google Scholar
- ASHE Higher Education Report (2014). Effects from student engagement online, 40(6), 67–73. https://doi.org/10.1002/aehe.20018.
- Association of American Colleges and Universities (AAC&U). (2007). College learning for the new global century: A report from the national leadership council for liberal education & America's promise. Washington, DC: Author.Google Scholar
- Bigatel, P., & Williams, V. (2015). Measuring student engagement in an online program. Online journal of distance learning and administration. Retrieved from http://www.westga.edu/~distance/ojdla/summer182/bigatel_williams182.html.
- Blackmon, S. J., & Major, C. (2012). Student experiences in online courses: A qualitative research synthesis. The Quarterly Review of Distance Education, 13(2), 77–85.Google Scholar
- Boston, W. E., & Ice, P. (2011). Assessing retention in online learning: An administrative perspective. Online Journal of Distance Learning Administration. Retrieved from, http://www.westga.edu/~distance/ojdla/summer142/boston_ice142.html.
- Boston, W., Diaz, S. R., Gibson, A. M., Ice, P., Richardson, J., & Swan, K. (2010). An exploration of the relationship between indicators of the Community of Inquiry framework and retention in online programs. Journal of Asynchronous Learning Networks, 14(1), 3–19.Google Scholar
- Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin, 3, 7.Google Scholar
- Dayton, D., & Vaughn, M. M. (2007). Developing a quality assurance process to guide the design and assessment of online courses. Technical Communication, 54(4), 475–489.Google Scholar
- Edel-Malizia, S., & Brautigam, K. (2014). Gauging the quality of online learning by measuring 21st century engagement, in Proceedings of the 13th European Conference on E-Learning, Copenhagen (pp. 700–703). Copenhagen: Aalborg University.Google Scholar
- Garrison, D. R. (2007). Online community of inquiry review: Social, cognitive, and teaching presence issues. Journal of Asynchronous Learning Networks, 7(1), 61–72.Google Scholar
- Gray, J. A., & DiLoreto, M. (2016). The effects of student engagement, student satisfaction, and perceived learning in online learning environments. International Journal of Educational Leadership Preparation, 11(1), n1.Google Scholar
- Jones, B. F., Valdez, G., Nowakowski, J., & Rasmussen, C. (1995). Plugging in: Choosing and using educational technology. Washington, DC: North central regional educational lab., oak brook, IL; Council for Educational Development and Research.Google Scholar
- Kuh, G. D. (2008). Excerpt from high-impact educational practices: What they are, who has access to them, and why they matter. Association of American Colleges and Universities. Retrieved from http://ueeval.ucr.edu/teaching_practices_inventory/Kuh_2008.pdf.
- Lehman, R. M., & Conceição, S. C. O. (2014). Motivating and retaining online students: Research-based strategies that work. San Francisco: Jossey-Bass/Wiley.Google Scholar
- Means, B. (1993). Using technology to support education reform (pp. 20402–29328). US Government Printing Office, Superintendent of Documents, Mail Stop: SSOP, Washington, DC.Google Scholar
- National Survey of Student Engagement. (2000). Improving the college experience: National benchmarks of effective educational practice. Bloomington: Indiana University Center for Postsecondary Research and Planning.Google Scholar
- North Central Regional Educational Laboratory & the Metiri Group. (2003). enGuage 21st century skills: Literacy in the digital age. Retrieved September 30, 2016, from http://pict.sdsu.edu/engauge21st.pdf.
- Phipps, R., & Merisotis, J. (2000). Quality on the line: Benchmarks for success in internet-based distance education. Retrieved from http://www.ihep.org/research/publications/quality-line-benchmarks-success-internet-based-distance-education.
- Quality Matters (QM). (2011). Helping you deliver on your online promise. Retrieved from https://www.qualitymatters.org/.
- Revere, L., & Kovach, J. V. (2011). Online technologies for engaged learning: A meaningful synthesis for educators. The Quarterly Review of Distance Education, 12(2), 113–124.Google Scholar
- Southard, S., & Mooney, M. (2015). A comparative analysis of distance education quality assurance standards. The Quarterly Review of Distance Education, 16(1), 55–68.Google Scholar
- Street, H. D. (2010). Factors influencing a learner’s decision to drop-out or persist in higher education distance learning. Online Journal of Distance Learning Administration, 13(4). Retrieved from http://wp.westga.edu/wp-content/uploads/sites/40/ojdla/winter134/street134.pdf.
- Swaner, L., & Brownell, J. (2009). Outcomes of high impact practices for undeserved students. A review of the literature. Prepared for the Association of American Colleges and Universities.Google Scholar
- Yin, K. (2009). Case study research: Design and method (4th ed.). Thousand Oaks: Sage.Google Scholar