Skip to main content

Advertisement

SpringerLink
Log in

Usability research in educational technology: a state-of-the-art systematic review

  • Research Article
  • Published:
Educational technology research and development Aims and scope Submit manuscript

Abstract

This paper presents a systematic literature review characterizing the methodological properties of usability studies conducted on educational and learning technologies in the past 20 years. PRISMA guidelines were followed to identify, select, and review relevant research and report results. Our rigorous review focused on (1) categories of educational and learning technologies that have been the focus of usability evaluation, (2) specific usability evaluation methods used, (3) outcome measures, and (4) research limitations. Findings revealed a diverse range of usability evaluation methods employed for different types of educational/learning technologies and the contexts in which those methods were used, with the majority of usability studies being performed on e-learning technologies within higher education contexts. Specific methods, instrumentation, and types of usability research found to be dominant in reviewed studies were further analyzed and classified, with findings suggesting inquiry methods using questionnaires were most prevalent. Prevalent outcome measures were also synthesized, with findings suggesting that the majority of usability research focuses on issues of technological usability, with very few studies considering pedagogical and socio-cultural aspects of usability. A number of limitations were found, including conceptual and procedural flaws, fundamental misunderstanding of usability evaluation methods, and inappropriate application of usability methods, suggesting potentially problematic and unreliable results. These findings are discussed in-depth, and implications for future research are provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Abulfaraj, A., & Steele, A. (2020). Coherent heuristic evaluation (CoHE): toward increasing the effectiveness of heuristic evaluation for novice evaluators. In A. Marcus & E. Rosenzweig (Eds.), International Conference on Human-Computer Interaction (pp. 3–20). Springer.

    Google Scholar 

  • Ackerman, R., Parush, A., Nassar, F., & Shtub, A. (2016). Metacognition and system usability: Incorporating metacognitive research paradigm into usability testing. Computers in Human Behavior, 54, 101–113. https://doi.org/10.1016/j.chb.2015.07.041

    Article  Google Scholar 

  • Alaswad, Z., & Nadolny, L. (2015). Designing for Game-Based Learning: The Effective Integration of Technology to Support Learning. Journal of Educational Technology Systems, 43(4), 389–402. https://doi.org/10.1177/0047239515588164

    Article  Google Scholar 

  • Albert, B., & Tullis, T. (2013). Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Newnes.

    Google Scholar 

  • Ally, M., & Prieto-Blázquez, J. (2014). What is the future of mobile learning in education? International Journal of Educational Technology in Higher Education, 11(1), 142–151. https://doi.org/10.7238/rusc.v11i1.2033

    Article  Google Scholar 

  • Alroobaea, R. S., Al-Badi, A. H., & Mayhew, P. J. (2013). AlRoobaea, R., Al-Badi, A. H., & Mayhew, P. J. (2013, March). A framework for generating domain-specific heuristics for evaluating online educational websites-Further validation. In 2nd International Conference on Human Computer Interaction and Learning Technology (ICHCILT 2013), Abu Dhabi, UAE.

  • Ardito, C., Costabile, M. F., Marsico, M. D., Lanzilotti, R., Levialdi, S., Roselli, T., & Rossano, V. (2006). An approach to usability evaluation of e-learning applications. Universal Access in the Information Society, 4(3), 270–283. https://doi.org/10.1007/s10209-005-0008-6

    Article  Google Scholar 

  • Arruda Filho, E. J. M., Nogueira, A. C. L., & da Costa, E. M. S. (2021). Social Influence effect on consumers’ intention to adopt mobile banking services. Information Systems Management. https://doi.org/10.1080/10580530.2021.1965678

    Article  Google Scholar 

  • Babatunde, F. O., MacDermid, J., Grewal, R., Macedo, L., & Szekeres, M. (2020). Development and Usability Testing of a Web-Based and Therapist-Assisted Coping Skills Program for Managing Psychosocial Problems in Individuals With Hand and Upper Limb Injuries: Mixed Methods Study. JMIR Human Factors, 7(2), e17088. https://doi.org/10.2196/17088

    Article  Google Scholar 

  • Barbosa, D., Schneider, G., Mossmann, J., & Santos, P. D. (2020). Usability and gameplay evaluation on mobile games: A user-centred application proposal. International Journal of Learning Technology, 15(2), 107–129. https://doi.org/10.1504/IJLT.2020.109570

    Article  Google Scholar 

  • Barrie, M., Socha, J. J., Mansour, L., & Patterson, E. S. (2019). Mixed reality in medical education: a narrative literature review. Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care. https://doi.org/10.1177/2327857919081006

    Article  Google Scholar 

  • Batra, S., & Bishu, R. R. (2007). Web usability and evaluation: issues and concerns. In N. Aykin (Ed.), International Conference on Usability and Internationalization (pp. 243–249). Springer.

    Google Scholar 

  • Benson, L., Elliott, D., Grant, M., Holschuh, D., Kim, B., Kim, H., Lauber, E., Loh, S., & Reeves, T. C. (2002). Usability and instructional design heuristics for e-learning evaluation (pp. 1615–1621). Association for the Advancement of Computing in Education (AACE).

  • Bernard, L., Raina, S., Taylor, B., & Kaza, S. (2021). Minimizing Cognitive Overload in Cybersecurity Learning Materials: An Experimental Study Using Eye-Tracking. IFIP World Conference on Information Security Education (pp. 47–63). Springer.

    Google Scholar 

  • Bernhaupt, R. (2010). User experience evaluation in entertainment. In R. Bernhaupt (Ed.), Evaluating user experience in games (pp. 3–7). Springer.

    Chapter  Google Scholar 

  • Bernsen, N. O., & Dybkjær, L. (2009). Multimodal usability. Springer Science & Business Media. https://doi.org/10.1007/978-1-84882-553-6

    Article  Google Scholar 

  • Bertolino, A., Braione, P., Angelis, G. D., Gazzola, L., Kifetew, F., Mariani, L.,…Tonella, P. (2021). A Survey of Field-based Testing Techniques. ACM Computing Surveys (CSUR), 54(5), 1–39.

    Article  Google Scholar 

  • Bourges-Waldegg, P., Moreno, L., & Rojano, T. (2000). The role of usability on the implementation and evaluation of educational technology. In Proceedings of the 33rd Annual Hawaii International Conference on System Sciences (pp. 7-pp). IEEE. https://doi.org/10.1109/HICSS.2000.926722

  • Bowman, D. A., Gabbard, J. L., & Hix, D. (2002). A Survey of Usability Evaluation in Virtual Environments: Classification and Comparison of Methods. Presence: Teleoperators and Virtual Environments, 11(4), 404–424. https://doi.org/10.1162/105474602760204309

    Article  Google Scholar 

  • Buzhardt, J., Abbott, M., Greenwood, C., & Tapia, Y. (2004). Usability Testing of the ClassWide Peer Tutoring-Learning Management System. Journal of Special Education Technology, 20(1), 19–29. https://doi.org/10.1177/016264340502000102

    Article  Google Scholar 

  • Calhoun, C., Sahay, S., & Wilson, M. (2021). Instructional Design Evaluation. Design for Learning. https://edtechbooks.org/id/instructional_design_evaluation

  • Cassino, R., & Tucci, M. (2011). Developing usable web interfaces with the aid of automatic verification of their formal specification. Journal of Visual Languages & Computing, 22(2), 140–149. https://doi.org/10.1016/j.jvlc.2010.12.001

    Article  Google Scholar 

  • Cayola, L., & Macías, J. A. (2018). Systematic guidance on usability methods in user-centered software development. Information and Software Technology, 97, 163–175.

    Article  Google Scholar 

  • Chang, V. (2016). Review and discussion: E-learning for academia and industry. International Journal of Information Management, 36(3), 476–485. https://doi.org/10.1016/j.ijinfomgt.2015.12.007

    Article  Google Scholar 

  • Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46.

    Article  Google Scholar 

  • Conn, V. S., Valentine, J. C., Cooper, H. M., & Rantz, M. J. (2003). Grey Literature in Meta-Analyses. Nursing Research, 52(4), 256–261. https://doi.org/10.1097/00006199-200307000-00008

    Article  Google Scholar 

  • Connan, V., Marcon, M. A., Mahmud, F. H., Assor, E., Martincevic, I., Bandsma, R. H., Vresk, L., & Walsh, C. M. (2019). Online education for gluten-free diet teaching: Development and usability testing of an e-learning module for children with concurrent celiac disease and type 1 diabetes. Pediatric Diabetes, 20(3), 293–303. https://doi.org/10.1111/pedi.12815

    Article  Google Scholar 

  • Costabile, M. F., De Marsico, M., Lanzilotti, R., Plantamura, V. L., & Roselli, T. (2005). On the usability evaluation of e-learning applications. In Proceedings of the 38th Annual Hawaii International Conference on System Sciences (pp. 6b-6b). IEEE. https://doi.org/10.1109/HICSS.2005.468

  • Coursaris, C. K., & Kim, D. J. (2011). A meta-analytical review of empirical mobile usability studies. Journal of Usability Studies, 6(3), 117–171.

    Google Scholar 

  • Criollo-C, S., Guerrero-Arias, A., Jaramillo-Alcázar, Á., & Luján-Mora, S. (2021). Mobile learning technologies for education: benefits and pending issues. Applied Sciences, 11(9), 4111. https://doi.org/10.3390/app11094111

    Article  Google Scholar 

  • Crowther, M. S., Keller, C. C., & Waddoups, G. L. (2004). Improving the quality and effectiveness of computer-mediated instruction through usability evaluations. British Journal of Educational Technology, 35(3), 289–303. https://doi.org/10.1111/j.0007-1013.2004.00390.x

    Article  Google Scholar 

  • Dabbagh, N., Fake, H., & Zhang, Z. (2019). Student Perspectives of Technology use for Learning in Higher Education. Revista Iberoamericana De Educación a Distancia, 22(1), 127. https://doi.org/10.5944/ried.22.1.22102

    Article  Google Scholar 

  • Darren, B., & Thom, N. (2015). New tools and techniques for understanding non-conscious consumer decisions. Applied Marketing Analytics, 1(3), 214–220.

    Google Scholar 

  • Davids, M. R., Chikte, U. M. E., & Halperin, M. L. (2013). An efficient approach to improve the usability of e-learning resources: The role of heuristic evaluation. Advances in Physiology Education, 37(3), 242–248. https://doi.org/10.1152/advan.00043.2013

    Article  Google Scholar 

  • Davis, R., Gardner, J., & Schnall, R. (2020). A review of usability evaluation methods and their use for testing eHealth HIV interventions. Current HIV/AIDS Reports, 17(3), 203–218. https://doi.org/10.1007/s11904-020-00493-3

    Article  Google Scholar 

  • de Abreu Cybis, W., Betiol, A. H., & Faust, R. (2015). Ergonomia e Usabilidade 3ª edição: Conhecimentos, Métodos e Aplicações. Novatec Editora.

  • de Lima Salgado, A., & Freire, A. P. (2014). Lima Salgado, A. D., & Freire, A. P. (2014, June). Heuristic evaluation of mobile usability: A mapping study. In: International Conference on Human-Computer Interaction (pp. 178–188). Springer, Cham. https://doi.org/10.1007/978-3-319-07227-2_18

  • Derouin, R. E., Fritzsche, B. A., & Salas, E. (2005). E-Learning in Organizations. Journal of Management, 31(6), 920–940. https://doi.org/10.1177/0149206305279815

    Article  Google Scholar 

  • Dingli, A., & Cassar, S. (2014). An intelligent framework for website usability. Advances in Human-Computer Interaction. https://doi.org/10.1155/2014/479286

    Article  Google Scholar 

  • Falkowska, J., Sobecki, J., & Pietrzak, M. (2016). Eye tracking usability testing enhanced with EEG analysis. International Conference of Design, User Experience, and Usability (pp. 399–411). Cham: Springer.

    Google Scholar 

  • Fernandez, A., Insfran, E., & Abrahão, S. (2011). Usability evaluation methods for the web: A systematic mapping study. Information and Software Technology, 53(8), 789–817. https://doi.org/10.1016/j.infsof.2011.02.007

    Article  Google Scholar 

  • Flagg, B. N. (2013). Formative Evaluation for Educational Technologies. Routledge.

    Book  Google Scholar 

  • Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378.

    Article  Google Scholar 

  • Fleiss, J. L., Levin, B., & Paik, M. C. (2013). Statistical methods for rates and proportions. John Wiley & Sons.

    Google Scholar 

  • Frøkjær, E., Hertzum, M., & Hornbæk, K. (2000). Measuring usability: are effectiveness, efficiency, and satisfaction really correlated?. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 345–352). https://doi.org/10.1145/332040.332455

  • Gladman, T., Tylee, G., Gallagher, S., Mair, J., Rennie, S. C., & Grainger, R. (2020). A tool for rating the value of health education mobile apps to enhance student learning (MARuL): Development and usability study. JMIR MHealth and UHealth, 8(7), e18015. https://doi.org/10.2196/18015

    Article  Google Scholar 

  • Goldberg, J., & Wichansky, A. (2003). Eye tracking in usability evaluation: A practitioner’s guide. The Mind’s Eye (pp. 493–516). Elsevier.

    Chapter  Google Scholar 

  • Grant, M. M. (2019). Difficulties in defining mobile learning: Analysis, design characteristics, and implications. Educational Technology Research and Development, 67(2), 361–388. https://doi.org/10.1007/s11423-018-09641-4

    Article  Google Scholar 

  • Haddaway, N. R., Collins, A. M., Coughlin, D., & Kirk, S. (2015). The role of google scholar in evidence reviews and its applicability to grey literature searching. PLoS ONE, 10(9), e0138237. https://doi.org/10.1371/journal.pone.0138237

    Article  Google Scholar 

  • Hadjerrouit, S. (2012). Investigating technical and pedagogical usability issues of collaborative learning with Wikis. Informatics in Education, 11(1), 45–64.

    Article  Google Scholar 

  • Haq, R., Li, B., Jovicic, A., Dastur, D., Trinkaus, M., & Kong, A. (2018). Web-based oncology educational tool for medical trainees on oncology rotation—results of a pilot study. Journal of Cancer Education, 33(4), 788–797. https://doi.org/10.1007/s13187-016-1151-x

    Article  Google Scholar 

  • Hasan, L., Morris, A., & Probets, S. (2013). E-commerce websites for developing countries – a usability evaluation framework. Online Information Review, 37(2), 231–251. https://doi.org/10.1108/OIR-10-2011-0166

    Article  Google Scholar 

  • Hinkle, D., Wiersma, W., & Jurs, S. (2003). Applied statistics for the behavioral sciences. Houghton Mifflin.

    Google Scholar 

  • Hornbæk, K. (2006). Current practice in measuring usability: Challenges to usability studies and research. International Journal of Human-Computer Studies, 64(2), 79–102. https://doi.org/10.1016/j.ijhcs.2005.06.002

    Article  Google Scholar 

  • Huizenga, J. C., ten Dam, G. T. M., Voogt, J. M., & Admiraal, W. F. (2017). Teacher perceptions of the value of game-based learning in secondary education. Computers & Education, 110, 105–115. https://doi.org/10.1016/j.compedu.2017.03.008

    Article  Google Scholar 

  • Ingrassia, P. L., Mormando, G., Giudici, E., Strada, F., Carfagna, F., Lamberti, F., & Bottino, A. (2020). Augmented reality learning environment for basic life support and defibrillation training: Usability study. Journal of Medical Internet Research, 22(5), e14910. https://doi.org/10.2196/14910

    Article  Google Scholar 

  • Ioannidis, J. P. A. (2005). Why Most Published Research Findings Are False. PLoS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124

    Article  Google Scholar 

  • ISO. (2011). ISO/IEC 25010:2011(en), Systems and software engineering—Systems and software Quality Requirements and Evaluation (SQuaRE)—System and software quality models. (n.d.). Retrieved September 9, 2021, from https://www.iso.org/obp/ui/#iso:std:iso-iec:25010:ed-1:v1:en:sec:4.4

  • ISO. (2019). ISO 9241–210:2019(en), Ergonomics of human-system interaction—Part 210: Human-centred design for interactive systems. Retrieved September 9, 2021, from https://www.iso.org/obp/ui/#iso:std:iso:9241:-210:ed-2:v1:en

  • Issa, T., & Isaias, P. (2015). Usability and human computer interaction (HCI). In T. Issa & P. Isaias (Eds.), Sustainable design (pp. 19–36). London: Springer.

    Chapter  Google Scholar 

  • Ivory, M. Y., & Hearst, M. A. (2001). The state of the art in automating usability evaluation of user interfaces. ACM Computing Surveys, 33(4), 470–516. https://doi.org/10.1145/503112.503114

    Article  Google Scholar 

  • Jackson, J. L., & Kuriyama, A. (2019). How often do systematic reviews exclude articles not published in english? Journal of General Internal Medicine, 34(8), 1388–1389. https://doi.org/10.1007/s11606-019-04976-x

    Article  Google Scholar 

  • Jahnke, I., Schmidt, M., Pham, M., & Singh, K. (2020). Sociotechnical-Pedagogical Usability for Designing and Evaluating Learner Experience in Technology-Enhanced Environments. Learner and User Experience Research.

  • Jeffries, R., & Desurvire, H. (1992). Usability testing vs heuristic evaluation: was there a contest? ACM SIGCHI Bulletin, 24(4), 39–41.

    Article  Google Scholar 

  • Kahiigi, E. K., Ekenberg, L., Hansson, H., Danielson, F. F. T., & Danielson, M. (2008). Exploring the e-Learning State of Art. Electronic Journal of E-Learning, 6(2), 149–160.

    Google Scholar 

  • Kavitha, V., & Lohani, R. (2019). A critical study on the use of artificial intelligence, e-Learning technology and tools to enhance the learners experience. Cluster Comput, 22, 6985–6989. https://doi.org/10.1007/s10586-018-2017-2

    Article  Google Scholar 

  • Kearney, M., Schuck, S., Burden, K., & Aubusson, P. (2012). Viewing mobile learning from a pedagogical perspective. Research in Learning Technology. https://doi.org/10.3402/rlt.v20i0.14406

    Article  Google Scholar 

  • Keegan, D., & Ireland, D. (2005). The incorporation of mobile learning into mainstream education and training. In World Conference on Mobile Learning, Cape Town (Vol. 11, pp. 1–17).

  • Khajouei, R., Ameri, A., & Jahani, Y. (2018). Evaluating the agreement of users with usability problems identified by heuristic evaluation. International Journal of Medical Informatics, 117, 13–18. https://doi.org/10.1016/j.ijmedinf.2018.05.012

    Article  Google Scholar 

  • Khajouei, R., & Farahani, F. (2020). A combination of two methods for evaluating the usability of a hospital information system. BMC Medical Informatics and Decision Making, 20(1), 84. https://doi.org/10.1186/s12911-020-1083-6

    Article  Google Scholar 

  • Khanum, M. A., & Trivedi, M. C. (2013). Exploring verbalization and collaboration during usability evaluation with children in context. arXiv preprint arXiv:1306.4094.

  • King, L. A. (2009). Visual navigation patterns and cognitive load. In International Conference on Foundations of Augmented Cognition (pp. 254–259). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02812-0_30

  • Kirkwood, A., & Price, L. (2014). Technology-enhanced learning and teaching in higher education: What is ‘enhanced’ and how do we know? A critical literature review. Learning, Media and Technology, 39(1), 6–36. https://doi.org/10.1080/17439884.2013.770404

    Article  Google Scholar 

  • Koszalka, T. A., Russ-Eft, D. F., & Reiser, R. (2013). Instructional Designer Competencies: The Standards (Fourth Edition). IAP.

  • Kraemer, H. C., Kupfer, D. J., Clarke, D. E., Narrow, W. E., & Regier, D. A. (2012). DSM-5: How reliable is reliable enough? American Journal of Psychiatry, 169(1), 13–15.

    Article  Google Scholar 

  • Krug, S. (2009). Rocket surgery made easy: The do-it-yourself guide to finding and fixing usability problems. New Riders.

  • Kumar, B. A., & Goundar, M. S. (2019). Usability heuristics for mobile learning applications. Education and Information Technologies, 24(2), 1819–1833. https://doi.org/10.1007/s10639-019-09860-z

    Article  Google Scholar 

  • Kumar, B. A., Goundar, M. S., & Chand, S. S. (2020). A framework for heuristic evaluation of mobile learning applications. Education and Information Technologies, 25(4), 3189–3204. https://doi.org/10.1007/s10639-020-10112-8

    Article  Google Scholar 

  • Kumar, B. A., & Mohite, P. (2018). Usability of mobile learning applications: A systematic literature review. Journal of Computers in Education, 5(1), 1–17. https://doi.org/10.1007/s40692-017-0093-6

    Article  Google Scholar 

  • Lai, J. W. M., & Bower, M. (2019). How is the use of technology in education evaluated? A systematic review. Computers & Education, 133, 27–42. https://doi.org/10.1016/j.compedu.2019.01.010

    Article  Google Scholar 

  • Lam, J., Yau, J., & Cheung, S. K. S. (2010). A review of mobile learning in the mobile age. In International Conference on Hybrid Learning (pp. 306–315). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-14657-2_28

  • Lau, R. W. H., Yen, N. Y., Li, F., & Wah, B. (2014). Recent development in multimedia e-learning technologies. World Wide Web, 17(2), 189–198. https://doi.org/10.1007/s11280-013-0206-8

    Article  Google Scholar 

  • Law, E.L.-C., & Abrahão, S. (2014). Interplay between User Experience (UX) evaluation and system development. International Journal of Human-Computer Studies, 72(6), 523–525. https://doi.org/10.1016/j.ijhcs.2014.03.003

    Article  Google Scholar 

  • Law, E.L.-C., & Sun, X. (2012). Evaluating user experience of adaptive digital educational games with Activity Theory. International Journal of Human-Computer Studies, 70(7), 478–497. https://doi.org/10.1016/j.ijhcs.2012.01.007

    Article  Google Scholar 

  • Lee, M. J. W., Georgieva, M., Alexander, B., Craig, E., & Richter, J. (2021). State of XR & Immersive Learning Outlook Report 2021. Walnut, CA: Immersive Learning Research Network.

  • Leow, M. C., Wang, L. Y. K., Lau, S. H., & Tan, C. K. (2016). Usability of RPG-based Learning Framework. International Journal of Human-Computer Interaction, 32(8), 643–653. https://doi.org/10.1080/10447318.2016.1183863

    Article  Google Scholar 

  • Lim, C. J., & Lee, S. (2007). Pedagogical usability checklist for ESL/EFL e-learning websites. Journal of Convergence and Information Technology, 2(3), 67–76.

    Google Scholar 

  • Liu, D., Bhagat, K. K., Gao, Y., Chang, T.-W., & Huang, R. (2017). The potentials and trends of virtual reality in education. In Virtual, augmented, and mixed realities in education (pp. 105–130). Springer, Singapore. https://doi.org/10.1007/978-981-10-5490-7_7

  • Lyons, L., & Mallavarapu, A. (2021). Collective Usability: Using Simulation Tools to Explore Embodied Design Challenges in Immersive, Shared Mixed-Reality Experiences. Educational Technology & Society, 24(2), 120–135. https://www.jstor.org/stable/27004936

  • Lyzara, R., Purwandari, B., & Zulfikar, M. F. (2019). E-government usability evaluation: Insights from a systematic literature review. In Proceedings of the 2nd International Conference on Software Engineering and Information Management (pp. 249–253).

  • Machado Neto, O., & Pimentel, M. da G. (2013). Heuristics for the assessment of interfaces of mobile devices. In Proceedings of the 19th Brazilian Symposium on Multimedia and the Web. https://doi.org/10.1145/2526188.2526237

  • Machado, A., Oliveira, A., Jácome, C., Pereira, M., Moreira, J., Rodrigues, J., Aparício, J., Jesus, L. M. T., & Marques, A. (2018). Usability of Computerized Lung Auscultation-Sound Software (CLASS) for learning pulmonary auscultation. Medical & Biological Engineering & Computing, 56(4), 623–633. https://doi.org/10.1007/s11517-017-1697-8

    Article  Google Scholar 

  • Mankoff, J., Dey, A. K., Hsieh, G., Kientz, J., Lederer, S., & Ames, M. (2003). Heuristic evaluation of ambient displays. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 169–176). Association for Computing Machinery, New York. https://doi.org/10.1145/642611.642642

  • Maramba, I., Chatterjee, A., & Newman, C. (2019). Methods of usability testing in the development of eHealth applications: A scoping review. International Journal of Medical Informatics, 126, 95–104. https://doi.org/10.1016/j.ijmedinf.2019.03.018

    Article  Google Scholar 

  • Marell-Olsson, E., & Jahnke, I. (2019). Wearable technology in a dentistry study program: Potential and challenges of smart glasses for learning at the workplace. In I. Buchem, R. Klamma, & F. Wild (Eds.), Perspectives on wearable enhanced learning (WELL) (pp. 433–451). Springer.

    Chapter  Google Scholar 

  • Martin, F., & Ertzberger, J. (2013). Here and now mobile learning: An experimental study on the use of mobile technology. Computers & Education, 68, 76–85. https://doi.org/10.1016/j.compedu.2013.04.021

    Article  Google Scholar 

  • Martin, S., Diaz, G., Sancristobal, E., Gil, R., Castro, M., & Peire, J. (2011). New technology trends in education: Seven years of forecasts and convergence. Computers & Education, 57(3), 1893–1906. https://doi.org/10.1016/j.compedu.2011.04.003

    Article  Google Scholar 

  • Martín-Gutiérrez, J., Mora, C. E., Añorbe-Díaz, B., & González-Marrero, A. (2017). Virtual technologies trends in education. Eurasia Journal of Mathematics, Science and Technology Education, 13(2), 469–486.

    Google Scholar 

  • McDonald, J. K., & Yanchar, S. C. (2020). Towards a view of originary theory in instructional design. Educational Technology Research and Development, 68(2), 633–651. https://doi.org/10.1007/s11423-019-09734-8

    Article  Google Scholar 

  • McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276–282.

    Article  Google Scholar 

  • Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., Group, T. P. (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLOS Medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097

    Article  Google Scholar 

  • Moore, J. L., Dickson-Deane, C., & Liu, M. Z. (2014). Designing CMS courses from a pedagogical usability perspective. In A. Benson & D. Whitworth (Eds.), Perspectives in instructional technology and distance education: Research on course management systems in higher education (pp. 143–169). Charlotte, NC: Information Age.

    Google Scholar 

  • Moore, J. L., Dickson-Deane, C., & Galyen, K. (2011). e-Learning, online learning, and distance learning environments: Are they the same? The Internet and Higher Education, 14(2), 129–135.

    Article  Google Scholar 

  • Moradian, S., Krzyzanowska, M. K., Maguire, R., Morita, P. P., Kukreti, V., Avery, J., Liu, G., Cafazzo, J., & Howell, D. (2018). Usability evaluation of a mobile phone-based system for remote monitoring and management of chemotherapy-related side effects in cancer patients: Mixed-methods study. JMIR Cancer, 4(2), e10932. https://doi.org/10.2196/10932

    Article  Google Scholar 

  • Moubayed, A., Injadat, M., Nassif, A. B., Lutfiyya, H., & Shami, A. (2018). E-Learning: Challenges and research opportunities using machine learning & data analytics. IEEE Access, 6, 39117–39138. https://doi.org/10.1109/ACCESS.2018.2851790

    Article  Google Scholar 

  • Nakamura, W., Teixeira de Oliveira, E., & Conte, T. (2018). Applying design science research to develop a technique to evaluate the usability and user experience of learning management systems. Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação-SBIE), 29(2), 953. https://doi.org/10.5753/cbie.sbie.2018.953

  • Navarro, C. X., Molina, A. I., Redondo, M. A., & Juárez-Ramírez, R. (2016). Framework to evaluate M-learning systems: A Technological and pedagogical approach. IEEE Revista Iberoamericana De Tecnologias Del Aprendizaje, 11(1), 33–40. https://doi.org/10.1109/RITA.2016.2518459

    Article  Google Scholar 

  • Nayebi, F., Desharnais, J.-M., & Abran, A. (2012). The state of the art of mobile application usability evaluation. In 25th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE) (pp. 1–4). IEEE. https://doi.org/10.1109/CCECE.2012.6334930

  • Nicholson, P. (2007). A History of E-Learning. In B. Fernández-Manjón, J. M. Sánchez-Pérez, J. A. Gómez-Pulido, M. A. Vega-Rodríguez, & J. Bravo-Rodríguez (Eds.), Computers and Education. Springer.

    Google Scholar 

  • Nielsen, J. (1994b). Enhancing the explanatory power of usability heuristics. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI ’94), pp. 152–158.

  • Nielsen, J. (1994a). 10 Usability Heuristics for User Interface Design. Nielsen Norman Group. https://www.nngroup.com/articles/ten-usability-heuristics/

  • Nielsen, J. (1990). Evaluating hypertext usability. In D. H. Jonassen & H. Mandl (Eds.), Designing hypermedia for learning (pp. 147–168). Springer.

    Chapter  Google Scholar 

  • Nokelainen, P. (2004). Conceptual definition of the technical and pedagogical usability criteria for digital learning material. In EdMedia+ Innovate Learning (pp. 4249–4254). Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/11688/

  • Nokelainen, P. (2005). The technical and pedagogical usability criteria for digital learning material. In EdMedia+ Innovate Learning (pp. 1011–1016). Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/20212/

  • Nokelainen, P. (2006). An empirical assessment of pedagogical usability criteria for digital learning material with elementary school students. Journal of Educational Technology & Society, 9(2), 178–197.

    Google Scholar 

  • Nyang’or, J., Villiers, R. D., & Ssemugabi, S. (2013). A framework for usability evaluation of an offline e-learning tutorial and its application in usability testing. In EdMedia+ Innovate Learning (pp. 1097–1105). Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/112097/

  • Oliver, M. (2000). An introduction to the evaluation of learning technology. Journal of Educational Technology & Society, 3(4), 20–30.

    Google Scholar 

  • Oztekin, A., Delen, D., Turkyilmaz, A., & Zaim, S. (2013). A machine learning-based usability evaluation method for eLearning systems. Decision Support Systems, 56, 63–73.

    Article  Google Scholar 

  • Page, M. J., McKenzie, J. E., Bossuyt, P. M., et al. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Systematic Reviews, 10, 89. https://doi.org/10.1186/s13643-021-01626-4

    Article  Google Scholar 

  • Park, B. K., Kim, J. Y., & Rogers, V. E. (2020). Development and usability evaluation of a facebook-based intervention program for childhood cancer patients: mixed methods study. Journal of Medical Internet Research, 22(7), e18779. https://doi.org/10.2196/18779

    Article  Google Scholar 

  • Paz, F., & Pow-Sang, J. A. (2014). Current trends in usability evaluation methods: A systematic review. In 2014 7th International Conference on Advanced Software Engineering and Its Applications, 2014, pp. 11–15. https://doi.org/10.1109/ASEA.2014.10

  • Paz, F., & Pow-Sang, J. A. (2016). A systematic mapping review of usability evaluation methods for software development process. International Journal of Software Engineering and Its Applications, 10(1), 165–178.

    Article  Google Scholar 

  • Pérez-Sanagustín, M., Nussbaum, M., Hilliger, I., Alario-Hoyos, C., Heller, R. S., Twining, P., & Tsai, C.-C. (2017). Research on ICT in K-12 schools – A review of experimental and survey-based studies in computers & education 2011 to 2015. Computers & Education, 104, A1–A15. https://doi.org/10.1016/j.compedu.2016.09.006

    Article  Google Scholar 

  • Perry, G. T., Kulpa, C. C., Pinheiro, E., & Eichler, M. L. (2012). Lessons from an educational game usability evaluation. International Journal of Interactive Mobile Technologies (IJIM), 6(2), 23. https://doi.org/10.3991/ijim.v6i2.1888

    Article  Google Scholar 

  • Pham, M., Singh, K., & Jahnke, I. (2021). Socio-technical-pedagogical usability of online courses for older adult learners. Interactive Learning Environments. https://doi.org/10.1080/10494820.2021.1912784

    Article  Google Scholar 

  • Piasecki, J., Waligora, M., & Dranseika, V. (2018). Google Search as an Additional Source in Systematic Reviews. Science and Engineering Ethics, 24(2), 809–810. https://doi.org/10.1007/s11948-017-0010-4

    Article  Google Scholar 

  • Plantak Vukovac, D., Kirinic, V., & Klicek, B. (2010). A comparison of usability evaluation methods for e-learning systems. DAAAM International Scientific Book. https://doi.org/10.2507/daaam.scibook.2010.27

    Article  Google Scholar 

  • Plass, J. L., Homer, B. D., & Kinzer, C. K. (2015). Foundations of Game-Based Learning. Educational Psychologist, 50(4), 258–283. https://doi.org/10.1080/00461520.2015.1122533

    Article  Google Scholar 

  • Polly, D., Allman, B., Casto, A., & Norwood, J. (2018). Sociocultural perspectives of learning. In R. E. West (Eds.), Foundations of learning and instructional design technology: The past, present, and future of learning and instructional design technology. EdTech Books. Retrieved from https://edtechbooks.org/lidtfoundations/sociocultural_perspectives_of_learning

  • Punchoojit, L., & Hongwarittorrn, N. (2017). Usability Studies on Mobile User Interface Design Patterns: A Systematic Literature Review. Advances in Human-Computer Interaction, 2017, e6787504. https://doi.org/10.1155/2017/6787504

    Article  Google Scholar 

  • Quiñones, D., & Rusu, C. (2017). How to develop usability heuristics: A systematic literature review. Computer Standards & Interfaces, 53, 89–122. https://doi.org/10.1016/j.csi.2017.03.009

    Article  Google Scholar 

  • Quiñones, D., Rusu, C., & Rusu, V. (2018). A methodology to develop usability/user experience heuristics. Computer Standards & Interfaces, 59, 109–129. https://doi.org/10.1016/j.csi.2018.03.002

    Article  Google Scholar 

  • Rajanen, D., Clemmensen, T., Iivari, N., Inal, Y., Rızvanoğlu, K., Sivaji, A., & Roche, A. (2017). UX professionals’ definitions of usability and UX–A comparison between Turkey, Finland, Denmark, France and Malaysia. In R. Bernhaupt, G. Dalvi, A. Joshi, D. K. Balkrishan, J. O’Neill, & M. Winckler (Eds.), Ifip conference on human-computer interaction (pp. 218–239). Springer.

    Google Scholar 

  • Rebelo, F., Noriega, P., Duarte, E., & Soares, M. (2012). Using virtual reality to assess user experience. Human Factors, 54(6), 964–982. https://doi.org/10.1177/0018720812465006

    Article  Google Scholar 

  • Reeves, T. C. (1994). Evaluating what really matters in computer-based education. In M. Wild & D. Kirkpatrick (Eds.), Computer education: New perspectives (pp. 219–246). MASTEC.

    Google Scholar 

  • Reeves, T. C., & Hedberg, J. C. (2003). Interactive learning systems evaluation. Educational Technology Publications.

    Google Scholar 

  • Resnik, L. (2011). Development and testing of new upper-limb prosthetic devices: Research designs for usability testing. The Journal of Rehabilitation Research and Development, 48(6), 697. https://doi.org/10.1682/JRRD.2010.03.0050

    Article  Google Scholar 

  • Robinson, H. A., Sheffield, A., Phillips, A. S., & Moore, M. (2017). “Introduction to Teaching Online”: Usability Evaluation of Interactivity in an Online Social Constructivist Course. TechTrends, 61(6), 533–540. https://doi.org/10.1007/s11528-017-0187-z

    Article  Google Scholar 

  • Rodrigues, H., Almeida, F., Figueiredo, V., & Lopes, S. L. (2019). Tracking e-learning through published papers: A systematic review. Computers & Education, 136, 87–98. https://doi.org/10.1016/j.compedu.2019.03.007

    Article  Google Scholar 

  • Rosell-Aguilar, F. (2017). State of the app: A taxonomy and framework for evaluating language learning mobile applications. CALICO Journal, 34(2), 243–258.

    Article  Google Scholar 

  • Rosenbaum, S. (2000). Not just a hammer: when and how to employ multiple methods in usability programs. In Proceedings of UPA.

  • Rosenbaum, S. (2008). The Future of Usability Evaluation: Increasing Impact on Value. In E. L. C. Law, E. T. Hvannberg, & G. Cockton (Eds.), Maturing Usability: Quality in Software, Interaction and Value (pp. 344–378). Springer.

    Chapter  Google Scholar 

  • Ruiz, J. G., Mintzer, M. J., & Leipzig, R. M. (2006). The impact of E-learning in medical education. Academic Medicine: Journal of the Association of American Medical Colleges, 81(3), 207–212. https://doi.org/10.1097/00001888-200603000-00002

    Article  Google Scholar 

  • Russell, J. D., & Blake, B. L. (1988). Formative and Summative Evaluation of Instructional Products and Learners. Educational Technology, 28(9), 22–28. http://www.jstor.org/stable/44426917

  • Santoso, H. B., Schrepp, M., Isal, R. Y. K., Utomo, A. Y., & Priyogi, B. (2016). Measuring User experience of the student-centered e-Learning environment. Journal of Educators Online, 13(1), 58–79.

    Article  Google Scholar 

  • Scanlon, E., & Issroff, K. (2005). Activity theory and higher education: Evaluating learning technologies. Journal of Computer Assisted Learning, 21(6), 430–439. https://doi.org/10.1111/j.1365-2729.2005.00153.x

    Article  Google Scholar 

  • Schmidt, M., Tawfik, A. A., Jahnke, I., & Earnshaw, Y. (2020). Learner and User Experience Research. EdTech Books. https://edtechbooks.org/ux

  • Schmidt, M. & Tawfik, A. A. (2022). Activity Theory as a Lens for Developing and Applying Personas and Scenarios in Learning Experience Design. The Journal of Applied Instructional Design, 11(1). https://edtechbooks.org/jaid_11_1/activity_theory_as_a

  • Schmidt, M., & Huang, R. (2022). Defining learning experience design: Voices from the field of learning design & technology. TechTrends, 66, 141–158. https://doi.org/10.1007/s11528-021-00656-y

    Article  Google Scholar 

  • Schnall, R., Cho, H., & Liu, J. (2018). Health Information Technology Usability Evaluation Scale (Health-ITUES) for usability assessment of mobile health technology: Validation study. JMIR mHealth and uHealth, 6(1), e8851. https://doi.org/10.2196/mhealth.8851

    Article  Google Scholar 

  • Schober, P., Boer, C., & Schwarte, L. A. (2018). Correlation coefficients: Appropriate use and interpretation. Anesthesia & Analgesia, 126(5), 1763–1768.

    Article  Google Scholar 

  • Seppälä, P., & Alamäki, H. (2003). Mobile learning in teacher training. Journal of Computer Assisted Learning, 19(3), 330–335. https://doi.org/10.1046/j.0266-4909.2003.00034.x

    Article  Google Scholar 

  • Sharples, M. (2000). The design of personal mobile technologies for lifelong learning. Computers & Education, 34(3), 177–193. https://doi.org/10.1016/S0360-1315(99)00044-5

    Article  Google Scholar 

  • Shekhar, A., & Marsden, N. (2018). Cognitive Walkthrough of a learning management system with gendered personas. In Proceedings of the 4th Conference on Gender & IT. https://doi.org/10.1145/3196839.3196869

    Article  Google Scholar 

  • Shilbayeh, S. A. R., & Ismail, S. A. E. R. (2020). Patient experience with an educational mobile health application: A pilot study on usability and feasibility in a Saudi population. Cogent Psychology, 7(1), 1843883. https://doi.org/10.1080/23311908.2020.1843883

    Article  Google Scholar 

  • Silius, K., Tervakari, A. M., & Pohjolainen, S. (2003). A multidisciplinary tool for the evaluation of usability, pedagogical usability, accessibility and informational quality of web-based courses. The Eleventh International PEG Conference: Powerful ICT for Teaching and Learning, 28, 1–10.

    Google Scholar 

  • Spitzer, R. L., Williams, J. B., & Endicott, J. (2012). Standards for DSM-5 reliability. American Journal of Psychiatry, 169(5), 537–537.

    Article  Google Scholar 

  • Squires, D., & Preece, J. (1996). Usability and learning: Evaluating the potential of educational software. Computers & Education, 27(1), 15–22. https://doi.org/10.1016/0360-1315(96)00010-3

    Article  Google Scholar 

  • Squires, D., & Preece, J. (1999). Predicting quality in educational software. Interacting with Computers, 11(5), 467–483.

    Article  Google Scholar 

  • Srikong, M., & Wannapiroon, P. (2020). Immersive technology for medical education: Technology enhance immersive learning experiences. Siriraj Medical Journal, 72(3), 265–271. https://doi.org/10.33192/Smj.2020.36

    Article  Google Scholar 

  • Stern, C., Jordan, Z., & McArthur, A. (2014). Developing the review question and inclusion criteria. AJN the American Journal of Nursing, 114(4), 53–56.

    Article  Google Scholar 

  • Stevens, D., & Kitchenham, A. (2011). An analysis of mobile learning in education, business, and medicine. In A. Kitchenham (Ed.), Models for interdisciplinary mobile learning: Delivering information to students (pp. 1–25). IGI Global.

    Google Scholar 

  • Storey, M. A., Phillips, B., Maczewski, M., & Wang, M. (2002). Evaluating the usability of Web-based learning tools. Journal of Educational Technology & Society, 5(3), 91–100.

    Google Scholar 

  • Sugar, W. (2014). Development and formative evaluation of multimedia case studies for Instructional Design and Technology students. TechTrends, 58(5), 36–52. https://doi.org/10.1007/s11528-014-0785-z

    Article  Google Scholar 

  • Tan, W., Liu, D., & Bishu, R. (2009). Web evaluation: Heuristic evaluation vs. user testing. International Journal of Industrial Ergonomics, 39(4), 621–627. https://doi.org/10.1016/j.ergon.2008.02.012

    Article  Google Scholar 

  • Tawfik, A. A., Gatewood, J., Gish-Lieberman, J. J., & Hampton, A. J. (2021). Toward a definition of learning experience design. Technology, Knowledge and Learning, 27(1), 309–334. https://doi.org/10.1007/s10758-020-09482-2

    Article  Google Scholar 

  • Tessmer. (1993). Planning and Conducting Formative Evaluations. Routledge. https://doi.org/10.4324/9780203061978

    Article  Google Scholar 

  • Tondello, G. F., Kappen, D. L., Ganaba, M., & Nacke, L. E. (2019). Gameful design heuristics: a gamification inspection tool. In M. Kurosu (Ed.), International Conference on Human-Computer Interaction (pp. 224–244). Springer.

    Google Scholar 

  • Turner, C. W., Lewis, J. R., & Nielsen, J. (2006). Determining usability test sample size. International Encyclopedia of Ergonomics and Human Factors, 3(2), 3084–3088.

    Google Scholar 

  • Van Der Linden, J., Amadieu, F., Vayre, E., & Van De Leemput, C. (2019). User experience and social influence: A new perspective for UX theory. In A. Marcus & W. Wang (Eds.), International Conference on Human-Computer Interaction (pp. 98–112). Springer.

    Google Scholar 

  • Walji, M. F., Kalenderian, E., Piotrowski, M., Tran, D., Kookal, K. K., Tokede, O., White, J. M., Vaderhobli, R., Ramoni, R., Stark, P. C., Kimmes, N. S., Lagerweij, M., & Patel, V. L. (2014). Are three methods better than one? A comparative assessment of usability evaluation methods in an EHR. International Journal of Medical Informatics, 83(5), 361–367. https://doi.org/10.1016/j.ijmedinf.2014.01.010

    Article  Google Scholar 

  • Wang, J. Y., Wu, H. K., & Hsu, Y. S. (2017). Using mobile applications for learning: Effects of simulation design, visual-motor integration, and spatial ability on high school students’ conceptual understanding. Computers in Human Behavior, 66, 103–113.

    Article  Google Scholar 

  • Weston, T. (2004). Formative evaluation for implementation: evaluating educational technology applications and lessons. American Journal of Evaluation, 25(1), 51–64. https://doi.org/10.1177/109821400402500104

    Article  Google Scholar 

  • Yáñez Gómez, R., Cascado Caballero, D., & Sevillano, J.-L. (2014). Heuristic evaluation on mobile interfaces: A New Checklist. The Scientific World Journal, 2014, 1–19. https://doi.org/10.1155/2014/434326

    Article  Google Scholar 

  • Yang, T., Linder, J., & Bolchini, D. (2012). DEEP: Design-oriented evaluation of perceived usability. International Journal of Human-Computer Interaction, 28(5), 308–346.

    Article  Google Scholar 

  • Yukselturk, E., Altıok, S., & Başer, Z. (2018). Using game-based learning with kinect technology in foreign language education course. Journal of Educational Technology & Society, 21(3), 159–173.

    Google Scholar 

  • Yun, S. J., Kang, M.-G., Yang, D., Choi, Y., Kim, H., Oh, B.-M., & Seo, H. G. (2020). Cognitive training using fully immersive, enriched environment virtual reality for patients with mild cognitive impairment and mild dementia: feasibility and usability study. JMIR Serious Games, 8(4), e18127. https://doi.org/10.2196/18127

    Article  Google Scholar 

  • Zabed Ahmed, S. M. (2008). A comparison of usability techniques for evaluating information retrieval system interfaces. Performance Measurement and Metrics, 9(1), 48–58. https://doi.org/10.1108/14678040810869422

    Article  Google Scholar 

  • Zaharias, P., & Koutsabasis, P. (2012). Heuristic evaluation of e-learning courses: A comparative analysis of two e-learning heuristic sets. Campus-Wide Information Systems, 29(1), 45–60. https://doi.org/10.1108/10650741211192046

    Article  Google Scholar 

  • Zali, Z., & Fadzlah, A. F. A. (2016). An initial theoretical usability evaluation model for assessing defence mobile e-based application system. International Conference on Information and Communication Technology (ICICTM), 2016, 198–202. https://doi.org/10.1109/ICICTM.2016.7890800

    Article  Google Scholar 

  • Zardari, B. A., Hussain, Z., Arain, A. A., Rizvi, W. H., & Vighio, M. S. (2021). QUEST e-learning portal: Applying heuristic evaluation, usability testing and eye tracking. Universal Access in the Information Society, 20(3), 531–543. https://doi.org/10.1007/s10209-020-00774-z

    Article  Google Scholar 

  • Zeng, J., Parks, S., & Shang, J. (2020). To learn scientifically, effectively, and enjoyably: A review of educational games. Human Behavior and Emerging Technologies, 2(2), 186–195. https://doi.org/10.1002/hbe2.188

    Article  Google Scholar 

  • Zhang, J., Johnson, T. R., Patel, V. L., Paige, D. L., & Kubose, T. (2003). Using usability heuristics to evaluate patient safety of medical devices. Journal of Biomedical Informatics, 36(1), 23–30. https://doi.org/10.1016/S1532-0464(03)00060-1

    Article  Google Scholar 

  • Zurita, G., Baloian, N., Peñafiel, S., & Jerez, O. (2019). Applying pedagogical usability for designing a mobile learning application that support reading comprehension. Multidisciplinary Digital Publishing Institute Proceedings, 31(1), 6. https://doi.org/10.3390/proceedings2019031006

    Article  Google Scholar 

Download references

Acknowledgements

We thank Dr. Noah Glaser for his guidance on the methodology used in this research study.

Funding

Research reported in this manuscript was not funded.

Author information

Authors and Affiliations

  1. University of Florida, 2423 Norman Hall, PO BOX 117048, Gainesville, FL, 32611, USA

    Jie Lu, Matthew Schmidt, Minyoung Lee & Rui Huang

Authors
  1. Jie Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthew Schmidt
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Minyoung Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rui Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthew Schmidt.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical approval

The research study described in this manuscript did not involve human subjects and/or animals other than the authors themselves.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A

See Table 6.

Table 6 Search terms and conditions applied in each database
Full size table

Appendix B

See Table 7.

Table 7 The pre-developed coding scheme with examples
Full size table

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, J., Schmidt, M., Lee, M. et al. Usability research in educational technology: a state-of-the-art systematic review. Education Tech Research Dev 70, 1951–1992 (2022). https://doi.org/10.1007/s11423-022-10152-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-022-10152-6

Keywords

Search

Navigation