With the growing emphasis on developing 21st century skills among today’s youth, there is continued optimism about the possibilities granted by increasing access to networked technologies, particularly for encouraging youth to pursue their interests and take ownership of their learning. Yet, research demonstrates the importance of adult support in realizing the promise of achieving these outcomes. Designers of such systems are thus faced with the need to create youth-centered spaces that also provide adult facilitation of learning. This paper presents an adaptation of the traditional heuristic evaluation method which provides designers of online learning systems with a holistic view of how adult learning support is enabled across the system. We describe how the heuristic evaluation method was adapted, and through a case example analyzing one online social learning system used in a middle school context, we demonstrate how it can be used to help identify areas for improvement and promising areas for further research. We also present a framework of heuristics which reflect specific educator learning support roles that have been found to be important for youth learning, particularly for supporting 21st century skills. This work contributes a novel heuristic evaluation method that can help designers of online learning platforms attend not only the experiences of learners, but also to how educators are enabled to support their learning.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Since we view this study as relevant to both formal and informal contexts, we use educator as general term to describe the adult(s) who interact with learners in a learning environment. Depending on the context, the educator may more specifically be referred to as the teacher, professor, mentor, etc.
Ahn, J. (2013, April). What can we learn from Facebook activity?: Using social learning analytics to observe new media literacy skills. In Proceedings of the third international conference on learning analytics and knowledge (pp. 135–144). Association for Computing Machinery.
Allen, I. & Seaman, J. (2011). Going the distance: Online education in the US 2011. Sloan Consortium Report.
Alsumait, A., & Al-Osaimi, A. (2009, December). Usability heuristics evaluation for child e-learning applications. In Proceedings of the 11th international conference on information integration and web-based applications & services (pp. 425–430). Association for Computing Machinery.
Ardito, C., Costabile, M. F., De Marsico, M., Lanzilotti, R., Levialdi, S., Roselli, T., et al. (2006). An approach to usability evaluation of e-learning applications. Universal Access in the Information Society, 4(3), 270–283.
Attewell, P. (2001). Comment: The first and second digital divides. Sociology of Education, 74(3), 252–259.
Barron, B., Gomez, K., Pinkard, N., & Martin, C. K. (2014). Digital youth network: Cultivating digital media citizenship in urban communities. Cambridge, MA: MIT Press.
Barron, B., Martin, C. K., Takeuchi, L., & Fithian, R. (2009). Parents as learning partners in the development of technological fluency. International Journal of Learning and Media, 1(2), 55–77.
Bell, P., Lewenstein, B., Shouse, A. W., & Feder, M. (Eds.). (2009). Learning science in informal environments: People, Places, and Pursuits. Washington, DC: National Academies Press.
boyd, D., & Ellison, N. (2007). Social network sites: Definition, history, and scholarship. Journal of Computer-Mediated Communication. https://doi.org/10.1111/j.1083-6101.2007.00393.x.
Brennan, K., Monroy-Hernández, A., & Resnick, M. (2010). Making projects, making friends: Online community as catalyst for interactive media creation. New Directions for Student Leadership, 2010(128), 75–83.
Dringus, L. P., & Cohen, M. S. (2005). An adaptable usability heuristic checklist for online courses. In Frontiers in education, 2005. FIE’05. Proceedings 35th annual conference (pp. T2H-6–T2H-11). IEEE.
Georgsson, M., Weir, C., & Staggers, N. (2014). Revisiting heuristic evaluation methods to improve the reliability of findings. MIE, 205, 930–934.
Greenhow, C. (2011). Online social networks and learning. On the Horizon, 19(1), 4–12.
Greenhow, C., & Robelia, E. (2009). Old communication, new literacies: Social network sites as social learning resources. Journal of Computer-Mediated Communication, 14(4), 1130–1161.
Greenhow, C., Robelia, B., & Hughes, J. E. (2009). Learning, teaching, and scholarship in a digital age: Web 2.0 and classroom research: What path should we take now? Educational Researcher, 38(4), 246–259.
Hargittai, E., & Walejko, G. (2008). The participation divide: Content creation and sharing in the digital age. Information, Community and Society, 11(2), 239–256.
Ito, M., Baumer, S., Bittanti, M., et al. (2010). Hanging out, messing around, and geeking out: Kids living and learning with new media. Cambridge, MA: MIT Press.
Jenkins, H., Purushotma, R., Weigel, M., Clinton, K., & Robison, A. (2009). Confronting the challenges of participatory culture: Media education for the 21st century. Cambridge: MIT Press.
Lenhart, A. (2015). Teens, social media & technology overview 2015. Washington, DC: Pew Internet & American Life Project.
Martin, C. K., Nacu, D., & Pinkard, N. (2016). Revealing opportunities for 21st century learning: An approach to interpreting user trace log data. Journal of Learning Analytics, 3(2), 37–87.
MMS Education. (2012). Survey of K-12 educators on social networking, online communities, and Web 2.0 tools. Report from MMS Education, Newtown, PA.
Nacu, D., Martin, C. K., Pinkard, N., & Gray, T. (2014). Analyzing educators’ online interactions: A framework of online learning support roles. Learning, Media and Technology, 41(2), 283–305.
Nacu, D., Martin, C. K., & Pinkard, N. (2017). An interface design approach to encouraging online contributions among underrepresented youth. In Y. A. Rankin & J. O. Thomas (Eds.), Moving students of color from consumers to producers of technology (pp. 174–196). Hershey, PA: IGI Global. https://doi.org/10.4018/978-1-5225-2005-4.ch009.
Nielsen, J. (1994). Heuristic evaluation. In Usability inspection methods (Vol. 17(1), pp. 25–62)
Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. In CHI’90 Proceedings of the SIGCHI conference on human factors in computing systems (pp. 249–256). Seattle, WA: Association for Computing Machinery.
Partnership for 21st Century Learning. (2016). Framework for 21st century learning. Retrieved from http://www.p21.org/storage/documents/docs/P21_framework_0816.pdf
Reeves, T., Benson, L., Elliott, D., Grant, M., Holschuh, D., Kim, B., et al. (2002). Usability and instructional design heuristics for e-learning evaluation. In Proceedings of world conference on educational multimedia, hypermedia and telecommunications (pp. 1615–1621). Charlottesville, VA: AACE.
Reich, J., & Ito, M. (2017). From good intentions to real outcomes: Equity by design in learning technologies. Irvine, CA: Digital Media and Learning Research Hub.
Reich, J., Murnane, R., & Willett, J. (2012). The state of wiki usage in US K–12 schools: Leveraging Web 2.0 data warehouses to assess quality and equity in online learning environments. Educational Researcher, 41(1), 7–15.
Squires, D., & Preece, J. (1999). Predicting quality in educational software: Evaluating for learning, usability and the synergy between them. Interacting with Computers, 11(5), 467–483.
Staker, H., & Horn, M. B. (2012). Classifying K-12 blended learning. Watertown, MA: Innosight Institute.
Wikipedia contributors. Learning management system. Wikipedia, The Free Encyclopedia. September 2, 2017, 22:06 UTC. Retrieved August 1, 2017, from https://en.wikipedia.org/w/index.php?title=Learning_management_system&oldid=798608757.
Zaharias, P., & Poylymenakou, A. (2009). Developing a usability evaluation method for e-learning applications: Beyond functional usability. International Journal of Human-Computer Interaction, 25(1), 75–98.
This study was funded by National Science Foundation (Grant Number 1325004).
Conflict of interest
The authors declare that they have no conflict of interest.
About this article
Cite this article
Nacu, D., Martin, C.K. & Pinkard, N. Designing for 21st century learning online: a heuristic method to enable educator learning support roles. Education Tech Research Dev 66, 1029–1049 (2018). https://doi.org/10.1007/s11423-018-9603-0
- Heuristic evaluation
- Online teaching roles
- User interfaces
- 21st century learning
- Social learning networks