Designing for 21st century learning online: a heuristic method to enable educator learning support roles


With the growing emphasis on developing 21st century skills among today’s youth, there is continued optimism about the possibilities granted by increasing access to networked technologies, particularly for encouraging youth to pursue their interests and take ownership of their learning. Yet, research demonstrates the importance of adult support in realizing the promise of achieving these outcomes. Designers of such systems are thus faced with the need to create youth-centered spaces that also provide adult facilitation of learning. This paper presents an adaptation of the traditional heuristic evaluation method which provides designers of online learning systems with a holistic view of how adult learning support is enabled across the system. We describe how the heuristic evaluation method was adapted, and through a case example analyzing one online social learning system used in a middle school context, we demonstrate how it can be used to help identify areas for improvement and promising areas for further research. We also present a framework of heuristics which reflect specific educator learning support roles that have been found to be important for youth learning, particularly for supporting 21st century skills. This work contributes a novel heuristic evaluation method that can help designers of online learning platforms attend not only the experiences of learners, but also to how educators are enabled to support their learning.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3


  1. 1.

    Since we view this study as relevant to both formal and informal contexts, we use educator as general term to describe the adult(s) who interact with learners in a learning environment. Depending on the context, the educator may more specifically be referred to as the teacher, professor, mentor, etc.


  1. Ahn, J. (2013, April). What can we learn from Facebook activity?: Using social learning analytics to observe new media literacy skills. In Proceedings of the third international conference on learning analytics and knowledge (pp. 135–144). Association for Computing Machinery.

  2. Allen, I. & Seaman, J. (2011). Going the distance: Online education in the US 2011. Sloan Consortium Report.

  3. Alsumait, A., & Al-Osaimi, A. (2009, December). Usability heuristics evaluation for child e-learning applications. In Proceedings of the 11th international conference on information integration and web-based applications & services (pp. 425–430). Association for Computing Machinery.

  4. Ardito, C., Costabile, M. F., De Marsico, M., Lanzilotti, R., Levialdi, S., Roselli, T., et al. (2006). An approach to usability evaluation of e-learning applications. Universal Access in the Information Society, 4(3), 270–283.

    Article  Google Scholar 

  5. Attewell, P. (2001). Comment: The first and second digital divides. Sociology of Education, 74(3), 252–259.

    Article  Google Scholar 

  6. Barron, B., Gomez, K., Pinkard, N., & Martin, C. K. (2014). Digital youth network: Cultivating digital media citizenship in urban communities. Cambridge, MA: MIT Press.

    Google Scholar 

  7. Barron, B., Martin, C. K., Takeuchi, L., & Fithian, R. (2009). Parents as learning partners in the development of technological fluency. International Journal of Learning and Media, 1(2), 55–77.

    Article  Google Scholar 

  8. Bell, P., Lewenstein, B., Shouse, A. W., & Feder, M. (Eds.). (2009). Learning science in informal environments: People, Places, and Pursuits. Washington, DC: National Academies Press.

    Google Scholar 

  9. boyd, D., & Ellison, N. (2007). Social network sites: Definition, history, and scholarship. Journal of Computer-Mediated Communication.

    Google Scholar 

  10. Brennan, K., Monroy-Hernández, A., & Resnick, M. (2010). Making projects, making friends: Online community as catalyst for interactive media creation. New Directions for Student Leadership, 2010(128), 75–83.

    Google Scholar 

  11. Dringus, L. P., & Cohen, M. S. (2005). An adaptable usability heuristic checklist for online courses. In Frontiers in education, 2005. FIE’05. Proceedings 35th annual conference (pp. T2H-6–T2H-11). IEEE.

  12. Georgsson, M., Weir, C., & Staggers, N. (2014). Revisiting heuristic evaluation methods to improve the reliability of findings. MIE, 205, 930–934.

    Google Scholar 

  13. Greenhow, C. (2011). Online social networks and learning. On the Horizon, 19(1), 4–12.

    Article  Google Scholar 

  14. Greenhow, C., & Robelia, E. (2009). Old communication, new literacies: Social network sites as social learning resources. Journal of Computer-Mediated Communication, 14(4), 1130–1161.

    Article  Google Scholar 

  15. Greenhow, C., Robelia, B., & Hughes, J. E. (2009). Learning, teaching, and scholarship in a digital age: Web 2.0 and classroom research: What path should we take now? Educational Researcher, 38(4), 246–259.

    Article  Google Scholar 

  16. Hargittai, E., & Walejko, G. (2008). The participation divide: Content creation and sharing in the digital age. Information, Community and Society, 11(2), 239–256.

    Article  Google Scholar 

  17. Ito, M., Baumer, S., Bittanti, M., et al. (2010). Hanging out, messing around, and geeking out: Kids living and learning with new media. Cambridge, MA: MIT Press.

    Google Scholar 

  18. Jenkins, H., Purushotma, R., Weigel, M., Clinton, K., & Robison, A. (2009). Confronting the challenges of participatory culture: Media education for the 21st century. Cambridge: MIT Press.

    Google Scholar 

  19. Lenhart, A. (2015). Teens, social media & technology overview 2015. Washington, DC: Pew Internet & American Life Project.

    Google Scholar 

  20. Martin, C. K., Nacu, D., & Pinkard, N. (2016). Revealing opportunities for 21st century learning: An approach to interpreting user trace log data. Journal of Learning Analytics, 3(2), 37–87.

    Article  Google Scholar 

  21. MMS Education. (2012). Survey of K-12 educators on social networking, online communities, and Web 2.0 tools. Report from MMS Education, Newtown, PA.

  22. Nacu, D., Martin, C. K., Pinkard, N., & Gray, T. (2014). Analyzing educators’ online interactions: A framework of online learning support roles. Learning, Media and Technology, 41(2), 283–305.

    Article  Google Scholar 

  23. Nacu, D., Martin, C. K., & Pinkard, N. (2017). An interface design approach to encouraging online contributions among underrepresented youth. In Y. A. Rankin & J. O. Thomas (Eds.), Moving students of color from consumers to producers of technology (pp. 174–196). Hershey, PA: IGI Global.

    Google Scholar 

  24. Nielsen, J. (1994). Heuristic evaluation. In Usability inspection methods (Vol. 17(1), pp. 25–62)

  25. Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. In CHI’90 Proceedings of the SIGCHI conference on human factors in computing systems (pp. 249–256). Seattle, WA: Association for Computing Machinery.

  26. Partnership for 21st Century Learning. (2016). Framework for 21st century learning. Retrieved from

  27. Reeves, T., Benson, L., Elliott, D., Grant, M., Holschuh, D., Kim, B., et al. (2002). Usability and instructional design heuristics for e-learning evaluation. In Proceedings of world conference on educational multimedia, hypermedia and telecommunications (pp. 1615–1621). Charlottesville, VA: AACE.

  28. Reich, J., & Ito, M. (2017). From good intentions to real outcomes: Equity by design in learning technologies. Irvine, CA: Digital Media and Learning Research Hub.

    Google Scholar 

  29. Reich, J., Murnane, R., & Willett, J. (2012). The state of wiki usage in US K–12 schools: Leveraging Web 2.0 data warehouses to assess quality and equity in online learning environments. Educational Researcher, 41(1), 7–15.

    Article  Google Scholar 

  30. Squires, D., & Preece, J. (1999). Predicting quality in educational software: Evaluating for learning, usability and the synergy between them. Interacting with Computers, 11(5), 467–483.

    Article  Google Scholar 

  31. Staker, H., & Horn, M. B. (2012). Classifying K-12 blended learning. Watertown, MA: Innosight Institute.

    Google Scholar 

  32. Wikipedia contributors. Learning management system. Wikipedia, The Free Encyclopedia. September 2, 2017, 22:06 UTC. Retrieved August 1, 2017, from

  33. Zaharias, P., & Poylymenakou, A. (2009). Developing a usability evaluation method for e-learning applications: Beyond functional usability. International Journal of Human-Computer Interaction, 25(1), 75–98.

    Article  Google Scholar 

Download references


This study was funded by National Science Foundation (Grant Number 1325004).

Author information



Corresponding author

Correspondence to Denise Nacu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Nacu, D., Martin, C.K. & Pinkard, N. Designing for 21st century learning online: a heuristic method to enable educator learning support roles. Education Tech Research Dev 66, 1029–1049 (2018).

Download citation


  • Heuristic evaluation
  • Online teaching roles
  • User interfaces
  • 21st century learning
  • Social learning networks