Educational Technology Research and Development

, Volume 66, Issue 4, pp 1029–1049 | Cite as

Designing for 21st century learning online: a heuristic method to enable educator learning support roles

  • Denise Nacu
  • Caitlin K. Martin
  • Nichole Pinkard
Development Article


With the growing emphasis on developing 21st century skills among today’s youth, there is continued optimism about the possibilities granted by increasing access to networked technologies, particularly for encouraging youth to pursue their interests and take ownership of their learning. Yet, research demonstrates the importance of adult support in realizing the promise of achieving these outcomes. Designers of such systems are thus faced with the need to create youth-centered spaces that also provide adult facilitation of learning. This paper presents an adaptation of the traditional heuristic evaluation method which provides designers of online learning systems with a holistic view of how adult learning support is enabled across the system. We describe how the heuristic evaluation method was adapted, and through a case example analyzing one online social learning system used in a middle school context, we demonstrate how it can be used to help identify areas for improvement and promising areas for further research. We also present a framework of heuristics which reflect specific educator learning support roles that have been found to be important for youth learning, particularly for supporting 21st century skills. This work contributes a novel heuristic evaluation method that can help designers of online learning platforms attend not only the experiences of learners, but also to how educators are enabled to support their learning.


Heuristic evaluation Online teaching roles User interfaces 21st century learning Social learning networks 



This study was funded by National Science Foundation (Grant Number 1325004).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. Ahn, J. (2013, April). What can we learn from Facebook activity?: Using social learning analytics to observe new media literacy skills. In Proceedings of the third international conference on learning analytics and knowledge (pp. 135–144). Association for Computing Machinery.Google Scholar
  2. Allen, I. & Seaman, J. (2011). Going the distance: Online education in the US 2011. Sloan Consortium Report.Google Scholar
  3. Alsumait, A., & Al-Osaimi, A. (2009, December). Usability heuristics evaluation for child e-learning applications. In Proceedings of the 11th international conference on information integration and web-based applications & services (pp. 425–430). Association for Computing Machinery.Google Scholar
  4. Ardito, C., Costabile, M. F., De Marsico, M., Lanzilotti, R., Levialdi, S., Roselli, T., et al. (2006). An approach to usability evaluation of e-learning applications. Universal Access in the Information Society, 4(3), 270–283.CrossRefGoogle Scholar
  5. Attewell, P. (2001). Comment: The first and second digital divides. Sociology of Education, 74(3), 252–259.CrossRefGoogle Scholar
  6. Barron, B., Gomez, K., Pinkard, N., & Martin, C. K. (2014). Digital youth network: Cultivating digital media citizenship in urban communities. Cambridge, MA: MIT Press.Google Scholar
  7. Barron, B., Martin, C. K., Takeuchi, L., & Fithian, R. (2009). Parents as learning partners in the development of technological fluency. International Journal of Learning and Media, 1(2), 55–77.CrossRefGoogle Scholar
  8. Bell, P., Lewenstein, B., Shouse, A. W., & Feder, M. (Eds.). (2009). Learning science in informal environments: People, Places, and Pursuits. Washington, DC: National Academies Press.Google Scholar
  9. boyd, D., & Ellison, N. (2007). Social network sites: Definition, history, and scholarship. Journal of Computer-Mediated Communication. Scholar
  10. Brennan, K., Monroy-Hernández, A., & Resnick, M. (2010). Making projects, making friends: Online community as catalyst for interactive media creation. New Directions for Student Leadership, 2010(128), 75–83.Google Scholar
  11. Dringus, L. P., & Cohen, M. S. (2005). An adaptable usability heuristic checklist for online courses. In Frontiers in education, 2005. FIE’05. Proceedings 35th annual conference (pp. T2H-6–T2H-11). IEEE.Google Scholar
  12. Georgsson, M., Weir, C., & Staggers, N. (2014). Revisiting heuristic evaluation methods to improve the reliability of findings. MIE, 205, 930–934.Google Scholar
  13. Greenhow, C. (2011). Online social networks and learning. On the Horizon, 19(1), 4–12.CrossRefGoogle Scholar
  14. Greenhow, C., & Robelia, E. (2009). Old communication, new literacies: Social network sites as social learning resources. Journal of Computer-Mediated Communication, 14(4), 1130–1161.CrossRefGoogle Scholar
  15. Greenhow, C., Robelia, B., & Hughes, J. E. (2009). Learning, teaching, and scholarship in a digital age: Web 2.0 and classroom research: What path should we take now? Educational Researcher, 38(4), 246–259.CrossRefGoogle Scholar
  16. Hargittai, E., & Walejko, G. (2008). The participation divide: Content creation and sharing in the digital age. Information, Community and Society, 11(2), 239–256.CrossRefGoogle Scholar
  17. Ito, M., Baumer, S., Bittanti, M., et al. (2010). Hanging out, messing around, and geeking out: Kids living and learning with new media. Cambridge, MA: MIT Press.Google Scholar
  18. Jenkins, H., Purushotma, R., Weigel, M., Clinton, K., & Robison, A. (2009). Confronting the challenges of participatory culture: Media education for the 21st century. Cambridge: MIT Press.Google Scholar
  19. Lenhart, A. (2015). Teens, social media & technology overview 2015. Washington, DC: Pew Internet & American Life Project.Google Scholar
  20. Martin, C. K., Nacu, D., & Pinkard, N. (2016). Revealing opportunities for 21st century learning: An approach to interpreting user trace log data. Journal of Learning Analytics, 3(2), 37–87.CrossRefGoogle Scholar
  21. MMS Education. (2012). Survey of K-12 educators on social networking, online communities, and Web 2.0 tools. Report from MMS Education, Newtown, PA.Google Scholar
  22. Nacu, D., Martin, C. K., Pinkard, N., & Gray, T. (2014). Analyzing educators’ online interactions: A framework of online learning support roles. Learning, Media and Technology, 41(2), 283–305.CrossRefGoogle Scholar
  23. Nacu, D., Martin, C. K., & Pinkard, N. (2017). An interface design approach to encouraging online contributions among underrepresented youth. In Y. A. Rankin & J. O. Thomas (Eds.), Moving students of color from consumers to producers of technology (pp. 174–196). Hershey, PA: IGI Global. Scholar
  24. Nielsen, J. (1994). Heuristic evaluation. In Usability inspection methods (Vol. 17(1), pp. 25–62)Google Scholar
  25. Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. In CHI’90 Proceedings of the SIGCHI conference on human factors in computing systems (pp. 249–256). Seattle, WA: Association for Computing Machinery.Google Scholar
  26. Partnership for 21st Century Learning. (2016). Framework for 21st century learning. Retrieved from
  27. Reeves, T., Benson, L., Elliott, D., Grant, M., Holschuh, D., Kim, B., et al. (2002). Usability and instructional design heuristics for e-learning evaluation. In Proceedings of world conference on educational multimedia, hypermedia and telecommunications (pp. 1615–1621). Charlottesville, VA: AACE.Google Scholar
  28. Reich, J., & Ito, M. (2017). From good intentions to real outcomes: Equity by design in learning technologies. Irvine, CA: Digital Media and Learning Research Hub.Google Scholar
  29. Reich, J., Murnane, R., & Willett, J. (2012). The state of wiki usage in US K–12 schools: Leveraging Web 2.0 data warehouses to assess quality and equity in online learning environments. Educational Researcher, 41(1), 7–15.CrossRefGoogle Scholar
  30. Squires, D., & Preece, J. (1999). Predicting quality in educational software: Evaluating for learning, usability and the synergy between them. Interacting with Computers, 11(5), 467–483.CrossRefGoogle Scholar
  31. Staker, H., & Horn, M. B. (2012). Classifying K-12 blended learning. Watertown, MA: Innosight Institute.Google Scholar
  32. Wikipedia contributors. Learning management system. Wikipedia, The Free Encyclopedia. September 2, 2017, 22:06 UTC. Retrieved August 1, 2017, from
  33. Zaharias, P., & Poylymenakou, A. (2009). Developing a usability evaluation method for e-learning applications: Beyond functional usability. International Journal of Human-Computer Interaction, 25(1), 75–98.CrossRefGoogle Scholar

Copyright information

© Association for Educational Communications and Technology 2018

Authors and Affiliations

  1. 1.DePaul UniversityChicagoUSA
  2. 2.Digital Youth NetworkChicagoUSA
  3. 3.Northwestern UniversityEvanstonUSA

Personalised recommendations