Skip to main content
SpringerLink
Log in

Discovering Generative Uncertainty in Learning Analytics Dashboards

  • Chapter
  • First Online:
Visualizations and Dashboards for Learning Analytics

Part of the book series: Advances in Analytics for Learning and Teaching ((AALT))

Abstract

The proliferation of learning dashboards in K–12 education calls for deeper knowledge of how such tools fit into existing data use routines in schools. An exemplar routine is coaching cycles, where teachers and experienced coaches collaborate on using data to improve student learning. Within this context, our mixed-method study draws from interviews and think-aloud sessions about dashboard visualizations, conducted with teachers and instructional coaches from four school districts in the United States. Our analyses illuminate how different professional roles express varied patterns of response when facing LA dashboards. The analyses uncover a particular pattern of asking questions to resolve uncertainty that leads to further reflection and action. We discuss how uncertainty towards data and visualizations can be productive for teacher learning and the implications of designing for uncertainty in dashboards. Mapping LA dashboards to educators’ daily routines is important to promote the uptake of analytics towards improving instructional practices.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  • Bertrand, M., & Marsh, J. A. (2015). Teachers’ sensemaking of data and implications for equity. American Educational Research Journal, 52(5), 861–893.

    Article  Google Scholar 

  • Bocala, C., & Boudett, K. P. (2015). Teaching educators habits of mind for using data wisely. Teachers College Record, 117(4), n4.

    Article  Google Scholar 

  • Borko, H., Livingston, C., & Shavelson, R. J. (1990). Teachers’ thinking about instruction. Remedial and Special Education, 11(6), 40–49.

    Article  Google Scholar 

  • Buckingham Shum, S., Ferguson, R., & Martinez-Maldonaldo, R. (2019). Human-Centred learning analytics. Journal of Learning Analytics, 6(2), 1–9. https://doi.org/10.18608/jla.2019.62.1.

    Article  Google Scholar 

  • Campos, F., Ahn, J., Digiacomo, D., Nguyen, H., & Hays, M. (in press). One chart, many meanings: Making sense of sensemaking in learning analytics dashboard design. Journal of Learning Analytics.

    Google Scholar 

  • Charters, E. (2003). The use of think-aloud methods in qualitative research: An Introduction to think-aloud methods. Brock Education: A Journal of Educational Research and Practice, 12(2), 68–82.

    Google Scholar 

  • Cobb, P., Jackson, K., Henrick, E., & Smith, T. M. (2020). Systems for instructional improvement: Creating coherence from the classroom to the district office. Harvard Education Press.

    Google Scholar 

  • Dervin, B. (2005). Sense-making theory and practice: An overview of user interests in knowledge seeking and use. Journal of Knowledge Management.

    Google Scholar 

  • Dewey, J. (1910). What is thought. How We Think, 1–13.

    Google Scholar 

  • Duval, E. (2011, February). Attention please! Learning analytics for visualization and recommendation. In Proceedings of the 1st international conference on learning analytics and knowledge (pp. 9–17).

    Chapter  Google Scholar 

  • Echeverria, V., Martinez-Maldonado, R., Shum, S. B., Chiluiza, K., Granda, R., & Conati, C. (2018). Exploratory versus explanatory visual learning analytics: Driving teachers’ attention through educational data storytelling. Journal of Learning Analytics, 5(3), 72–97.

    Article  Google Scholar 

  • Epp, C. D., & Bull, S. (2015). Uncertainty representation in visualizations of learning analytics for learners: Current approaches and opportunities. IEEE Transactions on Learning Technologies, 8(3), 242–260.

    Article  Google Scholar 

  • Few, S. (2013). Information dashboard design: Displaying data for at-a-glance monitoring (Vol. 5). Burlingame: Analytics Press.

    Google Scholar 

  • Floden, R. E., & Clark, C. M. (1987). Preparing teachers for uncertainty.

    Google Scholar 

  • Fonteyn, M. E., Kuipers, B., & Grobe, S. J. (1993). A description of Think aloud method and protocol analysis. Qualitative Health Research, 3(4), 430–441.

    Article  Google Scholar 

  • Garner, B., Thorne, J. K., & Horn, I. S. (2017). Teachers interpreting data for instructional decisions: Where does equity come in? Journal of Educational Administration.

    Google Scholar 

  • Grammel, L., Tory, M., & Storey, M. A. (2010). How information visualization novices construct visualizations. IEEE Transactions on Visualization and Computer Graphics, 16(6), 943–952.

    Article  Google Scholar 

  • Greller, W., & Drachsler, H. (2012). Translating learning into numbers: A generic framework for learning analytics. Journal of Educational Technology & Society, 15(3), 42–57.

    Google Scholar 

  • Helsing, D. (2007). Regarding uncertainty in teachers and teaching. Teaching and Teacher Education, 23(8), 1317–1333.

    Article  Google Scholar 

  • Horn, I. S., Kane, B. D., & Wilson, J. (2015). Making sense of student performance data: Data use logics and mathematics teachers’ learning opportunities. American Educational Research Journal, 52(2), 208–242.

    Article  Google Scholar 

  • Jackson, K., Henrick, E., Cobb, P., Kochmanski, N., & Nieman, H. (2016). Practical measures to improve the quality of small-group and whole-class discussion [white paper]. Retrieved January, 1, 2019.

    Google Scholar 

  • Johnson, C. R., & Sanderson, A. R. (2003). A next step: Visualizing errors and uncertainty. IEEE Computer Graphics and Applications, 23(5), 6–10.

    Article  Google Scholar 

  • Klein, G., Phillips, J. K., Rall, E. L., & Peluso, D. A. (2007, January). A data-frame theory of sensemaking. In Expertise out of context: Proceedings of the sixth international conference on naturalistic decision making (pp. 113–155). New York, NY: Lawrence Erlbaum Assoc.

    Google Scholar 

  • Knight, J. (2011). What good coaches do. Educational Leadership, 69(2), 18–22.

    Google Scholar 

  • Knight, S., Buckingham Shum, S., & Littleton, K. (2013). Collaborative sensemaking in learning analytics.

    Google Scholar 

  • Lee, S., Kim, S. H., Hung, Y. H., Lam, H., Kang, Y. A., & Yi, J. S. (2015). How do people make sense of unfamiliar visualizations?: A grounded model of novice’s information visualization sensemaking. IEEE Transactions on Visualization and Computer Graphics, 22(1), 499–508.

    Article  Google Scholar 

  • Liu, Z., & Stasko, J. (2010). Mental models, visual reasoning and interaction in information visualization: A top-down perspective. IEEE Transactions on Visualization and Computer Graphics, 16(6), 999–1008.

    Article  Google Scholar 

  • Marsh, J. A., Sloan McCombs, J., & Martorell, F. (2010). How instructional coaches support data-driven decision making: Policy implementation and effects in Florida middle schools. Educational Policy, 24(6), 872–907.

    Article  Google Scholar 

  • Martinez-Maldonado, R., Echeverria, V., Fernandez Nieto, G., & Buckingham Shum, S. (2020, April). From data to insights: A layered storytelling approach for multimodal learning analytics. In Proceedings of the 2020 chi conference on human factors in computing systems (pp. 1–15).

    Google Scholar 

  • McCoy, C., & Shih, P. (2016). Teachers as producers of data analytics: A case study of a teacher-focused educational data science program. Journal of Learning Analytics, 3(3), 193–214.

    Article  Google Scholar 

  • Means, B., Chen, E., DeBarger, A., & Padilla, C. (2011). Teachers’ ability to use data to inform instruction: Challenges and supports. Office of Planning, Evaluation and Policy Development, US Department of Education.

    Google Scholar 

  • Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Sage.

    Google Scholar 

  • Munthe, E. (2003). Teachers’ workplace and professional certainty. Teaching and Teacher Education, 19(8), 801–813.

    Article  Google Scholar 

  • Murnane, R. J., Sharkey, N. S., & Boudett, K. P. (2005). Using student-assessment results to improve instruction: Lessons from a workshop. Journal of Education for Students Placed at Risk, 10(3), 269–280.

    Article  Google Scholar 

  • Nelson, T., & Slavit, D. (2008). Supported teacher collaborative inquiry. Teacher Education Quarterly, 35(1), 99–116.

    Google Scholar 

  • Potter, K., Rosen, P., & Johnson, C. R. (2011, August). From quantification to visualization: A taxonomy of uncertainty visualization approaches. In IFIP Working Conference on Uncertainty Quantification (pp. 226–249). Berlin/Heidelberg: Springer.

    Google Scholar 

  • Romat, H., Henry Riche, N., Hinckley, K., Lee, B., Appert, C., Pietriga, E., & Collins, C. (2019, May). ActiveInk: (Th)Inking with data. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1–13).

    Google Scholar 

  • Santagata, R. (2011). From teacher noticing to a framework for analyzing and improving classroom lessons. Mathematics teacher noticing: Seeing through teachers’ eyes, 152, 168.

    Google Scholar 

  • Schildkamp, K. (2019). Data-based decision-making for school improvement: Research insights and gaps. Educational Research, 61(3), 257–273.

    Article  Google Scholar 

  • Schön, D. A. (1987). Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. Jossey-Bass.

    Google Scholar 

  • Schwendimann, B. A., Rodriguez-Triana, M. J., Vozniuk, A., Prieto, L. P., Boroujeni, M. S., Holzer, A., … Dillenbourg, P. (2016). Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies, 10(1), 30–41.

    Article  Google Scholar 

  • Sherin, M. G., & Russ, R. S. (2014). The role of interpretive frames. In Digital video for teacher education (pp. 11–28). Routledge.

    Google Scholar 

  • Srinivasan, A., Brehmer, M., Lee, B., & Drucker, S. M. (2018, April). What’s the difference? Evaluating variations of multi-series Bar charts for visual comparison tasks. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (pp. 1-12).

    Google Scholar 

  • Thomas, J. J., & Cook, K. A. (2006). A visual analytics agenda. IEEE Computer Graphics and Applications, 26(1), 10–13.

    Article  Google Scholar 

  • Thomson, J., Hetzler, E., MacEachren, A., Gahegan, M., & Pavel, M. (2005, March). A typology for visualizing uncertainty. In Visualization and Data Analysis 2005 (Vol. 5669, pp. 146-157). International society for optics and photonics.

    Google Scholar 

  • van Es, E. A., & Sherin, M. G. (2002). Learning to notice: Scaffolding new teachers’ interpretations of classroom interactions. Journal of Technology and Teacher Education, 10(4), 571–596.

    Google Scholar 

  • van Es, E. A., & Sherin, M. G. (2006). How different video club designs support teachers in “learning to notice”. Journal of Computing in Teacher Education, 22(4), 125–135.

    Google Scholar 

  • van Es, E. A., & Sherin, M. G. (2008). Mathematics teachers’“learning to notice” in the context of a video club. Teaching and Teacher Education, 24(2), 244–276.

    Article  Google Scholar 

  • van Leeuwen, A., & Rummel, N. (2020, March). Comparing teachers’ use of mirroring and advising dashboards. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge (pp. 26-34).

    Google Scholar 

  • Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57(10), 1500–1509.

    Article  Google Scholar 

  • Verbert, K., Ochoa, X., De Croon, R., Dourado, R. A., & De Laet, T. (2020, March). Learning analytics dashboards: The past, the present and the future. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge (pp. 35-40).

    Google Scholar 

  • Voyiatzaki, E., & Avouris, N. (2014). Support for the teacher in technology-enhanced collaborative classroom. Education and Information Technologies, 19(1), 129–154.

    Article  Google Scholar 

  • Wardrip, P. S., & Shapiro, R. B. (2016). Digital media and data: Using and designing technologies to support learning in practice. Learning, Media and Technology, 41(2), 187–192.

    Article  Google Scholar 

  • Wise, A. F., & Jung, Y. (2019). Teaching with analytics: Towards a situated model of instructional decision-making. Journal of Learning Analytics, 6(2), 53–69.

    Article  Google Scholar 

  • Wise, A. F., Sarmiento, J. P., & Boothe, M., Jr. (2021, April). Subversive learning analytics. In LAK21: 11th International Learning Analytics and Knowledge Conference (pp. 639–645).

    Chapter  Google Scholar 

  • Xhakaj, F., Aleven, V., & McLaren, B. M. (2017, September). Effects of a teacher dashboard for an intelligent tutoring system on teacher knowledge, lesson planning, lessons and student learning. In European conference on technology enhanced learning (pp. 315–329). Cham: Springer.

    Google Scholar 

  • Yeager, D., Bryk, A., Muhich, J., Hausman, H., & Morales, L. (2013). Practical measurement (p. 78712). Palo Alto, CA: Carnegie Foundation for the Advancement of Teaching.

    Google Scholar 

  • Yi, J. S., ah Kang, Y., Stasko, J., & Jacko, J. A. (2007). Toward a deeper understanding of the role of interaction in information visualization. IEEE Transactions on Visualization and Computer Graphics, 13(6), 1224–1231.

    Article  Google Scholar 

Download references

Acknowledgments

This material is based upon work supported by the National Science Foundation under Grants No. 1719744, 1620851, 1621238, and 1620863. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the view of the National Science Foundation. We thank all teachers, instructional coaches, and district leaders who participated in this study.

Author information

Authors and Affiliations

  1. University of California – Irvine, Irvine, CA, USA

    Ha Nguyen & June Ahn

  2. New York University, New York City, NY, USA

    Fabio Campos

Authors
  1. Ha Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fabio Campos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. June Ahn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ha Nguyen .

Editor information

Editors and Affiliations

  1. University of Mannheim, Mannheim, Germany

    Muhittin Sahin  & Dirk Ifenthaler  & 

  2. Curtin University, Perth, WA, Australia

    Dirk Ifenthaler

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Nguyen, H., Campos, F., Ahn, J. (2021). Discovering Generative Uncertainty in Learning Analytics Dashboards. In: Sahin, M., Ifenthaler, D. (eds) Visualizations and Dashboards for Learning Analytics. Advances in Analytics for Learning and Teaching. Springer, Cham. https://doi.org/10.1007/978-3-030-81222-5_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-81222-5_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-81221-8

  • Online ISBN: 978-3-030-81222-5

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation