Skip to main content
SpringerLink
Log in

Evaluating the Usability of a Professional Modeling Tool Repurposed for Middle School Learning

  • Published:
Journal of Science Education and Technology Aims and scope Submit manuscript

Abstract

This paper reports the results of a three-stage usability test of a modeling tool designed to support learners’ deep understanding of the impacts of climate change on ecosystems. The design process involved repurposing an existing modeling technology used by professional scientists into a learning tool specifically designed for middle school students. To evaluate usability, we analyzed students’ task performance and task completion time as they worked on an activity with the repurposed modeling technology. In stage 1, we conducted remote testing of an early modeling prototype with urban middle school students (n = 84). In stages 2 and 3, we used screencasting software to record students’ mouse and keyboard movements during collaborative think-alouds (n = 22) and conducted a qualitative analysis of their peer discussions. Taken together, the study findings revealed two kinds of usability issues that interfered with students’ productive use of the tool: issues related to the use of data and information, and issues related to the use of the modeling technology. The study findings resulted in design improvements that led to stronger usability outcomes and higher task performance among students. In this paper, we describe our methods for usability testing, our research findings, and our design solutions for supporting students’ use of the modeling technology and use of data. The paper concludes with implications for the design and study of modeling technologies for science learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Alibrandi M (2003) GIS in the classroom: using geographic information systems in social studies and environmental science. Heinemann, Portsmouth

    Google Scholar 

  • Baker TR (2005) Internet-based GIS mapping in support of K-12 education. Prof Geogr 57(1):44–50

    Google Scholar 

  • Baker TR, White SH (2003) The effects of G.I.S. on students’ attitudes, self-efficacy, and achievement middle school science classrooms. J Geogr 102(6):243–254

    Article  Google Scholar 

  • Bausmith JM, Leinhardt G (1998) Middle-school students’ map construction: understanding complex spatial displays. J Geogr 97(3):93–107

    Article  Google Scholar 

  • Beach JH, Stewart AM, Vorontsov, GY (2002) Mapping distributed life with distributed computation. In: Proceedings of the 22nd annual Esri international user conference. Redlands, CA: Esri Publications

  • Benyon D (1993) Adaptive systems: a solution to usability problems. User Model User Adap Inter 3(1):65–87

    Article  Google Scholar 

  • Board College (2009) Science: college board standards for college success. The College Board Press, New York

    Google Scholar 

  • Bury KF (1984) The iterative development of usable computer interfaces. In Proceedings of IFIP INTERACT’84 international conference on human–computer interaction, pp 743–748. London, UK, Sept 4–7

  • Ceaparu I, Lazar J, Bessiere K, Robinson J, Schneiderman B (2004) Determining causes and severity of end-user frustration. Int J Hum Comput Interact 17(3):333–356

    Article  Google Scholar 

  • Coll R, France B, Taylor I (2005) The role of models/and analogies in science education: implications from research. Int J Sci Edu 27(2):183–198

    Article  Google Scholar 

  • Collins A (2011) A study of expert theory formation: the role of different model types and domain frameworks. In: Khine MS, Saleh IM (eds) Models and modeling: cognitive tools for scientific enquiry. Springer, New York, pp 23–40

    Google Scholar 

  • Crowther MS, Keller CC, Waddoups GL (2004) Improving the quality and effectiveness of computer-mediated instruction through usability evaluations. British J Edu Technol 35(3):289–303

    Article  Google Scholar 

  • Csikszentmihalyi M (1997) Finding flow: the psychology of engagement with everyday life. Basic Books, New York

    Google Scholar 

  • Edelson DC (2004). Designing GIS software for education: a workshop report for the GIS community. The Geographic Data in Education Initiative at Northwestern University

  • Edelson DC, Gordin D (1998) Visualization for learning: a framework for adapting scientist’s tools. Comput Geosci 24(7):607–616

    Article  Google Scholar 

  • Gobert JD, Pallant A (2004) Fostering students’ epistemologies of models via authentic model-based tasks. J Sci Educ Technol 13(1):7–22

    Article  Google Scholar 

  • Hewitt J, Brett C (2011) Engaging learners in the identification of key ideas in complex online discussions. In Spada H, Stahl G, Miyake N, Law N (eds) In: Proceedings of the computer-supported collaborative learning conference, pp 960–961, Hong Kong, July 4–8

  • IPCC (2007) Climate change 2007: synthesis report. In: Core writing team, Pachauri RK, Reisinger A (eds) Contribution of working groups I, II and III to the fourth assessment report of the intergovernmental panel on climate change. IPCC, Geneva, Switzerland

  • Johnson RR, Salvo MJ, Zoetewey MW (2007) User-centered technology in participatory culture: two decades “Beyond a narrow conception of usability testing”. Proc IEEE Transact Prof Commun 50(4):320–332

    Article  Google Scholar 

  • Kaplan DE, Black JB (2003) Mental models and computer-based scientific inquiry learning: effects of mechanism cues on adolescent representation and reasoning about causal systems. J Sci Educ Technol 12(4):483–493

    Article  Google Scholar 

  • Keating T, Barnett MG, Barab SA, Hay KE (2002) The virtual solar system project: developing conceptual understanding of astronomical concepts through building three-dimensional computational models. J Sci Educ Technol 11(3):261–275

    Article  Google Scholar 

  • Landsberger HenryA (1958) Hawthorne revisited: management and the worker: its critics, and developments in human relations in industry. Cornell University Press, Ithaca

    Google Scholar 

  • Lazar J, Jones A, Shneiderman B (2006) Workplace user frustration with computers: an exploratory investigation of the causes and severity. Behav Inf Technol 25(3):239–251

    Article  Google Scholar 

  • Lehrer R, Schauble L (2006) Scientific thinking and science literacy. In: Damon W, Lerner R, Renninger KA, Sigel IE (eds) Handbook of child psychology, vol 4, 6th edn. Wiley, Hoboken

    Google Scholar 

  • Manouselis N, Drachsler H, Vuorikari R, Hummel H, Koper, K (2011) Recommender sytems in technology enhanced learning. In: Ricci F, Rokach L, Shapira B, Kantor PB (eds) Recommener systems handbook. Springer, New York, pp 387–415

  • Michigan Department of Education. K-7 Content grade level expectations in science, version 1.09. Retrieved from: http://michigan.gov/mde/

  • Miles MB, Huberman AM (1994) Qualitative data analysis: an expanded sourcebook, 2nd edn. Sage Publications, Thousand Oaks

    Google Scholar 

  • Miyake N (1986) Constructive interaction and the iterative process of understanding. Cognitive Science 10:151–177

    Article  Google Scholar 

  • National Research Council (2012) A framework for K-12 science education: practices, crosscutting concepts, and core ideas. National Academies Press, Washington

    Google Scholar 

  • Nelson BD, Aron RH, Francek MA (1992) Clarification of selected misconceptions in physical geography. J Geogr 91(2):76–80

    Article  Google Scholar 

  • Nielsen J (1993) Iterative user-interface design. IEEE Comput 26(11):32–41

    Article  Google Scholar 

  • Nielsen J (2003) Usability 101: an introduction to usability. Jakob Nielsen’s Alertbox. Base article retrieved 2 Nov 2010, from: http://www.useit.com/alertbox/20030825.html

  • Nielsen J (2010). Children’s websites: usability issues in designing for kids. Jakob Nielsen’s Alertbox. Article retrieved 2 Nov 2010, from: http://www.useit.com/alertbox/children.html

  • Nielsen J, Landauer TK (1993) A mathematical model of the finding of usability problems. In Proceedings of ACM INTERCHI’93 conference. ACM Press, Amsterdam, pp 206–213

  • Pallant A, Tinker RF (2004) Reasoning with atomic-scale molecular dynamic models. J Sci Educ Technol 13(1):51–66

    Article  Google Scholar 

  • Preece J, Rogers Y, Sharp H (2002) Interaction design: beyond human-computer interaction. Wiley, New York

    Google Scholar 

  • Quintana C, Reiser BJ, Davis EA, Krajcik J, Fretz E, Duncan RG, Kyza E, Edelson D, Soloway E (2004) A scaffolding design framework for software to support science inquiry. J Learn Sci 13(3):337–386

    Article  Google Scholar 

  • Schwarz CV, White BY (2005) Metamodeling knowledge: developing students’ understanding of scientific modeling. Cogn Instr 32(2):165–205

    Article  Google Scholar 

  • Sengupta P, Wilensky U (2011) Lowering the learning threshold: multi-agent-based models and learning electricity (pp 141–171). In Khine MS, Saleh IM (eds) Models and modeling: cognitive tools for scientific enquiry. Springer, New York, pp 23–40

  • Shapiro AM (2008) Hypermedia design as learner scaffolding. Edu Tech Res Dev 56:29–44

    Article  Google Scholar 

  • Shneiderman B, Bederson BB (2005) Maintaining concentration to achieve task completion. In: Proceedings of the 2005 designing user experience conference. AIGA Press, San Francisco

  • Shorkey CT, Crocker SB (1981) Frustration theory: a source of unifying concepts for generalist practice. Soc Work 26(5):374–379

    Google Scholar 

  • Sim G, MacFarlane S, Read J (2006) All work and no play: measuring fun, usability and learning in software for children. Comput Educ 46(3):235–248

    Article  Google Scholar 

  • Soloway E, Guzdial M, Hay KE (1994) Learner-centered design: the challenge for HCI in the 21st century. Interactions 1(2):36–48

    Article  Google Scholar 

  • Songer NB (2006) BioKIDS: an animated conversation on the development of complex reasoning in science (pp 355–369). In: Sawyer RK (ed) Cambridge handbook of the learning sciences. Cambridge University Press, New York

    Google Scholar 

  • Songer NB, Kelcey B, Gotwals AW (2009) How and when does complex reasoning occur? Empirically driven development of a learning progression focused on complex reasoning about biodiversity. J Res Sci Teach 46(6):610–631

    Article  Google Scholar 

  • Sullivan P (1989) Beyond a narrow conception of usability testing. Proc IEEE Transac Prof Commun 32(4):256–264

    Article  Google Scholar 

  • Taylor I, Barker M, Jones A (2003) Promoting mental model building in astronomy education. Int J Sci Edu 25(10):1205–1225

    Article  Google Scholar 

  • Turner CW, Lewis JR, Nielsen J (2006) Determining usability test sample size. In: Karwowski W (ed) International encyclopedia of ergonomics and human factors, vol 4, 6th edn. CRC Press, Boca Raton

    Google Scholar 

  • Uttal DH (2000) Seeing the big picture: map use and the development of special cognition. Dev Sci 3(3):247–264

    Article  Google Scholar 

  • Varma K, Linn MC (2011) Using interactive technology to support students’ understanding of the greenhouse effect and global warming. J Sci Educ Technol. doi:10.1007/s10956-011-9337-9

    Google Scholar 

  • Virzi RA (1992) Refining the test phase of usability evaluation: how many subjects is enough? Hum Factors 34(4):457–468

    Google Scholar 

Download references

Acknowledgments

This research is based upon work supported by the National Science Foundation under Grant No. 0918590. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Author information

Authors and Affiliations

  1. School of Education, University of Michigan, 610 East University Avenue, Ann Arbor, MI, 48109, USA

    Vanessa L. Peters & Nancy Butler Songer

Authors
  1. Vanessa L. Peters
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nancy Butler Songer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vanessa L. Peters.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Peters, V.L., Songer, N.B. Evaluating the Usability of a Professional Modeling Tool Repurposed for Middle School Learning. J Sci Educ Technol 22, 681–696 (2013). https://doi.org/10.1007/s10956-012-9422-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10956-012-9422-8

Keywords

Search

Navigation