Journal of Science Education and Technology

, Volume 23, Issue 1, pp 145–159 | Cite as

A Meta-analysis Method to Advance Design of Technology-Based Learning Tool: Combining Qualitative and Quantitative Research to Understand Learning in Relation to Different Technology Features



Educators design and create various technology tools to scaffold students’ learning. As more and more technology designs are incorporated into learning, growing attention has been paid to the study of technology-based learning tool. This paper discusses the emerging issues, such as how can learning effectiveness be understood in relation to different technology features? And how can pieces of qualitative and quantitative results be integrated to achieve a broader understanding of technology designs? To address these issues, this paper proposes a meta-analysis method. Detailed explanations about the structure of the methodology and its scientific mechanism are provided for discussions and suggestions. This paper ends with an in-depth discussion on the concerns and questions that educational researchers might raise, such as how this methodology takes care of learning contexts.


Methodology Meta-analysis Technology tool 


  1. Allison PD (2008) Convergence failures in logistic regression. In: Paper presented at the SAS Global Forum 2008 Conference, San Antonio, TexasGoogle Scholar
  2. Anderson T, Shattuck J (2012) Design-based research: a decade of progress in education research? Educ Res 41(1):16–25Google Scholar
  3. Annetta LA, Cheng M-T, Holmes S (2010) Assessing twenty-first century skills through a teacher created video game for high school biology students. Res Sci Technol Educ 28(2):101–114CrossRefGoogle Scholar
  4. Barab S (2006) Design-based research: a methodological toolkit for the learning scientist. In: Sawyer RK (ed) The Cambridge handbook of the learning sciences. Cambridge University Press, Cambridge, pp 153–168Google Scholar
  5. Barab S, Squire K (2004) Design-based research: putting a stake in the ground. J Learn Sci 13(1):1–14CrossRefGoogle Scholar
  6. Birchfield D, Megowan-Romanowicz C (2009) Earth science learning in SMALLab: a design experiment for mixed reality. Int J Comput Support Collab Learn 4(4):403–421CrossRefGoogle Scholar
  7. Brown AL (1992) Design experiments: theoretical and methodological challenges in creating complex interventions in classroom settings. J Learn Sci 2(2):141–178CrossRefGoogle Scholar
  8. Butler KA, Lumpe A (2008) Student use of scaffolding software: relationships with motivation and conceptual understanding. J Sci Educ Technol 17(5):427–436CrossRefGoogle Scholar
  9. Caspersona JM, Linn MC (2006) Using visualizations to teach electrostatics. Am J Phys 74(4):316–323CrossRefGoogle Scholar
  10. Clark D, D’Angelo C, Menekse M (2009) Initial structuring of online discussions to improve learning and argumentation: incorporating students’ own explanations as seed comments versus an augmented-preset approach to seeding discussions. J Sci Educ Technol 18(4):321–333CrossRefGoogle Scholar
  11. Collins A (1992) Toward a design science of education. In: Scanlon E, O’Shea T (eds) New directions in educational technology. Springer, New York, pp 15–22CrossRefGoogle Scholar
  12. Collins A, Joseph D, Bielaczyc K (2004) Design research: theoretical and methodological issues. J Learn Sci 13(1):15–42CrossRefGoogle Scholar
  13. Davis EA (2000) Scaffolding students’ knowledge integration: prompts for reflection in KIE. Int J Sci Educ 22(8):819–837CrossRefGoogle Scholar
  14. de Vries E, Lund K, Baker M (2002) Computer-mediated epistemic dialogue: explanation and argumentation as vehicles for understanding scientific notions. J Learn Sci 11(1):63–103CrossRefGoogle Scholar
  15. Dori YJ, Sasson I (2008) Chemical understanding and graphing skills in an honors case-based computerized chemistry laboratory environment: the value of bidirectional visual and textual representations. J Res Sci Teach 45(2):219–250CrossRefGoogle Scholar
  16. Ebenezer JV (2001) A hypermedia environment to explore and negotiate students’ conceptions: animation of the solution process of table salt. J Sci Educ Technol 10(1):73–92CrossRefGoogle Scholar
  17. Ebenezer J, Kaya ON, Ebenezer DL (2011) Engaging students in environmental research projects: perceptions of fluency with innovative technologies and levels of scientific inquiry abilities. J Res Sci Teach 48(1):94–116CrossRefGoogle Scholar
  18. Edelson DC (2002) Design research: what we learn when we engage in design. J Learn Sci 11(1):105–121CrossRefGoogle Scholar
  19. Edelson DC, Gordin DN, Pea RD (1999) Addressing the challenges of inquiry-based learning through technology and curriculum design. J Learn Sci 8(3/4):391–450Google Scholar
  20. Geier R, Blumenfeld PC, Marx RW, Krajcik JS, Fishman B, Soloway E et al (2008) Standardized test outcomes for students engaged in inquiry-based science curricula in the context of urban reform. J Res Sci Teach 45(8):922–939CrossRefGoogle Scholar
  21. Hosmer DW, Lemeshow S (1989) Applied logistic regression. Wiley, New YorkGoogle Scholar
  22. Huppert J, Lomask SM, Lazarowitz R (2002) Computer simulations in the high school: students’ cognitive stages, science process skills and academic achievement in microbiology. Int J Sci Educ 24(8):803–821CrossRefGoogle Scholar
  23. Kim MC, Hannafin MJ, Bryan LA (2007) Technology-enhanced inquiry tools in science education: an emerging pedagogical framework for classroom practice. Sci Educ 91(6):1010–1030CrossRefGoogle Scholar
  24. Linn M (2003) Technology and science education: starting points, research programs, and trends. Int J Sci Educ 25(6):727–758CrossRefGoogle Scholar
  25. Mayer RE (2003) Learning environments: the case for evidence-based practice and issue-driven research. Educ Psychol Rev 15(4):359–366CrossRefGoogle Scholar
  26. Menard S (1995) Applied logistic regression analysis. Sage Publications, Thousand Oaks, CAGoogle Scholar
  27. Mistler-Jackson M, Songer N (2000) Student motivation and Internet technology: are students empowered to learn science? J Res Sci Teach 37(5):459–479CrossRefGoogle Scholar
  28. Moss DM (2003) A window on science: exploring the JASON project and student conceptions of science. J Sci Educ Technol 12(1):21–30CrossRefGoogle Scholar
  29. National Research Council (2002) Scientific research in education. National Academy Press, Washington, DCGoogle Scholar
  30. National Research Council (2011) A framework for K-12 science education: practices, crosscutting concepts, and core ideas, Committee on a Conceptual Framework for New K-12 Science Education Standards. Board on Science Education, Division of Behavioral and Social Sciences and Education. The National Academies Press, Washington, DCGoogle Scholar
  31. Oliver K (2009) An investigation of concept mapping to improve the reading comprehension of science texts. J Sci Educ Technol 18(5):402–414CrossRefGoogle Scholar
  32. O’Neill DK (2001) Knowing when you’ve brought them in: scientific genre knowledge and communities of practice. J Learn Sci 10(3):223–264CrossRefGoogle Scholar
  33. Parnafes O (2007) What does “fast” mean? Understanding the physical world through computational representations. J Learn Sci 16(3):415–450CrossRefGoogle Scholar
  34. Parr CS, Jones T, Songer NB (2004) Evaluation of a handheld data collection interface for science learning. J Sci Educ Technol 13(2):233–242CrossRefGoogle Scholar
  35. Pine J, Aschbacher P, Roth E, Jones M, McPhee C, Martin C et al (2006) Fifth graders’ science inquiry abilities: a comparative study of students in hands-on and textbook curricula. J Res Sci Teach 43(5):467–484CrossRefGoogle Scholar
  36. Puntambekar S, Stylianou A, Goldstein J (2007) Comparing classroom enactments of an inquiry curriculum: lessons learned from two teachers. J Learn Sci 16(1):81–130Google Scholar
  37. Quintana C, Reiser BJ, Davis EA, Krajcik J, Fretz E, Duncan RG et al (2004) A scaffolding design framework for software to support science inquiry. J Learn Sci 13(3):337–386CrossRefGoogle Scholar
  38. Rogers Y, Price S (2008) The Role of mobile devices in facilitating collaborative inquiry in situ. Res Pract Technol Enhanc Learn 3(3):209–229CrossRefGoogle Scholar
  39. Sandoval WA, Reiser B (2004) Explanation-driven inquiry: integrating conceptual and epistemic scaffolds for scientific inquiry. Sci Educ 88(3):345–372CrossRefGoogle Scholar
  40. Shavelson RJ, Phillips DC, Towne L, Feuer MJ (2003) On the science of education design studies. Educ Res 32(1):25–28Google Scholar
  41. Shin E (2006) Using Geographic Information System (GIS) to improve fourth graders’ geographic content knowledge and map skills. J Geogr 105(3):109–120CrossRefGoogle Scholar
  42. Slavin RE (2002) Evidence-based education policies: transforming educational practice and research. Educ Res 31(7):15–21Google Scholar
  43. Slotta JD, Linn MC (2009) WISE science: web-based inquiry in classroom. Teachers College Press, New York, NYGoogle Scholar
  44. Songer NB, Lee H-S, Kam R (2002) Technology-rich inquiry science in urban classrooms: what are the barriers to inquiry pedagogy? J Res Sci Teach 39(2):128–150CrossRefGoogle Scholar
  45. Songer NB, Lee H-S, McDonald S (2003) Research towards an expanded understanding of inquiry science beyond one idealized standard. Sci Educ 87(4):490–516CrossRefGoogle Scholar
  46. Stieff M, Wilensky U (2003) Connected chemistry—incorporating interactive simulations into the chemistry classroom. J Sci Educ Technol 12(3):285–302CrossRefGoogle Scholar
  47. Tolentino L, Birchfield D, Megowan-Romanowicz C, Johnson-Glenberg MC, Kelliher A, Martinez C (2009) Teaching and learning in the mixed-reality science classroom. J Sci Educ Technol 18(6):501–517CrossRefGoogle Scholar
  48. Waight N, Abd-El-Khalick F (2007) The impact of technology on the enactment of inquiry in a technology enthusiast’s sixth grade science classroom. J Res Sci Teach 44(Generic):154–182Google Scholar
  49. Wallace RM, Kupperman J, Krajcik J, Soloway E (2000) Science on the web: students online in a sixth-grade classroom. J Learn Sci 9(1):75–104CrossRefGoogle Scholar
  50. Williams M, Linn MC (2002) WISE inquiry in fifth grade biology. Res Sci Educ 32:415–436CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Curriculum and Instruction, Lynch School of EducationBoston CollegeChestnut HillUSA

Personalised recommendations