Maximizing research and development resources: identifying and testing “load-bearing conditions” for educational technology innovations

  • Jennifer IritiEmail author
  • William Bickel
  • Christian Schunn
  • Mary Kay Stein
Development Article


Education innovations often have a complicated set of assumptions about the contexts in which they are implemented, which may not be explicit. Education technology innovations in particular may have additional technical and cultural assumptions. As a result, education technology research and development efforts as well as scaling efforts can be slowed or made less efficacious because some of these basic assumptions (called load bearing conditions) about the match and prerequisites for the innovation are not met. The assumptions-based planning model is adapted as a methodology to help identify the load-bearing conditions for innovations. The process and impact of its use with two cases of education technology-oriented research and development efforts is reported. The work demonstrates the potential value of this LBC process for recruiting, selecting, and supporting research sites, for innovation designers to target efforts that strengthen implementation and support of scaling. Recommendations are made for others engaged in partnerships with education providers around developing, implementing and testing new education technology based innovations in more effective ways.


Research and development Implementation Assumptions-based planning Scalability Education technology Innovations Formative evaluation 



The authors would like to thank Robin Shoop and the Robot Algebra and BLOOM team members for their contributions to the process described in this manuscript.


This work was made possible by two National Science Foundation-funded Projects: Robot Algebra Project (DRL-1029404) and Modeling Engineered Levers for the 21st Century Teaching of STEM (DRL-1027629).

Compliance with ethical standards

The work described in this manuscript is not considered human subjects research by the University of Pittsburgh’s Institutional Review Board.

Conflict of Interest

The authors declare that they have no conflict of interest.


  1. Bickel, W. E., & Iriti, J. E. (2012). Study of organizational context characteristics that influence implementation of robotics units. Pittsburgh, PA: Evaluation for Learning Project, Learning Research and Development Center, University of Pittsburgh. Google Scholar
  2. Boulet, G. (2009). Rapid prototyping: An efficient way to collaboratively design and develop e-learning content. Retrieved from
  3. Burkhardt, H., & Schoenfeld, A. H. (2003). Improving educational research: Toward a more useful, more influential, and better-funded enterprise. Educational Researcher, 32(9), 3–14.CrossRefGoogle Scholar
  4. Cavanagh, S. (2014). K-12 district leaders evolving into smarter ed-tech consumers. Education Week, 33(35)Google Scholar
  5. Chen, H. T. (1990). Theory-driven evaluations. Newbury Park: Sage.Google Scholar
  6. Coburn, C. E., & Stein, M. K. (2010). Key lessons about the relationship between research and practice. In C. E. Coburn & M. K. Stein (Eds.), Research and practice in education: Building alliances, bridging the divide (pp. 201–226). Lanham, MD: Rowman & Littlefield.Google Scholar
  7. Culp, K. M., Hawkins, J., & Honey, M. (1999). Review paper on educational technology research and development. Waltham: Center for Children and Technology, Education Development Center.Google Scholar
  8. Davis, E. A., & Krajcik, J. S. (2005). Designing educative curriculum materials to promote teacher learning. Educational Researcher, 34(3), 3.CrossRefGoogle Scholar
  9. Dewar, J. A., Builder, C. H., Hix, W. M., & Levin, M. H. (1993). Assumptions-based planning: A planning tool for very uncertain times. Santa Monica: RAND Corporation.Google Scholar
  10. Dickard, N. (Ed.) (2003). The sustainability challenge: Taking ed-tech to the next level. Washington, DC: The Benton Foundation Communications Policy Program & EDC Center for Children and Technology. Retrieved from
  11. Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.CrossRefGoogle Scholar
  12. Education Superhighway. (2014). Connecting America’s students: Opportunities for action. Retrieved from
  13. Ertmer, P. A., Ottenbreit-Leftwich, A. T., Sadik, O., Sendurur, E., & Sendurur, P. (2012). Teacher beliefs and technology integration practices: A critical relationship. Computers & Education, 59(2), 423–435.CrossRefGoogle Scholar
  14. European Commission. (2013). Opening up Education. memo Retrieved from
  15. Fishman, B., Marx, R. W., Blumenfeld, P., Krajcik, J., & Soloway, E. (2009). Creating a framework for research on systemic technology innovations. Journal of the Learning Sciences, 13(1), 43–76.CrossRefGoogle Scholar
  16. Hew, K. F., & Brush, T. (2007). Integrating technology into K-12 teaching and learning: Current knowledge gaps and recommendations for future research. Education Technology Research and Development, 55, 223–252.CrossRefGoogle Scholar
  17. Iriti, J. E., Nelson, C. A., & Bickel, W. E. (2010). Ball-Rowland Unified School District partnership evaluation report. Pittsburgh, PA: Learning Research and Development Center, University of Pittsburgh.Google Scholar
  18. Jones, T. S., & Richey, R. C. (2000). Rapid prototyping methodology in action: A developmental study. Education Technology Research and Development, 48(2), 63–80.CrossRefGoogle Scholar
  19. Kelly, A. E., Lesh, R. A., & Baek, J. Y. (2000). Handbook of design research methods in education: Innovation in science, technology, engineering, and mathematics teaching and learning. New York: Routledge.Google Scholar
  20. Kessler, A., Boston, M., & Stein, M. K. (2014). Conceptualizing teacher’s practices in supporting students’ mathematical learning in computer-directed learning environments. In Proceedings of the 11th  International Conference of the Learning Sciences (ICLS) (Vol. 1). Boulder, CO.Google Scholar
  21. Kraus, R. (2008). Inquiry teaching methods: A multiple case study. Chicago: Illinois Institute of Technology.Google Scholar
  22. Martin, P. (2014). Navigating choice: Toward better ed-tech product discovery. Digital Promise:
  23. Meyer, D. Z., Meyer, A. A., Nabb, K., Connell, M., & Avery, L. (2007). A theoretical and empirical exploration of intrinsic problems in designing inquiry activities. Education Technology Research and Development, 55, 223–252.CrossRefGoogle Scholar
  24. Molinar, M. (2014). Startups in education set record in raising money. Education Week, 33(8).Google Scholar
  25. Molinar, M. (2015). Evaluating the outlook for the ed-tech industry in 2015. Education Week, Marketplace K-12 blogs. Google Scholar
  26. Open Education Challenge. (2014). Open Education Challenge “The Results.”
  27. Raudenbush, S. W. (2007). Designing field trial of educational innovations. In B. Schneider & S. McDonald (Eds.), Scale up in education (Vol. II). New York: Rowman & Littlefield.Google Scholar
  28. Reynolds, A. J. (1998). Confirmatory program evaluation: A method for strengthening causal inference. American Journal of Evaluation, 19(2), 203–221.CrossRefGoogle Scholar
  29. Rogers, E. M. (1995). Diffusion of innovations (4th ed.). New York: Free Press.Google Scholar
  30. Schunn, C. D., & Stein, M. K. (2010a). The Robot Algebra Project. Proposal submitted to the National Science Foundation ITEST Collaborative Research Strategy. Pittsburgh, PA: Learning Research and Development Center.Google Scholar
  31. Schunn, C. D., & Stein, M. K. (2010b). Modeling Engineered Levers for the 21st Century Teaching of STEM. Proposal submitted to the National Science Foundation DRK-12. Pittsburgh, PA: Learning Research and Development Center.Google Scholar
  32. Schunn, C. D., & Stein, M. K. (2013a). The Robot Algebra Project: Final Report. Submitted to the National Science Foundation. Pittsburgh, PA: Learning Research and Development Center.Google Scholar
  33. Schunn, C. D., & Stein, M. K. (2013b). Modeling Engineered Levers for the 21st Century Teaching of STEM: Interim Report. Submitted to the National Science Foundation DRK-12. Pittsburgh, PA: Learning Research and Development Center.Google Scholar
  34. Shoop, R. (2012). Robotics education research partner expectations. Pittsburgh: Carnegie Mellon’s Robotics Academy.Google Scholar
  35. Wachira, P., & Keengwe, J. (2011). Technology integration barriers: Urban school mathematics teachers’ perspectives. Journal of Science Education and Technology, 20, 17–25.CrossRefGoogle Scholar
  36. White House Office of Science and Technology Policy. (2014). In Preparing Americans with 21st Century skills: Science, Technology, Engineering, and Mathematics (STEM) education in the 2015 budget. www.whitehouse.gove/ostp.

Copyright information

© Association for Educational Communications and Technology 2015

Authors and Affiliations

  • Jennifer Iriti
    • 1
    Email author
  • William Bickel
    • 1
  • Christian Schunn
    • 1
  • Mary Kay Stein
    • 1
  1. 1.University of Pittsburgh’s Learning Research and Development CenterPittsburghUSA

Personalised recommendations