Analysis of Propagation Plans in NSF-Funded Education Development Projects

Abstract

Increasing adoption and adaptation of promising instructional strategies and materials has been identified as a critical component needed to improve science, technology, engineering, and mathematics (STEM) education. This paper examines typical propagation practices and resulting outcomes of proposals written by developers of educational innovations. These proposals were analyzed using the Designing for Sustained Adoption Assessment Instrument (DSAAI), an instrument developed to evaluate propagation plans, and the results used to predict the likelihood that a successful project would result in adoption by others. We found that few education developers propose strong propagation plans. Afterwards, a follow-up analysis was conducted to see which propagation strategies developers actually used to help develop, disseminate, and support their innovations. A web search and interviews with principal investigators were used to determine the degree to which propagation plans were actually implemented and to estimate adoption of the innovations. In this study, we analyzed 71 education development proposals funded by the National Science Foundation and predicted that 80% would be unsuccessful in propagating their innovations. Follow-up data collection with a subset of these suggests that the predictions were reasonably accurate.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. Alfred P. Sloan Foundation: STEM Higher Education (2016). http://www.sloan.org/major-program-areas/stem-higher-education/the-science-of-learning-stem. Accessed 11 April 2016

  2. Australian Learning and Teaching Council (2008) The ALTC dissemination framework. Retrieved from http://www.acu.edu.au/__data/assets/pdf_file/0005/161519/dissemination_altcframework_2008.pdf. Accessed February 2017

  3. Beach AL, Henderson C, Finkelstein N (2012) Facilitating change in undergraduate STEM education. Change 44(6):52–59. doi:10.1080/00091383.2012.728955

    Article  Google Scholar 

  4. Borrego M, Henderson C (2014) Increasing the use of evidence-based teaching in STEM higher education: a comparison of eight change strategies. J Eng Educ 103(2):220–252

    Article  Google Scholar 

  5. Borrego M, Froyd JE, Hall TS (2010) Diffusion of engineering education innovations: a survey of awareness and adoption rates in U.S. engineering departments. J Eng Educ 99(3):185–207. doi:10.1002/j.2168-9830.2010.tb01056.x

    Article  Google Scholar 

  6. Bourrie DM, Cegielski CG, Jones-Farmer LA, Sankar CS (2014) Identifying characteristics of dissemination success using an expert panel. Decis Sci J Innovat Educ 12(4):357–380

    Article  Google Scholar 

  7. Brookhart SM, Chen F (2015) The quality and effectiveness of descriptive rubrics. Edu Rev 67(3):343–368

  8. Dancy M, Henderson C (2010) Pedagogical practices and instructional change of physics faculty. Am J Phys 78(10):1056–1063

    Article  Google Scholar 

  9. Fairweather J (2008). Linking evidence and promising practices in science, technology, engineering, and mathematics (STEM) undergraduate education. In: Board of Science Education, National Research Council, & The National Academies (Eds.). Washington, DC

  10. Farquhar JD, Surry DW (1994) Adoption analysis: an additional tool for instructional developers. IETI 31(1):19–25

    Google Scholar 

  11. Fishman B, Marx RW, Blumenfeld P, Krajcik J, Soloway E (2004) Creating a framework for research on systemic technology innovations. J Learn Sci 13(1):43–76

    Article  Google Scholar 

  12. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F (2005) Implementation research: a synthesis of the literature. University of South Florida, National Implementation Research Network, Tampa

    Google Scholar 

  13. Froyd JE (2001) Developing a dissemination plan. Paper presented at the Frontiers in Education Conference, Reno

    Google Scholar 

  14. Gannaway D, Hinton T, Berry B, Moore K (2013) Cultivating change: disseminating innovation in higher education teaching and learning. Innov Educ Teach Int 50(4):410–421

    Article  Google Scholar 

  15. Ghaith G, Yaghi H (1997) Relationships among experience, teacher efficacy, and attitudes toward the implementation of instructional innovation. Teach Teach Edu 13(4):451–458

    Article  Google Scholar 

  16. Hazen BT, Wu Y, Sankar CS (2012) Factors that influence dissemination in engineering education. IEEE Trans Educ 55(3):384–393

    Article  Google Scholar 

  17. Henderson C (2008) Promoting instructional change in new faculty: an evaluation of the physics and astronomy new faculty workshop. Am J Phys 76(2):176–187

    Article  Google Scholar 

  18. Henderson C, Dancy MH (2007) Barriers to the use of research-based instructional strategies: the influence of both individual and situational characteristics. PRST PER 3(2):020102

    Google Scholar 

  19. Henderson C and Dancy M H (2011) Increasing the impact and diffusion of STEM education innovations, a white paper commissioned for the characterizing the impact and diffusion of engineering education innovations forum

  20. Henderson C, Finkelstein N, Beach A (2010) Beyond dissemination in college science teaching: an introduction to four core change strategies. J Col Sci Teach 39(5):18–25

  21. Henderson C, Beach A, Finkelstein N (2011) Facilitating change in undergraduate STEM instructional practices: an analytic review of the literature. J Res Sci Teach 48(8):952–984. doi:10.1002/tea.20439

    Article  Google Scholar 

  22. Henderson C, Cole R, Froyd J and Khatri R (2012) Five claims about effective propagation, a white paper prepared for January 30–31, 2012 meetings with NSF TUES Program Directors

  23. Henderson C, Cole R, Froyd J, Gilbuena D, Khatri R, Stanford C (2015) Designing educational innovations for sustained adoption: a how-to guide for education developers who want to increase the impact of their work. Increase the Impact, Kalamazoo

    Google Scholar 

  24. Hinton T, Gannaway D, Berry B, Moore K (2011) The D-cubed guide: planning for effective dissemination. Australian Teaching and Learning Council, Sydney

    Google Scholar 

  25. Hora MT (2012) A situative analysis of the relationship between faculty beliefs and teaching practice: implications for instructional improvement at the postsecondary level. Wisconsin center for education research working paper no. 2012–10

  26. Hora MT, Anderson C (2012) Perceived norms for interactive teaching and their relationship to instructional decision-making: a mixed methods study. High Edu 64(4):573–592

    Article  Google Scholar 

  27. I-Corps for Learning (2016). https://www.asee.org/i-corps-l/about. Accessed 3 March 2016

  28. Jonsson A, Svingby G (2007) The use of scoring rubrics: reliability, validity and educational consequences. Educ Res Rev 2(2):130–144

    Article  Google Scholar 

  29. Kezar AJ (2012) The path to pedagogical reform in the sciences. Lib Educ 98(1):40–45

    Google Scholar 

  30. Kezar AJ, Eckel PD (2002) The effect of institutional culture on change strategies in higher education: universal principles or culturally responsive concepts? J High Educ 73(4):435–460

    Article  Google Scholar 

  31. Khatri R, Henderson C, Cole R, Froyd JE, Friedrichsen D, Stanford C (2016) Designing for sustained adoption: a model of developing educational innovations for successful propagation. Physical Review Physics Education Research 12:010112

    Article  Google Scholar 

  32. Khatri R, Henderson C, Cole RS, Froyd J, Friedrichsen D, Stanford C (2017) Characteristics of well-propagated teaching innovations for undergraduate STEM disciplines. Int J of STEM Edu 4(2):1–10

  33. King H (2003) Disseminating educational developments. In: Kahn P, Baume D (eds) A guide to staff and educational development. Kogan Page, London, pp 96–115

    Google Scholar 

  34. Lattuca LR (2011) Influences on engineering faculty members’ decisions about educational innovations: a systems view of curricular and instructional change. A White Paper Prepared for Characterizing the Impact of Diffusion of Engineering Education Innovations Forum

  35. Lattuca LR, Stark JS (2009) Shaping the college curriculum: academic plans in context, 2nd edn. Jossey-Bass, San Francisco

    Google Scholar 

  36. Lindblom YS, Trigwell K, Nevgi A, Ashwin P (2006) How approaches to teaching are affected by discipline and teaching context. Stud High Educ 31(3):285–298

    Article  Google Scholar 

  37. Litzinger TA, Zappe SE, Borrego M, Froyd JE, Newstetter W, Tonso KL et al (2011) Writing effective evaluation and dissemination plans for innovations in engineering education. Paper presented at the ASEE Annual Conference & Exposition, Vancouver

    Google Scholar 

  38. Lund TJ, Stains M (2015) The importance of context: an exploration of factors influencing the adoption of student-centered teaching among chemistry, biology, and physics faculty. Int J STEM Edu 2:13. doi:10.1186/s40594-015-0026-8

    Article  Google Scholar 

  39. McKenzie J, Alexander S, Harper C, Anderson S (2005) Dissemination, adoption and adaptation of project innovations in higher education. University of Technology, Sydney, Sydney

    Google Scholar 

  40. McMartin F, Giersch S, Tront J, Shumar W (2012) A tale of two studies: is dissemination working? In Proceedings of the 12th ACM/IEEE-CS joint conference on Digital Libraries. ACM, Washington, DC, USA, pp 47–50

  41. National Research Council (2012) Discipline-based education research: understanding and improving learning in undergraduate science and engineering. The National Acedemies Press, Washington DC

    Google Scholar 

  42. National Science Foundation (2009) Course, curriculum, and laboratory improvement (CCLI) Program Solicitation 09–529. Education & Human Resources Division of Undergraduate Education. Retrieved from https://www.nsf.gov/pubs/2009/nsf09529/nsf09529.html. Accessed November 2016

  43. National Science Foundation (2015) Improving undergraduate STEM education: education and human resources (IUSE: EHR) program solicitation NSF 15–585. Education & Human Resources Division of Undergraduate Education (https://www.nsf.gov/pubs/2015/nsf15585/nsf15585.htm). Accessed November 2016

  44. Norton L, Richardson T, Hartley J, Newstead S, Mayes J (2005) Teachers’ beliefs and intentions concerning teaching in higher education. High. Edu. 50(4):537–571

    Article  Google Scholar 

  45. Olson S, Riordan DG (2012) Report to the president. Engage to excel: producing one million additional college graduates with degrees in science, technology, engineering, and mathematics. President’s Council of Advisors on Science and Technology, Washington DC

    Google Scholar 

  46. Prosser M, Trigwell K (1997) Relations between perceptions of the teaching environment and approaches to teaching. Brit J Educ Psychol 67(1):25–35

    Article  Google Scholar 

  47. Rogers EM (2003) Diffusion of innovations, 5th edn. Free Press, New York, NY

    Google Scholar 

  48. Seymour E (2002) Tracking the processes of change in US undergraduate education in science, mathematics, engineering, and technology. Sci Educ 86(1):79–105. doi:10.1002/sce.1044

    Article  Google Scholar 

  49. Singer E (1996) Espoused teaching paradigms of college faculty. Res High Educ 37(6):659–679

    Article  Google Scholar 

  50. Smith K (2012) Lessons learnt from literature on the diffusion of innovative learning and teaching practices in higher education. Innov. Educ. Teach. Int. 49(2):173–182

    Article  Google Scholar 

  51. Southwell D, Gannaway D, Orrell J, Chalmers D, Abraham C (2005) Strategies for effective dissemination of project outcome. Carrick Institute for Learning and Teaching in Higher Education, Sydney

    Google Scholar 

  52. Southwell D, Gannaway D, Orrell J, Chalmers D, Abraham C (2010) Strategies for effective dissemination of the outcomes of teaching and learning projects. Journal of Higher Education Policy and Management 32(1):55–67

    Article  Google Scholar 

  53. Stains M, Pilarz M, Chakraverty D (2015) Short and long-term impacts of the Cottrell scholars collaborative new faculty workshop. J Chem Educ 92:1466–1476

    Article  Google Scholar 

  54. Stanford C, Cole R, Froyd JE, Friedrichsen D, Khatri R, Henderson C (2016) Supporting sustained adoption of education innovations: the designing for sustained adoption assessment instrument. Int J STEM Edu 3:1

    Article  Google Scholar 

  55. Stark JS (2000) Planning introductory college courses: content, context and form. Instr Sci 28(5):413–438

    Article  Google Scholar 

  56. The Howard Hughes Medical Institute: Building Authentic Research Experiences (2016). http://www.hhmi.org/advance-science/building-authentic-research-experiences. Accessed 11 April 2016

  57. The Leona M. and Harry B. Helmsley Charitable Trust: Our Grants (2016). http://helmsleytrust.org/our-grants. Accessed 11 April 2016

  58. Tront J, McMartin F and Muramatsu B (2011) Improving the dissemination of CCLI (TUES) educational innovations. In: Proceedings of the ASEE/IEEE Frontiers in Education Conference

  59. Turpen C, Finkelstein N (2009) Not all interactive engagement is the same: variations in physics professors’ implementation of peer instruction. PRST PER 5(2):020101

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Science Foundation under grant nos.1122446, 1122416, and 1236926. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. We are grateful for the contributions of PIs who have allowed us to analyze their proposals and members of the STEM education community for sharing their knowledge of well-propagated innovations.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Renee Cole.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Permissions

In this article, Table 2 has been published in a previous journal article by the authors. This table is printed in Stanford, C., Cole, R., Froyd, J. E., Friedrichsen, D., Khatri, R., & Henderson, C. Supporting sustained adoption of education innovations: The Designing for Sustained Adoption Assessment Instrument. International Journal of STEM Education. (2016). This is an open access journal where the copyright is retained by the authors. In the article, we have cited it accordingly as a reprint and have referenced where the original publication can be found.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Stanford, C., Cole, R., Froyd, J. et al. Analysis of Propagation Plans in NSF-Funded Education Development Projects. J Sci Educ Technol 26, 418–437 (2017). https://doi.org/10.1007/s10956-017-9689-x

Download citation

Keywords

  • Effective propagation strategies
  • Educational innovations
  • Proposal writing
  • Diffusion
  • Transfer of innovation
  • Broader impacts
  • Dissemination