What Makes Research Useful for Public School Educators?

Abstract

In this study, we explored the extent to which educators discuss and prioritize Rogers’ (Diffusion of innovations, The Free Press: New York, 1995) five attributes of innovations—relative advantage, compatibility, complexity, observability, and trialability—in the context of research use. Using a directed content analysis of 54 semi-structured interviews and exemplar quotes, we describe how educators mentioned compatibility most frequently, but also commonly invoked observability and complexity in their discussions of research use. Our results also revealed key differences between educators in executive and non-executive roles. We discuss the implications of our findings for closing the research-practice gap in school-based mental health services and psychosocial interventions.

This is a preview of subscription content, access via your institution.

Notes

  1. 1.

    County names are pseudonyms to protect the confidentiality of study participants.

  2. 2.

    Because this method of calculating reliability could potentially result in high scores even if coders identified different segments for each code in an interview, we also calculated coder reliability by segment-code pairs. Over all of the segment-code pairs in our study, our coders demonstrated high inter-rater reliability (κ= 0.87). This suggests that coders were consistently coding attributes in the same segments.

References

  1. Atkins, M. S., Hoagwood, K. E., Kutash, K., & Seidman, E. (2010). Toward the integration of education and mental health in schools. Administration and Policy in Mental Health and Mental Health Services Research, 37, 40–47. https://doi.org/10.1007/s10488-010-0299-7.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Atkins, M. S., & Lakind, D. (2013). Usual care for clinicians, unusual care for their clients: Rearranging priorities for children’s mental health services. Administration and Policy in Mental Health and Mental Health Services Research, 40, 48–51. https://doi.org/10.1007/s10488-012-0453-5.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Bartels, N. (2003). How teachers and researchers read academic articles. Teaching and Teacher Education, 19, 737–753.

    Article  Google Scholar 

  4. Barwick, M. A., Barac, R., Akrong, L. M., Johnson, S., & Chaban, P. (2014). Bringing evidence to the classroom: Exploring educator notions of evidence and preferences for practice change. International Education Research, 2(4), 1–15.

    Article  Google Scholar 

  5. Behrstock-Sherratt, E., Drill, K., & Miller, S. (2011). Is the supply in demand? Exploring how, when and why teachers use research (revised Ed). Washington, DC: American Institutes for Research.

    Google Scholar 

  6. Boardman, A. G., Argüelles, M. E., Vaughn, S., Hughes, M. T., & Klingner, J. (2005). Special education teachers’ views of research-based practices. The Journal of Special Education, 39, 168–180.

    Article  Google Scholar 

  7. Cappella, E., Frazier, S. L., Atkins, M. S., Schoenwald, S. K., & Glisson, C. (2008). Enhancing schools’ capacity to support children in poverty: An ecological model of school-based mental health services. Administration and Policy in Mental Health and Mental Health Services Research, 35, 395–409. https://doi.org/10.1007/s10488-008-0182-y.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Carnine, D. (1997). Bridging the research-to-practice gap. Exceptional Children, 63(4), 513–521.

    Article  Google Scholar 

  9. Coburn, C. E., Honig, M. I., & Stein, M. K. (2009). What is the evidence on districts’ use of evidence? In J. D. Bransford, D. J. Stipek, N. J. Vye, L. M. Gomez & D. Lam (Eds.), The role of research in educational improvement (pp. 67–86). Cambridge: Harvard Education Press.

    Google Scholar 

  10. Coburn, C. E., Penuel, W., & Geil, K. E. (2013). Research-practice partnerships: A strategy for leveraging research for educational improvement in school districts. New York: William T. Grant Foundation.

    Google Scholar 

  11. Coburn, C. E., & Talbert, J. E. (2006). Conceptions of evidence use in school districts: Mapping the terrain. American Journal of Education, 112, 469–495.

    Article  Google Scholar 

  12. Cook, B. G., Smith, G. J., & Tankersley, M. (2012). Evidence-based practice in education. In K. R. Harris, S. Graham & T. Urdan (Eds.), APA educational psychology handbook: Vol 1. Theories, constructs, and critical issues (pp. 493–525). Washington DC: APA.

    Google Scholar 

  13. Corcoran, T., Fuhrman, S. H., & Belcher, C. L. (2001). The district role in instructional improvement. Phi Delta Kappan, 83, 78–84.

    Article  Google Scholar 

  14. Cousins, J. B., & Leithwood, K. A. (1986). Current empirical research on evaluation utilization. Review of Educational Research, 56, 331–364.

    Article  Google Scholar 

  15. Cousins, J. B., & Leithwood, K. A. (1993). Enhancing knowledge utilization as a strategy for school improvement. Knowledge Creation Diffusion Utilization, 14, 305–333.

    Google Scholar 

  16. Cousins, J. B., & Walker, C. A. (2000). Predictors of educators’ valuing of systematic inquiry in schools. Canadian Journal of Program Evaluation, Special Issue, 25–52.

  17. Curtis, J. R., Wenrich, M. D., Carline, J. D., Shannon, S. E., Ambrozy, D. M., & Ramsey, P. G. (2001). Understanding physicians’ skills at providing end-of-life care: Perspectives of patients, families, and health care workers. Journal of General Internal Medicine, 16, 41–49. https://doi.org/10.1111/j.1525-1497.2001.00333.x.

    CAS  PubMed  PubMed Central  Google Scholar 

  18. Dagenais, C., Lysenko, L., Abrami, P. C., Bernard, R. M., Ramde, J., & Janosz, M. (2012). Use of research-based information by school practitioners and determinants of use: A review of empirical research. Evidence & Policy, 8, 285–309. https://doi.org/10.1332/174426412X654031.

    Article  Google Scholar 

  19. Everton, T., Galton, M., & Pell, T. (2000). Teachers’ perspectives on educational research: knowledge and context. Journal of Education for Teaching, 26, 167–182.

    Article  Google Scholar 

  20. Every Student Succeeds Act (ESSA) 2015, 20 U.S.C.A. § 6301 et seq. (U.S. Government Publishing Office, 2015).

  21. Farley-Ripple, E. N. (2012). Research use in school district central office decision making: A case study. Educational Management Administration, and Leadership, 40, 786–806. https://doi.org/10.1177/1741143212456912.

    Article  Google Scholar 

  22. Finnigan, K. S., Daly, A. J., & Che, J. (2013). Systemwide reform in districts under pressure: The role of social networks in defining, acquiring, using, and diffusing research evidence. Journal of Educational Administration, 51, 476–497. https://doi.org/10.1108/09578231311325668.

    Article  Google Scholar 

  23. Gottfredson, D. C., Cook, T. D., Gardner, F. E. M., Gorman-Smith, D., Howe, G. W., Sandler, I. N., & Zafft, K. M. (2015). Standards of evidence for efficacy, effectiveness, and scale-up research in prevention science. Prevention Science, 16, 893–926.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Greenberg, M. T., Weissberg, R. P., O’Brien, M. U., Zins, J. E., Fredericks, L., Resnik, H., & Elias, M. J. (2003). Enhancing school-based prevention and youth development through coordinated social, emotional, and academic learning. American Psychologist, 58, 466–474.

    Article  PubMed  Google Scholar 

  25. Hallfors, D., & Godette, D. (2002). Will the ‘Principles of Effectiveness’’ improve prevention practice? Early findings from a diffusion study. Health Education Research, 17, 461–470.

    CAS  Article  PubMed  Google Scholar 

  26. Hoagwood, K., & Johnson, J. (2003). School psychology: a public health framework I. From evidence-based practices to evidence-based policies. Journal of School Psychology, 41, 3–21.

    Article  Google Scholar 

  27. Hofferth, S. L. (2009). Changes in American children’s time. Electronic International Journal of Time Use Research, 6(1), 26–47.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Hofferth, S. L., & Sandberg (2001). How American children spend their time. Journal of Marriage and Family, 63, 295–308.

    Article  Google Scholar 

  29. Honig, M. I., & Coburn, C. (2008). Evidence-based decision making in school district central offices: Toward a policy and research agenda. Educational Policy, 22, 578–608. https://doi.org/10.1177/0895904807307067.

    Article  Google Scholar 

  30. Hsieh, H., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15, 1277–1288. https://doi.org/10.1177/1049732305276687.

    Article  PubMed  Google Scholar 

  31. Hultman, G., & Hörberg, C. R. (1998). Knowledge competition and personal ambition: A theoretical framework for knowledge utilization and action in context. Science Communication, 19, 328–348.

    Article  Google Scholar 

  32. Humble, A. M. (2009). Technique triangulation for validation in directed content analysis. International Journal of Qualitative Methods, 8, 34–51.

    Article  Google Scholar 

  33. Kena, G., Hussar, W., McFarland, J., de Brey, C., Musu-Gillette, L., Wang, X., Zhang, J., Rathbun, A., Wilkinson-Flicker, S., Diliberti, M., Barmer, A., Bullock Mann, F., & Velez, Dunlop, E. (2016). The Condition of Education 2016 (NCES 2016-144). Washington, DC: U.S. Department of Education, National Center for Education Statistics. Retrieved October 11, 2016 from http://nces.ed.gov/pubsearch.

  34. Kennedy, M. M. (1997). The connection between research and practice. Educational Researcher, 26, 4–12.

    Article  Google Scholar 

  35. Kratochwill, T. R., & Stoiber, K. C. (2000). Diversifying theory and science: Expanding the boundaries of empirically supported interventions in school psychology. Journal of School Psychology, 38, 349–358.

    Article  Google Scholar 

  36. Latham, G. (1993). Do educators use the literature of the profession? National Association of Secondary School Principals Bulletin, 77, 63–67.

    Google Scholar 

  37. Long, A. C. J., Sanetti, L. M. H., Collier-Meek, M. A., Gallucci, J., Altschaefl, M., & Kratochwill, T. R. (2016). An exploratory investigation of teachers’ intervention planning and perceived implementation barriers. Journal of School Psychology, 55, 1–26.

    Article  PubMed  Google Scholar 

  38. Lysenko, L. V., Abrami, P. C., Bernard, R. M., Dagenais, C., & Janosz, M. (2014). Educational research in educational practice: Predictors of use. Canadian Journal of Education, 37, 1–26.

    Google Scholar 

  39. Miretsky, D. (2007). A view of research from practice: Voices of teachers. Theory Into Practice, 46, 272–280. https://doi.org/10.1080/00405840701593857.

    Article  Google Scholar 

  40. Neal, J. W., Neal, Z. P., Kornbluh, M., Mills, K. J., & Lawlor, J. A. (2015a). Brokering the research-practice gap: A typology. American Journal of Community Psychology, 56(3/4), 422–435. https://doi.org/10.1007/s10464-015-9745-8.

    Article  PubMed  Google Scholar 

  41. Neal, Z. P., Neal, J. W., Lawlor, J. A., & Mills, K. J. (2015b). Small worlds or worlds apart? Using network theory to understand the research-practice gap. Psychosocial Intervention, 24, 177–184. https://doi.org/10.1016/j.psi.2015.07.006.

    Article  Google Scholar 

  42. Neal, Z. P., Neal, J. W., Mills, K. J., & Lawlor, J. A. (in press). Making or buying evidence: Using transaction cost economics to understand decision making in public schools. Evidence & Policy. https://doi.org/10.1332/174426416X14778277473701.

  43. No Child Left Behind (NCLB) Act of 2001, 20 U.S.C.A. § p. 6301 et seq. (West 2003).

  44. Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015). Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 42, 533–544.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Reinke, W. M., Stormont, M., Herman, K. C., Puri, R., & Goel, N. (2011). Supporting children’s mental health in schools: Teacher perceptions of needs, roles, and barriers. School Psychology Quarterly, 26, 1–13. https://doi.org/10.1037/a0022714.

    Article  Google Scholar 

  46. Ringeisen, H., Henderson, K., & Hoagwood, K. (2003). Context matters: Schools and the “research to practice gap” in children’s mental health. School Psychology Review, 32, 153–168.

    Google Scholar 

  47. Rogers, E. M. (1995). Diffusion of innovations (4th edn.). New York: The Free Press.

    Google Scholar 

  48. Rones, M., & Hoagwood, K. (2000). School-based mental health services: A research review. Clinical Child and Family Psychology Review, 3, 223–241.

    CAS  Article  PubMed  Google Scholar 

  49. Tseng, V. (2012). The use of research in policy and practice. Society for Research on Child Development Social Policy Report, 26, 1–23.

    Google Scholar 

  50. U.S. Census Bureau (2014). 20102014 American Community Survey 5-Year Estimates. Retrieved from http://factfinder.census.gov.

  51. U.S. Census Bureau, Population Division. (2016). Annual Estimates of the Resident Population: April 1, 2010 to July 1, 2015. Retrieved from http://factfinder.census.gov.

  52. U.S. Department of Education, Safe and Drug-free Schools Program. (1998) Notice of final principles of effectiveness. Federal Register, 63, 29902–29906.

    Google Scholar 

  53. Walker, H. M. (2004). Commentary: Use of evidence-based interventions in schools: Where we’ve been, where we are, and where we need to go. School Psychology Review, 33, 298–407.

    Google Scholar 

  54. Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, L., et al. (2008). Bridging the gap between prevention research and practice: The interactive systems framework for dissemination and implementation. American Journal of Community Psychology, 41, 171–181.

    Article  PubMed  Google Scholar 

  55. Weiss, C. H. (1998). Have we learned anything new about the use of evaluation? American Journal of Evaluation, 19, 21–33.

    Article  Google Scholar 

  56. Weiss, C. H., & Bucavalas, M. J. (1980). Social science research and decision-making. New York: Columbia University Press.

    Google Scholar 

  57. Williams, D., & Coles, L. (2007). Teachers’ approaches to finding and using research evidence: An information literacy perspective. Educational Research, 49, 185–206.

    Article  Google Scholar 

  58. Wisdom, J. P., Chor, K. H. B., Hoagwood, K. E., & Horwitz, S. M. (2014). Innovation adoption: A review of theories and constructs. Administration and Policy in Mental Health and Mental Health Services Research, 41, 480–502. https://doi.org/10.1007/s10488-013-0486-4.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

This study was funded by Officer’s Research Award (#182241) and a Use of Research Evidence Award (#183010) from the William T. Grant Foundation. Additional support for this research also came from an R21 research grant from the National Institute of Mental Health (#1R21MH100238-01A1).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jennifer Watling Neal.

Ethics declarations

Conflict of interest

All authors of this manuscript (i.e., Jennifer Watling Neal, Zachary P. Neal, Jennifer A. Lawlor, Kristen J. Mills, and Kathryn McAlindon) declare that they have no conflicts of interest. This research was approved by Michigan State University’s IRB (#x12-1011e, #x14-706e, #x14-1173e).

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Appendix 1

Appendix 1

Interview Protocol

The full interview began with a series of questions asking the educator to describe a recent social skills program or practice that had been adopted or considered for adoption. These questions focused on how they (or their district) learned about the program, how they evaluated it for adoption, and who participated in the process. The questions in this section did not directly ask about “research” or “evidence” and the interviewer was instructed to not raise these issues.

The second part of the interview, which is the focus of this study, included the following questions about research:

figurea

The interview concluded with basic demographic questions about work history, education, and racial, ethnic, and gender identity.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Neal, J.W., Neal, Z.P., Lawlor, J.A. et al. What Makes Research Useful for Public School Educators?. Adm Policy Ment Health 45, 432–446 (2018). https://doi.org/10.1007/s10488-017-0834-x

Download citation

Keywords

  • Diffusion
  • Innovation
  • Research use
  • Districts
  • Schools
  • Directed content analysis