Advertisement

Studying Professional Development as Part of the Complex Ecosystem of STEM Higher Education

  • Nathan EmeryEmail author
  • Jessica Middlemis Maher
  • Diane Ebert-May
Article

Abstract

Professional development in teaching is a critical component of ongoing work to improve student learning outcomes in higher education, especially STEM education. While there are many large-scale professional development programs designed to help participants change the way STEM is taught, few have thoroughly evaluated the outcomes to determine whether faculty members have adopted new techniques and transferred what they learned to their teaching practice. Importantly, without substantive assessment of long-term professional development outcomes, we are left with little evidence of program effectiveness. In this article we examine the current state of professional development evaluation in STEM higher education, propose possible study design elements to use when investigating the impact of professional development on instructors, and describe a novel longitudinal research design for the evaluation of professional development activities.

Keywords

Professional development STEM Higher education Faculty Teaching 

Notes

Acknowledgements

The authors acknowledge funding from the National Science Foundation DUE 1623834 entitled “Collaborative Research: Investigating Contextual Factors that Impact Early-Career Faculty Teaching Practice,” participation from faculty members in the longitudinal study, and support from the departmental faculty and chairs in the participating institutions.

References

  1. Amundsen, C., & Wilson, M. (2012). Are we asking the right questions? A conceptual review of the educational development literature in higher education. Review of Educational Research, 82, 90–126.CrossRefGoogle Scholar
  2. Austin, A.E. (2011). Promoting evidence-based change in undergraduate science education. Washington, DC: National Academies National Research Council.Google Scholar
  3. Austin, A. E. (1996). Institutional and departmental cultures: The relationship between teaching and research. New Directions for Institutional Research, 1996(90), 57–66.CrossRefGoogle Scholar
  4. Bloom, H. S., Michalopoulos, C., & Hill, C. J. (2005). Using experiments to assess nonexperimental comparison-group methods for measuring program effects. In H. S. Bloom (Ed.), Learning more from social experiments: Evolving analytic approaches (pp. 173–235). New York, NY: Russell Sage Foundation.Google Scholar
  5. Borrego, M., & Henderson, C. (2014). Increasing the use of evidence-based teaching in STEM higher education: A comparison of eight change strategies. Journal of Engineering Education, 103, 220–252.CrossRefGoogle Scholar
  6. Brewer, C. A., & Smith, D. (2011). Vision and change in undergraduate biology education: A call to action. Washington, DC: American Association for the Advancement of Science.Google Scholar
  7. Brownell, S. E., & Tanner, K. D. (2012). Barriers to faculty pedagogical change: Lack of training, time, incentives, and tensions with professional identity? CBE Life Sciences Education, 11, 339–346.CrossRefGoogle Scholar
  8. Burdick, D., Doherty, T., & Schoenfeld, N. (2015). Encouraging faculty attendance at professional development events. To Improve the Academy, 34, 367–405.CrossRefGoogle Scholar
  9. Campbell, C. M., & O’Meara, K. A. (2014). Faculty agency: Departmental contexts that matter in faculty careers. Research in Higher Education, 55, 49–74.CrossRefGoogle Scholar
  10. Chalmers, D., & Gardiner, D. (2015). Studies in educational evaluation: An evaluation framework for identifying the effectiveness and impact of academic teacher development programmes. Studies in Educational Evaluation, 46, 81–91.CrossRefGoogle Scholar
  11. Connolly, M. R., Lee, Y. G., & Savoy, J. N. (2018). The effects of doctoral teaching development on early-career STEM scholars’ college teaching self-efficacy. CBE Life Sciences Education, 17, ar14.CrossRefGoogle Scholar
  12. Connolly, M. R., & Millar, S. B. (2006). Using workshops to improve instruction in STEM courses. Metropolitan Universities, 17, 53–65.Google Scholar
  13. Cook, T. D., Shadish, W. R., & Wong, V. C. (2008). Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within-study comparisons. Journal of Policy Analysis and Management, 27, 724–750.CrossRefGoogle Scholar
  14. Derting, T. L., & Ebert-May, D. (2010). Learner-centered inquiry in undergraduate biology: Positive relationships with long-term student achievement. CBE Life Sciences Education, 9, 462–472.CrossRefGoogle Scholar
  15. Derting, T. L., Ebert-may, D., Henkel, T. P., Maher, J. M., Arnold, B., & Passmore, H. A. (2016). Assessing faculty professional development in STEM higher education: Sustainability of outcomes. Science Advances, 2, e1501422.CrossRefGoogle Scholar
  16. Desimone, L. M. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38, 181–199.CrossRefGoogle Scholar
  17. Ebert-May, D., Derting, T. L., Henkel, T. P., Maher, J. M., Momsen, J. L., Arnold, B., & Passmore, H. A. (2015). Breaking the cycle: Future faculty begin teaching with learner-centered strategies after professional development. CBE Life Sciences Education, 14, ar22.CrossRefGoogle Scholar
  18. Ebert-may, D., Derting, T. L., Hodder, J., Momsen, J. L., Long, T. M., & Jardeleza, S. E. (2011). What we say is not what we do: Effective evaluation of faculty professional development programs. BioScience, 61, 550–558.CrossRefGoogle Scholar
  19. Entwistle, N., McCune, V., & Hounsell, J. (2002). Approaches to study and perceptions of university teaching–learning environments: Concepts, measures and preliminary findings. Edinburgh, Scotland: Enhancing Teaching-Learning Environments in Undergraduate Courses Project, University of Edinburgh, Coventry University, and Durham University.Google Scholar
  20. Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2011). How to design and evaluate research in education. New York, NY: McGraw-Hill Humanities/Social Sciences/Languages.Google Scholar
  21. Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111, 8410–8415.CrossRefGoogle Scholar
  22. Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38, 915–945.CrossRefGoogle Scholar
  23. Gast, I., Schildkamp, K., & van der Veen, J. T. (2017). Team-based professional development interventions in higher education: A systematic review. Review of Educational Research, 87, 736–767.CrossRefGoogle Scholar
  24. Gormally, C., Evans, M., & Brickman, P. (2014). Feedback about teaching in higher ed: Neglected opportunities to promote change. CBE Life Sciences Education, 13, 187–199.CrossRefGoogle Scholar
  25. Guskey, T. R. (2002). Professional development and teacher change. Teachers and Teaching: Theory and Practice, 8, 381–391.CrossRefGoogle Scholar
  26. Hall, G. E., & Hord, S. M. (1987). Change in schools: Facilitating the process. New York, NY: State University of New York Press.Google Scholar
  27. Handelsman, J., Ebert-May, D., Beichner, R., Bruns, P., Chang, A., DeHaan, R.,...Wood, W.B. (2004). Policy forum: Scientific teaching. Science, 304, 521–522.CrossRefGoogle Scholar
  28. Hearn, J. C., & Anderson, M. S. (2002). Conflict in academic departments: An analysis of disputes over faculty promotion and tenure. Research in Higher Education, 43, 503–529.CrossRefGoogle Scholar
  29. Henderson, C., Beach, A., & Finkelstein, N. (2011). Facilitating change in undergraduate STEM instructional practices: An analytic review of the literature. Journal of Research in Science Teaching, 48, 952–984.CrossRefGoogle Scholar
  30. Hines, S. R. (2015). Setting the groundwork for quality faculty development evaluation: A five-step approach. The Journal of Faculty Development, 29, 5–12.Google Scholar
  31. Hines, S. R. (2017). Evaluating centers for teaching and learning: A field-tested model. To Improve the Academy, 36, 89–100.CrossRefGoogle Scholar
  32. Hurney, C. A., Brantmeier, E. J., Good, M. R., Harrison, D., & Meixner, C. (2016). The faculty learning outcome assessment framework. The Journal of Faculty Development, 30, 69–77.Google Scholar
  33. Katz, D., & Kahn, R. L. (1978). The social psychology of organizations. New York, NY: John Wiley & Sons.Google Scholar
  34. Kirkpatrick, D. L. (1959a). Techniques for evaluating training programs. Journal of the American Society for Training and Development, 13, 3–9.Google Scholar
  35. Kirkpatrick, D. L. (1959b). Techniques for evaluating training programs: Part 2-learning. Journal of the American Society for Training and Development, 13, 21–26.Google Scholar
  36. Kirkpatrick, D. L. (1960a). Techniques for evaluating training programs: Part 3-behavior. Journal of the American Society for Training and Development, 14, 13–18.Google Scholar
  37. Kirkpatrick, D. L. (1960b). Techniques for evaluating training programs: Part 4-results. Journal of the American Society for Training and Development, 14, 28–32.Google Scholar
  38. Kneale, P., Winter, J., Turner, R., Spowart, L., & Muneer, R. (2016). Evaluating teaching development activities in higher education: A toolkit. York, United Kingdom: Higher Education Academy.Google Scholar
  39. Lingoes, J. C. (1968). The multivariate analysis of qualitative data. Multivariate Behavioral Research, 3, 61–94.CrossRefGoogle Scholar
  40. Lok, P., & Crawford, J. (1999). The relationship between commitment and organizational culture, subculture, leadership style and job satisfaction in organizational change and development. Leadership and Organization Development Journal, 20, 365–374.CrossRefGoogle Scholar
  41. Manduca, C. A., Iverson, E. R., Luxenberg, M., Heather Macdonald, R., McConnell, D. A., Mogk, D. W., & Tewksbury, B. J. (2017). Improving undergraduate STEM education: The efficacy of discipline-based professional development. Science Advances, 3, 1–16.CrossRefGoogle Scholar
  42. Peterson, M. W. (1976). The academic department: Perspectives from theory and research. New Directions for Institutional Research, 10, 21–38.CrossRefGoogle Scholar
  43. Pfund, C., Miller, S., Brenner, K., Bruns, P., Chang, A., Ebert-May, D.,... Handelsman, J. (2009). Summer institute to improve university science teaching. Science, 324, 470–471.CrossRefGoogle Scholar
  44. Phuong, T. T., Cole, S. C., & Zarestky, J. (2018). A systematic literature review of faculty development for teacher educators. Higher Education Research and Development, 37, 373–389.CrossRefGoogle Scholar
  45. Piburn, M., Sawada, D., Turley, J., Falconer, K., Benford, R., Bloom, I., & Judson, E. (2000). Reformed teaching observation protocol (RTOP) reference manual. Tempe, AZ: Arizona Collaborative for Excellence in the Preparation of Teachers.Google Scholar
  46. Porter, A. C. (1988). Understanding teaching - a model for assessment. Journal of Teacher Education, 39, 2–7.CrossRefGoogle Scholar
  47. Porter, S. R., Whitcomb, M. E., & Weitzer, W. H. (2004). Multiple surveys of students and survey fatigue. New Directions for Institutional Research, 2004, 63–73.CrossRefGoogle Scholar
  48. Robinson, O. C. (2014). Sampling in interview-based qualitative research: A theoretical and practical guide. Qualitative Research in Psychology, 11, 25–41.CrossRefGoogle Scholar
  49. Saxon, D., Garratt, D., Gilroy, P., & Cairns, C. (2003). Collecting data in the information age: Exploring web-based survey methods in educational research. Research in Education, 69, 51–66.CrossRefGoogle Scholar
  50. Seymour, E. (2002). Tracking the processes of change in US undergraduate education in science, mathematics, engineering, and technology. Science Education, 86, 79–105.CrossRefGoogle Scholar
  51. Steinert, Y., Mann, K., Anderson, B., Barnett, B. M., Centeno, A., Naismith, L.,...Dolmans, D. (2016). A systematic review of faculty development initiatives designed to enhance teaching effectiveness: A 10-years update: BEME guide no. 40. Medical Teacher, 38, 142–159.CrossRefGoogle Scholar
  52. Stes, A., Clement, M., & Van Petegem, P. (2007). The effectiveness of a faculty training programme: Long-term and institutional impact. International Journal for Academic Development, 12, 99–109.CrossRefGoogle Scholar
  53. Stes, A., & Hoekstra, A. (2015). Convergence in diversity: Evaluating faculty development across the globe. Studies in Educational Evaluation, 46, 1–3.CrossRefGoogle Scholar
  54. Stes, A., Min-Leliveld, M., Gijbels, D., & Van Petegem, P. (2009). The impact of instructional development in higher education: The state-of-the-art of the research. Educational Research Review, 5, 25–49.CrossRefGoogle Scholar
  55. Stewart, M. (2014). Making sense of a teaching programme for university academics: Exploring the longer-term effects. Teaching and Teacher Education, 38, 89–98.CrossRefGoogle Scholar
  56. Tennill, M. M., & Cohen, M. W. (2013). 9: Assessing the long-term impact of a professional development program. To Improve the Academy, 32, 145–159.CrossRefGoogle Scholar
  57. Trigwell, K., & Prosser, M. (2004). Development and use of the approaches to teaching inventory. Educational Psychology Review, 16, 409–424.CrossRefGoogle Scholar
  58. Truong, M. H., Juillerat, S., & Gin, D. H. (2016). Good, fast, cheap: How centers of teaching and learning can capitalize in today's resource-constrained context. To Improve the Academy, 35, 180–195.CrossRefGoogle Scholar
  59. Walczyk, J. J., Ramsey, L. L., & Zha, P. (2007). Obstacles to instructional innovation according to college sciences and mathematics faculty. Journal of Research in Science Teaching, 44, 85–106.CrossRefGoogle Scholar
  60. Walter, E. M., Beach, A. L., Henderson, C., Williams, C. T., & Ceballos Madrigal, I. (2019). Exploring organizational climate for teaching in postsecondary settings: The development and validation of the survey of climate for instructional improvement. Manuscript submitted for publication.Google Scholar
  61. White, R. T., & Arzi, H. J. (2005). Longitudinal studies: Designs, validity, practicality, and value. Research in Science Education, 35, 137–149.CrossRefGoogle Scholar
  62. Williams, C. T., Walter, E. M., Henderson, C., & Beach, A. L. (2015). Describing undergraduate STEM teaching practices: A comparison of instructor self-report instruments. International Journal of STEM Education, 2, 18.CrossRefGoogle Scholar
  63. Wright, M. C., Goldwasser, M., Jacobson, W., & Dakes, C. (2017). Assessment from an educational development perspective. To Improve the Academy, 36, 39–49.CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  • Nathan Emery
    • 1
    Email author
  • Jessica Middlemis Maher
    • 2
  • Diane Ebert-May
    • 1
  1. 1.Department of Plant BiologyMichigan State UniversityEast LansingUSA
  2. 2.Delta Program in Research, Teaching, and LearningUniversity of Wisconsin–MadisonMadisonUSA

Personalised recommendations