Abstract
Increasingly, postsecondary education institutions are articulating institutional learning outcomes that define core competencies and evidence student learning to justify their value; however, concerns have arisen due to lack of authenticity, faculty autonomy, and representation of diverse disciplinary perspectives and worldviews (Lucas in Policy and Society 33(3):215–224, 2014). To actively engage faculty in co-designing institutional rubrics and learning outcomes assessment, we designed an action research project with the goal of examining and improving assessment practices related to institutional learning outcomes. The project is now in its fourth action research cycle; this paper describes the first two years of trialing a faculty-led community of practice approach to course-embedded assessment we call Strategic Assessment of Institutional Learning. We further describe the landscape of institutional learning outcomes assessment, two iterative cycles of action research, and seven themes that surfaced during the project. These seven themes informed the direction of learning outcomes assessment at our university: 1) student informed consent, 2) trust and community, 3) assignment selection, 4) rubric clarity and disciplinary variation, 5) framing the degree of student achievement, 6) implications for course redesign, and 7) faculty motivation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
Access to data is restricted. As noted, due to challenges with consent rates and inter-rater reliability, we did not draw conclusions, in aggregate, of student achievement of institutional learning outcomes, yet. Access to faculty co-researchers' focus group debrief and collaborative self-study data can be made available on request by contacting the corresponding author Dr. Alana Hoare at ahoare@tru.ca.
References
Alstete, J. W. (1995). Benchmarking in higher education: Adapting best practices to improve quality. ASHE-ERIC Higher Education Report No. 5, 1–112.
Banta, T. W. & Blaich, C. (2010). Closing the assessment loop. Change: The Magazine of Higher Learning, 43(1), 22–27. https://doi.org/10.1080/00091383.2011.538642
Baume, D., & Popovic, C. (2016). Introduction: Some issues in academic development. In C. Popovic & D. Baume (Eds.), Advancing practice in academic development (pp. 1–16). Routledge.
Biggs, J. B. (2014). The reflective institution: Assuring and enhancing the quality of teaching and learning. Higher Education, 4(1), 221–238.
Bosman, L. & Voglewede, P. (2019). How can a faculty community of practice change classroom practices? College Teaching, 67(3), 177–187. https://doi-org.proxy1.lib.uwo.ca/10.1080/87567555.2019.1594149
Canner, J., Dahlen, S. P. C., Gage, O., Graff, N., Gregg, S., Shapiro, D. F., Patterson, V. W., & Wood, S. E. (2020). Engaging Faculty in Assessment of Institutional Learning Outcomes. Assessment Update, 32, 1–16. https://doi.org/10.1002/au.30210
Carr, W. & Kemmis, S. (1986). Becoming critical: Education, knowledge and action research. Falmer.
Costa, A., & Kallick, B. (1993). Through the lens of a critical friend. Educational Leadership, 51(2), 49–51.
Deller, F., Pichette, J., & Watkins, E. K. (2018). Driving academic quality: Lessons from Ontario’s skills assessment project. Higher Education Quality Council of Ontario. https://heqco.ca/priorities/learning-outcomes/learning-outcomes-assessment-consortium/driving-academic-quality-summarizes-lessons-from-heqcos-learning-outcomes-assessment-consortium/
DiPietro, C., Dyjur, P., Fitzpatrick, K., Grant, K., Hoessler, C., Kalu, F., Richards, J., Skene, A., & Wolfe, P. (2022). Educational development guide series: No. 4. A comprehensive guide to working with higher education curriculum development, review and renewal projects. (P. Dyjur & A. Skene, Eds.). Educational Developers Caucus.
Dzimiska, M., Fijalkowska, J., & Sulkowski, L. (2018). Trust-based quality culture conceptual model for higher education institutions. Sustainability, 10(8), 2599, 1–22. https://doi.org/10.3390/su10082599
Eaton, J. S. (2021). The Role of Quality Assurance and the Values of Higher Education. In: van't Land, H., Corcoran, A., Iancu, DC. (eds) The Promise of Higher Education. Springer, Cham. https://doi.org/10.1007/978-3-030-67245-4_28
Ewell, P. T. (2009). Assessment, accountability, and improvement: Revisiting the tension, Occasional Paper #1. National Institute for Learning Outcomes and Assessment. https://www.learningoutcomesassessment.org/documents/PeterEwell_008.pdf
Harrison, M. & Lockwood, B. (2001). What price teaching quality. http://www2.warwick.ac.uk/fac/soc/economics/staff/academic/harrison/comment/quality.pdf
Hoessler, C. & Hoare, A. (2022). Strategic assessment of institutional learning: Practitioner handbook. https://sail.pressbooks.tru.ca/
Jackson, S., & Bohrer, J. (2010). Quality assurance in higher education: Recent developments in the United Kingdom. Research in Comparative and International Education, 5(1), 77–87.
Jeppesen, A., Hoessler, C., Fewer, J., & Mulhall, S. (2019). Responding to Outcome-Based Curriculum Development. Transformative Dialogues, 12(1). https://www.kpu.ca/sites/default/files/Transformative%20Dialogues/TD.12.1_Jeppesen_Hossler_Fewer%26Mulhall_Outcome-Based_Curriculum_Development.pdf
Kemmis, S. & McTaggart, R. (2007). The action research planner: Doing critical participatory action research. SAGE.
Kezar, A., & Eckel, P. D. (2015). The effect of institutional culture on change strategies in higher education: Universal principles or culturally responsive concepts? The Journal of Higher Education, 73(4), 436–460. https://doi.org/10.1080/00221546.2002.11777159
Koshy, E., Koshy, V., & Waterman, H. (2010). Action research in healthcare. SAGE.
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge University Press. https://doi.org/10.1017/CBO9780511815355
Lucas, L. (2014). Academic resistance to quality assurance processes in higher education in the UK. Policy and Society, 33(3), 215–224. https://doi.org/10.1016/j.polsoc.2014.09.006
Madsen, S. S., Habbestad, H., & Borch, I. H. (2023). Valuable unintended learning outcomes when practicum for student teachers in kindergartens is carried out online. Education and Information Technologies, 28, 37–55.
Maki, P. L. (2002). Developing an assessment plan to learn about student learning. The Journal of Academic Librarianship, 28(21), 8–13.
Manfra, M. M. (2019). Action research and systematic, intentional change in teaching practice. Review of Research in Education, 43(1), 163–196. https://doi.org/10.3102/0091732X18821132
Massa, L. J. & Kasimatis, M. (2017). Meaningful and manageable program assessment: A how-to guide for higher education faculty. Stylus.
McDaniel, E. A., Felder, B. D., Gordon, L., Hrutka, M. E., & Quinn, S. (2000). New faculty roles in learning outcome education: The experiences of four models and institutions. Innovative Higher Education, 25(2), 143–157. https://doi.org/10.1023/A:1007529007265
Ministry of Advanced Education, Skills & Training, British Columbia. (2018, August). Quality assurance process audit handbook. https://www2.gov.bc.ca/gov/content/education-training/post-secondary-education/institution-resources-administration/degree-authorization/degree-quality-assessment-board/quality-assurance-process-audit
Prinsloo, P., & Slade, S. (2015). Student privacy self-management: Implications for learning analytics. Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 83–92). ACM. https://doi.org/10.1145/2723576.2723585
Pyrko, I., Dorfler, V., & Eden, C. (2017). Thinking together: What makes communities of practice work? Human Relations, 70(4), 389–409. https://doi.org/10.1177/0018726716661040
Reason, P. & Bradbury, H. (2008) The SAGE handbook of action research: Participative inquiry and practice (2nd edition). SAGE.
Reich, A. Z., Collins, G. R., DeFranco, A. L., & Pieper, S. L. (2019). A recommended closed-loop assessment of learning outcomes process for hospitality programs: The experience of two programs, Part 1. International Hospitality Review, 33(1), 41–52. https://doi.org/10.1108/IHR-09-2018-0010
Schoepp, K., & Benson, S. (2016). Meta-assessment: Assessing the learning outcomes assessment program. Innovative Higher Education, 41, 287–301. https://doi.org/10.1007/s10755-015-9347-2
Schoepp, K., & Tezcan-Unal, B. (2016). Examining the effectiveness of a learning outcomes assessment program: A four frames perspective. Innovative Higher Education, 42, 305–319. https://doi.org/10.1007/s10755-016-9384-5
Solove, D. (2013). Introduction: Privacy self-management and the consent dilemma. In 126 Harvard Law Review, 1880, (p. 1882), http://ssrn.com/abstract=2171018
Tsai, Y.-S., Moreno-Marcos, P. M., Jivet, I., Scheffel, M., Tammets, K., Kollom, K., & Gašević, D. (2018). The SHEILA Framework: Informing Institutional Strategies and Policy Processes of Learning Analytics. Journal of Learning Analytics, 5(3), 5–20. https://doi.org/10.18608/jla.2018.53.2
Wenger, E. (1998). Communities of practice: Learning, meaning and identity. Cambridge University Press.
Wenger, E., McDermott, R., & Snyder, W.M. (2002). Cultivating communities of practice. Harvard Business Review Press.
Wenger-Trayner, E. & Wenger-Trayner, B. (2015). Introduction to communities of practice: A brief overview of the concept and its uses. https://wenger-trayner.com/introduction-to-communities-of-practice/
Acknowledgements
We thank the university community for their engagement and support in this project, and for institutional funding for the SAIL research project.
Funding
Partial funding for professional development was provided to faculty involved in the first two pilots. All authors were employees of the institution at the time of the research. Authors have no other financial or non-financial interests that are directly or indirectly related to the study.
Author information
Authors and Affiliations
Contributions
The first two authors, Dr. Carolyn Hoessler and Dr. Alana Hoare, were the co-principal investigators for the first two action research cycles of the SAIL project. At the time of publication, we are now in our fourth action research cycle. All authors contributed to co-identifying themes and writing sections individually or in pairs as co-investigators. The first two authors guided the writing process, edited and prepared the final manuscript.
Corresponding author
Ethics declarations
Ethical Approval
Study received Ethical approval by the institutional Research Ethics Board (details in the manuscript text). Students provided consent for their data (opt-in consent in year 1 or opt-out in year 2 as designed with students), faculty were co-researchers on the pilots.
Competing Interest
The authors have no competing interests to declare that are relevant to the content of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Hoessler, C., Hoare, A., Austin, LA. et al. Faculty in Action: Researching a Community of Practice Approach to Institutional Learning Outcomes Assessment. J Form Des Learn 7, 171–181 (2023). https://doi.org/10.1007/s41686-023-00084-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41686-023-00084-6