Abstract
This article explores the organizational impediments and facilitators that influence the implementation of student learning outcomes assessment (SLOA). This review points to the importance of culture, leadership, and organizational policies to the implementation of SLOA. However, we need to approach research differently, both conceptually and methodologically, if we want to understand these key factors better. I argue that our understanding of implementation conditions is superficial due to systemic weaknesses in the research. The article provides a framework for defining these terms clearly, suggests theories that can be applied, and reviews key methodological changes that can improve the quality of research.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Some studies take a broader definition and look at leadership as persons or groups who can mobilize human, material, and symbolic resources toward specific ends (Gray 1997).
This is more definitive as there are multiple, large scale studies that have identified this finding.
A few areas have mixed results and are therefore not reported on, but I mention them in this footnote. There is a tension between whether the central administration or departments /decentralized units should lead data collection efforts. Research findings are mixed in this area, and these findings may relate to institutional differences (e.g., research universities being more decentralized while small liberal arts campuses can be successful with more centralized processes). Some studies have found that centralized support and offices can facilitate assessment practices on campus (Banta et al. 1996; Peterson et al. 1999). Other studies have identified decentralizing assessment efforts as creating more buy-in and more assessment activities (Peterson et al. 1999). Data seems to favor a decentralized decision-making processes as better for implementation of assessment efforts (Peterson et al. 1999). Some researchers suggest that patterns of centralization and decentralization should vary over the implementation of assessment or depending on the outcome pursued (e.g., assessing individual students or a sustained large population such as undergraduates) (Patton et al 1996; Volkwein 2010). Another administrative structure that has been examined is whether assessment is located within academic or student affairs, institutional research, or some other functional area. Placement within academic affairs is expected to contribute most to building internal support for student assessment (Ewell 1988a). There is limited empirical support for whether the location administratively of student assessment efforts makes a difference. Also, the administrative location has not been studied in conjunction with the use of campus wide teams.
It is important to note that theories must be brought to bear carefully, with a full understanding of the concepts. It should be noted that cultural theories were brought to the study of SLOA and in some ways have provided an important framing of the issue. Cultural theories helped direct people to explore culture, to think more broadly than a quick policy fix, and to think about implementation of assessment as requiring a new set of values and as being a paradigm shift for campuses. In these ways, it has been extremely important. The problem is that various researchers used differing cultural theories that were emerging in the 1980s and 1990s somewhat uncritically – without examining the assumptions, comparing various theories, or placing their studies in the context of other theories. Cultural theories were also often used tacitly and not explicitly drawing from any particular theory or set of culture assumptions.
Care must be taken though as cultural theories were misapplied to the study of SLOA, and what started as an overarching framework to examine leadership and organizational polices and structures became too loosely applied and the concepts all merged. The same problem can occur with distributed leadership.
References
Andrade, M. (2011). Managing Change—Engaging Faculty in Assessment Opportunities. Innovative Higher Education., 36(4), 217–23.
Aper, J. P. (1993). Higher education and the state: Accountability and the roots of student outcomes assessment. Higher Education Management, 5, 365–376.
Astin, H., & Leland, C. (1991). Women of influence, Women of vision. San Francisco, CA: Jossey Bass.
Banta, T. (1997). Moving assessment forward: Enabling conditions and stumbling blocks. In P. J. Gray & T. W. Banta (Eds.), The campus-level impact of assessment: Progress, problems. New Directions for Higher Education, #100 (pp. 79–91). San Francisco, CA: Jossey Bass.
Banta, T. W., Lund, J. P., Black, K. E., & Oblander, F. W. (Eds.). (1996). Assessment in practice: Putting principles to work on college campuses. San Francisco, CA: Jossey-Bass.
Bensimon, E. (2005). Closing the achievement gap in higher education: An organizational learning perspective. In A. Kezar (Ed.), Higher education as a learning organization: Promising concepts and approaches. New Directions for Higher Education,131 (pp. 99–112). San Francisco, CA: Jossey-Bass.
Bess, J., & Dee, J. (2008). Understanding college and university organization. Sterling, VA: Stylus.
Birnbaum, R. (2001). Management fads in higher education. San Francisco, CA: Jossey Bass.
Boud, D. (2002). Sustainable assessment: Rethinking assessment for the learning society. Studies in Continuing Education, 22(2), 151–167.
Braskamp, L. A. (1991). Purposes, issues, and principles of assessment. NCA Quarterly, 66(2), 417–429.
Brewer, P., Denney, L., & Struhar, W. (1997). General education assessment: Starting and restarting. Dayton, OH: Sinclair Community College.
Cowart, S. C. (1990). A survey on using student outcomes measures to assess institutional effectiveness. Iowa City, IA: American College Testing Program.
Eisenman, C. D. (1991). Faculty participation in assessment programs. NCA Quarterly, 66(2), 458–464.
Ewell, P. T. (1987). Establishing a campus-based assessment program. In D. F. Halpern (Ed.), Student outcomes assessment: What institutions stand to gain. New Directions for Higher Education, 59 (pp. 9–24). San Francisco, CA: Jossey-Bass.
Ewell, P. T. (1988a). Implementing assessment: Some organizational issues. In T. W. Banta (Ed.), Implementing outcomes assessment: Promise and perils. New Directions for Institutional Research, 59 (pp. 15–28). San Francisco, CA: Jossey-Bass.
Ewell, P. T. (1997). Strengthening assessment for academic quality improvement. In M. W. Peterson, D. D. Dill, L. A. Mets, & Associates (Eds.), Planning and management for a changing environment: A handbook on redesigning postsecondary institutions (pp. 360–381). San Francisco, CA: Jossey-Bass.
Ewell, P., & Jones, D. (1993). Action matters: The case for indirect measures in assessing higher education’s national education goals. The Journal of General Education, 42(2), 123–148.
Francis, P., & Steven, D. (2003). The SUNY assessment initiative: Initial campus and system perspectives. Assessment and Evaluation in Higher Education, 28(3), 333–349.
Fraser, S. (2008). Leadership and assessment: Strengthening the nexus. Final report for the Australian Learning and Teaching Council. Sydney, Australia: Macquarie Press.
Gibbs, G. (1995). Assessing Student Centered Courses. Oxford, England: Oxford Centre for Staff Development.
Gill, W. E. (1993, June). Conversations about accreditation: Middle States Association of Colleges and Schools. Focusing on outcomes assessment in the accreditation process. Paper presented at the Double Feature Conference on Assessment and Continuous Quality Improvement of the American Association for Higher Education, Chicago, IL.
Gray, P. (1997). Viewing assessment as an innovation: Leadership and the change process. In P. J. Gray & T. W. Banta (Eds.), The campus-level impact of assessment: Progress problems and possibilities. New Directions for Higher education, 100 (pp. 5–15). San Francisco, CA: Jossey-Bass.
Hadden, C., & Davies, T. (2002). From innovation to institutionalization: The role of administrative leadership in the assessment process. Community College Journal of Research and Practice, 26, 243–260.
Huba, M., & Freed, J. (2000). Learner-centered assessment on college campuses. Boston, MA: Allyn & Bacon.
Huber, G. (1991). Organizational learning: The contributing processes and the literature. Organization Science, 2(2), 88–115.
Hutchings, P., & Marchese, T. (1990). Watching assessment: Questions, stories, prospects. Change, 22(5), 12–38.
Kezar, A. (Ed.). (2005). Higher education as a learning organization: Promising concepts and approaches. New Directions for Higher Education, 131. San Francisco, CA: Jossey-Bass.
Komives, S., & Wegner, W. (2009). Leadership for a better world: Understanding the social change model of leadership. San Francisco, CA: Jossey-Bass.
Kuh, G., & Ikenberry, S. (2009). More than you think, less than we need: Learning outcomes assessment in American Higher Education. National Institute for Learning Outcomes. Retrieved from http://www.learningoutcomesassessment.org/NILOAsurveyresults09.htm
Lind, C. J., & McDonald, M. (2003). Creating an assessment culture: A case study of success and struggles. In S. Van Kollenberg (Ed.), Collection of Papers in Self Study and Institutional Improvement, 3 (pp. 13–22). Chicago, IL: Higher Learning Commission.
Magruder, W. J., & Young, C. C. (1993). Value-added talent development in general education. In T. W. Banta, J. P. Lund, K. E. Black, & F. W. Oblander (Eds.), Assessment in practice: Putting principles to work on college campuses (pp. 169–171). San Francisco, CA: Jossey-Bass.
Magruder, J., McManis, M., & Young, C. (1997). The right idea and the right time: Development of a transformational assessment culture. In P. J. Gray & T. W. Banta (Eds.), The campus -level impact of assessment: Progress problems and possibilities, 100 (pp. 17–29). San Francisco, CA: Jossey Bass.
Maki, P. (2010). Coming to terms with student outcomes assessment: Faculty and administrators integrating assessment in their work and institutional culture. Sterling: VA, Stylus.
Mather, J. (1991). Accreditation and assessment: A staff perspective. NCA Quarterly, 66(2), 397–405.
Mentkowski, M. (1991). Creating a context where institutional assessment yields educational improvement. Journal of General Education, 40, 255–283.
Muffo, J. A. (1992). The status of student outcomes assessment at NASULGC member institutions. Research in Higher Education, 33, 765–774.
Palomba, C., & Banta, T. (2001). Assessing student competence in accredited disciplines. San Francisco, CA: Jossey Bass.
Patton, G. W., Dasher-Alston, R., Ratteray, O. M. T., & Kait, M. B. (1996). Outcomes assessment in the Middle States Region: A report on the 1995 outcomes assessment survey. Philadelphia, PA: Commission on Higher Education of the Middle States Association of Colleges and Schools.
Peterson, M. W., Einarson, M. K., Augustine, C. H., & Vaughan, D. S. (1999). Institutional support for student assessment: Methodology and results of a national survey. Stanford, CA: Stanford University, National Center for Postsecondary Improvement.
Peterson, M., & Augustine, C. (2000). External and internal influences on institutional approaches to student assessment” Accountability or improvement? Research in Higher Education, 41(4), 443–479.
Peterson, M. W., & Einarson, M. K. (2000). An analytic framework of institutional support for student assessment. In J. Smart (Ed.), Higher education: Handbook of theory and research, XV (pp. 219–267). New York, NY: Agathon Press.
Peterson, M., & Einarson, M. K. (2001). What are colleges doing about student assessment? Does it make a difference? The Journal of Higher Education, 72(6), 629–669.
Schein, E. (1985). Organizational culture and leadership. San Francisco, CA: Jossey Bass.
Seagraves, B., & Dean, L. (2010). Conditions supporting a culture of assessment in student affairs as small colleges and universities. Journal of Student Affairs Research, 47(3), 303–320. doi:10.2202/1949-6605.6.
Sell, G. (1989). An organizational perspective for effective practice of assessment. New Directions, (67), 21–41.
Serban, A. (2004). Assessment of student learning outcomes at the institutional level. Developing and Implementing assessment of student learning outcomes. New Directions for Community Colleges, 126, 17–27.
Smith, D., & Paker, S. (2005). Organizational learning: A tool for diversity and institutional effectiveness. In A. Kezar (Ed.), Higher education as a learning organization: Promising concepts and approaches. New Directions for Higher Education, 131 (pp. 113–126). San Francisco, CA: Jossey-Bass.
Stogdill, R. (1974). The handbook of leadership. New York, NY: Free Press.
Volkwein, F. (2010). In F. Volkwein (ed), Special issue: Assessing student outcomes. Issue no. s1. A model for assessing institutional effectiveness. New Directions for Institutional Research, 1,13-28.
Volkwein, F., Lattuca, L., Harper, B., & Domingo, R. (2006). Measuring the impact of professional accreditation on student experiences and learning outcomes. Research in Higher Education, 48(2), 251–282.
Wang, X., & Hurley, S. (2012). Assessment as a scholarly activity?: Faculty perceptions of and willingness to engage in student learning assessment. The Journal of General Education, 61(1), 1–15.
Weick, K. (1995). Sensemaking in organizations. Thousand Oaks, CA: Sage.
Williford, A. (1997). Ohio University’s multidimensional institutional impact and assessment plan. In P. J. Gray & T. W. Banta (Eds.), The campus -level impact of assessment: Progress problems and possibilities. 100 vol. 100 of what??? (pp. 32–41). San Francisco, CA: Jossey Bass.
Wojcik, R., Martin, E., & Kennedy, J. (2003). From crisis to culture: Inspiring, cultivating and guiding a culture of assessment. In S. Van Kollenberg (Ed.), Collection of Papers in Self Study and Institutional Improvement, 3 (pp. 27–30). Chicago, IL: Higher Learning Commission.
Wolff, R. A., & Harris, O. D. (1995). Using assessment to develop a culture of evidence. In D. Halpern (Ed.), Changing college classrooms: New teaching and learning strategies. San Francisco, CA: Jossey Bass.
Young, C., & Knight, M. (1993). Providing leadership for organizational change. In T. Banta & Associates (Eds.), Making a difference (pp. 25–39). San Francisco, CA: Jossey Bass.
Other Sources
Astin, A. W. (1991). Assessment for excellence: The philosophy and practice of assessment and evaluation in higher education. New York, NY: American Council on Education/Macmillan.
Banta, T. W. (1991). Contemporary approaches to assessing student achievement of general education outcomes. The Journal of General Education, 40, 203–223.
Banta, T. W., & Associates (Eds.). (1993). Making a difference: Outcomes of a decade of assessment in higher education. San Francisco, CA: Jossey-Bass.
Berquist, W. (2007). The six cultures of the academy. San Francisco, CA: Jossey Bass.
Biggs, J. (1999). Teaching for quality learning at the University. Buckingham, England: SRHE and Open University Press.
Broadfoot, P. (1998). Quality standards and control in higher education: What price life-long learning? International Studies in Sociology of Education, 8(2), 155–180.
Cole, J. J. K., Nettles, M. T., & Sharp, S. (1997). Assessment of teaching and learning for improvement and accountability: State governing, coordinating board and regional accreditation association policies and practices. Ann Arbor, MI: University of Michigan, National Center for Postsecondary Improvement.
Cook, C. E. (1989). FIPSE's role in assessment: Past, present, and future. Assessment Update, 1(2), 1–3.
Education Commission of the States. (1991). Assessing college outcomes: What state leaders need to know. Denver, CO: Author.
Ewell, P. T. (1988b). Outcomes, assessment, and academic improvement: In search of usable knowledge. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. IV, pp. 53–108). New York, NY: Agathon Press.
Frye, J. H. (1994). Educational paradigms in the professional literature of the community college. In J. C. Smart (Ed.), Higher education: Handbook of theory and research, Vol. X (pp. 181–224). New York, NY: Agathon Press.
Gray, P. J., & Banta, T. W. (Eds.). (1997). The campus-level impact of assessment: Progress problems and possibilities. New Directions for Higher Education,100). San Francisco, CA: Jossey-Bass.
Gray, P., Patil, A., & Codner, G. (2008). The background of quality assurance in higher education and engineering education. In A. Patil & P. Gray (Eds.), Engineering education quality assurance: A global perspective. New York, NY: Springer.
Heifetz, R. A. (1994). Adaptive leadership: Tools and tactics for changing your organization and the world. Cambridge, MA: Harvard Press.
Jankowski, N. & Makela, J. Exploring the landscape: What institutional websites reveal about student learning outcomes assessment activities. National institute for Learning Outcomes. Retrieved from http://www.learningoutcomesassessment.org/NILOAwebscanresults2010.htm
Kells, H. R. (1992). An analysis of the nature and recent development of performance indicators in higher education. Higher Education Management, 4(2), 131–138.
Kezar, A., Lester, J., Glenn, W., & Nokamato, J. (2008). Examining contextual features that affect implementation of equity initiatives. Journal of Higher Education., 79(2), 125–159.
Freeman, L. (2006). The Development of Social Network Analysis. Vancouver, Canada: Empirical Press.
Loacker, G., & Mentkowski, M. (1993). Creating a culture where assessment improves learning. In T. W. Banta & Associates (Eds.), Making a difference: Outcomes of a decade of assessment in higher education (pp. 5–24). San Francisco, CA: Jossey-Bass.
Maki, P. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling: VA, Stylus.
Marchese, T. (1987). Third down, ten years to go. AAHE Bulletin, 40(4), 3–8.
Nonaka, I., & Takeuchi, H. (1995). The knowledge creating company. New York, NY: Oxford University Press.
Nunley, C., Bers., T, & Manning, T. (2011). Learning outcomes assessment in community colleges. NILOA Occasional paper.
Peterson, M. W. (1998a). Assessing institutional support for student assessment. In T. W. Banta (Ed.), Assessment Update, 10 (4) (pp. 1–16). San Francisco, CA: Jossey-Bass.
Peterson, M. W., Einarson, M. K., & Augustine, C. H. (1997). Inventory of institutional support for student assessment. National survey instrument. Stanford, CA: Stanford University, National Center for Postsecondary Improvement.
Peterson, M. W., Augustine, C. H., Einarson, M. K., & Vaughan, D. S. (1999a). Designing student assessment to strengthen institutional performance in associate of arts institutions. Stanford, CA: Stanford University, National Center for Postsecondary Improvement.
Peterson, M. W., Augustine, C. H., Einarson, M. K., & Vaughan, D. S. (1999b). Designing student assessment to strengthen institutional performance in baccalaureate institutions. Stanford, CA: Stanford University, National Center for Postsecondary Improvement.
Peterson, M. W., Einarson, M. K., Augustine, C. H., & Vaughan, D. S. (1999). Institutional support for student assessment: Methodology and results of a national survey. Stanford, CA: Stanford University, National Center for Postsecondary Improvement.
Peterson, M. W., Einarson, M. K., Trice, A. G., & Nichols, A. R. (1997a). Improving organizational and administrative support for student assessment: A review of the research literature. Stanford, CA: Stanford University, National Center for Postsecondary Improvement.
Peterson, M. W., Einarson, M. K., Trice, A. G., & Nichols, A. R. (1997b). An analytic framework of institutional support for student assessment. Stanford, CA: Stanford University, National Center for Postsecondary Improvement.
Richardson, R. (1993). Creating effective learning environments. Denver, CO: Education Commission of the States.
Schlitz, S., O’Connor, M., Pang, Y., Stryker, D., Markell, S., Krupp, E., Byers, C., Jones, S., & Redfern, A. (2012). Developing a Culture of Assessment through a Self-formed Faculty Learning Community. International Journal of Teaching and Learning in Higher Education, 21(1), 133–147.
Terenzini, P. T. (1989). Assessment with open eyes: Pitfalls in studying student outcomes. Journal of Higher Education, 60, 644–664.
Wolff, R. A. (1992). CSU and assessment–second down and eight yards to go: A view from the scrimmage line. In Student outcomes assessment: What makes it work? (pp. 73-80). Long Beach, CA: California State University, Institute for Teaching and Learning.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kezar, A. Institutionalizing Student Outcomes Assessment: The Need for Better Research to Inform Practice. Innov High Educ 38, 189–206 (2013). https://doi.org/10.1007/s10755-012-9237-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10755-012-9237-9