Accountability systems, in various and evolving designs, have struggled to spur increases in the quality, efficiency, and productivity of higher education institutions. The effort to identify “what works” and disseminate higher education “best practices” to improve institutional effectiveness is ongoing under numerous accountability, assessment, and research initiatives. To characterize what we know and what we need to know from a “scholarship of best practices,” this paper analyzes the methods and goals of contemporary academic and institutional research investigating the effectiveness of collegiate educational practices.
To address our concern that the search for “best practices” in its current form will be ineffectual, we propose the creation of evidence-based inquiry councils (EBICs) as a central feature of a comprehensive accountability system designed to integrate knowledge of institutional contexts, educational processes, and learning outcomes for the purpose of increasing the educational effectiveness of colleges and universities.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
2005 Lumina Foundation annual report. (2006). Indianapolis, IN: Lumina Foundation for Education.
Abelmann, C., & Elmore, R. (1999). When accountability knocks, will anyone answer? Philadelphia: Consortium for Policy Research in Education, University of Pennsylvania.
Accountability for better results: A national imperative for higher education. (Report of the National Commission on Accountability in Higher Education)(2005). Report of the National Commission on Accountability in Higher Education). Denver, CO: State Higher Education Executive Officers.
Alexander, F. K. (2000). The changing face of accountability. Journal of Higher Education, 71(4), 411.
ALFI Assessment Toolkit. (n.d.). Retrieved May 14, 2005, from http://www.cael.org/alfi/isas.html
Alon, S. (2005). Model mis-specification in assessing the impact of financial aid on academic outcomes. Research in Higher Education, 46(1), 109–125.
Ayers, D. F. (2005). Neoliberal ideology in community college mission statements: A critical discourse analysis. Review of Higher Education, 28(4), 527–549.
Bailey, T., Calcagno, J. C., Jenkins, D., Leinbach, T., & Kienzl, G. (2006). Is student right to know all you should know? An analysis of community college graduation rates. Research in Higher Education, 47(5), 491–519.
Bailey, T., Jenkins, D., & Leinbach, T. (2005). Graduation rates, student goals, and measuring community college effectiveness (CCRC Brief No. 28). New York City: Community College Research Center, Teachers College, Columbia University.
Bannan-Ritland, B. (2003). The role of design in research: The integrative learning design framework. Educational Researcher, 32(1), 21–24.
Banta, T. W. (Ed.). (2004). Community college assessment. San Francisco: Jossey-Bass.
Barak, R. J., & Kniker, C. R. (2002). Benchmarking by state higher education boards. In B. E. Bender & J. H. Schuh (Eds.), Using benchmarking to inform practices in higher education (Summer ed., Vol. 118). San Francisco: Jossey Bass.
Bauman, G. L. (2005). Promoting organizational learning in higher education to achieve equity in educational outcomes In A. Kezar (Ed.), Higher education as a learning organization: Promising concepts and approaches (Vol. 131). San Francisco: Jossey Bass.
Bender, B. E., & Schuh, J. H. (Eds.). (2002). Using benchmarking to inform practice in higher education (Summer ed., Vol. 118). San Francisco: Jossey Bass.
Bensimon, E. M. (2004, January/ February). The diversity scorecard: A learning approach to institutional change. Change, 45–52.
Bensimon, E. M. (2005). Closing the achievement gap in higher education: An organizational learning perspective. In A. Kezar (Ed.), Higher education as a learning organization: Promising concepts and approaches (Vol. 131). San Francisco: Jossey-Bass.
Bensimon, E. M., Polkinghorne, D. E., Bauman, G. L., & Vallejo, E. (2004). Doing research that makes a difference. Journal of Higher Education, 75(1), 104–126.
Berliner, D. C. (2002). Educational research: The hardest science of all. Educational Researcher, 31(8), 18–20.
Bogue, F. G. (1998). Changing leadership metaphors: From a culture of blame to a culture of responsibility. Conference on Values in Higher Education. Knoxville: University of Tennessee.
Bollag, B. (2004, July 16). Opening the door on accreditation. Chronicle of Higher Education, p. 22.
Borden, V. M. H. (2004, March/ April). Accommodating student swirl: When traditional students are no longer the tradition. Change, 10–17.
Boulding, K. (1996). In praise of inefficiency. In D. W. Breneman, L. L. Leslie, & R. E. Anderson (Eds.), Finance in higher education pp. 415–418. Boston, MA: Pearson Custom Publishing.
Bowen, H. (1996). What determines the costs of higher education? In D. W. Breneman, L. L. Leslie, & R. E. Anderson (Eds.), Finance in higher education pp. 113–128). Boston, MA: Pearson Custom Publishing.
Bowen, W. G., Kurzweil, M. A., & Tobin, E. M. (2005). Equity and excellence in American higher education. Charlottesville: University of Virginia Press.
Burke, J. C. (2005a). The many faces of accountability. In J. C. Burke et al. (Eds.), Achieving accountability in higher education pp. 1–24. San Francisco: Jossey-Bass.
Burke, J. C. (2005b). Reinventing accountability: From bureaucratic rules to performance results. In J. C. Burke et al. (Eds.), Achieving accountability in higher education pp. 216–245. San Francisco: Jossey-Bass.
Burke, J. C. (2005c). The three corners of the accountability triangle: Serving all, submitting to none. In J. C. Burke et al. (Eds.), Achieving accountability in higher education pp. 296–324. San Francisco: Jossey-Bass.
Burke, J. C. et al. (2002). Funding public colleges and universities for performance: Popularity, problems, and prospects. Albany: Rockefeller Institute Press.
Burke, J. C., & Minassians, H. (2003). Performance reporting: “Real” accountability or accountability “lite” (Seventh Annual Survey). Albany: Nelson A. Rockefeller Institute of Government, State University of New York.
Callan, P. M., & Finney, J. E. (2005). State-by-state report cards: Public purposes and accountability for a new century. In J. C. Burke et al. (Eds.), Achieving accountability in higher education pp. 189–215. San Francisco: Jossey-Bass.
CCSSE (2004). About CCSSE. Retrieved September 14, 2004, from www.ccsse.org
Chatterji, M. (2005). Evidence on “what works”: An argument for extended-term mixed method (ETMM) evaluation designs. Educational Researcher, 34( 5), 14–24.
Clark, B. R. (1983). The higher education system: Academic organization in cross-national perspective. Berkeley: University of California Press.
Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32( 1), 9–13.
Cohen, D. K., Raudenbush, S. W., & Ball, D. L. (2003). Resources, instruction, and research. Educational Evaluation and Policy Analysis, 25( 2), 119–142.
Dellow, D. A., & Romano, R. M. (2002). Measuring outcomes: Is the first-time, full-time cohort appropriate for the community college? Community College Review, 30( 2), 42–54.
Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32( 1), 5–8.
DesJardins, S. L. (2002). Understanding and using efficiency and equity criteria in the study of higher education policy. In J. C. Smart & W. G. Tierney (Eds.), Higher education: Handbook of theory and research pp. 173–219. New York: Agathon Press.
DesJardins, S. L., Ahlburg, D. A., & McCall, B. P. (2006). An integrated model of application, admission, enrollment, and financial aid. Journal of Higher Education, 77( 3), 381–429.
Dill, D. D. (2003). Allowing the market to rule: the case of the United States. Higher Education Quarterly, 57( 2), 136–157.
DiMaggio, P. J., & Powell, W. W. (1983). The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological Review, 48, 147–160.
Doerfel, M. L., & Ruben, B. D. (2002). Developing more adaptive, innovative, and interactive organizations. In B. E. Bender & J. H. Schuh (Eds.), Using benchmarking to inform practices in higher education (Summer ed., Vol. 118). San Francisco: Jossey Bass.
Dougherty, K. J. (2002, April 1–5). Performance accountability and community colleges: Forms, impacts, and problems. Paper presented at the American Educational Research Association, New Orleans, LA.
Dougherty, K. J., & Hong, E. (2005). State systems of performance accountability for community colleges: Impacts and lessons for policymakers (Achieving the Dream Policy Brief ). Boston, MA: Jobs for the Future.
Dowd, A. C. (2003, March). From access to outcome equity: Revitalizing the democratic mission of the community college. Annals of the American Academy of Political and Social Science, 586, 92–119.
Dowd, A. C. (2004, May 12). Income and financial aid effects on persistence and degree attainment in public colleges. Retrieved May 12, 2004, from http://epaa.asu.edu/epaa/v12n21/
Dowd, A. C. (2005). Data don’t drive: Building a practitioner-driven culture of inquiry to assess community college performance (Research Report). Indianapolis, IN: Lumina Foundation for Education.
Dowd, A. C., Bensimon, E. M., & Gabbard, G. (2006). Transfer Access Self-Assessment Inventory [assessment instrument]. Los Angeles: Center for Urban Education, Rossier School of Education, University of Southern California.
Dowd, A. C., Bensimon, E. M., Gabbard, G., Singleton, S., Macias, E., Dee, J., et al. (2006). Transfer access to elite colleges and universities in the United States: Threading the needle of the American dream. Boston and Los Angeles: University of Massachusetts Boston and the University of Southern California.
Dowd, A. C., & Cheslock, J. J. (2006). An estimate of the two-year transfer population at elite institutions and of the effects of institutional characteristics on transfer access. Boston, MA and Tucson, AZ: University of Massachusetts Boston and University of Arizona.
Dwyer, C. A., Millett, C. M., & Payne, D. G. (2006). A culture of evidence: Postsecondary assessment and learning outcomes. Princeton, NJ: Educational Testing Service.
Dynarski, S. (2002a). The behavioral and distributional implications of aid for college. American Economic Review, 92( 2), 279–285.
Dynarski, S. (2002b). Loans, liquidity, and schooling decisions. Boston, MA: Harvard University, Kennedy School of Government.
Erickson, F., & Gutierrez, K. (2002). Culture, rigor, and science in educational research. Educational Researcher, 1( 8), 21–24.
Erisman, W., & Gao, L. (2006). Making accountability work: Community colleges and statewide higher education accountability systems. Washington, DC: Institute for Higher Education Policy.
Ewell, P. T. (1991). To capture the ineffable: New forms of assessment in higher education. Review of Research in Education, 17, 75–125.
Ewell, P. T. (2002). An emerging scholarship: A brief history of assessment. In T. W. Banta (Ed.), Building a scholarship of assessment. San Francisco, CA: Jossey Bass.
Feuer, M. J., Towne, L., & Shavelson, R. J. (2002). Scientific culture and educational research. Educational Researcher, 31( 8), 4–14.
Field, K. (2005a, October 14). Educators cast a wary eye at U. S. panel: Some fear federal intrusion into academe; others are pleased by the attention. Chronicle of Higher Education, p. A1.
Field, K. (2005b). “Political rigidity” in academe undermines federal support for higher education, senator tells commission. Retrieved December 13, 2005, from http://chronicle.com
Field, K. (2006). Draft report from federal panel takes aim at academe. Chronicle of Higher Education, p. A21.
Fleming, B. (2004, July 14). House republicans defend proposals for holding colleges more accountable. Retrieved July 14, 2004, from http://chronicle.com/
Gabbard, G., Singleton, S., Macias, E., Dee, J., Bensimon, E. M., Dowd, A. C., et al. (2006). Practices supporting transfer of low-income community college transfer students to selective institutions: Case study findings. Boston, MA and Los Angeles, CA: University of Massachusetts Boston and University of Southern California.
Greater expectations: A new vision for learning as a nation goes to college. (2002). Washington, D.C: Association of American Colleges and Universities.
Greene, J. C. (2000). Understanding social programs through evaluation. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research 2nd ed., pp. 981–999. Thousand Oaks: Sage Publications.
Gumport, P. J., & Pusser, B. (1995). A case of bureaucratic accretion: A case of context and consequences. Journal of Higher Education, 66( 5), 493–520.
Higher education revenues and expenditures: A study of institutional costs. (1998). Arlington, VA: Research Associates of Washington.
Hirschhorn, L. (1990). The workplace within: Psychodynamics of organizational life. Cambridge, MA: MIT Press.
Hurley, R. G. (2002). Identification and assessment of community college peer institution selection systems. Community College Review, 29( 4), 1–27.
Identifying and implementing educational practices supported by rigorous evidence: A user friendly guide. (n.d). Retrieved November 1, 2004, from http://www.ed.gov/print/rschstat/research/pubs/rigorousevid/guide/html
Indicators of Engagement Project. (n.d.). Retrieved September 14, 2004, from http://www.compact.org/indicators/indicators.html
Jaquette, O. (2006). Making performance accountability work: English lessons for U.S. community colleges (Achieving the Dream Policy Brief ). Boston, MA: Jobs for the Future.
Kaplan, R. S., & Norton, D. P. (1998). The balanced scorecard: Measures that drive performance. In H. B. Review (Ed.), On measuring corporate performance pp. 123–145. Boston: Harvard Business School Press.
Koestenbaum, P., & Block, P. (2001). Freedom and accountability at work: Applying philosophic insight to the real world. San Francisco: Jossey-Bass/Pfeiffer.
Kuh, G. D. (2001). Assessing what really matters to student learning. Change, 33, 10–29.
Kuh, G. D. (2005). Imagine asking the client: Using student and alumini surveys for accountability in higher education. In J. C. Burke et al. (Eds.), Achieving accountability in higher education pp. 148–172. San Francisco: Jossey-Bass.
Lagemann, E. C. (2002). Useable knowledge in education. Retrieved September 13, 2005, from http://www.spencer.org/publications/
Lather, P. (2004). This IS your father’s paradigm: Government intrusion and the case of qualitative research in education. Qualitative Inquiry, 10( 1), 15–34.
Lazerson, M. (n.d.). Discontent in the field of dreams: American higher education, 1945–1990. Stanford, CA: National Center for Postsecondary Improvement, Stanford University.
Lederman, D. (2006). Commission report, take 2. Retrieved July 29, 2006, from http://insidehighered.com
Leveille, D. E. (2005). An emerging view on accountability in American higher education (Center for Studies in Higher Education Research and Occasional Paper Series No. CSHE.8.05). Berkeley: University of California, Berkeley.
Levin, H. L., & McEwan, P. J. (2001). Cost-effectiveness analysis. Thousand Oaks: SagePublications.
Levin, H. L., & McEwan, P. J. (2002). Cost-effectiveness and educational policy. In H. L. Levin & P. J. McEwan (Eds.), Cost-effectiveness and educational policy pp. 1–17 Larchmont, NY: Eye on Education.
Levin, J. S. (2001). Globalizing the community college: Strategies for change in the twenty-first century. New York: Palgrave.
Lindblom, C. E., & Cohen, D. K. (1979). Usable knowledge: Social science and social problem solving. New Haven, CT: Yale University Press.
Linsenmeier, D. M., Rosen, H. S., & Rouse, C. E. (2001). Financial aid packages and college enrollment decisions: An econometric case study (Working Paper No. 459). Princeton, NJ: Princeton University, Industrial Relations Section.
Lipman-Blumen, J., & Leavitt, H. J. (1999). Hot groups. New York: Oxford University Press.
Maki, P. L. (2004). Assessing for learning. Herndon, VA: Stylus.
Marti, C. N. (2004). Overview of the CCSSE instrument and psychometric properties. Retrieved September 14, 2004, from http://www.ccsse.org/aboutsurvey/psychometrics.pdf
Massy, W. F. (2005). Academic audit for accountability and improvement. In J. C. Burke et al. (Eds.), Achieving accountability in higher education pp. 173–197. San Francisco: Jossey-Bass.
McEwan, E. K., & McEwan, P. J. (Eds.). (2003). Making sense of research: What’s good, what’s not, and how to tell the difference. Thousand Oaks, CA: Corwin Press.
Miller, D. L. (1998). Nothing to teach! No way to teach it! Together with the obligation to teach! Dilemmas in the rhetoric of assessment and accountability, Conference on values in higher education. University of Tennessee, Knoxville, TN: University of Tennessee.
Moore, W. S. (2002). Accountability is more than “accounting”: Promoting and sustaining institutional assessment-as-learning. Journal of Applied Research in the Community College, 9( 2), 89–96.
Our students best work: A framework of accountability worthy of our mission. (2004). Washington, D.C: Association of American Colleges and Universities.
Palmer, P. (1998). The courage to teach: Exploring the inner landscape of a teacher’s life. San Francisco: Jossey-Bass.
Pena, E. V., Bensimon, E. M., & Colyar, J. (2006). Contextual problem defining: Learning to think and act. Liberal Education, 92( 2), 48–55.
Peterson, M. W., & Einarson, M. K. (2001). What are colleges doing about student assessment? Does it make a difference? Journal of Higher Education, 72( 6), 629–669.
Polkinghorne, D. E. (2004). Practice and the human sciences: The case for a judgment-based practice of care. Albany: State University of New York Press.
Pusser, B., & Turner, J. K. (2004, March/April). Student mobility: Changing patterns challenging policymakers. Change, 36–43.
Raudenbush, S. W. (2005). Learning from attempts to improve schooling: The contribution of methodological diversity. Educational Researcher, 25–31.
Richardson, R. C., & Ramirez, A. (2005). Policies and performance: The U.S. cross-case analysis. New York The Alliance for International Higher Education Policies, New York University.
Richardson, R. C., & Smalling, T. R. (2005). Accountability and governance. In J. C. Burke et al. (Eds.), Achieving accountability in higher education pp. 55–77. San Francisco: Jossey-Bass.
Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A systematic approach (7th ed.). Thousand Oaks: Sage Publications.
Rouse, C. E. (1998). Do two-year colleges increase overall educational attainment? Policy Analysis and Management, 17( 4), 595–620.
Rueda, R. (2006). A sociocultural perspective on individual and institutional change: The equity for all project. Los Angeles: Center for Urban Education, University of Southern California.
Scientifically-based research. (n.d.). Retrieved March 15, 2004, from http://www.ecs.org/html/educationIssues/Research/primer/researchsays.asp
Selingo, J. (2006). To avoid federal mandate, colleges should devise own gauges of student learning, land-grant group says. Retrieved April 7, 2006, from http://chronicle.com
Sergiovanni, T. J. (1992). Moral leadership: Getting to the heart of school improvement. San Francisco: Jossey-Bass Publishers.
Serving adult learners in higher education: Principles of effectiveness. (2000). Retrieved May 14, 2005, from http://www.cael.org/
Shavelson, R. J., Phillips, D. C., Towne, L., & Feuer, M. J. (2003). On the science of education design studies. Educational Researcher, 32( 1), 25–28.
Simon, H. A. (1997). Administrative behavior: A study of decision-making processes in administrative organization. New York: The Free Press.
St. Pierre, E. A. (2002). “Science” rejects postmodernism. Educational Researcher, 31(8), 25–27.
Stake, R. E. (1995). The art of case study research. Thousand Oaks: Sage Publications.
Strout, E. (2004, July 13). Colleges should become more accountable, Smith’s leader says, or face the legislative consequences. Retrieved July 13, 2004, from http://chronicle.com/daily/2004/07/20040713n.htm
Tharp, R. G. (1993). Institutional and social context of educational practice and reform. In E. A. Forman, N. Minick, & C. A. Stone (Eds.), Contexts for learning: Sociocultural dynamics in children’s development. New York: Oxford University Press.
Tharp, R. G., & Gallimore, R. (1988). Rousing minds to life: Teaching and learning in social context. New York: Cambridge University Press.
Thomas, J. B., Clark, S. M., & Gioia, D. A. (1993). Strategic sensemaking and organizational performance: Linkages among scanning, interpretation, action, and outcomes. Academy of Management Journal, 36( 2), 239–270.
Titus, M. A. (in Press). Detecting selection bias, using propensity score matching, and estimating treatment effects: An application to the private returns to a master’s degree. Research in Higher Education.
Toutkoushian, R. K., & Danielson, C. (2002). Using performance indicators to evaluate decentralized budgeting systems and institutional performance. In D. Priest, W. Becker, D. Hossler, & E. P. St. John (Eds.), Incentive-based budgeting systems in public universities. Northampton, MA: Edward Elgar Publishing.
Trow, M. (1996). Trust, markets and accountability in higher education: A comparative perspective. Higher Education Policy, 9( 4), 309–324.
U.S. Department of Education. (2006). A test of leadership: Charting the future of U.S. higher education. Washington, D.C.
Volkwein, J. F. (2003). Using and enhancing existing data to respond to campus challenges. In F. K. Stage & K. Manning (Eds.), Research in the college context pp. 183–207. New York: Brunner-Routledge.
Volkwein, J. F., & Grunig, S. D. (2005). Resources and reputation in higher education: Double, double, toil and trouble. In J. C. Burke et al. (Eds.), Achieving accountability in higher education pp. 246–274. San Francisco: Jossey-Bass.
Walleri, R. D. (2003). The role of institutional research in the comprehensive community college. Journal of Applied Research in the Community College, 11( 1), 49–56.
Weiss, C. H. (1975). Evaluation research in the political context. In E. L. Struening & M. Guttentag (Eds.), Handbook of Evaluation Research Vol. 1, pp. 13–26. Beverly Hills, CA: Sage Publications.
Wellman, J. V. (2002). Statewide higher education accountability: Issues, options, and strategies for success. In N. G. Association (Ed.), Higher expectations: Second in a series of essays on the future of higher education pp. 7–15. Washington, DC
What we know about access and success in postsecondary education: Informing Lumina Foundation’s strategic mission. (2006). Retrieved July 30, 2006, from www.luminafoundation.org/
What Works Clearinghouse: A trusted source of scientific evidence of what works in education. (2006). U.S. Department of Education Institute of Education Sciences.
What Works Clearinghouse: Study design classification. (2006). U.S. Department of Education Institute of Education Sciences.
What Works Clearinghouse: Study review standards. (2006). U.S. Department of Education Institute of Education Sciences.
Wolff, R. A. (2005). Accountability and accreditation: Can reforms match increasing demands? In J. C. Burke et al. (Eds.), Achieving accountability in higher education pp. 78–103. San Francisco: Jossey-Bass.
Wyner, J. (2006, February 10). Educational equity and the transfer student. Chronicle of Higher Education, p. B6.
Zaritsky, R., Kelly, A. E., Flowers, W., Rogers, E., & O’Neill, P. (2003). Clinical design sciences: A view from sister design efforts. Educational Researcher, 32( 1), 32–34.
Zemsky, R. M. (2005). The dog that doesn’t bark: Why markets neither limit prices nor promote. In J. C. Burke et al. (Eds.), Achieving accountability in higher education pp. 275–295. San Francisco: Jossey-Bass.
Zemsky, R. M., Massy, W. F., & Oedel, P. (1993, May/June). On reversing the ratchet. Change, The Magazine of Higher Learning, 25, 56–62.
Zumeta, W. (2001). Public policy and accountability in higher education: Lessons from the past and present for the new millenium. In D. E. Heller (Ed.), The states and public higher education policy: Affordability, access, and accountability. Baltimore: Johns Hopkins University Press.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2007 Springer
About this chapter
Cite this chapter
Dowd, A.C., Tong, V.P. (2007). Accountability, Assessment, and the Scholarship of “Best Practice”. In: Smart, J.C. (eds) Higher Education: Handbook of Theory and Research. Higher Education: Handbook of Theory and Research, vol 22. Springer, Dordrecht. https://doi.org/10.1007/978-1-4020-5666-6_2
Download citation
DOI: https://doi.org/10.1007/978-1-4020-5666-6_2
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-5665-9
Online ISBN: 978-1-4020-5666-6
eBook Packages: Humanities, Social Sciences and LawEducation (R0)