Abstract
Starting with the premise that better assessment leads to more informed decisions about student learning, we investigated the factors that lead to assessment improvement. We used “meta-assessment” (i.e., evaluating the assessment process) to identify academic programs in which the assessment process had improved over a two-year period. The use of both quantitative and qualitative methods allowed us to understand the factors leading to assessment improvement better. Through these efforts, we discovered that a program’s assessment environment and use of resources were the predominant factors leading to improvement. One resource in particular, assessment consultation, was the most cited reason for improved assessment.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Astin, A., Banta, T., Cross, K., El-Khawas, E., Ewell, P., Hutchings, P., Marchese, T.J., McClenney, K.M., Mentkowski, M., Miller, M.A., Moran, E.T., & Wright, B. D. (n. d.). American Association of Higher Education (AAHE) Principles of good practice for assessing student learning. Retrieved from: http://assessment.uconn.edu/docs/resources/AAHE_Principles_of_Good_Practice.pdf
Banta, T. W. (1997). Moving assessment forward: Enabling conditions and stumbling blocks. New Directions for Higher Education, 100, 79–91. doi:10.1002/he.10007
Banta, T. W., Lund, J. P., Black, K. E., & Oblander, F. W. (1996). Assessment in practice: Putting principles to work on college campuses. San Francisco, CA: Jossey-Bass.
Erwin, T. D. (1991). Assessing student learning and development: A guide to the principles, goals, and methods of determining college outcomes. San Francisco, CA: Jossey-Bass.
Fulcher, K. H., & Orem, C. D. (2010). Evolving from quantity to quality: A new yardstick for assessment. Journal of Research and Practice in Assessment, 5, 1–10. Retrieved from http://www.virginiaassessment.org/rpa/5/FulcherandOrem.pdf
Fulcher, K.H., Swain, M., & Orem, C.D., (2012). Expectations for assessment reports: A descriptive analysis. Assessment Update, 24(1), 1–2, 14–16. doi:10.1002/au.241
Goodrich, H. (1997). Understanding rubrics. Educational Leadership, 54(4), 14–17.
Linkon, S. (2005). How can assessment work for us? Academe Online, Retrieved from http://www.aaup.org/AAUP/pubsres/academe/2005/JA/Feat/link.htm
Ndoye, A., & Parker, M. A. (2010). Creating and sustaining a culture of assessment. Planning for Higher Education, 38(2), 28–39.
Orem, C. (2012). Demonstrating validity evidence of meta-assessment scores using generalizability theory. Harrisonburg, VA: James Madison University (Unpublished doctoral dissertation).
Ory, J. C. (1992). Meta-assessment: Evaluating assessment activities. Research in Higher Education, 33, 467–481.
Pringle, C., & Michel, M. (2007). Assessment practices in AACSB-accredited business schools. The Journal of Education for Business, 82, 202–211.
Shavelson, R. J. (2010). Measuring college learning responsibly: Accountability in a new era. Stanford, CA: Stanford University Press.
Suskie, L. (2009). Assessing student learning: A common sense guide (2nd ed.). San Francisco, CA: Jossey-Bass.
Author information
Authors and Affiliations
Corresponding author
Appendix: Interview Protocol
Appendix: Interview Protocol
Thank you for agreeing to participate in this study. You have been asked to participate because your program’s Quality of Assessment scores improved dramatically from 2008–2009 to 2009–2010. We’re trying to understand how and why your program made such a dramatic improvement. We hope to share the results of this study both within and outside of our University’s community. As you read in the consent form, your responses will be kept confidential. We only have 30 minutes, so at some point I may ask you to finish your thoughts for the sake of time.
-
I.
Questions about the Assessment Coordinator
How long have you been the Assessment Coordinator for your program?
Before becoming Assessment Coordinator, did you have an interest in assessment?
Did you have any prior experience with assessment?
Are there other people in your program that assist you with assessment? How?
Which aspects of assessment are the easiest for you? Which are the most challenging?
-
II.
Factors contributing to Quality of Assessment score improvement
As you know, we asked to meet because we’re interested in learning about the characteristics of programs that made large gains in their Quality of Assessment scores over the past year. So, what factors do you think may have contributed to your program’s Quality of Assessment score improving from 2008–2009 to 2009-2010?
-
III.
Scale
Now I’d like you to take a few minutes to respond to these items. Feel free to ask me any questions about any of the items. Do you have any new thoughts or ideas on why your program’s APT score improved?
-
IV.
Summary
Verify and summarize the main factors identified by the assessment coordinator.
-
V.
Closing questions
What are some ways that assessment in your program could become even better than it already is?
What advice might you give to the assessment coordinator of a program that was new or struggling with its assessment?
I’m going to give you a card with contact information for the assessment center’s resources, just so that you know we are here to help you. So if you have any questions, please feel free to contact us.
Rights and permissions
About this article
Cite this article
Rodgers, M., Grays, M.P., Fulcher, K.H. et al. Improving Academic Program Assessment: A Mixed Methods Study. Innov High Educ 38, 383–395 (2013). https://doi.org/10.1007/s10755-012-9245-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10755-012-9245-9