Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Developing Performance Indicators for Assessing Classroom Teaching Practices and Student Learning

Abstract

Several states are requiring institutions to document changes in student outcomes. Regional and specialized accrediting agencies are also changing their review criteria from measuring inputs to assessing indicators of student learning. This article describes the results of an evaluation project that sought to develop performance indicators of learning gains for undergraduate engineering students. Specifically, the study investigated the relationship between classroom practices and students' gains in professional competencies. More than 1,250 students from 7 universities participated. Findings show that the instructional practices of Instructor Interaction and Feedback, Collaborative Learning, and Clarity and Organization are significantly and positively associated with gains in students' self-reported gains in problem-solving skills, group skills, and understanding of engineering as an occupation. The indicators meet several conditions recommended by the assessment literature. They are (1) meaningful to the user, (2) reliable and valid, and (3) index observable behaviors rather than subjective impressions.

This is a preview of subscription content, log in to check access.

REFERENCES

  1. Anaya, G. (1999). College impact on student learning: comparing the use of self-reported gains, standardized test scores, and college grades. Research in Higher Education 40(5): 499-527.

  2. Abrami, P. C. (1985). Dimensions of effective college instruction. Review of Higher Education 8(3): 211-228.

  3. Accreditation Board for Engineering and Technology (ABET). (1998). ABET 2000: Criteria for Accrediting Programs in Engineering in the United States. Baltimore: Auhor.

  4. Astin, A. W. (1993). What Matters in College? Four Critical Years Revisited. San Francisco: Jossey-Bass.

  5. Augustine, N. R. (1996). Rebuilding engineering education. Chronicle of Higher Education, May 24, 1996: B1-B2.

  6. Baxter-Magolda, M. D. (1992). Knowing and Reasoning in College: Gender-Related Patterns in Students' Intellectual Development. San Francisco: Jossey-Bass.

  7. Belenky, M. F., Clinchy, B. M., Goldberger, N., and Tarule, J. M. (1986). Women's Ways of Knowing: The Development of Self, Voice, and Mind. New York: Basic Books.

  8. Black, K. M. (1994). An industry view of engineering education. Journal of Engineering Education 83(1): 26-28.

  9. Borden, V. M. H., and Banta, T. W. (1994). Using Performance Indicators to Guide Strategic Decision Making. San Francisco: Jossey-Bass.

  10. Bruffee, K. A. (1984). Collaborative learning and the “conversation of mankind” College English 46: 635-652.

  11. Bucciarelli, L. L. (1988). An ethnographic perspective on engineering design. Design Studies 9(3): 159-168.

  12. Burke, J. C., and Serban, A. M. (1998). Performance Funding for Public Higher Education: Fad or Trend? New Directions for Institutional Research, No. 97. San Francisco: Jossey-Bass.

  13. Cabrera, A. F., and Nora, A. (1994). College students' perceptions of prejudice and discrimination and their feelings of alienation: A construct validation approach. Review of Education/Pedagogy/Cultural Studies 16(3-4): 387-409.

  14. Cabrera, A. F., Nora, A., Bernal, E., Terenzini, P., and Pascarella, P. (November 1998). Collaborative learning: preferences, gains in cognitive and affective outcomes, and openness to diversity among college students. Paper presented before the 1998 Annual Meeting of the Association for the Study of Higher Education, Miami, FL.

  15. Cabrera, A. F., Nora, A., Terenzini, P. T., Pascarella, P. T., and Hagedorn, L. S. (1999). Campus racial climate and the adjustment of students to college: a comparison between White students and African American students. Journal of Higher Education 70(2): 134-160.

  16. Cardelle, M., and Corno, L. (1981). Effects on second language learning of variations in written feddback on homework assignments. TESOL Quarterly 15(3): 251-261.

  17. Dally, J. W., and Zhang, G. M. (1993). A freshman engineering design course. Journal of Engineering Education 82(2): 83-91.

  18. Dougherty, C. and Fantaske, P. (1996). Defining expectations for problem-solving skills. In E. A. Jones (ed.), Preparing Competent College Graduates: Setting New and Higher Expectations for Student Learning, New Directions for Higher Education, Vol. 96, pp. 55-66. San Francisco: Jossey-Bass.

  19. Drew, D. (1996). Aptitude Revisited: Rethinking Math and Science Education for America's Next Century. Baltimore: Johns Hopkins University Press.

  20. Dym, C. L. (1994). Teaching design to freshman: style and content. Journal of Engineering Education 83(4): 303-310.

  21. Eimers, M. T., and Pike, G. R. (1997). Minority and nonminority adjustment to college: differences or similarities. Research in Higher Education 38(1): 77-98.

  22. Ewell, P. (1996). Identifying indicators of curriculum quality. In J. G. Gaff, J. L. Ratcliff, and Associates (eds.), Handbook of the Undergraduate Curriculum: A Comprehensive Guide to Purposes, Structures, Practices, and Change, pp. 608-632. San Francisco: Jossey-Bass.

  23. Ewell, P. T. (1987). Assessment, Accountability and Improvement. Washington, DC: American Association for Higher Education.

  24. Ewell, P. T. (1998). National trends in assessing student learning. Journal of Engineering Education 87(2): 107-113.

  25. Feldman, K. A. (1989). Instructional effectiveness of college teachers as judged by teachers themselves, current and former students, colleagues, administrators, and external (neutral) observers. Research in Higher Education 30(2): 137-194.

  26. Finkelstein, M. J., Seal, R. K., and Schuster, J. H. (1998). The New Academic Generation: A Profession in Transformation. Baltimore: Johns Hopkins University Press.

  27. Fleming, J. (1984). Blacks in College: A Comparative Study of Students' Success in Black and in White Institutions. San Francisco: Jossey-Bass.

  28. Gaff, J. G., Ratcliff, J. L., and Associates. (1996). Handbook of the Undergraduate Curriculum: A Comprehensive Guide to Purposes, Structures, Practices, and Change. San Francisco: Jossey-Bass.

  29. Gamson, Z. F. (1994). Collaborative learning comes of age. In S. Kadel and J. A. Keehner (eds.), Collaborative Learning: A Source Book for Higher Education, Vol. II, pp. 5-17. State College, PA: National Center for Teaching, Learning, and Assessment.

  30. Hackett, E. R., and Carrigan, S. D. (1998). Performance indicators: information in search of a valid and reliable use. Education Policy Archives 6(17): 1-27. [http://epaa.asu. edu/epaa/v6n17/]

  31. Hall, R. M., and Sandler, B. (1982). The Classroom Climate: A Chilly One for Women? Project on the Status and Education of Women. Washington, DC: Association of American Colleges.

  32. Hurtado, S. (1992). The campus racial climate: contexts of conflict. Journal of Higher Education 63: 539-569.

  33. Jehn, K. A. (1997). A qualitative analysis of conflict types and dimensions in organizational groups. Administrative Science Quarterly 42: 530-557.

  34. Johnson, D. W., Johnson, R. T., and Smith, K. A. (1991). Cooperative learning: increasing college faculty instructional productivity. ASHE-ERIC Higher Education Report No. 4. Washington DC: The George Washington University, School of Education and Human Development.

  35. Jones, E. A. (1994). Goals Inventories. State College, PA: National Center on Postsecondary Teaching, Learning and Assessment, Center for the Study of Higher Education, The Pennsylvania State University.

  36. Jones, E. A. (1996). Preparing Competent College Graduates: Setting New and Higher Expectations for Student Learning. New Directions for Higher Education, No. 96. San Francisco: Jossey-Bass.

  37. Jones, E. A., Dougherty, C., and Fantaske, P. (1997). Defining Essential Critical Reading. and Problem-Solving Outcomes: Perspectives of Faculty, Employers and Policy Makers. State College, PA: National Center on Postsecondary Teaching, Learning and Assessment, Center for the Study of Higher Education, The Pennsylvania State University.

  38. Jones, E. A., Hoffman, S., Moore, L., Ratcliff, J., Tibbetts, S., and Click, B. (1994). Essential Skills in Writing, Speech and Listening, and Critical Thinking: Perspectives of Faculty, Employers and Policy Makers. State College, PA: National Center on Post-secondary Teaching, Learning and Assessment, Center for the Study of Higher Education, The Pennsylvania State University.

  39. Kellogg Commission on the Future of State and Land-Grant Universities. (1997). Returning to Our Roots: The Student Experience. Washington DC: National Association of State Universities and Land-Grant Colleges.

  40. Kuh, G. D., Douglas, K. B., Lund, J. P., and Ramin-Gyurnek, J. (1994). Student Learning Outside the Classroom: Transcending Artificial Boundaries. Washington, DC: ASHE-ERIC Education Report No. 8.

  41. Kulik, J., and Kulik, C. L. (1979). College teaching. In P. Peterson & H. Walberg (eds.), Research on Teaching: Concepts, Findings and Implications, pp. 70-93. Berkeley, CA: McCutcheon.

  42. Levine, M. E., and Levine, R. L. (1991). A critical examination of academic retention programs for at-risk minority college students. Journal of College Student Development 32: 323-334.

  43. Lopez, C. L. (1996). Opportunities for improvement: Advice from Consultant-Evaluators on Programs to Assess Student Learning. Chicago: North Central Accreditation Com-mission on Institutions of Higher Education.

  44. Lundeberg, M. A., and Diemert, S. (1995). Influence of social interaction on cognition: connected learning in science. Journal of Higher Education 66(3): 312-335.

  45. Martínez-Alemán, A. (1997). Understanding and investigating female friendship's educative value. Journal of Higher Education 68(2): 119-159.

  46. McKeachie, W. (1988). Teaching thinking. Update 2(1): 1.

  47. McKeachie, W. (1990). Research on college teaching: the historical background. Journal of Educational Psychcology, 82(2): 189-200.

  48. McMartin, F., Van Duzer, E., and Agogino, A. (1998). Bridging diverse institutions, multiple engineering departments, and industry: a case study in assessment planning. Journal of Engineering Education 87(2): 157-165.

  49. Middle States Association of Colleges and Universities. (1996). Framework for Outcomes Assessment. Philadelphia: Commission on Higher Education.

  50. Murray, H. G. (1991). Effective teaching behaviors in the college classroom. In J. C. Smart (ed.), Higher Education: Handbook of Theory and Research, Vol. VII, pp. 135-172. New York: Agathon Press.

  51. National Education Goals Panel. (1992). The National Education Goals Report: Building a Nation of Learners. Washington, DC: U.S. Government Printing Office.

  52. National Science Foundation, Directorate for Education and Human Resources. (1996). Shaping the Future: New Expectations for Undergraduate Education in Science, Mathematics, Engineering, and Technology (NSF-96-139). Washington, DC: National Science Foundation.

  53. Nedwek, B. P., and Neal, J. E. (1994). Performance indicators and regional management tools: a comparative assessment of projects in North American and Europe. Research in Higher Education 35(1): 75-104.

  54. Oakes, J. (1990). Opportunities, achievement, and choice: women and minority students in science and mathematics. Review of Research in Education 16: 153-222.

  55. Pascarella, E., and Terenzini, P. T. (1991). How College Affects Students. San Francisco: Jossey-Bass.

  56. Performance Indicators Task Force. (June 1998). A proposal for merit-based performance funding for the State University of New York. Unpublished report. New York: University at Albany.

  57. Pike, G. R. (1995). The relationship between self reports of college experiences and achievement test scores, Research in Higher Education 36(1): 1-22.

  58. Sax, L. J. (1994). Retaining tomorrow's scientists: exploring the factors that keep minority and female college students interested in science careers. Journal of Women and Minorities in Science and Engineering 1: 45-61

  59. Schon, D. A. (1987). Educating the Reflective Practitioner. San Francisco: Jossey-Bass.

  60. Seymour, E., and Hewitt, N. M. (1997). Talking About Leaving: Why Undergraduates Leave the Sciences. Boulder, CO: Westview Press.

  61. Stark, J. S., and Latucca, L. R. (1997). Shaping the College Curriculum: Academic Plans in Action. MA: Allyn & Bacon.

  62. Stark, J. S., and Lowther, M. A. (1989). Exploring common ground in liberal and professional education. In R. A. Armour and B. Fuhrmann (eds.), Integrating Liberal Learning and Professional Education, New Directions for Teaching and Learning, No. 40, pp. 7-20. San Francisco: Jossey-Bass.

  63. Strenta, A. C., Elliott, R., Adair, R., Matier, M., and Scott, J. (1994). Choosing and learning sciences in highly selective institutions. Research in Higher Education 35(5): 513-548.

  64. Terenzini, P. T., Springer, L., Pascarella, E. T., and Nora, A. (1995). Influences affecting the development of students' critical thinking skills. Research in Higher Education 36(1): 23-40.

  65. Tinto, V. (1997). Classrooms as communities: Exploring the educational character of student persistence. Journal of Higher Education 68(6): 599-623.

  66. Tinto, V. (1998). Colleges as communities: Taking research on student persistence seriously. Review of Higher Education 21(2): 167-177.

  67. Volkwein, J. F. (November 1991). Improved measures of academic and social integration and their association with measures of student growth. Paper presented at the annual meeting of the Association for the Study of Higher Education, Boston.

  68. Volkwein, J. F., and Cabrera, A.F. (1998). Student measures associated with favorable classroom experiences. Paper presented at the Association of Institutional Researchers Forum, Minneapolis, MN.

  69. Volkwein, J. F., and Lorang, W. (1996). Characteristics of extenders: full-time students who take light credit loads and graduate in more than four years. Research in Higher Education 37(1).

  70. Whiteley, M. A., Porter, J. D., and Fenske, R. H. (1992). The Premier for Institutional Research. Florida: Association for Institutional Research.

  71. Whitt, E. J., Pascarella, E. T., Edison, M. I., Nora, A., and Terenzini, P. T. (1998). Perceptions of achilly climateand cognitive outcomes during college. Paper presented before the annual meeting of the Association of Educational Researchers, San Diego.

Download references

Author information

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Cabrera, A.F., Colbeck, C.L. & Terenzini, P.T. Developing Performance Indicators for Assessing Classroom Teaching Practices and Student Learning. Research in Higher Education 42, 327–352 (2001). https://doi.org/10.1023/A:1018874023323

Download citation

  • student outcomes
  • classroom practices
  • assessment