Advertisement

Research in Higher Education

, Volume 42, Issue 3, pp 327–352 | Cite as

Developing Performance Indicators for Assessing Classroom Teaching Practices and Student Learning

  • Alberto F. Cabrera
  • Carol L. Colbeck
  • Patrick T. Terenzini
Article

Abstract

Several states are requiring institutions to document changes in student outcomes. Regional and specialized accrediting agencies are also changing their review criteria from measuring inputs to assessing indicators of student learning. This article describes the results of an evaluation project that sought to develop performance indicators of learning gains for undergraduate engineering students. Specifically, the study investigated the relationship between classroom practices and students' gains in professional competencies. More than 1,250 students from 7 universities participated. Findings show that the instructional practices of Instructor Interaction and Feedback, Collaborative Learning, and Clarity and Organization are significantly and positively associated with gains in students' self-reported gains in problem-solving skills, group skills, and understanding of engineering as an occupation. The indicators meet several conditions recommended by the assessment literature. They are (1) meaningful to the user, (2) reliable and valid, and (3) index observable behaviors rather than subjective impressions.

student outcomes classroom practices assessment 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

REFERENCES

  1. Anaya, G. (1999). College impact on student learning: comparing the use of self-reported gains, standardized test scores, and college grades. Research in Higher Education 40(5): 499-527.Google Scholar
  2. Abrami, P. C. (1985). Dimensions of effective college instruction. Review of Higher Education 8(3): 211-228.Google Scholar
  3. Accreditation Board for Engineering and Technology (ABET). (1998). ABET 2000: Criteria for Accrediting Programs in Engineering in the United States. Baltimore: Auhor.Google Scholar
  4. Astin, A. W. (1993). What Matters in College? Four Critical Years Revisited. San Francisco: Jossey-Bass.Google Scholar
  5. Augustine, N. R. (1996). Rebuilding engineering education. Chronicle of Higher Education, May 24, 1996: B1-B2.Google Scholar
  6. Baxter-Magolda, M. D. (1992). Knowing and Reasoning in College: Gender-Related Patterns in Students' Intellectual Development. San Francisco: Jossey-Bass.Google Scholar
  7. Belenky, M. F., Clinchy, B. M., Goldberger, N., and Tarule, J. M. (1986). Women's Ways of Knowing: The Development of Self, Voice, and Mind. New York: Basic Books.Google Scholar
  8. Black, K. M. (1994). An industry view of engineering education. Journal of Engineering Education 83(1): 26-28.Google Scholar
  9. Borden, V. M. H., and Banta, T. W. (1994). Using Performance Indicators to Guide Strategic Decision Making. San Francisco: Jossey-Bass.Google Scholar
  10. Bruffee, K. A. (1984). Collaborative learning and the “conversation of mankind” College English 46: 635-652.Google Scholar
  11. Bucciarelli, L. L. (1988). An ethnographic perspective on engineering design. Design Studies 9(3): 159-168.Google Scholar
  12. Burke, J. C., and Serban, A. M. (1998). Performance Funding for Public Higher Education: Fad or Trend? New Directions for Institutional Research, No. 97. San Francisco: Jossey-Bass.Google Scholar
  13. Cabrera, A. F., and Nora, A. (1994). College students' perceptions of prejudice and discrimination and their feelings of alienation: A construct validation approach. Review of Education/Pedagogy/Cultural Studies 16(3-4): 387-409.Google Scholar
  14. Cabrera, A. F., Nora, A., Bernal, E., Terenzini, P., and Pascarella, P. (November 1998). Collaborative learning: preferences, gains in cognitive and affective outcomes, and openness to diversity among college students. Paper presented before the 1998 Annual Meeting of the Association for the Study of Higher Education, Miami, FL.Google Scholar
  15. Cabrera, A. F., Nora, A., Terenzini, P. T., Pascarella, P. T., and Hagedorn, L. S. (1999). Campus racial climate and the adjustment of students to college: a comparison between White students and African American students. Journal of Higher Education 70(2): 134-160.Google Scholar
  16. Cardelle, M., and Corno, L. (1981). Effects on second language learning of variations in written feddback on homework assignments. TESOL Quarterly 15(3): 251-261.Google Scholar
  17. Dally, J. W., and Zhang, G. M. (1993). A freshman engineering design course. Journal of Engineering Education 82(2): 83-91.Google Scholar
  18. Dougherty, C. and Fantaske, P. (1996). Defining expectations for problem-solving skills. In E. A. Jones (ed.), Preparing Competent College Graduates: Setting New and Higher Expectations for Student Learning, New Directions for Higher Education, Vol. 96, pp. 55-66. San Francisco: Jossey-Bass.Google Scholar
  19. Drew, D. (1996). Aptitude Revisited: Rethinking Math and Science Education for America's Next Century. Baltimore: Johns Hopkins University Press.Google Scholar
  20. Dym, C. L. (1994). Teaching design to freshman: style and content. Journal of Engineering Education 83(4): 303-310.Google Scholar
  21. Eimers, M. T., and Pike, G. R. (1997). Minority and nonminority adjustment to college: differences or similarities. Research in Higher Education 38(1): 77-98.Google Scholar
  22. Ewell, P. (1996). Identifying indicators of curriculum quality. In J. G. Gaff, J. L. Ratcliff, and Associates (eds.), Handbook of the Undergraduate Curriculum: A Comprehensive Guide to Purposes, Structures, Practices, and Change, pp. 608-632. San Francisco: Jossey-Bass.Google Scholar
  23. Ewell, P. T. (1987). Assessment, Accountability and Improvement. Washington, DC: American Association for Higher Education.Google Scholar
  24. Ewell, P. T. (1998). National trends in assessing student learning. Journal of Engineering Education 87(2): 107-113.Google Scholar
  25. Feldman, K. A. (1989). Instructional effectiveness of college teachers as judged by teachers themselves, current and former students, colleagues, administrators, and external (neutral) observers. Research in Higher Education 30(2): 137-194.Google Scholar
  26. Finkelstein, M. J., Seal, R. K., and Schuster, J. H. (1998). The New Academic Generation: A Profession in Transformation. Baltimore: Johns Hopkins University Press.Google Scholar
  27. Fleming, J. (1984). Blacks in College: A Comparative Study of Students' Success in Black and in White Institutions. San Francisco: Jossey-Bass.Google Scholar
  28. Gaff, J. G., Ratcliff, J. L., and Associates. (1996). Handbook of the Undergraduate Curriculum: A Comprehensive Guide to Purposes, Structures, Practices, and Change. San Francisco: Jossey-Bass.Google Scholar
  29. Gamson, Z. F. (1994). Collaborative learning comes of age. In S. Kadel and J. A. Keehner (eds.), Collaborative Learning: A Source Book for Higher Education, Vol. II, pp. 5-17. State College, PA: National Center for Teaching, Learning, and Assessment.Google Scholar
  30. Hackett, E. R., and Carrigan, S. D. (1998). Performance indicators: information in search of a valid and reliable use. Education Policy Archives 6(17): 1-27. [http://epaa.asu. edu/epaa/v6n17/]Google Scholar
  31. Hall, R. M., and Sandler, B. (1982). The Classroom Climate: A Chilly One for Women? Project on the Status and Education of Women. Washington, DC: Association of American Colleges.Google Scholar
  32. Hurtado, S. (1992). The campus racial climate: contexts of conflict. Journal of Higher Education 63: 539-569.Google Scholar
  33. Jehn, K. A. (1997). A qualitative analysis of conflict types and dimensions in organizational groups. Administrative Science Quarterly 42: 530-557.Google Scholar
  34. Johnson, D. W., Johnson, R. T., and Smith, K. A. (1991). Cooperative learning: increasing college faculty instructional productivity. ASHE-ERIC Higher Education Report No. 4. Washington DC: The George Washington University, School of Education and Human Development.Google Scholar
  35. Jones, E. A. (1994). Goals Inventories. State College, PA: National Center on Postsecondary Teaching, Learning and Assessment, Center for the Study of Higher Education, The Pennsylvania State University.Google Scholar
  36. Jones, E. A. (1996). Preparing Competent College Graduates: Setting New and Higher Expectations for Student Learning. New Directions for Higher Education, No. 96. San Francisco: Jossey-Bass.Google Scholar
  37. Jones, E. A., Dougherty, C., and Fantaske, P. (1997). Defining Essential Critical Reading. and Problem-Solving Outcomes: Perspectives of Faculty, Employers and Policy Makers. State College, PA: National Center on Postsecondary Teaching, Learning and Assessment, Center for the Study of Higher Education, The Pennsylvania State University.Google Scholar
  38. Jones, E. A., Hoffman, S., Moore, L., Ratcliff, J., Tibbetts, S., and Click, B. (1994). Essential Skills in Writing, Speech and Listening, and Critical Thinking: Perspectives of Faculty, Employers and Policy Makers. State College, PA: National Center on Post-secondary Teaching, Learning and Assessment, Center for the Study of Higher Education, The Pennsylvania State University.Google Scholar
  39. Kellogg Commission on the Future of State and Land-Grant Universities. (1997). Returning to Our Roots: The Student Experience. Washington DC: National Association of State Universities and Land-Grant Colleges.Google Scholar
  40. Kuh, G. D., Douglas, K. B., Lund, J. P., and Ramin-Gyurnek, J. (1994). Student Learning Outside the Classroom: Transcending Artificial Boundaries. Washington, DC: ASHE-ERIC Education Report No. 8.Google Scholar
  41. Kulik, J., and Kulik, C. L. (1979). College teaching. In P. Peterson & H. Walberg (eds.), Research on Teaching: Concepts, Findings and Implications, pp. 70-93. Berkeley, CA: McCutcheon.Google Scholar
  42. Levine, M. E., and Levine, R. L. (1991). A critical examination of academic retention programs for at-risk minority college students. Journal of College Student Development 32: 323-334.Google Scholar
  43. Lopez, C. L. (1996). Opportunities for improvement: Advice from Consultant-Evaluators on Programs to Assess Student Learning. Chicago: North Central Accreditation Com-mission on Institutions of Higher Education.Google Scholar
  44. Lundeberg, M. A., and Diemert, S. (1995). Influence of social interaction on cognition: connected learning in science. Journal of Higher Education 66(3): 312-335.Google Scholar
  45. Martínez-Alemán, A. (1997). Understanding and investigating female friendship's educative value. Journal of Higher Education 68(2): 119-159.Google Scholar
  46. McKeachie, W. (1988). Teaching thinking. Update 2(1): 1.Google Scholar
  47. McKeachie, W. (1990). Research on college teaching: the historical background. Journal of Educational Psychcology, 82(2): 189-200.Google Scholar
  48. McMartin, F., Van Duzer, E., and Agogino, A. (1998). Bridging diverse institutions, multiple engineering departments, and industry: a case study in assessment planning. Journal of Engineering Education 87(2): 157-165.Google Scholar
  49. Middle States Association of Colleges and Universities. (1996). Framework for Outcomes Assessment. Philadelphia: Commission on Higher Education.Google Scholar
  50. Murray, H. G. (1991). Effective teaching behaviors in the college classroom. In J. C. Smart (ed.), Higher Education: Handbook of Theory and Research, Vol. VII, pp. 135-172. New York: Agathon Press.Google Scholar
  51. National Education Goals Panel. (1992). The National Education Goals Report: Building a Nation of Learners. Washington, DC: U.S. Government Printing Office.Google Scholar
  52. National Science Foundation, Directorate for Education and Human Resources. (1996). Shaping the Future: New Expectations for Undergraduate Education in Science, Mathematics, Engineering, and Technology (NSF-96-139). Washington, DC: National Science Foundation.Google Scholar
  53. Nedwek, B. P., and Neal, J. E. (1994). Performance indicators and regional management tools: a comparative assessment of projects in North American and Europe. Research in Higher Education 35(1): 75-104.Google Scholar
  54. Oakes, J. (1990). Opportunities, achievement, and choice: women and minority students in science and mathematics. Review of Research in Education 16: 153-222.Google Scholar
  55. Pascarella, E., and Terenzini, P. T. (1991). How College Affects Students. San Francisco: Jossey-Bass.Google Scholar
  56. Performance Indicators Task Force. (June 1998). A proposal for merit-based performance funding for the State University of New York. Unpublished report. New York: University at Albany.Google Scholar
  57. Pike, G. R. (1995). The relationship between self reports of college experiences and achievement test scores, Research in Higher Education 36(1): 1-22.Google Scholar
  58. Sax, L. J. (1994). Retaining tomorrow's scientists: exploring the factors that keep minority and female college students interested in science careers. Journal of Women and Minorities in Science and Engineering 1: 45-61Google Scholar
  59. Schon, D. A. (1987). Educating the Reflective Practitioner. San Francisco: Jossey-Bass.Google Scholar
  60. Seymour, E., and Hewitt, N. M. (1997). Talking About Leaving: Why Undergraduates Leave the Sciences. Boulder, CO: Westview Press.Google Scholar
  61. Stark, J. S., and Latucca, L. R. (1997). Shaping the College Curriculum: Academic Plans in Action. MA: Allyn & Bacon.Google Scholar
  62. Stark, J. S., and Lowther, M. A. (1989). Exploring common ground in liberal and professional education. In R. A. Armour and B. Fuhrmann (eds.), Integrating Liberal Learning and Professional Education, New Directions for Teaching and Learning, No. 40, pp. 7-20. San Francisco: Jossey-Bass.Google Scholar
  63. Strenta, A. C., Elliott, R., Adair, R., Matier, M., and Scott, J. (1994). Choosing and learning sciences in highly selective institutions. Research in Higher Education 35(5): 513-548.Google Scholar
  64. Terenzini, P. T., Springer, L., Pascarella, E. T., and Nora, A. (1995). Influences affecting the development of students' critical thinking skills. Research in Higher Education 36(1): 23-40.Google Scholar
  65. Tinto, V. (1997). Classrooms as communities: Exploring the educational character of student persistence. Journal of Higher Education 68(6): 599-623.Google Scholar
  66. Tinto, V. (1998). Colleges as communities: Taking research on student persistence seriously. Review of Higher Education 21(2): 167-177.Google Scholar
  67. Volkwein, J. F. (November 1991). Improved measures of academic and social integration and their association with measures of student growth. Paper presented at the annual meeting of the Association for the Study of Higher Education, Boston.Google Scholar
  68. Volkwein, J. F., and Cabrera, A.F. (1998). Student measures associated with favorable classroom experiences. Paper presented at the Association of Institutional Researchers Forum, Minneapolis, MN.Google Scholar
  69. Volkwein, J. F., and Lorang, W. (1996). Characteristics of extenders: full-time students who take light credit loads and graduate in more than four years. Research in Higher Education 37(1).Google Scholar
  70. Whiteley, M. A., Porter, J. D., and Fenske, R. H. (1992). The Premier for Institutional Research. Florida: Association for Institutional Research.Google Scholar
  71. Whitt, E. J., Pascarella, E. T., Edison, M. I., Nora, A., and Terenzini, P. T. (1998). Perceptions of achilly climateand cognitive outcomes during college. Paper presented before the annual meeting of the Association of Educational Researchers, San Diego.Google Scholar

Copyright information

© Human Sciences Press, Inc. 2001

Authors and Affiliations

  • Alberto F. Cabrera
    • 1
  • Carol L. Colbeck
    • 2
  • Patrick T. Terenzini
    • 2
  1. 1.Center for the Study of Higher Education (CSHE)The Pennsylvania State UniversityUniversity Park
  2. 2.Center for the Study of Higher Education (CSHE)USA

Personalised recommendations