Advertisement

Towards effective group work assessment: even what you don’t see can bias you

  • Gahgene Gweon
  • Soojin Jun
  • Susan Finger
  • Carolyn Penstein Rosé
Article

Abstract

In project-based learning (PBL) courses, which are common in design and technology education, instructors regard both the process and the final product to be important. However, conducting an accurate assessment for process feedback is not an easy task because instructors of PBL courses often have to make judgments based on a limited view of group work. In this paper, we provide explanations about how in practice instructors actually exhibit cognitive biases and judgments made using incomplete information in the context of an engineering design education classroom. More specifically, we hypothesize that instructors would be susceptible to human errors that are well known in social psychology, the halo effect and the fundamental attribution error, because they have a limited view of group work when they facilitate distributed and remote groups. Through this study, we present two main contributions, namely (1) insights based on classroom data about limitations of current instructor assessment practices, (2) an illustration of using principles from social psychology as a lens for exploring important design questions for designing tools that monitor support oversight of group work. In addition to the study, we illustrate how the findings from our classroom study can be used for effective group assessments.

Keywords

Instructor assessment Halo effect Fundamental attribution error Project-based learning Group work 

References

  1. Balzer, W. K., & Sulsky, L. M. (1992). Halo and performance appraisal research: A critical examination. Journal of Applied Psychology, 77(6), 975–985.CrossRefGoogle Scholar
  2. Becker, B. E., & Cardy, R. L. (1986). Influence of halo error on appraisal effectiveness: A conceptual and empirical reconsideration. Journal of Applied Psychology, 71(4), 662–671.CrossRefGoogle Scholar
  3. Beehr, T. A., Ivanitskaya, L., Hansen, C. P., Erofeev, D., & Gudanowski, D. M. (2001). Evaluation of 360 degree feedback ratings: Relationships with each other and with performance and selection predictors. Journal of Organizational Behavior, 22(7), 775–788.CrossRefGoogle Scholar
  4. Bober, M., Sullivan, H., Lowther, D., & Harrison, P. (1998). Instructional practices of teachers enrolled in educational technology and general educational programs. Educational Technology Research and Development, 46(3), 81–97.CrossRefGoogle Scholar
  5. Burger, J. (1991). Changes in attributions over time: The ephemeral fundamental attribution error. Social Cognition, 9(2), 182–193.CrossRefGoogle Scholar
  6. Carroll, J. S. (1978). Causal attributions in expert parole decisions. Journal of Personality and Social Psychology, 36(12), 1501–1511.CrossRefGoogle Scholar
  7. Chen, M. (2003). Visualizing the pulse of a classroom. In Proceedings of international multimedia conference, pp. 555–561, Berkeley, CA: ACM Press.Google Scholar
  8. Cook, M., & Klumper, D. (1999). Metacognitive, social and interpersonal skills and aptitudes in officer performance with distributed teams. In Paper presented at RTO HFM workshop on Officer Selection, in Monterey, USA.Google Scholar
  9. Cooper, W. (1981). Ubiquitous halo. Psychological Bulletin, 90(2), 218–244.CrossRefGoogle Scholar
  10. DiMicco, J., Hoolenbach, K., & Bender, W. (2006). Using visualizations to review a group’s interaction dynamics. In Proceedings of the SIGCHI conference on human factors in computing systems, pp 706–711, New York: ACM Press.Google Scholar
  11. Dimitracopoulou, A., Hoppe, U., & Dillenbourg, P. (2004). Interaction analysis supporting participants during technology based collaborative activities. In Paper presented at CSCL symposium, October 7–9, in Kaleidoscope Noe, Lausanne.Google Scholar
  12. Dutson, A. J., Todd, R. H., Magleby, S. P., & Sorensen, C. D. (1997). A review of literature on teaching design through project-oriented capstone courses. Journal of Engineering Education, 76(1), 17–28.CrossRefGoogle Scholar
  13. Feeley, T. (2002). Evidence of halo effects in student evaluations of communication instruction. Communication Education, 51(3), 225–236.CrossRefGoogle Scholar
  14. Gipps, C. (2005). What is the role for ICT-based assessment in universities? Studies in Higher Education, 30(2), 171–180.CrossRefGoogle Scholar
  15. Gómez Puente, S. M., van Eijck, M., & Jochems, W. (2013). Empirical validation of characteristics of design-based learning in higher education. International Journal of Engineering Education, 29(2), 491–503.Google Scholar
  16. Gopinath, C. (1999). Alternatives to instructor assessment of class participation. Journal of Education for Business, 75(1), 10–14.CrossRefGoogle Scholar
  17. Harvey, J., Town, J. P., & Yarkim, K. (1981). How fundamental is the fundamental attribution error? Journal of Personality and Social Psychology, 40(2), 346–349.CrossRefGoogle Scholar
  18. Jochems, W., & Kreijns, K. (2006). Measuring social aspects of distributed learning groups. European Educational Research Journal, 5(2), 110–121.CrossRefGoogle Scholar
  19. Johari, A., & Bradshaw, A. (2008). Project-based learning in an internship program: A qualitative study of related roles and their motivational attributes. Education Technology Research and Development, 56, 329–359.CrossRefGoogle Scholar
  20. Kay, J., Maisonneuve, N., Yacef, K., & Reimann, P. (2006). Wattle tree: What’ll it tell us? University of Sydney Technical Report. Google Scholar
  21. Kelsey, D. M., Kearney, P., Plax, T. G., Allen, T. H., & Ritter, K. J. (2004). College students’ attributions of teacher misbehaviors. Communication Education, 53(1), 40–55.CrossRefGoogle Scholar
  22. Kimbell, R. (2007). E-assessment in project e-scape. Design and Technology Education: An International Journal, 12(2), 66–76.Google Scholar
  23. Lambart, E., Sharma, A., & Levy, M. (1997). What information can relationship marketers obtain from customer evaluations of salespeople? Industrial Marketing Management, 26(2), 177–187.CrossRefGoogle Scholar
  24. Madan, A., Caneel, R., and Pentland, A. (2004). GroupMedia: Distributed multimodal interfaces. In Proceedings of sixth international conference on multimodal interfaces ICMI04. Google Scholar
  25. McPherson, M., & Young, S. L. (2004). What students think when teachers get upset: Fundamental attribution error and student generated reasons for teacher anger. Communication Quarterly, 52(4), 357–369.CrossRefGoogle Scholar
  26. Meier, A., Spada, H., & Rummel, N. (2007). Evaluating collaboration: A rating scheme for assessing the quality of collaborative process. International Journal of Computer-Supported Collaborative Learning, 2, 63–86.CrossRefGoogle Scholar
  27. Nicol, D., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218.CrossRefGoogle Scholar
  28. Phielix, C., Prins, F., & Kirschner, P. (2010). Awareness of group performance in a CSCL-environment: Effects of peer feedback and reflection. Computers and Human Behavior, 26(2), 151–161.CrossRefGoogle Scholar
  29. Pianesi, F., Zancarnaro, M., Not, E., Leonardi, C., Falcon, V., & Lepri, B. (2008). Multimodal support to group dynamics. Personal and Ubiquitous Computing, 12(3), 181–195.CrossRefGoogle Scholar
  30. Price, M., Handley, K., & Millar, J. (2011). Feedback: Focusing attention on engagement. Studies in Higher Education, 36(8), 879–896.CrossRefGoogle Scholar
  31. Reimann, P., Yacef, K., & Kay, J. (2011). Analyzing collaborative interactions with data mining methods for the benefit of learning. In S. Puntambekar, G. Erkens, & C. Hmelo-Silver (Eds.), Analyzing interactions in CSCL: Methods, approaches and issue (pp. 161–186). New York: Springer.CrossRefGoogle Scholar
  32. Ren, Y., Kiesler, S., & Fussell, S. R. (2008). Multiple group coordination in complex and dynamic task environments: Interruptions, coping mechanisms, and technology recommendations. Journal of Management Information Systems, 25(1), 107–133.CrossRefGoogle Scholar
  33. Rienks, R.J., Zhang, D., Gatica-Perez, D., & Post, W. (2006) Detection and application of influence rankings in small group meetings. In Proceedings of eighth international conference on multimodal interfaces ICMI’06. Google Scholar
  34. Rohde, M., Klamma, R., Jarke, M., & Wulf, V. (2007). Reality is our laboratory: Communities of practice in applied computer science. Behavior and Information Technology, 26(1), 81–94.CrossRefGoogle Scholar
  35. Ross, L. (1977). The intuitive psychologist and his shortcomings: Distortions in the attribution process. In L. Berkowitz (Ed.), Advances in experimental social psychology (pp. 173–220). New York: Academic Press.Google Scholar
  36. Tetlock, P. (1985). Accountability: A social check on the fundamental attribution error. Social Psychology Quarterly, 48(3), 227–236.CrossRefGoogle Scholar
  37. Thorndike, E. L. (1920). A constant error on psychological rating. Journal of Applied Psychology, 4(1), 25–29.CrossRefGoogle Scholar
  38. Weinberger, A., & Fischer, F. (2005). A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers and Education, 46(1), 71–95.CrossRefGoogle Scholar
  39. Wong, J., Oh, L. M., Ou, J., Rosé, C. P., Yang, J., & Fussell, S.R. (2007). Sharing a single expert among multiple partners. In Proceedings of the SIGCHI conference on human factors in computing systems, pp. 261–270. New York: ACM Press.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  • Gahgene Gweon
    • 1
  • Soojin Jun
    • 2
  • Susan Finger
    • 3
  • Carolyn Penstein Rosé
    • 4
  1. 1.Knowledge Service EngineeringKorea Advanced Institute of Science and Technology (KAIST)DaejeonRepublic of Korea
  2. 2.Graduate School of Communication and ArtsYonsei UniversitySeoulKorea
  3. 3.Civil and Environmental EngineeringCarnegie Mellon UniversityPittsburghUSA
  4. 4.LTICarnegie Mellon UniversityPittsburghUSA

Personalised recommendations