Skip to main content

Advertisement

SpringerLink
Log in

Towards effective group work assessment: even what you don’t see can bias you

  • Published:
International Journal of Technology and Design Education Aims and scope Submit manuscript

Abstract

In project-based learning (PBL) courses, which are common in design and technology education, instructors regard both the process and the final product to be important. However, conducting an accurate assessment for process feedback is not an easy task because instructors of PBL courses often have to make judgments based on a limited view of group work. In this paper, we provide explanations about how in practice instructors actually exhibit cognitive biases and judgments made using incomplete information in the context of an engineering design education classroom. More specifically, we hypothesize that instructors would be susceptible to human errors that are well known in social psychology, the halo effect and the fundamental attribution error, because they have a limited view of group work when they facilitate distributed and remote groups. Through this study, we present two main contributions, namely (1) insights based on classroom data about limitations of current instructor assessment practices, (2) an illustration of using principles from social psychology as a lens for exploring important design questions for designing tools that monitor support oversight of group work. In addition to the study, we illustrate how the findings from our classroom study can be used for effective group assessments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Balzer, W. K., & Sulsky, L. M. (1992). Halo and performance appraisal research: A critical examination. Journal of Applied Psychology, 77(6), 975–985.

    Article  Google Scholar 

  • Becker, B. E., & Cardy, R. L. (1986). Influence of halo error on appraisal effectiveness: A conceptual and empirical reconsideration. Journal of Applied Psychology, 71(4), 662–671.

    Article  Google Scholar 

  • Beehr, T. A., Ivanitskaya, L., Hansen, C. P., Erofeev, D., & Gudanowski, D. M. (2001). Evaluation of 360 degree feedback ratings: Relationships with each other and with performance and selection predictors. Journal of Organizational Behavior, 22(7), 775–788.

    Article  Google Scholar 

  • Bober, M., Sullivan, H., Lowther, D., & Harrison, P. (1998). Instructional practices of teachers enrolled in educational technology and general educational programs. Educational Technology Research and Development, 46(3), 81–97.

    Article  Google Scholar 

  • Burger, J. (1991). Changes in attributions over time: The ephemeral fundamental attribution error. Social Cognition, 9(2), 182–193.

    Article  Google Scholar 

  • Carroll, J. S. (1978). Causal attributions in expert parole decisions. Journal of Personality and Social Psychology, 36(12), 1501–1511.

    Article  Google Scholar 

  • Chen, M. (2003). Visualizing the pulse of a classroom. In Proceedings of international multimedia conference, pp. 555–561, Berkeley, CA: ACM Press.

  • Cook, M., & Klumper, D. (1999). Metacognitive, social and interpersonal skills and aptitudes in officer performance with distributed teams. In Paper presented at RTO HFM workshop on Officer Selection, in Monterey, USA.

  • Cooper, W. (1981). Ubiquitous halo. Psychological Bulletin, 90(2), 218–244.

    Article  Google Scholar 

  • DiMicco, J., Hoolenbach, K., & Bender, W. (2006). Using visualizations to review a group’s interaction dynamics. In Proceedings of the SIGCHI conference on human factors in computing systems, pp 706–711, New York: ACM Press.

  • Dimitracopoulou, A., Hoppe, U., & Dillenbourg, P. (2004). Interaction analysis supporting participants during technology based collaborative activities. In Paper presented at CSCL symposium, October 7–9, in Kaleidoscope Noe, Lausanne.

  • Dutson, A. J., Todd, R. H., Magleby, S. P., & Sorensen, C. D. (1997). A review of literature on teaching design through project-oriented capstone courses. Journal of Engineering Education, 76(1), 17–28.

    Article  Google Scholar 

  • Feeley, T. (2002). Evidence of halo effects in student evaluations of communication instruction. Communication Education, 51(3), 225–236.

    Article  Google Scholar 

  • Gipps, C. (2005). What is the role for ICT-based assessment in universities? Studies in Higher Education, 30(2), 171–180.

    Article  Google Scholar 

  • Gómez Puente, S. M., van Eijck, M., & Jochems, W. (2013). Empirical validation of characteristics of design-based learning in higher education. International Journal of Engineering Education, 29(2), 491–503.

    Google Scholar 

  • Gopinath, C. (1999). Alternatives to instructor assessment of class participation. Journal of Education for Business, 75(1), 10–14.

    Article  Google Scholar 

  • Harvey, J., Town, J. P., & Yarkim, K. (1981). How fundamental is the fundamental attribution error? Journal of Personality and Social Psychology, 40(2), 346–349.

    Article  Google Scholar 

  • Jochems, W., & Kreijns, K. (2006). Measuring social aspects of distributed learning groups. European Educational Research Journal, 5(2), 110–121.

    Article  Google Scholar 

  • Johari, A., & Bradshaw, A. (2008). Project-based learning in an internship program: A qualitative study of related roles and their motivational attributes. Education Technology Research and Development, 56, 329–359.

    Article  Google Scholar 

  • Kay, J., Maisonneuve, N., Yacef, K., & Reimann, P. (2006). Wattle tree: What’ll it tell us? University of Sydney Technical Report.

  • Kelsey, D. M., Kearney, P., Plax, T. G., Allen, T. H., & Ritter, K. J. (2004). College students’ attributions of teacher misbehaviors. Communication Education, 53(1), 40–55.

    Article  Google Scholar 

  • Kimbell, R. (2007). E-assessment in project e-scape. Design and Technology Education: An International Journal, 12(2), 66–76.

    Google Scholar 

  • Lambart, E., Sharma, A., & Levy, M. (1997). What information can relationship marketers obtain from customer evaluations of salespeople? Industrial Marketing Management, 26(2), 177–187.

    Article  Google Scholar 

  • Madan, A., Caneel, R., and Pentland, A. (2004). GroupMedia: Distributed multimodal interfaces. In Proceedings of sixth international conference on multimodal interfaces ICMI04.

  • McPherson, M., & Young, S. L. (2004). What students think when teachers get upset: Fundamental attribution error and student generated reasons for teacher anger. Communication Quarterly, 52(4), 357–369.

    Article  Google Scholar 

  • Meier, A., Spada, H., & Rummel, N. (2007). Evaluating collaboration: A rating scheme for assessing the quality of collaborative process. International Journal of Computer-Supported Collaborative Learning, 2, 63–86.

    Article  Google Scholar 

  • Nicol, D., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218.

    Article  Google Scholar 

  • Phielix, C., Prins, F., & Kirschner, P. (2010). Awareness of group performance in a CSCL-environment: Effects of peer feedback and reflection. Computers and Human Behavior, 26(2), 151–161.

    Article  Google Scholar 

  • Pianesi, F., Zancarnaro, M., Not, E., Leonardi, C., Falcon, V., & Lepri, B. (2008). Multimodal support to group dynamics. Personal and Ubiquitous Computing, 12(3), 181–195.

    Article  Google Scholar 

  • Price, M., Handley, K., & Millar, J. (2011). Feedback: Focusing attention on engagement. Studies in Higher Education, 36(8), 879–896.

    Article  Google Scholar 

  • Reimann, P., Yacef, K., & Kay, J. (2011). Analyzing collaborative interactions with data mining methods for the benefit of learning. In S. Puntambekar, G. Erkens, & C. Hmelo-Silver (Eds.), Analyzing interactions in CSCL: Methods, approaches and issue (pp. 161–186). New York: Springer.

    Chapter  Google Scholar 

  • Ren, Y., Kiesler, S., & Fussell, S. R. (2008). Multiple group coordination in complex and dynamic task environments: Interruptions, coping mechanisms, and technology recommendations. Journal of Management Information Systems, 25(1), 107–133.

    Article  Google Scholar 

  • Rienks, R.J., Zhang, D., Gatica-Perez, D., & Post, W. (2006) Detection and application of influence rankings in small group meetings. In Proceedings of eighth international conference on multimodal interfaces ICMI’06.

  • Rohde, M., Klamma, R., Jarke, M., & Wulf, V. (2007). Reality is our laboratory: Communities of practice in applied computer science. Behavior and Information Technology, 26(1), 81–94.

    Article  Google Scholar 

  • Ross, L. (1977). The intuitive psychologist and his shortcomings: Distortions in the attribution process. In L. Berkowitz (Ed.), Advances in experimental social psychology (pp. 173–220). New York: Academic Press.

    Google Scholar 

  • Tetlock, P. (1985). Accountability: A social check on the fundamental attribution error. Social Psychology Quarterly, 48(3), 227–236.

    Article  Google Scholar 

  • Thorndike, E. L. (1920). A constant error on psychological rating. Journal of Applied Psychology, 4(1), 25–29.

    Article  Google Scholar 

  • Weinberger, A., & Fischer, F. (2005). A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers and Education, 46(1), 71–95.

    Article  Google Scholar 

  • Wong, J., Oh, L. M., Ou, J., Rosé, C. P., Yang, J., & Fussell, S.R. (2007). Sharing a single expert among multiple partners. In Proceedings of the SIGCHI conference on human factors in computing systems, pp. 261–270. New York: ACM Press.

Download references

Author information

Authors and Affiliations

  1. Knowledge Service Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon, Republic of Korea

    Gahgene Gweon

  2. Graduate School of Communication and Arts, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 120-749, Korea

    Soojin Jun

  3. Civil and Environmental Engineering, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA, 15213, USA

    Susan Finger

  4. LTI, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA, 15213, USA

    Carolyn Penstein Rosé

Authors
  1. Gahgene Gweon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Soojin Jun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Susan Finger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Carolyn Penstein Rosé
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Soojin Jun.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gweon, G., Jun, S., Finger, S. et al. Towards effective group work assessment: even what you don’t see can bias you. Int J Technol Des Educ 27, 165–180 (2017). https://doi.org/10.1007/s10798-015-9332-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10798-015-9332-1

Keywords

Search

Navigation