Skip to main content
SpringerLink
Log in

Designing Scalable and Sustainable Peer Interactions Online

  • Chapter
  • First Online:
Design Thinking Research

Part of the book series: Understanding Innovation ((UNDINNO))

Abstract

When students work with peers, they learn more actively, build richer knowledge structures, and connect material to their lives. However, not every peer learning experience online sees successful adoption. This chapter first introduces PeerStudio, an assessment platform that leverages the large number of students’ peers in online classes to enable rapid feedback on in-progress work. Students submit their draft, give rubric-based feedback on two peers’ drafts, and then receive peer feedback. Students can integrate the feedback and repeat this process as often as they desire. PeerStudio demonstrates how rapid feedback on in-progress work improves course outcomes. We then articulate and address three adoption and implementation challenges for peer learning platforms such as PeerStudio. First, peer interactions struggle to bootstrap critical mass. However, class incentives can signal importance and spur initial usage. Second, online classes have limited peer visibility and awareness, so students often feel alone even when surrounded by peers. We find that highlighting interdependence and strengthening norms can mitigate this issue. Third, teachers can readily access “big” aggregate data but not “thick” contextual data that helps build intuitions, so software should guide teachers’ scaffolding of peer interactions. We illustrate these challenges through studying 8500 students’ usage of PeerStudio and another peer learning platform: Talkabout. Efficacy is measured through sign-up and participation rates and the structure and duration of student interactions. This research demonstrates how large classes can leverage their scale to encourage mastery through rapid feedback and revision, and suggests secret ingredients to make such peer interactions sustainable at scale.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Anderson S, Rodin J (1989) Is bad news always bad?: cue and feedback effects on intrinsic motivation. J Appl Soc Psychol 19(6):449–467

    Article  Google Scholar 

  • Andrade HG (2001) The effects of instructional rubrics on learning to write. Current Issues Educ 4:4

    Google Scholar 

  • Andrade HG (2005) Teaching with rubrics: the good, the bad, and the ugly. Coll Teach 53(1):27–31

    Article  Google Scholar 

  • André P, Bernstein M, Luther K (2012) Who gives a tweet? In: Proceedings of the ACM 2012 conference on computer supported cooperative work—CSCW’12, ACM Press, p 471

    Google Scholar 

  • Bakshy E, Karrer B, Adamic LA (2009) Social influence and the diffusion of user-created content. In: Proceedings of the 10th ACM conference on electronic commerce, pp 325–334

    Google Scholar 

  • Balcazar FE, Hopkins BL, Suarez Y (1986) A critical, objective review of performance feedback. J Organ Behav Manag 7:2

    Google Scholar 

  • Bransford J, Brown A, Cocking R (2000) How people learn. The National Academies Press, Washington, DC

    Google Scholar 

  • Bransford J, Schwartz DL (1999) Rethinking transfer: a simple proposal with multiple implications. Rev Res Educ 24:61–100

    Google Scholar 

  • Breslow L, Pritchard D, DeBoer J, Stump G, Ho A, Seaton D (2013) Studying learning in the worldwide classroom: Research into edX’s first MOOC

    Google Scholar 

  • Brooks M, Basu S, Jacobs C, Vanderwende L (2014) Divide and correct: using clusters to grade short answers at scale. Learning at Scale

    Google Scholar 

  • Buxton B (2007) Sketching user experiences: getting the design right and the right design. Morgan Kaufmann, San Francisco

    Google Scholar 

  • Cambre J, Kulkarni C, Bernstein MS, Klemmer SR (2014) Talkabout: small-group discussions in massive global classes. Learning@Scale

    Google Scholar 

  • Carlson PA, Berry FC (2003) Calibrated peer review and assessing learning outcomes. In: Frontiers in education conference

    Google Scholar 

  • Chen M (2001) Design of a virtual auditorium. In: MULTIMEDIA’01 Proceedings of the ninth ACM international conference on multimedia, pp 19–28

    Google Scholar 

  • Cheng J, Adamic L, Dow P, Kleinberg J, Leskovec J (2014) Can cascades be predicted? In: Proceedings of the 23rd international conference on world wide web, pp 925–936

    Google Scholar 

  • Cialdini R, Goldstein N (2004) Social influence: compliance and conformity. Annu Rev Psychol 55:591–621

    Article  Google Scholar 

  • Coetzee D, Fox A, Hearst MA, Hartmann B (2014) Chatrooms in MOOCs: all talk and no action. In: Proceeding of the ACM conference on learning @ scale, ACM Press, pp 127–136

    Google Scholar 

  • Coetzee D, Lim S, Fox A, Hartmann B, Hearst MA (2015) Structuring interactions for large-scale synchronous peer learning. In: CSCW: ACM Conference on computer supported collaborative work

    Google Scholar 

  • Crouch CH, Mazur E (2001) Peer instruction: ten years of experience and results. Am J Phys 69(9):970

    Article  Google Scholar 

  • Dawes RM (1979) The robust beauty of improper linear models in decision making. Am Psychol 34(7):571

    Article  Google Scholar 

  • Dourish P, Bell G (2007) The infrastructure of experience and the experience of infrastructure: meaning and structure in everyday encounters with space. Environ Plan B Plan Des 34(3):414

    Article  Google Scholar 

  • Dourish P, Bellotti V (1992) Awareness and coordination in shared workspaces. In: Proceeding CSCW’92 Proceedings of the 1992 ACM conference on computer-supported cooperative work, pp 107–114

    Google Scholar 

  • Dow SP, Heddleston K, Klemmer SR (2009) The efficacy of prototyping under time constraints. In: Proceeding of the ACM conference on creativity and cognition, ACM Press, p 165

    Google Scholar 

  • Dow S, Fortuna J, Schwartz D, Altringer B, Klemmer S (2011) Prototyping dynamics: sharing multiple designs improves exploration, group rapport, and results. In: Proceedings of the 2011 annual conference on human factors in computing systems, pp 2807–2816

    Google Scholar 

  • Erickson T, Kellogg W (2000) Social translucence: an approach to designing systems that support social processes. In: ACM Transactions on Computer-Human Interaction (TOCHI)—Special issue on human-computer interaction in the new millennium, Part 1 7(1), pp 59–83

    Google Scholar 

  • Ericsson KA, Krampe RT, Tesch-Römer C (1993) The role of deliberate practice in the acquisition of expert performance. Psychol Rev 100:3

    Article  Google Scholar 

  • Falchikov N, Goldfinch J (2000) Student peer assessment in higher education: a meta-analysis comparing peer and teacher marks. Rev Educ Res 70(3):287–322

    Article  Google Scholar 

  • Fast E, Lee C, Aiken A, Bernstein MS, Koller D, Smith E (2013) Crowd-scale interactive formal reasoning and analytics. In: Proceedings of the 26th annual ACM symposium on user interface software and technology

    Google Scholar 

  • Gray WD, Boehm-Davis DA (2000) Milliseconds matter: an introduction to microstrategies and to their use in describing and predicting interactive behavior. J Exp Psychol 6(4):322–335

    Google Scholar 

  • Greenberg S (2009) Embedding a design studio course in a conventional computer science program. In: Creativity and HCI: From Experience to Design in Education. Springer, pp 23–41

    Google Scholar 

  • Grudin J (1994) Groupware and social dynamics: eight challenges for developers. Commun ACM 37(1):92–105

    Article  Google Scholar 

  • Guskey TR (2007) Closing achievement gaps: revisiting Benjamin S. Bloom’s “Learning for Mastery”. J Adv Acad 19(1):8–31

    Google Scholar 

  • Hattie J, Timperley H (2007) The power of feedback. Rev Educ Res 77(1):81–112

    Article  Google Scholar 

  • Heffernan N, Heffernan C, Dietz K, Soffer D, Pellegrino JW, Goldman SR, Dailey M (2012) Improving mathematical learning outcomes through automatic reassessment and relearning. AERA

    Google Scholar 

  • Kim S-M, Pantel P, Chklovski T, Pennacchiotti M (2006) Automatically assessing review helpfulness. In: Proceedings of the 2006 conference on empirical methods in natural language processing, association for computational linguistics, pp 423–430

    Google Scholar 

  • Kittur A, Chi EH, Suh B (2008) Crowdsourcing user studies with mechanical turk. In: Proceeding of CHI, ACM Press, p 453

    Google Scholar 

  • Kizilcec RF, Schneider E (2015) Motivation as a lens to understand online learners: towards data-driven design with the OLEI scale

    Google Scholar 

  • Klemmer SR (2015) Katayanagi Lecture at CMU: the power of examples

    Google Scholar 

  • Klemmer SR, Hartmann B, Takayama L (2006) How bodies matter: five themes for interaction design. In: Proceedings of the 6th conference on designing interactive systems, pp 140–149

    Google Scholar 

  • Kluger AN, DeNisi A (1996) The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull 119(2):254–284

    Article  Google Scholar 

  • Konstan JA, Walker JD, Brooks DC, Brown K, Ekstrand MD (2014) Teaching recommender systems at large scale. In: Proceeding of the ACM conference on learning @ scale conference, ACM Press

    Google Scholar 

  • Kraut RE, Resnick P, Kiesler S, Burke M, Chen Y, Kittur N, Konstan J, Ren Y, Riedl J (2012) Building successful online communities: evidence-based social design (Google eBook), MIT Press

    Google Scholar 

  • Krosnick JA (1999) Survey research. Annu Rev Psychol 50(1):537–567

    Article  Google Scholar 

  • Kulik JA, Kulik C-LC (1987) Timing of feedback and verbal learning. Rev Educ Res 58(1):79–97

    Article  Google Scholar 

  • Kulkarni C, Wei KP, Le H, Chia D, Papadopoulos K, Cheng J, Koller D, Klemmer SR (2013) Peer and self assessment in massive online classes. ACM Trans Comput-Hum Interact (TOCHI) 20(6):33

    Article  Google Scholar 

  • Kulkarni C, Socher R, Bernstein MS, Klemmer SR (2014) Scaling short-answer grading by combining peer assessment with algorithmic scoring. In: ACM conference on learning @ scale, ACM

    Google Scholar 

  • Kulkarni C, Cambre J, Kotturi Y, Bernstein M, Klemmer S (2015) Talkabout: making distance matter with small groups in massive classes. In: CSCW: ACM conference on computer supported collaborative work

    Google Scholar 

  • Latham GP, Locke EA (1991) Self-regulation through goal setting. Organ Behav Hum Decis Process 50(2):212–247

    Article  Google Scholar 

  • Ling K, Beenen G (2005) Using social psychology to motivate contributions to online communities. J Comput-Mediat Commun 10(4):00

    Article  Google Scholar 

  • Marsh RL, Landau JD, Hicks JL (1996) How examples may (and may not) constrain creativity. Mem Cogn 24(5):669–680

    Article  Google Scholar 

  • O’Donnell AM, Dansereau DF (1995) Scripted cooperation in student dyads: a method for analyzing and enhancing academic learning and performance. In: Interaction in cooperative groups: the theoretical anatomy of group learning. Cambridge University Press, Cambridge, UK, pp 120–141

    Google Scholar 

  • Plattner H, Meinel C, Leifer L (eds) (2015) Design thinking research: making design thinking foundational. Springer, Heidelber

    Google Scholar 

  • Porter L, Lee CB, Simon B (2013) Halving fail rates using peer instruction: a study of four computer science courses. In: Proceeding SIGCSE’13 Proceeding of the 44th ACM tech nical symposium on computer science education, pp 177–182

    Google Scholar 

  • Rosenberg J, Lorenzo M, Mazur E (2007) Peer instruction: making science engaging. In: Handbook of college science teaching. NSTA Press, Arlington, VA, pp 77–85

    Google Scholar 

  • Smith M, Wood W, Adams W, Wieman C, Knight J, Guild N, Su T (2009) Why peer discussion improves student performance on in-class concept questions. Science 323(5910):122–124

    Article  Google Scholar 

  • Sommers N (1982) Responding to student writing. Coll Compos Commun 33(2):148–156

    Article  Google Scholar 

  • Stephens-Martinez K, Hearst MA, Fox A (2014) Monitoring MOOCs: which information sources do instructors value? In: Proceedings of the first ACM conference on learning @ scale conference, pp 79–88

    Google Scholar 

  • Tinapple D, Olson L, Sadauskas J (2013) CritViz: web-based software supporting peer critique in large creative classrooms. Bull IEEE Techn Comm Learn Technol 15(1):29

    Google Scholar 

  • Turkle S (2011) Alone together: why we expect more from technology and less from each other. Basic Books, New York

    Google Scholar 

Download references

Acknowledgments

We are grateful to Scott Plous, Ed Hutchins and all instructors who used Talkabout and PeerStudio for their thought-provoking discussions and feedback; Courtney Noh and Julia Cambre for contributing to software development; Coursera and OpenEdX for platform integration and encouraging instructors to try PeerStudio; and the many students who taught and learned with their peers and shared their stories with us. This research was funded in part through NSF grants #1351131 and #1444865, the Hasso Plattner Design Thinking Program, and the Siebel Scholars Program. This research was conducted under Stanford IRB protocol #30324 and #30319.

Author information

Authors and Affiliations

  1. Stanford University, Stanford, CA, USA

    Chinmay Kulkarni & Michael S. Bernstein

  2. University of California, San Diego, CA, USA

    Yasmine Kotturi & Scott Klemmer

Authors
  1. Chinmay Kulkarni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yasmine Kotturi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael S. Bernstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Scott Klemmer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chinmay Kulkarni .

Editor information

Editors and Affiliations

  1. Hasso Plattner Institute for Software Systems Engineering, Potsdam, Germany

    Hasso Plattner

  2. Hasso Plattner Institute for Software Systems Engineering, Potsdam, Germany

    Christoph Meinel

  3. Stanford University, Stanford, California, USA

    Larry Leifer

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Kulkarni, C., Kotturi, Y., Bernstein, M.S., Klemmer, S. (2016). Designing Scalable and Sustainable Peer Interactions Online. In: Plattner, H., Meinel, C., Leifer, L. (eds) Design Thinking Research. Understanding Innovation. Springer, Cham. https://doi.org/10.1007/978-3-319-40382-3_14

Download citation

Publish with us

Policies and ethics

Search

Navigation