Skip to main content
SpringerLink
Log in

Critical Reflections on Multivocal Analysis and Implications for Design-Based Research

  • Chapter
  • First Online:
Productive Multivocality in the Analysis of Group Interactions

Part of the book series: Computer-Supported Collaborative Learning Series ((CULS,volume 15))

Abstract

This chapter presents critical reflections on the multivocal analyses presented in the preceding chapters in this volume by Teplovs and Fujita, Law and Wong, and Chiu on the asynchronous discussion data collected in an online graduate education course using Knowledge Forum. The multivocal analyses are discussed along five dimensions: theoretical assumptions, purpose of analysis, unit of analysis/unit of interaction, data representations, and manipulations on data representations. The diverse interpretations and findings of pivotal moments are explicated in light of broader dynamic group processes that support knowledge building in online graduate course contexts. The implications of multivocal analysis for design-based research are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Abell, P. (2004). Narrative explanations: An alternative to variable-centered explanation? Annual Review of Sociology, 30, 287–310.

    Article  Google Scholar 

  • American Psychological Association. (2009). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author.

    Google Scholar 

  • Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research. Educational Researcher, 41(2), 16–25.

    Article  Google Scholar 

  • Barab, S. A. (2006). Design-based research: A methodological toolkit for the learning scientist. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 153–169). New York, NY: Cambridge University Press.

    Google Scholar 

  • Barab, S. A., & Kirschner, D. (2001). Guest editor's introduction: Rethinking methodology in the learning sciences. The Journal of the Learning Sciences, 10(1&2), 5–12.

    Article  Google Scholar 

  • Barab, S. A., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13(1), 1–14.

    Article  Google Scholar 

  • Bereiter, C. (2002a). Design research for sustained innovation. Cognitive Studies, Bulletin of the Japanese Cognitive Science Society, 9(3), 321–327.

    Google Scholar 

  • Bereiter, C. (2002b). Education and mind in the knowledge age. Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Bereiter, C., & Scardamalia, M. (2003). Learning to work creatively with knowledge. In E. De Corte, L. Verschaffel, N. Entwistle, & J. van Merrienboer (Eds.), Powerful learning environments: Unravelling basic components and dimensions (pp. 55–68). Oxford: Elsevier Science.

    Google Scholar 

  • Blei, D., Ng, A. Y., & Jordan, M. I. (2003). Journal of Machine Learning Research, 3, 993–1022.

    Google Scholar 

  • Brett, C., Forrester, B. C., & Fujita, N. (2009). Online learning journals as an instructional and self-assessment tool for epistemological growth. Canadian Journal of Learning and Technology, 35(1).

    Google Scholar 

  • Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141–178.

    Article  Google Scholar 

  • Brown, A. L., & Campione, J. C. (1990). Communities of learning and thinking, or a context by any other name. Contributions to Human Development, 21, 108–126.

    Google Scholar 

  • Brown, A. L., & Campione, J. C. (1994). Guided discovery in a community of learners. In K. McGilly (Ed.), Classroom lessons: Integrating cognitive theory and classroom practice (pp. 229–270). Cambridge, MA: Bradford Books/MIT Press.

    Google Scholar 

  • Burtis, J. (1998). Analytic Toolkit for Knowledge Forum. Centre for Applied Cognitive Science, The Ontario Institute for Studies in Education/University of Toronto.

    Google Scholar 

  • Cacciamani, S., Cesareni, D., Martini, F., Ferrini, T., & Fujita, N. (2012). Influence of participation, facilitator styles, and metacognitive reflection on knowledge building in online university courses. Computers & Education, 58(3), 874–884.

    Article  Google Scholar 

  • Chiu, M. M. (2000). Group problem solving processes: Social interactions and individual actions. Journal for the Theory of Social Behavior, 30(1), 27–50.

    Article  Google Scholar 

  • Chiu, M. M. & Fujita, N. (in press). Statistical discourse analysis of online discussions: Informal cognition, social metacognition and knowledge creation. In S. C. Tan, H. J. So, and J. Yeo (Eds.) Knowledge creation in education. New York: Springer.

    Google Scholar 

  • Chiu, M. M., & Kuo, S. W. (2009). From metacognition to social metacognition. Journal of Education Research, 3(4), 1–19.

    Google Scholar 

  • Chiu, M. M. (2014). Cognition and social metacognition: A statistical discourse analysis of an online discussion. In D. D. Suthers et al. (Eds.) Productive Multivocality in the Analysis of Group Interactions, Computer-Supported Collaborative Learning Series 15, DOI 10.1007/978-1-4614-8960-3_23, ©Springer Science+Business Media New York 2013.

    Google Scholar 

  • Chiu, M. M., & Pawlikowski, M. (2013). Social metacognition and micro-creativity. In E. G. Carayannis (Ed.), Encyclopedia of creativity, invention, innovation, and entrepreneurship. New York, NY: Springer.

    Google Scholar 

  • Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9–13.

    Article  Google Scholar 

  • Collins, A. (1992). Toward a design science of education. In E. Scanlon & T. O'Shea (Eds.), New directions in educational technology (pp. 15–20). Berlin: Springer.

    Chapter  Google Scholar 

  • Collins, A. (1999). The changing infrastructure of education research. In E. C. Lagemann & L. S. Shulman (Eds.), Issues in education research: Problems and possibilities (pp. 289–298). San Francisco, CA: Jossey-Bass.

    Google Scholar 

  • Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. The Journal of the Learning Sciences, 13(1), 15–42.

    Article  Google Scholar 

  • de Laat, M., Lally, V., Lipponen, L., & Simons, R.-J. (2007). Investigating patterns of interaction in networked learning and computer-supported collaborative learning: A role for Social Network Analysis. International Journal of Computer-Supported Collaborative Learning, 2, 87–103.

    Article  Google Scholar 

  • Dede, C. (2004). If design-based research is the answer, what is the question? The Journal of the Learning Sciences, 13(1), 105–114.

    Article  Google Scholar 

  • diSessa, A., & Cobb, P. (2004). Ontological innovation and the role of theory in design experiments. The Journal of the Learning Sciences, 13(1), 77–103.

    Article  Google Scholar 

  • Donald, J. (2002). Learning to think: Disciplinary perspectives. San Francisco, CA: Jossey-Bass.

    Google Scholar 

  • Edelson, D. C. (2002). Design research: What we learn when we engage in design. The Journal of the Learning Sciences, 11(1), 105–121.

    Article  Google Scholar 

  • Fujita, N. (2009). Group processes supporting the development of progressive discourse in online graduate courses. Ph.D. Doctoral Dissertation, University of Toronto, Toronto, ON.

    Google Scholar 

  • Fujita, N. (2014). Online graduate education course using Knowledge Forum. In D. D. Suthers et al. (Eds.) Productive Multivocality in the Analysis of Group Interactions, Computer-Supported Collaborative Learning Series 15, DOI 10.1007/978-1-4614-8960-3_20, ©Springer Science+Business Media New York 2013.

    Google Scholar 

  • Fujita, N., & Teplovs, C. (2009). Automating analysis of collaborative discourse: Identifying idea clusters. In C. O'Malley, D. Suthers, P. Reimann, & A. Dimitracopoulou (Eds.), Computer supported collaborative learning practices (Vol. 2, pp. 162–169). Rhodes, Greece: International Society of the Learning Sciences.

    Google Scholar 

  • Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence and computer conferencing in distance education. The American Journal of Distance Education, 15(1), 7–23.

    Article  Google Scholar 

  • Gunawardena, C. N., Lowe, C. A., & Anderson, T. (1997). Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397–431.

    Article  Google Scholar 

  • Hacker, D. J., & Bol, L. (2004). Metacognitive theory. In D. M. McInerney & S. van Etten (Eds.), Big theories revisited (Vol. 4, pp. 275–297). Greenwich, CT: Information Age.

    Google Scholar 

  • Hewitt, J. (1999). The Vocabulary Analyser for Knowledge Forum. Centre for Applied Cognitive Science, The Ontario Institute for Studies in Education/University of Toronto.

    Google Scholar 

  • Hewitt, J. G. (2001). Beyond threaded discourse. Journal of Educational Telecommunications, 7(3), 207–221.

    Google Scholar 

  • Hewitt, J. G. (2005). Towards an understanding of how threads die in asynchronous computer conferences. The Journal of the Learning Sciences, 14(4), 567–589.

    Article  Google Scholar 

  • Hoadley, C. (2002). Creating context: Design-based research in creating and understanding CSCL. In G. Stahl (Ed.), Computer support for collaborative learning 2002 (pp. 453–462). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Jacobbe, T., & Horton, R. M. (2010). Elementary school teachers' comprehension of data displays. Statistics Education Research Journal, 9(1), 27–45.

    Google Scholar 

  • Johnson, M., Bull, S., Reimann, P., & Fujita, N. (2011). Epistemic beliefs and open learner models. In D. Pérez-Marin, M. Kravcik, & O. C. Santos (Eds.), Proceedings of the International Workshop on Personalization Approaches in Learning Environments (PALE 2011) (pp. 19–24). Girona, Spain: Springer.

    Google Scholar 

  • Kelly, A. E., Baek, J. Y., Lesh, R. A., & Bannan-Ritland, B. (2008). Enabling innovations in education and systematizing the impact. In A. E. Kelly, R. A. Lesh, & J. Y. Baek (Eds.), Handbook of Design Research Methods in Education. New York, NY: Routledge.

    Google Scholar 

  • Labaree, D. F. (2003). The peculiar problems of preparing educational researchers. Educational Researcher, 32(4), 13–22.

    Article  Google Scholar 

  • Landauer, T. K., Laham, D., & Derr, M. (2004). From paragraph to graph: Latent semantic analysis for information visualization. PNAS, 101, 5214–5219.

    Article  Google Scholar 

  • Law, N. & Wong, O.-W. (2014). Exploring pivotal moments in students’ knowledge building progress using participation and discourse marker indicators as heuristic guides. In D. D. Suthers et al. (Eds.) Productive Multivocality in the Analysis of Group Interactions, Computer-Supported Collaborative Learning Series 15, DOI 10.1007/978-1-4614-8960-3_22, ©Springer Science+Business Media New York 2014.

    Google Scholar 

  • Lee, E., Chan, C., & van Aalst, J. (2006). Students assessing their own collaborative knowledge building. International Journal of Computer-Supported Collaborative Learning, 1(1), 57–87.

    Article  Google Scholar 

  • Leech, N. L., & Onwuegbuzie, A. J. (2007). An array of qualitative data analysis tools: A call for data analysis triangulation. School Psychology Quarterly, 22(4), 557–584.

    Article  Google Scholar 

  • Maxwell, J. A. (2004). Causal explanation, qualitative research, and scientific inquiry in education. Educational Researcher, 33(2), 3–11.

    Article  Google Scholar 

  • Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: A sourcebook of new methods (2nd ed.). Thousand Oaks, CA: Sage.

    Google Scholar 

  • Onwuegbuzie, A. J., & Dickinson, W. B. (2008). Mixed methods analysis and information visualization: Graphical display for effective communication of research results. The Qualitative Report, 13(2), 204–225.

    Google Scholar 

  • Reimann, P. (2011). Design-based research. In L. Markauskaite, P. Freebody, & J. Irwin (Eds.), Methodological choices and research designs for educational and social change: Linking scholarship, policy and practice (pp. 37–50). New York, NY: Springer.

    Google Scholar 

  • Rourke, L., & Kanuka, H. (2007). Barriers to online critical discourse. International Journal of Computer-Supported Collaborative Learning, 2(1), 105–126.

    Article  Google Scholar 

  • Sandoval, W. (2013). Conjecture mapping: An approach to systematic educational design research. Journal of the Learning Sciences, 22(1), 1–19.

    Article  Google Scholar 

  • Scardamalia, M., & Bereiter, C. (1991). Higher levels of agency in knowledge building: A challenge for the design of new knowledge media. The Journal of the Learning Sciences, 1(1), 37–68.

    Article  Google Scholar 

  • Scardamalia, M., & Bereiter, C. (1994). Computer support for knowledge-building communities. The Journal of the Learning Sciences, 3(3), 265–283.

    Google Scholar 

  • Scardamalia, M. (2002). Collective cognitive responsibility for the advancement of knowledge. In B. Smith (Ed.), Liberal education in a knowledge society (pp. 67–98). Chicago: Open Court.

    Google Scholar 

  • Scardamalia, M., & Bereiter, C. (2003). Knowledge Building. Encyclopedia of Education (2nd ed.). New York, NY: Macmillan Reference.

    Google Scholar 

  • Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and technology. In K. Sawyer (Ed.), Cambridge Handbook of the Learning Sciences. New York, NY: Cambridge University Press.

    Google Scholar 

  • Suthers, D., Dwyer, N., Medina, R., & Vatrapu, R. (2010). A framework for conceptualizing, representing, and analyzing distributed interaction. International Journal of Computer-Supported Collaborative Learning, 5(1), 5–42.

    Article  Google Scholar 

  • Tashakkori, A., & Teddlie, C. (2003). Handbook of mixed methods in social & Behavioral research. Thousand Oaks: Sage.

    Article  Google Scholar 

  • Teplovs, C., & Fujita, N. (2009). Asynchronous Knowledge Forum postings from a graduate course in education. Paper presented at the Pinpointing Pivotal Moments in Collaboration Workshop, STELLAR Alpine Rendez-Vous 2009, Garmisch-Partenkirchen, Germany.

    Google Scholar 

  • Teplovs, C. (2008). The Knowledge Space Visualizer: A tool for visualizing online discourse. In G. Kanselaar, V. Jonker, P. A. Kirschner, & F. Prins (Eds.), Proceedings of the International Conference of the Learning Sciences 2008: Cre8 a learning world. Utrecht, NL: International Society of the Learning Sciences.

    Google Scholar 

  • Teplovs, C., Fujita, N., & Vatrapu, R. (2011). Generating predictive models of learner community dynamics through visualization of networks based on social network analysis and latent semantic analysis. In P. Long, G. Siemens, G. Conole, & D. Gasevic (Eds.), Proceedings of the First International Conference on Learning Analytics & Knowledge 2011 (pp. 147–152). New York, NY: ACM.

    Google Scholar 

  • Teplovs, C. & Fujita, N. (2013). Socio-dynamic Latent Semantic Learner Models. In D. D. Suthers et al. (Eds.) Productive Multivocality in the Analysis of Group Interactions, Computer-Supported Collaborative Learning Series 15, DOI 10.1007/978-1-4614-8960-3_21, ©Springer Science+Business Media New York 2013.

    Google Scholar 

  • Teplovs, C., Donoahue, Z., Scardamalia, M., & Philip, D. (2007). Tools for concurrent, embedded, and transformative assessment of knowledge building processes and progress. In C. A. Chinn, G. Erkens, & S. Puntambekar (Eds.), Proceedings of the 8th International Conference on Computer Supported Collaborative Learning (pp. 721–723). New Brunswick, NJ: International Society of the Learning Sciences.

    Google Scholar 

  • Thagard, P. (1989). Explanatory coherence. Behavioral and Brain Sciences, 12, 435–502.

    Google Scholar 

  • Tscholl, M., & Dowell, J. (2009). Analysis of uptakes and individualistic appropriation in Knowledge Forum: Theoretical perspectives and an example. Paper presented at the Pinpointing Pivotal Moments in Collaboration Workshop, STELLAR Alpine Rendez-Vous 2009, Garmisch-Partenkirchen, Germany.

    Google Scholar 

  • Vatrapu, R., Teplovs, C., Fujita, N., & Bull, S. (2011). Towards visual analytics for teachers' dynamic diagnostic pedagogical decision-making. In P. Long, G. Siemens, G. Conole, & D. Gasevic (Eds.), Proceedings of the First International Conference on Learning Analytics & Knowledge 2011 (pp. 93–98). New York, NY: ACM.

    Google Scholar 

  • Wise, A., & Chiu, M. M. (2011). Analyzing temporal patterns of knowledge construction in a role-based online discussion. International Journal of Computer-Supported Collaborative Learning, 6, 445–470.

    Article  Google Scholar 

  • Woodruff, E., & Brett, C. (1999). Collaborative knowledge building: Preservice teachers and elementary students talking to learn. Language and Education, 13(4), 280–302.

    Article  Google Scholar 

  • Zampa, V., & Lemaire, B. (2002). Latent Semantic Analysis for User Modeling. Journal of Intelligent Information Systems, 19, 15–30.

    Article  Google Scholar 

  • Zhang, J., Scardamalia, M., Reeve, R., & Messina, R. (2009). Designs for collective cognitive responsibility in knowledge building communities. Journal of the Learning Sciences, 18(1), 7–44.

    Article  Google Scholar 

Download references

Acknowledgements

  I would like to thank Chris Teplovs, Nancy Law, Wing Wong, and Ming Ming Chiu for their insightful analyses of this dataset. This data was originally collected with funding from the Social Sciences Research Council of Canada and the Canadian Foundation for Innovation awarded to Clare Brett at the University of Toronto.

Author information

Authors and Affiliations

  1. University of Windsor, Windsor, ON, Canada

    Nobuko Fujita

Authors
  1. Nobuko Fujita
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nobuko Fujita .

Editor information

Editors and Affiliations

  1. Dept. of Information & Computer Sciences, University of Hawai'i at Manoa, Honolulu, Hawaii, USA

    Daniel D. Suthers

  2. ICAR Research Lab, CNRS — University of Lyon, LYON, France

    Kristine Lund

  3. Language Technologies Inst. & HCI Inst., Carnegie Mellon University, Pittsburgh, Pennsylvania, USA

    Carolyn Penstein Rosé

  4. Problemshift, Inc., Windsor, Ontario, Canada

    Chris Teplovs

  5. University of Hong Kong Centre for Info. Technology in Education, Hong Kong, Hong Kong SAR

    Nancy Law

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media New York

About this chapter

Cite this chapter

Fujita, N. (2013). Critical Reflections on Multivocal Analysis and Implications for Design-Based Research. In: Suthers, D., Lund, K., Rosé, C., Teplovs, C., Law, N. (eds) Productive Multivocality in the Analysis of Group Interactions. Computer-Supported Collaborative Learning Series, vol 15. Springer, Boston, MA. https://doi.org/10.1007/978-1-4614-8960-3_24

Download citation

Publish with us

Policies and ethics

Search

Navigation