Skip to main content
SpringerLink
Log in

Scoping the Emerging Field of Quantitative Ethnography: Opportunities, Challenges and Future Directions

  • Conference paper
  • First Online:
Advances in Quantitative Ethnography (ICQE 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1312))

Included in the following conference series:

Abstract

Quantitative Ethnography (QE) is an emerging methodological approach that combines ethnographic and statistical tools to analyze both Big Data and smaller data to study human behavior and interactions. This paper presents a methodological scoping review of 60 studies employing QE approaches with an intention to characterize and establish where the boundaries of QE might and should be in order to establish the identity of the field. The key finding is that QE researchers have enough commonality in their approach to the analysis of human behavior with a strong focus on grounded analysis, the validity of codes and consistency between quantitative models and qualitative analysis. Nonetheless, in order to reach a larger audience, the QE community should attend to a number of conceptual and methodological issues (e.g. interpretability). We believe that the strength of work from individual researchers reported in this review and initiatives such as the recently established International Society for Quantitative Ethnography (ISQE) can present a powerful force to shape the identity of the QE community.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Brown, B., Chui, M., Manyika, J.: Are you ready for the era of ‘big data’. McKinsey Q. 4(1), 24–35 (2011)

    Google Scholar 

  2. Shaffer, D.W.: Quantitative Ethnography. Cathcart Press, Madison (2017)

    Google Scholar 

  3. Wu, B., Hu, Y., Ruis, A., Wang, M.: Analysing computational thinking in collaborative programming: a quantitative ethnography approach. J. Comput. Assist. Learn. 35(3), 421–434 (2019)

    Article  Google Scholar 

  4. Shaffer, D.W., Collier, W., Ruis, A.: A tutorial on epistemic network analysis: analyzing the structure of connections in cognitive, social, and interaction data. J. Learn. Anal. 3(3), 9–45 (2016)

    Article  Google Scholar 

  5. Misfeldt, M., Spikol, D., Bruun, J., Saqr, M., Kaliisa, R., Ruis, A., Eagan, B.: Quantitative ethnography as a framework for network analysis–a discussion of the foundations for network approaches to learning analysis. In: LAK 2010 Companion Proceedings (2020)

    Google Scholar 

  6. Levac, D., Colquhoun, H., O’Brien, K.K.: Scoping studies: advancing the methodology. Implement. Sci. 5(1) (2010). https://doi.org/10.1186/1748-5908-5-69

  7. Munn, Z., Peters, M.D., Stern, C., Tufanaru, C., McArthur, A., Aromataris, E.: Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med. Res. Methodol. 18(1) (2018). https://doi.org/10.1186/s12874-018-0611-x

  8. Gee, J.P.: Discourse, small d, big D. In: Tracy, K., Sandel, T., Ilie, C. (eds.) The International Encyclopedia of Language and Social Interaction, pp. 1–5 (2015)

    Google Scholar 

  9. Geertz, C.: Deep play: notes on the Balinese cockfight. In: Crothers, L., Lockhart, C. (eds.) The Interpretation of Cultures. Selected Essays, pp. 412–453. Palgrave Macmillan, New York (1973)

    Google Scholar 

  10. Wise, A.F., Shaffer, D.W.: Why theory matters more than ever in the age of big data. J. Learn. Anal. 2(2), 5–13 (2015)

    Article  Google Scholar 

  11. Shaffer, D.W., Ruis, A.R.: Epistemic network analysis: a worked example of theory-based learning analytics. In: Lang, C., Siemens, G., Wise, A.F., Gašević, D. (eds.) Handbook of Learning Analytics, pp. 175–187. Society for Learning Analytics Research (2017)

    Google Scholar 

  12. Rupp, et al.: Modeling learning progressions in epistemic games with epistemic network analysis: principles for data analysis and generation. In: LeaPS 2009 Proceedings (2009)

    Google Scholar 

  13. Arastoopour Irgens, G., Shaffer, D.W.: Measuring social identity development in epistemic games. In: CSCL 2013 Proceedings, pp. 42–48 (2013)

    Google Scholar 

  14. Frey, K.S., Kwak-Tanquay, S., Nguyen, H.A., Onyewuenyi, A.C., Strong, Z.H., Waller, I.A.: Adolescents’ views of third-party vengeful and reparative actions. In: ICQE 2019 Proceedings, pp. 89–105 (2019)

    Google Scholar 

  15. D’Angelo, A.L.D., Ruis, A.R., Collier, W., Shaffer, D.W., Pugh, C.M.: Evaluating how residents talk and what it means for surgical performance in the simulation lab. Am. J. Surg. 220(1), 37–43 (2020)

    Article  Google Scholar 

  16. Eagan, B.R., Rogers, B., Pozen, R., Marquart, C., Shaffer, D.W.: rhoR: Rho for inter-rater reliability (Version 1.1.0) (2016)

    Google Scholar 

  17. Shaffer, D.W., et al.: The nCoder: A Technique for Improving the Utility of Inter-Rater Reliability Statistics. Epistemic Games Group Working Paper 2015-01 (2015)

    Google Scholar 

  18. Cai, Z., Siebert-Evenstone, A., Eagan, B., Shaffer, D.W., Hu, X., Graesser, A.C.: nCoder+: a semantic tool for improving recall of nCoder coding. In: ICQE 2019 Proceedings, pp. 41–54 (2019)

    Google Scholar 

  19. Zörgő, S., Peters, G.J.Y.: Epistemic network analysis for semi-structured interviews and other continuous narratives: Challenges and insights. In: ICQE 2019 Proceedings, pp. 267–277 (2019)

    Google Scholar 

  20. Buckingham Shum, S., Echeverria, V., Martinez-Maldonado, R.: The Multimodal Matrix as a quantitative ethnography methodology. In: ICQE 2019 Proceedings, pp. 26–40 (2019)

    Google Scholar 

  21. Major, L., Warwick, P., Rasmussen, I., Ludvigsen, S., Cook, V.: Classroom dialogue and digital technologies: a scoping review. Educ. Inf. Technol. 23(5), 1995–2028 (2018)

    Article  Google Scholar 

  22. Espino, D., Lee, S., Eagan, B., Hamilton, E.: An initial look at the developing culture of online global meet-ups in establishing a collaborative, STEM media-making community. In: CSCL 2019 Proceedings, pp. 608–611 (2019)

    Google Scholar 

  23. Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.): Advances in Quantitative Ethnography: First International Conference, ICQE 2019, Madison, WI, USA, 20–22 October 2019, Proceedings, vol. 1112. Springer Nature (2019)

    Google Scholar 

  24. Sullivan, S., et al.: Using epistemic network analysis to identify targets for educational interventions in trauma team communication. Surgery 163(4), 938–943 (2018)

    Article  Google Scholar 

  25. Wooldridge, Abigail R., Haefli, R.: Using epistemic network analysis to explore outcomes of care transitions. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 245–256. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_21

    Chapter  Google Scholar 

  26. Shaffer, D.W.: Big data for thick description of deep learning. In: Millis, K., Long, D., Magliano, J., Wiemer, K. (eds.) Deep Comprehension, pp. 265–277. Routledge, New York (2018)

    Chapter  Google Scholar 

  27. Arastoopour Irgens, G., et al.: Modeling and measuring high school students’ computational thinking practices in science. J. Sci. Educ. Tech. 29(1), 137–161 (2020)

    Article  Google Scholar 

  28. Ferreira, R., Kovanović, V., Gašević, D., Rolim, V.: Towards combined network and text analytics of student discourse in online discussions. In: AIED 2018 Proceedings, pp. 111–126 (2018)

    Google Scholar 

  29. Lund, K., Quignard, M., Shaffer, D.W.: Gaining insight by transforming between temporal representations of human interaction. J. Learn. Anal. 4(3), 102–122 (2017)

    Google Scholar 

  30. Nash, P., Shaffer, D.W.: Mentor modeling: the internalization of modeled professional thinking in an epistemic game. J. Comput. Assist. Learn. 27(2), 173–189 (2011)

    Article  Google Scholar 

  31. Bauer, E., et al.: Using ENA to analyze pre-service teachers’ diagnostic argumentations: a conceptual framework and initial applications. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 14–25. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_2

    Chapter  Google Scholar 

  32. Hu, S., Torphy, K.T., Chen, Z., Eagan, B.: How do US teachers align instructional resources to the common core state standards: a case of Pinterest. In: SMSociety 2018 Proceedings, pp. 315–319 (2018)

    Google Scholar 

  33. Pratt, S.M.: A mixed methods approach to exploring the relationship between beginning readers’ dialog about their thinking and ability to self-correct oral reading. Read. Psychol. 41(1), 1–43 (2020)

    Article  MathSciNet  Google Scholar 

  34. Karumbaiah, S., Baker, R.S., Barany, A., Shute, V.: Using epistemic networks with automated codes to understand why players quit levels in a learning game. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 106–116. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_9

    Chapter  Google Scholar 

  35. Peters-Burton, E.E.: Outcomes of a self-regulated learning curriculum model. Sci. Educ. 24(7–8), 855–885 (2015)

    Article  Google Scholar 

  36. Swiecki, Z., Ruis, A.R., Farrell, C., Shaffer, D.W.: Assessing individual contributions to collaborative problem solving: a network analysis approach. Comput. Hum. Behav. 104 105876 (2020)

    Google Scholar 

  37. Zhang, S., Liu, Q., Cai, Z.: Exploring primary school teachers’ technological pedagogical content knowledge (TPACK) in online collaborative discourse: an epistemic network analysis. Br. J. Educ. Technol. 50(6), 3437–3455 (2019)

    Google Scholar 

  38. Siebert-Evenstone, A., Arastoopour Irgens, G., Collier, W., Swiecki, Z., Ruis, A.R., Shaffer, D.W.: In search of conversational grain size: Modelling semantic structure using moving stanza windows. J. Learn. Anal. 4(3), 123–139 (2017)

    Google Scholar 

  39. Melzner, N., Greisel, M., Dresel, M., Kollar, I.: Using Process Mining (PM) and Epistemic Network Analysis (ENA) for Comparing Processes of Collaborative Problem Regulation. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 154–164. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_13

    Chapter  Google Scholar 

  40. Gašević, D., Joksimović, S., Eagan, B.R., Shaffer, D.W.: SENS: network analytics to combine social and cognitive perspectives of collaborative learning. Comput. Hum. Behav. 92, 562–577 (2019)

    Google Scholar 

  41. Swiecki, Z., Shaffer, D.W.: iSENS: an integrated approach to combining epistemic and social network analyses. In: LAK 2010 Proceedings, pp. 305–313 (2020)

    Google Scholar 

  42. Bagley, E., Shaffer, D.W.: Epistemic mentoring in virtual and face-to-face environments. In: ICLS 2012 Proceedings, pp. 256–260 (2012)

    Google Scholar 

  43. Svarovsky, G.N.: Exploring complex engineering learning over time with epistemic network analysis. J-PEER 1(2), 4 (2011)

    Google Scholar 

  44. Rolim, V., Ferreira, R., Lins, R.D., Gašević, D.: A network-based analytic approach to uncovering the relationship between social and cognitive presences in communities of inquiry. Internet High. Educ. 42, 53–65 (2019)

    Google Scholar 

  45. Ognjanović, I., Gašević, D., Dawson, S.: Using institutional data to predict student course selections in higher education. Internet High. Educ. 29, 49–62 (2016)

    Article  Google Scholar 

  46. Shaffer, D.W.: QE-COVID data challenge. Why QE? [White paper]. (2020). https://sites.google.com/wisc.edu/qe-covid-data-challenge/why-qe

  47. Wu, B., Hu, Y., Ruis, A., Wang, M.: Analysing computational thinking in collaborative programming: a quantitative ethnography approach. J. Comput. Assist. Learn. 35(3), 421–434 (2019)

    Google Scholar 

  48. Swiecki, Z., Shaffer, D.W.: Toward a taxonomy of team performance visualization tools. In: ICLS 2018 Proceedings, pp. 144–151 (2018)

    Google Scholar 

  49. Swiecki, Z., Marquart, C., Sachar, A., Hinojosa, C., Ruis, A.R., Shaffer, D.W.: Designing an Interface for sharing quantitative ethnographic research data. In: ICQE 2019 Proceedings, pp. 334–341 (2019)

    Google Scholar 

  50. Herder, T., et al.: Supporting teachers’ intervention in students’ virtual collaboration using a network based model. In: LAK 2008 Proceedings, pp. 21–25 (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Education, University of Oslo, Oslo, Norway

    Rogers Kaliisa

  2. Centre for the Science of Learning and Technology, University of Bergen, Bergen, Norway

    Kamila Misiejuk

  3. Department of Education and Human Development, Clemson University, Clemson, South Carolina, USA

    Golnaz Arastoopour Irgens

  4. Department of Computer Science, University of Copenhagen, Copenhagen, Denmark

    Morten Misfeldt

Authors
  1. Rogers Kaliisa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kamila Misiejuk
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Golnaz Arastoopour Irgens
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Morten Misfeldt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rogers Kaliisa .

Editor information

Editors and Affiliations

  1. University of Wisconsin–Madison, Madison, WI, USA

    Andrew R. Ruis

  2. Pepperdine University, Malibu, CA, USA

    Seung B. Lee

Appendix I

Appendix I

List of studies included in the scoping review but not cited in the main text (n = 36)

Authors

Authors

Andrist, Collier, Gleicher, Mutlu, Shaffer (2015)

Orrill, Shaffer (2012)

Andrist, Ruis, Shaffer (2018)

Peters-Burton (2015)

Arastoopour Irgens, Shaffer, Swiecki, Ruis, Chesler (2015)

Peters-Burton, Parrish, Mulvey (2019)

Arastoopour Irgens, Shaffer (2015)

Phillips, Kovanović, Mitchell, Gašević (2019)

Barany, Foster (2019)

Ruis, Rosser, Nathwani, Beams, Jung, Pugh (2019)

Bressler, Bodzin, Eagan, Tabatabai (2019)

Ruis, Rosser, Quandt-Walle, Nathwani, Shaffer, Pugh (2018)

Brown, Nagar, Orrill, Weiland, Burke (2016)

Ruis, Siebert-Evenstone, Pozen, Eagan, Shaffer (2019)

Cai, Eagan, Dowell, Pennebaker, Shaffer, Graesser (2017)

Shah, Foster, Talafian, Barany (2019)

Chesler, Ruis, Collier, Swiecki, Arastoopour Irgens, Shaffer (2015)

Siebert-Evenstone, Shaffer (2019)

Csanadi, Eagan, Shaffer, Kollar, Fischer (2017)

Sinclair, Ferreira, Gašević, Lucas, Lopez (2019)

Espino, Lee, Van Tress, Baker, Hamilton (2020)

Sung, Cao, Ruis, Shaffer (2019)

Fisher, Hirshfield, Siebert-Evenstone, Arastoopour Irgens, Koretsky (2016)

Talafian, Shah, Barany, Foster (2019)

Fougt, Siebert-Evenstone, Eagan, Tabatabai, Misfeldt (2018)

Wakimoto, Sasaki, Hirayama, Mochizuki, Eagan, Yuki, Kato (2019)

Foster, Shah, Barany, Talafian (2019)

Whitelock-Wainwright, Tsai, Lyons, Kaliff, Bryant, Ryan, Gašević (2020)

Knight, Arastoopour Irgens, Shaffer, Buckingham Shum, Littleton (2014)

Wooldridge, Carayon, Shaffer, Eagan (2018)

Lim, Dawson, Joksimović, Gašević (2019)

Yi, Lu, Leng (2019)

Nachtigall, Sung (2019)

Yue, Hu, Xiao (2019)

Oner (2020)

 

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kaliisa, R., Misiejuk, K., Irgens, G.A., Misfeldt, M. (2021). Scoping the Emerging Field of Quantitative Ethnography: Opportunities, Challenges and Future Directions. In: Ruis, A.R., Lee, S.B. (eds) Advances in Quantitative Ethnography. ICQE 2021. Communications in Computer and Information Science, vol 1312. Springer, Cham. https://doi.org/10.1007/978-3-030-67788-6_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-67788-6_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-67787-9

  • Online ISBN: 978-3-030-67788-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation