Skip to main content
SpringerLink
Log in

Is QE Just ENA?

  • Conference paper
  • First Online:
Advances in Quantitative Ethnography (ICQE 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1785))

Included in the following conference series:

Abstract

In the emerging field of quantitative ethnography (QE), epistemic network analysis (ENA) has featured prominently, to the point where multiple scholars in the QE community have asked some variation on the question: Is QE just ENA? This paper is an attempt to address this question systematically. We review arguments that QE should be considered a background and justification for using ENA as well as arguments that ENA should be considered merely one approach to implementing QE ideas. We conclude that ENA is used in QE, but not exclusively; and that QE uses ENA, but not exclusively; but that the answer to this question is less important than the reflexive thinking about methodology that has been a key focus of the QE community. Our hope is that, rather than a definitive answer to this question, this paper provides some ways to think about the relationships between theory, methods, and analytic techniques as the QE community continues to grow.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Bernard also argued for the qualitative examination of quantitative data, again sounding strikingly similar to more recent arguments. He claims that qualitative analysis of quantitative data is “the search for, and the presentation of, meaning in the results of quantitative data processing.” He argues that without such work, quantitative studies are “puerile.”

  2. 2.

    Technically, Shaffer and Serlin argued that such a statistical analysis would generalize to a hypothetical sample taken from “all the things that we might have recorded about these students in the given context from a particular perspective.” Thus, statistical significance meant that the analysis was saturated in the sense that the results generalize to other possible data that might have been collected or examined under the original circumstances.

  3. 3.

    Shaffer and Ruis present three forms of fairness (theory, community, and data) together and then discusses subgroup fairness separately. However, we believe it is conceptually clearer to think of four co-equal criteria for fairness. We also note that these criteria do not explicitly reference ethical issues in theory (such as plagiarism), interactions with a community (such as informed consent), data (such as p-hacking), and subgroups (such as unconscious bias). However, we take these as shared assumptions about acceptable research practices.

  4. 4.

    The use of capitalization denotes the difference between events in the world ([l]ower case) and claims about a culture ([U]pper case). This terminology comes from Gee [18] and Shaffer [45].

  5. 5.

    The Ship of Theseus is a paradox raised by Heraclitus of Ephesus (and others, including Thomas Hobbes) asking whether an object that had all of its parts replaced was still the same object.

References

  1. Andrade, A., Maddox, B., Edwards, D., Chopade, P., Khan, S.: Quantitative multimodal interaction analysis for the assessment of problem-solving skills in a collaborative online game. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 281–290. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_24

    Chapter  Google Scholar 

  2. Andrist, S., Collier, W., Gleicher, M., Mutlu, B., Shaffer, D.W.: Look together: Analyzing gaze coordination with epistemic network analysis. Front. Psychol. 6 (2015)

    Google Scholar 

  3. Bagley, E.: Epistemic Mentoring in Virtual and Face-to-Face Environments. Ph.D. thesis, University of Wisconsin-Madison (2012)

    Google Scholar 

  4. Bagley, E.A., Shaffer, D.W.: Promoting civic thinking through epistemic game play. In: Ferdig, R. (ed.) Discoveries in Gaming and Computer-Mediated Simulations: New Interdisciplinary Applications, pp. 111–127. IGI Global (2011)

    Google Scholar 

  5. Bakharia, A.: On the equivalence of inductive content analysis and topic modeling. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 291–298. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_25

    Chapter  Google Scholar 

  6. Bernard, H.R.: Qualitative data, quantitative analysis. CAM February, pp. 9–11 (1996)

    Google Scholar 

  7. Borden, F., Collier, W., Marquart, C., Arastoopour, G., Srinivasan, A., Shaffer, D.W.: Epistemic Network Analysis Webkit (2014)

    Google Scholar 

  8. Bowman, D., et al.: The mathematical foundations of epistemic network analysis. In: Ruis, A.R., Lee, S.B. (eds.) ICQE 2021. CCIS, vol. 1312, pp. 91–105. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67788-6_7

    Chapter  Google Scholar 

  9. Brohinsky, J., Marquart, C., Wang, J., Ruis, A.R., Shaffer, D.W.: Trajectories in epistemic network analysis. In: Ruis, A.R., Lee, S.B. (eds.) ICQE 2021. CCIS, vol. 1312, pp. 106–121. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67788-6_8

    Chapter  Google Scholar 

  10. Buckingham Shum, S., Echeverria, V., Martinez-Maldonado, R.: The multimodal matrix as a quantitative ethnography methodology. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 26–40. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_3

    Chapter  Google Scholar 

  11. Chi, M.T.H.: Quantifying qualitative analyses of verbal data: A practical guide 6(3), 271–315 (1997)

    Google Scholar 

  12. Clements, F.E.: Quantitative method in ethnography

    Google Scholar 

  13. Cohen, J.: Kappa: Coefficient of concordance. Educ. Psychol. Measur. 20(37), 37–46 (1960)

    Article  Google Scholar 

  14. Crowley, K., Jacobs, M.: Building islands of expertise in everyday family activity. In: Leinhardt, G., Crowley, K., Knutson, K. (eds.) Learning Conversations in Museums, pp. 333–356. Lawrence Erlbaum Associates (2002)

    Google Scholar 

  15. Darden, L.: Discoveries and the emergence of new fields in science. In: PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association, vol. 1978(1), pp. 149–160 (1978)

    Google Scholar 

  16. Elmoazen, R., Saqr, M., Tedre, M., Hirsto, L.: A systematic literature review of empirical research on epistemic network analysis in education. IEEE Access 10, 17330–17348 (2022)

    Article  Google Scholar 

  17. Fuchs, K.H.: Minimally invasive surgery. Endoscopy 34, 154–159 (2002)

    Article  Google Scholar 

  18. Gee, J.P.: An Introduction to Discourse Analysis: Theory and Method, 4th ed. Routledge (2014)

    Google Scholar 

  19. Geertz, C.: Thick description: Toward an interpretive theory of culture. In: The Interpretation of Cultures: Selected Essays, pp. 3–30. Basic Books (1973)

    Google Scholar 

  20. Glesne, C.: Becoming Qualitative Researchers: An Introduction. Pearson (2015)

    Google Scholar 

  21. Goffman, E.: Frame Analysis: An Essay on the Organization of Experience. Harvard University Press (1974)

    Google Scholar 

  22. Goodman, N.: Ways of Worldmaking. Hackett (1978)

    Google Scholar 

  23. Hascoët, M., Dragicevic, P.: Visual Comparison of Document Collections Using Multi-Layered Graphs (2011)

    Google Scholar 

  24. Hutt, S., et al.: Quick Red Fox: An App Supporting a New Paradigm in Qualitative Research on AIED for STEM (2021)

    Google Scholar 

  25. Keir, S.S.: Middle Class Black Families in Austin, Texas: An Exploratory Analysis of Husbands and Wives. Ph.D. thesis, University of Texas at Austin (1987)

    Google Scholar 

  26. Kleinman, A.: The cultural meanings and social uses of illness: A role for medical anthropology and clinically oriented social science in the development of primary care theory and research 16(3), 539–545 (1983)

    Google Scholar 

  27. Knowles, M., Shaffer, D.W.: Hierarchical epistemic network analysis. In: Ruis, A.R., Lee, S.B. (eds.) Second International Conference on Quantitative Ethnography: Conference Proceedings Supplement, pp. 31–34. ISQE (2021)

    Google Scholar 

  28. Lave, J., Wenger, E.: Situated learning: Legitimate peripheral participation. Cambridge University Press (1991)

    Google Scholar 

  29. Li, Qi.: Embodying Data: Chinese Aesthetics, Interactive Visualization and Gaming Technologies. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-5069-0

  30. Marquart, C., Swiecki, Z., Collier, W., Eagan, B., Woodward, R., Shaffer, D.W.: rENA: R statistical package for Epistemic Network Analysis (2018)

    Google Scholar 

  31. Marquart, C., Swiecki, Z., Hinojosa, C., Collier, W., Shaffer, D.W.: Epistemic Network Analysis Webkit (2018)

    Google Scholar 

  32. Marquart, C.L., Swiecki, Z., Eagan, B.R., Williamson Shaffer, D.: ncodeR: Techniques for automated classifiers (2018)

    Google Scholar 

  33. Mehl, M.R.: The Sounds of Social Life: Exploring Students’ Daily Social Environments and Natural Conversations. Ph.D. thesis, University of Texas at Austin (2004)

    Google Scholar 

  34. Melzner, N., Greisel, M., Dresel, M., Kollar, I.: Using process mining (PM) and epistemic network analysis (ENA) for comparing processes of collaborative problem Regulation. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 154–164. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_13

    Chapter  Google Scholar 

  35. Orrill, C., Shaffer, D.W.: Exploring connectedness: Applying ENA to teacher knowledge. In: Van Aalst, J., Thompson, K., Jacobson, M.J., Reimann, P. (eds.) The Future of Learning: Proceedings of the 10th International Conference of the Learning Sciences (ICLS 2012), vol. 1, pp. 175–179 (2012)

    Google Scholar 

  36. Orrill, C., Shaffer, D.W., Burke, J.: Exploring coherence in teacher knowledge using epistemic network analysis. In: Paper presented at the Annual Meeting of the American Educational Research Association (2013)

    Google Scholar 

  37. Pelillo, M.: Replicator equations, maximal cliques, and graph isomorphism. In: Kearns, M., Solla, S., Cohn, D. (eds.) Advances in Neural Information Processing Systems. MIT Press (1998)

    Google Scholar 

  38. Peters, G.J., Zörgő, S.: Introduction to the reproducible open coding kit (ROCK). PsyArXiv Preprints (2019)

    Google Scholar 

  39. Porter, C., et al.: A systematic review of quantitative ethnography methods. In: Ruis, A., Lee, S. (eds.) Second International Conference on Quantitative Ethnography: Conference Proceedings Supplement, pp. 35–38. International Society for Quantitative Ethnography (2021)

    Google Scholar 

  40. Shaffer, D.W.: Epistemic frames for epistemic games. Comput. Educ. 46(3), 223–234 (2006)

    Article  Google Scholar 

  41. Shaffer, D.W.: Quantitative ethnography: Measuring complex thinking using grounded data mining. Paper presented at Colorado State University (2016)

    Google Scholar 

  42. Shaffer, D.W.: Quantitative Ethnography. Cathcart Press (2017)

    Google Scholar 

  43. Shaffer, D.W., et al.: The nCoder: A technique for improving the utility of inter-rater reliability statistics (2015)

    Google Scholar 

  44. Shaffer, D.W., et al.: Epistemic network analysis: A prototype for 21st century assessment of learning, vol. 1, pp. 33–53 (2009)

    Google Scholar 

  45. Shaffer, D.W., Ruis, A.R.: How we code. In: Ruis, A.R., Lee, S.B. (eds.) ICQE 2021. CCIS, vol. 1312, pp. 62–77. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67788-6_5

    Chapter  Google Scholar 

  46. Shaffer, D.W., Serlin, R.: What good are statistics that don’t generalize?. vol. 33, pp. 14–25 (2004)

    Google Scholar 

  47. Siebert-Evenstone, A.L., Arastoopour, G., Collier, W., Swiecki, Z., Ruis, A.R., Shaffer, D.W.: In search of conversational grain size: Modeling semantic structure using moving stanza windows. In: Looi, C.K., Polman, J., Cress, U., Reimann, P. (eds.) Transforming Learning, Empowering Learners: The International Conference of the Learning Sciences (ICLS) 2016, vol. I, pp. 631–638 (2016)

    Google Scholar 

  48. Tan, S.C., Wang, X., Li, L.: The development trajectory of shared epistemic agency in online collaborative learning: A study combing network analysis and sequential analysis. J. Educ. Comput. Res. (2021) (in press)

    Google Scholar 

  49. Zörgő, S., Swiecki, Z., Ruis, A.R.: Exploring the effects of segmentation on semi-structured interview data with epistemic network analysis. In: Ruis, A.R., Lee, S.B. (eds.) ICQE 2021. CCIS, vol. 1312, pp. 78–90. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67788-6_6

    Chapter  Google Scholar 

Download references

Acknowledgements

This work was funded in part by the National Science Foundation (DRL-1713110, DRL-2100320, DRL-2201723), the Wisconsin Alumni Research Foundation, and the Office of the Vice Chancellor for Research and Graduate Education at the University of Wisconsin-Madison. The opinions, findings, and conclusions do not reflect the views of the funding agencies, cooperating institutions, or other individuals.

Author information

Authors and Affiliations

  1. University of Wisconsin–Madison, Madison, WI, 53711, USA

    David Williamson Shaffer & Andrew R. Ruis

Authors
  1. David Williamson Shaffer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrew R. Ruis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Williamson Shaffer .

Editor information

Editors and Affiliations

  1. University of Oslo, Oslo, Norway

    Crina Damşa

  2. Drexel University School of Education, Philadelphia, PA, USA

    Amanda Barany

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Shaffer, D.W., Ruis, A.R. (2023). Is QE Just ENA?. In: Damşa, C., Barany, A. (eds) Advances in Quantitative Ethnography. ICQE 2022. Communications in Computer and Information Science, vol 1785. Springer, Cham. https://doi.org/10.1007/978-3-031-31726-2_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-31726-2_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-31725-5

  • Online ISBN: 978-3-031-31726-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation