Skip to main content
SpringerLink
Log in

Establishing Trustworthiness Through Algorithmic Approaches to Qualitative Research

  • Conference paper
  • First Online:
Advances in Quantitative Ethnography (ICQE 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1312))

Included in the following conference series:

Abstract

Establishing trustworthiness is a fundamental component of qualitative research. In the following paper, we document how combining natural language processing (NLP), with human analysis by researchers, can help analysts develop insights from qualitative data and establish trustworthiness for the analysis process. We document the affordances of such an approach to strengthen three specific aspects of trustworthiness in qualitative research: credibility, dependability, and confirmability. We illustrate this workflow and shed light on its implications for trustworthiness from our own, recent research study of educators’ experiences with the 2020 COVID-19 pandemic; a context that compelled our research team to analyze our data efficiently to best aid the community, but also establish rigor and trustworthiness of our process.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Change history

  • 22 January 2021

    In the originally published version of the chapter 4, the name of the author was spelled incorrectly. The author’s name has been changed as Ashlee Belgrave.

References

  1. Anfara Jr, V.A., Brown, K.M., Mangione, T.L.: Qualitative analysis on stage: Making the research process more public. Educ. Res. 31(7), 28–38 (2002)

    Google Scholar 

  2. Bakharia, A.: On the equivalence of inductive content analysis and topic modeling. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 291–298. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_25

    Chapter  Google Scholar 

  3. Bakharia, A., Corrin, L.: Using recent advances in contextual word embeddings to improve the quantitative ethnography workflow. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 299–306. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_26

    Chapter  Google Scholar 

  4. Boyd, R.L.: Psychological text analysis in the digital humanities. In: Hai-Jew, S. (ed.) Analytics in Digital Humanities. Multimedia Systems and Applications, Springer, Cham (2019). https://doi.org/10.1007/978-3-319-54499-1_7

    Chapter  Google Scholar 

  5. Bullough, R.V., Jr., Pinnegar, S.: Guidelines for quality in autobiographical forms of self-study research. Educ. Res. 30(3), 13–21 (2001)

    Article  Google Scholar 

  6. Cai, Z., Siebert-Evenstone, A., Eagan, B., Shaffer, D.W., Hu, X., Graesser, A.C.: nCoder+: a semantic tool for improving recall of nCoder coding. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 41–54. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_4

    Chapter  Google Scholar 

  7. Charmaz, K., Denzin, N.K., Lincoln, Y.S.: Handbook of qualitative research. Sage Publications, Thousand Oaks (2000)

    Google Scholar 

  8. Chen, N.C., Drouhard, M., Kocielnik, R., Suh, J., Aragon, C.R.: Using machine learning to support qualitative coding in social science: shifting the focus to ambiguity. ACM Trans. Interact. Intell. Syst. (TiiS) 8(2), 1–20 (2018)

    Article  Google Scholar 

  9. DiMaggio, P., Nag, M., Blei, D.: Exploiting affinities between topic modeling and the sociological perspective on culture: application to newspaper coverage of US government arts funding. Poetics 41(6), 570–606 (2013)

    Article  Google Scholar 

  10. Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. KDD 96(34), 226–231 (1996)

    Google Scholar 

  11. Fekete, J. D., Dufournaud, N.: Compus: visualization and analysis of structured documents for understanding social life in the 16th century. In: Proceedings of the Fifth ACM Conference on Digital Libraries, pp. 47–55 (2000)

    Google Scholar 

  12. Goldberg, Y.: Neural network methods for natural language processing. Synth. Lect. Hum. Lang. Technol. 10(1), 1–309 (2017)

    Article  Google Scholar 

  13. Harris, Z.S.: Distributional structure. Word 10(2–3), 146–162 (1954)

    Article  Google Scholar 

  14. Li, H., Schnieders, J.Z.Y., Bobek, B.L.: Theme analyses for open-ended survey responses in education research on summer melt phenomenon. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 128–140. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_11

    Chapter  Google Scholar 

  15. Lincoln, Y.S., Guba, E.G.: But is it rigorous? Trustworthiness and authenticity in naturalistic evaluation. New Direct. Program Eval. 1986(30), pp. 73–84 (1986)

    Google Scholar 

  16. Marquart, C.L., Swiecki, Z., Eagan, B., Shaffer, D.W.: ncodeR (Version 0.1.2) (2018). https://cran.r-project.org/web/packages/ncodeR/ncodeR.pdf

  17. Marshall, C., Rossman, G.B.: Designing Qualitative Research. Sage Publications, Thousand Oaks (2014)

    Google Scholar 

  18. Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., Dean, J. Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)

    Google Scholar 

  19. Muralidharan, A., Hearst, M. A.: Supporting exploratory text analysis in literature study. Liter. Linguist. Comput. 28(2), 283–295 (2012)

    Google Scholar 

  20. Ophir, Y., Walter, D., Marchant, E.R.: A collaborative way of knowing: bridging computational communication research and grounded theory ethnography. J. Commun. 70(3), 447–472 (2020)

    Article  Google Scholar 

  21. Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)

    Google Scholar 

  22. Rahmah, N., Sitanggang, I.S.: Determination of optimal epsilon (eps) value on DBscan algorithm to clustering data on peatland hotspots in sumatra. In IOP Conference Series: Earth and Environmental Science, vol. 31, no. 1, p. 012012. IOP Publishing (2016)

    Google Scholar 

  23. Robinson, R.L., Navea, R., Ickes, W.: Predicting final course performance from students’ written self-introductions: a LIWC analysis. J. Lang. Soc. Psychol. 32(4), 469–479 (2013)

    Article  Google Scholar 

  24. Shaffer, D. W. Quantitative ethnography (2017). Lulu.com

  25. Tausczik, Y.R., Pennebaker, J.W.: The psychological meaning of words: LIWC and computerized text analysis methods. J. Lang. Soc. Psychol. 29(1), 24–54 (2010)

    Article  Google Scholar 

  26. Verma, M., Srivastava, M., Chack, N., Diswar, A.K., Gupta, N.: A comparative study of various clustering algorithms in data mining. Int. J. Eng. Res. Appl. (IJERA) 2(3), 1379–1384 (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of California-Irvine, Irvine, USA

    Ha Nguyen, June Ahn, Ashlee Belgrave, Jiwon Lee, Lora Cawelti, Ha Eun Kim, Yenda Prado, Rossella Santagata & Adriana Villavicencio

Authors
  1. Ha Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. June Ahn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ashlee Belgrave
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiwon Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lora Cawelti
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ha Eun Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yenda Prado
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Rossella Santagata
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Adriana Villavicencio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ha Nguyen .

Editor information

Editors and Affiliations

  1. University of Wisconsin–Madison, Madison, WI, USA

    Andrew R. Ruis

  2. Pepperdine University, Malibu, CA, USA

    Seung B. Lee

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nguyen, H. et al. (2021). Establishing Trustworthiness Through Algorithmic Approaches to Qualitative Research. In: Ruis, A.R., Lee, S.B. (eds) Advances in Quantitative Ethnography. ICQE 2021. Communications in Computer and Information Science, vol 1312. Springer, Cham. https://doi.org/10.1007/978-3-030-67788-6_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-67788-6_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-67787-9

  • Online ISBN: 978-3-030-67788-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation