Assessing Quality and Naturally Occurring Data

  • Nikki Kiyimba
  • Jessica Nina Lester
  • Michelle O’Reilly


Ensuring quality in qualitative research has been subject to considerable debate across time, and while the qualitative community has moved towards some consensus, some tension remains. The heterogeneity of the different methodological approaches has created some difficulties in creating universal criteria against what qualitative health research can be judged. However, there are some common discourses in the field, and some agreed markers (that may be applied in slightly different ways). In the chapter, we consider these broad quality indicators and discuss how they apply to research that uses naturally occurring data.


Quality Qualitative research Universal criteria Universality Heterogeneity 


  1. Barbour, R. (2001). Checklists for improving rigour in qualitative research: A case of the tail wagging the dog? British Medical Journal, 322, 1115–1117.CrossRefGoogle Scholar
  2. Barbour, R., & Barbour, M. (2003). Evaluating and synthesizing qualitative research: Then need to develop a distinctive approach. Journal of Evaluation in Clinical Practice, 9(2), 179–186.CrossRefGoogle Scholar
  3. Caelli, K., Ray, L., & Mill, J. (2003). ‘Clear as mud’: Toward greater clarity in generic qualitative research. International Journal of Qualitative Methods, 2(2), 1–13. Retrieved January 1, 2012 from, Scholar
  4. Charmaz, K. (2006). Constructing grounded theory: Practical guide through qualitative analysis. London: Sage.Google Scholar
  5. Collingridge, D., & Gantt, E. (2008). The quality of qualitative research. American Journal of Medical Quality, 23(5), 389–395.CrossRefGoogle Scholar
  6. Daymon, C., & Holloway, I. (2010). Qualitative research methods in public relations and marketing communications. Hove: Routledge.CrossRefGoogle Scholar
  7. Demuth, C. (2018). Generalization from single cases and the concept of double dialogicality. Integrative Psychological & Behavioral Science, 52, 77–93.CrossRefGoogle Scholar
  8. Devers, K. (1999). How will we know “good” qualitative research when we see it? Beginning the dialogue in health services research. Health Services Research, 34(5), 1153–1188.PubMedPubMedCentralGoogle Scholar
  9. Finlay, L. (2002). “Outing” the researcher: The provenance, process, and practice of reflexivity. Qualitative Health Research, 12, 531–545.CrossRefGoogle Scholar
  10. Flick, U. (2014). An introduction to qualitative research (5th ed.). London: Sage.Google Scholar
  11. Francis, J., Johnston, M., Robertson, C., Glidewell, L., Entwistle, V., Eccles, M., & Grimshaw, J. (2010). What is adequate sample size? Operationalising data saturation for theory-based interview studies. Psychology and Health, 25(10), 1229–1245.CrossRefGoogle Scholar
  12. Freeman, M., de Marrais, K., Preissle, J., Roulston, K., & St. Pierre, E. (2007). Standards of evidence in qualitative research: An incitement to discourse. Educational Researcher, 36(1), 25–32.CrossRefGoogle Scholar
  13. Garfinkel, H. (1967). Studies in ethnomethodology. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
  14. Gergen, K., & Gergen, M. (1991). From theory to reflexivity in research practice. In F. Steier (Ed.), Method and reflexivity: Knowing as systemic social construction (pp. 76–95). London: Sage.Google Scholar
  15. Gregory, D., Russell, C., & Phillips, L. (1997). Beyond textual perfection: Transcribers as vulnerable persons. Qualitative Health Research, 7(2), 294–300.CrossRefGoogle Scholar
  16. Guba, E. G., & Lincoln, Y. S. (1981). Effective evaluation: Improving the usefulness of evaluation results through responsive and naturalistic approaches. San Fransisco: Jossey-Bass.Google Scholar
  17. Guba, E., & Lincoln, Y. (1989). Fourth generation evaluation. Newbury Park, CA: Sage.Google Scholar
  18. Guba, E., & Lincoln, Y. (2005). Paradigmatic controversies, contradictions, and emerging influences. In N. Denzin & Y. Lincoln (Eds.), The SAGE handbook of qualitative research (3rd ed., pp. 191–215). Thousand Oaks, CA: Sage.Google Scholar
  19. Hammersley, M. (2007). The issue of quality in qualitative research. International Journal of Research and Method in Education, 30(3), 287–305.CrossRefGoogle Scholar
  20. Hancock, M., Amankwaa, L., Revell, M., & Mueller, D. (2016). Focus group data saturation: A new approach to data analysis. The Qualitative Report, 21(11), 2121–2130.Google Scholar
  21. Harré, R. (2012). Positioning theory: moral dimensions of social-cultural psychology. In J. Valsiner (Ed.), The Oxford Handbook of Culture and Psychology (pp. 191–206). New York: Oxford University.Google Scholar
  22. Holloway, I., & Biley, F. (2011). Being a qualitative researcher. Qualitative Health Research, 21(7), 968–975.CrossRefGoogle Scholar
  23. Kiyimba, N., & O’Reilly, M. (2016a). An exploration of the possibility for secondary traumatic stress amongst transcriptionists: A grounded theory approach. Qualitative Research in Psychology, 13(1), 92–108.CrossRefGoogle Scholar
  24. Kiyimba, N., & O’Reilly, M. (2016b). The risk of secondary traumatic stress in the qualitative transcription process: A research note. Qualitative Research, 16(4), 468–476.CrossRefGoogle Scholar
  25. Kvale, S. (1996). Interviews: An introduction to qualitative research interviewing. Thousand Oaks, CA: Sage.Google Scholar
  26. Lewis, J., & Ritchie, J. (2003). Generalising from qualitative research. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice: A guide for social science students and researchers (pp. 263–286). London: Sage.Google Scholar
  27. Lincoln, Y., & Guba, E. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage.Google Scholar
  28. Linell, P. (2009). Rethinking language, mind, and word dialogically: Interactional and contextual theories of human sense-making. Charlotte, NC: Information Age.Google Scholar
  29. Lynch, M. (2000). Against reflexivity as an academic virtue and source of privileged knowledge. Theory, Culture and Society, 17(3), 26–54.CrossRefGoogle Scholar
  30. Macbeth, D. (2001). On ‘reflexivity’ in qualitative research: Two readings and a third. Qualitative Inquiry, 7(1), 26–54.CrossRefGoogle Scholar
  31. Maxwell, J., & Chmiel, M. (2013). Generalization in and from qualitative analysis. In U. Flick (Ed.), The SAGE handbook of qualitative data analysis (pp. 540–553). London: Sage.Google Scholar
  32. Mays, N., & Pope, C. (2000). Quality in qualitative health research. In C. Pope & N. Mays (Eds.), Qualitative research in health care (pp. 89–102). London: BMJ Books.Google Scholar
  33. Meyrick, J. (2006). What is good qualitative research? A first step towards a comprehensive approach to judging rigour/quality. Journal of Health Psychology, 11(5), 799–808.CrossRefGoogle Scholar
  34. Mitchell, J. C. (1983). Case and situational analysis. Sociological Review, 31(2), 187–211.CrossRefGoogle Scholar
  35. Morse, J., Barrett, M., Mayan, M., Olson, K., & Spiers, J. (2002). Verification strategies for establishing reliability and validity in qualitative research. International Journal for Qualitative Methods, 1(2), 1–19.CrossRefGoogle Scholar
  36. O’Reilly, M., & Kiyimba, N. (2015). Advanced qualitative research: A guide to contemporary theoretical debates. London: Sage.Google Scholar
  37. O’Reilly, M., & Parker, N. (2013). Unsatisfactory saturation: A critical exploration of the notion of saturated sample sizes in qualitative research. Qualitative Research, 13(2), 190–197.CrossRefGoogle Scholar
  38. O’Reilly, M., Parker, N., & Hutchby, I. (2011). Ongoing processes of managing consent: The empirical ethics of using video-recording in clinical practice and research. Clinical Ethics, 6, 179–185.CrossRefGoogle Scholar
  39. Patton, M. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage.Google Scholar
  40. Peräkylä, A. (2004). Reliability and validity in research based on naturally occurring social interaction. In D. Silverman (Ed.), Qualitative research: Theory, method and practice (2nd ed., pp. 283–304). London: Sage.Google Scholar
  41. Ravenek, M., & Rudman, D. (2013). Bridging conceptions of quality in moments of qualitative research. International Journal of Qualitative Methods, 12, 436–456.CrossRefGoogle Scholar
  42. Ravitch, S., & Carl, N. M. (2016). Qualitative research: Bridging the conceptual, theoretical and methodological. London: Sage.Google Scholar
  43. Salvatore, S., & Valsiner, J. (2010). Between the general and the unique. Overcoming the nomothetic versus idiographic opposition. Theory and Psychology, 20(6), 817–833.Google Scholar
  44. Sandelowski, M. (1986). The problem of rigour in qualitative research. Advances in Nursing Science, 8, 27–37.CrossRefGoogle Scholar
  45. Sandelowski, M. (1993). Rigor or rigor mortis: The problem of rigor in qualitative research revisited. Advances in Nursing Science, 16(2), 1–8.CrossRefGoogle Scholar
  46. Schön, D. (1983). The reflective practitioner. New York, NY: Basic Books.Google Scholar
  47. Seale, C. (1999). Quality in qualitative research. Qualitative Inquiry, 5(4), 465–478.CrossRefGoogle Scholar
  48. Shotter, J. (2008). Conversational realities revisited: Life, language, body and world. Chagrin Falls, OH: Taos Institute Publications.Google Scholar
  49. Silverman, D. (2010). Doing qualitative research (4th ed.). London: Sage.Google Scholar
  50. Spencer, L., Ritchie, J., Lewis, J., & Dillon, L. (2003). Quality in qualitative evaluation: A framework for assessing research evidence. London: Government Chief Social Researcher’s Office, Prime Minister’s strategy Unit. Scholar
  51. Stenbacka, C. (2001). Qualitative research requires quality concepts of its own. Management Decision, 39(7), 551–555.CrossRefGoogle Scholar
  52. Stiles, W. (1993). Quality control in qualitative research. Clinical Psychology Review, 13, 593–618.CrossRefGoogle Scholar
  53. Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19(6), 349–357.CrossRefGoogle Scholar
  54. Tracy, S. (2010). Qualitative quality: Eight “Big-Tent” criteria for excellent qualitative research. Qualitative Inquiry, 16(10), 837–851.CrossRefGoogle Scholar
  55. Valsiner, J., & Brinkmann, S. (2016). Beyond the “variables”: Developing metalanguage for psychology. In S. Klempe & R. Smith (Eds.), Centrality of history for theory construction in psychology, Annals of theoretical psychology (Vol. 14, pp. 75–90). Cham: Springer.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2019

Authors and Affiliations

  • Nikki Kiyimba
    • 1
  • Jessica Nina Lester
    • 2
  • Michelle O’Reilly
    • 3
  1. 1.Department of Social and Political ScienceUniversity of ChesterChesterUK
  2. 2.School of EducationIndiana UniversityBloomingtonUSA
  3. 3.The Greenwood Institute of Child HealthUniversity of LeicesterLeicesterUK

Personalised recommendations