Skip to main content

Preparing for analysis: a practical guide for a critical step for procedural rigor in large-scale multisite qualitative research studies

Quality & Quantity volume 52, pages 815–828 (2018)Cite this article

Abstract

Despite attention to technical aspects of conducting qualitative research, few studies describe procedures for managing pre-analysis activities in large-scale multisite qualitative projects. As part of the data collection for the Criminal Justice Drug Abuse Treatment Studies, a national multisite research cooperative with nine research centers (RCs) conducted 700 semi-structured interviews to evaluate interorganizational strategies in three protocols focused on improving addiction- and health related service delivery in criminal justice settings. Guided by the research cooperative, RCs collaborated on many aspects of the qualitative data activities (e.g., codebook development and coding activities), however, pre-analysis procedures, such as organizing and managing resources, were primarily managed at the RC level. The present study examines these various practices and describes two exemplary approaches to pre-analysis, with emphasis on lessons learned and recommendations for coordinating resources, managing data, and maintaining fidelity to the study procedures for large-scale qualitative projects.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3

References

  • Atkinson, P.: Qualitative research—unity and diversity [25 paragraphs]. In: Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, vol. 6, no. 3, Art. 26 (2005). http://nbn-resolving.de/urn:nbn:de:0114-fqs0503261. Retrieved 16 Jan 2017

  • ATLAS.ti. (Version 6): Scientific Software Development. Berlin (2011). http://atlasti.com/

  • Barry, C.A., Britten, N., Barber, N., Bradley, C., Stevenson, F.: Using reflexivity to optimize teamwork in qualitative research. Qual. Health Res. 9(1), 26–44 (1999)

    Article  Google Scholar 

  • Charmaz, K.: Grounded theory. In: Emerson, R.M. (ed.) Contemporary Field Research: Perspectives and Formulations, pp. 335–352. Waveland Press, Prospect Heights (1995)

    Google Scholar 

  • Corti, L., Day, A., Backhouse, G.: Confidentiality and informed consent: issues for consideration in the preservation of and provision of access to qualitative data archives. In: Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, ISSN 1438-5627 (2000). Retrieved 30 Oct 2015

  • Corti, L., Van den Eynden, V., Bishop, L., Woollard, M.: Managing and Sharing Research Data: A Guide to Good Practice. Sage, Beverly Hills (2014)

    Google Scholar 

  • Ducharme, J.L., Chandler, R.K., Wiley, T.R.A.: Implementing drug abuse treatment services in criminal justice settings: introduction to the CJ-DATS study protocol series. Health Justice 1, 1–6 (2013)

    Article  Google Scholar 

  • Elliott, R., Fischer, C.T., Rennie, D.L.: Evolving guidelines for publication of qualitative research studies in psychology and related fields. Br. J. Clin. Psychol. 38, 215–229 (1999)

    Article  Google Scholar 

  • Elo, S., Kyngas, H.: The qualitative content analysis process. J. Adv. Nurs. 62, 107–115 (2008)

    Article  Google Scholar 

  • Emerson, M.R.: Contemporary Field Research: Perspectives and Formulations. Waveland Press, Prospect Heights (2001)

    Google Scholar 

  • Gale, N.K., Heath, G., Cameron, E., Rashid, S., Redwood, S.: Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med. Res. Methodol. 13(1), 117 (2013). doi:10.1186/1471-2288-13-117

    Article  Google Scholar 

  • Glaser, G.B.: The constant comparative method in qualitative analysis. Soc. Probl. 12, 436–445 (1965)

    Article  Google Scholar 

  • Glaser, G.B., Strauss, A.L.: Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine, Chicago (1967)

    Google Scholar 

  • Hak, T., Bernt, T.: Coder training: theoretical training on practical socialization? Qual. Sociol. 19, 235–257 (1996)

    Article  Google Scholar 

  • Hruschka, D., Schwartz, D., St. John, D.C., Picone-Decaro, E., Jenkins, R.A., Carey, J.: Reliability in coding open-ended data: lessons learned from HIV behavioral research. Field Methods 16, 307–331 (2004)

    Article  Google Scholar 

  • Inter-university Consortium for Political and Social Research. Deductive disclosure risk. https://www.icpsr.umich.edu/icpsrweb/content/DSDR/disclosure.html. Retrieved 9 May 2016

  • Knoblauch, H., Flick, U., Maeder, C.: Qualitative methods in Europe: The variety of social research [10 paragraphs]. In:Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, vol. 6, no. 3, Art. 34 (2005). http://nbn-resolving.de/urn:nbn:de:0114-fqs0503342. Retrieved 16 Jan 2017

  • Lofland, J., Snow, D.A., Anderson, L., Lofland, L.H.: Analyzing Social Settings: A Guide to Qualitative Observation and Analysis. Wadsworth, Belmont (2006)

    Google Scholar 

  • Maxwell, A.J.: Designing a qualitative study. In: Bickman, L., Rog, D.J. (eds.) Handbook of Applied Social Science Research Methods, pp. 69–110. Sage, Thousand Oaks (1998)

    Google Scholar 

  • Monette, R.D., Sullivan, T.J., DeJong, C.R.: Applied Social Research. Harcourt Press, Orlando (2002)

    Google Scholar 

  • Proctor, K.E., Knudsen, K.J., Fedoravicius, N., Hovmand, P., Rosen, A., Perron, B.: Implementation of evidence-based practices in community behavioral health: agency director perspectives. Adm. Policy Mental Health Mental Health Serv. Res. 34, 479–488 (2007)

    Article  Google Scholar 

  • Thomas, R.D.: A general inductive approach for analyzing qualitative evaluation data. Am. J. Eval. 27, 237–246 (2006)

    Article  Google Scholar 

Download references

Acknowledgements

This study is funded under a cooperative agreement from the U.S. Department of Health and Human Services, National Institutes of Health, National Institute on Drug Abuse (NIH/NIDA), with support from the Substance Abuse and Mental Health Services Administration (SAMHSA) and the Bureau of Justice Assistance, U.S. Department of Justice. The authors gratefully acknowledge the collaborative contributions by NIDA; the Coordinating Center, AMAR International, Inc.; and the Research Centers participating in CJ-DATS. The Research Centers include: Arizona State University and Maricopa County Adult Probation (U01DA025307); University of Connecticut and the Connecticut Department of Correction (U01DA016194); University of Delaware and the Delaware Department of Corrections (U01DA016230); Friends Research Institute and the Maryland Department of Public Safety Correctional Services’ Division of Parole and Probation (U01DA025233); University of Kentucky and the Kentucky Department of Corrections (U01DA016205); National Development and Research Institutes, Inc. and the Colorado Department of Corrections (U01DA016200); University of Rhode Island, Rhode Island Hospital and the Rhode Island Department of Corrections (U01DA016191); Texas Christian University and the Illinois Department of Corrections (U01DA016190); Temple University and the Pennsylvania Department of Corrections (U01DA025284); and the University of California at Los Angeles and the Washington State Department of Corrections (U01DA016211). The authors would also like to acknowledge Dr. Tisha Wiley at NIDA for the direction and guidance on CJ-DATS. The views and opinions expressed in this report are those of the authors and do not represent the views of NIDA nor any of the sponsoring organizations, agencies, CJ-DATS partner sites, or the U.S. government.

Author information

Authors and Affiliations

  1. University of Louisiana at Lafayette, 104 E. University Circle, Lafayette, LA, 70504, USA

    Yang Yang

  2. Texas Christian University, Fort Worth, TX, USA

    Jennifer Pankow & Kevin Knight

  3. Abt Associates, Inc., Cambridge, MA, USA

    Holly Swan

  4. University of Nevada, Reno, NV, USA

    Jennifer Willett

  5. Friends Research Institute, Inc., Baltimore, MD, USA

    Shannon Gwin Mitchell

  6. George Mason University, Fairfax, VA, USA

    Danielle S. Rudes

Authors
  1. Yang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jennifer Pankow
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Holly Swan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jennifer Willett
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shannon Gwin Mitchell
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Danielle S. Rudes
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Kevin Knight
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yang Yang.

Appendix: An example of the checklist for team logistics

Appendix: An example of the checklist for team logistics

Formatting, Labeling, and De-identification

  • Activities are completed

  • Check for accuracy

Managing Data Inventory

  • Documents have been moved to proper folders

  • All interview transcripts have been accounted for in folders

Hermeneutic Unit (HU) Management

  • HU has been installed in the designated directory

  • Confirm all documents are accounted for in HU

  • HU back-up has been completed weekly

Sharing Data

  • Confirm de-identification of pathways before uploading

  • Confirm all documents have been accounted for in HU

  • Confirm all files have been archived

Establishing Inter-coder Reliability

  • Team members have completed codebook training

  • Agreement among coder has been calculation

  • Set up additional training, if needed

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yang, Y., Pankow, J., Swan, H. et al. Preparing for analysis: a practical guide for a critical step for procedural rigor in large-scale multisite qualitative research studies. Qual Quant 52, 815–828 (2018). https://doi.org/10.1007/s11135-017-0490-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-017-0490-y

Keywords

  • Qualitative research
  • Procedural rigor
  • Team building
  • Inter-coder reliability
  • Multisite research