Skip to main content

Advertisement

SpringerLink
Log in

Reporting Formative Qualitative Research to Support the Development of Quantitative Preference Study Protocols and Corresponding Survey Instruments: Guidelines for Authors and Reviewers

  • Original Research Article
  • Published:
The Patient - Patient-Centered Outcomes Research Aims and scope Submit manuscript

Abstract

Background

Formative qualitative research is foundational to the methodological development process of quantitative health preference research (HPR). Despite its ability to improve the validity of the quantitative evidence, formative qualitative research is underreported.

Objective

To improve the frequency and quality of reporting, we developed guidelines for reporting this type of research. The guidelines focus on formative qualitative research used to develop robust and acceptable quantitative study protocols and corresponding survey instruments in HPR.

Methods

In December 2018, a steering committee was formed as a means to accumulate the expertise of the HPR community on the reporting guidelines (21 members, seven countries, multiple settings and disciplines). Using existing guidelines and examples, the committee constructed, revised, and refined the guidelines. The guidelines underwent beta testing by three researchers, and further revisions to the guidelines were made based on their feedback as well as on comments from members of the International Academy of Health Preference Research (IAHPR) and the editorial board of The Patient: Patient-Centered Outcomes Research.

Results

The guidelines have five components: introductory material (4 domains), methods (12), results/findings (2), discussion (2), and other (2). They are concordant with existing guidelines, published examples, beta-testing results, and expert comments.

Conclusions

Publishing formative qualitative research is a necessary step toward strengthening the foundation of any quantitative study, enhancing the relevance of its preference evidence. The guidelines should aid researchers, reviewers, and regulatory agencies and promote transparency within HPR more broadly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Data Availability

Data sharing is not applicable to this article because no datasets were generated or analyzed during the current study.

References

  1. US Department of Health and Human Resources Food and Drug Administration. Patient-focused drug development: collecting comprehensive and representative input Guidance for industry, food and drug administration staff, and other stakeholders. Silver Spring; 2018.

  2. 21st Century Cures Act. H.R. 34, 114th congress. 2016. https://www.gpo.gov/fdsys/pkg/BILLS-114hr34enr/pdf/BILLS-114hr34enr.pdf. Accessed 30 Sept 2019.

  3. Onwuegbuzie AJ, Bustamante RM, Nelson JA. Mixed research as a tool for developing quantitative instruments. J Mix Methods Res. 2010;4(1):56–78.

    Google Scholar 

  4. Soekhai V, de Bekker-Grob EW, Ellis A, Vass C. Discrete choice experiments in health economics: past, present and future. Pharmacoeconomics. 2019;37:201–26.

    Google Scholar 

  5. US Department of Health and Human Resources Food and Drug Administration. Patient preference information—submission, review in PMAs, HDE applications, and de novo requests, and inclusion in device labeling: draft guidance for industry, food and drug administration staff, and other stakeholders. Silver Spring, MD; 2015.

  6. Bryman A. Social research methods. 2nd ed. Oxford: Oxford University Press; 2004.

    Google Scholar 

  7. Remler DK, Van Ryzin GG. Research methods in practice: Strategies for description and causation. Thousand Oaks: Sage Publications, Inc.; 2011.

    Google Scholar 

  8. Coast J. Qualitative methods for health economics. London: Rowman & Littlefield International; 2017.

    Google Scholar 

  9. US Department of Health and Human Resources Food and Drug Administration. Guidance for industry patient-reported outcome measures: use in medical product development to support labeling claims. Silver Spring, MD; 2009.

  10. Magasi S, Ryan G, Revicki D, Lenderking W, Hays RD, Brod M, et al. Content validity of patient-reported outcome measures: perspectives from a PROMIS meeting. Qual Life Res. 2012;21(5):739–46.

    PubMed  Google Scholar 

  11. Coast J, Horrocks S. Developing attributes and levels for discrete choice experiments using qualitative methods. J Health Serv Res Policy. 2007;12(1):25–30.

    PubMed  Google Scholar 

  12. Coast J, Al-Janabi H, Sutton EJ, Horrocks SA, Vosper AJ, Swancutt DR, et al. Using qualitative methods for attribute development for discrete choice experiments: issues and recommendations. Health Econ. 2012;21(6):730–41.

    PubMed  Google Scholar 

  13. Vass C, Rigby D, Payne K. The role of qualitative research methods in discrete choice experiments. Med Decis Making. 2017;37(3):298–313.

    PubMed  PubMed Central  Google Scholar 

  14. Clark MD, Determann D, Petrou S, Moro D, de Bekker-Grob EW. Discrete choice experiments in health economics: a review of the literature. Pharmacoeconomics. 2014;32(9):883–902.

    PubMed  Google Scholar 

  15. Louviere JJ, Lancsar E. Choice experiments in health: the good, the bad, the ugly and toward a brighter future. Health Econ Policy Law. 2009;4(Pt 4):527–46.

    PubMed  Google Scholar 

  16. Louviere J, Hensher D, Swait J. Stated choice methods—analysis and application. Cambridge: Cambridge University; 2000.

    Google Scholar 

  17. Janssen EM, Bridges JFP. Art and science of instrument development for stated-preference methods. Patient. 2017;10(4):377–9.

    PubMed  Google Scholar 

  18. Brod M, Tesler LE, Christensen TL. Qualitative research and content validity: developing best practices based on science and experience. Qual Life Res. 2009;18(9):1263–78.

    PubMed  Google Scholar 

  19. Fargher EA, Eddy C, Newman W, Qasim F, Tricker K, Elliott RA, et al. Patients’ and healthcare professionals’ views on pharmacogenetic testing and its future delivery in the NHS. Pharmacogenomics. 2007;8(11):1511–9.

    PubMed  Google Scholar 

  20. Payne K, Fargher EA, Roberts SA, Tricker K, Elliott RA, Ratcliffe J, et al. Valuing pharmacogenetic testing services: a comparison of patients’ and health care professionals’ preferences. Value Health. 2011;14(1):121–34.

    PubMed  Google Scholar 

  21. Witter S, Ha BTT, Shengalia B, Vujicic M. Understanding the ‘four directions of travel’: qualitative research into the factors affecting recruitment and retention of doctors in rural Vietnam. Hum Resour Health. 2011;9(1):20.

    PubMed  PubMed Central  Google Scholar 

  22. Vujicic M, Shengelia B, Alfano M, Thu HB. Physician shortages in rural Vietnam: using a labor market approach to inform policy. Soc Sci Med. 2011;73(7):970–7.

    PubMed  Google Scholar 

  23. Dancet EAF, Van Empel IWH, Rober P, Nelen WLDM, Kremer JAM, d’Hooghe TM. Patient-centred infertility care: a qualitative study to listen to the patient’s voice. Hum Reprod. 2011;26(4):827–33.

    CAS  PubMed  Google Scholar 

  24. van Empel IWH, Dancet EAF, Koolman XHE, Nelen WLDM, Stolk EA, Sermeus W, et al. Physicians underestimate the importance of patient-centredness to patients: a discrete choice experiment in fertility care. Hum Reprod. 2011;26(3):584–93.

    PubMed  Google Scholar 

  25. Cohen DJ, Crabtree BF. Evaluative criteria for qualitative research in health care: controversies and recommendations. Ann Fam Med. 2008;6(4):331–9.

    PubMed  PubMed Central  Google Scholar 

  26. Coast J, McDonald R, Baker R. Issues arising from the use of qualitative methods in health economics. J Health Serv Res Policy. 2004;9(3):171–6.

    PubMed  Google Scholar 

  27. O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245–51.

    PubMed  Google Scholar 

  28. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

    PubMed  Google Scholar 

  29. Kitto SC, Chesters J, Grbich C. Quality in qualitative research. Med J Aust. 2008;188(4):243–6.

    PubMed  Google Scholar 

  30. Hammarberg K, Kirkman M, de Lacey S. Qualitative research methods: when to use them and how to judge them. Hum Reprod. 2016;31(3):498–501.

    CAS  PubMed  Google Scholar 

  31. Côté L, Turgeon J. Appraising qualitative research articles in medicine and medical education. 2005;27(1):71–5.

    Google Scholar 

  32. UK EQUATOR Centre. EQUATOR network: enhancing the QUAlity and transparency of health research. Center for Statistics in Medicine (CSM), NDORMS, University of Oxford. https://www.equator-network.org/. Accessed 14 Jan 2019.

  33. Springer. Additional information for authors. Spring. 2019. https://www.springer.com/adis/journal/40271. Accessed 14 Jan 2019.

  34. Danner M, Vennedey V, Hiligsmann M, Fauser S, Stock S. Focus groups in elderly ophthalmologic patients: setting the stage for quantitative preference elicitation. Patient. 2016;9(1):47–57.

    PubMed  Google Scholar 

  35. de Bekker-Grob EW, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ. 2012;21(2):145–72.

    PubMed  Google Scholar 

  36. Cheung KL, Wijnen BF, Hollin IL, Janssen EM, Bridges JF, Evers SM, et al. Using best-worst scaling to investigate preferences in health care. Pharmacoeconomics. 2016;34:1195–209. https://doi.org/10.1007/s40273-016-0429-5.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Kløjgaard ME, Bech M, Søgaard R. Designing a stated choice experiment: the value of a qualitative process. J Choice Model. 2012;5(2):1–18.

    Google Scholar 

  38. Coast J. Presenting and publishing qualitative research in health economics. In: Coast J, editor. Qualitative methods for health economics. London: Rowman & Littlefield International Ltd; 2017.

    Google Scholar 

  39. Hollin IL, Young C, Hanson C, Bridges JFP. Developing a patient-centered benefit-risk survey: a community-engaged process. Value Health. 2016;19(6):751–7.

    PubMed  Google Scholar 

  40. Janssen EM, Segal JB, Bridges JF. A framework for instrument development of a choice experiment: an application to type 2 diabetes. Patient. 2016;9(5):465–79.

    PubMed  Google Scholar 

  41. dosReis S, Castillo WC, Ross M, Fitz-Randolph M, Vaughn-Lee A, Butler B. Attribute development using continuous stakeholder engagement to prioritize treatment decisions: a framework for patient-centered research. Value Health. 2016;19(6):758–66.

    PubMed  Google Scholar 

  42. Owen-Smith A, Coast J. Understanding sampling and recruitment. In: Coast J, editor. Qualitative methods for health economics. London: Rowman & Littlefield International; 2017.

    Google Scholar 

  43. Daly J, Willis K, Small R, Green J, Welch N, Kealy M, et al. A hierarchy of evidence for assessing qualitative health research. J Clin Epidemiol. 2007;60(1):43–9.

    PubMed  Google Scholar 

  44. Patton M. Qualitative evaluation and research methods. 2nd ed. Newbury Park: Sage Publications; 1990.

    Google Scholar 

  45. Kerr C, Nixon A, Wild D. Assessing and demonstrating data saturation in qualitative inquiry supporting patient-reported outcomes research. Expert Rev Pharmacoecon Outcomes Res. 2010;10(3):269–81.

    PubMed  Google Scholar 

  46. Federal Policy for the Protection of Human Subjects. 82 FR 7149. Jan 19, 2017. https://www.govinfo.gov/content/pkg/FR-2017-01-19/pdf/2017-01058.pdf. Accessed 30 Sept 2019.

  47. Al-Janabi H, Coast J, Flynn TN. What do people value when they provide unpaid care for an older person? A meta-ethnography with interview follow-up. Soc Sci Med. 2008;67(1):111–21.

    PubMed  Google Scholar 

  48. Abiiro GA, Leppert G, Mbera GB, Robyn PJ, De Allegri M. Developing attributes and attribute-levels for a discrete choice experiment on micro health insurance in rural Malawi. BMC Health Serv Res. 2014;14:235.

    PubMed  PubMed Central  Google Scholar 

  49. Sutton EJ, Coast J. Development of a supportive care measure for economic evaluation of end-of-life care using qualitative methods. Palliat Med. 2014;28(2):151–7.

    PubMed  Google Scholar 

  50. Canaway A, Al-Janabi H, Kinghorn P, Bailey C, Coast J. Development of a measure (ICECAP-Close Person Measure) through qualitative methods to capture the benefits of end-of-life care to those close to the dying for use in economic evaluation. Palliat Med. 2017;31(1):53–62.

    PubMed  Google Scholar 

  51. Stevens KJ. Working with children to develop dimensions for a preference-based, generic, pediatric, health-related quality-of-life measure. Qual Health Res. 2010;20(3):340–51.

    PubMed  Google Scholar 

  52. Stevens K. Developing a descriptive system for a new preference-based measure of health-related quality of life for children. Qual Life Res. 2009;18(8):1105–13.

    PubMed  Google Scholar 

  53. Ke KM, Mackichan F, Sandy JR, Ness AR, Hollingworth W. Parents’ perspectives on centralized cleft services for children: the development of a DCE questionnaire. Oral Dis. 2013;19(2):185–92.

    CAS  PubMed  Google Scholar 

  54. Michaels-Igbokwe C, Lagarde M, Cairns J, Terris-Prestholt F. Using decision mapping to inform the development of a stated choice survey to elicit youth preferences for sexual and reproductive health and HIV services in rural Malawi. Soc Sci Med. 2014;105:93–102.

    PubMed  Google Scholar 

  55. Beusterien K, Chan E, Such P, De Jong Laird A, Heres S, Amos K, et al. Development of a stated-preference instrument to prioritize treatment goals in recent onset schizophrenia. Curr Med Res Opin. 2017;33(12):2129–36.

    PubMed  Google Scholar 

  56. Bridges JF, Gallego G, Kudo M, Okita K, Han KH, Ye SL, et al. Identifying and prioritizing strategies for comprehensive liver cancer control in Asia. BMC Health Serv Res. 2011;11(1):298.

    PubMed  PubMed Central  Google Scholar 

  57. Ryden A, Chen S, Flood E, Romero B, Grandy S. Discrete choice experiment attribute selection using a multinational interview study: treatment features important to patients with type 2 diabetes mellitus. Patient. 2017;10(4):475–87.

    PubMed  Google Scholar 

  58. Gilbert C, Keay L, Palagyi A, Do VQ, McCluskey P, White A, et al. Investigation of attributes which guide choice in cataract surgery services in urban Sydney, Australia. Clin Exp Optom. 2018;101(3):363–71.

    PubMed  Google Scholar 

  59. Staniszewska S, Brett J, Mockford C, Barber R. The GRIPP checklist: strengthening the quality of patient and public involvement reporting in research. Int J Technol Assess Health Care. 2011;27(04):391–9.

    PubMed  Google Scholar 

  60. Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ. 2017;358:j3453.

    CAS  PubMed  PubMed Central  Google Scholar 

  61. Barbour RS. Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? BMJ. 2001;322(7294):1115–7.

    CAS  PubMed  PubMed Central  Google Scholar 

  62. Lincoln YS. Emerging criteria for quality in qualitative and interpretive research. Qual Inq. 1995;1(3):275–89.

    Google Scholar 

Download references

Acknowledgements

The authors of this manuscript are grateful to the members of the steering committee who commented on draft versions of this manuscript: Olalekan Lee Aiyegbusi, Esther de Bekker-Grob, Emuella Flood, Gisselle Gallego, Ellen Janssen, Rachel Milte, Axel Mühlbacher, Semra Ozdemir Van Dyk, Jason Ong, Thomas Poder, Fern Terris-Prestholt, and Albert Wu. The authors also wish to acknowledge Emily Mulivihill, who provided input on previous versions of this manuscript. The authors thank all members of the IAHPR and The Patient editorial board for their participation in this process. Finally, the authors acknowledge the authors who tested these guidelines and provided feedback: Richard De Abreu Lourenco, Maria McCarthy, Emily Holmes, and Seda Erdem.

Author information

Authors and Affiliations

  1. Department of Health Services Administration and Policy, College of Public Health, Temple University, 1301 Cecil B. Moore Ave., Room 537, Philadelphia, PA, 19122, USA

    Ilene L. Hollin

  2. Department of Economics College of Arts and Sciences, University of South Florida, Tampa, FL, USA

    Benjamin M. Craig

  3. Health Economics Bristol, Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK

    Joanna Coast

  4. Health Division, Kantar, New York, NY, USA

    Kathleen Beusterien

  5. Health Preference Assessment Group, RTI Health Solutions, Didsbury, Manchester, UK

    Caroline Vass

  6. Department of Epidemiology, Janssen R&D, LLC, Titusville, NJ, USA

    Rachael DiSantostefano

  7. RTI International, Research Triangle Park, NC, USA

    Holly Peay

Authors
  1. Ilene L. Hollin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benjamin M. Craig
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joanna Coast
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kathleen Beusterien
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Caroline Vass
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rachael DiSantostefano
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Holly Peay
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ILH made substantial contributions to the conception and design of the work, drafted the manuscript, revised the manuscript critically for intellectual content, and provided final approval of the manuscript. BMC made substantial contributions to the conception and design of the work, revised the manuscript critically, and provided final approval of the manuscript. JC, KB, CV, RD, and HP made substantial contributions to the design of the work, revised the manuscript critically, and provided final approval of the manuscript. All authors agree to be accountable for all aspects of the work.

Corresponding author

Correspondence to Ilene L. Hollin.

Ethics declarations

Funding

No sources of funding were used to conduct this study or prepare this manuscript.

Conflict of Interest

Ilene L. Hollin, Benjamin M. Craig, Joanna Coast, Kathleen Beusterien, Caroline Vass, Rachael DiSantostefano, and Holly Peay have no conflicts of interest that are directly relevant to the content of this article.

Additional information

This article is part of the topical collection “Formative qualitative evidence for health preference and outcomes research”.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hollin, I.L., Craig, B.M., Coast, J. et al. Reporting Formative Qualitative Research to Support the Development of Quantitative Preference Study Protocols and Corresponding Survey Instruments: Guidelines for Authors and Reviewers. Patient 13, 121–136 (2020). https://doi.org/10.1007/s40271-019-00401-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40271-019-00401-x

Search

Navigation