Reporting Formative Qualitative Research to Support the Development of Quantitative Preference Study Protocols and Corresponding Survey Instruments: Guidelines for Authors and Reviewers



Formative qualitative research is foundational to the methodological development process of quantitative health preference research (HPR). Despite its ability to improve the validity of the quantitative evidence, formative qualitative research is underreported.


To improve the frequency and quality of reporting, we developed guidelines for reporting this type of research. The guidelines focus on formative qualitative research used to develop robust and acceptable quantitative study protocols and corresponding survey instruments in HPR.


In December 2018, a steering committee was formed as a means to accumulate the expertise of the HPR community on the reporting guidelines (21 members, seven countries, multiple settings and disciplines). Using existing guidelines and examples, the committee constructed, revised, and refined the guidelines. The guidelines underwent beta testing by three researchers, and further revisions to the guidelines were made based on their feedback as well as on comments from members of the International Academy of Health Preference Research (IAHPR) and the editorial board of The Patient: Patient-Centered Outcomes Research.


The guidelines have five components: introductory material (4 domains), methods (12), results/findings (2), discussion (2), and other (2). They are concordant with existing guidelines, published examples, beta-testing results, and expert comments.


Publishing formative qualitative research is a necessary step toward strengthening the foundation of any quantitative study, enhancing the relevance of its preference evidence. The guidelines should aid researchers, reviewers, and regulatory agencies and promote transparency within HPR more broadly.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 49.95

Price includes VAT for USA

Fig. 1
Fig. 2

Data Availability

Data sharing is not applicable to this article because no datasets were generated or analyzed during the current study.


  1. 1.

    US Department of Health and Human Resources Food and Drug Administration. Patient-focused drug development: collecting comprehensive and representative input Guidance for industry, food and drug administration staff, and other stakeholders. Silver Spring; 2018.

  2. 2.

    21st Century Cures Act. H.R. 34, 114th congress. 2016. Accessed 30 Sept 2019.

  3. 3.

    Onwuegbuzie AJ, Bustamante RM, Nelson JA. Mixed research as a tool for developing quantitative instruments. J Mix Methods Res. 2010;4(1):56–78.

  4. 4.

    Soekhai V, de Bekker-Grob EW, Ellis A, Vass C. Discrete choice experiments in health economics: past, present and future. Pharmacoeconomics. 2019;37:201–26.

  5. 5.

    US Department of Health and Human Resources Food and Drug Administration. Patient preference information—submission, review in PMAs, HDE applications, and de novo requests, and inclusion in device labeling: draft guidance for industry, food and drug administration staff, and other stakeholders. Silver Spring, MD; 2015.

  6. 6.

    Bryman A. Social research methods. 2nd ed. Oxford: Oxford University Press; 2004.

  7. 7.

    Remler DK, Van Ryzin GG. Research methods in practice: Strategies for description and causation. Thousand Oaks: Sage Publications, Inc.; 2011.

  8. 8.

    Coast J. Qualitative methods for health economics. London: Rowman & Littlefield International; 2017.

  9. 9.

    US Department of Health and Human Resources Food and Drug Administration. Guidance for industry patient-reported outcome measures: use in medical product development to support labeling claims. Silver Spring, MD; 2009.

  10. 10.

    Magasi S, Ryan G, Revicki D, Lenderking W, Hays RD, Brod M, et al. Content validity of patient-reported outcome measures: perspectives from a PROMIS meeting. Qual Life Res. 2012;21(5):739–46.

  11. 11.

    Coast J, Horrocks S. Developing attributes and levels for discrete choice experiments using qualitative methods. J Health Serv Res Policy. 2007;12(1):25–30.

  12. 12.

    Coast J, Al-Janabi H, Sutton EJ, Horrocks SA, Vosper AJ, Swancutt DR, et al. Using qualitative methods for attribute development for discrete choice experiments: issues and recommendations. Health Econ. 2012;21(6):730–41.

  13. 13.

    Vass C, Rigby D, Payne K. The role of qualitative research methods in discrete choice experiments. Med Decis Making. 2017;37(3):298–313.

  14. 14.

    Clark MD, Determann D, Petrou S, Moro D, de Bekker-Grob EW. Discrete choice experiments in health economics: a review of the literature. Pharmacoeconomics. 2014;32(9):883–902.

  15. 15.

    Louviere JJ, Lancsar E. Choice experiments in health: the good, the bad, the ugly and toward a brighter future. Health Econ Policy Law. 2009;4(Pt 4):527–46.

  16. 16.

    Louviere J, Hensher D, Swait J. Stated choice methods—analysis and application. Cambridge: Cambridge University; 2000.

  17. 17.

    Janssen EM, Bridges JFP. Art and science of instrument development for stated-preference methods. Patient. 2017;10(4):377–9.

  18. 18.

    Brod M, Tesler LE, Christensen TL. Qualitative research and content validity: developing best practices based on science and experience. Qual Life Res. 2009;18(9):1263–78.

  19. 19.

    Fargher EA, Eddy C, Newman W, Qasim F, Tricker K, Elliott RA, et al. Patients’ and healthcare professionals’ views on pharmacogenetic testing and its future delivery in the NHS. Pharmacogenomics. 2007;8(11):1511–9.

  20. 20.

    Payne K, Fargher EA, Roberts SA, Tricker K, Elliott RA, Ratcliffe J, et al. Valuing pharmacogenetic testing services: a comparison of patients’ and health care professionals’ preferences. Value Health. 2011;14(1):121–34.

  21. 21.

    Witter S, Ha BTT, Shengalia B, Vujicic M. Understanding the ‘four directions of travel’: qualitative research into the factors affecting recruitment and retention of doctors in rural Vietnam. Hum Resour Health. 2011;9(1):20.

  22. 22.

    Vujicic M, Shengelia B, Alfano M, Thu HB. Physician shortages in rural Vietnam: using a labor market approach to inform policy. Soc Sci Med. 2011;73(7):970–7.

  23. 23.

    Dancet EAF, Van Empel IWH, Rober P, Nelen WLDM, Kremer JAM, d’Hooghe TM. Patient-centred infertility care: a qualitative study to listen to the patient’s voice. Hum Reprod. 2011;26(4):827–33.

  24. 24.

    van Empel IWH, Dancet EAF, Koolman XHE, Nelen WLDM, Stolk EA, Sermeus W, et al. Physicians underestimate the importance of patient-centredness to patients: a discrete choice experiment in fertility care. Hum Reprod. 2011;26(3):584–93.

  25. 25.

    Cohen DJ, Crabtree BF. Evaluative criteria for qualitative research in health care: controversies and recommendations. Ann Fam Med. 2008;6(4):331–9.

  26. 26.

    Coast J, McDonald R, Baker R. Issues arising from the use of qualitative methods in health economics. J Health Serv Res Policy. 2004;9(3):171–6.

  27. 27.

    O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245–51.

  28. 28.

    Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

  29. 29.

    Kitto SC, Chesters J, Grbich C. Quality in qualitative research. Med J Aust. 2008;188(4):243–6.

  30. 30.

    Hammarberg K, Kirkman M, de Lacey S. Qualitative research methods: when to use them and how to judge them. Hum Reprod. 2016;31(3):498–501.

  31. 31.

    Côté L, Turgeon J. Appraising qualitative research articles in medicine and medical education. 2005;27(1):71–5.

  32. 32.

    UK EQUATOR Centre. EQUATOR network: enhancing the QUAlity and transparency of health research. Center for Statistics in Medicine (CSM), NDORMS, University of Oxford. Accessed 14 Jan 2019.

  33. 33.

    Springer. Additional information for authors. Spring. 2019. Accessed 14 Jan 2019.

  34. 34.

    Danner M, Vennedey V, Hiligsmann M, Fauser S, Stock S. Focus groups in elderly ophthalmologic patients: setting the stage for quantitative preference elicitation. Patient. 2016;9(1):47–57.

  35. 35.

    de Bekker-Grob EW, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ. 2012;21(2):145–72.

  36. 36.

    Cheung KL, Wijnen BF, Hollin IL, Janssen EM, Bridges JF, Evers SM, et al. Using best-worst scaling to investigate preferences in health care. Pharmacoeconomics. 2016;34:1195–209.

  37. 37.

    Kløjgaard ME, Bech M, Søgaard R. Designing a stated choice experiment: the value of a qualitative process. J Choice Model. 2012;5(2):1–18.

  38. 38.

    Coast J. Presenting and publishing qualitative research in health economics. In: Coast J, editor. Qualitative methods for health economics. London: Rowman & Littlefield International Ltd; 2017.

  39. 39.

    Hollin IL, Young C, Hanson C, Bridges JFP. Developing a patient-centered benefit-risk survey: a community-engaged process. Value Health. 2016;19(6):751–7.

  40. 40.

    Janssen EM, Segal JB, Bridges JF. A framework for instrument development of a choice experiment: an application to type 2 diabetes. Patient. 2016;9(5):465–79.

  41. 41.

    dosReis S, Castillo WC, Ross M, Fitz-Randolph M, Vaughn-Lee A, Butler B. Attribute development using continuous stakeholder engagement to prioritize treatment decisions: a framework for patient-centered research. Value Health. 2016;19(6):758–66.

  42. 42.

    Owen-Smith A, Coast J. Understanding sampling and recruitment. In: Coast J, editor. Qualitative methods for health economics. London: Rowman & Littlefield International; 2017.

  43. 43.

    Daly J, Willis K, Small R, Green J, Welch N, Kealy M, et al. A hierarchy of evidence for assessing qualitative health research. J Clin Epidemiol. 2007;60(1):43–9.

  44. 44.

    Patton M. Qualitative evaluation and research methods. 2nd ed. Newbury Park: Sage Publications; 1990.

  45. 45.

    Kerr C, Nixon A, Wild D. Assessing and demonstrating data saturation in qualitative inquiry supporting patient-reported outcomes research. Expert Rev Pharmacoecon Outcomes Res. 2010;10(3):269–81.

  46. 46.

    Federal Policy for the Protection of Human Subjects. 82 FR 7149. Jan 19, 2017. Accessed 30 Sept 2019.

  47. 47.

    Al-Janabi H, Coast J, Flynn TN. What do people value when they provide unpaid care for an older person? A meta-ethnography with interview follow-up. Soc Sci Med. 2008;67(1):111–21.

  48. 48.

    Abiiro GA, Leppert G, Mbera GB, Robyn PJ, De Allegri M. Developing attributes and attribute-levels for a discrete choice experiment on micro health insurance in rural Malawi. BMC Health Serv Res. 2014;14:235.

  49. 49.

    Sutton EJ, Coast J. Development of a supportive care measure for economic evaluation of end-of-life care using qualitative methods. Palliat Med. 2014;28(2):151–7.

  50. 50.

    Canaway A, Al-Janabi H, Kinghorn P, Bailey C, Coast J. Development of a measure (ICECAP-Close Person Measure) through qualitative methods to capture the benefits of end-of-life care to those close to the dying for use in economic evaluation. Palliat Med. 2017;31(1):53–62.

  51. 51.

    Stevens KJ. Working with children to develop dimensions for a preference-based, generic, pediatric, health-related quality-of-life measure. Qual Health Res. 2010;20(3):340–51.

  52. 52.

    Stevens K. Developing a descriptive system for a new preference-based measure of health-related quality of life for children. Qual Life Res. 2009;18(8):1105–13.

  53. 53.

    Ke KM, Mackichan F, Sandy JR, Ness AR, Hollingworth W. Parents’ perspectives on centralized cleft services for children: the development of a DCE questionnaire. Oral Dis. 2013;19(2):185–92.

  54. 54.

    Michaels-Igbokwe C, Lagarde M, Cairns J, Terris-Prestholt F. Using decision mapping to inform the development of a stated choice survey to elicit youth preferences for sexual and reproductive health and HIV services in rural Malawi. Soc Sci Med. 2014;105:93–102.

  55. 55.

    Beusterien K, Chan E, Such P, De Jong Laird A, Heres S, Amos K, et al. Development of a stated-preference instrument to prioritize treatment goals in recent onset schizophrenia. Curr Med Res Opin. 2017;33(12):2129–36.

  56. 56.

    Bridges JF, Gallego G, Kudo M, Okita K, Han KH, Ye SL, et al. Identifying and prioritizing strategies for comprehensive liver cancer control in Asia. BMC Health Serv Res. 2011;11(1):298.

  57. 57.

    Ryden A, Chen S, Flood E, Romero B, Grandy S. Discrete choice experiment attribute selection using a multinational interview study: treatment features important to patients with type 2 diabetes mellitus. Patient. 2017;10(4):475–87.

  58. 58.

    Gilbert C, Keay L, Palagyi A, Do VQ, McCluskey P, White A, et al. Investigation of attributes which guide choice in cataract surgery services in urban Sydney, Australia. Clin Exp Optom. 2018;101(3):363–71.

  59. 59.

    Staniszewska S, Brett J, Mockford C, Barber R. The GRIPP checklist: strengthening the quality of patient and public involvement reporting in research. Int J Technol Assess Health Care. 2011;27(04):391–9.

  60. 60.

    Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ. 2017;358:j3453.

  61. 61.

    Barbour RS. Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? BMJ. 2001;322(7294):1115–7.

  62. 62.

    Lincoln YS. Emerging criteria for quality in qualitative and interpretive research. Qual Inq. 1995;1(3):275–89.

Download references


The authors of this manuscript are grateful to the members of the steering committee who commented on draft versions of this manuscript: Olalekan Lee Aiyegbusi, Esther de Bekker-Grob, Emuella Flood, Gisselle Gallego, Ellen Janssen, Rachel Milte, Axel Mühlbacher, Semra Ozdemir Van Dyk, Jason Ong, Thomas Poder, Fern Terris-Prestholt, and Albert Wu. The authors also wish to acknowledge Emily Mulivihill, who provided input on previous versions of this manuscript. The authors thank all members of the IAHPR and The Patient editorial board for their participation in this process. Finally, the authors acknowledge the authors who tested these guidelines and provided feedback: Richard De Abreu Lourenco, Maria McCarthy, Emily Holmes, and Seda Erdem.

Author information

ILH made substantial contributions to the conception and design of the work, drafted the manuscript, revised the manuscript critically for intellectual content, and provided final approval of the manuscript. BMC made substantial contributions to the conception and design of the work, revised the manuscript critically, and provided final approval of the manuscript. JC, KB, CV, RD, and HP made substantial contributions to the design of the work, revised the manuscript critically, and provided final approval of the manuscript. All authors agree to be accountable for all aspects of the work.

Correspondence to Ilene L. Hollin.

Ethics declarations


No sources of funding were used to conduct this study or prepare this manuscript.

Conflict of Interest

Ilene L. Hollin, Benjamin M. Craig, Joanna Coast, Kathleen Beusterien, Caroline Vass, Rachael DiSantostefano, and Holly Peay have no conflicts of interest that are directly relevant to the content of this article.

Additional information

This article is part of the topical collection “Formative qualitative evidence for health preference and outcomes research”.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hollin, I.L., Craig, B.M., Coast, J. et al. Reporting Formative Qualitative Research to Support the Development of Quantitative Preference Study Protocols and Corresponding Survey Instruments: Guidelines for Authors and Reviewers. Patient 13, 121–136 (2020) doi:10.1007/s40271-019-00401-x

Download citation