FormalPara Key Summary Points

Research requirements and budget and time constraints should be considered when choosing the most appropriate research method.

Qualitative concept elicitation interviews provided the greatest depth of understanding of the patient experience but incurred the highest costs and burden to research team and participants.

More time and cost-effective social media listening and literature review methods can be used to supplement qualitative concept elicitation interview data and provide early identification of measurement concepts.

Introduction

Qualitative research with patients, caregivers, and clinicians is often used as a means of providing an in-depth understanding of the patient experience of health conditions in terms of the symptoms experienced and the impact that the condition has on their health-related quality of life (HRQoL), for example, the physical, functional, and psychological impact of diseases and treatments. Such research can also be used to support the development, selection, and modification of patient-reported outcome (PRO) measures and other Clinical Outcome Assessments (COAs) [1,2,3].

Understanding, measuring, and communicating the holistic patient experience of health conditions and treatments is of increasing importance to stakeholders (including industry, regulatory authorities, health technology assessment [HTA] bodies, payers, clinicians, and patient organisations) to inform drug development [4, 5]. This increased focus on the patient perspective in healthcare research has been driven, in part, by recent US Food and Drug Administration (FDA) Patient-Focused Drug Development (PFDD) guidance [1]. This shift is also reflected in wider patient engagement initiatives, including more involvement of patient advocates in HTA decision making [6], organisations like the British Medical Journal and the Patient-Centred outcomes Research Institute involving patient advocates in their article and grant development processes [7, 8], and an increased emphasis on patient-centredness in clinical settings [9, 10]. All these patient-centric activities ultimately aim to support better service outcomes and increased patient satisfaction.

A range of qualitative methods are utilised by healthcare researchers to obtain patient perspectives on health conditions and treatments. Commonly used methods include qualitative concept elicitation (CE) interviews, focus groups, Delphi panels, group concept mapping, and open-ended surveys. In addition, targeted literature reviews and social media listening (SML) are other approaches used to gain an understanding of the patient experience of diseases and treatments.

Each qualitative research method, however, has strengths and weaknesses in terms of the breadth and depth of data generated, cost and time to conduct the research, and the burden on participants. For example, FDA guidelines [1, 2], reports from academic representatives [11], and findings from a qualitative method comparison study [12] suggest that qualitative CE interviews are still the strongest option for eliciting an in-depth understanding of the patient experience, generating rich and nuanced data, and are more scientifically rigorous than less traditional methods (e.g. SML). However, guidance and available evidence suggest that less traditional methods are faster and cheaper to conduct, present a lower burden for participants, can afford a larger sample size, and so still have potential for supporting the development of PRO measures [1, 2, 11, 12]. Thus, the latter may be considered in studies that have restricted budget and timelines.

The ideal situation for making direct comparisons of different qualitative research methods is when multiple qualitative research methods have been used to explore the same condition. However, only one such study was identified by the current authors. This study comprised an empirical evaluation of the merits and limitations of qualitative CE interviews, group concept mapping, and SML to explore the patient experience of ankylosing spondylitis [12]. A study specifically comparing the relative value of qualitative CE interviews, SML, and literature reviews has yet to be conducted; therefore, in this article empirical evidence is examined to compare the relative value of these three qualitative research methods for understanding the lived experience of presbyopia.

Presbyopia occurs when the physiologically normal age-related reduction in the eye's focusing range reaches a point, when optimally corrected for distance vision, where the clarity of near vision is insufficient to satisfy an individual's requirements [13, 14]. Presbyopia typically starts to develop in the 4th decade of life and is experienced in over 80% of people over this age. It was estimated there were 1.3 billion people living with presbyopia worldwide in 2017, which is projected to increase to 1.8 billion by 2050 [13]. It is hypothesised to be caused by a loss of lens elasticity preventing focal point change [15, 16]. While the aetiology of this condition is not fully understood, research suggests that an increase in lens rigidity is the primary causative mechanism [17, 18]. It is not currently possible to stop or reverse the ageing process that causes presbyopia; generally, it is corrected with glasses, contact lenses, or refractive surgery, or managed by the use of magnifiers [19]. Individuals with presbyopia have difficulty with tasks that rely on near vision [20, 21]. As a result, presbyopia has a significant impact on individuals’ HRQoL [20,21,22,23,24,25,26]. Despite its high prevalence, there is a lack of research into the lived experience of the condition [20]. It is important to understand the patient experience of presbyopia through qualitative research to develop adequate PRO measures to assess the impact of presbyopia on individual HRQoL and functioning.

In the current study, a targeted, structured literature review was first conducted to identify and critically evaluate the most commonly cited PRO assessments used in clinical trials and HRQoL studies conducted in individuals with presbyopia and to review the qualitative literature to identify and better understand concepts that are relevant to measure the patient experience of the condition [27]. As part of the literature review, studies that reported on the HRQoL of individuals with presbyopia were identified. Second, an SML study was conducted to explore how individuals used social media to describe their experience of presbyopia regarding the symptoms experienced and the impacts of presbyopia on their HRQoL. Aspects related to adjustments to the condition and the impact of current treatment options were also explored. This involved the review of posts from Twitter, online forums, blogs, and news outlets to supplement the literature review and provide further insight into the lived experience of presbyopia [28]. Third, qualitative CE interviews with individuals with presbyopia and healthcare professionals (HCPs), who were experienced in treating presbyopia, were conducted to gain a comprehensive understanding of the individual experience of phakic presbyopia (with the natural crystalline lens still intact) and to identify all relevant visual function symptoms and associated functional impacts [29]. A further aim of the study was to develop a conceptual model based on the findings from the literature review, SML study, and qualitative CE interviews.

Although the overall aim of these studies was to conduct qualitative research to gain a comprehensive understanding of the individual experience of phakic presbyopia, the aim of this article is to compare the insights obtained across the three sources of qualitative data: a literature review, an SML review, and qualitative CE interviews with individuals with presbyopia and HCPs. The number of concepts identified, depth of data, cost and time implications, and the value of the patient insights generated were explored and compared for each method.

Methods

Study Design

SML

The SML study aimed to explore how individuals used social media to describe their experience of presbyopia regarding the symptoms experienced and the impacts of presbyopia on their HRQoL. Aspects related to adjustments to the condition and the impact of current treatment options were also explored [28]. The study was a non-interventional retrospective analysis of social media data available on public platforms.

Literature Review

The objectives of the literature review were to identify and critically evaluate the most commonly cited PRO assessments used in clinical trials and HRQoL studies conducted in individuals with presbyopia and to review the qualitative literature to identify and better understand concepts that are relevant to measure the patient experience of the condition. The methods and findings of this review have been detailed in a separate article [27]. The present article focuses on the findings from the second objective—in-depth review of the qualitative research articles identified—to better understand the key symptoms and impact concepts of presbyopia.

Qualitative CE Interviews

The qualitative CE interviews aimed to gain an in-depth and unbiased understanding of the individual experience of presbyopia and to identify all relevant visual function symptoms and associated functional impact concepts. A further aim of the study was to develop a conceptual model based on the findings from the literature review, SML study, and qualitative CE interviews. Both individuals with phakic presbyopia and HCPs who were experienced in treating presbyopia were interviewed between February 2018 and July 2019. The interviews also included cognitive debriefing of the Near Activity Vision Questionnaire Presbyopia (NAVQ-P), but those methods and findings are not the focus of the present article [30]. Ethical approval and oversight was provided by Copernicus Group Independent Review Board (CGIRB; IRB ref: ADE1-18-049), an Independent Review Board (IRB) in the US for the US interviews, prior to any study related activities in the US. It was also obtained from Salus IRB (IRB ref: Novartis CUNR844A2001) for the German interviews prior to any study-related activities in Germany. In line with ethical requirements for research in France at the time of the research, a letter was submitted to the Conseil National de l'Ordre des Médecins (CNOM) to notify them of the study procedures and payments to recruiting health professionals. The study was performed in accordance with the Helsinki Declaration of 1864 and its later amendments [31]. Written and verbal informed consent was obtained from every participant before each interview was conducted and before any other study activities. Consent was obtained to publish the findings.

Individuals with presbyopia aged between 40 and 65 years from the US, Germany, and France were interviewed face-to-face over three rounds; each interview included 30–45 min of CE. Participants were recruited through a partner recruitment agency who worked closely with clinical sites to identify participants based on the eligibility criteria and study quotas. Recruiting clinician’s provided information on the date of diagnosis of presbyopia, visual acuity of each eye, clinician-rated severity of binocular distance-corrected near visual acuity (DCNVA) at 40 cm, and clinician-rated severity of near addition (ADD) to support confirmation of diagnosis. Individuals who had experienced lens extraction or replacement or those who had an ocular condition which affected their visual acuity (other than short-sightedness [myopia] or long-sightedness [hyperopia]) were excluded from the study (see supplementary material for the eligibility criteria). The inclusion criteria and study sample quotas were implemented to ensure a diverse and representative sample that reflected typical characteristics of individuals with presbyopia and to represent cohorts that may be targeted for future treatments.

The HCP interviews were conducted via telephone over two rounds and included 30 min of CE [29]. HCPs were either ophthalmologists or optometrists and were identified based on their expertise and contribution to the research field. All HCPs had at least one key contribution to the research area of presbyopia in terms of publications and/or congress sessions. Further information regarding the inclusion and exclusion criteria can be found in Appendix C.

Data Collection

SML

The search was conducted using Salesforce Social Studio [32] database to identify relevant content posted between May 2017 and August 2017 on publicly accessible social media sources including: Twitter, forums, blogs, and news posts (see supplementary material for the SML search strategy). The key words were searched in English, German, French, Spanish, Italian, and Japanese.

Literature Review

Keyword searches were performed in bibliographic databases (Medline and Embase) (see supplementary material for search strategy) up until October 2017. All clinical trials with PRO-supported endpoints in presbyopia were identified on ClinicalTrials.gov.

Qualitative CE Interviews

Trained, experienced interviewers conducted the interviews using a semi-structured interview guide. The interview guide included a list of questions and probes to guide the conversation. Topics of interest were based on the previously conducted literature and SML review. The guide was pilot tested in a mock interview conducted by the interviewers, and minor updates were made to the interview guide between rounds to ensure all topics of interest were fully explored. All interviews were audio-recorded and transcribed verbatim.

Data Analysis

SML

Content that was originally non-English was translated into English using Google Translate. Relevant content was then manually analysed through a process of data tagging by channel type (e.g. Twitter, forums, blogs, or news) and categorised according to the type of stakeholder where possible (e.g. individuals with presbyopia, physicians, and support groups), sentiment (positive, neutral, or negative), and key concepts (e.g. symptoms, impacts, treatment, lived experiences, and adjustments to presbyopia). Of the relevant content, that which was posted by individuals with presbyopia was identified, and symptom and impact concepts were extracted. The researchers used quotations to support the key concepts. Any direct quotations were anonymised by translation of non-English direct quotations into English, by paraphrasing any originally English quotations while retaining meaning, and by removing the username of content authors.

Literature Review

A manual review of relevant qualitative research articles was undertaken to identify the frequency of key presbyopia symptoms and impact concepts by paper. A list of verbatim statements was extracted to support each symptom and impact concept, where possible.

Qualitative CE Interviews

A qualitative analysis plan (QAP) was developed detailing how the qualitative data would be evaluated. Verbatim transcripts were coded using ATLAS.Ti [33] software and thematic analysis methods, involving identification and coding of quotes related to concepts/themes of interest. Thematic analysis is a foundational, theory-free, qualitative analysis method, which offers flexibility to provide a rich, detailed, and complex synthesis of data that meets a very specific and applied aim [34, 35]. Concept frequency was determined by counting the number of participants who mentioned a concept at least once during the interview. After analysing each transcript, a list of participant verbatim statements was generated for each symptom and impact concept identified. This information was used to refine an assessment of functional impairments in a presbyopia population.

Conceptual saturation is often used to determine the adequacy of qualitative sample sizes [36], and has been described as the point at which no new insights are likely to be obtained from analysis of further interviews [35, 37]. Conceptual saturation was evaluated by dividing the sample into five equal groups of ten individuals with phakic presbyopia (in the order the interviews were conducted). If no new concepts emerged in the last set of transcripts, then it was considered evidence that saturation had been achieved. Conceptual saturation analysis demonstrated that all concepts had been fully explored within this sample. Conceptual saturation was not assessed for the HCP interview sample because of the small sample size (n = 7).

Development of Conceptual Model

A key aim of the study was to develop a conceptual model to summarise the presbyopia symptom and impact concepts that emerged from the literature review, SML study, and qualitative CE interviews. A preliminary conceptual model developed based on findings from the literature review and SML study was updated iteratively at each stage of the qualitative CE interview study. All concepts identified across the three research methods were included in the model and grouped with similar concepts. This resulted in a number of domains, including: causes of presbyopia, exacerbating factors, visual function symptoms, physical (secondary) symptoms, proximal functional impacts, distal impacts on wider quality of life, adjustments to the condition, and impact of current correction options. See Fig. 1 for the conceptual model development process. The final version of the conceptual model included concepts identified from across the three qualitative methods.

Fig. 1
figure 1

Flowchart illustrating the conceptual model development process

Full size image

Comparison of Methods

Findings from the three qualitative methods were compared by the research team. The number of concepts elicited by each method and the degree of overlap was determined by reviewing the conceptual model, which featured a key indicating the source of each concept. Depth of data was assessed by comparing quotes elicited from each qualitative method. Comparing the number of concepts and depth of data helped to determine the scientific value of each method. Finally, a comparison of the impact on the research budget required, timelines, and participants’ time for each method was conducted. This comparison provided insight into the value and feasibility of each method.

Results

Sample Results

Literature Review

A total of 703 records were identified from initial searches. Of these, 120 records were deemed relevant publications that were selected for full-text review. Following full-text review, 35 articles were identified as being relevant to the qualitative experience of presbyopia, 18 articles qualitatively explored the individual experience of presbyopia, and 17 articles were identified as relevant to the PRO aspect of the literature review (the latter is summarised in a separate publication) [27]. See Fig. 2 for a flow chart of the selection process.

Fig. 2
figure 2

Flow chart illustrating the study selection process in the literature review

Full size image

SML

A large number of posts were downloaded (n = 4456), which was then reduced by 50% through random sampling to make the number of posts manageable to review (n = 2229). Relevant posts aligned to the study objectives were identified (n = 1470). Of the relevant posts, 270/1470 were identified as having been posted by individuals with presbyopia. These 270 posts from individuals were subsequently analysed, and symptoms and impact concepts were extracted. See Fig. 3 for a flow chart illustrating the sampling process.

Fig. 3
figure 3

Flow chart illustrating the post sampling in the SML review

Full size image

Qualitative CE Interviews

Fifty individuals with clinician-confirmed phakic presbyopia (US n = 30, France n = 10, Germany n = 10) and seven HCPs (US n = 3, France n = 2, Germany n = 1, Japan n = 1) participated in in-depth, face-to-face qualitative CE interviews. The total sample size of 50 participants ensured the study included participants from three countries across a range of demographic and clinical characteristics (age, sex, race, and disease severity). Adequate sample size was further confirmed using conceptual saturation analysis (see Data analysis section).

Comparison of Methods: Number of Concepts Identified

Across all three research methods, a total of 158 concepts were identified. These 158 concepts were categorised by symptom and impact domains, as presented in a combined conceptual model (see Fig. 4). The symptoms of presbyopia were separated into ‘visual function symptoms’ such as difficulty with near vision, and ‘physical (secondary) symptoms’ such as eye strain and headaches. Similarly, impacts of presbyopia were separated into ‘proximal impacts’, which are impacts that are a direct effect of the presbyopia symptoms, and ‘distal impacts’, which may be a result of a proximal impact rather than directly an impact of symptoms. The model also included additional domains discussed in the qualitative interviews, including ‘adjustments to the condition’, such as holding things further away, and ‘impact of correction options’, such as reliance on glasses, costs, or irritated eyes.

Fig. 4
figure 4

Conceptual model of presbyopia

Full size image

Of the total of 158 concepts, the CE interviews yielded the highest number of concepts (n = 151/158, 96%), with SML yielding a third of the concepts (n = 51/158, 32%) and the literature review yielding the fewest concepts (n = 33/158, 21%). The SML and literature review identified fewer visual functioning symptoms (e.g. blurry vision: n = 2/7 and n = 1/7, respectively) compared to the qualitative CE interviews (n = 7/7). All secondary symptoms (e.g. headaches) were identified in qualitative CE interviews (n = 9/9), whereas SML identified 5/9 symptoms (56%) and the literature review did not identify any. Proximal functional impacts (e.g. difficulty seeing objects in near vision, difficulty reading small print, and daily living impacts) were almost all identified in qualitative CE interviews (n = 41/42, 98%) but only half emerged through SML (n = 21/42, 50%) and even fewer from the literature review (n = 13/42, 31%). All concepts related to distal impacts on HRQoL (e.g. emotional, social, and work concepts) were identified in qualitative CE interviews (n = 38/38) but again emerged less frequently through SML (n = 9/38, 24%) and the literature review (n = 3/38, 8%). Concepts related to adjustments to the condition (e.g. squinting to see, holding reading material at a distance) were all identified in qualitative CE interviews (n = 25/25) but were found very infrequently through the SML (n = 5/25, 20%) and the literature review (n = 2/25, 8%). Impacts of correction aids were mostly identified in qualitative CE interviews (n = 28/34, 82%), but again came up less frequently through the literature review (n = 13/34, 38%) and the SML (n = 8/34, 24%).

Comparison of Methods: Depth of Data

The depth of data obtained from each qualitative research methodology was dependent on aspects including access to direct quotes from individuals with presbyopia, the ability to probe individuals further, and the range of topics explored (which was related to the study activity aim).

Unsurprisingly, the qualitative CE interviews provided a more in-depth exploration of concepts and sub-concepts. This was likely due to the interviewer having the opportunity to probe and explore topics further. In addition, it was also possible to be confident that the interview sample all had HCP-confirmed diagnosis of presbyopia and to collect demographic and clinical characteristics to fully characterise the sample. Gathering such data was not possible for the SML review, and in the literature, sometimes the clinical characteristics summarised were limited and generalisability was restricted because of the lack of geographical representation of the recruited samples. The SML review also generally generated short quotes, which often lacked detail and context; in some cases, this was likely due to character limits for specific social media platforms (e.g. Twitter). Without the opportunity to probe topics further, some quotes were vague and ambiguous. Additionally, it was not possible to confirm the individuals’ diagnosis or characterise the sample. The initial exclusion of 50% of posts collected during the SML review may have led to the loss of relevant data; however, this was unavoidable given the large number of posts identified, many of which were not relevant to the research. The literature review yielded some quotes from qualitative studies, but the number available was very limited; most of the findings from the literature review were summaries of the identified concepts only, without supporting quotes and additional descriptions.

Despite the literature and SML reviews eliciting more top-line data compared to the qualitative interviews, these methods were fundamental in shaping the interview guide content that was used for the qualitative interviews. The insight provided by the literature and SML reviews shaped the type of questions and concepts included in the guide and ultimately led to rich in-depth data being elicited in the qualitative interviews, highlighting the cumulative value of these methods.

Comparison of Methods: Budgetary Considerations

When considering the costs of each of these qualitative research approaches, aspects considered included the time required and professional costs to conduct the research, ethics submissions required for interviews, cost of software or papers, recruitment and participant compensation, and transcription costs. Overall, qualitative interviews required approximately four times (4 ×) more budget than a SML review and 1.5 × more budget than a literature review. The SML incurred the least costs as, although it had costs associated with the software needed to search social media platforms, the professional costs to run and analyse the posts were less than the literature review. The literature review incurred costs related to purchase of papers and researcher time to develop the search strategy, screen abstracts, and then review full-text papers. Qualitative CE interviews were by far the most expensive option, including the cost of extensive researcher time associated with the development of a study protocol and study documents to be submitted for ethical approval, interviewing time, transcription costs, travel costs for conducting face-to-face interviews, recruitment, participant compensation, and translation of any study documents and transcripts (for interviews conducted in non-English-speaking countries).

Comparison of Methods: Burden to Research Team and Participants

Considerations for research team burden include time required to develop and run the search strategy or developing study documents, abstract screening or data searching, review of papers or posts, recruitment, screening of participants, and conduct and analysis of interviews. Qualitative interviews took approximately twice (2 ×) as many project team hours as a SML review and literature review.

Literature review timelines can vary depending on the number of articles to be reviewed, but a relatively focussed/targeted literature review can typically be conducted in approximately 3–6 months; in this study it required approximately 4 months. Aspects that should be considered when planning literature review timelines include the development and running of search terms in databases, abstract screening, and review of papers.

SML typically has similar timelines to a literature review depending on the number of relevant posts that are yielded and the proportion of posts that are reviewed. Tasks that require time include defining and running the search strategy, random sampling to reduce the number of downloaded posts to a manageable size, filtering/reviewing the data to establish which content is relevant, and review of relevant posts.

The qualitative CE interviews were by far the most time-consuming option out of the three qualitative methods, taking approximately 9 months in total. Tasks involved the development of study documents and time required to submit for ethical review, recruitment, screening of participants, conduct of interviews, transcription of interview audio recordings, and analysis of interview transcripts. As interviews were conducted in multiple countries, time was also required to translate study documents and transcripts. Notably, however, conducting the interviews remotely by telephone/videoconferencing software can often be quicker, less burdensome for participant and interviewer, and more efficient than conducting interviews in person, but perhaps with some loss of opportunity to build rapport. Regardless of the interview format, however, qualitative CE interviews still posed a burden on participants’ time, which was not an issue for literature reviews or SML.

Discussion

This article sought to compare the benefits and limitations of three approaches to obtaining qualitative data to provide insights related to presbyopia symptoms and HRQoL impacts: a literature review, SML review, and qualitative CE interviews. The concepts identified, depth of patient insights and data, cost and time implications, and value of each method have been summarised.

Breadth and depth of data

SML uncovered more concepts than the literature review across all domains but one. Qualitative CE interviews identified more concepts across all domains and allowed greater depth of exploration than both the SML study and literature review. Notably, qualitative CE interviews allow for dialogue and probing, which are not possible because of the secondary nature of SML and literature review methods. These findings are aligned with FDA guidance and a previous study, which suggests qualitative CE interviews provide richer, more nuanced data than less traditional methods like SML [1, 2, 12, 36].

Budgetary and time considerations

Compared to the qualitative CE interview study, the SML and literature review were quicker and much more cost-effective due to substantially less researcher time being required and no recruitment or participant compensation costs. It must be noted, however, that 50% of the SML posts were deleted to ensure the final number of posts to review was manageable. These findings are consistent with a previous study comparing the time and cost burden of qualitative CE interviews and SML in examining patient-reported condition symptoms and impacts [12].

Stakeholder Acceptance

Despite being less costly and quicker to run, SML and literature reviews are generally only accepted by researchers and regulators in conjunction with evidence from qualitative CE interviews, likely due primarily to the greater depth and stronger credibility of the data, as described here. The FDA positions qualitative CE interviews as a primary means of gathering patient data; however, they also require a literature review as an initial step for development of a hypothesised conceptual framework for a PRO [38]. SML reviews are recommended as a method to gather supplemental information to complement other qualitative methods [2]. Furthermore, the FDA’s PFDD guidance provides significantly more information related to qualitative CE interviews than SML or literature reviews [1, 2].

There are also other factors that might be driving the greater stakeholder acceptance of qualitative CE interview studies compared to the other methods. Qualitative CE interviews are conducted by trained and skilled interviewers who have the opportunity to elicit rich and nuanced data. In contrast, the data available in SML and literature review studies are limited to brief quotes (particularly in the case of SML), which are comparatively static because of the absence of study-specific probing. Qualitative CE interview data may also be viewed as more robust than SML data, as participant characteristics and diagnoses can be verified and diversity in the sample can be ensured. Furthermore, qualitative CE interviews offer a higher level of data protection through the development of protocols, opportunity to obtain written informed consent, and oversight by an ethical review board before research even commences with patients. For literature reviews, in most cases the primary data collection will have been performed to similar high standards, although, depending on the journal that is not guaranteed. However, although data in SML were anonymised to protect patient privacy, seeking informed consent is not possible since it is not feasible or practical to contact participants. Nevertheless, as SML studies are relatively inexpensive and quick and easy to implement compared to qualitative CE interviews, they are arguably still valuable, perhaps in most cases to perform early on in a research programme, to provide early insights that can be used to help inform the design of the interview guide for in-depth qualitative interviews. They may also tap into more honest and elicit conversations, shared in a real-world setting with a more diverse cross-section of the population.

Limitations

The study has a number of limitations. First, findings may differ in future research comparing qualitative methods depending on the amount of qualitative literature or social media discussion for a specific condition. There is typically little available published research regarding the patient experience of a rare condition; however, in a well-researched condition such as arthritis or asthma, a literature review will likely provide greater insight into the important concepts. Similarly, some conditions are more commonly discussed in social media than others.

Second, findings may differ depending on the symptoms of a condition. A previous qualitative methods comparison study generating ankylosing spondylitis insights found that SML identified more concepts than qualitative CE interviews. The researchers hypothesised that the anonymity of posting online may have facilitated the discussion of sensitive or stigmatised symptoms/concepts (e.g. erectile dysfunction and skin symptoms) and therefore a greater breadth of data [12]. It is possible that symptoms and impacts of presbyopia are not perceived as sensitive by individuals and therefore were discussed freely in the interview setting in the current study. As such, the comparative success of SML and qualitative interviewing might depend on the extent to which symptoms/domains are stigmatised or embarrassing for individuals.

Third, the population of interest might affect the data gathered in certain methods. For the SML study, for example, it is possible the sample was not representative of the whole population of interest, particularly as the prevalence of presbyopia increases with age, while internet usage decreases with age [39]. Furthermore, the SML study only found a small proportion of posts about quality of life from the total number of posts identified during the search. As presbyopia increases in severity with age, individuals whose quality of life is most affected by their presbyopia may be of an age where social media use is less common [28]. Studies in conditions with younger populations of interest may therefore uncover more representative data via SML.

Finally, from a study design perspective, the qualitative CE interviews conducted in this study included a large sample of individuals with presbyopia (n = 50) across three countries. While conceptual saturation might have been achieved with a smaller sample size, the larger sample size was necessary to provide cross-cultural insights. It must be noted that the qualitative CE interviews were based on a sample restricted by eligibility criteria, whereas the literature review and SML studies are likely to have included individuals with a broader range of characteristics. Although no major bias or differences were observed between the concepts identified, it is possible that the concept identified in the qualitative CE interviews may not be transferable to the wider population, for example, non-phakic presbyopia, in other non-western geographical locations. Equally, although SML posts may be obtained from a more geographically diverse sample, it is possible that the translation of posts from different languages using Google Translate may have altered or obscured the true meaning of the posts.

Recommendations

The chosen qualitative research methods should be considered in line with the requirements of the research project. For example, qualitative CE interviews are better suited than SML and literature reviews when participant characteristics and diagnoses are required. A qualitative interview approach might also be more appropriate than SML and literature reviews in under-researched conditions and where breadth and depth of data are of paramount importance. Conversely, SML may be more suitable than qualitative CE interviews in gathering insights about conditions with potentially sensitive symptoms. Furthermore, a particular advantage of SML is that it can allow data to be gathered from a far larger and representative sample that is geographically dispersed (i.e. from a wide range of countries, locations)—something not easily achieved for in-depth qualitative research. Purely conducting a literature review without supplementing with prospective qualitative CE interview research can be appropriate for well-researched conditions or if only a summary of the available data is required to answer the research question rather than detailed descriptions.

One limitation of articles published in peer-reviewed journals is that word count limitations mean that limited space is available for providing many detailed participant quotes. However, it is notable that an increasing number of journals allow authors to include supplementary files in support of a qualitative research article. Such files allow for greater detail and depth of qualitative data to be made available while keeping the main manuscript body a manageable length. If researchers were to use such supplementary files to provide access to detailed qualitative datasets that include tables of all supportive quotes for concepts identified, it would substantially improve the value and insight that could be obtained from qualitative articles in the literature. The findings of this study suggest that such depth of data is still rarely provided (which is also the anecdotal experience of these authors). We encourage journals to make it a requirement for qualitative researchers to provide more ready access to qualitative datasets in this way. This would reduce research wastage and increase the value that can be gained from a given literature review study.

The value of different qualitative research methods should also be evaluated in terms of cost and time considerations. Where there are greater time and budgetary constraints, literature reviews and SML can be prioritised. Such methods also present a reduced burden to the research team and participants and may also provide early identification of relevant concepts that could then be explored further through interviews, creating a more efficient interview process.

Overall, the ideal is to collect data from as many sources as possible since agreement/convergence across data sources provides a level of validation and reassurance that nothing important has been missed. However, the feasibility of this is again dependent on time, budget, and resources. Furthermore, stakeholder preferences should be considered. Evidence from in-depth qualitative CE interview research (whether published or collected prospectively) remains most credible and valued by most stakeholders, particularly regulators. It is worth exploring before research commences to confirm that your research method will be accepted by the relevant stakeholders.

Conclusions

Notwithstanding the time and budgetary considerations, prospective, in-depth qualitative research through CE interviews with participants remains the gold standard for obtaining in-depth understanding of health concepts such as symptoms and functional impacts. However, literature reviews and SML can supplement such data, providing greater geographical diversity and confidence in results. As evidence regarding the comparative value of qualitative methods grows, and if the depth of data provided in supplementary files to published articles can be increased, perhaps reduced reliance on prospective qualitative research will come with time.