Canadian Journal of Anesthesia/Journal canadien d'anesthésie

, 56:71

Posing the research question: not so simple

  • Lehana Thabane
  • Tara Thomas
  • Chenglin Ye
  • James Paul
Special Article

DOI: 10.1007/s12630-008-9007-4

Cite this article as:
Thabane, L., Thomas, T., Ye, C. et al. Can J Anesth/J Can Anesth (2009) 56: 71. doi:10.1007/s12630-008-9007-4

Abstract

Purpose

The success of any research process relies, in part, on how well investigators are able to translate a clinical problem into a research question—a task that is not so simple for novice investigators. The PICOT approach requires that the framing of the research question specify the target Population, the Intervention of interest, the Comparator intervention, key Outcomes, and the Time frame over which the outcomes are assessed. This paper describes the use of the PICOT structure in framing research questions and examines PICOT criteria as applied to the anesthesia literature. We also provide a roadmap for applying the PICOT format in identifying and framing clear research questions.

Methods

In addition to searching MEDLINE for the literature on framing research questions, we performed a systematic review of articles published in four key anesthesia journals in 2006, including Anesthesiology, Anesthesia & Analgesia, the British Journal of Anaesthesia, and the Canadian Journal of Anesthesia.

Results

Three hundred thirteen articles (n = 313) were included in this review, with the following distribution by study design: 139 (44%) randomized controlled trials, 129 (41%) cohort studies, and 45 (15%) case-controlled, cross-sectional studies or systematic reviews. Overall, 96% (95% confidence interval: 91,100) of articles did not apply the PICOT approach in reporting the research question.

Conclusions

The PICOT approach may be helpful in defining and clearly stating the research question. It remains to be determined whether or not compliance with the PICOT style, or any other format for framing research questions, is associated with a higher quality of research reporting.

Poser la question de recherche: pas si simple!

Résumé

Objectif

La réussite de tout processus de recherche s’appuie en partie sur la capacité des chercheurs à traduire un problème clinique en question de recherche, une tâche ardue pour les chercheurs débutants. L’approche PICOT exige que la formulation de la question de recherche spécifie la Population cible, l’Intervention à l’étude, l’intervention de Comparaison, les devenirs clés (Outcomes) et un cadre Temporel au cours duquel les devenirs sont évalués. Cet article décrit l’utilisation de la structure PICOT dans la formulation des questions de recherche et examine les critères PICOT lorsqu’ils sont appliqués à la littérature en anesthésie. Nous fournissons également une feuille de route visant à l’application du format PICOT pour identifier et formuler des questions de recherche claires.

Méthode

En plus de recherches sur MEDLINE pour trouver la littérature touchant à la formulation de questions de recherche, nous avons effectué une révision systématique des articles publiés dans quatre revues clés d’anesthésie en 2006, soit Anesthesiology, Anesthesia & Analgesia, le British Journal of Anaesthesia, et le Journal canadien d’anesthésie.

Résultats

Trois cent treize articles (n = 313) ont été inclus dans cette révision, lesquels étaient distribués selon le devis de l’étude : 139 (44 %) études randomisées contrôlées, 129 (41 %) études de cohorte, et 45 (15 %) études transversales cas-témoin ou revues systématiques. Au total, 96% (intervalle de confiance 95 %: 91,100) des articles n’utilisaient pas l’approche PICOT dans la détermination de la question.

Conclusion

L’approche PICOT peut être utile pour définir et formuler clairement la question de recherche. Toutefois, il reste à déterminer si le fait de suivre le style PICOT – ou tout autre format de formulation des questions de recherche – est associé ou non à une meilleure qualité dans la présentation des recherches.

Research is defined as any systematic activity designed to contribute to generalizeable knowledge (expressed as theories, principles, or statements about relationships).1 Every clinical research project starts with a clinical problem that may arise when what is currently done in practice does not seem to achieve the desired outcomes. To address the problem, researchers examine possible options, with the aim of investigating whether the options can perform better than the current practice. In some cases, the problem can involve trying to understand why undesirable outcomes occur, that is, identifying factors that can explain why such outcomes occur. This paper aims to provide a roadmap for identifying and framing clear research questions.

The success of any research process relies, in part, on how well investigators are able to turn a clinical problem into a research question—something that is not so simple for novice investigators. Getting it right is key, because the research question is the number one driver that “determines the research architecture, strategy, and methodology.”2 In other words, getting the question right increases the likelihood of finding a solution to the problem,3 i.e., it is a formula for successful search for answers.4 A clearly defined question can also enhance the clarity of the thought process in developing the protocol, informing the design, and guiding analysis decisions, including ensuring publication.5 Therefore, it is imperative that, if any energy or resources are to be spent doing research, they should first be spent on getting the research question right. To help researchers judge their success in moulding their clinical problem into a research question, it is crucial that they know what constitutes a good research question. In general, it is always best to focus on a single primary research question. This is the one that drives the design. Attempting to answer several research questions can also be a good research strategy, because it can make good use of the resources. However, such additional questions can best be considered as secondary questions or, with appropriate adjustment for multiple testing, within a well defined hierarchy of primary questions.

How to identify a research question

In general, a good research question should be appropriate, meaningful, and purposeful.5 Table 1 provides a summary of the FINER criteria,6 often used to define the desirable properties of a good research question, together with some suggestions on how to achieve each attribute. The FINER criteria state that a research question must be feasible, interesting, novel, ethical, and relevant.
Table 1

Strategies for assessing the FINER criteriaa

Criterion

Strategies for achieving success

Is the research question feasible?

• Do a pilot to assess feasibility.

• Consider modifying inclusion criteria.

• Get collaborators, learn the skills, consult other experts.

• Use less costly designs (e.g., paired designed, cross-over designs).

• Choose common outcomes.

• Use continuous versus binary outcomes.

Is it interesting?

Check if it:

• Interests you as a researcher,

• Interests your collaborators,

• Interests the stakeholders.

Is it novel?

• Be familiar with the literature.

• Get guidance from experienced researchers.

• Get a mentor.

It is ethical?

• Be familiar with research ethics guidelines. Examples include:

    ✓ The Declaration of Helsinki28

    ✓ Tri-council Policy Statement (TCPS)29

    ✓ Good Clinical Practice (GCP)30

• Get Research Ethics approval prior to conducting research.

It is relevant—to scientific knowledge, policy, or future directions?

• Be familiar and up-to-date with the literature.

• Be familiar with policy debates

• Get guidance from experienced researchers or mentors.

• Search information about the national and global burden of disease.

aRefers to reference 6

Knowing the desirable attributes of a good question and understanding how to achieve them can facilitate identifying the clinical problems that are worth the expenditure of intellectual energy and resources. This may be easy for experienced researchers, but novice or new researchers would need guidance. Here we mention some of the common strategies used to identify clinical research problems.69
  1. (1)

    Relying on one’s own clinical experience or practice;

     
  2. (2)

    discussing issues with other researchers at professional meetings;

     
  3. (3)

    following developments in the literature and identifying gaps in the literature;

     
  4. (4)

    discussing issues with a mentor;

     
  5. (5)

    being alert to new ideas and technological advances;

     
  6. (6)

    brainstorming with friends and colleagues;

     
  7. (7)

    keeping the imagination roaming;

     
  8. (8)

    searching information about the national and global burden of disease; and

     
  9. (9)

    using focus groups.

     

The risks of a poorly formulated research question

As noted above, a clear and focused research question will help to determine research collaborations and set the direction for the selection of appropriate study design and the most appropriate methods of statistical analysis and sample size determination. A poorly formulated research question poses several risks or threats. First, researchers are likely to adopt an erroneous design. Second, it can create confusion10 and hinder the thought process, including impede the development of a clear protocol.5 Third, it can jeopardize publication efforts.5 Fourth, it is difficult for the reader to determine whether the answer is relevant when the question is not clear.11 Fifth, an unclear question can make it difficult to interpret the results of the study. Sixth, an unclear research question makes it difficult to determine whether or not a study fulfills inclusion criteria for systematic review and meta-analysis.12,13 This would, in turn, create challenges in determining whether it is necessary to collect more information by running additional studies to answer the question. Lastly, when the research question is not clearly stated, people reading the study may fail to understand the objective of the study, and this could negatively impact the likelihood of the study being cited by other researchers.

How to frame the research question

The PICOT approach

What is the research question? Anyone reading the report should be able to answer this first question.11,14 The general principle is that the title should reflect the research question; if it does not, the abstract should, followed by the text. The question should be framed in such a way that it is easily understood and can be rephrased in the reader’s own words.11,14 First introduced in 1995,15 the PICO format, later expanded to PICOT9 (Table 2), is now a widely recommended strategy for framing research questions. Since its inception, several authors have advocated its use in framing research questions in different areas, including nursing,16 palliative medicine,5 transfusion medicine,17 occupational health,18 clinical epidemiology,9 systematic reviews,12,13,19 and information searching.2022
Table 2

PICOT formata

Letter

Stands for

Meaning

P

Patient population of interest

What patient population or problem are you trying to address?

I

Intervention or issue of interest

What will you do for the patient or problem?

C

Comparison with another intervention/issue

What are the alternatives to your chosen intervention?

O

Outcome of interest

What will be improved for the patient or problem?

T

Time frame

At what time following the intervention do you decide it is doing more good than harm?

aRefers to reference 9

The PICOT approach requires that the framing of the research question specify the target Population, the Intervention of interest, the Comparator intervention, key Outcomes, and the Time frame over which the outcomes are assessed. The population can be described by certain eligibility criteria, qualifying disease condition of interest, or geographic location. The intervention is a controlled maneuver or exposure that can be manipulated and is often a new, experimental, or innovative approach. The primary goal may be to compare the intervention with an alternative standard (control), placebo (no intervention), or approach. The effect is evaluated by comparing outcomes in the underlying intervention groups. Note; the allocation of patients into intervention groups need not be random, although random allocation is generally considered the best approach in generating evidence.23 It is also important to state the key outcomes, which may be either clinical or process outcomes. Table 3 provides a summary of the desirable properties of key outcomes in interventional research. The assessment of outcomes is completed over a specified time frame that is chosen (based on clinical considerations) to create the optimal difference between the intervention and the control groups (i.e., intervention effect).
Table 3

Desired properties of primary outcomes

A good primary outcome/endpoint should:

• be appropriate (should be fitting for the objectives of the study);

• be objective (i.e., should require less subjective judgment to measure);

• be valid (i.e., should measure that which is intended);

• be reproducible/precise/reliable (i.e., should easily be reproduced in different times/settings);

• be clinically available (i.e., should be available as part of clinical care);

• be easily quantifiable (i.e., should be easily measured);

• be efficient (i.e., should be affordable to measure in terms of time and cost);

• be sensitive (i.e., should correctly specify presence of disease or condition of interest);

• be specific (i.e., should correctly specify absence of disease or condition of interest);

• be responsive (i.e., should be sensitive to changes in treatment). That is, it should:

    – rapidly reflect the response to treatment; and

    – accurately reflect the response to treatment; and

• be straightforward (i.e., should allow easy interpretation of results).

It is worth noting that the PICOT format is generally applicable to comparative studies or studies of association between exposure and outcome(s). Other useful approaches exist in the literature.22,24 We focused on the PICOT format, because it is a routinely advocated approach in framing research questions in evidence-based medicine.25 The use of PICOT has also been shown to be associated with improvements in search results for clinical information in PubMed.20

A cross-section review of articles published in four leading anesthesia journals in 2006

We performed a systematic review of papers published in 2006 in four leading anesthesia journals, including Anesthesiology (Anes), Anesthesia & Analgesia (A&A), the British Journal of Anaesthesia (BJA), and the Canadian Journal of Anesthesia (CJA). We applied stratified random sampling (with journal as the stratum) to select up to 80 articles for those journals that had published more than 80 eligible papers. The aim was to determine the extent of variance from the PICOT approach used by the various authors in framing the research questions, objectives, or hypotheses in their papers. We selected all papers that included results of original research, systematic reviews of comparative studies, and studies of association between exposure and outcome(s). The different types of research designs we selected included randomized controlled trials, cohort designs, case-control, cross-sectional, and quasi-experimental designs. We excluded commentaries as well as case-reports, because they are primarily descriptive. We focused mainly on the primary or key question of the study to determine whether the question/objective or hypothesis clearly indicated the target population, the intervention, the control, the key outcome, and the timing of the assessment. We first reviewed the Title, then the Abstract, followed by the Introduction or Methods sections. Two reviewers [C.Y. and T.T.] abstracted data from all included papers to determine whether PICOT was used in framing the primary or key research question. Agreement between reviewers was evaluated using kappa statistics. The results varied from 0.73 to 1, indicating good to perfect agreement. If consensus could not be reached, disagreements were resolved through consensus discussion with a third reviewer (L.T.).

Three hundred and thirteen articles (n = 313) were included in the review, with the following distribution by study design: 139 (44%) randomized controlled trials, 129 (41%) cohort studies, 2 (1%) case-control studies, 25 (8%) cross-sectional studies, and 18 (6%) systematic reviews (Fig. 1). Overall, 96% (95% confidence interval: 91,100) of articles were at variance with the PICOT approach in framing the research question, the objective, or the hypothesis—that is, up to 100% of the papers did not have a well-framed research question/objective, as judged by the PICOT criteria. Corresponding estimates of the percent of papers that failed to adopt each PICOT element were as follows: Population: 39% (95% confidence interval: 27%, 50%); Intervention/exposure: 12% (5%, 20%); Comparator: 54% (43%, 65%); Outcome: 16% (8%, 25%); and Time frame: 89% (82%, 96%) (Table 4). Thus, not stating the time frame was the key reason for the research question/objective not meeting the PICOT criteria. This was followed by incomplete or non-identification of the comparator. A similar pattern appeared across all four journals.
https://static-content.springer.com/image/art%3A10.1007%2Fs12630-008-9007-4/MediaObjects/12630_2008_9007_Fig1_HTML.gif
Fig. 1

Selection process for articles

Table 4

Percentage of articles published in four anesthesia journals in 2006 that do not state PICOT elements (95% confidence interval)

PICOT elements

CJA (n = 73)

Anes (n = 80)

A&A (n = 80)

BJA (n = 80)

All (n = 313)

P = population

51 (39, 62)

41 (30, 52)

30 (20, 40)

34 (23, 44)

39 (27, 50)

I = intervention

21 (11, 30)

15 (7, 23)

4 (0, 8)

10 (3, 17)

12 (5, 20)

C = comparator

56 (45, 68)

64 (53, 74)

43 (32, 53)

54 (43, 65)

54 (43, 65)

O = outcome

32 (21, 42)

5 (0, 10)

14 (6, 21)

16 (8, 24)

16 (8, 25)

T = time frame

93 (87, 99)

89 (82, 96)

85 (77, 93)

89 (82, 96)

89 (82, 96)

A = at least one element

100

96 (92, 100)

90 (83, 97)

96 (92, 100)

96 (91, 100)

CJA Canadian Journal of Anesthesia; Anes Anesthesiology; A&A Anesthesia & Analgesia; BJA British Journal of Anaesthesia

We acknowledge that this type of review has several limitations—one, in particular, is the use of published research articles instead of drawing on research or grant proposals. In reality, most peer-reviewed journals do not require using the PICOT format.

Resources

Table 5 provides a list of several valuable resources that researchers can explore for more information on determining and framing research questions.
Table 5

Resources

Session topic

Key references

How to identify research problems or ideas

• Articles

    a. Buelow JM. Identifying a Researchable Problem. Clin Nurs Specialist 2006;20(4):175

    b. Chulay M. Good Research Ideas for Clinicians. AACN Adv Crit Care 2006;17(3):253–265

• Text books

    a. Haynes BR, Sackett DL, Guyatt GH, Tugwell P. Clinical Epidemiology: How to do Clinical Practice Research. Third Edition. Lippincott Williams & Wilkins, Philadelphia, PA 2006

    b. Hulley SB, Cummungs SR, Browner WS, Grady D, Newman TB. Designing Clinical Research, 3rd Edition. Lippincott Williams & Wilkins, Philadelphia, PA 2007

    c. DePoy E, Gitlin LN. Introduction to Research: Understanding and Applying Multiple Strategies. 3rd Edition. Elsevier Mosby: Philadelphia, PA 2005

How to frame questions using PICOT (see Table 2)

• Articles

    a. Heddle NM. The research question. Transfusion 2007 Jan;47:15

    b. Johnston L, Fineout-Overhold E. Teaching EBP: “Getting from Zero to One.” Moving from Recognizing and Admitting Uncertainties to Asking Searchable, Answerable Questions. Worldviews on Evidence-Based Nursing 2005;2(2):98

    c. Stone PW. Popping the (PICO) question in research and evidence-based practice. Appl Nurs Res 2002 Aug;15(3):197–198

    d. McKibbon KA, Marks S. Posing Clinical Questions: Framing the Question for Scientific Inquiry. AACN Clinical Issues 2001 Nov;12(4):477–481

    e. Geddes J. Asking structured and focused clinical questions: essential first step of evidence-based practice. Evidence-Based Mental Health 1999;2:35–36

    f. Counsell C. Formulating Questions and Locating Primary Studies for Inclusion in Systematic Reviews. Ann Inter Med. 1997 Sep;127(5):380–387

    g. Durbin Jr CG. How to Come Up With a Good Research Question: Framing the Hypothesis. Respiratory Care 2004 Oct;49(10):1195

• Text books

    a. Haynes BR, Sackett DL, Guyatt GH, Tugwell P. Clinical Epidemiology: How to do Clinical Practice Research, 3rd Edition. Lippincott Williams & Wilkins, Philadelphia, PA 2006

    b. Hulley SB, Cummungs SR, Browner WS, Grady D, Newman TB. Designing Clinical Research, 3rd Edition. Lippincott Williams & Wilkins, Philadelphia, PA 2007

    c. Systematic Reviews: Synthesis of Best Evidence for Health Care Decisions (eds. Cynthia Mulrow, Deborah Cook). American College of Physicians: Philadelphia, PA 1998

• Internet

    a. http://library.wcsu.edu/web/assistance/research/nursing/tutorial/c_picot/: Western Connecticut University Library resources

    b. http://consortiumlibrary.org/hsis/researchaids/handouts/ebp.php: Health Sciences Information Services Consortium library resources

Framing research questions using other frameworks

PICOS: (stands for Patient population or Problem, (22)Intervention (treatment/test), Comparison (group or treatment), Outcomes, and Setting or study type)

    a. http://consortiumlibrary.org/hsis/researchaids/handouts/ebp.php: Health Sciences Information Services Consortium library resources

PESICO (stands for Person (or problem), Environments, Stakeholders, Intervention, Comparison and Outcome)

    a. Schlosser RW, Koul R, Costello J. Asking well-built questions for evidence-based practice in augmentative and alternative communication. J Commu Dis 2007 Jun;40(3):225–238

Identifying research questions in Anesthesia

• Articles

    a. Pronovost PJ, Berenholtz SM, Dorman T, Merritt WT, Martinez EA, Guyatt GH. Evidence-based medicine in anesthesiology. Anesth Analg 2001 Mar;92(3):787–94

Application of the PICOT approach: examples referring to the literature

Following the approach by Heddle,17 Fig. 2 provides a roadmap with an illustrative example showing how to apply the principles discussed above while using the PICOT format in phrasing a research question. We offer the following examples from the anesthesia literature to demonstrate how to apply the PICOT approach in posing a research question. First, we pose the question/objective as stated in the publication. Second, we identify the missing elements, according to the PICOT criteria in the framing of the question. Third, we suggest an alternative way of framing the question using the PICOT model.
https://static-content.springer.com/image/art%3A10.1007%2Fs12630-008-9007-4/MediaObjects/12630_2008_9007_Fig2_HTML.gif
Fig. 2

Roadmap for determining and framing researchable questions

Example one: In a recently published study examining prediction of massive blood transfusion in cardiac surgery, the objective of this three-parallel group randomized controlled trial was “to determine if recovery from postoperative anemia is accelerated in patients randomized to receive early postoperative intravenous iron therapy alone or in combination with recombinant erythropoietin.”26 According to the PICOT criteria, the missing elements in this objective include: (1) the target population; (2) the outcome (i.e., how anemia is measured); and (3) the timing of the assessment/measurement. Restating the research question using PICOT would result in the following: “In patients without preoperative anemia undergoing cardiac or orthopedic surgery, does treatment with: (1) intravenous iron alone; or (2) intravenous iron with recombinant erythropoietin; compared with (3) placebo, administered a day after surgery, increase hemoglobin concentration 7 days after surgery?”

Example two: Using a cohort design, the purpose or objective of a recently published study, identifying the independent risk factors for fentanyl-induced cough, was “to determine how the probability of fentanyl-induced cough is affected by patient characteristics and/or anesthetic technique.”27 According to the PICOT criteria, the missing elements in this objective include specification of: (1) the target population; (2) the comparator(s); (3) the outcome (i.e., how fentanyl-induced cough is measured); and (4) the time frame. Restating the objective using PICOT criteria would generate the following: “In patients undergoing elective surgery under general anesthesia accompanied by intravenous fentanyl, are the type of pre-anesthesic medications (benzodiazepines, clonidine, hydroxyzine) and patient characteristics associated with increased risk of fentanyl-induced cough measured by development of bronchial asthma one minute following the administration of intravenous fentanyl?

Both of these revised research questions provide clear and concise summaries of the criteria involved in the studies, in terms of the population, intervention, comparator intervention, outcomes, and time frame. This format leaves nothing to the reader’s imagination at the beginning of the manuscript; consequently, the reader is better guided as to what to anticipate, with regard to methods and results. Furthermore, by choosing the correct methodology and analysis approach at the start of the project, researchers may be more likely to successfully answer their research question.

Conclusions

In conclusion, it is important, at the outset, to spend the energy and resources necessary to establish a clearly defined research objective/question prior to study design. The research question should guide the research design, methods, and analytic strategies, including the personnel with whom to collaborate. Although it is optimal to focus on a single primary research question, it is equally useful to have a clearly defined hierarchy of research questions, with appropriate adjustment of multiplicity for multiple primary or secondary questions. Conceiving the research question requires scholarship (reading and critically interpreting the literature), research experience (or guidance from a mentor), and awareness of societal/professional trends. We recommend consideration be given to adopting the PICOT style. The framework of the research question should specify the target Population, the Intervention, the Comparator intervention, and the main Outcomes, including the Timing of the assessment of outcomes. The research question should be meaningful, appropriate, and purposeful. It should satisfy the FINER criteria (Feasible, Interesting, Novel, Ethical,2830 and Relevant). It remains to be determined whether or not compliance with the PICOT style, or any other format for framing research questions, is associated with a higher quality of research reporting.

Conflicts of interest

None declared.

Copyright information

© Canadian Anesthesiologists’ Society 2008

Authors and Affiliations

  • Lehana Thabane
    • 1
    • 2
  • Tara Thomas
    • 3
  • Chenglin Ye
    • 4
  • James Paul
    • 3
  1. 1.Department of Clinical Epidemiology and BiostatisticsMcMaster UniversityHamiltonCanada
  2. 2.Biostatistics Unit, 3 Martha Wing, St Joseph’s Healthcare HamiltonHamiltonCanada
  3. 3.Department of AnesthesiaMcMaster UniversityHamiltonCanada
  4. 4.Department of Mathematics and StatisticsMcMaster UniversityHamiltonCanada