The Patient: Patient-Centered Outcomes Research

, Volume 2, Issue 3, pp 179–190 | Cite as

Public Involvement in Setting a National Research Agenda

A Mixed Methods Evaluation
  • Sandy Oliver
  • David G. Armes
  • Gill Gyte
Original Research Article


Background: A growing body of literature supports the inclusion of patients, other service users, and the wider public in guiding health technology assessment, particularly in relation to interventions and outcomes for evaluative studies.

Objective: To describe the input and influence of public involvement in setting the agenda for a national research program.

Methods: The data source was the commissioned research of the UK National Health Service Health Technology Assessment (HTA) program, 1999–2004. The study consisted of a mixed methods evaluation employing document analyses, key informant interviews, and structured non-participant observations. Routine management records of the HTA program were examined for public influence on research topics. The nature and influence of contributions from the public were compared with those of other experts. Structured observations of advisory panel meetings investigated how discussion and decisions related to patient and public perspectives and how panel members responded to public input to the program. Semi-structured interviews gathered the perceptions of staff and advisory panel members.

Results: The public provided unique contributions both as external experts and as panel members. The value and influence of many of these contributions were acknowledged by staff and panel members. Input from external public experts was least where recruitment was passive (through a website) and where contributions were required in a research question format that may have been unfamiliar to non-researchers. However, public influence at this stage was at least of the same order as that of professional suggestions. Input was most where recruitment effort was greater, where contributions could be made in an open format, and where the responsibility for integrating these into a research question format lay with research program staff. Public experts contributing at this stage often influenced research plans. Their contributions resulted in some important changes, including making patient and carer perspectives explicit, changing the focus of the research, adding new outcomes, refuting the need for the planned research, providing up-to-date prevalence data, and providing plain English background text. At their best, public members of advisory panels were seen as providing useful comment and encouraging greater sensitivity to patient perspectives among other panel members. At their worst, they were seen as lobbying for particular patient groups.

Conclusions: Public involvement has influenced decisions about research commissioned by the HTA program with only relatively minor changes to the procedures and resources for managing the program required. This results in outcomes research that incorporates patient and public preferences and values, and that is freely available for evidence-informed health services.


National Health Service Service User Health Technology Assessment Panel Member Public Involvement 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This evaluation was funded by the National Coordinating Centre for Health Technology Assessment (NCCHTA). Public involvement was initiated by Ruairidh Milne. Subsequently, the work was guided by the Public Involvement Steering Group of the NCCHTA, which included, at various times, Gillian Fletcher, John Gabbay, Lynn Kerridge, Jean Mossman, Joy Stokes, and Pamela Young. The authors are very grateful to staff at the NCCHTA for their patience and care in allowing access to documents and for generously giving their time to explain working procedures to the evaluation team. The authors appreciate the time given by NCCHTA staff and members of advisory panels and the Commissioning Board to enable them to be interviewed about their work and perceptions of public involvement in the Health Technology Assessment program. The authors have no conflicts of interest that are directly relevant to the content of this study.


  1. 1.
    Abelson J, Giacomini M, Lehoux P, et al. Bringing ‘the public’ into health technology assessment and coverage policy decisions: from principles to practice. Health Policy 2007; 82(1): 37–50PubMedCrossRefGoogle Scholar
  2. 2.
    Bridges J, Jones C. Patient based health technology assessment: a vision of what might one day be possible. Int J Technol Assess Health Care 2007; 23(1): 30–5PubMedCrossRefGoogle Scholar
  3. 3.
    Hanley B, Truesdale A, King A, et al. Involving consumers in designing, conducting, and interpreting randomised controlled trials: questionnaire survey. BMJ 2001; 322: 519–23PubMedCrossRefGoogle Scholar
  4. 4.
    Sakala C, Gyte G, Henderson S, et al. Consumer-professional partnership to improve research: experience of the Cochrane Collaboration’s Pregnancy and Childbirth Group. Birth 2001; 28: 133–7PubMedCrossRefGoogle Scholar
  5. 5.
    Oliver S, Oakley L, Lumley J, et al. Smoking cessation programmes in pregnancy: systematically addressing development, implementation, women’s concerns and effectiveness. Health Educ J 2001; 60(4): 362–70CrossRefGoogle Scholar
  6. 6.
    Rees R, Oliver S. An example from health promotion. In: Coren E, editor. Collection of examples of service user and carer participation in systematic reviews. London: Social Care Institute for Excellence, 2007: 15–34 [online]. Available from URL: [Accessed 2007 Aug 08]Google Scholar
  7. 7.
    Oliver S, Gray J. A bibliography of research reports about patients’, clinicians’ and researchers’ priorities for new research. London: James Lind Alliance, 2006 DecGoogle Scholar
  8. 8.
    Research and Development Directorate, Department of Health. Best research for best health: a new national health research strategy. London: Department of Health, 2006Google Scholar
  9. 9.
    Resnik DB. Setting biomedical research priorities: justice, science, and public participation. Kennedy Inst Ethics J 2001; 11(2): 181–204PubMedCrossRefGoogle Scholar
  10. 10.
    Consumers’ Health Forum of Australia. A model framework for consumer and community participation in health and medical research. Canberra (ACT): National Health and Medical Research Council (NHMRC), 2004 [online]. Available from URL: [Accessed 2009 Jul 10]Google Scholar
  11. 11.
    National Institutes of Health Director’s Council of Public Representatives (COPR). Enhancing public input and transparency in the National Institutes of Health research priority-setting process [online]. Available from URL: [Accessed 2007 May 23]
  12. 12.
    O’Donnell M, Entwistle V. Consumer involvement in decisions about what health-related research is funded. Health Policy 2004; 70(3): 281–90PubMedCrossRefGoogle Scholar
  13. 13.
    Oliver S, Clarke-Jones L, Rees R, et al. Involving consumers in research and development agenda setting for the NHS: developing an evidence-based approach. Health Technol Assess 2004; 8(15): 1–148PubMedGoogle Scholar
  14. 14.
    National Institute for Health Research. NIHR Health Technology Assessment Programme. Published research: full listing [online]. Available from URL: [Accessed 2009 Jul 10]
  15. 15.
    Oliver S, Milne R, Bradburn J, et al. Investigating consumer perspectives on evaluating health technologies. Evaluation 2001; 7: 468–86CrossRefGoogle Scholar
  16. 16.
    Oliver S, Milne R, Bradburn J, et al. Involving consumers in a needs-led research programme: a pilot project. Health Expect 2001; 4: 18–28PubMedCrossRefGoogle Scholar
  17. 17.
    Royle J, Oliver S. Consumer involvement in the Health Technology Assessment programme. Int J Technol Assess Health Care 2004; 20(4): 493–7PubMedCrossRefGoogle Scholar
  18. 18.
    Hanley B, Bradburn J, Barnes M, et al. Involving the public in NHS, public health, and social care research: briefing notes for researchers. Eastleigh: INVOLVE Support Unit, 2004 [online]. Available from URL: [Accessed 2008 Aug 08]Google Scholar
  19. 19.
    Hammersley M, Atkinson P. Ethnography: principles in practice. London: Routledge, 1995Google Scholar
  20. 20.
    Emerson R. Observational field work. Annu Rev Sociol 1981; 7: 351–78CrossRefGoogle Scholar
  21. 21.
    Burgess RG. In the field: an introduction to field research. London: Unwin Hyman, 1984CrossRefGoogle Scholar
  22. 22.
    Finnegan R. Using documents. In: Sapsford R, Jupp V, editors. Data collection and analysis. London: Sage, 1996: 138–51Google Scholar
  23. 23.
    Strauss A, Corbin J. Basics of qualitative research: techniques and procedures for developing grounded theory. Newbury Park: Sage, 1990: 35, 52Google Scholar
  24. 24.
    UK Data Protection Act 1998. Chapter 29, part IV, para 33 (1) [online]. Available from URL: [Accessed 2008 Aug 08]
  25. 25.
    Oliver S, Selai C. Guidelines for consumers interested in peer reviewing: II. Ideas to help consumers peer-review Cochrane protocols and reviews. London: University of London, 2001 [online]. Available from URL: [Accessed 2008 Aug 08]Google Scholar
  26. 26.
    Horey D, Gyte G, Sakala C, et al. What do you mean? Collated consumer feedback on Cochrane reviews and protocols on topics in pregnancy. Cochrane Colloquium Proceedings; 2004 Oct; Ottawa (ON) [online]. Available from URL: [Accessed 2008 Aug 08]
  27. 27.
    Boote J, Telford R, Cooper C. Consumer involvement in health research: a review and research agenda. Health Policy 2002; 61(2): 213–36PubMedCrossRefGoogle Scholar
  28. 28.
    Nilsen ES, Myrhaug HT, Johansen M, et al. Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material. Cochrane Database Syst Rev 2006; (3): CD004563Google Scholar
  29. 29.
    Collins HM, Evans R. The third wave of science studies: studies of expertise and experience. Soc Stud Sci 2002 Apr; 32(2): 235–96CrossRefGoogle Scholar

Copyright information

© Adis Data Information BV 2009

Authors and Affiliations

  1. 1.Social Science Research Unit, Institute of EducationUniversity of LondonLondonUK

Personalised recommendations