Abstract
Background
Eczema (syn. Atopic Eczema or Atopic Dermatitis) is a chronic, relapsing, itchy skin condition which probably results from a combination of genetic and environmental factors. The Global Resource of EczemA Trials (GREAT) is a collection of records of randomised controlled trials (RCTs) for eczema treatment produced from a highly sensitive search of six reference databases. We sought to assess the sensitivity of the GREAT database as a tool to save future researchers repeating extensive bibliographic searches.
Methods
All Cochrane systematic review on treatments for eczema and five non-Cochrane systematic reviews on eczema were identified as a reference set to assess the utility of the GREAT database in identifying randomised controlled trials (RCTs). RCTs included in the systematic reviews were checked for inclusion in the GREAT database by two independent authors. A third author resolved any disagreements.
Results
Five Cochrane and six non-Cochrane systematic reviews containing a total of 105 RCTs of eczema treatments were included. Of these, 95 fitted the inclusion criteria for the GREAT database and 88 were published from 2000 onwards. Of the 88 eligible studies, 92% were found in the GREAT database. Seven trials were not included in the GREAT database - two of these were reported within a review paper and one as an abstract with no trial results.
Conclusions
The sensitivity of the GREAT database for trials from 2000 onwards was high (75/88 trials, 94%). Sensitivity for the period prior to 2000 was less sensitive, due to differences in how the trials were identified prior to this time.
‘Dual’ filtering for new records has recently become part of the GREAT database methodology and should further improve the sensitivity of the database in time. The GREAT database can be considered as a primary source for future systematic reviews including randomised controlled trials of eczema treatments, but searches should be supplemented by checking reference lists for eligible trials, searching trial registries and contacting pharmaceutical companies for unpublished studies.
Similar content being viewed by others
Background
Eczema is a chronic, relapsing, itchy skin condition (syn. Atopic Eczema or Atopic Dermatitis) which is the result of a combination of genetic and environmental factors [1].
Searching for published reports of randomised controlled trials (RCTs) can be time consuming and complex. In order to facilitate the speedy identification of RCTs of eczema treatments, the Global Resource of EczemA Trials database (GREAT) was created. This free, publicly accessible database (www.greatdatabase.org.uk) contains summary details of published randomised controlled trials (RCTs) on the treatment of eczema. Trials are identified using a highly sensitive search of six databases [2]. The aim of the GREAT database is to save research time by: 1) creating only one database to search for trials of eczema treatment; and 2) providing enough pertinent information about each trial to allow systematic reviewers and guideline writers to decide whether the trial fits their inclusion criteria without having to obtain and scrutinise the full papers for all possible trials themselves.
If potential users of the GREAT database are not convinced that it will provide a comprehensive list of all eczema RCTs, then they are far less likely to use it as their main database to search and will conduct their own searches. This will not save research time or help to prevent research wastage as was the original intention.
The objective of this study was to evaluate the sensitivity of the GREAT database by comparing those studies found in systematic reviews with those in the GREAT database. Sensitivity was defined as the number of relevant RCTs identified in the GREAT database, divided by the total number of relevant RCTs in the included systematic reviews.
Methods
All methods described below were undertaken by two authors independently, with any discrepancies being resolved by another author acting as an arbiter.
As there is no obvious ‘gold standard’ resource of trials of eczema treatments, systematic reviews were considered the ‘gold standard’ and the GREAT database was the ‘comparator’.
It was decided, a priori, to restrict the comparison of the GREAT database to all published Cochrane reviews of eczema treatments, and five non-Cochrane reviews of eczema treatments. The latter were those with the most recent search end dates. Reviews also had to have a full list of citations for the included studies and sufficient details of the review’s inclusion criteria to allow a fair comparison with the GREAT database. The validation only compared included RCTs, as the GREAT database only includes RCTs.
Results of this study are presented as being for the period from 2000 onwards and pre-2000 due to differences in the way that the searches were conducted during these periods. RCTs published prior to 2000 were identified from a previously published systematic review of eczema treatments [3], whereas RCTs published from 2000 onwards were identified prospectively as the GREAT database was developed.
Identifying systematic reviews of eczema treatments
The mapping of systematic reviews of atopic eczema project developed by the Centre of Evidence Based Dermatology [4,5] has records of systematic reviews from 2000 onwards (including both Cochrane and non-Cochrane reviews). This map of systematic reviews was mined independently by two authors for all Cochrane systematic reviews and five non-Cochrane reviews involving eczema treatments. Any disagreements about which systematic reviews to include as per the pre-specified inclusion criteria were resolved by the arbiter before the list was finalised.
Identifying included and excluded trials from systematic reviews
Two authors independently scrutinised the included systematic reviews to obtain the list of citations of included RCTs studies (other included studies depending on the nature of the review). Any disagreements between the two authors about the included and excluded studies were resolved by the arbiter. Excluded studies were defined as those described as such in the review text or a table(s). Any trials not in English and not in GREAT had the appropriate data extracted by a researcher with the required language skills, if further information was required.
Identifying trials from the GREAT database
Two authors independently attempted to find all the included and excluded trials identified from the systematic reviews, using the search facility and exploration of the GREAT database via the browse and filter facilities. Where an included or excluded trial was not found in the GREAT database, the reasons for this were explored and recorded.
The reported inclusion criteria for each systematic review were used to identify any additional RCTs in the GREAT database which could have been eligible for inclusion in the review, but were not in the list of included studies of that review. Where an RCT was identified in the GREAT database that was not in the included studies list of one or more of the systematic reviews, the reasons for this were explored, consulting the full trial paper and GREAT if necessary. Any disagreements between the two author’s results were resolved by the independent arbiter before the results were analysed.
Results
Summary of results
A search of the mapping of eczema systematic reviews [4] revealed 59 eczema treatment systematic reviews published since 2000. Of these, six were Cochrane reviews [6-11] and were included in this validation study. Five non-Cochrane systematic reviews [12-16] with the most recent search end dates and which matched all other inclusion criteria were included in the study.
The six Cochrane reviews included 72 randomised controlled trials (RCTs) on eczema treatments and no other types of studies. All of the reviews searched The Cochrane Library, MEDLINE, and Embase; three searched the Allied and Complementary Medicine Database (AMED) [7,9,11]; one searched the Cumulative Index to Nursing and Allied Health Literature (CINAHL) [11]; two searched the Latin American and Caribbean Health Sciences database (LILACS) [7,9]; three searched PsycINFO® [7,9,10]; and two searched the ISI Web of Science [7,9]. Four [6,8,10,11] of the six Cochrane reviews searched various other databases and repositories such as trials databases, the Food and Drug Administration (FDA) website, European Medicines Agency (EMEA) website, and Chinese databases. The Cochrane reviews included in this study covered topical calcineurin inhibitors, antibacterials, dietary interventions, Chinese herbal medicine, psychological and educational interventions and probiotics.
The five non- Cochrane reviews included 33 RCTs on eczema treatments that could be included in this validation study. In addition, 52 other studies included in the non-Cochrane reviews were excluded from the validation study as they were either controlled clinical trials, cohort studies, or RCTs involving other dermatoses.
Four reviews searched the Cochrane Library [12-14,16],, four searched MEDLINE [12,14,15], three searched Embase [12-14]; one searched Current Contents [15], one searched HomInform [15], and two searched PubMed [13,15]. Four [12,14-16] out of five of the reviews undertook manual searching of the reference lists of included trials, reviews and other sources such as textbooks and guidelines. The non-Cochrane reviews included in this study covered homeopathy, azathioprine, topical calcineurin inhibitors, topical corticosteroids.
Assessment of trials published from 2000 onwards in the GREAT database
Trials missing from GREAT but found in systematic reviews
The GREAT database did not contain all the included RCTs for four [6,8,9,14] of the eleven included systematic reviews. There were 13 out of 88 trials in total (see Table1) that were not found in the GREAT database. The reasons for this are discussed below.
Trial missing with reason
One of the trials [17], was included in a Cochrane review but did not fulfil the GREAT inclusion criteria as it assessed changes in bacterial numbers rather than clinical outcomes.
Five trials [18-22] were included in a Cochrane review, but had been identified through a pharmaceutical company trials database [23] and did not contain enough information in the trial records provided to be certain whether these trials appear in the GREAT database or not (see Table2).
Trials missing in error
Seven trials 7/88 (8%) were identified from systematic reviews and fitted the GREAT inclusion criteria, but were not present in the GREAT database (see Table2). Two of these trials [24,25], were additional citations for a trial that was already included in the GREAT database. Of the remaining five trials two were reported only within a review paper [26-28] and one [29] was published as a conference abstract and did not report trial results.
Potential implications of trials missing from GREAT
One Cochrane review from which one of the missing in error trials was included may possibly not have drawn such strong conclusions about the level of problems with adverse event if the trial had not been included. The other two Cochrane reviews and two non-Cochrane reviews from which one of the missing in error trials were included would have been very unlikely to have drawn different conclusions if the trials had not been included.
Trials published from 2000 onwards found in the GREAT database that potentially should have been included in systematic reviews
For one of the included Cochrane reviews on anti-staphylococcal interventions [8], three trials [30-32] were identified using the GREAT database that were not included in the review. We believe from the trial reports and the reported inclusion criteria of the review, that these three trials should have been included in this review. It is possible that the review authors managed to gather additional information about these trials which resulted in them being excluded; however, as the trials are not listed as excluded studies when they would have been so close to inclusion, this is less likely.
Assessment of pre 2000 trials missing from the GREAT database
As trials published before 2000 were added to GREAT directly from a systematic review [3], a strict validation comparison was not performed for trials before 2000.
Four trials [33-36] published before 2000 and included in one Cochrane review [8], were not present in the GREAT database. All four trials were returned in the search strategy for the Cochrane systematic review [8] and so they were not found exclusively through hand searching.
It is most likely that the four trials [33-36] were not selected for inclusion in the overarching systematic review that provided the source for pre-2000 RCTs due to errors in the filtering process; or possibly the trials were excluded as they did not match the systematic review [3] inclusion criteria.
Discussion
The GREAT database is free to use and has been specifically designed to be quick and easy to access by all potential users, including guideline writers, patient information developers, methodologists, healthcare practitioners, and patients. The GREAT database offers much more than a standard bibliographic database; which provides citations and abstracts of articles only. Researchers identifying references from other databases would be well advised to cross-check them in the GREAT database, in order to take advantage of the additional data extraction and analysis available there.
In this validation study, the GREAT database was found to include 92% of the RCTs identified in published systematic reviews, and was also able to identify an additional three trials that had not been included in published reviews.
Two trial reports were counted as missing where the trial itself was reported in the GREAT database, but not the particular citations used in the systematic review. This is something that could helpfully be addressed in the future by linking all reports and abstracts from the same trial to the same trial record in the GREAT database. This would prevent bias being introduced by multiple reporting of the same trial.
Although the GREAT database is comprehensive, if conducting a comprehensive systematic review, other methods of identifying trials may be required, such as searching trials databases, as well as hand-searching and contacting of trial authors. In addition, reviews including studies other than RCTs will require additional searches of bibliographic databases to search for observational and non-randomised studies.
Conclusions
Strengths and weaknesses
Although using published systematic reviews as the gold standard for validation of the GREAT database is a useful tool for establishing coverage of the database, direct comparison can be challenging due to differences in eligibility criteria for reviews compared to the GREAT database. This can lead to an apparent lowering of the sensitivity of the database if different eligibility criteria have been applied.
Many treatments that the GREAT database covers have not been assessed in this validation study, as only a selection of systematic reviews were used. Sensitivity of the database may differ depending on the treatment being reviewed and so ongoing validation of the GREAT database is required.
Reviewing the GREAT database against all of the Cochrane systematic reviews of eczema treatments has ensured a complete comparison has been made using a very rigorous search methodology. The main weakness with this comparison is that authors of Cochrane reviews are expected to contact trial authors for information not present in the trial report. The GREAT database methodology does not include this due to time and resource constraints.
One review included RCTs that were identified from a trials database in which most trials were reported to be unpublished. Unfortunately it is impossible to be sure if five of the missing trials are in the GREAT database or not, as insufficient information is available in the public domain. For all eczema trials to be included in GREAT they must be published. Efforts to get all clinical trials registered and published, such as the AllTrials campaign [37], are helping to achieve this.
Implications for the future development of the GREAT database
One area for improvement of the GREAT database is dual filtering of references in order to reduce errors of omission of potentially eligible RCTs. Dual filtering involves two authors independently assessing the search results to determine eligibility of the included studies, with disagreements being resolved by a third party as necessary. Dual filtering has recently become part of the GREAT database methodology and should further improve the sensitivity of the database. It is our intention to apply dual-filtering retrospectively in order to ensure complete coverage of the GREAT database, but in the meantime, additional searches of bibliographic databases may be required, especially for RCTs published prior to 2000.
Other possible sources for trials to add to the GREAT database could be the interrogation of unpublished trials databases, searching of additional specialist databases such as PsycINFO® or Chinese databases, or adding all citations found for a particular trial (including conference abstracts). Whether or not to widen the inclusion criteria of the GREAT database beyond its current scope requires careful consideration as to its feasibility, impact and resource implications.
The GREAT database is sufficiently sensitive for most users, and is a resource that is quick and simple to interrogate; particularly for those looking for the best evidence for specific treatments that have not yet been included in a full systematic review, or for people looking for new trial evidence published after a systematic review’s search end date. The more the GREAT database is used and added to on an iterative basis, the better it will be.
Although there are other databases, such as for transfusion [38], as far as the authors are aware there are no other databases similar to GREAT for any other dermatological condition. It is therefore particularly important that the impact of the GREAT database on eczema research is assessed in order to enhance its potential to improve patient care and methodology of clinical trials in dermatology research. Over the next 5–10 years, similar databases for other common skin diseases such as psoriasis, acne, hand eczema and skin cancer might be developed with the aim of facilitating more effective mobilisation of evidence for clinical and research uses. The resources required to develop and maintain databases such as GREAT for the public good should not be underestimated. Collating the evidence and designing and testing the database required 20 months of researcher time, specialist IT input and hosting costs. Ongoing updates now require around 2 weeks of researcher time per month.
References
Williams HC, Wuthrich B. In: Williams HC, editor. Atopic Dermatits, vol. 1. Cambridge: Cambridge University Press; 2000.
Nankervis H, Maplethorpe A, Williams HC. Mapping randomized controlled trials of treatments for eczema–the GREAT database (the Global Resource of EczemA Trials: a collection of key data on randomized controlled trials of treatments for eczema from 2000 to 2010). BMC Dermatol. 2011;11:10.
Hoare C, Li Wan Po A, Williams H. Systematic review of treatments for atopic eczema. Health Technol Assess. 2000;4(37):1–191.
Mapping of systematic reviews of atopic eczema. [http://www.nottingham.ac.uk/research/groups/cebd/documents/methodological-resources/maponaewebsitefeb2013.pdf]
Futamura M, Thomas KS, Grindlay DJC, Doney EJ, Torley D, Williams HC. Mapping systematic reviews on atopic eczema—an essential resource for dermatology professionals and researchers. PLoS ONE. 2013;8(3):e58484.
Ashcroft DM, Chen LC, Garside R, Stein K, Williams HC. Topical pimecrolimus for eczema. Cochrane Database Syst Rev. 2007;4:CD005500.
Bath-Hextall F, Delamere FM, Williams HC. Dietary exclusions for established atopic eczema. Cochrane Database Syst Rev. 2008;1:CD005203.
Birnie AJ, Bath-Hextall FJ, Ravenscroft JC, Williams HC. Interventions to reduce Staphylococcus aureus in the management of atopic eczema. Cochrane Database Syst Rev. 2008;3:CD003871.
Boyle RJ, Bath-Hextall FJ, Leonardi-Bee J, Murrell DF, Tang ML. Probiotics for treating eczema. Cochrane Database Syst Rev. 2008;4:CD006135.
Ersser SJ, Latter S, Sibley A, Satherley PA, Welbourne S. Psychological and educational interventions for atopic eczema in children. Cochrane Database Syst Rev. 2007;3:CD004054.
Zhang W, Leonard T, Bath-Hextall F, Chambers CA, Lee C, Humphreys R, et al. Chinese herbal medicine for atopic eczema. Cochrane Database Syst Rev. 2005;2:CD002291.
Schram ME, Borgonjen RJ, Bik CM, van der Schroeff JG, van Everdingen JJ, Spuls PI. Off-label use of azathioprine in dermatology: a systematic review. Arch Dermatol. 2011;147(4):474–88.
Yin Z, Xu J, Luo D. Efficacy and tolerance of tacrolimus and pimecrolimus for atopic dermatitis: A meta-analysis. J Biomed Res. 2011;25(6):385–91.
Svensson A, Chambers C, Ganemo A, Mitchell SA. A systematic review of tacrolimus ointment compared with corticosteroids in the treatment of atopic dermatitis. Curr Med Res Opin. 2011;27(7):1395–406.
Simonart T, Kabagabo C, De Maertelaer V. Homoeopathic remedies in dermatology: a systematic review of controlled clinical trials. Br J Dermatol. 2011;165(4):897–905.
Schmitt J, von Kobyletzki L, Svensson A, Apfelbacher C. Efficacy and tolerability of proactive treatment with topical corticosteroids and calcineurin inhibitors for atopic eczema: systematic review and meta-analysis of randomized controlled trials. Br J Dermatol. 2011;164(2):415–28.
Boguniewicz M, Sampson H, Leung SB, Harbeck R, Leung DY. Effects of cefuroxime axetil on Staphylococcus aureus colonization and superantigen production in atopic dermatitis. J Aller Clin Immunol. 2001;108(4):651–2.
Novartis. A 4-week, randomized, multicenter, double-blind, vehicle-controlled, parallel-group clinical trial to evaluate the efficacy and safety of pimecrolimus cream 1% in the short-term treatment of patients with mild to moderate Atopic Dermatitis (Eczema). In: Novartis Trials Database. 2005.
Novartis. A 6-month, randomized, multicenter, parallel-group, double-blind, vehicle-controlled study to evaluate the efficacy and safety of pimecrolimus 1% cream twice daily vs. standard of care in the management of mild to severe atopic dermatitis in adults.In: Novartis Trials Database. 2005.
Novartis. A 12 week multicenter study consisting of a 6 week double blind, randomized, vehicle controlled, parallel group phase, followed by a 6 week open label phase, to assess the safety and efficacy of pimecrolimus cream 1% in mild to moderate head and neck atopic dermatitis of patients intolerant of topical corticosteroid. In: Novartis Trials Database; 2006.
Novartis. Confirmatory study in pediatric patients with atopic dermatitis- multicenter, randomized, double-blind, parallel-group, vehicle-controlled study with a 26-weeks treatment phase to determine the efficacy, safety of pimecrolimus cream for long-term treatment of pediatric patients with atopic dermatitis. In: Novartis Trials Database. 2005.
Novartis. A multicenter, 5-week, randomized, double-blind, placebo controlled, parallel group study exploring the effects of pimecrolimus cream 1% (twice daily application) vs. placebo control (twice daily application) on the molecular and cellular profile of adult male patients with atopic dermatitis. In: Novartis Trials Database. 2006.
Clinical Trials Results database. [http://www.novctrd.com/ctrdWebApp/clinicaltrialrepository/public/login.jsp]
Staab D, Kaufmann R, Brautigam M, Wahn U, Group CAC-S. Treatment of infants with atopic eczema with pimecrolimus cream 1% improves parents’ quality of life: a multicenter, randomized trial. Pediatr Allergy Immunol. 2005;16(6):527–33.
Thaci DF-HR, Foelster-Holst R, Hoger P, Kaufmann R, Brautigam M, Weidinger G, et al. Pimecrolimus cream 1% is effective in the treatment of atopic eczema in infants irrespective of clinical score. J Eur Acad Dermatol Venereol. 2003;17 Suppl 3:2.37.
Kirjavainen PV, Salminen SJ, Isolauri E. Probiotic bacteria in the management of atopic disease: underscoring the importance of viability. J Pediatr Gastroenterol Nutr. 2003;36(2):223–7.
Nakagawa H. Comparison of the efficacy and safety of 0.1% tacrolimus ointment with topical corticosteroids in adult patients with atopic dermatitis: review of randomised, double-blind clinical studies conducted in Japan. Clin Drug Investig. 2006;26(5):235–46.
Siegfried EKN, Molina C, Kianifard F, Abrams K. Safety and efficacy of early intervention with pimecrolimus cream 1% combined with corticosteroids for major flares in infants and children with atopic dermatitis Safety and efficacy of early intervention with pimecrolimus cream 1% combined with corticosteroids for major flares in infants and children with atopic dermatitis. J Dermatol Treat. 2006;17(3):143–50.
Breneman D, Fleischer Jr A, Abramovits W, Farber H. Tacrolimus ointment with and without topical steroids in patients with moderate to severe atopic dermatitis. J Am Acad Dermatol. 2006;54(3, Suppl 1 AB88):832. Abstract.
Hung SH, Lin YT, Chu CY, Lee CC, Liang TC, Yang YH, et al. Staphylococcus colonization in atopic dermatitis treated with fluticasone or tacrolimus with or without antibiotics. Ann Allergy Asthma Immunol. 2007;98(1):51–6.
Larsen FS, Simonsen L, Melgaard A, Wendicke K, Henriksen AS. An efficient new formulation of fusidic acid and betamethasone 17-valerate (fucicort lipid cream) for treatment of clinically infected atopic dermatitis. Acta Derm Venereol. 2007;87(1):62–8.
Ravenscroft JC, Layton AM, Eady EA, Murtagh MS, Coates P, Walker M, et al. Short-term effects of topical fusidic acid or mupirocin on the prevalence of fusidic acid resistant (FusR) Staphylococcus aureus in atopic eczema. Br J Dermatol. 2003;148(5):1010–7.
Polano MK, De Vries HR. Analysis of the results obtained in the treatment of atopic dermatitis with corticosteroid and neomycin containing ointments. Dermatologica. 1960;120:191–9.
Nilsson EJ, Henning CG, Magnusson J. Topical corticosteroids and Staphylococcus aureus in atopic dermatitis. J Am Acad Dermatol. 1992;27(1):29–34.
Leyden JJ, Marples RR, Kligman AM. Staphylococcus aureus in the lesions of atopic dermatitis. Br J Dermatol. 1974;90(5):525–30.
Fattah AA, El-Shiemy S, Faris R, Tadros SS. A comparative clinical evaluation of a new topical steroid ‘halcinonide’ and hydrocortisone in steroid-responsive dermatoses. J Int Med Res. 1976;4(4):228–31.
Alltrials. [http://www.alltrials.net/]
Transfusion evidence library. [http://evidentiapublishing.com/transfusion-evidence-library/]
Acknowledgements
This publication presents independent research funded by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research Programme (RP-PG-0407-10177). The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
Helen Nankervis - lead the creation of the GREAT database as part of her work on an NIHR funded Programme Grant (RP-PG-0407-10177).
John Ingram – current co-investigator of an eczema trial funded by Atopix Therapeutics Ltd. He has not received any reimbursements, fees, funding, or salary from involvement in the trial.
Kim Thomas – co-applicant for the NIHR funded Programme Grant (RP-PG-0407-10177) that supported this work.
Hywel Williams – co-applicant for the NIHR funded Programme Grant (RP-PG-0407-10177) that supported this work.
Alison Devine - No competing interests.
Elizabeth Doney – No competing interests.
Sherie Smith – No competing interests.
Finola Delamere – No competing interests.
Authors’ contributions
HN contributed to the design of the study, extracted data, coordinated data extraction, comparison and arbitration, drafted the first draft of the manuscript and revised the manuscript according to co-author comments. AD extracted data and commented on the final manuscript. HCW contributed to the design of the study and commented on the manuscript, JRI compared data extraction data and arbitrated where the data did not agree, ED contributed to the design of the study, extracted data and commented on the manuscript, FD contributed to the design of the study and commented on the manuscript, SS contributed to the design of the study and commented on the manuscript, KST contributed to the design of the study and commented on the manuscript. All authors read and approved the final manuscript.
Rights and permissions
This article is published under an open access license. Please check the 'Copyright Information' section either on this page or in the PDF for details of this license and what re-use is permitted. If your intended use exceeds what is permitted by the license or if you are unable to locate the licence and re-use information, please contact the Rights and Permissions team.
About this article
Cite this article
Nankervis, H., Devine, A., Williams, H.C. et al. Validation of the global resource of eczema trials (GREAT database). BMC Dermatol 15, 4 (2015). https://doi.org/10.1186/s12895-015-0024-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12895-015-0024-z