In 2002, a comprehensive methodological review of the acne literature over 50 years revealed great variation in severity assessment and outcome measures. The authors suggested a number of possible methodological improvements, since such variation makes comparisons across the literature difficult and hampers the development of evidence-based medicine in the field. To investigate whether any improvement has been achieved, we evaluated the recent literature on topical and systemic therapeutic interventions. In 18 papers, published in 2011, we identified 25 ways of reporting changes in lesion counts and 25 ways of reporting changes in assessed grade. Fourteen grading systems were used to evaluate the therapeutic response. Only three studies reported data on quality of life. Our finding reveals a continued and substantial variation in severity assessment and outcome measurement in the acne literature.
Globally, acne is a common skin disorder, affecting a majority of adolescents [1–4]. Although acne typically starts in puberty, it often persists into adulthood . It is associated with marked physical and psychological morbidity, and significantly affects the quality of life of people with the disease [6, 7•, 8••]. A variety of treatment options have been developed for acne, and are tailored to the severity and persistence of the disease. Over the last few decades, numerous clinical trials have been conducted to evaluate the efficacy and safety of such interventions, making acne therapy a highly studied area in dermatology. Study of the literature and current research reveals a continued strong interest in developing and evaluating treatments.
When evaluating therapeutic interventions, outcome measures are used to assess and classify the severity of the condition before the intervention, as well as the subsequent response to treatment. Outcome measures are, therefore, crucial tools to therapeutic research. Appropriate and uniform application of validated and reliable outcome measures minimizes methodological variation among clinical trials, making it possible to make meaningful comparisons across them. In short, the outcome measures make it possible to qualify changes in the course of a disease.
In 2002, a comprehensive methodological review of the acne literature over the last 50 years revealed great variation in outcome assessment . In this review, Lehmann et al.  studied randomized controlled trials (RCTs), evaluating miscellaneous acne therapies in patients without any complicating comorbidity. Among the evaluated trials, the authors identified more than 25 methods of assessing acne severity, including lesion counting, comparison with a photographic standard, and comparison with a text description. Furthermore, the way in which severity had been reported was found to be inconsistent across the studies, and this inconsistency was not limited to severity assessment. Even when the same principle was used, outcome measures were reported in various ways—the authors identified 19 methods of reporting lesion counts. The variation in severity assessment and outcome measures in the acne literature, reported by Lehmann et al. , outlined the difficulty of making meaningful comparisons among trials and emphasized the importance of using comparable outcome measures in future studies. In this review, we investigate whether any improvement has been achieved in the recent literature.
A systematic electronic literature search was carried out in Medline, using a mixture of free text terms and medical subject headings, including acne vulgaris, therapy and randomized. We considered trials for inclusion if they were RCTs and if they aimed to investigate the clinical efficacy of any medical and/or topical therapy on acne. We included papers in English, published from January1, 2011, to December 31, 2011. We excluded studies if they were split-face comparisons, or if they did not include clinical efficacy as an endpoint or contain original data. To obtain homogeneity among the evaluated studies, we also excluded follow-up studies in which investigation of recurrence ratio was the main objective. Additionally, to conduct a comparable evaluation to Lehmann and colleagues’ study , we excluded studies on patients with acne conglobata, acne fulminans, chloracne, rosacea, acne venenata, acne necrotica, or any complicating comorbidity such as an endocrine disease.
Eighteen papers evaluating various treatment options were identified. These were then carefully studied and methodological data concerning severity assessment and outcome measures were extracted for further analysis.
The 18 papers that met the inclusion and exclusion criteria [10–27] were published across 14 journals, and each represented one RCT, making the number of reviewed RCTs 18. Half of the trials (50 %) were conducted in US or European countries (including a study conducted in the United States, Canada, and Belize and a study in Europe, Mexico, and Australia); the remaining half were conducted in Asian countries.
The duration of the treatments studied varied from 4 weeks to 24 months, and in the majority of papers (67 %), no post-treatment data were reported. In the six papers reporting post-treatment data, the follow-up period varied from 4 weeks to 1 year.
The trials were distributed as five (28 %) in mild to moderate category, five (28 %) in moderate to severe category, four (22 %) in moderate category, one (6 %) in mild category, and none in severe category. One trial included participants with acne of any severity, and two did not use “mild moderate or severe” terminology to report the severity at the baseline.
The mean age and male-to-female ratio was reported in almost every trial (94 %). The location of acne was not reported in five trials (28 %), and, in about half of the cases (56 %), the authors did not report whether they had collected data on compliance.
Classification of Severity at the Baseline
The majority of the reviewed papers (89 %) used the terminology mild, moderate, or severe to report the severity of the disease; however, in one third of the studies, the number of lesions was not reported as an inclusion/exclusion criterion. Furthermore, the use of the terms mild, moderate, or severe was inconsistent across the studies, with several different grading systems being used to delineate the categories. For example, the mild to moderate category might be defined in one study as having 10 to 50 facial inflammatory lesions and 10 to 100 facial noninflammatory lesions, with no nodules or cysts, and in another as having less than 11 nodules. In 14 trials (78 %), the severity assessment was conducted according to nine different, acknowledged grading systems.
The following categories of outcome measures were identified: grading, lesion counting, participant evaluation, quality of life, biophysical measurements, bacterial examination, and recurrence (Fig. 1). The evaluation of efficacy was based mainly on grading and lesion counting. All studies reported at least one of the two categories, and 72 % reported both. In 16 studies (89 %), outcome measures included grading, and lesion counting was reported in 15 studies (83 %).
We identified 25 ways of reporting changes in the 14 grading systems used (Table 1). Of the methods used, six were global assessments based on the presence of dominant lesions and the extent of inflammation (eg, Investigator’s Global Assessment). Five were severity scores based on mathematical formulas providing a total score by multiplying the number of each lesion by its severity index (eg, Acne Severity Index score). Three were classifications based on the number of distinct lesions (eg, Korean Acne Grading System). Furthermore, the studies were not necessarily consistent when referring to specific grading systems; that is, the same scale might be referred to by different names and abbreviations across the studies (eg, IGA, ISGA, EGSS) or different formulas used to calculate the same severity score (eg, Acne Severity Index score).
We identified 25 ways of reporting changes in lesion counts (Table 2). These included changes in the number of lesions, percentage change in number of lesions, and percentage of participants achieving a certain degree of change in the number of lesions.
None of the studies provided health economic outcomes. Only three trials reported data on patients’ quality of life; however, each used a different instrument for that purpose. Of the three questionnaires used, one was dermatology specific (Skindex-16) and two were acne specific (Cardiff Acne Disability Index and Acne-Specific Quality of Life Questionnaire).
The seminal review by Lehmann et al.  reveals great variation in severity assessment and outcome measures in the acne literature from 1948 to 1999. The authors suggested a number of possible methodological improvements, since such variation makes comparisons across the literature difficult and hampers the development of evidence-based medicine in the field. To investigate whether any improvement has, in fact, been achieved, we evaluated the recent literature on topical and systemic therapeutic interventions. We studied English-language papers published in 2011 that reported results from RCTs on therapeutic interventions for acne vulgaris. Similar to the findings of Lehmann et al. , we found great variation among the trials in severity assessment and outcome measures. In 18 trials, we identified nine grading systems used to assess acne severity at the baseline. Furthermore, we identified 25 ways of reporting changes in lesion counts and 25 ways of reporting changes in grading, with the latter representing 14 grading systems. Data on economic outcomes were not reported in any study, and only three papers reported data on quality of life; however, each used a different instrument for that purpose.
The lack of homogeneity in acne outcome measures has also been addressed elsewhere. In a Cochrane review evaluating the effect of minocycline, the authors identified 50 outcomes in 26 studies; accordingly, they concluded that data pooling would not be appropriate . The lack of standardized outcome measures was also an issue in a Cochrane review evaluating the efficacy of oral contraceptives in acne .
There is no single standardized method to evaluate therapeutic interventions for acne. Lesion counting and grading represent two major categories of outcome measures used in clinical trials. Lesion counting is an objective measure used since the 1960s . It enables investigators to measure small changes in therapeutic response, and, in the case of targeted products that only affect a distinct lesion (eg, comedones), it makes it possible to evaluate the efficacy of the treatment better than general compounded scores. However, lesion counting has limitations because of its lack of evaluation of quantitative variables such as the degree of inflammation and the size of lesions. The use of lesion counting alone does not, therefore, provide adequate information on the severity of the disease. Yet the extent of involvement and degree of inflammation can be expressed by physician’s global assessment, a subjective measure that is not necessarily sensitive to therapeutic response of individual lesions.
A number of global assessment methods have been described, some containing multiple variables. A study investigating the reliability of global assessment and lesion counting in a group of 11 dermatologists demonstrated good intra-rater reliability (ICC ≥ 0.75) in lesion counting but somewhat less so in global assessment . Combining the two outcome categories may, however, provide a more balanced approach toward the evaluation of acne severity than is currently available.
Since the mid-90s, approval of new products by the US Food and Drug Administration (FDA) has been based on statistically significant superiority in two lesion counts (total lesion counts and either inflammatory or noninflammatory lesion counts) and in physician’s global assessment . In 2005, the FDA prepared guidance to the pharmaceutical industry on the development of products for the treatment of acne vulgaris . In this nonfinalized draft document made available by the FDA, baseline acne severity was referred to as a key element of the enrollment criteria and the use of co-primary endpoints that evaluate physician’s global assessment, as well as lesion counts, was recommended. It was also recommended that the physician’s global assessment be a qualitative evaluation of overall acne severity, and use of an ordinal scale was preferred with approximately five or six severity grades, one of which is “completely clear.” Additionally, it was recommended that the outcome of physician’s global assessment be reported as “success” or “failure,” where success was defined as “clear” (grade 0) or “almost clear” (grade 1) or as improvement of two grades from the baseline score.
A closer look at the evaluated trials reveals that six papers reported changes in at least two acne lesion counts as well as proportion of “success,” according to the FDA recommendations. Interestingly, all three RCTs, conducted in the United States, met the aforementioned recommendations, suggesting that recommendations may lead to the use of more comparable outcome measures.
Only three trials reported changes in quality of life, and each used a different instrument. In contrast, in a comparable study of psoriasis treatment, quality of life measures were shown to have been used increasingly often . Although combining lesion counting and physician’s global assessment provides a more balanced approach to evaluation of therapeutic interventions, quality of life measures would add valuable information on efficacy. In a study investigating the association of demographic factors with clinical severity and quality of life impact, the authors demonstrated that clinical grading and quality of life measures each differentiate distinct groups of acne patients most severely affected by acne [35•].
Reporting outcomes of acne treatment is apparently difficult, even though Lehman et al.  indicated important areas where methodological improvements were possible and necessary. In this review, we have identified a continued and substantial variation in severity assessment and outcome measurement 12 years later. Our findings emphasize the importance of using comparable outcome measures in future studies. The great variation among the trials, conducted outside the United States, illustrates the importance of recommendations and guidelines on standardization. A future international agreement recommending the use of standardized, validated, and reliable outcome measures would permit performance of meaningful comparisons across the studies, which will provide a strong foundation for both evidence-based medicine and decision making in everyday practice.
Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance
Ghodsi SZ, Orawa H, Zouboulis CC. Prevalence, severity, and severity risk factors of acne in high school pupils: a community-based study. J Invest Dermatol. 2009;129(9):2136–41.
Dreno B, Poli F. Epidemiology of acne. Dermatology. 2003;206(1):7–10.
Law MP, Chuh AA, Lee A, Molinari N. Acne prevalence and beyond: acne disability and its predictive factors among Chinese late adolescents in Hong Kong. Clin Exp Dermatol. 2010;35(1):16–21.
Rademaker M, Garioch JJ, Simpson NB. Acne in schoolchildren: no longer a concern for dermatologists. BMJ. 1989;298(6682):1217–9.
Collier CN, Harper JC, Cafardi JA, Cantrell WC, Wang W, Foster KW, et al. The prevalence of acne in adults 20 years and older. J Am Acad Dermatol. 2008;58(1):56–9.
Barnes LE, Levender MM, Fleischer Jr AB, Feldman SR. Quality of life measures for acne patients. Dermatol Clin. 2012;30(2):293–300. ix.
• Dunn LK, O’Neill JL, Feldman SR. Acne in adolescents: quality of life, self-esteem, mood, and psychological disorders. Dermatol Online J. 2011;17(1):1. This review provides information on the negative impact of acne on quality of life.
•• Williams HC, Dellavalle RP, Garner S. Acne vulgaris. Lancet. 2012;379(9813):361–72. This is a comprehensive review of acne vulgaris.
Lehmann HP, Robinson KA, Andrews JS, Holloway V, Goodman SN. Acne therapy: a methodologic review. J Am Acad Dermatol. 2002;47(2):231–40.
Choudhury S, Chatterjee S, Sarkar DK, Dutta RN. Efficacy and safety of topical nadifloxacin and benzoyl peroxide versus clindamycin and benzoyl peroxide in acne vulgaris: a randomized controlled trial. Indian J Pharmacol. 2011;43(6):628–31.
Eichenfield LF, AlioSaenz AB. Safety and efficacy of clindamycin phosphate 1.2%-benzoyl peroxide 3% fixed-dose combination gel for the treatment of acne vulgaris: a phase 3, multicenter, randomized, double-blind, active- and vehicle-controlled study. J Drugs Dermatol. 2011;10(12):1382–96.
Agarwal US, Besarwal RK, Bhola K. Oral isotretinoin in different dose regimens for acne vulgaris: a randomized comparative trial. Indian J Dermatol Venereol Leprol. 2011;77(6):688–94.
Kobayashi M, Nakagawa T, Fukamachi K, Nakamura M, Tokura Y. Efficacy of combined topical treatment of acne vulgaris with adapalene and nadifloxacin: a randomized study. J Dermatol. 2011;38(12):1163–6.
Lee HE, Ko JY, Kim YH, Yoo SR, Moon SH, Kim NI, et al. A double-blind randomized controlled comparison of APDDR-0901, a novel cosmeceutical formulation, and 0.1% adapalene gel in the treatment of mild-to-moderate acne vulgaris. Eur J Dermatol. 2011;21(6):959–65.
Abels C, Kaszuba A, Michalak I, Werdier D, Knie U, Kaszuba A. A 10% glycolic acid containing oil-in-water emulsion improves mild acne: a randomized double-blind placebo-controlled trial. J Cosmet Dermatol. 2011;10(3):202–9.
Adalatkhah H, Pourfarzi F, Sadeghi-Bazargani H. Flutamide versus a cyproterone acetate-ethinyl estradiol combination in moderate acne: a pilot randomized clinical trial. Clin Cosmet Investig Dermatol. 2011;4:117–21.
Schmidt N, Gans EH. Clindamycin 1.2% tretinoin 0.025% gel versus clindamycin gel treatment in acne patients: a focus on Fitzpatrick Skin Types. J Clin Aesthet Dermatol. 2011;4(6):31–40.
Tanghetti E, Dhawan S, Green L, Ling M, Downie J, Germain MA, et al. Clinical evidence for the role of a topical anti-inflammatory agent in comedonal acne: findings from a randomized study of dapsone gel 5% in combination with tazarotene cream 0.1% in patients with acne vulgaris. J Drugs Dermatol. 2011;10(7):783–92.
Morganti P, Berardesca E, Guarneri B, Guarneri F, Fabrizi G, Palombo P, et al. Topical clindamycin 1% vs. linoleic acid-rich phosphatidylcholine and nicotinamide 4% in the treatment of acne: a multicentre-randomized trial. Int J Cosmet Sci. 2011;33(5):467–76.
Pazoki-Toroudi H, Nilforoushzadeh MA, Ajami M, Jaffary F, Aboutaleb N, Nassiri-Kashani M, et al. Combination of azelaic acid 5% and clindamycin 2% for the treatment of acne vulgaris. Cutan Ocul Toxicol. 2011;30(4):286–91.
Maleszka R, Turek-Urasinska K, Oremus M, Vukovic J, Barsic B. Pulsed azithromycin treatment is as effective and safe as 2-week-longer daily doxycycline treatment of acne vulgaris: a randomized, double-blind, noninferiority study. Skinmed. 2011;9(2):86–94.
Dreno B, Kaufmann R, Talarico S, Torres LV, Rodriguez-Castellanos MA, Gomez-Flores M, et al. Combination therapy with adapalene-benzoyl peroxide and oral lymecycline in the treatment of moderate to severe acne vulgaris: a multicentre, randomized, double-blind controlled study. Br J Dermatol. 2011;165(2):383–90.
Trifu V, Tiplica GS, Naumescu E, Zalupca L, Moro L, Celasco G. Cortexolone 17alpha-propionate 1% cream, a new potent antiandrogen for topical treatment of acne vulgaris. A pilot randomized, double-blind comparative study vs. placebo and tretinoin 0.05% cream. Br J Dermatol. 2011;165(1):177–83.
Hayashi N, Kawashima M. Efficacy of oral antibiotics on acne vulgaris and their effects on quality of life: a multicenter randomized controlled trial using minocycline, roxithromycin and faropenem. J Dermatol. 2011;38(2):111–9.
Lee JW, Yoo KH, Park KY, Han TY, Li K, Seo SJ, et al. Effectiveness of conventional, low-dose and intermittent oral isotretinoin in the treatment of acne: a randomized, controlled comparative study. Br J Dermatol. 2011;164(6):1369–75.
Babayeva L, Akarsu S, Fetil E, Gunes AT. Comparison of tretinoin 0.05% cream and 3% alcohol-based salicylic acid preparation in the treatment of acne vulgaris. J Eur Acad Dermatol Venereol. 2011;25(3):328–33.
Babaeinejad S, Khodaeiani E, Fouladi RF. Comparison of therapeutic effects of oral doxycycline and azithromycin in patients with moderate acne vulgaris: What is the role of age? J Dermatolog Treat. 2011;22(4):206–10.
Garner SE, Eady EA, Popescu C, Newton J, Li WA. Minocycline for acne vulgaris: efficacy and safety. Cochrane Database Syst Rev. 2003;(1):CD002086.
Arowojolu AO, Gallo MF, Grimes DA, Garner SE. Combined oral contraceptive pills for treatment of acne. Cochrane Database Syst Rev. 2004;(3):CD004425.
Witkowski JA, Simons HM. Objective evaluation of demethylchlortetracycline hydrochloride in the treatment of acne. JAMA. 1966;196(5):397–400.
Tan JK, Fung K, Bulger L. Reliability of dermatologists in acne lesion counts and global assessments. J Cutan Med Surg. 2006;10(4):160–5.
Executive summary, Dermatologic and Ophthalmic Drugs Advisory Committee, November 4-5, 2002. Availible at: http://www.fda.gov/ohrms/dockets/ac/02/briefing/3904B1_02_Executive%20summary.htm
Guidance for Industry Acne Vulgaris: Developing Drugs for Treatment, U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research, September 2005. Availible at: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM071292.pdf
Morsy H, Kamp S, Jemec GB. Outcomes in randomized controlled trials in psoriasis: what has changed over the last 20 years? J Dermatolog Treat. 2007;18(5):261–7.
• Tan JK, Li Y, Fung K, Gupta AK, Thomas DR, Sapra S, et al. Divergence of demographic factors associated with clinical severity compared with quality of life impact in acne. J Cutan Med Surg. 2008;12(5):235–42. This article provides information on demographic factors associated with quality of life impact.
K. Zarchi: none; G. B. E. Jemec: advisory board member to Abbott Laboratories and Pfizer, consultant to Coloplast, grant from LEO Pharma, payment for giving lectures from Galderma, travel/accommodations/meeting expenses covered/reimbursed by MSD, and research support from Actelion, Janssen-Pharma, and Abbott Laboratories.
About this article
Cite this article
Zarchi, K., Jemec, G.B.E. Severity Assessment and Outcome Measures in Acne Vulgaris. Curr Derm Rep 1, 131–136 (2012). https://doi.org/10.1007/s13671-012-0016-8
- Acne vulgaris
- Acne severity
- Outcome measures
- Severity assessment
- Randomized controlled trials
- Grading systems
- Topical and systemic therapeutic interventions