Introduction

Scarring is a frequent occurrence among patients with acne. Most patients have some degree of scarring and approximately 40% of all acne patients have clinically relevant scars [1, 2]. There is a significant correlation between acne severity and likelihood of acne scarring [3], but scars can occur with acne of any severity, including mild [1, 2]. In an international study of over 6000 patients, scarring was present in 26.7% of subjects with mild acne, 51.3% of those with moderate acne, and 76.7% of those with severe acne [3].

In 2010, the Global Alliance to Improve Outcomes in Acne group created an initiative to improve understanding of acne scarring. Acne scarring can involve either loss (atrophic) or excessive accumulation (hypertrophic) of tissue with differing pathogenesis and clinical management [4]. Atrophic acne scarring is more common in clinical practice; accordingly the initial focus was on atrophic acne scarring. Up to that time, few studies of acne scarring had been conducted and most focused on management. There were relatively few standardized and validated tools for clinical assessment of acne-related scarring; even expert dermatologists may disagree about how to describe acne scars in a consistent way [2, 59]. No validated patient assessment instruments specific for acne scarring were found in the literature and only sparse studies of impact of atrophic acne scarring on quality of life (QoL) [10]. Patients with acne scarring (particularly on the face) seek treatment for these scars, implying that there is an impact on their lives [11]. Further, psychosocial morbidity may be linked with acne scarring [4, 12, 13]. However, no QoL tools specifically address the impact of atrophic acne scarring and limited data suggest that acne QoL measures are not sensitive for acne scarring [14]. Ilgen et al. [14] tested the Acne Quality of Life Scale (AQOLS) and the Dermatology Life Quality Index (DLQI) in 108 patients and found no difference between those with or without scarring. It should be noted that neither the AQOLS nor the DLQI was designed for the purpose of assessing acne scars; thus, it would not be surprising if they were not sensitive or specific in evaluating the effects of scarring. A more recent acne-specific QoL scale [Acne Symptom and Impact Scale (ASIS)] included an item about “scars/scabs/dark marks”, but this tool addressed acne as a global disease and was not specific for atrophic acne scars [15]. To address specifically acne scarring evaluation, we developed two instruments to help patients describe the severity of their atrophic acne scars [Self-assessment of Clinical Acne-Related Scars (SCARS) questionnaire] and to assess the impact of atrophic acne scars on QoL [Facial Acne Scar Quality of Life (FASQoL) questionnaire] [16]. The aim of this study was to develop these scar tools based on patients’ input according to Food and Drug Administration (FDA) guidance on patient-reported outcomes (PROs). Measurement properties of the two instruments will be tested in a further step.

Methods

The protocol for the creation of the patient assessment tools followed the classical steps for developing and qualitatively testing a PRO instrument in accordance with the US FDA PRO Guidance [16]. Since the FDA PRO Guidance notes that a fundamental consideration in the development and use of a PRO instrument is whether an instrument’s concepts are clearly defined and appropriate to the population of interest, these steps include: (1) elicitation of the concepts of interest regarding appearance, symptoms, impacts, bothersomeness [through concept elicitation (CE) interviews with specific questionnaires]; (2) generation of items, selection and implementation of the most relevant signs (items); and (3) testing of patient understanding of the draft instrument through cognitive interviews (CIs). Details of the methodology used are presented in Fig. 1.

Fig. 1
figure 1

Methodology for developing patient-reported outcomes tool

CE interviews were designed to probe aspects of import to patients and items explored included symptoms, impact, description of appearance and severity of atrophic acne scars, and bothersomeness. The number of CE interviews was determined based on saturation achievement—the point when no more unique concepts arose. In an independent sample of subjects, understanding of concepts and item wording was tested through CIs. All the interviews were conducted in the USA.

Ethical Conduct of the Interviews

All interviews were performed in accordance with the ethical principles of the Declaration of Helsinki and consistent with Good Clinical Practice and applicable regulatory requirements. Medical and all other information collected during the study was kept confidential. Copernicus Group, Durham, NC, USA, served as the Independent Review Board (IRB).

Recruitment

After obtaining IRB approval, patients were recruited through a recruitment agency that accepted centralized IRB approval based on the finalized study inclusion and exclusion criteria (same for both sets of interviews). The recruitment agency was provided with study objectives prior to the start of recruitment; dermatologists from an existing database were contacted to identify patients with atrophic acne scars. Adult candidates (18 years old and above) completed a Subject Information and Consent Form including an Authorization to Use and Disclose Personal Health Information for Research Form. Adolescent patients (12–17 year olds) and their parents/legal guardians completed an Adolescent Assent Form and an accompanying Parental Permission and Health Insurance Portability and Accountability Act (HIPAA) Form. After written informed consent and authorization was obtained, the clinicians confirmed patient eligibility for the study in person.

Inclusion/Exclusion Criteria

All subjects had atrophic acne scars; active acne could be present, and recruitment was designed to include equal numbers of subjects with mild, moderate, and severe acne. Subjects also had to be willing and able to provide informed consent or assent (for adolescent subjects), to speak US English, and to participate in study procedures. Subjects were excluded if they had hypertrophic and/or keloidal scars, a skin condition in the area of acne scars that would interfere with study procedures (e.g., plaque psoriasis, tattoo, birthmark, facial hair), or a significant medical condition that would confound study results or interfere significantly with a subject’s involvement in the study. Targeted patients had a range of active acne severity (no active acne with only scars to grade 4 severity with scars) [2] and were grouped in two groups of skin types: light to medium (Fitzpatrick skin type I–IV) and dark skin (Fitzpatrick skin type V and VI) [3].

For the CE interviews, 30 subjects were recruited and interviewed with a range of acne lesion and acne scar severities. Severity was classified using Investigator Global Assessment (IGA) and the Scar Global Assessment (SGA). An additional group of 20 subjects with atrophic acne scars was recruited to participate in individual face-to-face CIs.

Conduct of Interviews

A subgroup of experts from the Global Alliance to Improve Outcomes in Acne constituted a Board of Experts for the development of these tools, providing definitions of atrophic scars for a user manual guide for the recruitment process. Patients with scars excluded from the assessment according to the guide were not evaluated. Atrophic acne scars were defined as the permanent loss of tissue following acne lesions (not induced by manipulation or excoriation).

All interviews were conducted by trained personnel (National Institutes of Health Participant Protection training). Prior to the interviews, mock interviews were conducted to review the interview guide and ensure good interviewing practices. Interviews lasted up to 60 min and interviewers followed a semi-structured Interview Guide. Subjects were asked open-ended questions, followed by more targeted, probing questions if needed. The open-ended approach ensured that subjects were not unduly biased and elicited the language subjects use to describe what they believe to be the symptoms and impacts of their acne scars. In addition to the open-ended questions, subjects were asked to rank the severity of scars in photos to further engage them and elicit more information on symptoms, impacts and description of appearance and severity of atrophic acne scars. Subjects were asked to focus responses on atrophic acne scars and to exclude pigmentary changes, scars from other etiologies (e.g., trauma, varicella), and hypertrophic/keloidal scars.

Data Handling

Interviews were recorded and transcribed, and coding for key concepts was performed, then analyzed. The content of SCARS and FASQoL tools was selected based upon results from the CE interviews. Special attention was given to the importance and relevance of concepts based on the frequency with which they were reported by interviewed patients.

Cognitive Interviews

The objectives of the CIs were to evaluate each item from the instruments for adequacy of the following properties: relevance of all of the items (and domains, if any) covered by the instruments; comprehensiveness (content validity) of the items; comprehensibility, applicability, and acceptability of the items; and instruments or items appear to measure what they should measure. Furthermore, CIs were conducted to evaluate clarity and ease of comprehension on first reading.

The CIs were conducted in two phases. During the first phase, subjects were asked to complete the draft instruments and provide feedback. They were asked to think aloud about the process they used to arrive at each answer and to identify words, terms, or concepts that were difficult to interpret or understand. They were also asked if they would make changes to improve the usefulness of the tools. Following this first phase, the instruments were revised based on patient feedback as well as expert input. The revised instruments were cognitively debriefed in a second phase of CIs. Similar to the first phase of CIs, patients were asked to complete the PRO instruments using a think aloud method. Patient feedback on item/concept relevance, comprehensiveness and interpretability was also collected. The revised instrument was re-submitted to the IRB for approval before use in the second phase of CIs. The interviews lasted approximately 1 h and were audio-recorded (with subjects’ prior consent) and transcribed verbatim and anonymized. The results from these interviews were coded and compared in a fashion similar to that used with the CE interviews.

Statistical Analysis

Analysis of qualitative data collected during interviews involved constant comparison and traditional content tabulations. Qualitative data were analyzed using ATLAS.ti version 7.0 (Atlas.ti GmbH, Berlin, Germany), a software package designed specifically for the analysis of qualitative data. Researchers trained in qualitative methods and coding developed a coding scheme that was applied to all transcripts. A saturation grid was developed to compare the amount of new information elicited and analyzed periodically during the interview process to determine that saturation had been reached. The goal of the saturation grid was to compare the amount of new or “novel” information that was observed. Responses from the interviews were grouped by domain, concept, and patient. Further analysis compared the number of responses elicited from the first 25% of patients to the next 25% of patients, then from the first 50% of patients to the next 25% of patients, and finally from the first 75% of patients to the last 25% of patients. This allowed researchers to determine if and when new concepts were elicited as data collection neared completion and, ultimately, to determine if saturation was met within the number of interviews conducted. The saturation results determined whether or not additional interviews should be conducted.

Results

Patient Populations

Group 1: Concept Elicitation

A total of 30 subjects participated in the CE interviews (15 adults and 15 adolescents). The mean age of the adults was 31.0 years (range 20.3–50.6 years) and the mean age of adolescents was 16.0 years (range 13.8–17.8 years). Most adults (73.3%) were female; 40.0% of the adolescents were female. Approximately half of subjects had mild acne (adults 56.7%, adolescents 50.0%), one-third had moderate acne (adults 23.3%, adolescents 33.3%) and a small group had severe acne (adults 20.0%, adolescents 16.7%). Fitzpatrick I–III types were regrouped in light to medium skin class, and Fitzpatrick IV–VI types into dark skin class. Table 1 presents information about the duration of acne and acne scars as well as scar severity for the group.

Table 1 Scar and acne severity in concept elicitation interview population

Group 2: Cognitive Interviews

Twenty CIs were conducted in two phases to allow adjustment and improvement of the SCARS and FASQoL instruments after the first phase. Subjects included eight adolescents (mean age 16.6 years) and 12 adults (mean age 35.6 years). Among these, 11 (55%) were female; 11 (55%) had a light to medium skin type while 9 (45%) had a dark skin type; 2 patients (10%) had scars only, 6 (30%) had mild active acne, 7 (35%) had moderate active acne, and 5 (25%) had severe acne.

Concept Elicitation

Among all concepts (n = 60) expressed by patients across the symptom and impact domains, the vast majority (n = 58) were reported prior to the final quartile of interviews. The two concepts that were reported for the first time in the final quartile [sensitivity (as a symptom of acne scars) and talking too much (as a coping mechanism to distract from acne scars)] were mentioned by only one patient each (3.3%). Therefore, these concepts are considered idiosyncratic to a small minority of patients. As such, saturation was considered to be achieved for appearance, severity, symptom, and impact concepts. The sample size of 30 patients was sufficient to conclude that no new meaningful concepts would emerge even if additional interviews were conducted.

Appearance, Severity, and Symptoms Associated with Acne Scars

During CE interviews, patients discussed the physical attributes and severity of their acne scars (Fig. 2). These were organized under the domains appearance, severity, and symptoms. Among the appearance factors, frequently reported concepts included color (n = 24, 80.0%), size (n = 24, 80.0%), and location/area (n = 22, 73.3%). Somewhat frequently reported concepts included texture (n = 13, 43.3%), number (n = 13, 43.3%), shape (n = 12, 40.0%), and depth (n = 11, 36.6%). Among the severity factors (i.e., factors patients considered when describing scar severity), frequently reported concepts were color (n = 21, 70.0%), number (n = 20, 66.6%), size (n = 19, 63.3%), and location/area (n = 17, 56.6%); somewhat frequently reported concepts were depth (n = 12, 40.0%) and texture (n = 11, 36.6%).

Fig. 2
figure 2

Factors patients identified important in determining acne scar severity

Most Bothersome Appearance Factors

In the discussion of appearance factors associated with acne scars, patients highlighted those factors that were most bothersome: color (n = 7, 23.3%), depth (n = 2, 6.6%), number (n = 2, 6.6%), and size (n = 1, 3.3%). Patients explained that these appearance factors were bothersome because they felt that their scars were noticeable (n = 7, 23.3%), numerous (n = 2, 6.6%), dark (n = 2, 6.6%), created an uneven skin tone (n = 1, 3.3%), and would not go away (n = 1, 3.3%).

Impact on Patients’ Life

During CE interviews, patients explained the effect that acne scars had on their lives (Fig. 3). Among the acne scar impact concepts, two were frequently reported: feeling less attractive (n = 25, 83.3%) and feeling self-conscious (n = 20, 66.6%). Other impacts included confidence (n = 10, 33.3%), annoyed (n = 10, 33.3%), worried about scars not going away (n = 9, 30.0%), depressed feeling (n = 8, 26.6%), upset by negative comments from others (n = 10, 33.3%), avoiding or changing social activities (n = 7, 23.3%), dating (n = 7, 23.3%), bothered having to spend time putting make-up on (n = 7, 23.3%), and impacts to work/school life (n = 8, 26.6%). Although two coping mechanisms and moderating factors (having to cover scars with make-up and washing one’s face) were somewhat frequently reported, these do not pertain directly to either scar severity or impacts, and therefore were not included in the development of the PRO. On the other hand, the impact on patients’ relationships (specifically dating), though infrequently reported (n = 4, 13.3%), was recommended for inclusion to broadly understand the impact of acne scars on patients’ relationships with friends, family, and significant others.

Fig. 3
figure 3

Concepts most bothersome to acne scar subjects

Most Bothersome Life Impacts

Most commonly, patients identified impacts to their appearance/attractiveness (n = 5, 16.6%), confidence (n = 4, 13.3%), social activities (n = 4, 13.3%), and the inconvenience of using make-up (n = 4, 13.3%) as the most bothersome. Three patients (10.0%) each reported worrying about the permanence of acne scars and impacts to work/school life as the most bothersome, while another two patients (6.6%) each reported worrying about what others think, people staring, romantic relationships, and emotional impacts generally. One patient (3.3%) each reported that comments from others, feeling embarrassed, and feeling self-conscious were the most bothersome impacts.

Operationalization of the Concepts and Instrument Development

The results of the CE interviews and initial drafts of items were presented to the clinical experts from the Global Alliance. Content for both instruments was selected based upon results from the CE interviews. In particular, special attention was given to the importance and relevance of concepts based on the frequency of report by interviewed patients.

Item Wording and Structure

The specific wording for the instruments was based on the actual words and phrases that patients used to talk about their scars during the CE interviews and on standard language conventions used for instrument development (e.g., avoiding medical jargon and use of vague terms). The definition of atrophic acne scars included in both PRO instruments was informed by the words patients used as well as expert input. The terms that were most used by both adult and adolescent patients to describe atrophic acne scars were “indents” and “holes.” Other terms suggested by the experts for consideration such as dips, pits, pinholes, sunken areas, and stretches were mentioned infrequently by patients (n < 3). Therefore, the definition included in the instructions of both PROs was “atrophic acne scars are indents or holes in the skin from previous acne spots.”

Recall Period

To use the SCARS tool, patients are instructed to look in the mirror and assess appearance concepts at the present moment. This method was used to minimize measurement bias introduced by memory problems. For the FASQoL, the recall of recent states (no more than 1 week) was preferable and also served to minimize measurement bias introduced by memory problems while still providing the patient a long enough timeframe to experience the impacts being assessed.

Response Options

All of the items in the instruments include response options based on a five-point Likert-type scale. Both instruments were designed to be a self-report in the clinical research setting. In the current development program, subjects were asked to complete the questionnaire using a paper-and-pen format.

Cognitive Interview Results

Phase I

The following revisions were made to the instructions to help patients understand the differences between active acne and acne scars and reflect patient language.

  • The language for active acne lesions and atrophic acne scars was simplified;

  • To help patients understand active acne and acne scars, “active acne” was revised to “zits, breakouts, pimples, whiteheads, or blackheads,” and “acne scars” was revised to “indents or holes” in the items.

  • All references to post active acne lesions were removed;

  • “Facial jewelry/piercings” was added at the beginning of the VAS instructions based on expert feedback;

  • The item wording was revised from “uneven” to “rough” based on patient feedback during CEs and to help patients understand the intended interpretation.

Phase II

No new items were added to the SCARS initially developed, as patient suggestions were inconsistent with one another or were not relevant to the measurement goals. All patients (n = 12, 100.0%) demonstrated no difficulty interpreting the instructions as intended. No revisions were made, but training is recommended prior to the use of the instrument in clinical trials to help patients understand the differences between active acne and atrophic acne scars. Patients appeared to have difficulty interpreting the concept of roughness for item 5 and the item was deleted. Item 6 (noticeability) was reworded to ensure that patients consider the visibility of their acne scars to themselves and not to others. The order of items 2 (amount) and 3 (size) was reversed to help patients with the understanding of coverage and amount of acne scars.

Overall, subjects provided positive feedback on the SCARS and FASQoL questionnaires. The majority of concepts were relevant and understood by subjects; items were worded so that subjects could provide meaningful responses. SCARS is a 5-item questionnaire (Table 2). The FASQoL includes 10 items assessing atrophic acne scar-related impacts (Table 3).

Table 2 SCARS patient-reported outcome tool
Table 3 Facial acne scar quality of life (FASQoL) questionnaire

Discussion

Skin disease can have profound and long-lasting impact on those affected [17]. While many with atrophic acne scars seek treatment, little is known about the impact of this condition. It is generally assumed that acne scars have a negative impact on QoL [18]. The Global Alliance scar working group is working to develop a variety of acne-scarring specific tools for both subjects and clinicians, with a goal of providing a more standardized and focused understanding of acne scars’ severity. This report describes the development of a patient-reported severity assessment (SCARS) and a QoL assessment tool (FASQoL), both developed in accordance with the FDA PRO Guidance. These tools are intended to provide information about changes that occur during acne treatment (including changes in scar severity and possible prevention of new scars), and we feel this justifies the inclusion of patients with both active acne and scars.

During the development process, several considerations emerged. First, differentiation of active acne, post-inflammatory erythema and hyperpigmentation from scars is difficult for the lay public. We found it is essential to incorporate lay language, with the simplest terminology possible. Important concepts for subjects in determining acne scar severity included number, size, location/area, depth, and texture (skin evenness). Patients also indicated color was important. However, color is confounded when active acne or post-acne lesions are present [post-inflammatory hyperpigmentation (PIH)], therefore an item on color was not included.

SCARS is a 5-item instrument with one hypothesized domain that asks subjects to rate the severity of acne scars as seen in a mirror. The tool starts with two visual analog scales to remind the patient to separate active acne lesions from acne scars. Use of a short photographic training manual for patients could help to optimize the sensitivity of this tool. FASQoL is a 10-item instrument with three domains assessing the impact of scars on emotions, social functioning and work/school on 5-point rating scales with a recall period of the past 7 days. The content of the two tools has been validated from patient’s perspective and is ready for psychometric validation.

We believe the development of scar-specific tools is important since there is some evidence showing that existing tools lack sensitivity and specificity to assess the impact of scars. In general, more generic health-related instruments are less sensitive than disease-specific measures [19]. To our knowledge, no prior patient-reported atrophic acne scar-specific measure has been developed. Brown et al. [19] reported the development of a patient-reported scar assessment tool [patient-reported impact of scars measure (PRISM)], but it includes scars of differing etiologies and assesses both symptoms and QoL impact. It is the first tool we know of based on patient input that measures the global impact of scars from the patient’s viewpoint. It would be interesting to compare PRISM with our tools to find areas of agreement and disagreement. Similarly, the tool developed by Alexis et al. [15] was designed primarily to assess active acne, but it incorporates some questions about scars/scabs/marks. We feel this could be useful to provide an overall assessment of the patient’s experience of acne, but may not be applicable as an outcome measure for management of the group of patients with scars, but no active acne.

The study had some limitations; first, interviews were conducted only in the USA. Adaptation of the two PRO instruments for use in other patient populations will be needed, following the conduct of translatability testing and additional CIs. Translatability testing and additional patient interviews would ensure content validity across other languages and cultures. In addition, there is a need to assess the measurement properties of the two PRO instruments to provide evidence for the reliability and construct validity of the instruments. Finally, responsiveness to treatment should be assessed.

Conclusions

As treatments emerge for acne scarring, tools that address the patient’s perspective are becoming increasingly important. We have created tools to inform clinicians about the patients’ perspectives that can be used in both routine practice and clinical studies to help clinicians manage patients with acne scars. We followed the rigorous recommendations given by FDA guidance for developing two PRO tools (SCARS and FASQoL) and found that the concepts included are relevant to the condition and that the tools created are easily understood, simple to complete, and contain information relevant to daily activities. To assess sensitivity to treatment benefit, SCARS and FASQoL should be evaluated in a clinical trial with a treatment known to improve and/or prevent scars.