The study used an Internet panel survey to collect measures of consumer perceptions about emerging drug safety information and the effects of modifying a sample DSC on a specific drug safety issue. Half of the study sample was randomized electronically to receive a previously developed drug safety message about a fictitious drug used for smoking cessation; the other half of the sample was exposed to a revised version of the same message (see electronic supplementary material 1 for additional information about the sample allocation process). Study investigators were blinded to the randomization process. Inclusion criteria included ensuring that at least half of participants or a family member used a prescription drug in the past year and that at least one-third of the participants had no more than a high-school education. These and other sociodemographic characteristics were also considered in developing the weight variable. Sample size was determined based on power calculations for the primary outcomes with at least a 90 % power for detecting small effects (Cohen’s d = 0.2). Human subjects approval was obtained for this research. No changes were made to the study design after the study commenced.
For the previous version of the drug safety message (‘Standard version’), we used a DSC released online in a single-page, long-form format before FDA instituted the multi-tabbed DSC format, with more plain language and other general content modifications. When creating the revised version, the drug name was changed to ‘Smoquit,’ a fictitious name. For the revised version of the drug safety message (‘Revised version’), we applied plain language , clear communication , and health literacy principles to test whether the changes would improve readability and uptake of the information, or influence other measures. The reading level of the Standard version was grade 11 and for the Revised version it was grade 8, using the Simple Measure of Gobbledygook (SMOG) test . Table 1 presents a side-by-side comparison of various elements of the Standard and Revised versions of the message.
Briefly, the Revised version retained the same general content as the Standard version but used the active voice, made behavioral recommendations more action oriented, used less complex language, ‘chunked’ information with additional subheadings, and provided both quantitative and qualitative data with appropriate context and explanation. For example, the text describing a meta-analysis was revised to use simpler terminology and given a plain language subheading called ‘Looking at the best evidence.’ Language used in the Standard version was reworded in the Revised version and was also added to the main safety announcement to explain to patients the likelihood of experiencing an adverse event when taking the medication. The Standard version reported these numbers only in the data summary section aimed at healthcare professionals, stating that “there was a low incidence of major cardiovascular events occurring within 30 days of treatment discontinuation (Smoquit 0.31 % [13/4190]).” The Revised version was reworded to state “the chance of someone having a heart-related problem if they took Smoquit was 31 in 10,000 (0.31 %).”
Data Collection and Panel Survey
Data were collected using a KnowledgePanel®  probability-based Internet sample that is designed to be representative of US households. GfK, the developer of KnowledgePanel, uses address-based sampling of a computerized file updated every 2 months that contains all delivery point addresses serviced by the US Postal Service (over 125 million records with 97 % coverage of US households). This approach reduces sampling biases that are introduced through the use of random digit dialing and Internet-based sampling methods because it includes non-telephone and non-Internet households.
Individuals who agree to be on the panel can use their own computers connected to the Internet to take surveys; netbooks and Internet access are provided to panel members living in non-Internet households. Panel ‘case managers’ provide telephone support to households that require help connecting their computers to the Internet, accessing their e-mail, and accessing and responding to Internet surveys. KnowledgePanel consists of about 50,000 US residents aged 18 or older, and includes people living in cell-phone-only households. Numerous internal and independent assessments of the representativeness of KnowledgePanel samples have been conducted; no evidence of selection bias or bias attributable to time in the panel were identified .
The survey was fielded between July 24 and August 20, 2013.
The comprehension and behavioral intention questions were specifically developed to align with the intervention for this study. The health literacy items are from a validated scale. All measures were prespecified and were not changed once data collection commenced.
Comprehension We measured respondents’ comprehension of the Smoquit DSC using five survey items (α = 0.70) developed specifically for this study and based on the content included in the DSC. The questions examined respondents’ awareness and knowledge about the following: (1) How common are major cardiovascular or heart-related events? (2) Who is most likely to have heart-related problems? (3) When should patients taking Smoquit contact their healthcare professional? (4) What is the likelihood of experiencing an adverse event? and (5) What are FDA’s recommendations for patients taking Smoquit? Each survey item had a single correct answer; consequently, we created a five-item ‘comprehension index’ reflecting the proportion of correct answers to these questions. Respondents were told at the end of the survey that the message was about a fictitious drug.
Message Assessment, Risk Perception, and Trust of the Source Respondents were asked five questions about the utility of the DSC; specifically, whether it was (1) clear, (2) informative, (3) convincing, (4) helpful, and (5) did they learn something new. These questions used a 1–4 response scale, with 1 = Strongly disagree and 4 = Strongly agree. We summed the responses to these items to create a 5-item ‘message assessment’ scale (α = 0.91). We also asked respondents about how much they trust the information in the DSC and how much risk they think there is for a person with heart or blood vessel disease who is taking Smoquit (using a 4-item response scale of 1 = None, 2 = Some; 3 = A fair amount; 4 = A lot, with a ‘Don’t know’ option). The focus for this last question was on individuals who already have heart or blood vessel disease because they are at increased risk.
Behavioral Intentions The survey included six questions about respondents’ intended behaviors/actions as a result of being exposed to the DSC; that is, how likely would they be to (1) talk with their HCP about Smoquit, (2) report any symptoms described in the message to their HCP, (3) report any symptoms described in the message to the FDA, (4) look for more information about the medicine, (5) discontinue taking the drug. The responses to these questions each had a 1–7 scale with end points labeled 1 = Strongly disagree and 7 = Strongly agree. The sixth question asked if a respondent would take Smoquit to try and stop smoking if their doctor prescribed it; responses were on a 1–4 scale with end points labeled 1 = Strongly disagree to 4 = Strongly agree.
Sociodemographic, health insurance, and geographic data were collected from respondents as part of their participation in KnowledgePanel. Participants were classified by GfK as to whether they had Internet access or were given it as a part of their panel participation.
Sources of Prescription Drug Information and Information Preferences To assess what information sources people consult about prescription drugs, we asked the following question: “Before taking a prescription drug, what information source(s) would you use to learn about it?” A total of 20 different response options were provided in the following categories: (1) People (such as doctor or pharmacist), (2) Traditional media (such as radio or magazine), (3) Internet or mobile sources, and (4) Other.
To assess what kind of information people want about prescription drugs, we asked the following question: “When you are looking for information about prescription drugs, what kind of information do you want?” Response options included ‘General use of the medicine,’ ‘Safety information,’ ‘Possible side effects,’ ‘Dosage information or how much to take,’ ‘What it is best used for,’ ‘I don’t look for that information,’ and ‘Other.’
Health Literacy To evaluate health literacy level, we used a five-item version of the Health Literacy Skills Instrument (HLSI) [25, 26], a computer-based instrument that measures a range of health literacy skills in the general population (α = 0.75). This five-item version of the HLSI focuses largely on numeracy-related skills. Respondents use real-world stimuli such as charts, tables, maps, and other images to answer questions about a range of health-related topics. Each question has only one correct response option, and scale scores are computed as the percentage of correct responses ranging from 0 to 100.
Prescription Drug Utilization Prescription drug use was assessed with one item: “In the past 12 months, have you taken a prescription drug? Examples of prescription drugs include antibiotics, antidepressants, and insulin.”
Chi-square tests were used to determine if the two DSC groups (Standard version versus Revised version) had comparable demographic characteristics. Responses between the two groups on all dependent and independent variables were compared using Chi-square tests for categorical variables and t-tests for continuous variables. Finally, a linear regression model was used to compare comprehension index scores based on DSC condition while controlling for other factors that may have an impact on comprehension, including demographics (i.e., gender, age, education level, race/ethnicity, income, insurance, and geographic region), household Internet access, prescription drug use in the past 12 months, health literacy, risk perceptions, and trust in the message. We also ran an analysis to determine the proportion of individuals who did not accurately respond to the question about overall risk of Smoquit and who also did not answer the question correctly about the overall risk of Smoquit. Analyses were conducted using SAS version 9.3 survey analysis procedures, and survey weights were applied to represent the national population and adjusted for differential nonresponse. The difference between weighted and unweighted frequencies for all variables was no more than 4 percentage points.