Abstract
Purpose
The Statin-Associated Muscle Symptom Clinical Index (SAMS-CI) is a method for assessing the likelihood that a patient’s muscle symptoms (e.g., myalgia or myopathy) were caused or worsened by statin use. The objectives of this study were to prepare the SAMS-CI for clinical use, estimate its inter-rater reliability, and collect feedback from physicians on its practical application.
Methods
For content validity, we conducted structured in-depth interviews with its original authors as well as with a panel of independent physicians. Estimation of inter-rater reliability involved an analysis of 30 written clinical cases which were scored by a sample of physicians. A separate group of physicians provided feedback on the clinical use of the SAMS-CI and its potential utility in practice.
Results
Qualitative interviews with providers supported the content validity of the SAMS-CI. Feedback on the clinical use of the SAMS-CI included several perceived benefits (such as brevity, clear wording, and simple scoring process) and some possible concerns (workflow issues and applicability in primary care). The inter-rater reliability of the SAMS-CI was estimated to be 0.77 (confidence interval 0.66–0.85), indicating high concordance between raters. With additional provider feedback, a revised SAMS-CI instrument was created suitable for further testing, both in the clinical setting and in prospective validation studies.
Conclusions
With standardized questions, vetted language, easily interpreted scores, and demonstrated reliability, the SAMS aims to estimate the likelihood that a patient’s muscle symptoms were attributable to statins. The SAMS-CI may support better detection of statin-associated muscle symptoms in clinical practice, optimize treatment for patients experiencing muscle symptoms, and provide a useful tool for further clinical research.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Statins are among the most widely prescribed drugs in the USA [1], and statin-associated muscle symptoms are the most commonly reported adverse events of statin therapy [2]. Approximately 60% of adults who no longer take statins cite muscle pain as the primary reason for discontinuation [3]. Since statins reduce the risk of cardiovascular events, and adherence to statin therapy correlates with reduced cardiovascular mortality, the presence of muscle-related adverse events associated with statin therapy represents a major clinical and public health concern [4, 5]. In fact, one study by Graham et al. indicated that patients with intolerance to statins experienced higher healthcare resource use, higher likelihood of cardiovascular events, and lower likelihood of achieving their low-density lipoprotein cholesterol goal compared to a matched cohort without intolerance to statins [6]. Likewise, a study by Serban and colleagues demonstrated that, among Medicare beneficiaries, intolerance to statins was associated with higher recurrent risk of myocardial infarction and coronary revascularization [7]. The presence of statin-associated muscle symptoms also negatively influences the ability of patients to perform activities of daily living and engage in physical activity, which is concerning because of the strong inverse relationship between physical activity and mortality [8, 9].
While no gold standard measure exists for the identification of statin-associated muscle symptoms, progress toward a clear definition came in a 2014 publication from the Statin Muscle Safety Task Force of the National Lipid Association (NLA). In this work, Rosenson et al. [2] defined the spectrum of statin-associated muscle events to include, in increasing order of severity: myalgia (described as flu-like symptoms), myopathy (muscle weakness), myositis (muscle inflammation), myonecrosis (muscle enzyme elevation or increase in creatine kinase), and clinical rhabdomyolysis. To date, there are no standardized measurement instruments to accurately diagnose statin-associated muscle symptoms. Therefore, in order to determine the likelihood that muscle-related effects are attributed to statin use, the NLA Statin Muscle Safety Task Force proposed a new methodology for assessing the likelihood that a statin-treated patient’s myalgia or myopathy were caused or worsened by statin use (Fig. 1).
The method was originally referred to as the Statin Myalgia Clinical Index (SMCI), but was re-named the Statin-Associated Muscle Symptom Clinical Index (SAMS-CI) to reflect its breadth of assessing all forms of statin-associated muscle symptoms outlined by the NLA, not just myalgia. The SAMS-CI includes four separate ratings: the first regards the location and patterns of the muscle symptoms and the remaining three address the timing of symptoms relative to starting, stopping (dechallenge), and rechallenging with statins. As with any new measurement tool, documentation of its development, measurement properties, and practical application are required. This paper therefore reports on: (1) preparing the SAMS-CI for clinical use, including documentation of content validity; (2) estimating its inter-rater reliability; and (3) collecting initial feedback from physicians on the potential clinical use of the tool. These efforts are taken to prepare the tool not only for clinical use, but also for research, such as in an ongoing validation study with an independent cohort to establish the SAMS-CI’ predictive validity.
Methods
The New England Institutional Review Board approved this study. The three phases of the study are further described in Fig. 2 and consist of: (1) preparing the SAMS-CI for clinical use, (2) estimating its inter-rater reliability, and (3) gathering feedback from clinicians on its potential for clinical use.
Preparing the SAMS-CI for Clinical Use
For clarity, the original proposed method published by the NLA Statin Muscle Safety Task Force will be referred to as the SMCI (Fig. 1). The intermediate version following author interviews and review will be called the interim SAMS-CI (not shown), and the version prepared for clinical use is denoted as the SAMS-CI (Fig. 3).
Adaptation of the original SMCI for routine clinical use proceeded with three activities, as shown in Fig. 2. First, between October 16 and 22, 2014, hour-long telephone interviews were conducted with a convenience sample of three of the five authors of the SMCI (RSR, BT, and TAJ, also authors of this paper). These semi-structured interviews covered several topics, including the thought process behind each question; the response options, scoring algorithm, and importance weights; the precise definition of certain terms; the type of patient best suited to take the SMCI; and reasons for excluding certain concepts from the SMCI. The interviews were audiotaped and transcribed for analysis.
Second, we conducted a review of the SMCI with regard to everyday use, assessing any missing features and applying survey design principles to the content and layout. Changes to the SMCI following the author interviews and this review resulted in the interim SAMS-CI.
Third, between January 23 and 30, 2015, we conducted content validation interviews of the interim SAMS-CI among 10 clinicians who had no involvement in drafting the SMCI or SAMS-CI. The sample included two primary care physicians and eight cardiologists across the USA (further description available in Online Resource 1). To be included, clinicians needed to write an average of 40 or more statin prescriptions per month, have treated at least 10 patients with statin-associated muscle symptoms in the previous year, and have been in practice post-residency for at least 5 years. These interviews followed a rigorous, standardized qualitative methodology using a semi-structured interview guide to gather detailed information from respondents on the conceptual domains, language clarity, suggested changes, and other reactions to each element of the interim SAMS-CI (instructions, questions, response options, and scoring). The interviews were audiotaped and transcribed for analysis. Respondents were blinded to the identity of the sponsor. We applied the information gathered in these interviews to generate the final SAMS-CI (Fig. 3).
Estimating Inter-rater Reliability
The next phase of development was to test the reliability of the SAMS-CI. The inter-rater reliability was estimated by the intra-class correlation coefficient (ICC). Using the notation of Shrout and Fleiss [10], we calculated ICC (2,1) as:
where BMS is between-case variance, JMS is between-rater variance, EMS is residual (error), k is number of raters (e.g., clinicians), and n is number of clinical cases. A 95% confidence interval was constructed around the ICC using Fisher’s z transformation. In this methodology, statistical power is largely determined by the number of clinical cases rated by each respondent rather than the number of respondents [11]. Thus, we utilized 10 respondents rating 30 case studies, which were sufficiently powered to detect a true ICC above 0.65 with an alpha of 0.05 and a power of 0.85.
Our sample of 10 raters included four primary care physicians and six cardiologists, as shown in Table 1. This sample of clinicians was drawn separately from the content validation sample, but the inclusion criteria were identical. Raters were blinded to the identity of the researchers and sponsor.
Each case presented a hypothetical patient with hypercholesterolemia, and the set of cases was designed to cover the full range of possible scores on the SAMS-CI. We used an Excel function to randomly draw 10 clinical states from each of the three possible ratings: “probable,” “possible,” and “unlikely,” as defined by the total SAMS-CI scores. The “probable” vignettes were randomly drawn from among the 14 ways to score “probable” from the SAMS-CI items, for a sampling fraction of 71%. Likewise, the “possible” vignettes were randomly drawn from the 35 “possible” states, for a sampling fraction of 29%. Lastly, the “unlikely” vignettes were randomly drawn from the 59 “unlikely” states, for a sampling fraction of 17%. In all cases, the hypothetical patient had experienced two or more previous episodes of muscle symptoms with statin use. Table 2 contains a sample clinical case from the study, Online Resource 2 presents a screenshot of the SAMS-CI as it appeared to study participants, and Online Resource 3 details all of the clinical cases included in the study.
All cases were developed by a practicing cardiologist (otherwise unaffiliated with this study), who wrote each case to fit the target SAMS-CI score as selected above. The cases were reviewed and edited by two measurement scientists, a clinical nurse, and a copyeditor.
The 30 cases were programmed into an online survey system [Qualtrics.com; Provo, UT, USA], and raters completed the SAMS-CI for all the cases at their convenience within a 2-week period between May 18 and June 1, 2015. The order of presentation of cases was randomized by rater and they were required to provide a rating for every case.
Collecting Feedback on Potential Clinical Use of the SAMS-CI
In the third phase, the 20 clinicians who participated in the content validation (n = 10) and inter-rater reliability (n = 10) studies also gave input on the feasibility of using the SAMS-CI in their practice (Fig. 2). They reported on which staff they thought could administer the SAMS-CI to patients, how the SAMS-CI might be integrated with electronic medical systems, and any potential barriers inhibiting its use.
The content validation interview respondents addressed this topic at the end of the in-depth instrument review. The inter-rater reliability respondents addressed the topic during a 30-min telephone interview conducted after they had completed the 30 clinical cases online. These interviews followed a rigorous, standardized qualitative methodology using a semi-structured interview guide, and were audiotaped and transcribed for analysis.
Results
Preparing the SAMS-CI for Clinical Use
The revisions to the original SMCI were based on the three sources of information described above: author interviews, review with regard to clinical use, and content validation interviews. Key changes included the addition of a large, visually salient scoring system; revision of question and response option text for simplicity, clarity, and descriptive value; and addition of instructions regarding the overall instrument and certain questions.
Crucially, formatting changes were also made to clarify which questions applied to which muscle symptom episodes. In the SMCI, the patient is assumed to have had a minimum of two previous episodes of muscle symptoms, yet the SAMS-CI should also be relevant for patients who have had only one previous episode, perhaps arising in the primary care setting. In this case, as seen in the finalized SAMS-CI for clinical use shown in Fig. 3, questions A through C could be completed initially, and D could be completed after a statin rechallenge. To accommodate this likely clinical situation, the SAMS-CI is presented in two distinct columns side by side, with slightly different versions for the case of one previous episode of muscle symptoms versus two or more episodes.
Estimating Inter-Rater Reliability
All 10 raters scored all 30 clinical cases, resulting in 300 SAMS-CI ratings in total with no missing data. Table 3 shows that, among the 30 clinical cases, on average 8.6 of the 10 raters calculated the correct SAMS-CI rating of “probable,” “possible”, or “unlikely”. This average was slightly higher for the extremes—9.0 for “probable” and 8.9 for “unlikely”—and slightly lower for the middle case of “possible” (7.9). The estimated ICC of the SAMS-CI was 0.77, with a 95% confidence interval of 0.66 to 0.85.
Feedback on Potential Clinical Use of the SAMS-CI
All 20 clinicians interviewed regarding the SAMS-CI found it to be clear and brief enough for use in clinical practice, and 12 reported that they would use the SAMS-CI in their own clinical practice. Considerations in favor of using the SAMS-CI in routine clinical practice mentioned by cardiologists and primary care physicians included the following:
-
Brief and clear wording and scoring processes.
-
Ability of non-physician staff such as nurses or physician assistants to complete it with patients.
-
Usefulness in the context of patient counseling, such as triggering a conversation about alternate causes, building the patient’s confidence in the clinical assessment, and educating patients generally about the relationship between statins and muscle symptoms.
Clinicians also cited some possible concerns about routine use:
-
Fitting the SAMS-CI into constrained workflows and brief patient visits.
-
Concern that the SAMS-CI assesses just one aspect of statin intolerance (the experience of muscle symptoms), and must be interpreted in the context of other clinical information, such as recent physical exertion or hypothyroidism.
-
Need for information from two previous episodes of muscle symptoms to calculate a score.
Clinicians also expressed interest in embedding the SAMS-CI into electronic medical record systems, which would streamline the administration of the tool and centralize the results for access by clinical staff.
Discussion
The SAMS-CI offers a framework for attributing muscle symptoms to statin use. The initial form of the instrument, the SMCI, had not been previously optimized for routine clinical use. This mixed-method study revised the SMCI to produce a new version (the SAMS-CI) ready for use and further testing, both in clinical practice and in research on its predictive validity with real patients. Methods in this study included qualitative research on content validation and a quantitative study of inter-rater reliability. Qualitative results supported the content of the SAMS-CI, and the inter-rater reliability of the SAMS-CI was estimated to be 0.77 (above the customary minimum ICC threshold of 0.7 [12]), indicating strong correspondence between raters. This study also elicited several potential strengths and weaknesses of the SAMS-CI as integrated into clinical care.
Although muscle symptoms often co-occur with statin use, statins are not necessarily their cause. Both the ODYSSEY ALTERNATIVE and GAUSS-3 studies [13, 14] recruited patients with a strong history of muscle-related statin intolerance and used a complex study design in order to enhance the selection of a “truly” statin-intolerant patient population. The ODYSSEY ALTERNATIVE study started with a placebo run-in period, during which patients who experienced muscle-related adverse events were excluded, followed by a randomized controlled trial including a statin control arm. Of the 47 patients who failed to complete the placebo run-in, 48.9% had at least one skeletal muscle event related to placebo and were excluded from the study. The GAUSS-3 study used a two-stage approach using a first-step cross-over procedure to allow identification of patients who developed muscle symptoms during atorvastatin administration, but not during placebo administration and vice-versa. During this cross-over procedure, 26.5% of patients discontinued for intolerable muscle symptoms with placebo (but not atorvastatin) and did not enter the second step of the study.
These two examples demonstrate that reported muscle symptoms are not always related to statin use and illustrate how difficult it is to properly identify the statin intolerance population. These rechallenge studies provide further evidence that the predominant cause of statin intolerance may be the “nocebo” effect, which is totally dependent on patient awareness of a treatment and its potential adverse effects [4, 15]. They may also arise from unrelated factors such as coincident underlying pathologies involving the musculoskeletal system, changes in activity routines, or psychological determinants [15].
The need for tools to identify statin intolerance is rapidly growing. Recently, two manuscripts were published which created an algorithm to identify statin-associated muscle symptoms from claims databases and/or electronic medical records [16, 17]. Furthermore, a patient-level tool, which allows patients to start a dialog about statin-associated muscle symptoms with healthcare providers, has also been developed [18]; and finally, the American College of Cardiology has developed a mobile application to identify statin-associated muscle symptoms. In our study, the SAMS-CI was developed to assist clinicians’ estimates of the likelihood that myalgia or myopathy are attributable to statins. The current study represents the first important step on the road to final validation.
A next step in this process will be to collect evidence about the performance of the SAMS-CI with actual statin patients in practice. Further research can establish the positive and negative predictive values of the SAMS-CI to proactively identify cases of statin-associated muscle symptoms, thereby averting unnecessary rechallenges. While the SAMS-CI has not yet been used in research contexts, it was developed in part from muscle symptom assessment protocols used in two recent clinical trials. The Effect of Statins on Skeletal Muscle Function study [19] used a dechallenge-rechallenge protocol in statin-naive patients, and the Coenzyme Q10 in Statin Myopathy Study [20] used a double-blind, placebo-controlled, randomized, cross-over protocol in patients with a previous history of statin-associated muscle symptoms. In these trials, 30–50% of patients with supposed statin-associated muscle symptoms actually had non-specific muscle pain (pain on placebo). Consequently, as a first step toward validation, we are carrying out a post hoc reanalysis of data from these two trials. Although the SAMS-CI did not exist at the time of these studies, enough detailed data was collected on each patient at baseline to complete the SAMS-CI retrospectively. These scores will then be compared to the patients’ ultimate outcomes in the studies, to estimate the SAMS-CI’ ability to predict statin-associated muscle symptoms among patients on statins as well as to identify patients who experienced muscle symptoms on placebo. These results may suggest further revision of the instrument before a dedicated, prospective study is carried out.
Conclusions
Attributing muscle symptoms to statin use has long been identified in the literature as a difficult problem to measure, standardize, and quantify. The SAMS-CI resolves some of these issues through the use of standard questions with vetted language, easily interpreted scores, and proven inter-rater reliability. As revised for clinical use, the SAMS-CI may support better detection of statin-associated muscle symptoms in clinical practice, better-optimized treatment for patients experiencing muscle symptoms, and stronger measurement in clinical research. Forthcoming research will seek to validate the SAMS-CI in a larger population and establish its predictive validity.
References
Kantor ED, Rehm CD, Haas JS, Chan AT, Giovannucci EL. Trends in prescription drug use among adults in the United States from 1999-2012. JAMA. 2015;314(17):1818–31.
Rosenson RS, Baker SK, Jacobson TA, Kopecky SL, Parker BA. The National Lipid Association’s muscle safety expert panel an assessment by the statin muscle safety task force: 2014 update. J Clin Lipidol. 2014;8(3 Suppl):S58–71.
Wei MY, Ito MK, Cohen JD, Brinton EA, Jacobson TA. Predictors of statin adherence, switching, and discontinuation in the USAGE survey: understanding the use of statins in America and gaps in patient education. J Clin Lipidol. 2013;7(5):472–83.
Collins R, Reith C, Emberson J, Armitage J, Baigent C, Blackwell L, et al. Interpretation of the evidence for the efficacy and safety of statin therapy. Lancet. 2016;388(10059):2532–61.
Mihaylova B, Emberson J, Blackwell L, Keech A, Simes J, Barnes EH, et al. The effects of lowering LDL cholesterol with statin therapy in people at low risk of vascular disease: meta-analysis of individual data from 27 randomised trials. Lancet. 2012;380(9841):581–90.
Graham JH, Sanchez RJ, Saseen JJ, Mallya UG, Panaccio MP, Evans MA. Clinical and economic consequences of statin intolerance in the United States: results from an integrated health system. J Clin Lipidol. 2016.
Serban MC, Colantonio LD, Mathripragada AD, Monda KL, Bittner VA, Banach M, Chen L, Huang L, Dent R, Kent ST, Muntner P, Rosenson RS. Statin intolerance and risk of coronary heart events and all-cause mortality following myocardial infarction. J Am Coll Cardiol. 2017;69(11);1386–95.
Taylor BA, Thompson PD. Muscle-related side-effects of statins: from mechanisms to evidence-based solutions. Curr Opin Lipidol. 2015;26(3):221–7.
Franklin BA, Lavie CJ. Impact of statins on physical activity and fitness: ally or adversary? Mayo Clin Proc. 2015;90(10):1314–9.
Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86(2):420–8.
Bonett DG. Sample size requirements for estimating intraclass correlations with desired precision. Stat Med. 2002;21(9):1331–5.
Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34–42.
Moriarty PM, Thompson PD, Cannon CP, Guyton JR, Bergeron J, Zieve FJ, et al. Efficacy and safety of alirocumab vs ezetimibe in statin-intolerant patients, with a statin rechallenge arm: the ODYSSEY ALTERNATIVE randomized trial. J Clin Lipidol. 2015;9(6):758–69.
Nissen SE, Stroes E, Dent-Acosta RE, Rosenson RS, Lehman SJ, Sattar N, et al. Efficacy and tolerability of evolocumab vs ezetimibe in patients with muscle-related statin intolerance: the GAUSS-3 randomized clinical trial. JAMA. 2016;315(15):1580–90.
Tobert JA, Newman CB. The nocebo effect in the context of statin intolerance. J Clin Lipidol. 2016;10(4):739–47.
Schulman KL, Lamerato LE, Dalal MR, Sung J, Jhaveri M, Koren A, et al. Development and validation of algorithms to identify statin intolerance in a US administrative database. Value Health. 2016;19(6):852–60.
Colantonio LD, Kent ST, Huang L, Chen L, Monda KL, Serban MC, et al. Algorithms to identify statin intolerance in Medicare administrative claim data. Cardiovasc Drugs Ther. 2016;30(5):525–33.
Jacobson TA, Edelman SV, Galipeau N, Shields AL, Mallya UG, Koren A, et al. Development and content validity of the Statin Experience Assessment Questionnaire (SEAQ) (c). Patient. 2016.
Parker BA, Capizzi JA, Grimaldi AS, Clarkson PM, Cole SM, Keadle J, et al. Effect of statins on skeletal muscle function. Circulation. 2013;127(1):96–103.
Taylor BA, Lorson L, White CM, Thompson PD. A randomized trial of coenzyme Q10 in patients with confirmed statin myopathy. Atherosclerosis. 2015;238(2):329–35.
Acknowledgements
The authors thank the physicians who provided feedback on optimization and clinical use of the SAMS-CI, and acknowledge the contributions of Rita Samuel, MD, William Sasiela, PhD (Regeneron Pharmaceuticals, Inc.) and Gloria Garcia-Palacios, PharmD (Sanofi) for providing critical review of the manuscript. Minor editorial and administrative support for later drafts was provided by Rachel Wright, PhD, of Prime (Knutsford, Cheshire, UK), funded by Sanofi and Regeneron Pharmaceuticals, Inc.
Author’s Contributions
All authors contributed to conception and design of the study. K Miller, M Bayliss, and M White conducted all interviews and managed the online data collection. R Rosenson, T Jacobson, and B Taylor contributed expert opinion as data for the study. K Miller and M Bayliss carried out the data analysis. All authors contributed to interpreting the data, drafting the manuscript, and revising it for important intellectual content. The authors were responsible for all content and editorial decisions, and gave final approval of the version to be submitted.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethical Approval
This article does not contain any studies with human participants performed by any of the authors.
Conflicts of Interest
Robert Rosenson has received grant support from Amgen, AstraZeneca, Catabasis, Medicines Company, Sanofi-Aventis; is a consultant/advisor for Akcea, Amgen, AstraZeneca, Eli Lilly, Regeneron Pharmaceuticals, Inc., and Sanofi-Aventis; has received honoraria from Kowa; and receives royalties from UpToDate, Inc. Kate Miller, Martha Bayliss, and Michelle White are employees of Optum Patient Insights, which was retained by Regeneron Pharmaceuticals, Inc. for this work. Beth Taylor served on Amgen’s Pharmacovigilance Monitoring Board and receives research support from Regeneron Pharmaceuticals, Inc. Robert Sanchez and Garen Manvelian are employees of Regeneron Pharmaceuticals, Inc. and have stock/ownership in Regeneron Pharmaceuticals, Inc. Marie Baccara-Dinet, Daniela Chibedi-De-Roche, and Irfan Khan are employees of Sanofi and have stock/ownership in Sanofi. Terry Jacobson is a consultant/advisory board member for Amarin, Amgen, AstraZeneca, Merck, Regeneron Pharmaceuticals, Inc., and Sanofi.
Role of the Funding Source
This work was funded by Sanofi and Regeneron Pharmaceuticals, Inc.
Electronic supplementary material
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Rosenson, R.S., Miller, K., Bayliss, M. et al. The Statin-Associated Muscle Symptom Clinical Index (SAMS-CI): Revision for Clinical Use, Content Validation, and Inter-rater Reliability. Cardiovasc Drugs Ther 31, 179–186 (2017). https://doi.org/10.1007/s10557-017-6723-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10557-017-6723-4