Quantifying a Rare Disease in Administrative Data: The Example of Calciphylaxis
- First Online:
- Cite this article as:
- Nigwekar, S.U., Solid, C.A., Ankers, E. et al. J GEN INTERN MED (2014) 29: 724. doi:10.1007/s11606-014-2910-1
- 860 Views
Calciphylaxis, a rare disease seen in chronic dialysis patients, is associated with significant morbidity and mortality. As is the case with other rare diseases, the precise epidemiology of calciphylaxis remains unknown. Absence of a unique International Classification of Diseases (ICD) code impedes its identification in large administrative databases such as the United States Renal Data System (USRDS) and hinders patient-oriented research. This study was designed to develop an algorithm to accurately identify cases of calciphylaxis and to examine its incidence and mortality.
DESIGN, PARTICIPANTS, AND MAIN MEASURES
Along with many other diagnoses, calciphylaxis is included in ICD-9 code 275.49, Other Disorders of Calcium Metabolism. Since calciphylaxis is the only disorder listed under this code that requires a skin biopsy for diagnosis, we theorized that simultaneous application of code 275.49 and skin biopsy procedure codes would accurately identify calciphylaxis cases. This novel algorithm was developed using the Partners Research Patient Data Registry (RPDR) (n = 11,451 chronic hemodialysis patients over study period January 2002 to December 2011) using natural language processing and review of medical and pathology records (the gold-standard strategy). We then applied this algorithm to the USRDS to investigate calciphylaxis incidence and mortality.
Comparison of our novel research strategy against the gold standard yielded: sensitivity 89.2 %, specificity 99.9 %, positive likelihood ratio 3,382.3, negative likelihood ratio 0.11, and area under the curve 0.96. Application of the algorithm to the USRDS identified 649 incident calciphylaxis cases over the study period. Although calciphylaxis is rare, its incidence has been increasing, with a major inflection point during 2006–2007, which corresponded with specific addition of calciphylaxis under code 275.49 in October 2006. Calciphylaxis incidence continued to rise even after limiting the study period to 2007 onwards (from 3.7 to 5.7 per 10,000 chronic hemodialysis patients; r = 0.91, p = 0.02). Mortality rates among calciphylaxis patients were noted to be 2.5–3 times higher than average mortality rates for chronic hemodialysis patients.
By developing and successfully applying a novel algorithm, we observed a significant increase in calciphylaxis incidence. Because calciphylaxis is associated with extremely high mortality, our study provides valuable information for future patient-oriented calciphylaxis research, and also serves as a template for investigating other rare diseases.
KEY WORDScalciphylaxisdialysisrenal diseaseresearch design
Calciphylaxis is classified as a rare disease by the National Institutes of Health’s Office of Rare Diseases Research;7 however, like many other rare diseases, calciphylaxis’ exact epidemiology remains unknown. Lack of data on fundamental issues such as incidence, prevalence, or mortality for any condition seriously impairs future patient-oriented translational research and limits investigators’ abilities to examine temporal trends, epidemiological associations, and development of diagnostic biomarkers and novel therapy targets. In fact, despite first descriptions of calciphylaxis over 50 years ago,8 there are persistent major knowledge gaps regarding its pathogenesis, risk factors, prognosis, and management strategies. The scientific literature on calciphylaxis consists predominantly of case reports, case series, and small cohorts, and its reported incidence and prevalence vary widely.2 To date, no effort has been made to systematically assess its epidemiology in a large administrative database. As noted in a recent editorial on calciphylaxis, “[this knowledge gap] is astounding considering chronic hemodialysis patients are one of the best and most regularly monitored patient cohorts in the world.”9
What are the barriers to investigating calciphylaxis in a large administrative database such as the United States Renal Data System (USRDS)? Like many other rare diseases, calciphylaxis does not have a unique International Classification of Disease (ICD) code, and databases such as USRDS derive data from the submitted claims according to such codes.10 Thus, even though we have a comprehensive cohort of chronic hemodialysis patients, we are limited in our ability to accurately identify calciphylaxis cases in such cohorts. We designed the present study 1) to test the operating characteristics (sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and area under the receiver operating characteristic curve) of a novel algorithm for identifying cases of calciphylaxis using administrative claims; and 2) to describe the epidemiology of calciphylaxis in the United States using a national administrative database of end-stage renal disease patients (USRDS).
Description of Databases Used for Development and Application of the Research Algorithm
Partners Research Patient Data Registry (RPDR)
RPDR serves as a central clinical data warehouse for over 1.8 million patients treated in the Partners Healthcare System, and has been described in detail.11 This data set contains information on patient demographics, diagnoses, laboratory data, and procedures, derived from a combination of claims submitted for billing purposes (including ICD-9 and CPT codes) and electronic medical records. RPDR also includes a narrative electronic provider note repository that can be queried using natural language processing software.
USRDS is the national data registry that collects, analyzes, and distributes information on the chronic kidney disease and end-stage renal disease population, including those undergoing hemodialysis or peritoneal dialysis or being considered for renal transplant.10 USRDS is funded by the National Institutes of Health and by the National Institute of Diabetes and Digestive and Kidney Diseases. This data registry contains information on patient demographics, diagnoses, procedures, treatments, and outcomes. Data are derived from the claims submitted for billing purposes based on billing codes such as ICD and CPT.
Description of Research Algorithm
Upon reviewing an ICD-9 register, we found calciphylaxis listed under ICD-9 diagnosis code 275.49,12 described as “Other Disorders of Calcium Metabolism (short description: Dis calcium metablsm). This code covers not only calciphylaxis, but also many other conditions, including chondrocalcinosis, nephrocalcinosis, vascular calcification, oxalate-related arthritis, apatite-related arthropathy, extra-skeletal calcification, intervertebral disc calcification, pseudohypoparathyroidism, pseudopseudohypoparathyroidism, and Bartter’s syndrome with hypercalciuria.
Definitive diagnosis of calciphylaxis is made by skin biopsy (either punch biopsy or excision procedure), confirming calcification in dermal arterioles along with mural thrombosis and septal panniculitis.13 Diagnostic skin procedures in the claims database are identified by ICD-9 code 86.1 or Current Procedural Terminology (CPT) codes 110xx.12,14 Although skin biopsy is part of the diagnostic work-up for calciphylaxis, it is not included in the diagnostic work-up for other disorders listed under the ICD-9 code 275.49. Therefore, we hypothesized that the simultaneous application of ICD-9 code 275.49 plus skin biopsy procedure codes (CPT codes 110xx and ICD-9 code 86.1) to claims in the USRDS would accurately identify suspected calciphylaxis cases.
Since hemodialysis is the most common dialysis modality in the United States (according to 2013 USRDS Atlas, 93 % of chronic dialysis patients were on hemodialysis in 2011, compared to 7 % on peritoneal dialysis),10 and most of the literature descriptions of calciphylaxis are in hemodialysis patients, we focused on patients using chronic hemodialysis for this study.
Development of Research Algorithm
Epidemiology of Calciphylaxis
After developing our novel research algorithm in the RPDR, we applied it to the USRDS database to estimate both calciphylaxis incidence and mortality among patients with calciphylaxis. The incidence of calciphylaxis in the USRDS was calculated by dividing the number of new cases of calciphylaxis in a given study year by the number of total chronic hemodialysis patients in the USRDS for that year. Mortality in patients with calciphylaxis was reported as deaths per 1,000 patient-years.
The following operating characteristics were computed for the research algorithm in the RPDR: sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and area under the receiver operating characteristic curve (AUC).
Spearman correlation was used to describe the trend in calciphylaxis incidence and mortality over time. Statistical assessment of the suspected inflection point in the incidence trend was conducted using simple linear regression. A binary indicator variable was created to represent years earlier than, at, or later than 2006, using calciphylaxis incidence as the outcome; the slopes of these two time periods were compared. All P values were two-sided, and values of P < 0.05 were considered statistically significant. All the analyses were conducted using SAS 9.3 version (Cary, NC).
Characteristics of the RPDR Chronic Hemodialysis Population and the Overall USRDS Chronic Dialysis Population10
RPDR (n = 11,451)
USRDS (S = 395,656)
Gender, % Male
Race, % Caucasian
Diabetes mellitus, %
Congestive heart failure, rate per 1,000 patient-years
Acute myocardial infarction, rate per 1,000 patient-years
Cerebrovascular accident or transient ischemic attack, rate per 1,000 patient-years
Development of Algorithm
The algorithm was developed in the RPDR, where we identified 74 incident cases of calciphylaxis over the pre-specified study period. From the original patient pool of 11,451 chronic hemodialysis patients in the RPDR, we identified 653 patients with “calciphylaxis” or “calcific uremic arteriolopathy” terms in their medical records, using natural language processing. Review of medical charts of these patients identified 102 patients with possible calciphylaxis. Seventy-five of these had a skin biopsy, through which the diagnosis was confirmed in 60 patients. Among the remaining 15 cases who had a skin biopsy, seven had a false negative skin biopsy (very high clinical suspicion, but the biopsy did not capture a representative dermal arteriole), and eight had a true negative skin biopsy. Seven of the 27 patients without a skin biopsy had clinical and imaging evidence for calciphylaxis.
Computation of Operating Characteristics of Novel Research Algorithm in the RPDR
Gold standard Novel algorithm
Epidemiology of Calciphylaxis
Annual Incidence of Calciphylaxis Derived from Application of Novel Research Strategy to the USRDS Database
Total chronic hemodialysis patients
Incident calciphylaxis cases
Annual incidence per 10,000 chronic hemodialysis patients
Calciphylaxis Incidence and Outcomes in the USRDS Database
Incidence of calciphylaxis steadily increased from year 2007 to 2011, as shown in the circled portion of Fig. 3. This trend was statistically significant (r = 0.91, p = 0.02). The most recent annual incidence of calciphylaxis was noted to be 5.7 cases per 10,000 chronic hemodialysis patients.
We have described development and application of an innovative research algorithm to investigate the epidemiology of calciphylaxis, a rare disease with significant morbidity and mortality seen in chronic hemodialysis patients. Despite the lack of a unique ICD code for calciphylaxis, by developing a novel algorithm that incorporates the simultaneous application of different billing codes to the largest administrative database for chronic hemodialysis patients (USRDS), we effectively described the increasing incidence of calciphylaxis in recent years (3.7 to 5.7 per 10,000 chronic hemodialysis patients), and strikingly, a 2.5-fold to threefold higher mortality rate among calciphylaxis patients compared to other chronic hemodialysis patients.
Thinking outside the box. Rare disease investigators must frequently create innovative approaches because of the challenges inherent to rare-disease research. In terms of identifying rare disease in large administrative databases, we believe that lack of unique billing codes is not necessarily an insurmountable barrier. We have demonstrated this in the case of calciphylaxis and trust that application of similar strategies for other diseases would be successful. In fact, a recent report describes a similar approach for esophageal atresia and trachea-esophageal fistula (EA/TEF), a rare complex condition in children.15 By simultaneous application of an ICD code for medical diagnosis with procedure codes, Sulkowski et al. accurately identified cases of EA/TEF in the Pediatric Health Information System, a database that contains information about each inpatient hospitalization at more than 40 freestanding children’s hospitals across the United States.
Efficient approaches to case identification in database research. Novel tools such as natural language processing can significantly improve the efficiency of case detection in large databases. For our research algorithm, we applied natural language processing to narrow down the number of medical records to be reviewed from 11,451 to 653. A number of recent publications have outlined innovative methods incorporating natural language processing into database research, a strategy likely to play a significant role in rare disease research in the genomics era.16–18
Meticulous interpretation of results. Our algorithm performed very well in the RPDR; however, upon application to a substantially larger database, we discovered a limitation of the algorithm that was produced by a change in the ICD-9 coding method. We explored reasons for this inflection in the published literature by considering possibilities such as practice pattern changes for hemodialysis patients and temporal changes in demographics and clinical characteristics.19–24 No such change could explain the magnitude of inflection noted between the study years 2006 and 2007. Meticulous review of ICD code revisions revealed a likely explanation: calciphylaxis was specifically added under ICD 275.49 in October 2006. After identifying this important limitation, we decided to focus on study years 2007 onwards to derive calciphylaxis incidence and mortality. The relatively smaller sample size of our development database compared to the USRDS, as well as differences in billing patterns between academic institutions and community settings, may account for our inability to identify this issue during the development phase.
Collaborative approach. Investigations in rare diseases are possible only if different disciplines converge and establish collaborative networks. In our case, we established collaborations across nephrology, cardiology, and bioinformatics, and also enlisted the help of our local multi-disciplinary calciphylaxis team to accurately identify calciphylaxis cases in the RPDR.
Cost-efficient approach. In the current research environment of tightly constrained funding, it is important to appreciate that approaches such as establishing large patient registries for rare diseases face significant challenges. In that regard, it is important to make use of existing resources (e.g., USRDS) by developing innovative strategies such as the one described here.
The ability to accurately identify calciphylaxis cases in the USRDS now prepares investigators to examine a number of risk factors that have been previously described only in smaller studies.3–5,25,26 Establishing temporal trends in calciphylaxis allows rigorous pharmacovigilance studies focused on therapeutics that are known to induce vascular calcification, e.g., vitamin D analogues, warfarin, calcium-based phosphate binders, and iron.27–29 Accurate identification of calciphylaxis cases in a large administrative database such as the USRDS also offers the opportunity to study whether certain commonly used therapies in dialysis patients, such as cinacalcet, sevelamer, or statins, offer any protection against calciphylaxis.25,30,31 Our study also provides fundamental epidemiological data, as future studies related to biomarkers, therapeutic target discovery, and clinical trials are planned for calciphylaxis.
Because we focused specifically on chronic hemodialysis patients, our findings may not be generalizable to calciphylaxis patients on chronic peritoneal dialysis, renal transplant recipients, or patients with normal renal function who develop calciphylaxis.32–35 The small number of calciphylaxis cases in the RPDR limited our ability to split the available cohort into development and validation data sets. We were unable to independently validate our novel research algorithm in the USRDS, since medical records and skin biopsy reports are not included in this administrative database. Further validation of such a research algorithm in an independent database is necessary to confirm our findings. Although our study showed a trend towards reduction in mortality rates among calciphylaxis patients in recent years, it is important to keep in mind that mortality rates for chronic hemodialysis patients with calciphylaxis are much higher than among chronic hemodialysis patients without calciphylaxis. Our study cannot explain if the trend in mortality reduction is a result of treatments such as sodium thiosulfate, or whether it is due to increased awareness and early recognition of the disease.36,37 A prospective study is needed to better understand calciphylaxis mortality rates and to assess the effectiveness of interventions to treat calciphylaxis. Similarly, unmasking detection signal bias and the possibility that increased awareness produced an apparent incidence increase cannot be entirely ruled out by any study of this nature. Since our research algorithm included skin biopsy as a pre-requisite to identify calciphylaxis cases, patients diagnosed based entirely on clinical suspicion were not eligible. This certainly leads us to believe that our incidence numbers are underestimates of true calciphylaxis incidence. However, a number of skin conditions in dialysis patients (e.g., necrotizing infections, vasculitis) may present with similar clinical features; in that regard, skin biopsy is a highly valuable tool in establishing the correct diagnosis. The accuracy of the algorithm will diminish if biopsies are ordered for cases with lower pre-test probability for calciphylaxis diagnosis.
In conclusion, we have developed a novel research algorithm to accurately identify calciphylaxis cases in a large administrative database. Calciphylaxis is rare, but its incidence appears to be rising in the chronic hemodialysis population and is associated with extremely high mortality rates. Our study provides valuable information for future patient-oriented research in calciphylaxis and also serves as a template for investigations in other rare diseases.
Conception and design: Sagar Nigwekar, Ravi Thadhani, Charles Herzog
Analysis and interpretation of data: Sagar Nigwekar, Craig Solid, William Eggert, Rajeev Malhotra, Alexander Turchin, Elizabeth Ankers, Ravi Thadhani, Charles Herzog
Drafting of the article: Sagar Nigwekar, Elizabeth Ankers
Critical revision of the article: Craig Solid, William Eggert, Elizabeth Ankers, Rajeev Malhotra, Alexander Turchin, Ravi Thadhani, Charles Herzog
Final approval of the article: Sagar Nigwekar, Craig Solid, William Eggert, Elizabeth Ankers, Rajeev Malhotra, Alexander Turchin, Ravi Thadhani, Charles Herzog
Administrative, technical and logistic support: Sagar Nigwekar, Elizabeth Ankers, Rajeev Malhotra
Collection and assembly of data: Sagar Nigwekar, Alexander Turchin, Craig Solid, William Eggert, Rajeev Malhotra, Ravi Thadhani, Charles Herzog
Sagar Nigwekar is supported by the Clinical Scientist in Nephrology Fellowship award from the American Kidney Fund and by Sanofi-Aventis Fellowship award. Ravi Thadhani is supported by National Institute of Health grants DK094872 and DK094486. Dr Herzog served as the director of the USRDS Cardiovascular Special Study Center through February 2014 and was supported by Contract No. HHSN267200715003C (National Institute of Diabetes and Digestive and Kidney Diseases, National Institutes of Health, Bethesda, Maryland).
Rajeev Malhotra was supported by the American Heart Association Fellow-to-Faculty Award #11FTF7290032 and is supported by the National Heart, Lung, and Blood Institute (K08HL111210).
Portion of this work was presented as an oral abstract at the American Society of Nephrology’s annual conference in Atlanta, GA in November 2013.
The authors would like to thank Thadhani Lab members for constructive input on this manuscript and Massachusetts General Hospital’s Multi-disciplinary Calciphylaxis Team for accurate identification of calciphylaxis cases in the RPDR. Authors would also like to thank Julia Wenger for statistical analysis help while revising this manuscript.
Data for this analysis were provided by the USRDS. The interpretation and reporting of these data are the responsibility of the authors and in no way should be seen as an official policy or interpretation of the US government.
Conflict of Interest
Sagar Nigwekar reports receiving lecture honoraria from Sanofi-Aventis and is a recipient of Nephrology Fellowship Award from Sanofi-Aventis.
Alexander Turchin reports receiving grant support from Merck, Sharp and Dohme, Inc., serves on the scientific Advisory Board of Monarch Medical Technologies, and is a consultant to GNS Healthcare.
Ravi Thadhani is a consultant to Fresenius Medical Care North America and has received a research grant from Abbott Laboratories.
Charles Herzog is a consultant to Abbott Laboratories, Abbvie, and Amgen. He also reports having received lecture honoraria from Fresenius Medical Care North America.