Rheumatology International

, Volume 38, Issue 9, pp 1651–1660 | Cite as

Performance of the 2016 ACR-EULAR classification criteria for primary Sjogren’s syndrome in a Korean cohort

  • Jennifer Lee
  • Jung Hee Koh
  • Ji-Won Kim
  • Yoon-Kyoung Sung
  • Shin-Seok Lee
  • Jung Yoon Choe
  • Seung-Cheol Shim
  • Hyun-Sook Kim
  • Hae-Rim Kim
  • Ji-Min Kim
  • Sung Ryul Kwon
  • Hyun-Ok Kim
  • Kichul Shin
  • Chang Hoon Lee
  • So-Hyang Chung
  • Seung-Ki Kwok
  • Ji Hyeon Ju
  • Sung-Hwan ParkEmail author
Classification Criteria


This study compared the performance of the newly proposed 2016 American College of Rheumatology (ACR)/European League Against Rheumatism (EULAR) criteria to the 2002 American–European Consensus Group (AECG) and 2012 ACR classification criteria for primary Sjogren’s syndrome (pSS) in well-characterized Korean patients. Patients with pSS from 12 university-affiliated hospitals in Korea were enrolled from October 2013 to January 2017. Clinical and laboratory data were reviewed. For the validation set, patients who underwent evaluation tests to rule out pSS at Seoul St. Mary’s hospital from November 2016 to December 2017 were analyzed. Baseline registry data were available in 458 patients, and 328 patients had sufficient data to determine the fulfillment of each criteria set. All three sets of criteria were met by 307 patients (93.6%). The newly proposed 2016 ACR/EULAR criteria were met by 325 patients (99.1%). The 2002 AECG and 2012 ACR criteria were met by 325 (99.1%) and 310 patients (94.5%), respectively. In a validation cohort consisting of 161 patients with pSS-related symptoms/signs, the sensitivity and specificity of the 2016 ACR/EULAR criteria were 100% [95% confidence interval (CI), 96.11–100.00] and 81.8% [95% CI, 76.15–94.26], respectively. Agreement between the 2016 criteria and 2012 or 2002 criteria was high (Cohen’s kappa 0.736 and 0.769, respectively). The newly proposed 2016 ACR/EULAR criteria were met by most patients diagnosed with pSS according to previous criteria and showed higher sensitivity and lower specificity compared with both previous criteria sets.


Classification criteria Sjogren’s syndrome 



This study was supported by a grant of the Korean Health Technology R&D Project, Ministry of Health & Welfare, Republic of Korea (HI13C0016).

Author contributions

JL and SHP contributed to the study design, data collection, data analysis, interpretation of the results, and preparation of the manuscript. JHK, JWK, YKS, SSL, JYC, SCS, HSK, HRK, JMK, SRK, HOK, KS, CHL, SHC, SKK, and JHJ contributed to data collection. All the authors read and approved the final manuscript.

Compliance with ethical standards

Conflict of interest

Jennifer Lee, Jung Hee Koh, Ji-Won Kim, Yoon-Kyoung Sung, Shin-Seok Lee, Jung Yoon Choe, Seung-Cheol Shim, Hyun-Sook Kim, Hae-Rim Kim, Ji-Min Kim, Sung Ryul Kwon, Hyun-Ok Kim, Kichul Shin, Chang Hoon Lee, So-Hyang Chung, Seung-Ki Kwok, Ji Hyeon Ju, and Sung-Hwan Park declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.


  1. 1.
    Fox RI (2005) Sjögren’s syndrome. Lancet 366:321–331CrossRefPubMedGoogle Scholar
  2. 2.
    Koh JH, Lee J, Chung SH, Kwok SK, Park SH (2015) Relation of autoimmune cytopenia to glandular and systemic manifestations in primary Sjögren Syndrome: analysis of 113 Korean patients. J Rheumatol 42:1817–1824CrossRefPubMedGoogle Scholar
  3. 3.
    Li X, Xu B, Ma Y, Li X, Cheng Q, Wang X, Wang G, Qian L, Wei L (2015) Clinical and laboratory profiles of primary Sjogren’s syndrome in a Chinese population: a retrospective analysis of 315 patients. Int J Rheum Dis 18:439–446CrossRefPubMedGoogle Scholar
  4. 4.
    Ramos-Casals M, Brito-Zerón P, Seror R, Bootsma H, Bowman SJ, Dörner T, Gottenberg JE, Mariette X, Theander E, Bombardieri S, De Vita S, Mandl T, Ng WF, Kruize A, Tzioufas A, Vitali C, EULAR Sjögren Syndrome Task Force (2015) Characterization of systemic disease in primary Sjögren’s syndrome: EULAR-SS Task Force recommendations for articular, cutaneous, pulmonary and renal involvements. Rheumatology 54:2230–2238PubMedGoogle Scholar
  5. 5.
    Reina D, Roig Vilaseca D, Torrente-Segarra V, Cerdà D, Castellvi I, Díaz Torné C, Moreno M, Narváez J, Ortiz V, Blavia R, Martín-Baranera M, Corominas H (2016) Sjögren’s syndrome-associated interstitial lung disease: a multicenter study. Reumatol Clin 12:201–205CrossRefPubMedGoogle Scholar
  6. 6.
    Arends S, Meiners PM, Moerman RV, Kroese FG, Brouwer E, Spijkervet FK, Vissink A, Bootsma H (2017) Physical fatigue characterises patient experience of primary Sjögren’s syndrome. Clin Exp Rheumatol 35:255–261PubMedGoogle Scholar
  7. 7.
    Carsons SE, Vivino FB, Parke A, Carteron N, Sankar V, Brasington R, Brennan MT, Ehlers W, Fox R, Scofield H, Hammitt KM, Birnbaum J, Kassan S, Mandel S (2017) Treatment Guidelines for Rheumatologic Manifestations of Sjögren’s Syndrome: use of biologic agents, management of fatigue, and inflammatory musculoskeletal pain. Arthritis Care Res (Hoboken) 69:517–527CrossRefGoogle Scholar
  8. 8.
    Howard Tripp N, Tarn J, Natasari A, Gillespie C, Mitchell S, Hackett KL, Bowman SJ, Price E, Pease CT, Emery P, Lanyon P, Hunter J, Gupta M, Bombardieri M, Sutcliffe N, Pitzalis C, McLaren J, Cooper A, Regan M, Giles I, Isenberg DA, Saravanan V, Coady D, Dasgupta B, McHugh N, Young-Min S, Moots R, Gendi N, Akil M, Griffiths B, Lendrem DW, Ng WF (2016) Fatigue in primary Sjögren’s syndrome is associated with lower levels of proinflammatory cytokines. RMD Open 2:e000282CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Cornec D, Devauchelle-Pensec V, Mariette X, Jousse-Joulin S, Berthelot JM, Perdriger A, Puéchal X, Le Guern V, Sibilia J, Gottenberg JE, Chiche L, Hachulla E, Yves Hatron P, Goeb V, Hayem G, Morel J, Zarnitsky C, Dubost JJ, Saliou P, Pers JO, Seror R, Saraux A (2017) Severe health-related quality of life impairment in active primary Sjögren’s syndrome and patient-reported outcomes: data from a large therapeutic trial. Arthritis Care Res (Hoboken) 69:528–535CrossRefGoogle Scholar
  10. 10.
    Cui Y, Li L, Yin R, Zhao Q, Chen S, Zhang Q, Shen B (2017) Depression in primary Sjögren’s syndrome: a systematic review and meta-analysis. Psychol Health Med 23:198–209CrossRefPubMedGoogle Scholar
  11. 11.
    Solans-Laqué R, López-Hernandez A, Bosch-Gil JA, Palacios A, Campillo M, Vilardell-Tarres M (2011) Risk, predictors, and clinical characteristics of lymphoma development in primary Sjögren’s syndrome. Semin Arthritis Rheum 41:415–423CrossRefPubMedGoogle Scholar
  12. 12.
    Theander E, Henriksson G, Ljungberg O, Mandl T, Manthorpe R, Jacobsson LT (2006) Lymphoma and other malignancies in primary Sjögren’s syndrome: a cohort study on cancer incidence and lymphoma predictors. Ann Rheum Dis 65:796–803CrossRefPubMedGoogle Scholar
  13. 13.
    Kim HJ, Kim KH, Hann HJ, Han S, Kim Y, Lee SH, Kim DS, Ahn HS (2017) Incidence, mortality, and causes of death in physician-diagnosed primary Sjögren’s syndrome in Korea: a nationwide, population-based study. Semin Arthritis Rheum 47:222–227CrossRefPubMedGoogle Scholar
  14. 14.
    Letaief H, Lukas C, Barnetche T, Gaujoux-Viala C, Combe B, Morel J (2018) Efficacy and safety of biological DMARDs modulating B cells in primary Sjögren’s syndrome: systematic review and meta-analysis. Joint Bone Spine 85:15–22CrossRefPubMedGoogle Scholar
  15. 15.
    Verstappen GM, Meiners PM, Corneth OBJ, Visser A, Arends S, Abdulahad WH, Hendriks RW, Vissink A, Kroese FGM, Bootsma H (2017) Attenuation of follicular helper T cell-dependent B cell hyperactivity by abatacept treatment in primary Sjögren’s syndrome. Arthritis Rheumatol 69:1850–1861CrossRefPubMedGoogle Scholar
  16. 16.
    Bloch KJ, Buchanan WW, Wohl MJ, Bunim JJ (1965) Sjoegren’s syndrome. a clinical, pathological, and SEROLOGICAL STUDY of sixty-two cases. Medicine (Baltimore) 44:187–231CrossRefGoogle Scholar
  17. 17.
    Shearn MA (1971) Sjögren’s syndrome. In: Smith LH Jr (ed) Major problems in internal medicine, vol 2. WB Saunders, PhiladelphiaGoogle Scholar
  18. 18.
    Daniels TE, Silverman S Jr, Michalski JP, Greenspan JS, Sylvester RA, Talal N (1975) The oral component of Sjögren’s syndrome. Oral Surg Oral Med Oral Pathol 39:875–885CrossRefPubMedGoogle Scholar
  19. 19.
    Ohfuji T (1977) Review on research reports; Annual report of the ministry of Health and Welfare. Sjögren’s Disease Research Committee, JapanGoogle Scholar
  20. 20.
    Manthorpe R, Frost-Larsen K, Isager H, Prause JU (1981) Sjögren’s syndrome. A review with emphasis on immunological features. Allergy 36:139–153CrossRefPubMedGoogle Scholar
  21. 21.
    Fox RI, Robinson CA, Curd JG, Kozin F, Howell FV (1986) Sjögren’s syndrome. Proposed criteria for classification. Arthritis Rheum 29:577–585CrossRefPubMedGoogle Scholar
  22. 22.
    Homma M, Tojo T, Akizuki M, Yamagata H (1986) Criteria for Sjögren’s syndrome in Japan. Scand J Rheumatol Suppl 61:26–27PubMedGoogle Scholar
  23. 23.
    Skopouli FN, Drosos AA, Papaioannou T, Moutsopoulos HM (1986) Preliminary diagnostic criteria for Sjögren’s syndrome. Scand J Rheumatol Suppl 61:22–25PubMedGoogle Scholar
  24. 24.
    Vitali C, Bombardieri S, Moutsopoulos HM, Balestrieri G, Bencivelli W, Bernstein RM, Bjerrum KB, Braga S, Coll J, de Vita S, Drosos AA, Ehrenfeld M, Hatron PY, Hay EM, Isenberg DA, Janin A, Kalden JR, Kater L, Konttinen YT, Maddison PJ, Maini RN, Manthorpe R, Meyer O, Ostuni P, Pennec Y, Prause JU, Richards A, Sauvezie B, Schiødt M, Sciuto M, Scully C, Shoenfeld Y, Skopouli FN, Smolen JS, Snaith ML, Tishler M, Todesco S, Valesini G, Venables PJ, Wattiaux MJ, Youinou P (1993) Preliminary criteria for the classification of Sjögren’s syndrome. Results of a prospective concerted action supported by the European Community. Arthritis Rheum 36:340–347CrossRefPubMedGoogle Scholar
  25. 25.
    Vitali C, Bombardieri S, Jonsson R, Moutsopoulos HM, Alexander EL, Carsons SE, Daniels TE, Fox PC, Fox RI, Kassan SS, Pillemer SR, Talal N, Weisman MH, European Study Group on Classification Criteria for Sjögren’s Syndrome (2002) Classification criteria for Sjögren’s syndrome: a revised version of the European criteria proposed by the American-European Consensus Group. Ann Rheum Dis 61:554–558CrossRefPubMedPubMedCentralGoogle Scholar
  26. 26.
    Fujibayashi T, Sugai S, Miyasaka N, Hayashi Y, Tsubota K (2004) Revised Japanese criteria for Sjögren’s syndrome (1999): availability and validity. Mod Rheumatol 14:425–434CrossRefPubMedGoogle Scholar
  27. 27.
    Shiboski SC, Shiboski CH, Criswell L, Baer A, Challacombe S, Lanfranchi H, Schiødt M, Umehara H, Vivino F, Zhao Y, Dong Y, Greenspan D, Heidenreich AM, Helin P, Kirkham B, Kitagawa K, Larkin G, Li M, Lietman T, Lindegaard J, McNamara N, Sack K, Shirlaw P, Sugai S, Vollenweider C, Whitcher J, Wu A, Zhang S, Zhang W, Greenspan J, Daniels T, Sjögren’s International Collaborative Clinical Alliance (SICCA) Research Groups (2012) American College of Rheumatology classification criteria for Sjögren’s syndrome: a data-driven, expert consensus approach in the Sjögren’s International Collaborative Clinical Alliance cohort. Arthritis Care Res (Hoboken) 64:475–487CrossRefGoogle Scholar
  28. 28.
    Shiboski CH, Shiboski SC, Seror R, Criswell LA, Labetoulle M, Lietman TM, Rasmussen A, Scofield H, Vitali C, Bowman SJ, Mariette X, International Sjögren’s Syndrome Criteria Working Group (2017) 2016 American College of Rheumatology/European League Against Rheumatism classification criteria for primary Sjögren’s syndrome: a consensus and data-driven methodology involving three international patient cohorts. Arthritis Rheumatol 69:35–45CrossRefPubMedGoogle Scholar
  29. 29.
    Tsuboi H, Hagiwara S, Asashima H, Takahashi H, Hirota T, Noma H, Umehara H, Kawakami A, Nakamura H, Sano H, Tsubota K, Ogawa Y, Takamura E, Saito I, Inoue H, Nakamura S, Moriyama M, Takeuchi T, Tanaka Y, Hirata S, Mimori T, Matsumoto I, Sumida T (2017) Comparison of performance of the 2016 ACR-EULAR classification criteria for primary Sjögren’s syndrome with other sets of criteria in Japanese patients. Ann Rheum Dis 76:1980–1985CrossRefPubMedPubMedCentralGoogle Scholar
  30. 30.
    Rasmussen A, Ice JA, Li H, Grundahl K, Kelly JA, Radfar L, Stone DU, Hefner KS, Anaya JM, Rohrer M, Gopalakrishnan R, Houston GD, Lewis DM, Chodosh J, Harley JB, Hughes P, Maier-Moore JS, Montgomery CG, Rhodus NL, Farris AD, Segal BM, Jonsson R, Lessard CJ, Scofield RH, Sivils KL (2014) Comparison of the American-European Consensus Group Sjogren’s syndrome classification criteria to newly proposed American College of Rheumatology criteria in a large, carefully characterised sicca cohort. Ann Rheum Dis 73:31–38CrossRefPubMedGoogle Scholar
  31. 31.
    Cornec D, Saraux A, Cochener B, Pers JO, Jousse-Joulin S, Renaudineau Y, Marhadour T, Devauchelle-Pensec V (2014) Level of agreement between 2002 American-European Consensus Group and 2012 American College of Rheumatology classification criteria for Sjögren’s syndrome and reasons for discrepancies. Arthritis Res Ther 16:R74CrossRefPubMedPubMedCentralGoogle Scholar
  32. 32.
    Shiboski CH, Shiboski SC, Seror R, Criswell LA, Labetoulle M, Lietman TM, Rasmussen A, Scofield H, Vitali C, Bowman SJ, Mariette X, International Sjögren’s Syndrome Criteria Working Group (2017) 2016 American College of Rheumatology/European League Against Rheumatism classification criteria for primary Sjögren's syndrome: a consensus and data-driven methodology involving three international patient cohorts. Ann Rheum Dis 76(1):9–16CrossRefPubMedGoogle Scholar
  33. 33.
    Shiboski CH, Baer AN, Shiboski SC, Lam M, Challacombe S, Lanfranchi HE, Schiødt M, Shirlaw P, Srinivasan M, Umehara H, Vivino FB, Akpek E, Bunya V, Vollenweider CF, Greenspan JS, Daniels TE, Criswell LA, Sjögren’s International Collaborative Clinical Alliance Research Groups (2018) Natural History and Predictors of Progression to Sjögren’s Syndrome Among Participants of the Sjögren’s International Collaborative Clinical Alliance Registry. Arthritis Care Res (Hoboken) 70:284–294CrossRefGoogle Scholar
  34. 34.
    Brito-Zerón P, Acar-Denizli N, Zeher M, Rasmussen A, Seror R, Theander E, Li X, Baldini C, Gottenberg JE, Danda D, Quartuccio L, Priori R, Hernandez-Molina G, Kruize AA, Valim V, Kvarnstrom M, Sene D, Gerli R, Praprotnik S, Isenberg D, Solans R, Rischmueller M, Kwok SK, Nordmark G, Suzuki Y, Giacomelli R, Devauchelle-Pensec V, Bombardieri M, Hofauer B, Bootsma H, Brun JG, Fraile G, Carsons SE, Gheita TA, Morel J, Vollenveider C, Atzeni F, Retamozo S, Horvath IF, Sivils K, Mandl T, Sandhya P, De Vita S, Sanchez-Guerrero J, van der Heijden E, Trevisani VFM, Wahren-Herlenius M, Mariette X, Ramos-Casals M, EULAR-SS Task Force Big Data Consortium (2017) Influence of geolocation and ethnicity on the phenotypic expression of primary Sjögren’s syndrome at diagnosis in 8310 patients: a cross-sectional study from the Big Data Sjögren Project Consortium. Ann Rheum Dis 76:1042–1050CrossRefPubMedGoogle Scholar
  35. 35.
    Danda D, Sharma R, Truong D, Koelsch KA, Kurien BT, Bagavant H, Deshmukh U, Kaufman CE, Lewis DM, Stone DU, Radfar L, Rasmussen A, Sivils KL, Scofield RH (2017) Anti-La positive, anti-Ro negative subset of primary Sjögren’s syndrome: anti-La is a reality but is the disease? Clin Exp Rheumatol 35:438–444PubMedGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Division of Rheumatology, Department of Internal Medicine, Seoul St. Mary’s Hospital, College of MedicineThe Catholic University of KoreaSeoulRepublic of Korea
  2. 2.Department of RheumatologyHanyang University Hospital for Rheumatic DiseasesSeoulRepublic of Korea
  3. 3.Department of RheumatologyChonnam National University Medical School and HospitalGwangjuRepublic of Korea
  4. 4.Division of RheumatologyDaegu Catholic University Medical CenterDaeguRepublic of Korea
  5. 5.Division of Rheumatology, Department of Internal Medicine, School of Medicine, Chungnam National UniversityChungnam National University HospitalDaejeonRepublic of Korea
  6. 6.Division of Rheumatology, Department of Internal Medicine, Soonchunhyang University Seoul HospitalSoonchunhyang University College of MedicineSeoulRepublic of Korea
  7. 7.Division of Rheumatology, Department of Internal MedicineKonkuk University School of MedicineSeoulRepublic of Korea
  8. 8.Division of Rheumatology, Department of Internal MedicineKeimyung University Dongsan Medical CentreDaeguRepublic of Korea
  9. 9.Division of Rheumatology, Department of Internal MedicineInha University School of MedicineIncheonRepublic of Korea
  10. 10.Division of Rheumatology, Department of Internal MedicineGyeongsang National University Changwon HospitalChangwonRepublic of Korea
  11. 11.Division of Rheumatology, Department of Internal MedicineSNU Boramae Medical CenterSeoulRepublic of Korea
  12. 12.Department of Internal MedicineWonkwang University College of MedicineIksanRepublic of Korea
  13. 13.Department of Ophthalmology and Visual Science, Seoul St Mary’s Hospital, College of MedicineThe Catholic University of KoreaSeoulRepublic of Korea

Personalised recommendations