Skip to main content

The burden of the variability introduced by the HEp-2 assay kit and the CAD system in ANA indirect immunofluorescence test

Immunologic Research volume 65, pages 345–354 (2017)Cite this article

Abstract

According to the recent recommendations of the American College of Rheumatology, ANA Task Force, IIF technique should be considered the gold standard in antinuclear antibodies (ANAs) testing. To overcome the lack of standardization, biomedical industries have developed several computer-aided diagnosis (CAD) systems. Two hundred and sixty-one consecutive samples with suspected autoimmune diseases were tested for ANA by means of IIF on routinely HEp-2 assay kit (Euroimmun AG). Assignment of result was made if consensus for positive/negative was reached by at least 2 out of 3 expert physicians. ANA-IIF was also carried out using 3 CAD systems: Zenit G-Sight (n = 84), Helios (n = 85) and NOVA View (n = 92); human evaluation was repeated on the same substrate of each CAD system (Immco, Aesku and Inova HEp-2 cells, respectively). To anonymize the results, we randomly named these three systems as A, B and C. We ran a statistical analysis computing several measures of agreement between the ratings, and we also improved the evaluation by using the Wilcoxon’s test for nonparametric data. Agreement between the human readings on routinely HEp-2 assay kit and human readings on CAD HEp-2 assay was substantial for A (k = 0.82) and B (k = 0.72), and almost perfect for C (k = 0.89). Such readings were statistically different only in case A. Comparing experts’ readings with the readings of CAD systems, when the samples were prepared using CAD HEp-2 assay kits, we found almost perfect agreement for B and C (k = 0.86; k = 0.82) and substantial agreement for A (k = 0.73). Again, human and CAD readings were statistically different only in A. When we compared the readings of medical experts on routinely HEp-2 assay kit with the output of the CAD systems that worked using their own slides, we found substantial agreement for all the systems (A: k = 0.62; B: k = 0.65; C: k = 0.71). Such readings were not statistically different. The change of the assay kit and/or the introduction of a CAD system affect the laboratory reporting, with an evident impact on the autoimmune laboratory workflow. The CAD systems may represent one of the most important novel elements of harmonization in the autoimmunity field, reducing intra- and inter-laboratory variability in a new vision of the diagnostic autoimmune platform.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Abbreviations

ACR:

American College of Rheumatology

ANA:

Antinuclear antibodies

CAD:

Computer-aided diagnosis

HEp-2:

Human epithelial cells

IIF:

Indirect immunofluorescence

IIM:

Idiopathic inflammatory myopathies

FITC:

Fluorescein isothiocyanate

SARD:

Systemic rheumatic diseases

SjS:

Sjögren’s syndrome

SLE:

Systemic lupus erythematosus

SSc:

Systemic sclerosis

References

  1. Solomon DH, Kavanaugh AJ, Schur PH, American College of Rheumatology Ad Hoc Committee on Immunologic Testing Guidelines. Evidence based guidelines for the use of immunologic tests: antinuclear antibody testing. Arthritis Rheum. 2002;47(4):434–44.

    Article  PubMed  Google Scholar 

  2. Fritzler MJ. Challenges to the use of autoantibodies as predictors of disease onset, diagnosis and outcomes. Autoimmun Rev. 2008;7(8):616–20.

    CAS  Article  PubMed  Google Scholar 

  3. Tozzoli R, Bizzaro N, Tonutti E, Villalta D, Bassetti D, Manoni F, Piazza A, Pradella M, Rizzotti P, Italian Society of Laboratory Medicine Study Group on the Diagnosis of Autoimmune Diseases. Guidelines for the laboratory use of autoantibody tests in the diagnosis and monitoring of autoimmune rheumatic diseases. Am J Clin Pathol. 2002;117:316–24.

    Article  PubMed  Google Scholar 

  4. Brand F, Martin F, Philipp S, Rößler J, Hansen B, Anderer U. Difference in fluorescence pattern of cytoplasmic and nuclear antigens in cultivated human cells dependent on the applied fixation procedure. Cell Prolif. 2005;38:205.

    Google Scholar 

  5. Wiik A, Hoier-Madsen M, Forslid J, Charles P, Meyrowitsch J. Antinuclear antibodies: a contemporary nomenclature using HEp-2 cells. J Autoimmun. 2010;35(3):276–90.

    CAS  Article  PubMed  Google Scholar 

  6. Hiemann R, Büttner T, Krieger T, Roggenbuck D, Sack U, Conrad K. Challenges of 417 automated screening and differentiation of non-organ specific autoantibodies 418 on HEp-2 cells. Autoimmun Rev. 2009;9:17–22.

    CAS  Article  PubMed  Google Scholar 

  7. Fenger M, Wiik A, Høier-Madsen M, Lykkegaard JJ, Rozenfeld T, Hansen MS, et al. Detection of antinuclear antibodies by solid-phase immunoassays and immunofluorescence analysis. Clin Chem. 2004;50(11):2141–7.

    CAS  Article  PubMed  Google Scholar 

  8. Pham BN, Albarede S, Guyard A, Burg E, Maisonneuve P. Impact of external quality assessment on antinuclear antibody detection performance. Lupus. 2005;14:113–9.

    CAS  Article  PubMed  Google Scholar 

  9. Bizzaro N, Tozzoli R, Tonutti E, Piazza A, Manoni F, Ghirardello A, et al. Variability between methods to determine ANA, anti-dsDNA and anti-ENA autoantibodies: a collaborative study with the biomedical industry. J Immunol Methods. 1998;219:99–107.

    CAS  Article  PubMed  Google Scholar 

  10. Song L, Hennink EJ, Young IT, Tanke HJ. Photobleaching kinetics of fluorescein in quantitative fluorescence microscopy. Biophys J. 1995;68:2588–600.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  11. Marcolongo R, Ruffatti A, Morozzi G. Presentazione linee guida del Forum Interdisciplinare per la Ricerca sulle Malattie Autoimmuni (F.I.R.M.A.). Reumatismo. 2003;55:9–21.

    Google Scholar 

  12. Kavanaugh A, Tomar R, Reveille J, Solomon DH, Homburger HA, American College of Pathologists. Guidelines for clinical use of the antinuclear antibody test and tests for specific autoantibodies to nuclear antigens. Arch Pathol Lab Med. 2000;124(1):71–81.

    CAS  PubMed  Google Scholar 

  13. Sack U, Conrad K, Csernok E, Frank I, Hiepe F, Krieger T, et al. Autoantibody detection using indirect immunofluorescence on HEp-2 cells, German EASI (European Autoimmunity Standardization Initiative). Ann N Y Acad Sci. 2009;1173:166–73.

    CAS  Article  PubMed  Google Scholar 

  14. Copple SS, Giles SR, Jaskowski TD, Gardiner AE, Wilson AM, Hill HR. Screening for IgG antinuclear autoantibodies by HEp-2 indirect fluorescent antibody assays and the need for standardization. Am J Clin Pathol. 2012;137(5):825–30.

    Article  PubMed  Google Scholar 

  15. Burlingame RW, Peebles C. Detection of antibodies. In: Pollard KM, editor. Autoantibodies and autoimmunity: molecular mechanisms in health and disease. Weinheim, Germany: Wiley-VCH; 2006. p. 159–88.

    Google Scholar 

  16. Egner W. The use of laboratory tests in the diagnosis of SLE. J Clin Pathol. 2000;53(6):424–32.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  17. Fenger M, Wiik A, Høier-Madsen M, Lykkegaard JJ, Rozenfeld T, Hansen MS, et al. Detection of antinuclear antibodies by solid-phase immunoassays and immunofluorescence analysis. Clin Chem. 2004;50:2141–7.

    CAS  Article  PubMed  Google Scholar 

  18. Bradwell AR, Hughes RG, Karim AR. In: Detrick B, Hamilton RG, Folds JD, editors. Immunofluorescent antinuclear antibody tests in Manual of Clinical Laboratory Immunology. 7th ed. Washington: ASM press; 2006. p. 995–1006.

    Google Scholar 

  19. Emlen W, O’neill L. Clinical significance of antinuclear antibodies: comparison of detection with immunofluorescence and enzyme-linked immunosorbent assays. Arthritis Rheum. 1997;40(9):1612–8.

    CAS  Article  PubMed  Google Scholar 

  20. Tonutti E, Bassetti D, Piazza A, Visentini D, Poletto M, Bassetto F, et al. Diagnostic accuracy of ELISA methods as an alternative screening test to indirect immunofluorescence for the detection of antinuclear antibodies. Evaluation of Five Commercial Kits. Autoimmunity. 2004;37:171–6.

    CAS  Article  Google Scholar 

  21. Tan EM, Smolen JS, McDougal JS, Butcher BT, Conn D, Dawkins R, et al. A critical evaluation of enzyme immunoassays for detection of antinuclear autoantibodies of defined specificities. I. Precision, sensitivity, and specificity. Arthritis Rheum. 1999;42:455–64.

    CAS  Article  PubMed  Google Scholar 

  22. Swaak AJ. Diagnostic significance of antinuclear antibodies in clinical practice. Ned Tijdschr Geneeskd. 2000;144(13):585–9.

    CAS  PubMed  Google Scholar 

  23. Nifli AP, Notas G, Mamoulaki M, Niniraki M, Ampartzaki V, Theodoropoulos PA, et al. Comparison of a multiplex, bead-based fluorescent assay and immunofluorescence methods for the detection of ANA and ANCA autoantibodies in human serum. J Immunol Methods. 2006;311(1–2):189–97.

    CAS  Article  PubMed  Google Scholar 

  24. Smith J, Onley D, Garey C, Crowther S, Cahir N, Johanson A, et al. Determination of ANA specificity using the UltraPlex platform. Ann N Y Acad Sci. 2005;1050:286–94.

    CAS  Article  PubMed  Google Scholar 

  25. Ghirardello A, Bendo R, Rampudda ME, Bassi N, Zampieri S, Doria A. Commercial blot assays in the diagnosis of systemic rheumatic diseases. Autoimmun Rev. 2009;8(8):645–9.

    CAS  Article  PubMed  Google Scholar 

  26. Somers K, Govarts C, Stinissen P, Somers V. Multiplexing approaches for autoantibody profiling in multiple sclerosis. Autoimmun Rev. 2009;8(7):573–9.

    CAS  Article  PubMed  Google Scholar 

  27. Liu H, Norman GL, Shums Z, Worman HJ, Krawitt EL, Bizzaro N, et al. PBC screen: an IgG/IgA dual isotype ELISA detecting multiple mitochondrial and nuclear autoantibodies specific for primary biliary cirrhosis. J Autoimmun. 2010;35(4):436–42.

    CAS  Article  PubMed  Google Scholar 

  28. Lea P, Keystone E, Mudumba S, Kahama A, Ding SF, Hansen J, et al. Advantages of multiplex proteomics in clinical immunology: the case of rheumatoid arthritis: novel IgXPLEX: planar microarray diagnosis. Clin Rev Allergy Immunol. 2011;41(1):20–35.

    CAS  Article  PubMed  Google Scholar 

  29. Bernardini S, Infantino M, Bellicampi L, Nuccetelli M, Afeltra A, Lori R, et al. Screening of antinuclear antibodies: comparison between enzyme immunoassay based on nuclear homogenates, purified or recombinant antigens and immunofluorescence assay. Clin Chem Lab Med. 2004;42(10):1155–60.

    CAS  Article  PubMed  Google Scholar 

  30. Tonutti E, Bassetti D, Piazza A, Visentini D, Poletto M, Bassetto F, et al. Diagnostic accuracy of ELISA methods as an alternative screening test to indirect immunofluorescence for the detection of antinuclear antibodies. Evaluation of five commercial kits. Autoimmunity. 2004;37(2):171–6.

    CAS  Article  Google Scholar 

  31. Sinclair D, Saas M, Williams D, Hart M, Goswami R. Can an ELISA replace immunofluorescence for the detection of antinuclear antibodies? The routine use of anti-nuclear antibody screening ELISAs. Clin Lab. 2007;53(3–4):183–91.

    PubMed  Google Scholar 

  32. Lopez-Hoyos M, Rodriguez-Valverde V, Martinez-Taboada V. Performance of antinuclear antibody connective tissue disease screen. Ann N Y Acad Sci. 2007;1109:322–9.

    Article  PubMed  Google Scholar 

  33. Op De Beeck K, Vermeesch P, Verscheueren P, Westhovens R, Marien G, Blockmans D, et al. Detection of antinuclear antibodies by indirect immunofluorescence and by solid phase assay. Autoimmun Rev. 2011;10(12):801–8.

    CAS  Article  PubMed  Google Scholar 

  34. Meroni PL, Schur PH. ANA screening: an old test with new recommendations. Ann Rheum Dis. 2010;69:1420–2.

    CAS  Article  PubMed  Google Scholar 

  35. Hiemann R, Buttner T, Krieger T, Roggenbuck D, Sack U, Conrad K. Challenges of automated screening and differentiation of non-organ specific autoantibodies on HEp-2 cells. Autoimmun Rev. 2009;9:17–22.

    CAS  Article  PubMed  Google Scholar 

  36. Rigon A, Buzzulini F, Soda P, Onofri L, Arcarese L, Iannello G, et al. Novel opportunities in automated classification of antinuclear antibodies on HEp-2 cells. Autoimmun Rev. 2011;10:647–52.

    CAS  Article  PubMed  Google Scholar 

  37. Rigon A, Soda P, Zennaro D, Iannello G, Afeltra A. Indirect Immunofluorescence in autoimmune diseases: assessment of digital images for diagnostic purpose. Cytomet B Clin Cytom. 2007;72B(6):472–7.

    Article  Google Scholar 

  38. Hiemann R, Büttner T, Krieger T, Roggenbuck D, Sack U, Conrad K. Challenges of automated screening and differentiation of non-organ specific autoantibodies on HEp-2 cells. Autoimmun Rev. 2009;9:17–22.

    CAS  Article  PubMed  Google Scholar 

  39. Hiemann R, Hilger N, Michel J, Nitschke J, Böhm A, Anderer U, et al. Automatic analysis of immunofluorescence patterns of HEp-2 cells. Ann N Y Acad Sci. 2007;1109:358–71.

    Article  PubMed  Google Scholar 

  40. Egerer K, Roggenbuck D, Hiemann R, et al. Automated evaluation of autoantibodies on human epithelial-2 cells as an approach to standardize cell-based immunofluorescence tests. Arthrit Res Ther. 2010;12(2):R40.

    Article  Google Scholar 

  41. Melegari A, Bonaguri C, Russo A, Luisita B, Trenti T, Lippi G. A comparative study on the reliability of an automated system for the evaluation of cell-based indirect immunofluorescence. Autoimmun Rev. 2012;11(10):713–6.

    CAS  Article  PubMed  Google Scholar 

  42. Kivity S, Gilburd B, Agmon-Levin N, Carrasco MG, Tzafrir Y, Sofer Y, et al. A novel automated indirect immunofluorescence autoantibody evaluation. Clin Rheumatol. 2012;31(3):503–9.

    Article  PubMed  Google Scholar 

  43. Voigt J, Krause C, Rohwäder E, Saschenbrecker S, Hahn M, Danckwardt M, et al. Automated indirect immunofluorescence evaluation of antinuclear autoantibodies on HEp-2 cells. Clin Dev Immunol. 2012;2012:651058.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Bossuyt X, Cooreman S, De Baere H, Verschueren P, Westhovens R, Blockmans D, et al. Detection of antinuclear antibodies by automated indirect immunofluorescence analysis. Clin Chim Acta. 2013;16(415):101–6.

    Article  Google Scholar 

  45. Bonroy C, Verfaillie C, Smith V, Persijn L, De Witte E, De Keyser F, et al. Automated indirect immunofluorescence antinuclear antibody analysis is a standardized alternative for visual microscope interpretation. Clin Chem Lab Med. 2013;51(9):1771–9.

    CAS  Article  PubMed  Google Scholar 

  46. Meroni PL, Bizzaro N, Cavazzana I, Borghi MO, Tincani A. Automated tests of ANA immunofluorescence as throughput autoantibody detection technology: strengths and limitations. BMC Med. 2014;12:38.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Bizzaro N, Antico A, Platzgummer S, Tonutti E, Bassetti D, Pesente F, Study Group on Autoimmune Diseases of the Italian Society of Laboratory Medicine, Italy, et al. Automated antinuclear immunofluorescence antibody screening: a comparative study of six computer-aided diagnostic systems. Autoimmun Rev. 2013;13:292–8.

    Article  PubMed  Google Scholar 

  48. Bonroy C, Verfaillie C, Smith V, Persijn L, De Witte E, De Keyser F, et al. Automated indirect immunofluorescence antinuclear antibody analysis is a standardized alternative for visual microscope interpretation. Clin Chem Lab Med. 2013;51:1771–9.

    CAS  PubMed  Google Scholar 

  49. Bossuyt X, Cooreman S, De Baere H, Verschueren P, Westhovens R, Blockmans D, et al. Detection of antinuclear antibodies by automated indirect immunofluorescence analysis. Clin Chim Acta. 2013;415:101–6.

    CAS  Article  PubMed  Google Scholar 

  50. Bizzaro N, Antico A, Platzgummer S, Tonutti E, Bassetti D, Pesente F, Study Group on Autoimmune Diseases of the Italian Society of Laboratory Medicine, Italy, et al. Automated antinuclear immunofluorescence antibody screening: a comparative study of six computer-aided diagnostic systems. Autoimmun Rev. 2013;13:292–8.

    Article  PubMed  Google Scholar 

  51. Maenhout TM, Bonroy C, Verfaillie C, Stove V, Devreese K. Automated indirect immunofluorescence microscopy enables the implementation of a quantitative internal quality control system for anti-nuclear antibody (ANA) analysis. Clin Chem Lab Med. 2014;52(7):989–98.

    CAS  Article  PubMed  Google Scholar 

  52. Zafrir Y, Gilburd B, Carrasco MG, Kivity S, Sánchez-Castañón M, López-Hoyos M, et al. Evaluation of an automated chemiluminescent immunoassay kit for antinuclear antibodies in autoimmune diseases. Immunol Res. 2013;56(2–3):451–6.

    CAS  Article  PubMed  Google Scholar 

  53. Fritzler MJ, Fritzler ML. The emergence of multiplexed technologies as diagnostic platforms in systemic autoimmune diseases. Curr Med Chem. 2006;13(21):2503–12.

    CAS  Article  PubMed  Google Scholar 

  54. Utz PJ. Multiplexed assays for identification of biomarkers and surrogate markers systemic lupus erythematosus. Lupus. 2004;13(5):304–11.

    CAS  Article  PubMed  Google Scholar 

  55. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20:37–46.

    Article  Google Scholar 

  56. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1997;33:159–74.

    Article  Google Scholar 

  57. Derrac J, García S, Molina D, Herrera F. A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput. 2011;1(1):3–18.

    Article  Google Scholar 

  58. Monce NM Jr, Cappel VL, Saqueton CB. A comparison of two fixatives on IFA HEp-2 slides for the detection of antinuclear antibodies. J Immunoassay. 1994;15:55–68.

    Article  PubMed  Google Scholar 

  59. Hahm D, Anderer U. Establishment of HEp-2 cell preparation for automated analysis of ANA fluorescence pattern. Cytomet A. 2006;69:178–81.

    Article  Google Scholar 

  60. Clinical and Laboratory Standards Institute (CLSI). Quality assurance of laboratory tests for autoantibodies to nuclear antigens: (1) Indirect fluorescent assay for microscopy and (2) microtiter enzyme immunoassay methods. Approved guideline; second edition. Wayne, PA: CLSI; 2006. CLSI document I/LA2-A2.

  61. Copple SS, Jaskowski TD, Giles R, Hill HR. Interpretation of ANA indirect immunofluorescence test outside the darkroomusing NOVA view compared tomanual microscopy. J Immunol Res. 2014;2014:149316.

    Article  PubMed  PubMed Central  Google Scholar 

  62. Bertin D, Jourde-Chiche N, Bongrand P, Bardin N. Original approach for automated quantification of antinuclear autoantibodies by indirect immunofluorescence. Clin Dev Immunol. 2013;2013:182172.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Gorgi Y, Dhaouadi T, Sfar I, Haouami Y, Abdallah TB, Raso G, et al. Comparative study of human and automated screening for antinuclear antibodies by immunofluorescence on HEp-2 cells. Int J Stat Med Res. 2015;4(3):270–6.

    Article  Google Scholar 

  64. Soda P, Iannello G, Vento M. A multiple experts system for classifying fluorescence intensity in antinuclear autoantibodies analysis. Pattern Anal Appl. 2009;12(3):215–26.

    Article  Google Scholar 

  65. Foggia P, Percannella G, Soda P, Vento M. Benchmarking HEp-2 cells classification methods. IEEE Trans Med Imaging. 2013;32(10):1878–89.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

We gratefully acknowledge Inova Diagnostics, San Diego, CA; Aesku Diagnostics, Wendelsheim, Germany, and A. Menarini Diagnostics, Florence, Italy, for participating in this study.

Author information

Authors and Affiliations

  1. Immunology and Allergy Laboratory, S.Giovanni di Dio Hospital, Via Torregalli, 3, 50143, Florence, Italy

    M. Infantino, F. Meacci, V. Grossi & M. Manfredi

  2. Rheumatology Unit, S.Giovanni di Dio Hospital, Via Torregalli, 3, 50143, Florence, Italy

    M. Benucci

  3. Computer Systems & Bioinformatics Laboratory, Department of Engineering, University Campus Bio-Medico, Rome, Italy

    M. Merone & P. Soda

Authors
  1. M. Infantino
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. F. Meacci
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. V. Grossi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M. Manfredi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. M. Benucci
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. M. Merone
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. P. Soda
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Infantino.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Infantino, M., Meacci, F., Grossi, V. et al. The burden of the variability introduced by the HEp-2 assay kit and the CAD system in ANA indirect immunofluorescence test. Immunol Res 65, 345–354 (2017). https://doi.org/10.1007/s12026-016-8845-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12026-016-8845-3

Keywords

  • Indirect immunofluorescence
  • HEp-2
  • Antinuclear antibodies
  • Automated classification system
  • Standardization