Immunologic Research

, Volume 65, Issue 1, pp 345–354 | Cite as

The burden of the variability introduced by the HEp-2 assay kit and the CAD system in ANA indirect immunofluorescence test

  • M. Infantino
  • F. Meacci
  • V. Grossi
  • M. Manfredi
  • M. Benucci
  • M. Merone
  • P. Soda
Mechanism in Autoimmunity

Abstract

According to the recent recommendations of the American College of Rheumatology, ANA Task Force, IIF technique should be considered the gold standard in antinuclear antibodies (ANAs) testing. To overcome the lack of standardization, biomedical industries have developed several computer-aided diagnosis (CAD) systems. Two hundred and sixty-one consecutive samples with suspected autoimmune diseases were tested for ANA by means of IIF on routinely HEp-2 assay kit (Euroimmun AG). Assignment of result was made if consensus for positive/negative was reached by at least 2 out of 3 expert physicians. ANA-IIF was also carried out using 3 CAD systems: Zenit G-Sight (n = 84), Helios (n = 85) and NOVA View (n = 92); human evaluation was repeated on the same substrate of each CAD system (Immco, Aesku and Inova HEp-2 cells, respectively). To anonymize the results, we randomly named these three systems as A, B and C. We ran a statistical analysis computing several measures of agreement between the ratings, and we also improved the evaluation by using the Wilcoxon’s test for nonparametric data. Agreement between the human readings on routinely HEp-2 assay kit and human readings on CAD HEp-2 assay was substantial for A (k = 0.82) and B (k = 0.72), and almost perfect for C (k = 0.89). Such readings were statistically different only in case A. Comparing experts’ readings with the readings of CAD systems, when the samples were prepared using CAD HEp-2 assay kits, we found almost perfect agreement for B and C (k = 0.86; k = 0.82) and substantial agreement for A (k = 0.73). Again, human and CAD readings were statistically different only in A. When we compared the readings of medical experts on routinely HEp-2 assay kit with the output of the CAD systems that worked using their own slides, we found substantial agreement for all the systems (A: k = 0.62; B: k = 0.65; C: k = 0.71). Such readings were not statistically different. The change of the assay kit and/or the introduction of a CAD system affect the laboratory reporting, with an evident impact on the autoimmune laboratory workflow. The CAD systems may represent one of the most important novel elements of harmonization in the autoimmunity field, reducing intra- and inter-laboratory variability in a new vision of the diagnostic autoimmune platform.

Keywords

Indirect immunofluorescence HEp-2 Antinuclear antibodies Automated classification system Standardization 

Abbreviations

ACR

American College of Rheumatology

ANA

Antinuclear antibodies

CAD

Computer-aided diagnosis

HEp-2

Human epithelial cells

IIF

Indirect immunofluorescence

IIM

Idiopathic inflammatory myopathies

FITC

Fluorescein isothiocyanate

SARD

Systemic rheumatic diseases

SjS

Sjögren’s syndrome

SLE

Systemic lupus erythematosus

SSc

Systemic sclerosis

References

  1. 1.
    Solomon DH, Kavanaugh AJ, Schur PH, American College of Rheumatology Ad Hoc Committee on Immunologic Testing Guidelines. Evidence based guidelines for the use of immunologic tests: antinuclear antibody testing. Arthritis Rheum. 2002;47(4):434–44.CrossRefPubMedGoogle Scholar
  2. 2.
    Fritzler MJ. Challenges to the use of autoantibodies as predictors of disease onset, diagnosis and outcomes. Autoimmun Rev. 2008;7(8):616–20.CrossRefPubMedGoogle Scholar
  3. 3.
    Tozzoli R, Bizzaro N, Tonutti E, Villalta D, Bassetti D, Manoni F, Piazza A, Pradella M, Rizzotti P, Italian Society of Laboratory Medicine Study Group on the Diagnosis of Autoimmune Diseases. Guidelines for the laboratory use of autoantibody tests in the diagnosis and monitoring of autoimmune rheumatic diseases. Am J Clin Pathol. 2002;117:316–24.CrossRefPubMedGoogle Scholar
  4. 4.
    Brand F, Martin F, Philipp S, Rößler J, Hansen B, Anderer U. Difference in fluorescence pattern of cytoplasmic and nuclear antigens in cultivated human cells dependent on the applied fixation procedure. Cell Prolif. 2005;38:205.Google Scholar
  5. 5.
    Wiik A, Hoier-Madsen M, Forslid J, Charles P, Meyrowitsch J. Antinuclear antibodies: a contemporary nomenclature using HEp-2 cells. J Autoimmun. 2010;35(3):276–90.CrossRefPubMedGoogle Scholar
  6. 6.
    Hiemann R, Büttner T, Krieger T, Roggenbuck D, Sack U, Conrad K. Challenges of 417 automated screening and differentiation of non-organ specific autoantibodies 418 on HEp-2 cells. Autoimmun Rev. 2009;9:17–22.CrossRefPubMedGoogle Scholar
  7. 7.
    Fenger M, Wiik A, Høier-Madsen M, Lykkegaard JJ, Rozenfeld T, Hansen MS, et al. Detection of antinuclear antibodies by solid-phase immunoassays and immunofluorescence analysis. Clin Chem. 2004;50(11):2141–7.CrossRefPubMedGoogle Scholar
  8. 8.
    Pham BN, Albarede S, Guyard A, Burg E, Maisonneuve P. Impact of external quality assessment on antinuclear antibody detection performance. Lupus. 2005;14:113–9.CrossRefPubMedGoogle Scholar
  9. 9.
    Bizzaro N, Tozzoli R, Tonutti E, Piazza A, Manoni F, Ghirardello A, et al. Variability between methods to determine ANA, anti-dsDNA and anti-ENA autoantibodies: a collaborative study with the biomedical industry. J Immunol Methods. 1998;219:99–107.CrossRefPubMedGoogle Scholar
  10. 10.
    Song L, Hennink EJ, Young IT, Tanke HJ. Photobleaching kinetics of fluorescein in quantitative fluorescence microscopy. Biophys J. 1995;68:2588–600.CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Marcolongo R, Ruffatti A, Morozzi G. Presentazione linee guida del Forum Interdisciplinare per la Ricerca sulle Malattie Autoimmuni (F.I.R.M.A.). Reumatismo. 2003;55:9–21.Google Scholar
  12. 12.
    Kavanaugh A, Tomar R, Reveille J, Solomon DH, Homburger HA, American College of Pathologists. Guidelines for clinical use of the antinuclear antibody test and tests for specific autoantibodies to nuclear antigens. Arch Pathol Lab Med. 2000;124(1):71–81.PubMedGoogle Scholar
  13. 13.
    Sack U, Conrad K, Csernok E, Frank I, Hiepe F, Krieger T, et al. Autoantibody detection using indirect immunofluorescence on HEp-2 cells, German EASI (European Autoimmunity Standardization Initiative). Ann N Y Acad Sci. 2009;1173:166–73.CrossRefPubMedGoogle Scholar
  14. 14.
    Copple SS, Giles SR, Jaskowski TD, Gardiner AE, Wilson AM, Hill HR. Screening for IgG antinuclear autoantibodies by HEp-2 indirect fluorescent antibody assays and the need for standardization. Am J Clin Pathol. 2012;137(5):825–30.CrossRefPubMedGoogle Scholar
  15. 15.
    Burlingame RW, Peebles C. Detection of antibodies. In: Pollard KM, editor. Autoantibodies and autoimmunity: molecular mechanisms in health and disease. Weinheim, Germany: Wiley-VCH; 2006. p. 159–88.Google Scholar
  16. 16.
    Egner W. The use of laboratory tests in the diagnosis of SLE. J Clin Pathol. 2000;53(6):424–32.CrossRefPubMedPubMedCentralGoogle Scholar
  17. 17.
    Fenger M, Wiik A, Høier-Madsen M, Lykkegaard JJ, Rozenfeld T, Hansen MS, et al. Detection of antinuclear antibodies by solid-phase immunoassays and immunofluorescence analysis. Clin Chem. 2004;50:2141–7.CrossRefPubMedGoogle Scholar
  18. 18.
    Bradwell AR, Hughes RG, Karim AR. In: Detrick B, Hamilton RG, Folds JD, editors. Immunofluorescent antinuclear antibody tests in Manual of Clinical Laboratory Immunology. 7th ed. Washington: ASM press; 2006. p. 995–1006.Google Scholar
  19. 19.
    Emlen W, O’neill L. Clinical significance of antinuclear antibodies: comparison of detection with immunofluorescence and enzyme-linked immunosorbent assays. Arthritis Rheum. 1997;40(9):1612–8.CrossRefPubMedGoogle Scholar
  20. 20.
    Tonutti E, Bassetti D, Piazza A, Visentini D, Poletto M, Bassetto F, et al. Diagnostic accuracy of ELISA methods as an alternative screening test to indirect immunofluorescence for the detection of antinuclear antibodies. Evaluation of Five Commercial Kits. Autoimmunity. 2004;37:171–6.CrossRefGoogle Scholar
  21. 21.
    Tan EM, Smolen JS, McDougal JS, Butcher BT, Conn D, Dawkins R, et al. A critical evaluation of enzyme immunoassays for detection of antinuclear autoantibodies of defined specificities. I. Precision, sensitivity, and specificity. Arthritis Rheum. 1999;42:455–64.CrossRefPubMedGoogle Scholar
  22. 22.
    Swaak AJ. Diagnostic significance of antinuclear antibodies in clinical practice. Ned Tijdschr Geneeskd. 2000;144(13):585–9.PubMedGoogle Scholar
  23. 23.
    Nifli AP, Notas G, Mamoulaki M, Niniraki M, Ampartzaki V, Theodoropoulos PA, et al. Comparison of a multiplex, bead-based fluorescent assay and immunofluorescence methods for the detection of ANA and ANCA autoantibodies in human serum. J Immunol Methods. 2006;311(1–2):189–97.CrossRefPubMedGoogle Scholar
  24. 24.
    Smith J, Onley D, Garey C, Crowther S, Cahir N, Johanson A, et al. Determination of ANA specificity using the UltraPlex platform. Ann N Y Acad Sci. 2005;1050:286–94.CrossRefPubMedGoogle Scholar
  25. 25.
    Ghirardello A, Bendo R, Rampudda ME, Bassi N, Zampieri S, Doria A. Commercial blot assays in the diagnosis of systemic rheumatic diseases. Autoimmun Rev. 2009;8(8):645–9.CrossRefPubMedGoogle Scholar
  26. 26.
    Somers K, Govarts C, Stinissen P, Somers V. Multiplexing approaches for autoantibody profiling in multiple sclerosis. Autoimmun Rev. 2009;8(7):573–9.CrossRefPubMedGoogle Scholar
  27. 27.
    Liu H, Norman GL, Shums Z, Worman HJ, Krawitt EL, Bizzaro N, et al. PBC screen: an IgG/IgA dual isotype ELISA detecting multiple mitochondrial and nuclear autoantibodies specific for primary biliary cirrhosis. J Autoimmun. 2010;35(4):436–42.CrossRefPubMedGoogle Scholar
  28. 28.
    Lea P, Keystone E, Mudumba S, Kahama A, Ding SF, Hansen J, et al. Advantages of multiplex proteomics in clinical immunology: the case of rheumatoid arthritis: novel IgXPLEX: planar microarray diagnosis. Clin Rev Allergy Immunol. 2011;41(1):20–35.CrossRefPubMedGoogle Scholar
  29. 29.
    Bernardini S, Infantino M, Bellicampi L, Nuccetelli M, Afeltra A, Lori R, et al. Screening of antinuclear antibodies: comparison between enzyme immunoassay based on nuclear homogenates, purified or recombinant antigens and immunofluorescence assay. Clin Chem Lab Med. 2004;42(10):1155–60.CrossRefPubMedGoogle Scholar
  30. 30.
    Tonutti E, Bassetti D, Piazza A, Visentini D, Poletto M, Bassetto F, et al. Diagnostic accuracy of ELISA methods as an alternative screening test to indirect immunofluorescence for the detection of antinuclear antibodies. Evaluation of five commercial kits. Autoimmunity. 2004;37(2):171–6.CrossRefGoogle Scholar
  31. 31.
    Sinclair D, Saas M, Williams D, Hart M, Goswami R. Can an ELISA replace immunofluorescence for the detection of antinuclear antibodies? The routine use of anti-nuclear antibody screening ELISAs. Clin Lab. 2007;53(3–4):183–91.PubMedGoogle Scholar
  32. 32.
    Lopez-Hoyos M, Rodriguez-Valverde V, Martinez-Taboada V. Performance of antinuclear antibody connective tissue disease screen. Ann N Y Acad Sci. 2007;1109:322–9.CrossRefPubMedGoogle Scholar
  33. 33.
    Op De Beeck K, Vermeesch P, Verscheueren P, Westhovens R, Marien G, Blockmans D, et al. Detection of antinuclear antibodies by indirect immunofluorescence and by solid phase assay. Autoimmun Rev. 2011;10(12):801–8.CrossRefPubMedGoogle Scholar
  34. 34.
    Meroni PL, Schur PH. ANA screening: an old test with new recommendations. Ann Rheum Dis. 2010;69:1420–2.CrossRefPubMedGoogle Scholar
  35. 35.
    Hiemann R, Buttner T, Krieger T, Roggenbuck D, Sack U, Conrad K. Challenges of automated screening and differentiation of non-organ specific autoantibodies on HEp-2 cells. Autoimmun Rev. 2009;9:17–22.CrossRefPubMedGoogle Scholar
  36. 36.
    Rigon A, Buzzulini F, Soda P, Onofri L, Arcarese L, Iannello G, et al. Novel opportunities in automated classification of antinuclear antibodies on HEp-2 cells. Autoimmun Rev. 2011;10:647–52.CrossRefPubMedGoogle Scholar
  37. 37.
    Rigon A, Soda P, Zennaro D, Iannello G, Afeltra A. Indirect Immunofluorescence in autoimmune diseases: assessment of digital images for diagnostic purpose. Cytomet B Clin Cytom. 2007;72B(6):472–7.CrossRefGoogle Scholar
  38. 38.
    Hiemann R, Büttner T, Krieger T, Roggenbuck D, Sack U, Conrad K. Challenges of automated screening and differentiation of non-organ specific autoantibodies on HEp-2 cells. Autoimmun Rev. 2009;9:17–22.CrossRefPubMedGoogle Scholar
  39. 39.
    Hiemann R, Hilger N, Michel J, Nitschke J, Böhm A, Anderer U, et al. Automatic analysis of immunofluorescence patterns of HEp-2 cells. Ann N Y Acad Sci. 2007;1109:358–71.CrossRefPubMedGoogle Scholar
  40. 40.
    Egerer K, Roggenbuck D, Hiemann R, et al. Automated evaluation of autoantibodies on human epithelial-2 cells as an approach to standardize cell-based immunofluorescence tests. Arthrit Res Ther. 2010;12(2):R40.CrossRefGoogle Scholar
  41. 41.
    Melegari A, Bonaguri C, Russo A, Luisita B, Trenti T, Lippi G. A comparative study on the reliability of an automated system for the evaluation of cell-based indirect immunofluorescence. Autoimmun Rev. 2012;11(10):713–6.CrossRefPubMedGoogle Scholar
  42. 42.
    Kivity S, Gilburd B, Agmon-Levin N, Carrasco MG, Tzafrir Y, Sofer Y, et al. A novel automated indirect immunofluorescence autoantibody evaluation. Clin Rheumatol. 2012;31(3):503–9.CrossRefPubMedGoogle Scholar
  43. 43.
    Voigt J, Krause C, Rohwäder E, Saschenbrecker S, Hahn M, Danckwardt M, et al. Automated indirect immunofluorescence evaluation of antinuclear autoantibodies on HEp-2 cells. Clin Dev Immunol. 2012;2012:651058.CrossRefPubMedPubMedCentralGoogle Scholar
  44. 44.
    Bossuyt X, Cooreman S, De Baere H, Verschueren P, Westhovens R, Blockmans D, et al. Detection of antinuclear antibodies by automated indirect immunofluorescence analysis. Clin Chim Acta. 2013;16(415):101–6.CrossRefGoogle Scholar
  45. 45.
    Bonroy C, Verfaillie C, Smith V, Persijn L, De Witte E, De Keyser F, et al. Automated indirect immunofluorescence antinuclear antibody analysis is a standardized alternative for visual microscope interpretation. Clin Chem Lab Med. 2013;51(9):1771–9.CrossRefPubMedGoogle Scholar
  46. 46.
    Meroni PL, Bizzaro N, Cavazzana I, Borghi MO, Tincani A. Automated tests of ANA immunofluorescence as throughput autoantibody detection technology: strengths and limitations. BMC Med. 2014;12:38.CrossRefPubMedPubMedCentralGoogle Scholar
  47. 47.
    Bizzaro N, Antico A, Platzgummer S, Tonutti E, Bassetti D, Pesente F, Study Group on Autoimmune Diseases of the Italian Society of Laboratory Medicine, Italy, et al. Automated antinuclear immunofluorescence antibody screening: a comparative study of six computer-aided diagnostic systems. Autoimmun Rev. 2013;13:292–8.CrossRefPubMedGoogle Scholar
  48. 48.
    Bonroy C, Verfaillie C, Smith V, Persijn L, De Witte E, De Keyser F, et al. Automated indirect immunofluorescence antinuclear antibody analysis is a standardized alternative for visual microscope interpretation. Clin Chem Lab Med. 2013;51:1771–9.PubMedGoogle Scholar
  49. 49.
    Bossuyt X, Cooreman S, De Baere H, Verschueren P, Westhovens R, Blockmans D, et al. Detection of antinuclear antibodies by automated indirect immunofluorescence analysis. Clin Chim Acta. 2013;415:101–6.CrossRefPubMedGoogle Scholar
  50. 50.
    Bizzaro N, Antico A, Platzgummer S, Tonutti E, Bassetti D, Pesente F, Study Group on Autoimmune Diseases of the Italian Society of Laboratory Medicine, Italy, et al. Automated antinuclear immunofluorescence antibody screening: a comparative study of six computer-aided diagnostic systems. Autoimmun Rev. 2013;13:292–8.CrossRefPubMedGoogle Scholar
  51. 51.
    Maenhout TM, Bonroy C, Verfaillie C, Stove V, Devreese K. Automated indirect immunofluorescence microscopy enables the implementation of a quantitative internal quality control system for anti-nuclear antibody (ANA) analysis. Clin Chem Lab Med. 2014;52(7):989–98.CrossRefPubMedGoogle Scholar
  52. 52.
    Zafrir Y, Gilburd B, Carrasco MG, Kivity S, Sánchez-Castañón M, López-Hoyos M, et al. Evaluation of an automated chemiluminescent immunoassay kit for antinuclear antibodies in autoimmune diseases. Immunol Res. 2013;56(2–3):451–6.CrossRefPubMedGoogle Scholar
  53. 53.
    Fritzler MJ, Fritzler ML. The emergence of multiplexed technologies as diagnostic platforms in systemic autoimmune diseases. Curr Med Chem. 2006;13(21):2503–12.CrossRefPubMedGoogle Scholar
  54. 54.
    Utz PJ. Multiplexed assays for identification of biomarkers and surrogate markers systemic lupus erythematosus. Lupus. 2004;13(5):304–11.CrossRefPubMedGoogle Scholar
  55. 55.
    Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20:37–46.CrossRefGoogle Scholar
  56. 56.
    Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1997;33:159–74.CrossRefGoogle Scholar
  57. 57.
    Derrac J, García S, Molina D, Herrera F. A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput. 2011;1(1):3–18.CrossRefGoogle Scholar
  58. 58.
    Monce NM Jr, Cappel VL, Saqueton CB. A comparison of two fixatives on IFA HEp-2 slides for the detection of antinuclear antibodies. J Immunoassay. 1994;15:55–68.CrossRefPubMedGoogle Scholar
  59. 59.
    Hahm D, Anderer U. Establishment of HEp-2 cell preparation for automated analysis of ANA fluorescence pattern. Cytomet A. 2006;69:178–81.CrossRefGoogle Scholar
  60. 60.
    Clinical and Laboratory Standards Institute (CLSI). Quality assurance of laboratory tests for autoantibodies to nuclear antigens: (1) Indirect fluorescent assay for microscopy and (2) microtiter enzyme immunoassay methods. Approved guideline; second edition. Wayne, PA: CLSI; 2006. CLSI document I/LA2-A2.Google Scholar
  61. 61.
    Copple SS, Jaskowski TD, Giles R, Hill HR. Interpretation of ANA indirect immunofluorescence test outside the darkroomusing NOVA view compared tomanual microscopy. J Immunol Res. 2014;2014:149316.CrossRefPubMedPubMedCentralGoogle Scholar
  62. 62.
    Bertin D, Jourde-Chiche N, Bongrand P, Bardin N. Original approach for automated quantification of antinuclear autoantibodies by indirect immunofluorescence. Clin Dev Immunol. 2013;2013:182172.CrossRefPubMedPubMedCentralGoogle Scholar
  63. 63.
    Gorgi Y, Dhaouadi T, Sfar I, Haouami Y, Abdallah TB, Raso G, et al. Comparative study of human and automated screening for antinuclear antibodies by immunofluorescence on HEp-2 cells. Int J Stat Med Res. 2015;4(3):270–6.CrossRefGoogle Scholar
  64. 64.
    Soda P, Iannello G, Vento M. A multiple experts system for classifying fluorescence intensity in antinuclear autoantibodies analysis. Pattern Anal Appl. 2009;12(3):215–26.CrossRefGoogle Scholar
  65. 65.
    Foggia P, Percannella G, Soda P, Vento M. Benchmarking HEp-2 cells classification methods. IEEE Trans Med Imaging. 2013;32(10):1878–89.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • M. Infantino
    • 1
  • F. Meacci
    • 1
  • V. Grossi
    • 1
  • M. Manfredi
    • 1
  • M. Benucci
    • 2
  • M. Merone
    • 3
  • P. Soda
    • 3
  1. 1.Immunology and Allergy LaboratoryS.Giovanni di Dio HospitalFlorenceItaly
  2. 2.Rheumatology UnitS.Giovanni di Dio HospitalFlorenceItaly
  3. 3.Computer Systems & Bioinformatics Laboratory, Department of EngineeringUniversity Campus Bio-MedicoRomeItaly

Personalised recommendations