European Radiology

, Volume 20, Issue 7, pp 1599–1606 | Cite as

Effect of reader experience on variability, evaluation time and accuracy of coronary plaque detection with computed tomography coronary angiography

  • Stefan C. Saur
  • Hatem AlkadhiEmail author
  • Paul Stolzmann
  • Stephan Baumüller
  • Sebastian Leschka
  • Hans Scheffel
  • Lotus Desbiolles
  • Thomas J. Fuchs
  • Gábor Székely
  • Philippe C. Cattin



To assess the effect of reader experience on variability, evaluation time and accuracy in the detection of coronary artery plaques with computed tomography coronary angiography (CTCA).


Three independent, blinded readers with three different experience levels twice labelled 50 retrospectively electrocardiography (ECG)-gated contrast-enhanced dual-source CTCA data sets (15 female, age 67.3 ± 10.4 years, range 46–86 years) indicating the presence or absence of coronary plaques. The evaluation times for the readings were recorded. Intra- and interobserver variability expressed as κ statistics and sensitivity, specificity, and negative and positive predictive values were calculated for plaque detection, with a consensus reading of the three readers taken as the standard of reference. A bootstrap method was applied in the statistical analysis to account for clustering.


Significant correlations were found between reader experience and, respectively, evaluation times (r = −0.59, p < 0.05) and intraobserver variability (r = 0.73, p < 0.05). The evaluation time significantly differed among the readers (p < 0.05). The observer variability for plaque detection, compared with the consensus, varied between κ = 0.582 and κ = 0.802. Variability of plaque detection was significantly smaller (p < 0.05) and more accurate (p < 0.05) for the most experienced reader.


Reader experience significantly correlated with observer variability, evaluation time and accuracy of coronary plaque detection at CTCA.


Coronary plaque Observer variability Evaluation time Reader experience Computed tomography coronary angiography 



This research has been supported by the National Center of Competence in Research, Computer Aided and Image Guided Medical Interventions of the Swiss National Science Foundation (


  1. 1.
    Olgac U, Poulikakos D, Saur SC et al (2009) Patient-specific three-dimensional simulation of LDL accumulation in a human left coronary artery in its healthy and atherosclerotic states. Am J Physiol Heart Circ Physiol 296:H1969–H1982CrossRefPubMedGoogle Scholar
  2. 2.
    Enrico B, Suranyi P, Thilo C et al (2009) Coronary artery plaque formation at coronary CT angiography: morphological analysis and relationship to hemodynamics. Eur Radiol 19:837–844CrossRefPubMedGoogle Scholar
  3. 3.
    Leber AW, Knez A, von Ziegler F et al (2005) Quantification of obstructive and nonobstructive coronary lesions by 64-slice computed tomography: a comparative study with quantitative coronary angiography and intravascular ultrasound. J Am Coll Cardiol 46:147–154CrossRefPubMedGoogle Scholar
  4. 4.
    Otsuka M, Bruining N, Van Pelt NC et al (2008) Quantification of coronary plaque by 64-slice computed tomography: a comparison with quantitative intracoronary ultrasound. Invest Radiol 43:314–321CrossRefPubMedGoogle Scholar
  5. 5.
    Beckman JA, Ganz J, Creager MA et al (2001) Relationship of clinical presentation and calcification of culprit coronary artery stenoses. Arterioscler Thromb Vasc Biol 21:1618–1622CrossRefPubMedGoogle Scholar
  6. 6.
    Ehara S, Kobayashi Y, Yoshiyama M et al (2004) Spotty calcification typifies the culprit plaque in patients with acute myocardial infarction: an intravascular ultrasound study. Circulation 110:3424–3429CrossRefPubMedGoogle Scholar
  7. 7.
    Fujii K, Carlier SG, Mintz GS et al (2005) Intravascular ultrasound study of patterns of calcium in ruptured coronary plaques. Am J Cardiol 96:352–357CrossRefPubMedGoogle Scholar
  8. 8.
    Hong M-K, Mintz GS, Lee CW et al (2007) Plaque ruptures in stable angina pectoris compared with acute coronary syndrome. Int J Cardiol 114:78–82CrossRefPubMedGoogle Scholar
  9. 9.
    Pregowski J, Tyczynski P, Mintz GS et al (2006) Intravascular ultrasound assessment of the spatial distribution of ruptured coronary plaques in the left anterior descending coronary artery. Am Heart J 151:898–901CrossRefPubMedGoogle Scholar
  10. 10.
    Shimada Y, Courtney BK, Nakamura M et al (2006) Intravascular ultrasonic analysis of atherosclerotic vessel remodeling and plaque distribution of stenotic left anterior descending coronary arterial bifurcation lesions upstream and downstream of the side branch. Am J Cardiol 98:193–196CrossRefPubMedGoogle Scholar
  11. 11.
    Wang JC, Normand S-LT, Mauri L et al (2004) Coronary artery spatial distribution of acute myocardial infarction occlusions. Circulation 110:278–284CrossRefPubMedGoogle Scholar
  12. 12.
    Hendel RC, Patel MR, Kramer CM et al (2006) ACCF/ACR/SCCT/SCMR/ASNC/NASCI/SCAI/SIR 2006 appropriateness criteria for cardiac computed tomography and cardiac magnetic resonance imaging: a report of the American College of Cardiology Foundation Quality Strategic Directions Committee Appropriateness Criteria Working Group, American College of Radiology, Society of Cardiovascular Computed Tomography, Society for Cardiovascular Magnetic Resonance, American Society of Nuclear Cardiology, North American Society for Cardiac Imaging, Society for Cardiovascular Angiography and Interventions, and Society of Interventional Radiology. J Am Coll Cardiol 48:1475–1497CrossRefPubMedGoogle Scholar
  13. 13.
    Hoffmann U, Moselewski F, Nieman K et al (2006) Noninvasive assessment of plaque morphology and composition in culprit and stable lesions in acute coronary syndrome and stable lesions in stable angina by multidetector computed tomography. J Am Coll Cardiol 47:1655–1662CrossRefPubMedGoogle Scholar
  14. 14.
    Achenbach S, Moselewski F, Ropers D et al (2004) Detection of calcified and noncalcified coronary atherosclerotic plaque by contrast-enhanced, submillimeter multidetector spiral computed tomography: a segment-based comparison with intravascular ultrasound. Circulation 109:14–17CrossRefPubMedGoogle Scholar
  15. 15.
    Hausleiter J, Meyer T, Hadamitzky M et al (2006) Prevalence of noncalcified coronary plaques by 64-slice computed tomography in patients with an intermediate risk for significant coronary artery disease. J Am Coll Cardiol 48:312–318CrossRefPubMedGoogle Scholar
  16. 16.
    Iriart X, Brunot S, Coste P et al (2007) Early characterization of atherosclerotic coronary plaques with multidetector computed tomography in patients with acute coronary syndrome. Eur Radiol 17:2581–2588CrossRefPubMedGoogle Scholar
  17. 17.
    van Werkhoven JM, Schuijf JD, Gaemperli O et al (2009) Prognostic value of multislice computed tomography and gated single-photon emission computed tomography in patients with suspected coronary artery disease. J Am Coll Cardiol 53:623–632CrossRefPubMedGoogle Scholar
  18. 18.
    Leber AW, Becker A, Knez A et al (2006) Accuracy of 64-Slice computed tomography to classify and quantify plaque volumes in the proximal coronary system: a comparative study using intravascular ultrasound. J Am Coll Cardiol 47:672–677CrossRefPubMedGoogle Scholar
  19. 19.
    Motoyama S, Kondo T, Sarai M et al (2007) Multislice computed tomographic characteristics of coronary lesions in acute coronary syndromes. J Am Coll Cardiol 50:319–326CrossRefPubMedGoogle Scholar
  20. 20.
    Brodoefel H, Reimann A, Heuschmid M et al (2008) Characterization of coronary atherosclerosis by dual-source computed tomography and HU-based color mapping: a pilot study. Eur Radiol 18:2466–2474CrossRefPubMedGoogle Scholar
  21. 21.
    Pohle K, Achenbach S, MacNeill B et al (2007) Characterization of non-calcified coronary atherosclerotic plaque by multi-detector row CT: comparison to IVUS. Atherosclerosis 190:174–180CrossRefPubMedGoogle Scholar
  22. 22.
    Schroeder S, Kopp AF, Baumbach A et al (2001) Noninvasive detection and evaluation of atherosclerotic coronary plaques with multislice computed tomography. J Am Coll Cardiol 37:1430–1435CrossRefPubMedGoogle Scholar
  23. 23.
    Leber AW, Knez A, Becker A et al (2004) Accuracy of multidetector spiral computed tomography in identifying and differentiating the composition of coronary atherosclerotic plaques: a comparative study with intracoronary ultrasound. J Am Coll Cardiol 43:1241–1247CrossRefPubMedGoogle Scholar
  24. 24.
    Ferencik M, Nieman K, Achenbach S (2006) Noncalcified and calcified coronary plaque detection by contrast-enhanced multi-detector computed tomography: a study of interobserver agreement. J Am Coll Cardiol 47:207CrossRefPubMedGoogle Scholar
  25. 25.
    Hoffmann H, Frieler K, Hamm B et al (2008) Intra- and interobserver variability in detection and assessment of calcified and noncalcified coronary artery plaques using 64-slice computed tomography. Int J Card Imaging 24:735CrossRefGoogle Scholar
  26. 26.
    Pflederer T, Schmid M, Ropers D et al (2007) Interobserver variability of 64-slice computed tomography for the quantification of non-calcified coronary atherosclerotic plaque. Rofo 179:953–957PubMedGoogle Scholar
  27. 27.
    Bruining N, Roelandt JRTC, Palumbo A et al (2007) Reproducible coronary plaque quantification by multislice computed tomography. Catheter Cardiovasc Interv 69:857–865CrossRefPubMedGoogle Scholar
  28. 28.
    Pugliese F, Hunink MGM, Gruszczynska K et al (2009) Learning curve for coronary CT angiography: what constitutes sufficient training? Radiology 251:359–368CrossRefPubMedGoogle Scholar
  29. 29.
    Diamond G, Forrester J (1979) Analysis of probability as an aid in the clinical diagnosis of coronary-artery disease. N Engl J Med 300:1350–1358PubMedCrossRefGoogle Scholar
  30. 30.
    Leschka S, Scheffel H, Desbiolles L et al (2007) Image quality and reconstruction intervals of dual-source CT coronary angiography: recommendations for ECG-pulsing windowing. Invest Radiol 42:543–549CrossRefPubMedGoogle Scholar
  31. 31.
    Budoff MJ, Achenbach S, Berman DS et al (2008) Task force 13: training in advanced cardiovascular imaging (computed tomography): endorsed by the American Society of Nuclear Cardiology, Society of Atherosclerosis Imaging and Prevention, Society for Cardiovascular Angiography and Interventions, and Society of Cardiovascular Computed Tomography. J Am Coll Cardiol 51:409CrossRefPubMedGoogle Scholar
  32. 32.
    Austen WG, Edwards JE, Frye RL (1975) A reporting system on patients evaluated for coronary artery disease. Report of the Ad Hoc Committee for Grading of Coronary Artery Disease, Council on Cardiovascular Surgery, American Heart Association. Circulation 5–40Google Scholar
  33. 33.
    Efron B, Tibshirani RJ (1993) An introduction to the bootstrap. Chapman & Hall, New YorkGoogle Scholar
  34. 34.
    Hothorn T, Buhlmann P, Dudoit S et al (2006) Survival ensembles. Biostat 7:355–373CrossRefGoogle Scholar
  35. 35.
    Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32:675–701CrossRefGoogle Scholar
  36. 36.
    Tessmann M, Vega-Higuera F, Fritz D et al (2009) Multi-scale feature extraction for learning-based classification of coronary artery stenosis. In: Nico K, Maryellen LG (eds) SPIE 726002Google Scholar
  37. 37.
    Renard F, Yongyi Y (2008) Image analysis for detection of coronary artery soft plaques in MDCT images. In: Proceedings of the 5th IEEE international symposium on biomedical imaging: from nano to macro, ISBI, Paris, France, 14–17 May 2008, pp 25–28Google Scholar
  38. 38.
    Becker C, Nikolaou K, Muders M et al (2003) Ex vivo coronary atherosclerotic plaque characterization with multi-detector-row CT. Eur Radiol 13:2094–2098CrossRefPubMedGoogle Scholar
  39. 39.
    Leschka S, Stolzmann P, Desbiolles L et al (2009) Diagnostic accuracy of high-pitch dual-source CT for the assessment of coronary stenoses: first experience. Eur Radiol 19:2896–2903. doi: 10.1007/s00330–009–1618–9 Google Scholar
  40. 40.
    Saur S, Cattin P, Desbiolles L et al (2009) Prediction rules for the detection of coronary artery plaques: evidence from cardiac CT. Invest Radiol 44:483–490Google Scholar

Copyright information

© European Society of Radiology 2010

Authors and Affiliations

  • Stefan C. Saur
    • 1
  • Hatem Alkadhi
    • 2
    • 3
    Email author
  • Paul Stolzmann
    • 2
  • Stephan Baumüller
    • 2
  • Sebastian Leschka
    • 2
  • Hans Scheffel
    • 2
  • Lotus Desbiolles
    • 2
  • Thomas J. Fuchs
    • 4
  • Gábor Székely
    • 1
  • Philippe C. Cattin
    • 1
    • 5
  1. 1.Computer Vision LaboratoryETH ZurichZurichSwitzerland
  2. 2.Institute of Diagnostic RadiologyUniversity Hospital ZurichZurichSwitzerland
  3. 3.Cardiac MR PET CT groupMassachusetts General Hospital and Harvard Medical SchoolBostonUSA
  4. 4.Department of Computer ScienceETH ZurichZurichSwitzerland
  5. 5.Medical Image Analysis CenterUniversity of BaselBaselSwitzerland

Personalised recommendations