Impact of acquisition and interpretation on total inter-observer variability in echocardiography: results from the quality assurance program of the STAAB cohort study

  • Caroline Morbach
  • Götz Gelbrich
  • Margret Breunig
  • Theresa Tiffe
  • Martin Wagner
  • Peter U. Heuschmann
  • Stefan StörkEmail author
Original Paper


Variability related to image acquisition and interpretation is an important issue of echocardiography in clinical trials. Nevertheless, there is no broadly accepted standard method for quality assessment of echocardiography in clinical research reports. We present analyses based on the echocardiography quality-assurance program of the ongoing STAAB cohort study (characteristics and course of heart failure stages A–B and determinants of progression). In 43 healthy individuals (mean age 50 ± 14 years; 18 females), duplicate echocardiography scans were acquired and mutually interpreted by one of three trained sonographers and an EACVI certified physician, respectively. Acquisition (AcV), interpretation (InV), and inter-observer variability (IOV; i.e., variability between the acquisition-interpretation sequences of two different observers), were determined for selected M-mode, B-mode, and Doppler parameters. We calculated Bland–Altman upper 95% limits of absolute differences, implying that 95% of measurement differences were smaller/equal to the given value: e.g. LV end-diastolic volume (mL): 25.0, 25.0, 27.9; septal e′ velocity (cm/s): 3.03, 1.25, 3.58. Further, 90, 85, and 80% upper limits of absolute differences were determined for the respective parameters. Both, acquisition and interpretation, independently and sizably contributed to IOV. As such, separate assessment of AcV and InV is likely to aid in echocardiography training and quality-assurance. Our results further suggest to routinely determine IOV in clinical trials as a comprehensive measure of imaging quality. The derived 95, 90, 85, and 80% upper limits of absolute differences are suggested as reproducibility targets of future studies, thus contributing to the international efforts of standardization in quality-assurance.


Echocardiography Quality Reproducibility Variability Observer Acquisition 





A-wave velocity


Peak late- diastolic lengthening velocity


Acquisition variability


Continuous wave


E-wave velocity


Peak early diastolic lengthening velocity


European Association of Cardiovascular Imaging


Ejection fraction


Intra-interpretation variability


Intraclass correlation coefficient


Interpretation variability


Inter-observer variability


Characteristics and course of heart failure stages A–B and determinants of progression cohort study


Left atrium


Left ventricle


Limits of agreement


Left ventricular end-diastolic diameter


Left ventricular end-diastolic volume


Left ventricular end-systolic volume


Mitral annular plane systolic excursion


Pulsed wave


Right atrium


Right ventricle


Peak systolic shortening velocity


Tricuspid annular plane systolic excursion


Tissue Doppler imaging



We thank Martina Bauer, lead sonographer of the Comprehensive Heart Failure Center (CHFC), for excellent echocardiography teaching, as well as Jasmin Simon, Martin Cramer and Frances Knobeloch, CHFC sonographers, for their valuable contribution to this study.


The study is supported by the German Ministry of Research and Education within the Comprehensive Heart Failure Centre Würzburg (BMBF 01EO1004 and 01EO1504). This publication was funded by the German Research Foundation (DFG) and the University of Wuerzburg in the funding programme Open Access Publishing.

Compliance with ethical standards

Conflict of interest

CM reports a co-investigatorship in a trial sponsored by NOVARTIS, a travel grant from THERMO FISHER and a speaker honorarium from AMGEN. GG reports grants from University Göttingen within the FIND-AF-randomized trial (FIND-AF-randomized is supported by an unrestricted grant to the University Göttingen from Boehringer-Ingelheim), grants from University Göttingen within DZHK analysis projects (the DZHK is funded by a BMBF grant), personal fees from Charité - Universitätsmedizin Berlin within the TIM-HF-II trial for data safety and monitoring board membership (TIM-HF-II is supported by a BMBF grant to the Charité), grants from University Hospital Würzburg within the MOOD-HF trial for biometry and steering committee membership (MOOD-HF is supported by a BMBF grant and an unrestricted grant to the University Hospital Würzburg from Lundbeck), outside the submitted work. PUH reports research grants from the German Ministry of Research and Education, European Union, Charité, Berlin Chamber of Physicians, German Parkinson Society, University Hospital Würzburg, Robert-Koch-Institute, German Heart Foundation, Charité–Universitätsmedizin Berlin (within MonDAFIS; MonDAFIS is supported by an unrestricted research grant to the Charité from Bayer), University Göttingen (within FIND-AF-randomized; FIND-AF randomized is supported by an unrestricted research grant to the University Göttingen from Boehringer-Ingelheim), and University Hospital Heidelberg (within RASUNOA-prime; RASUNOA-prime is supported by an unrestricted research grant to the University Hospital Heidelberg from Bayer, BMS, Boehringer-Ingelheim, Daiichi Sankyo), outside submitted work. SS reports research grants from the German Ministry of Education and Research, European Union, University Hospital Würzburg; participation in Data Safety Monitoring and Event Adjudication Boards in trials sponsored by ROCHE and MEDTRONIC; principal investigator in trials (co-)sponsored by BOEHRINGER, NOVARTIS, BAYER, LUNDBECK; speaker honaria by BOEHRINGER, SERVIER, NOVARTIS, ASTRA-ZENECA, PFIZER, BAYER. MW declares that he has no conflict of interest. TT declares that she has no conflict of interest. MB declares that she has no conflict of interest.

Ethical approval

The STAAB cohort study protocol and procedures received positive votes from the Ethics Committee of the Medical Faculty (vote 98/13) as well as from the data protection officer of the University of Würzburg (J-117.605-09/13). All procedures performed in this study were in accordance with the ethical standards of the institutional and national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

All participants provided written informed consent prior to any study examination.


  1. 1.
    Crowley AL, Yow E, Barnhart HX, Daubert MA, Bigelow R, Sullivan DC, Pencina M, Douglas PS (2016) Critical review of current approaches for echocardiographic reproducibility and reliability assessment in clinical research. J Am Soc Echocardiogr 29(12):1144–1154.e7. CrossRefPubMedGoogle Scholar
  2. 2.
    Pellikka PA, Douglas PS, Miller JG, Abraham TP, Baumann R, Buxton DB, Byrd BF 3rd, Chen P, Cook NL, Gardin JM, Hansen G, Houle HC, Husson S, Kaul S, Klein AL, Lang RM, Leong-Poi H, Lopez H, Mahmoud TM, Maslak S, McCulloch ML, Metz S, Nagueh SF, Pearlman AS, Pibarot P, Picard MH, Porter TR, Prater D, Rodriguez R, Sarano ME, Scherrer-Crosbie M, Shirali GS, Sinusas A, Slosky JJ, Sugeng L, Tatpati A, Villanueva FS, von Ramm OT, Weissman NJ, Zamani S (2013) American society of echocardiography cardiovascular technology and research summit: a roadmap for 2020. J Am Soc Echocardiogr 26(4):325–338. CrossRefPubMedGoogle Scholar
  3. 3.
    Kardys I, Deckers JW, Stricker BH, Vletter WB, Hofman A, Witteman J (2010) Distribution of echocardiographic parameters and their associations with cardiovascular risk factors in the Rotterdam Study. Eur J Epidemiol 25(7):481–490. CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Kou S, Caballero L, Dulgheru R, Voilliot D, De Sousa C, Kacharava G, Athanassopoulos GD, Barone D, Baroni M, Cardim N, Gomez De Diego JJ, Hagendorff A, Henri C, Hristova K, Lopez T, Magne J, De La Morena G, Popescu BA, Penicka M, Ozyigit T, Rodrigo Carbonero JD, Salustri A, Van De Veire N, Von Bardeleben RS, Vinereanu D, Voigt JU, Zamorano JL, Donal E, Lang RM, Badano LP, Lancellotti P (2014) Echocardiographic reference ranges for normal cardiac chamber size: results from the NORRE study. Eur Heart J Cardiovasc Imaging 15(6):680–690. CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    Jenkins C, Bricknell K, Hanekom L, Marwick TH (2004) Reproducibility and accuracy of echocardiographic measurements of left ventricular parameters using real-time three-dimensional echocardiography. J Am Coll Cardiol 44(4):878–886. CrossRefPubMedGoogle Scholar
  6. 6.
    Wagner M, Tiffe T, Morbach C, Gelbrich G, Stork S, Heuschmann PU, Consortium S (2016) Characteristics and course of heart failure stages A-B and determinants of progression—design and rationale of the STAAB cohort study. Eur J Prev Cardiol 24(5):468–479. CrossRefPubMedGoogle Scholar
  7. 7.
    Lang RM, Badano LP, Mor-Avi V, Afilalo J, Armstrong A, Ernande L, Flachskampf FA, Foster E, Goldstein SA, Kuznetsova T, Lancellotti P, Muraru D, Picard MH, Rietzschel ER, Rudski L, Spencer KT, Tsang W, Voigt JU (2015) Recommendations for cardiac chamber quantification by echocardiography in adults: an update from the American Society of Echocardiography and the European Association of Cardiovascular Imaging. Eur Heart J Cardiovasc Imaging 16(3):233–270. CrossRefPubMedGoogle Scholar
  8. 8.
    Bland JM, Altman DG (1999) Measuring agreement in method comparison studies. Stat Methods Med Res 8(2):135–160CrossRefPubMedGoogle Scholar
  9. 9.
    Thorstensen A, Dalen H, Amundsen BH, Aase SA, Stoylen A (2010) Reproducibility in echocardiographic assessment of the left ventricular global and regional function, the HUNT study. Eur J Echocardiogr 11(2):149–156. CrossRefPubMedGoogle Scholar
  10. 10.
    Stritzke J, Linsel-Nitschke P, Markus MR, Mayer B, Lieb W, Luchner A, Doring A, Koenig W, Keil U, Hense HW, Schunkert H, Investigators MK (2009) Association between degenerative aortic valve disease and long-term exposure to cardiovascular risk factors: results of the longitudinal population-based KORA/MONICA survey. Eur Heart J 30(16):2044–2053. CrossRefPubMedGoogle Scholar
  11. 11.
    Colan SD, Shirali G, Margossian R, Gallagher D, Altmann K, Canter C, Chen S, Golding F, Radojewski E, Camitta M, Carboni M, Rychik J, Stylianou M, Tani LY, Selamet Tierney ES, Wang Y, Sleeper LA, Pediatric Heart Network I (2012) The ventricular volume variability study of the Pediatric Heart Network: study design and impact of beat averaging and variable type on the reproducibility of echocardiographic measurements in children with chronic dilated cardiomyopathy. J Am Soc Echocardiogr 25(8):842–854.e6. CrossRefPubMedPubMedCentralGoogle Scholar
  12. 12.
    Otterstad JE, Froeland G, St John Sutton M, Holme I (1997) Accuracy and reproducibility of biplane two-dimensional echocardiographic measurements of left ventricular dimensions and function. Eur Heart J 18(3):507–513CrossRefPubMedGoogle Scholar
  13. 13.
    Barnhart HX, Yow E, Crowley AL, Daubert MA, Rabineau D, Bigelow R, Pencina M, Douglas PS (2014) Choice of agreement indices for assessing and improving measurement reproducibility in a core laboratory setting. Stat Methods Med Res 25(6):2939–2958. CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media B.V., part of Springer Nature 2018

Authors and Affiliations

  • Caroline Morbach
    • 1
    • 2
  • Götz Gelbrich
    • 3
    • 4
  • Margret Breunig
    • 1
    • 2
  • Theresa Tiffe
    • 1
    • 3
  • Martin Wagner
    • 1
    • 3
  • Peter U. Heuschmann
    • 3
    • 4
  • Stefan Störk
    • 1
    • 2
    Email author
  1. 1.Comprehensive Heart Failure CenterUniversity Hospital and University of WürzburgWürzburgGermany
  2. 2.Department of Medicine I, CardiologyUniversity Hospital WürzburgWürzburgGermany
  3. 3.Institute of Clinical Epidemiology and BiometryUniversity of WürzburgWürzburgGermany
  4. 4.Clinical Trial CenterUniversity Hospital WürzburgWürzburgGermany

Personalised recommendations