Skip to main content

Level of Agreement Between Human-Rated and Instrumented Balance Error Scoring System Scores

Annals of Biomedical Engineering volume 47, pages 2128–2135 (2019)Cite this article

Abstract

Clinicians have used the Balance Error Scoring System (BESS) to quantify postural control for concussion management. However, the reliability of the human rated BESS has varied prompting the development of instrumented BESSs. A cross-sectional design was used to determine the level of agreement (LOA) between human rated and instrumented BESS scores. Sixty participants completed the BESS on video. An instrumented mat was used to quantify BESS errors while a live human rater simultaneously scored the BESS. A second human rated BESS performance via video. Bland-Altman LOA analyses evaluated agreement between scoring methods (Mat-Human, Mat-Video, Video-Live) for each stance. Mean biases between scores, for each stance, with 95% confidence intervals (95%CIs) were calculated. Agreement between scoring methods was not assessed for the Firm-Double-Limb stance because very few errors were recorded. Agreement between both human raters and the mat was poor based on mean bias estimates > ± 1 and/or wide 95%CIs for all stances including BESS-Total. Agreement between the human raters was better, having displayed consistently smaller mean bias estimates and tighter 95%CIs for all stances and BESS Total. As a result, human rated and instrumented BESS scores may not be comparable. One method should be used to measure BESS errors for consistency.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

43,34 €

Price includes VAT (Finland)
Tax calculation will be finalised during checkout.

Figure 1
Figure 2
Figure 3
Figure 4

References

  1. Alberts, J. L., A. Thota, J. Hirsch, et al. Quantification of the balance error scoring system with mobile technology. Med. Sci. Sports Exerc. 47(10):2233–2240, 2015.

    Article  Google Scholar 

  2. Barlow, M., D. Schlabach, J. Peiffer, and C. Cook. Differences in change scores and the predictive validity of three commonly used measures following concussion in the middle school and high school aged population. Int. J. Sports Phys. Ther. 6(3):150–157, 2011.

    PubMed  PubMed Central  Google Scholar 

  3. Bell, D. R., K. M. Guskiewicz, M. A. Clark, and D. A. Padua. Systematic review of the balance error scoring system. Sports Health. 3(3):287–295, 2011.

    Article  Google Scholar 

  4. Broglio, S. P., R. C. Cantu, G. A. Gioia, et al. National Athletic Trainers’ Association position statement: management of sport concussion. J. Athl. Train. 49(2):245–265, 2014.

    Article  Google Scholar 

  5. Broglio, S. P., B. P. Katz, S. Zhao, M. McCrea, and T. McAllister. Test-retest reliability and interpretation of common concussion assessment tools: findings from the NCAA-DoD CARE Consortium. Sports Med. 48(5):1255–1268, 2018.

    Article  Google Scholar 

  6. Broglio, S. P., M. McCrea, T. McAllister, et al. A national study on the effects of concussion in collegiate athletes and US military service academy members: the NCAA-DoD Concussion Assessment, Research and Education (CARE) Consortium structure and methods. Sports Med. 47(7):1437–1451, 2017.

    Article  Google Scholar 

  7. Brown, H. J., G. P. Siegmund, K. M. Guskiewicz, K. Van Den Doel, E. Cretu, and J. S. Blouin. Development and validation of an objective balance error scoring system. Med. Sci. Sports Exerc. 46(8):1610–1616, 2014.

    Article  Google Scholar 

  8. Caccese, J. B., and T. W. Kaminski. Comparing computer-derived and human-observed scores for the balance error scoring system. J. Sport. Rehabil. 25(2):133–136, 2016.

    Article  Google Scholar 

  9. Chang, J. O., S. S. Levy, S. W. Seay, and D. J. Goble. An alternative to the balance error scoring system: using a low-cost balance board to improve the validity/reliability of sports-related concussion balance testing. Clin. J. Sport. Med. 24(3):256–262, 2014.

    Article  Google Scholar 

  10. Echemendia, R. J., W. Meeuwisse, P. McCrory, et al. The Sport Concussion Assessment Tool 5th Edition (SCAT5): background and rationale. Br J Sports Med 51(11):848–850, 2017.

    Article  Google Scholar 

  11. Finnoff, J. T., V. J. Peterson, J. H. Hollman, and J. Smith. Intrarater and interrater reliability of the Balance Error Scoring System (BESS). PM&R. 1(1):50–54, 2009.

    Article  Google Scholar 

  12. Gao, J., J. Hu, T. Buckley, K. White, and C. Hass. Shannon and renyi entropies to classify effects of mild traumatic brain injury on postural sway. PLoS ONE 6(9):e24446, 2011.

    CAS  Article  Google Scholar 

  13. Glass, S. M., A. Napoli, E. D. Thompson, I. Obeid, and C. A. Tucker. Validity of an automated balance error scoring system. J. Appl. Biomech. 35(1):32–36, 2018.

    Article  Google Scholar 

  14. Napoli, A., S. M. Glass, C. Tucker, and I. Obeid. The automated assessment of postural stability: balance detection algorithm. Ann. Biomed. Eng. 45(12):2784–2793, 2017.

    Article  Google Scholar 

  15. Patterson, J. A., R. Z. Amick, P. D. Pandya, N. Hakansson, and M. J. Jorgensen. Comparison of a mobile technology application with the balance error scoring system. Int. J. Athl. Ther. Train. 19(3):4–7, 2014.

    Article  Google Scholar 

  16. Powers, K. C., J. M. Kalmar, and M. E. Cinelli. Recovery of static stability following a concussion. Gait Posture. 39(1):611–614, 2014.

    Article  Google Scholar 

  17. Rahn, C., B. A. Munkasy, B. A. Joyner, and T. A. Buckley. Sideline performance of the Balance Error Scoring System during a live sporting event. Clin. J. Sport. Med. 25(3):248–253, 2015.

    Article  Google Scholar 

  18. Riemann, B. L., K. M. Guskiewicz, and E. W. Shields. Relationship between clinical and forceplate measures of postural stability. J. Sport Rehabil. 8(2):71–82, 1999.

    Article  Google Scholar 

  19. Slobounov, S., C. Cao, W. Sebastianelli, E. Slobounov, and K. Newell. Residual deficits from concussion as revealed by virtual time-to-contact measures of postural stability. Clin Neurophysiol. 119(2):281–289, 2008.

    Article  Google Scholar 

  20. Teyhen, D. S., S. W. Shaffer, J. A. Umlauf, et al. Automation to improve efficiency of field expedient injury prediction screening. J. Strength Cond. Res. 26(Suppl 2):S61–72, 2012.

    Article  Google Scholar 

  21. Valovich McLeod, T. C., W. B. Barr, M. McCrea, and K. M. Guskiewicz. Psychometric and measurement properties of concussion assessment tools in youth sports. J. Athl. Train. 41(4):399–408, 2006.

    PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

The views and opinions expressed are those of the investigators and they do not reflect the official position of the U.S. Army, the Department of Defense, or the U.S. Government. This work was supported by the Office of the Assistant Secretary of Defense for Health Affairs under Award # HU0001-15-2-0037/2988.

Author information

Affiliations

  1. John A. Feagin Jr. Sports Medicine Fellowship, Keller Army Hospital, United States Military Academy, 900 Washington Road, West Point, NY, 10996, USA

    Megan N. Houston, Steven R. Malvasi, Karen Y. Peck & Kenneth L. Cameron

  2. Sports Medicine Research Institute, University of Kentucky, Lexington, KY, USA

    Matthew C. Hoch

  3. MedStar Health, Washington, DC, USA

    Steven J. Svoboda

Authors
  1. Megan N. Houston
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthew C. Hoch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Steven R. Malvasi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Karen Y. Peck
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Steven J. Svoboda
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kenneth L. Cameron
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Megan N. Houston.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Associate Editor Stefan M. Duma oversaw the review of this article.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Houston, M.N., Hoch, M.C., Malvasi, S.R. et al. Level of Agreement Between Human-Rated and Instrumented Balance Error Scoring System Scores. Ann Biomed Eng 47, 2128–2135 (2019). https://doi.org/10.1007/s10439-019-02274-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10439-019-02274-5

Keywords

  • Postural stability
  • Concussion
  • Pressure mat