Skip to main content

Advertisement

SpringerLink
Log in

Automatic Tracking of Hyoid Bone Displacement and Rotation Relative to Cervical Vertebrae in Videofluoroscopic Swallow Studies Using Deep Learning

  • Published:
Journal of Imaging Informatics in Medicine Aims and scope Submit manuscript

Abstract

The hyoid bone displacement and rotation are critical kinematic events of the swallowing process in the assessment of videofluoroscopic swallow studies (VFSS). However, the quantitative analysis of such events requires frame-by-frame manual annotation, which is labor-intensive and time-consuming. Our work aims to develop a method of automatically tracking hyoid bone displacement and rotation in VFSS. We proposed a full high-resolution network, a deep learning architecture, to detect the anterior and posterior of the hyoid bone to identify its location and rotation. Meanwhile, the anterior-inferior corners of the C2 and C4 vertebrae were detected simultaneously to automatically establish a new coordinate system and eliminate the effect of posture change. The proposed model was developed by 59,468 VFSS frames collected from 1488 swallowing samples, and it achieved an average landmark localization error of 2.38 pixels (around 0.5% of the image with 448 × 448 pixels) and an average angle prediction error of 0.065 radians in predicting C2–C4 and hyoid bone angles. In addition, the displacement of the hyoid bone center was automatically tracked on a frame-by-frame analysis, achieving an average mean absolute error of 2.22 pixels and 2.78 pixels in the x-axis and y-axis, respectively. The results of this study support the effectiveness and accuracy of the proposed method in detecting hyoid bone displacement and rotation. Our study provided an automatic method of analyzing hyoid bone kinematics during VFSS, which could contribute to early diagnosis and effective disease management.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data Availability

Data generated or used for the study are available from the corresponding author by request.

References

  1. M. Aslam and M. F. Vaezi, ‘Dysphagia in the elderly’, Gastroenterol. Hepatol. (N. Y.), vol. 9, no. 12, pp. 784–795, Dec. 2013.

  2. M. Crary, L. Sura, A. Madhavan, and G. Carnaby-Mann, “Dysphagia in the elderly: management and nutritional considerations,” Clinical Interventions in Aging, p. 287, 7 2012.

  3. A. M. Namasivayam and C. M. Steele, “Malnutrition and dysphagia in long-term care: A systematic review,” Journal of Nutrition in Gerontology and Geriatrics, vol. 34, pp. 1–21, 1 2015.

  4. C. M. Sarabia-Cobo et al., ‘The incidence and prognostic implications of dysphagia in elderly patients institutionalized: A multicenter study in Spain’, Appl. Nurs. Res., vol. 30, pp. e6-9, May 2016.

  5. M. Streicher, R. Wirth, K. Schindler, C. C. Sieber, M. Hiesmayr, and D. Volkert, “Dysphagia in nursing homes—results from the nutritionday project,” Journal of the American Medical Directors Association, vol. 19, pp. 141–147.e2, 2 2018.

  6. P. E. Spronk, L. E. J. Spronk, J. Lut, E. Gnacke, D. Mijnes, B. van Munster, and A. Kroner, “Prevalence and characterization of dysphagia in hospitalized¨ patients,” Neurogastroenterology & Motility, vol. 32, 3 2020.

    Article  Google Scholar 

  7. M. González-Fernández, L. Ottenstein, L. Atanelov, and A. B. Christian, ‘Dysphagia after stroke: An overview’, Curr. Phys. Med. Rehabil. Rep., vol. 1, no. 3, pp. 187–196, Sep. 2013.

  8. R. Martino, N. Foley, S. Bhogal, N. Diamant, M. Speechley, and R. Teasell, “Dysphagia after stroke,” Stroke, vol. 36, pp. 2756–2763, 12 2005.

  9. J. Logemann, “Evaluation and treatment of swallowing disorders,” NSSLHA Journal, vol. 12, pp. 38–50, 11 1984.

  10. R. Martino, G. Pron, and N. Diamant, “Oropharyngeal dysphagia: Surveying practice patterns of the speech?language pathologist,” Dysphagia, vol. 19, 8 2004.

    Article  Google Scholar 

  11. B. Martin-Harris and B. Jones, “The videofluorographic swallowing study,” Physical Medicine and Rehabilitation Clinics of North America, vol. 19, pp. 769–785, 11 2008.

  12. G. Dengel, J. Robbins, and J. C. Rosenbek, “Image processing in swallowing and speech research,” Dysphagia, vol. 6, pp. 30–39, 3 1991.

  13. J. Logemann, P. Kahrilas, J. Begelman, W. Dodds, and B. Pauloski, “Interactive computer program for biomechanical analysis of videoradiographic studies of swallowing,” American Journal of Roentgenology, vol. 153, pp. 277–280, 8 1989.

  14. W. J. Dodds, J. A. Logemann, and E. T. Stewart, “Radiologic assessment of abnormal oral and pharyngeal phases of swallowing.,” American Journal of Roentgenology, vol. 154, pp. 965–974, 5 1990.

  15. J. G. J. van der Kruis, L. W. J. Baijens, R. Speyer, and I. Zwijnenberg, “Biomechanical analysis of hyoid bone displacement in videofluoroscopy: A systematic review of intervention effects,” Dysphagia, vol. 26, pp. 171–182, 6 2011.

  16. H. Opdebeeck, W. H. Bell, J. Eisenfeld, and D. Mishelevich, “Comparative study between the SFS and LFS rotation as a possible morphogenic mechanism,” Am. J. Orthod., vol. 74, pp. 509–521, Nov. 1978.

  17. N. Jose, S. Shetty, S. Mogra, V. Shetty, S. Rangarajan, and L. Mary, “Evaluation of hyoid bone position and its correlation with pharyngeal airway space in different types of skeletal malocclusion,” Contemporary Clinical Dentistry, vol. 5, p. 187, 2014.

    Article  PubMed  PubMed Central  Google Scholar 

  18. I. P. Adamidis and M. N. Spyropoulos, “The effects of lymphadenoid hypertrophy on the position of the tongue, the mandible and the hyoid bone,” Eur. J. Orthod., vol. 5, pp. 287–294, Nov. 1983.

  19. A. L. Perlman, D. J. VanDaele, and M. S. Otterbacher, “Quantitative assessment of hyoid bone displacement from video images during swallowing,” Journal of Speech, Language, and Hearing Research, vol. 38, pp. 579–585, 6 1995.

  20. G. H. McCullough, R. T. Wertz, J. C. Rosenbek, R. H. Mills, W. G. Webb, and K. B. Ross, “Inter- and intrajudge reliability for videofluoroscopic swallowing evaluation measures,” Dysphagia, vol. 16, pp. 110–118, 3 2001.

  21. M. A. Crary, G. D. C. (Mann), and M. E. Groher, “Biomechanical correlates of surface electromyography signals obtained during swallowing by healthy adults,” Journal of Speech, Language, and Hearing Research, vol. 49, pp. 186–193, 2 2006.

  22. P. M. Kellen, D. L. Becker, J. M. Reinhardt, and D. J. V. Daele, “Computer-assisted assessment of hyoid bone motion from videofluoroscopic swallow studies,” Dysphagia, vol. 25, pp. 298–306, 12 2010.

  23. I. Hossain, A. Roberts-South, M. Jog, and M. R. El-Sakka, “Semi-automatic assessment of hyoid bone motion in digital videofluoroscopic images,” Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, vol. 2, pp. 25–37, 1 2014.

  24. S. Mao, Z. Zhang, Y. Khalifa, C. Donohue, J. L. Coyle, and E. Sejdic, “Neck sensor-supported hyoid bone movement tracking during swallowing,” Royal Society Open Science, vol. 6, p. 181982, 7 2019.

  25. A. Voulodimos, N. Doulamis, A. Doulamis, and E. Protopapadakis, “Deep learning for computer vision: A brief review,” Computational Intelligence and Neuroscience, vol. 2018, pp. 1–13, 2018.

    Google Scholar 

  26. D. Shen, G. Wu, and H.-I. Suk, “Deep learning in medical image analysis.,” Annual review of biomedical engineering, vol. 19, pp. 221–248, 2017.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Z. Zhang, S. Mao, J. Coyle, and E. Sejdić, ‘Automatic annotation of cervical vertebrae in videofluoroscopy images via deep learning’, Med. Image Anal., vol. 74, no. 102218, p. 102218, Dec. 2021.

  28. Z. Zhang, J. L. Coyle, and E. Sejdić, ‘Automatic hyoid bone detection in fluoroscopic images using deep learning’, Sci. Rep., vol. 8, no. 1, p. 12310, Aug. 2018.

  29. D. Lee, W. H. Lee, H. G. Seo, B.-M. Oh, J. C. Lee, and H. C. Kim, “Online learning for the hyoid bone tracking during swallowing with neck movement adjustment using semantic segmentation,” IEEE Access, vol. 8, pp. 157451–157461, 2020.

    Article  Google Scholar 

  30. H.-I. Kim, Y. Kim, B. Kim, D. Y. Shin, S. J. Lee, and S.-I. Choi, “Hyoid bone tracking in a videofluoroscopic swallowing study using a deep-learning-based segmentation network,” Diagnostics, vol. 11, p. 1147, 6 2021.

  31. M.-Y. Hsiao, C.-H. Weng, Y.-C. Wang, S.-H. Cheng, K.-C. Wei, P.-Y. Tung, J.-Y. Chen, C.-Y. Yeh, and T.-G. Wang, “Deep learning for automatic hyoid tracking in videofluoroscopic swallow studies,” Dysphagia, 4 2022.

  32. S. M. Molfenter and C. M. Steele, “Kinematic and temporal factors associated with penetration–aspiration in swallowing liquids,” Dysphagia, vol. 29, pp. 269–276, 4 2014.

  33. Y. Matsuda, E. Ito, Y. Kimura, and K. Araki, “Hyoid bone position related to gender and aging using lateral cephalometric radiographs,” Orthod. Waves, vol. 77, pp. 226–231, Dec. 2018.

  34. K. Sun, Y. Zhao, B. Jiang, T. Cheng, B. Xiao, D. Liu, Y. Mu, X. Wang, W. Liu, and J. Wang, “High-resolution representations for labeling pixels and regions,” 4 2019.

  35. K. Sun, B. Xiao, D. Liu, and J. Wang, “Deep high-resolution representation learning for human pose estimation,” 2 2019.

  36. J. Wang, K. Sun, T. Cheng, B. Jiang, C. Deng, Y. Zhao, D. Liu, Y. Mu, M. Tan, X. Wang, W. Liu, and B. Xiao, “Deep high-resolution representation learning for visual recognition,” 8 2019.

  37. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 12 2014.

  38. K. Weiss, T. M. Khoshgoftaar, and D. Wang, “A survey of transfer learning,” Journal of Big Data, vol. 3, p. 9, 12 2016.

  39. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” pp. 248–255, IEEE, 6 2009.

  40. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 12 2015.

  41. Y. Chen, Z. Wang, Y. Peng, Z. Zhang, G. Yu, and J. Sun, “Cascaded pyramid network for multi-person pose estimation,” 11 2017.

  42. B. Xiao, H. Wu, and Y. Wei, “Simple baselines for human pose estimation and tracking”, arXiv [cs.CV], 17-Apr-2018.

  43. A. Ross and V. L. Willson, ‘Independent Samples T-Test’, in Basic and Advanced Statistical Tests, Rotterdam: SensePublishers, 2017, pp. 13–16.

    Google Scholar 

  44. A. Ross and V. L. Willson, “Paired Samples T-Test”, in Basic and Advanced Statistical Tests, Rotterdam: SensePublishers, 2017, pp. 17–19.

    Google Scholar 

  45. B. Cheng, B. Xiao, J. Wang, H. Shi, T. S. Huang, and L. Zhang, “Higherhrnet: Scale-aware representation learning for bottom-up human pose estimation,” 8 2019.

Download references

Funding

This work was supported by the Eunice Kennedy Shriver National Institute of Child Health & Human Development of the National Institute of Health under Award Number R01HD092239, while the data were collected under Award Number R01HD074819. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Health.

Author information

Authors and Affiliations

  1. Edward S. Rogers Department of Electrical and Computer Engineering, University of Toronto, Toronto, ON, Canada

    Wuqi Li & Ervin Sejdić

  2. Department of Head and Neck Surgery, The University of Texas MD Anderson Cancer Center, Houston, TX, USA

    Shitong Mao

  3. Department of Communication Science and Disorders, School of Health and Rehabilitation Sciences, University of Pittsburgh, Pittsburgh, PA, USA

    Amanda S. Mahoney & James L. Coyle

  4. North York General Hospital, Toronto, ON, Canada

    Ervin Sejdić

Authors
  1. Wuqi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shitong Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amanda S. Mahoney
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. James L. Coyle
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ervin Sejdić
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ervin Sejdić.

Ethics declarations

Ethics Approval

The Institutional Review Board at the University of Pittsburgh approved the protocol of this study, and all participants provided informed consent.

Competing Interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, W., Mao, S., Mahoney, A.S. et al. Automatic Tracking of Hyoid Bone Displacement and Rotation Relative to Cervical Vertebrae in Videofluoroscopic Swallow Studies Using Deep Learning. J Digit Imaging. Inform. med. (2024). https://doi.org/10.1007/s10278-024-01039-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10278-024-01039-4

Keywords

Search

Navigation