Skip to main content
SpringerLink
Log in

Automatic and Objective Facial Palsy Grading Index Prediction Using Deep Feature Regression

  • Conference paper
  • First Online:
Medical Image Understanding and Analysis (MIUA 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1248))

Included in the following conference series:

Abstract

One of the main reasons for a half-sided facial paralysis is a dysfunction of the facial nerve. Physicians have to assess such a unilateral facial palsy with the help of standardized grading scales to evaluate the treatment. However, such assessments are usually very subjective and they are prone to variance and inconsistency between physicians due to their varying experience. We propose an automatic non-biased method using deep features combined with a linear regression method for facial palsy grading index prediction. With an extension of the free software tool Auto-eFace we annotated images of facial palsy patients and healthy subjects according to a common facial palsy grading scale. In our experiments, we obtained an average grading error of 11%.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/dguari1/Auto-eFace.

References

  1. Banks, C.A., Bhama, P.K., Park, J., Hadlock, C.R., Hadlock, T.A.: Clinician-graded electronic facial paralysis assessment: the eface. Plast. Reconstr. Surg. 136(2), 223e–230e (2015)

    Article  Google Scholar 

  2. Barbosa, J., et al.: Efficient quantitative assessment of facial paralysis using iris segmentation and active contour-based key points detection with hybrid classifier. BMC Med. Imaging 16(1), 23 (2016)

    Article  Google Scholar 

  3. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  Google Scholar 

  4. Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)

    Article  Google Scholar 

  5. Daugman, J.: How iris recognition works. In: The Essential Guide to Image Processing, pp. 715–739. Elsevier (2009)

    Google Scholar 

  6. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)

    Google Scholar 

  7. Donahue, J., et al.: Decaf: a deep convolutional activation feature for generic visual recognition. In: International Conference on Machine Learning, pp. 647–655 (2014)

    Google Scholar 

  8. Finsterer, J.: Management of peripheral facial nerve palsy. Eur. Arch. Otorhinolaryngol. 265(7), 743–752 (2008)

    Article  Google Scholar 

  9. Gaber, A., Faher, M.F., Wahed, M.A.: Automated grading of facial paralysis using the kinect v2: a proof of concept study. In: International Conference on Virtual Rehabilitation, pp. 258–264. IEEE (2015)

    Google Scholar 

  10. Gaber, A., Taher, M.F., Wahed, M.A.: Quantifying facial paralysis using the kinect v2. In: Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 2497–2501. IEEE (2015)

    Google Scholar 

  11. Guarin, D.L., Dusseldorp, J., Hadlock, T.A., Jowett, N.: A machine learning approach for automated facial measurements in facial palsy. JAMA Facial Plast. Surg. 20(4), 335–337 (2018)

    Article  Google Scholar 

  12. Guarin, D.L., et al.: Toward an automatic system for computer-aided assessment in facial palsy. arXiv preprint arXiv:1910.11497 (2019)

  13. Haase, D., Kemmler, M., Guntinas-Lichius, O., Denzler, J.: Efficient measuring of facial action unit activation intensities using active appearance models. In: Machine Vision Applications, pp. 141–144 (2013)

    Google Scholar 

  14. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778. IEEE (2016)

    Google Scholar 

  15. House, J.W., Brackmann, D.E.: Facial nerve grading system. Otolaryngol. Head Neck Surg. 93(2), 146–147 (1985)

    Article  Google Scholar 

  16. Howard, A.G., et al.: Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)

  17. Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867–1874 (2014)

    Google Scholar 

  18. King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)

    Google Scholar 

  19. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105 (2012)

    Google Scholar 

  20. Lankton, S., Tannenbaum, A.: Localizing region-based active contours. IEEE Trans. Image Process. 17(11), 2029–2039 (2008)

    Article  MathSciNet  Google Scholar 

  21. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  22. Lin, T.Y., RoyChowdhury, A., Maji, S.: Bilinear CNN models for fine-grained visual recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1449–1457 (2015)

    Google Scholar 

  23. Modersohn, L., Denzler, J.: Facial paresis index prediction by exploiting active appearance models for compact discriminative features. In: International Conference on Computer Vision Theory and Applications, pp. 271–278 (2016)

    Google Scholar 

  24. Morales, D.R., Donnan, P.T., Daly, F., Staa, T.V., Sullivan, F.M.: Impact of clinical trial findings on bell’s palsy management in general practice in the uk 2001–2012: interrupted time series regression analysis. BMJ Open 3(7), e003121 (2013)

    Article  Google Scholar 

  25. Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: British Machine Vision Conference, p. 6 (2015)

    Google Scholar 

  26. Plumbaum, K., et al.: Inpatient treatment of patients with acute idiopathic peripheral facial palsy: a population-based healthcare research study. Clin. Otolaryngol. 42(6), 1267–1274 (2017)

    Article  Google Scholar 

  27. Ross, B.G., Fradet, G., Nedzelski, J.M.: Development of a sensitive clinical facial grading system. Otolaryngol. Head Neck Surg. 114(3), 380–386 (1996)

    Article  Google Scholar 

  28. Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: the first facial landmark localization challenge. In: IEEE International Conference on Computer Vision - Workshops, pp. 397–403 (2013)

    Google Scholar 

  29. Schaede, R.A., Volk, G.F., Modersohn, L., Barth, J.M., Denzler, J., Guntinas-Lichius, O.: Video instruction for synchronous video recording of mimic movement of patients with facial palsy. Laryngo-Rhino-Otologie (2017)

    Google Scholar 

  30. Simon, M., Rodner, E.: Neural activation constellations: unsupervised part model discovery with convolutional networks. In: IEEE International Conference on Computer Vision, pp. 1143–1151. IEEE (2015)

    Google Scholar 

  31. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  32. Stennert, E., Limberg, C., Frentrup, K.: An index for paresis and defective healing-an easily applied method for objectively determining therapeutic results in facial paresis (author’s transl). HNO 25(7), 238–245 (1977)

    Google Scholar 

  33. Tan, M., Le, Q.: EfficientNet: rethinking model scaling for convolutional neural networks. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Learning Research. PMLR, Long Beach, California, USA, 09–15 June 2019, vol. 97, pp. 6105–6114 (2019)

    Google Scholar 

  34. Thielker, J., Geißler, K., Granitzka, T., Klingner, C., Volk, G., Guntinas-Lichius, O.: Acute management of bell’s palsy. Curr. Otorhinolaryngol. Rep. 6(2), 161–170 (2018)

    Article  Google Scholar 

  35. Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995). https://doi.org/10.1007/978-1-4757-3264-1

    Book  MATH  Google Scholar 

  36. Volk, G.F., et al.: Functional outcome and quality of life after hypoglossal-facial jump nerve suture. Front. Surg. 7, 11 (2020)

    Article  Google Scholar 

  37. Volk, G.F., et al.: Reliability of grading of facial palsy using a video tutorial with synchronous video recording. The Laryngoscope 129(10), 2274–2279 (2019)

    Article  Google Scholar 

Download references

Acknowledgments

The research was supported by grant DE 735/15-1 and GU 463/12-1 of the German Research Foundation (DFG). We also gratefully acknowledge the support of NVIDIA Corporation with the donation of Titan Xp GPUs used for this research.

Author information

Authors and Affiliations

  1. Computer Vision Group, Friedrich Schiller University Jena, Ernst-Abbe-Platz 2, 07743, Jena, Germany

    Anish Raj, Oliver Mothes, Sven Sickert & Joachim Denzler

  2. Department of Otorhinolaryngology, Jena University Hospital, Am Klinikum 1, 07747, Jena, Germany

    Gerd Fabian Volk & Orlando Guntinas-Lichius

Authors
  1. Anish Raj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Oliver Mothes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sven Sickert
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gerd Fabian Volk
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Orlando Guntinas-Lichius
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Joachim Denzler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oliver Mothes .

Editor information

Editors and Affiliations

  1. University of Oxford, Oxford, UK

    Bartłomiej W. Papież

  2. University of Oxford, Oxford, UK

    Ana I. L. Namburete

  3. University of Oxford, Oxford, UK

    Mohammad Yaqub

  4. University of Oxford, Oxford, UK

    J. Alison Noble

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Raj, A., Mothes, O., Sickert, S., Volk, G.F., Guntinas-Lichius, O., Denzler, J. (2020). Automatic and Objective Facial Palsy Grading Index Prediction Using Deep Feature Regression. In: Papież, B., Namburete, A., Yaqub, M., Noble, J. (eds) Medical Image Understanding and Analysis. MIUA 2020. Communications in Computer and Information Science, vol 1248. Springer, Cham. https://doi.org/10.1007/978-3-030-52791-4_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-52791-4_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-52790-7

  • Online ISBN: 978-3-030-52791-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation