Optimising Knee Injury Detection with Spatial Attention and Validating Localisation Ability

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12722)


This work employs a pre-trained, multi-view Convolutional Neural Network (CNN) with a spatial attention block to optimise knee injury detection. An open-source Magnetic Resonance Imaging (MRI) data set with image-level labels was leveraged for this analysis. As MRI data is acquired from three planes, we compare our technique using data from a single-plane and multiple planes (multi-plane). For multi-plane, we investigate various methods of fusing the planes in the network. This analysis resulted in the novel ‘MPFuseNet’ network and state-of-the-art Area Under the Curve (AUC) scores for detecting Anterior Cruciate Ligament (ACL) tears and Abnormal MRIs, achieving AUC scores of 0.977 and 0.957 respectively. We then developed an objective metric, Penalised Localisation Accuracy (PLA), to validate the model’s localisation ability. This metric compares binary masks generated from Grad-Cam output and the radiologist’s annotations on a sample of MRIs. We also extracted explainability features in a model-agnostic approach that were then verified as clinically relevant by the radiologist.


Deep learning Musculoskeletal Magnetic Resonance Imaging Medical imaging Spatial attention Explainability 



This work was funded by Science Foundation Ireland through the SFI Centre for Research Training in Machine Learning (18/CRT/6183). This work is supported by the Insight Centre for Data Analytics under Grant Number SFI/12/RC/2289_P2. This work was funded by Enterprise Ireland Commercialisation Fund under Grant Number CF 201912481. We would also like to acknowledge and thank Professor Kevin McGuinness from Dublin City University (DCU) for his guidance on the research study.


  1. 1.
    Arun, N., et al.: Assessing the (un)trustworthiness of saliency maps for localizing abnormalities in medical imaging (2020)Google Scholar
  2. 2.
    Azcona, D., McGuinness, K., Smeaton, A.F.: A comparative study of existing and new deep learning methods for detecting knee injuries using the MRNet dataset. arXiv preprint arXiv:2010.01947 (2020)
  3. 3.
    Bien, N., et al.: Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of MRNet. PLOS Med. 15(11), 1–19 (2018). Scholar
  4. 4.
    Chattopadhay, A., Sarkar, A., Howlader, P., Balasubramanian, V.N.: Grad-CAM++: generalized gradient-based visual explanations for deep convolutional networks. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 839–847 (2018).
  5. 5.
    Deng, J., Dong, W., Socher, R., Li, L., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009).
  6. 6.
    Floruss, J., Vahlpahl, N.: Artificial intelligence in healthcare: acceptance of AI-based support systems by healthcare professionals (2020)Google Scholar
  7. 7.
    Graziani, M., Lompech, T., Müller, H., Andrearczyk, V.: Evaluation and comparison of CNN visual explanations for histopathology (2020)Google Scholar
  8. 8.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016).
  9. 9.
    Holzinger, A., Carrington, A., Müller, H.: Measuring the quality of explanations: the system causability scale (SCS). KI - Künstliche Intelligenz 34(2), 193–198 (2020). Scholar
  10. 10.
    Irmakci, I., Anwar, S.M., Torigian, D.A., Bagci, U.: Deep learning for musculoskeletal image analysis. In: 2019 53rd Asilomar Conference on Signals, Systems, and Computers, pp. 1481–1485. IEEE (2019).
  11. 11.
    Janik, A., Dodd, J., Ifrim, G., Sankaran, K., Curran, K.: Interpretability of a deep learning model in the application of cardiac MRI segmentation with an ACDC challenge dataset. In: Išgum, I., Landman, B.A. (eds.) Medical Imaging 2021: Image Processing, vol. 11596, pp. 852–863. International Society for Optics and Photonics, SPIE (2021).
  12. 12.
    Awan, M.J., Rahim, M.S.M., Salim, N., Mohammed, M.A., Garcia-Zapirain, B., Abdulkareem, K.H.: Efficient detection of knee anterior cruciate ligament from magnetic resonance imaging using deep learning approach. Diagnostics 11(1) (2021).
  13. 13.
    Kennedy, M., Dunne, C., Mulcahy, B., Molloy, M.: The sports’ clinic: a one year review of new referrals. Ir. Med. J. 86(1), 29–30 (1993)Google Scholar
  14. 14.
    Kim, B., et al.: Interpretability beyond feature attribution: quantitative testing with concept activation vectors (TCAV) (2018)Google Scholar
  15. 15.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2017)Google Scholar
  16. 16.
    Kosaka, M., et al.: Oblique coronal and oblique sagittal MRI for diagnosis of anterior cruciate ligament tears and evaluation of anterior cruciate ligament remnant tissue. Knee 21(1), 54–57 (2014). Scholar
  17. 17.
    Mader, K.: Attention on pretrained-vgg16 for bone age (2018).
  18. 18.
    Nyúl, L.G., Udupa, J.K.: On standardizing the MR image intensity scale. Magn. Reson. Med. 42(6), 1072–1081 (1999)CrossRefGoogle Scholar
  19. 19.
    Oktay, O., et al.: Attention U-Net: learning where to look for the pancreas (2018)Google Scholar
  20. 20.
    Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. Int. J. Comput. Vision 128(2), 336–359 (2019). Scholar
  21. 21.
    Skinner, S.: MRI of the knee. Aust. Fam. Physician 41(11), 867–869 (2012)Google Scholar
  22. 22.
    Smilkov, D., Thorat, N., Kim, B., Viégas, F., Wattenberg, M.: SmoothGrad: removing noise by adding noise (2017)Google Scholar
  23. 23.
    Su, H., Maji, S., Kalogerakis, E., Learned-Miller, E.: Multi-view convolutional neural networks for 3D shape recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 945–953 (2015).
  24. 24.
    Tao, Q., Ge, Z., Cai, J., Yin, J., See, S.: Improving deep lesion detection using 3d contextual and spatial attention (2019).
  25. 25.
    Tsai, C.H., Kiryati, N., Konen, E., Eshed, I., Mayer, A.: Knee injury detection using MRI with efficiently-layered network (ELNet). In: Proceedings of the Third Conference on Medical Imaging with Deep Learning, Montreal, QC, Canada. Proceedings of Machine Learning Research, vol. 121, pp. 784–794. PMLR, 06–08 July 2020.

Copyright information

© Springer Nature Switzerland AG 2021

Authors and Affiliations

  1. 1.Science Foundation Ireland Centre for Research Training in Machine LearningDublinIreland
  2. 2.School of MedicineUniversity College DublinDublinIreland
  3. 3.Department of RadiologyMater Misericordiae University HospitalDublinIreland
  4. 4.School of Electronic and Electrical EngineeringUniversity College DublinDublinIreland
  5. 5.School of Computer ScienceUniversity College DublinDublinIreland
  6. 6.Insight Centre for Data AnalyticsUniversity College DublinDublinIreland

Personalised recommendations