Abstract
A common difficulty in computer-assisted diagnosis is acquiring accurate and representative labeled data, required to train, test and monitor models. Concerning liver steatosis detection in ultrasound (US) images, labeling images with human annotators can be error-prone because of subjectivity and decision boundary biases. To overcome these limits, we propose comparative visual labeling (CVL), where an annotator labels the relative degree of a pathology in image pairs, that is combined with a RankNet to give per-image diagnostic scores. In a multi-annotator evaluation on a public steatosis dataset, CVL+RankNet significantly improves label quality compared to conventional single-image visual labeling (SVL) (0.97 versus 0.87 F1-score respectively, 95% CI significance). This is the first application of CVL for diagnostic medical image labeling, and it may stimulate more research for other diagnostic labeling tasks. We also show that Deep Learning (DL) models trained with CVL+RankNet or histopathology labels attain similar performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Araújo, A.R., et al.: Global epidemiology of non-alcoholic fatty liver disease/non-alcoholic steatohepatitis: what we need in the future. Liver Int. 38, 47–51 (2018)
Bang, K.B., Cho, Y.K.: Comorbidities and metabolic derangement of NAFLD. J. Lifestyle Med. 5(1), 7–13 (2015)
Bedossa, P.: Pathology of non-alcoholic fatty liver disease. Liver Int. 37(Suppl. 1), 85–89 (2017)
Burges, C., et al.: Learning to rank using gradient descent. In: Proceedings of the 22nd International Conference on Machine Learning, ICML 2005, Bonn, Germany, pp. 89–96. ACM, New York (2005). https://doi.org/10.1145/1102351.1102363. ISBN 1595931805
Byra, M., et al.: Transfer learning with deep convolutional neural network for liver steatosis assessment in ultrasound images. Int. J. Comput. Assist. Radiol. Surg. 41(1), 175–184 (2018)
Byra, M., et al.: Liver fat assessment in multiview sonography using transfer learning with convolutional neural networks. J. Ultrasound Med. 41(1), 175–184 (2021)
Chalasani, N., et al.: The diagnosis and management of non-alcoholic fatty liver disease: practice guideline by the American association for the study of liver diseases, American college of gastroenterology, and the American gastroenterological association. Hepatology 55(6), 2005–2023 (2012)
Doughty, H., Damen, D., Mayol-Cuevas, W.: Who’s better? Who’s best? pairwise deep ranking for skill determination. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6057–6066. IEEE Computer Society, Los Alamitos (2018). https://doi.org/10.1109/CVPR.2018.00634. https://doi.ieeecomputersociety.org/10.1109/CVPR.2018.00634
Dulai, P.S., Sirlin, C.B., Loomba, R.: MRI and MRE for non-invasive quantitative assessment of hepatic steatosis and fibrosis in NAFLD and NASH: clinical trials to clinical practice. J. Hepatol. 65(5), 1006–1016 (2016)
Khov, N., Sharma, A., Riley, T.R.: Bedside ultrasound in the diagnosis of nonalcoholic fatty liver disease. World J. Gastroenterol. WJG 20(22), 6821–6825 (2014)
Köppel, M., Segner, A., Wagener, M., Pensel, L., Karwath, A., Kramer, S.: Pairwise learning to rank by neural networks revisited: reconstruction, theoretical analysis and practical performance. In: Brefeld, U., Fromont, E., Hotho, A., Knobbe, A., Maathuis, M., Robardet, C. (eds.) ECML PKDD 2019. LNCS (LNAI), vol. 11908, pp. 237–252. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-46133-1_15
Kou, N.M., Li, Y., Wang, H., Leong Hou, U., Gong, Z.: Crowdsourced top-k queries by confidence-aware pairwise judgments. In: Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD 2017, Chicago, Illinois, USA, pp. 1415–1430. ACM, New York (2017). https://doi.org/10.1145/3035918.3035953. ISBN 9781450341974
Lee, S.S., Park, S.H.: Radiologic evaluation of nonalcoholic fatty liver disease. World J. Gastroenterol. WJG 20(23), 7392–7402 (2014)
Li, B., et al.: Learning from subjective ratings using auto-decoded deep latent embeddings. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12905, pp. 270–280. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87240-3_26. ISBN 978-3-030-87239-7
Lyu, J., et al.: Ultrasound volume projection image quality selection by ranking from convolutional ranknet. Comput. Med. Imaging Graph. 89, 7392–7402 (2021)
Petzold, G., et al.: Diagnostic accuracy of b-mode ultrasound and hepatorenal index for graduation of hepatic steatosis in patients with chronic liver disease. PLoS One 15(5), e0231044 (2020)
Prathama, F., et al.: Personalized recommendation by matrix cofactorization with multiple implicit feedback on pairwise comparison. Comput. Ind. Eng. 152, 295–318 (2021)
Rhyou, S.-Y., Yoo, J.-C.: Cascaded deep learning neural network for automated liver steatosis diagnosis using ultrasound images. Sensors 21(16), 5304 (2021)
Rinella, M.E., Sanyal, A.J.: Management of NAFLD: a stage-based approach. Nat. Rev. Gastroenterol. Hepatol. 13(4), 196–205 (2016)
Sekachev, B., et al.: opencv/cvat: v1.1.0. https://doi.org/10.5281/zenodo.4009388
Selvaraju, R.R., et al.: Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626 (2017)
Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI 2017, San Francisco, California, USA, pp. 4278–4284. AAAI Press (2017)
Talebi, H., et al.: Rank-smoothed pairwise learning in perceptual quality assessment. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 3413–3417 (2020)
Tapper, E.B., Lok, A.S.-F.: Use of liver imaging and biopsy in clinical practice. N. Engl. J. Med. 377(8), 756–768 (2017)
The GIMP Development Team. GIMP. Version 2.10.12, 12 June 2019. https://www.gimp.org
Younossi, Z.M., et al.: Global epidemiology of nonalcoholic fatty liver disease-meta-analytic assessment of prevalence, incidence, and outcomes. Hepatology 64(4), 1388–1389 (2016)
Yu, X., et al.: Group-aware contrastive regression for action quality assessment. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7899–7908 (2021)
Zamanian, H., et al.: Implementation of combinational deep learning algorithm for non-alcoholic fatty liver classification in ultrasound images. J. Biomed. Phys. Eng. 11(1), 73–84 (2021)
Zhang, Z., et al.: An improved pairwise comparison scaling method for subjective image quality assessment. In: 2017 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), pp. 1–6 (2017)
Acknowledgements
This work has received funding from France’s Région Grand Est. We also greatly thank the annotators for their invaluable participation and work on this study.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Saibro, G., Diana, M., Sauer, B., Marescaux, J., Hostettler, A., Collins, T. (2022). Automatic Detection of Steatosis in Ultrasound Images with Comparative Visual Labeling. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13433. Springer, Cham. https://doi.org/10.1007/978-3-031-16437-8_39
Download citation
DOI: https://doi.org/10.1007/978-3-031-16437-8_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16436-1
Online ISBN: 978-3-031-16437-8
eBook Packages: Computer ScienceComputer Science (R0)
