Skip to main content

Learning More for Free - A Multi Task Learning Approach for Improved Pathology Classification in Capsule Endoscopy

  • Conference paper
  • First Online:
Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 (MICCAI 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12907))


The progress in Computer Aided Diagnosis (CADx) of Wireless Capsule Endoscopy (WCE) is thwarted by the lack of data. The inadequacy in richly representative healthy and abnormal conditions results in isolated analyses of pathologies, that can not handle realistic multi-pathology scenarios. In this work, we explore how to learn more for free, from limited data through solving a WCE multicentric, multi-pathology classification problem. Learning more implies to learning more than full supervision would allow with the same data. This is done by combining self supervision with full supervision, under multi task learning. Additionally, we draw inspiration from the Human Visual System (HVS) in designing self supervision tasks and investigate if seemingly ineffectual signals within the data itself can be exploited to gain performance, if so, which signals would be better than others. Further, we present our analysis of the high level features as a stepping stone towards more robust multi-pathology CADx in WCE. Code accompanying this work will be made available on github.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others


  1. Ali, S., et al.: Endoscopy artifact detection (EAD 2019) challenge dataset. arXiv preprint arXiv:1905.03209 (2019)

  2. Atsawarungruangkit, A., Elfanagely, Y., Asombang, A.W., Rupawala, A., Rich, H.G.: Understanding deep learning in capsule endoscopy: can artificial intelligence enhance clinical practice? Artif. Intell. Gastrointest. Endosc. 1(2), 33–43 (2020)

    Article  Google Scholar 

  3. Baxter, J.: A bayesian/information theoretic model of learning to learn via multiple task sampling. Mach. Learn. 28(1), 7–39 (1997)

    Article  Google Scholar 

  4. Benton, A., Mitchell, M., Hovy, D.: Multitask learning for mental health conditions with limited social media data. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, vol. 1, Long Papers, pp. 152–162 (2017)

    Google Scholar 

  5. Bingel, J., Søgaard, A.: Identifying beneficial task relations for multi-task learning in deep neural networks. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, vol. 2, Short Papers, pp. 164–169. Association for Computational Linguistics, Valencia, Spain, April 2017.

  6. Caruana, R.: Multitask learning. Mach. Learn. 28(1), 41–75 (1997).

    Article  MathSciNet  Google Scholar 

  7. Caruana, R.: Multitask learning: a knowledge-based source of inductive bias. In: Proceedings of the Tenth International Conference on Machine Learning, pp. 41–48. Morgan Kaufmann (1993)

    Google Scholar 

  8. Ding, Z., et al.: Gastroenterologist-level identification of small-bowel diseases and normal variants by capsule endoscopy using a deep-learning model. Gastroenterology 157(4), 1044–1054 (2019)

    Article  Google Scholar 

  9. Hwang, Y., Park, J., Lim, Y.J., Chun, H.J.: Application of artificial intelligence in capsule endoscopy: where are we now? Clin. Endosc. 51(6), 547–551 (2018)

    Article  Google Scholar 

  10. Kendall, A., Gal, Y., Cipolla, R.: Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7482–7491 (2018)

    Google Scholar 

  11. Kokkinos, I.: Ubernet: training a universal convolutional neural network for low-, mid-, and high-level vision using diverse datasets and limited memory. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6129–6138 (2017)

    Google Scholar 

  12. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)

    Article  Google Scholar 

  13. Laiz, P., Vitria, J., Seguí, S.: Using the triplet loss for domain adaptation in WCE. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 399–405 (2019)

    Google Scholar 

  14. Leenhardt, R., Li, C., Le Mouel, J.P., Rahmi, G., Saurin, J.C., Cholet, F., Boureille, A., Amiot, X., Delvaux, M., Duburque, C., et al.: CAD-CAP: a 25,000-image database serving the development of artificial intelligence for capsule endoscopy. Endosc. Int. Open 8(3), E415 (2020)

    Article  Google Scholar 

  15. Liu, X., Gao, J., He, X., Deng, L., Duh, K., Wang, Y.y.: Representation learning using multi-task deep neural networks for semantic classification and information retrieval. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 912–921. Association for Computational Linguistics, Denver, Colorado, May–June 2015.,

  16. McAlindon, M.E., Ching, H.L., Yung, D., Sidhu, R., Koulaouzidis, A.: Capsule endoscopy of the small bowel. Ann. Transl. Med. 4(19), 369 (2016)

    Article  Google Scholar 

  17. McInnes, L., Healy, J., Melville, J.: UMAP: uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426 (2018)

  18. Misra, I., Shrivastava, A., Gupta, A., Hebert, M.: Cross-stitch networks for multi-task learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3994–4003 (2016)

    Google Scholar 

  19. Mohammed, A., Farup, I., Pedersen, M., Hovde, Ø., Yildirim Yayilgan, S.: Stochastic capsule endoscopy image enhancement. J. Imaging 4(6), 75 (2018)

    Article  Google Scholar 

  20. Muhammad, K., Khan, S., Kumar, N., Del Ser, J., Mirjalili, S.: Vision-based personalized wireless capsule endoscopy for smart healthcare: taxonomy, literature review, opportunities and challenges. Future Gener. Comput. Syst. 113, 266–280 (2020)

    Article  Google Scholar 

  21. Park, J., Cho, Y.K., Kim, J.H.: Current and future use of esophageal capsule endoscopy. Clin. Endosc. 51(4), 317–322 (2018)

    Article  Google Scholar 

  22. Seguí, S., Drozdzal, M., Pascual, G., Radeva, P., Malagelada, C., Azpiroz, F., Vitrià, J.: Generic feature learning for wireless capsule endoscopy analysis. Comput. Biol. Med. 79, 163–172 (2016)

    Article  Google Scholar 

  23. Soffer, S., Klang, E., Shimon, O., Nachmias, N., Eliakim, R., Ben-Horin, S., Kopylov, U., Barash, Y.: Deep learning for wireless capsule endoscopy: a systematic review and meta-analysis. Gastrointest. Endosc. 92(4), 831–839 (2020)

    Article  Google Scholar 

  24. Syed, S., Stidham, R.W.: Potential for standardization and automation for pathology and endoscopy in inflammatory bowel disease. Inflamm. Bowel Dis. 26(10), 1490–1497 (2020)

    Article  Google Scholar 

  25. Valério, M.T., Gomes, S., Salgado, M., Oliveira, H.P., Cunha, A.: Lesions multiclass classification in endoscopic capsule frames. Procedia Comput. Sci. 164, 637–645 (2019)

    Article  Google Scholar 

  26. Yang, Y.J.: The future of capsule endoscopy: the role of artificial intelligence and other technical advancements. Clin. Endosc. 53(4), 387–394 (2020)

    Article  Google Scholar 

  27. Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Facial landmark detection by deep multi-task learning. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 94–108. Springer, Cham (2014).

    Chapter  Google Scholar 

  28. Zheng, Y., Hawkins, L., Wolff, J., Goloubeva, O., Goldberg, E.: Detection of lesions during capsule endoscopy: physician performance is disappointing. Am. J. Gastroenterol. 107(4), 554–560 (2012)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Anuja Vats .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 2042 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Vats, A., Pedersen, M., Mohammed, A., Hovde, Ø. (2021). Learning More for Free - A Multi Task Learning Approach for Improved Pathology Classification in Capsule Endoscopy. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12907. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87233-5

  • Online ISBN: 978-3-030-87234-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics