Skip to main content
SpringerLink
Log in

A Petri Dish for Histopathology Image Analysis

  • Conference paper
  • First Online:
Artificial Intelligence in Medicine (AIME 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12721))

Included in the following conference series:

Abstract

With the rise of deep learning, there has been increased interest in using neural networks for histopathology image analysis, a field that investigates the properties of biopsy or resected specimens traditionally manually examined under a microscope by pathologists. However, challenges such as limited data, costly annotation, and processing high-resolution and variable-size images make it difficult to quickly iterate over model designs.

Throughout scientific history, many significant research directions have leveraged small-scale experimental setups as petri dishes to efficiently evaluate exploratory ideas. In this paper, we introduce a minimalist histopathology image analysis dataset (MHIST), an analogous petri dish for histopathology image analysis. MHIST is a binary classification dataset of 3,152 fixed-size images of colorectal polyps, each with a gold-standard label determined by the majority vote of seven board-certified gastrointestinal pathologists and annotator agreement level. MHIST occupies less than 400 MB of disk space, and a ResNet-18 baseline can be trained to convergence on MHIST in just 6 min using 3.5 GB of memory on a NVIDIA RTX 3090. As example use cases, we use MHIST to study natural questions such as how dataset size, network depth, transfer learning, and high-disagreement examples affect model performance.

By introducing MHIST, we hope to not only help facilitate the work of current histopathology imaging researchers, but also make the field more-accessible to the general community. Our dataset is available at https://bmirds.github.io/MHIST.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Jennings, B.H.: Drosophila - a versatile model in biology & medicine. Mater. Today, 14(5), 190–195 (2011). http://www.sciencedirect.com/science/article/pii/S1369702111701134

  2. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE, 86(11), 2278–2324 (1998). https://ieeexplore.ieee.org/document/726791

  3. Goodfellow, I.J., et al.: Generative adversarial networks (2014). https://arxiv.org/pdf/1406.2661.pdf

  4. Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2014). https://arxiv.org/pdf/1412.6980.pdf

  5. Srinidhi, C.L., Ciga, O., Martel, A.L.: Deep neural network models for computational histopathology: a survey. Med. Image Anal. 101813 (2020). https://www.sciencedirect.com/science/article/pii/S1361841520301778

  6. Arvaniti, E., et al.: Automated Gleason grading of prostate cancer tissue microarrays via deep learning. Nat. Sci. Rep. 8(1), 1–11 (2018). https://www.nature.com/articles/s41598-018-30535-1

  7. Bulten, W., et al.: Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study. Lancet Oncol. 21(2), 233–241 (2020). https://arxiv.org/pdf/1907.07980.pdf

  8. Hekler, A., et al.: Pathologist-level classification of histopathological melanoma images with deep neural networks. Euro. J. Cancer, 115, 79–83 (2019). http://www.sciencedirect.com/science/article/pii/S0959804919302758

  9. Shah, M., Wang, D., Rubadue, C., Suster, D., Beck, A.: Deep learning assessment of tumor proliferation in breast cancer histological images. In: 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 600–603 (2017). https://ieeexplore.ieee.org/abstract/document/8217719

  10. Ström, P., et al.: Pathologist-level grading of prostate biopsies with artificial intelligence. CoRR (2019). http://arxiv.org/pdf/1907.01368

  11. Wei, J.W., Tafe, L.J., Linnik, Y.A., Vaickus, L.J., Tomita, N., Hassanpour, S.: Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks. Sci. Rep. 9(1), 1–8 (2019). https://www.nature.com/articles/s41598-019-40041-7

  12. Zhang, Z., et al.: Pathologist-level interpretable whole-slide cancer diagnosis with deep learning. Nat. Mach. Intell. 1, 236–245 (2019). https://www.nature.com/articles/s42256-019-0052-1

  13. Bándi, P., et al.: From detection of individual metastases to classification of lymph node status at the patient level: the camelyon17 challenge. IEEE Trans. Med. Imaging, 38(2), 550–560 (2019). https://ieeexplore.ieee.org/document/8447230

  14. Colorectal cancer statistics. https://www.cancer.org/cancer/colon-rectal-cancer/about/key-statistics.html. Accessed 06 Jan 2021

  15. Rex, D.K., et al.: Colorectal cancer screening: Recommendations for physicians and patients from the U.S. multi-society task force on colorectal cancer. Gastroenterology, 153, 307–323 (2017). www.gastrojournal.org/article/S0016-5085(17)35599--3/fulltext

  16. Abdeljawad, K., Vemulapalli, K.C., Kahi, C.J., Cummings, O.W., Snover, D.C., Rex, D.K.: Sessile serrated polyp prevalence determined by a colonoscopist with a high lesion detection rate and an experienced pathologist. Gastrointest. Endosc. 81, 517–524 (2015). https://pubmed.ncbi.nlm.nih.gov/24998465/

  17. Farris, A.B., et al.: Sessile serrated adenoma: challenging discrimination from other serrated colonic polyps. Am. J. Surg. Pathol. 32, 30–35 (2008). https://pubmed.ncbi.nlm.nih.gov/18162767/

  18. Glatz, K., Pritt, B., Glatz, D., HArtmann, A., O’Brien, M.J., Glaszyk, H.: A multinational, internet-based assessment of observer variability in the diagnosis of serrated colorectal polyps. Am. J. Clin. Pathol. 127(6), 938–945 (2007). https://pubmed.ncbi.nlm.nih.gov/17509991/

  19. Khalid, O., Radaideh, S., Cummings, O.W., O’brien, M.J., Goldblum, J.R., Rex, D.K.: Reinterpretation of histology of proximal colon polyps called hyperplastic in 2001. World J. Gastroenterol. 15(30), 3767–3770 (2009). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2726454/

  20. Wong, N.A.C.S., Hunt, L.P., Novelli, M.R., Shepherd, N.A., Warren, B.F.: Observer agreement in the diagnosis of serrated polyps of the large bowel. Histopathology, 55(1), 63–66 (2009). https://pubmed.ncbi.nlm.nih.gov/19614768/

  21. Understanding your pathology report: Colon polyps (sessile or traditional serrated adenomas). https://www.cancer.org/treatment/understanding-your-diagnosis/tests/understanding-your-pathology-report/colon-pathology/colon-polyps-sessile-or-traditional-serrated-adenomas.html. Accessed 06 Jan 2021

  22. Gurudu, S.R., et al.: Sessile serrated adenomas: Demographic, endoscopic and pathological characteristics. World J. Gastroenterol. 16(27), 3402–3405 (2010). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2904886/

  23. Nagtegaal, I.D., et al.: The 2019 who classification of tumours of the digestive system. Histopathology, 76(2), 182–188 (2020). https://onlinelibrary.wiley.com/doi/abs/10.1111/his.13975

  24. Chilamkurthy, S., et al.: Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study. Lancet, 392(10162), 2388–2396 (2018). www.thelancet.com/journals/lancet/article/PIIS0140-6736(18)31645--3/fulltext

  25. Gulshan, V., et al.: Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA, 316(22), 2402–2410 (2016). https://jamanetwork.com/journals/jama/fullarticle/2588763

  26. Irvin, J., et al.: Chexpert: a large chest radiograph dataset with uncertainty labels and expert comparison. In: Association for the Advancement of Artificial Intelligence (AAAI) (2019). http://arxiv.org/pdf/1901.07031

  27. Kanavati, F., et al.: Weakly-supervised learning for lung carcinoma classification using deep learning. Nat. Sci. Rep. 10(1), 1–11 (2020). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7283481/

  28. Korbar, B., et al.: Deep learning for classification of colorectal polyps on whole-slide images. J. Pathol. Inform. 8 (2017). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5545773/

  29. Sertel, O., Kong, J., Catalyurek, U.V., Lozanski, G., Saltz, J.H., Gurcan, M.N.: Histopathological image analysis using model-based intermediate representations and color texture: follicular lymphoma grading. J. Sig. Process. Syst. 55(1), 169–183 (2009). https://link.springer.com/article/10.1007/s11265-008-0201-y

  30. Wang, S., Xing, Y., Zhang, L., Gao, H., Zhang, H.: Deep convolutional neural network for ulcer recognition in wireless capsule endoscopy: experimental feasibility and optimization. Computat. Math. Meth. Med. (2019). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6766681/

  31. Wei, J., Wei, J., Jackson, C., Ren, B., Suriawinata, A., Hassanpour, S.: Automated detection of celiac disease on duodenal biopsy slides: a deep learning approach. J. Pathol. Inform. 10(1), 7 (2019). http://www.jpathinformatics.org/article.asp?issn=2153-3539;year=2019;volume=10;issue=1;spage=7;epage=7;aulast=Wei;t=6

  32. Wei, J., et al.: Difficulty translation in histopathology images. In: Artificial Intelligence in Medicine (AIME) (2020). https://arxiv.org/pdf/2004.12535.pdf

  33. Zhou, J., et al.: Weakly supervised 3D deep learning for breast cancer classification and localization of the lesions in MR images. J. Magn. Reson. Imaging, 50(4), 1144–1151 (2019). https://onlinelibrary.wiley.com/doi/abs/10.1002/jmri.26721

  34. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015). http://arxiv.org/pdf/1512.03385

  35. Benkendorf, D.J., Hawkins, C.P.: Effects of sample size and network depth on a deep learning approach to species distribution modeling. Ecol. Inform. 60, 101137 (2020). http://www.sciencedirect.com/science/article/pii/S157495412030087X

  36. Kornblith, S., Shlens, J., Le, Q.V.: Do better imagenet models transfer better? CoRR (2018). http://arxiv.org/pdf/1805.08974

  37. Coudray, N., Moreira, A.L., Sakellaropoulos, T., Fenyö, D., Razavian, N., Tsirigos, A.: Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24(10), 1559–1567 (2017). https://www.nature.com/articles/s41591-018-0177-5

  38. Ehteshami Bejnordi, B., et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA, 318(22), 2199–2210 (2017). https://jamanetwork.com/journals/jama/fullarticle/2665774

  39. Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639), 115–118 (2017). https://www.nature.com/articles/nature21056

  40. Ghorbani, A., Natarajan, V., Coz, D., Liu, Y.: Dermgan: synthetic generation of clinical skin images with pathology (2019). https://arxiv.org/pdf/1911.08716.pdf

  41. Wei, J.W., et al.: Evaluation of a deep neural network for automated classification of colorectal polyps on histopathologic slides. JAMA Netw. Open, 3(4) (2020). https://jamanetwork.com/journals/jamanetworkopen/article-abstract/2764906

  42. Wei, J., et al.: Learn like a pathologist: curriculum learning by annotator agreement for histopathology image classification. In: Winter Conference on Applications of Computer Vision (WACV) (2020)

    Google Scholar 

  43. Brown, T.B., et al.: Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020). https://arxiv.org/pdf/2005.14165.pdf

  44. Strubell, E., Ganesh, A., McCallum, A.: Energy and policy considerations for deep learning in nlp. arXiv preprint arXiv:1906.02243 (2019). https://arxiv.org/pdf/1906.02243.pdf

  45. Rawal, A., Lehman, J., Such, F.P., Clune, J., Stanley, K.O.: Synthetic petri dish: a novel surrogate model for rapid architecture search. arXiv preprint arXiv:2005.13092 (2020). https://arxiv.org/pdf/2005.13092.pdf

  46. Greydanus, S.: Scaling down deep learning. arXiv preprint arXiv:2011.14439 (2020). https://arxiv.org/pdf/2011.14439.pdf

  47. Veeling, B.S., Linmans, J., Winkens, J., Cohen, T., Welling, M.: Rotation equivariant CNNs for digital pathology. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 210–218. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_24

    Chapter  Google Scholar 

  48. Cireşan, D.C., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Mitosis detection in breast cancer histology images with deep neural networks. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8150, pp. 411–418. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40763-5_51

    Chapter  Google Scholar 

  49. Veta, M., et al.: Predicting breast tumor proliferation from whole-slide images: the tupac16 challenge. Med. Image Anal. 54, 111–121 (2019). https://doi.org/10.1016/j.media.2019.02.012, http://www.sciencedirect.com/science/article/pii/S1361841518305231

  50. Aresta, G., et al.: Bach: Grand challenge on breast cancer histology images. Med. Image Anal.56, 122–139 (2019). https://doi.org/10.1016/j.media.2019.05.010, http://www.sciencedirect.com/science/article/pii/S1361841518307941

  51. Swiderska-Chadaj, Z., et al.: Learning to detect lymphocytes in immunohistochemistry with deep learning. Med. Image Anal. 58, (2019). https://doi.org/10.1016/j.media.2019.101547, http://www.sciencedirect.com/science/article/pii/S1361841519300829

Download references

Author information

Authors and Affiliations

  1. Dartmouth College, Hanover, NH, USA

    Jerry Wei, Naofumi Tomita, Lorenzo Torresani, Jason Wei & Saeed Hassanpour

  2. Dartmouth-Hitchcock Medical Center, Lebanon, NH, USA

    Arief Suriawinata, Bing Ren, Xiaoying Liu, Mikhail Lisovsky, Louis Vaickus, Charles Brown & Michael Baker

Authors
  1. Jerry Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arief Suriawinata
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bing Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoying Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mikhail Lisovsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Louis Vaickus
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Charles Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Michael Baker
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Naofumi Tomita
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Lorenzo Torresani
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Jason Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Saeed Hassanpour
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Brunel University London, Uxbridge, UK

    Allan Tucker

  2. University of Coimbra, Coimbra, Portugal

    Pedro Henriques Abreu

  3. University of Porto, Porto, Portugal

    Jaime Cardoso

  4. University of Porto, Porto, Portugal

    Pedro Pereira Rodrigues

  5. Universitat Rovira i Virgili, Tarragona, Spain

    David Riaño

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wei, J. et al. (2021). A Petri Dish for Histopathology Image Analysis. In: Tucker, A., Henriques Abreu, P., Cardoso, J., Pereira Rodrigues, P., Riaño, D. (eds) Artificial Intelligence in Medicine. AIME 2021. Lecture Notes in Computer Science(), vol 12721. Springer, Cham. https://doi.org/10.1007/978-3-030-77211-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-77211-6_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-77210-9

  • Online ISBN: 978-3-030-77211-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation