A Petri Dish for Histopathology Image Analysis

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12721)


With the rise of deep learning, there has been increased interest in using neural networks for histopathology image analysis, a field that investigates the properties of biopsy or resected specimens traditionally manually examined under a microscope by pathologists. However, challenges such as limited data, costly annotation, and processing high-resolution and variable-size images make it difficult to quickly iterate over model designs.

Throughout scientific history, many significant research directions have leveraged small-scale experimental setups as petri dishes to efficiently evaluate exploratory ideas. In this paper, we introduce a minimalist histopathology image analysis dataset (MHIST), an analogous petri dish for histopathology image analysis. MHIST is a binary classification dataset of 3,152 fixed-size images of colorectal polyps, each with a gold-standard label determined by the majority vote of seven board-certified gastrointestinal pathologists and annotator agreement level. MHIST occupies less than 400 MB of disk space, and a ResNet-18 baseline can be trained to convergence on MHIST in just 6 min using 3.5 GB of memory on a NVIDIA RTX 3090. As example use cases, we use MHIST to study natural questions such as how dataset size, network depth, transfer learning, and high-disagreement examples affect model performance.

By introducing MHIST, we hope to not only help facilitate the work of current histopathology imaging researchers, but also make the field more-accessible to the general community. Our dataset is available at


Histopathology images Deep learning Medical image analysis 


  1. 1.
    Jennings, B.H.: Drosophila - a versatile model in biology & medicine. Mater. Today, 14(5), 190–195 (2011).
  2. 2.
    Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE, 86(11), 2278–2324 (1998).
  3. 3.
    Goodfellow, I.J., et al.: Generative adversarial networks (2014).
  4. 4.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2014).
  5. 5.
    Srinidhi, C.L., Ciga, O., Martel, A.L.: Deep neural network models for computational histopathology: a survey. Med. Image Anal. 101813 (2020).
  6. 6.
    Arvaniti, E., et al.: Automated Gleason grading of prostate cancer tissue microarrays via deep learning. Nat. Sci. Rep. 8(1), 1–11 (2018).
  7. 7.
    Bulten, W., et al.: Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study. Lancet Oncol. 21(2), 233–241 (2020).
  8. 8.
    Hekler, A., et al.: Pathologist-level classification of histopathological melanoma images with deep neural networks. Euro. J. Cancer, 115, 79–83 (2019).
  9. 9.
    Shah, M., Wang, D., Rubadue, C., Suster, D., Beck, A.: Deep learning assessment of tumor proliferation in breast cancer histological images. In: 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 600–603 (2017).
  10. 10.
    Ström, P., et al.: Pathologist-level grading of prostate biopsies with artificial intelligence. CoRR (2019).
  11. 11.
    Wei, J.W., Tafe, L.J., Linnik, Y.A., Vaickus, L.J., Tomita, N., Hassanpour, S.: Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks. Sci. Rep. 9(1), 1–8 (2019).
  12. 12.
    Zhang, Z., et al.: Pathologist-level interpretable whole-slide cancer diagnosis with deep learning. Nat. Mach. Intell. 1, 236–245 (2019).
  13. 13.
    Bándi, P., et al.: From detection of individual metastases to classification of lymph node status at the patient level: the camelyon17 challenge. IEEE Trans. Med. Imaging, 38(2), 550–560 (2019).
  14. 14.
    Colorectal cancer statistics. Accessed 06 Jan 2021
  15. 15.
    Rex, D.K., et al.: Colorectal cancer screening: Recommendations for physicians and patients from the U.S. multi-society task force on colorectal cancer. Gastroenterology, 153, 307–323 (2017).
  16. 16.
    Abdeljawad, K., Vemulapalli, K.C., Kahi, C.J., Cummings, O.W., Snover, D.C., Rex, D.K.: Sessile serrated polyp prevalence determined by a colonoscopist with a high lesion detection rate and an experienced pathologist. Gastrointest. Endosc. 81, 517–524 (2015).
  17. 17.
    Farris, A.B., et al.: Sessile serrated adenoma: challenging discrimination from other serrated colonic polyps. Am. J. Surg. Pathol. 32, 30–35 (2008).
  18. 18.
    Glatz, K., Pritt, B., Glatz, D., HArtmann, A., O’Brien, M.J., Glaszyk, H.: A multinational, internet-based assessment of observer variability in the diagnosis of serrated colorectal polyps. Am. J. Clin. Pathol. 127(6), 938–945 (2007).
  19. 19.
    Khalid, O., Radaideh, S., Cummings, O.W., O’brien, M.J., Goldblum, J.R., Rex, D.K.: Reinterpretation of histology of proximal colon polyps called hyperplastic in 2001. World J. Gastroenterol. 15(30), 3767–3770 (2009).
  20. 20.
    Wong, N.A.C.S., Hunt, L.P., Novelli, M.R., Shepherd, N.A., Warren, B.F.: Observer agreement in the diagnosis of serrated polyps of the large bowel. Histopathology, 55(1), 63–66 (2009).
  21. 21.
  22. 22.
    Gurudu, S.R., et al.: Sessile serrated adenomas: Demographic, endoscopic and pathological characteristics. World J. Gastroenterol. 16(27), 3402–3405 (2010).
  23. 23.
    Nagtegaal, I.D., et al.: The 2019 who classification of tumours of the digestive system. Histopathology, 76(2), 182–188 (2020).
  24. 24.
    Chilamkurthy, S., et al.: Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study. Lancet, 392(10162), 2388–2396 (2018).
  25. 25.
    Gulshan, V., et al.: Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA, 316(22), 2402–2410 (2016).
  26. 26.
    Irvin, J., et al.: Chexpert: a large chest radiograph dataset with uncertainty labels and expert comparison. In: Association for the Advancement of Artificial Intelligence (AAAI) (2019).
  27. 27.
    Kanavati, F., et al.: Weakly-supervised learning for lung carcinoma classification using deep learning. Nat. Sci. Rep. 10(1), 1–11 (2020).
  28. 28.
    Korbar, B., et al.: Deep learning for classification of colorectal polyps on whole-slide images. J. Pathol. Inform. 8 (2017).
  29. 29.
    Sertel, O., Kong, J., Catalyurek, U.V., Lozanski, G., Saltz, J.H., Gurcan, M.N.: Histopathological image analysis using model-based intermediate representations and color texture: follicular lymphoma grading. J. Sig. Process. Syst. 55(1), 169–183 (2009).
  30. 30.
    Wang, S., Xing, Y., Zhang, L., Gao, H., Zhang, H.: Deep convolutional neural network for ulcer recognition in wireless capsule endoscopy: experimental feasibility and optimization. Computat. Math. Meth. Med. (2019).
  31. 31.
    Wei, J., Wei, J., Jackson, C., Ren, B., Suriawinata, A., Hassanpour, S.: Automated detection of celiac disease on duodenal biopsy slides: a deep learning approach. J. Pathol. Inform. 10(1), 7 (2019).;year=2019;volume=10;issue=1;spage=7;epage=7;aulast=Wei;t=6
  32. 32.
    Wei, J., et al.: Difficulty translation in histopathology images. In: Artificial Intelligence in Medicine (AIME) (2020).
  33. 33.
    Zhou, J., et al.: Weakly supervised 3D deep learning for breast cancer classification and localization of the lesions in MR images. J. Magn. Reson. Imaging, 50(4), 1144–1151 (2019).
  34. 34.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015).
  35. 35.
    Benkendorf, D.J., Hawkins, C.P.: Effects of sample size and network depth on a deep learning approach to species distribution modeling. Ecol. Inform. 60, 101137 (2020).
  36. 36.
    Kornblith, S., Shlens, J., Le, Q.V.: Do better imagenet models transfer better? CoRR (2018).
  37. 37.
    Coudray, N., Moreira, A.L., Sakellaropoulos, T., Fenyö, D., Razavian, N., Tsirigos, A.: Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24(10), 1559–1567 (2017).
  38. 38.
    Ehteshami Bejnordi, B., et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA, 318(22), 2199–2210 (2017).
  39. 39.
    Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639), 115–118 (2017).
  40. 40.
    Ghorbani, A., Natarajan, V., Coz, D., Liu, Y.: Dermgan: synthetic generation of clinical skin images with pathology (2019).
  41. 41.
    Wei, J.W., et al.: Evaluation of a deep neural network for automated classification of colorectal polyps on histopathologic slides. JAMA Netw. Open, 3(4) (2020).
  42. 42.
    Wei, J., et al.: Learn like a pathologist: curriculum learning by annotator agreement for histopathology image classification. In: Winter Conference on Applications of Computer Vision (WACV) (2020)Google Scholar
  43. 43.
    Brown, T.B., et al.: Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020).
  44. 44.
    Strubell, E., Ganesh, A., McCallum, A.: Energy and policy considerations for deep learning in nlp. arXiv preprint arXiv:1906.02243 (2019).
  45. 45.
    Rawal, A., Lehman, J., Such, F.P., Clune, J., Stanley, K.O.: Synthetic petri dish: a novel surrogate model for rapid architecture search. arXiv preprint arXiv:2005.13092 (2020).
  46. 46.
    Greydanus, S.: Scaling down deep learning. arXiv preprint arXiv:2011.14439 (2020).
  47. 47.
    Veeling, B.S., Linmans, J., Winkens, J., Cohen, T., Welling, M.: Rotation equivariant CNNs for digital pathology. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 210–218. Springer, Cham (2018). Scholar
  48. 48.
    Cireşan, D.C., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Mitosis detection in breast cancer histology images with deep neural networks. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8150, pp. 411–418. Springer, Heidelberg (2013). Scholar
  49. 49.
    Veta, M., et al.: Predicting breast tumor proliferation from whole-slide images: the tupac16 challenge. Med. Image Anal. 54, 111–121 (2019).,
  50. 50.
    Aresta, G., et al.: Bach: Grand challenge on breast cancer histology images. Med. Image Anal.56, 122–139 (2019).,
  51. 51.
    Swiderska-Chadaj, Z., et al.: Learning to detect lymphocytes in immunohistochemistry with deep learning. Med. Image Anal. 58, (2019).,

Copyright information

© Springer Nature Switzerland AG 2021

Authors and Affiliations

  1. 1.Dartmouth CollegeHanoverUSA
  2. 2.Dartmouth-Hitchcock Medical CenterLebanonUSA

Personalised recommendations