Advertisement

A Petri Dish for Histopathology Image Analysis

Conference paper
  • 276 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12721)

Abstract

With the rise of deep learning, there has been increased interest in using neural networks for histopathology image analysis, a field that investigates the properties of biopsy or resected specimens traditionally manually examined under a microscope by pathologists. However, challenges such as limited data, costly annotation, and processing high-resolution and variable-size images make it difficult to quickly iterate over model designs.

Throughout scientific history, many significant research directions have leveraged small-scale experimental setups as petri dishes to efficiently evaluate exploratory ideas. In this paper, we introduce a minimalist histopathology image analysis dataset (MHIST), an analogous petri dish for histopathology image analysis. MHIST is a binary classification dataset of 3,152 fixed-size images of colorectal polyps, each with a gold-standard label determined by the majority vote of seven board-certified gastrointestinal pathologists and annotator agreement level. MHIST occupies less than 400 MB of disk space, and a ResNet-18 baseline can be trained to convergence on MHIST in just 6 min using 3.5 GB of memory on a NVIDIA RTX 3090. As example use cases, we use MHIST to study natural questions such as how dataset size, network depth, transfer learning, and high-disagreement examples affect model performance.

By introducing MHIST, we hope to not only help facilitate the work of current histopathology imaging researchers, but also make the field more-accessible to the general community. Our dataset is available at https://bmirds.github.io/MHIST.

Keywords

Histopathology images Deep learning Medical image analysis 

References

  1. 1.
    Jennings, B.H.: Drosophila - a versatile model in biology & medicine. Mater. Today, 14(5), 190–195 (2011). http://www.sciencedirect.com/science/article/pii/S1369702111701134
  2. 2.
    Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE, 86(11), 2278–2324 (1998). https://ieeexplore.ieee.org/document/726791
  3. 3.
    Goodfellow, I.J., et al.: Generative adversarial networks (2014). https://arxiv.org/pdf/1406.2661.pdf
  4. 4.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2014). https://arxiv.org/pdf/1412.6980.pdf
  5. 5.
    Srinidhi, C.L., Ciga, O., Martel, A.L.: Deep neural network models for computational histopathology: a survey. Med. Image Anal. 101813 (2020). https://www.sciencedirect.com/science/article/pii/S1361841520301778
  6. 6.
    Arvaniti, E., et al.: Automated Gleason grading of prostate cancer tissue microarrays via deep learning. Nat. Sci. Rep. 8(1), 1–11 (2018). https://www.nature.com/articles/s41598-018-30535-1
  7. 7.
    Bulten, W., et al.: Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study. Lancet Oncol. 21(2), 233–241 (2020). https://arxiv.org/pdf/1907.07980.pdf
  8. 8.
    Hekler, A., et al.: Pathologist-level classification of histopathological melanoma images with deep neural networks. Euro. J. Cancer, 115, 79–83 (2019). http://www.sciencedirect.com/science/article/pii/S0959804919302758
  9. 9.
    Shah, M., Wang, D., Rubadue, C., Suster, D., Beck, A.: Deep learning assessment of tumor proliferation in breast cancer histological images. In: 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 600–603 (2017). https://ieeexplore.ieee.org/abstract/document/8217719
  10. 10.
    Ström, P., et al.: Pathologist-level grading of prostate biopsies with artificial intelligence. CoRR (2019). http://arxiv.org/pdf/1907.01368
  11. 11.
    Wei, J.W., Tafe, L.J., Linnik, Y.A., Vaickus, L.J., Tomita, N., Hassanpour, S.: Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks. Sci. Rep. 9(1), 1–8 (2019). https://www.nature.com/articles/s41598-019-40041-7
  12. 12.
    Zhang, Z., et al.: Pathologist-level interpretable whole-slide cancer diagnosis with deep learning. Nat. Mach. Intell. 1, 236–245 (2019). https://www.nature.com/articles/s42256-019-0052-1
  13. 13.
    Bándi, P., et al.: From detection of individual metastases to classification of lymph node status at the patient level: the camelyon17 challenge. IEEE Trans. Med. Imaging, 38(2), 550–560 (2019). https://ieeexplore.ieee.org/document/8447230
  14. 14.
    Colorectal cancer statistics. https://www.cancer.org/cancer/colon-rectal-cancer/about/key-statistics.html. Accessed 06 Jan 2021
  15. 15.
    Rex, D.K., et al.: Colorectal cancer screening: Recommendations for physicians and patients from the U.S. multi-society task force on colorectal cancer. Gastroenterology, 153, 307–323 (2017). www.gastrojournal.org/article/S0016-5085(17)35599--3/fulltext
  16. 16.
    Abdeljawad, K., Vemulapalli, K.C., Kahi, C.J., Cummings, O.W., Snover, D.C., Rex, D.K.: Sessile serrated polyp prevalence determined by a colonoscopist with a high lesion detection rate and an experienced pathologist. Gastrointest. Endosc. 81, 517–524 (2015). https://pubmed.ncbi.nlm.nih.gov/24998465/
  17. 17.
    Farris, A.B., et al.: Sessile serrated adenoma: challenging discrimination from other serrated colonic polyps. Am. J. Surg. Pathol. 32, 30–35 (2008). https://pubmed.ncbi.nlm.nih.gov/18162767/
  18. 18.
    Glatz, K., Pritt, B., Glatz, D., HArtmann, A., O’Brien, M.J., Glaszyk, H.: A multinational, internet-based assessment of observer variability in the diagnosis of serrated colorectal polyps. Am. J. Clin. Pathol. 127(6), 938–945 (2007). https://pubmed.ncbi.nlm.nih.gov/17509991/
  19. 19.
    Khalid, O., Radaideh, S., Cummings, O.W., O’brien, M.J., Goldblum, J.R., Rex, D.K.: Reinterpretation of histology of proximal colon polyps called hyperplastic in 2001. World J. Gastroenterol. 15(30), 3767–3770 (2009). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2726454/
  20. 20.
    Wong, N.A.C.S., Hunt, L.P., Novelli, M.R., Shepherd, N.A., Warren, B.F.: Observer agreement in the diagnosis of serrated polyps of the large bowel. Histopathology, 55(1), 63–66 (2009). https://pubmed.ncbi.nlm.nih.gov/19614768/
  21. 21.
  22. 22.
    Gurudu, S.R., et al.: Sessile serrated adenomas: Demographic, endoscopic and pathological characteristics. World J. Gastroenterol. 16(27), 3402–3405 (2010). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2904886/
  23. 23.
    Nagtegaal, I.D., et al.: The 2019 who classification of tumours of the digestive system. Histopathology, 76(2), 182–188 (2020). https://onlinelibrary.wiley.com/doi/abs/10.1111/his.13975
  24. 24.
    Chilamkurthy, S., et al.: Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study. Lancet, 392(10162), 2388–2396 (2018). www.thelancet.com/journals/lancet/article/PIIS0140-6736(18)31645--3/fulltext
  25. 25.
    Gulshan, V., et al.: Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA, 316(22), 2402–2410 (2016). https://jamanetwork.com/journals/jama/fullarticle/2588763
  26. 26.
    Irvin, J., et al.: Chexpert: a large chest radiograph dataset with uncertainty labels and expert comparison. In: Association for the Advancement of Artificial Intelligence (AAAI) (2019). http://arxiv.org/pdf/1901.07031
  27. 27.
    Kanavati, F., et al.: Weakly-supervised learning for lung carcinoma classification using deep learning. Nat. Sci. Rep. 10(1), 1–11 (2020). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7283481/
  28. 28.
    Korbar, B., et al.: Deep learning for classification of colorectal polyps on whole-slide images. J. Pathol. Inform. 8 (2017). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5545773/
  29. 29.
    Sertel, O., Kong, J., Catalyurek, U.V., Lozanski, G., Saltz, J.H., Gurcan, M.N.: Histopathological image analysis using model-based intermediate representations and color texture: follicular lymphoma grading. J. Sig. Process. Syst. 55(1), 169–183 (2009). https://link.springer.com/article/10.1007/s11265-008-0201-y
  30. 30.
    Wang, S., Xing, Y., Zhang, L., Gao, H., Zhang, H.: Deep convolutional neural network for ulcer recognition in wireless capsule endoscopy: experimental feasibility and optimization. Computat. Math. Meth. Med. (2019). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6766681/
  31. 31.
    Wei, J., Wei, J., Jackson, C., Ren, B., Suriawinata, A., Hassanpour, S.: Automated detection of celiac disease on duodenal biopsy slides: a deep learning approach. J. Pathol. Inform. 10(1), 7 (2019). http://www.jpathinformatics.org/article.asp?issn=2153-3539;year=2019;volume=10;issue=1;spage=7;epage=7;aulast=Wei;t=6
  32. 32.
    Wei, J., et al.: Difficulty translation in histopathology images. In: Artificial Intelligence in Medicine (AIME) (2020). https://arxiv.org/pdf/2004.12535.pdf
  33. 33.
    Zhou, J., et al.: Weakly supervised 3D deep learning for breast cancer classification and localization of the lesions in MR images. J. Magn. Reson. Imaging, 50(4), 1144–1151 (2019). https://onlinelibrary.wiley.com/doi/abs/10.1002/jmri.26721
  34. 34.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015). http://arxiv.org/pdf/1512.03385
  35. 35.
    Benkendorf, D.J., Hawkins, C.P.: Effects of sample size and network depth on a deep learning approach to species distribution modeling. Ecol. Inform. 60, 101137 (2020). http://www.sciencedirect.com/science/article/pii/S157495412030087X
  36. 36.
    Kornblith, S., Shlens, J., Le, Q.V.: Do better imagenet models transfer better? CoRR (2018). http://arxiv.org/pdf/1805.08974
  37. 37.
    Coudray, N., Moreira, A.L., Sakellaropoulos, T., Fenyö, D., Razavian, N., Tsirigos, A.: Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24(10), 1559–1567 (2017). https://www.nature.com/articles/s41591-018-0177-5
  38. 38.
    Ehteshami Bejnordi, B., et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA, 318(22), 2199–2210 (2017). https://jamanetwork.com/journals/jama/fullarticle/2665774
  39. 39.
    Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639), 115–118 (2017). https://www.nature.com/articles/nature21056
  40. 40.
    Ghorbani, A., Natarajan, V., Coz, D., Liu, Y.: Dermgan: synthetic generation of clinical skin images with pathology (2019). https://arxiv.org/pdf/1911.08716.pdf
  41. 41.
    Wei, J.W., et al.: Evaluation of a deep neural network for automated classification of colorectal polyps on histopathologic slides. JAMA Netw. Open, 3(4) (2020). https://jamanetwork.com/journals/jamanetworkopen/article-abstract/2764906
  42. 42.
    Wei, J., et al.: Learn like a pathologist: curriculum learning by annotator agreement for histopathology image classification. In: Winter Conference on Applications of Computer Vision (WACV) (2020)Google Scholar
  43. 43.
    Brown, T.B., et al.: Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020). https://arxiv.org/pdf/2005.14165.pdf
  44. 44.
    Strubell, E., Ganesh, A., McCallum, A.: Energy and policy considerations for deep learning in nlp. arXiv preprint arXiv:1906.02243 (2019). https://arxiv.org/pdf/1906.02243.pdf
  45. 45.
    Rawal, A., Lehman, J., Such, F.P., Clune, J., Stanley, K.O.: Synthetic petri dish: a novel surrogate model for rapid architecture search. arXiv preprint arXiv:2005.13092 (2020). https://arxiv.org/pdf/2005.13092.pdf
  46. 46.
    Greydanus, S.: Scaling down deep learning. arXiv preprint arXiv:2011.14439 (2020). https://arxiv.org/pdf/2011.14439.pdf
  47. 47.
    Veeling, B.S., Linmans, J., Winkens, J., Cohen, T., Welling, M.: Rotation equivariant CNNs for digital pathology. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 210–218. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00934-2_24CrossRefGoogle Scholar
  48. 48.
    Cireşan, D.C., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Mitosis detection in breast cancer histology images with deep neural networks. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8150, pp. 411–418. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-40763-5_51CrossRefGoogle Scholar
  49. 49.
    Veta, M., et al.: Predicting breast tumor proliferation from whole-slide images: the tupac16 challenge. Med. Image Anal. 54, 111–121 (2019).  https://doi.org/10.1016/j.media.2019.02.012, http://www.sciencedirect.com/science/article/pii/S1361841518305231
  50. 50.
    Aresta, G., et al.: Bach: Grand challenge on breast cancer histology images. Med. Image Anal.56, 122–139 (2019).  https://doi.org/10.1016/j.media.2019.05.010, http://www.sciencedirect.com/science/article/pii/S1361841518307941
  51. 51.
    Swiderska-Chadaj, Z., et al.: Learning to detect lymphocytes in immunohistochemistry with deep learning. Med. Image Anal. 58, (2019).  https://doi.org/10.1016/j.media.2019.101547, http://www.sciencedirect.com/science/article/pii/S1361841519300829

Copyright information

© Springer Nature Switzerland AG 2021

Authors and Affiliations

  1. 1.Dartmouth CollegeHanoverUSA
  2. 2.Dartmouth-Hitchcock Medical CenterLebanonUSA

Personalised recommendations