Unsupervised Pre-training Across Image Domains Improves Lung Tissue Classification

Conference paper
First Online:
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8848)

Abstract

The detection and classification of anomalies relevant for disease diagnosis or treatment monitoring is important during computational medical image analysis. Often, obtaining sufficient annotated training data to represent natural variability well is unfeasible. At the same time, data is frequently collected across multiple sites with heterogeneous medical imaging equipment. In this paper we propose and evaluate a semi-supervised learning approach that uses data from multiple sites (domains). Only for one small site annotations are available. We use convolutional neural networks to capture spatial appearance patterns and classify lung tissue in high-resolution computed tomography data. We perform domain adaptation via unsupervised pre-training of convolutional neural networks to inject information from sites or image classes for which no annotations are available. Results show that across site pre-training as well as pre-training on different image classes improves classification accuracy compared to random initialisation of the model parameters.

Keywords

Image Patch Hide Unit Unlabeled Data Convolutional Neural Network Misclassification Error 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

References

  1. 1.
    Fukushima, K.: Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern. 36(4), 193–202 (1980)CrossRefMATHGoogle Scholar
  2. 2.
    Ryu, J.H., Daniels, C.E., Hartman, T.E., Yi, E.S.: Diagnosis of interstitial lung diseases. In: Mayo Clinic Proceedings, vol. 82, pp. 976–986. Elsevier (2007)Google Scholar
  3. 3.
    Depeursinge, A., Sage, D., Hidki, A., Platon, A., Poletti, P.A., Unser, M., Muller, H.: Lung tissue classification using wavelet frames. In: 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6259–6262 (2007)Google Scholar
  4. 4.
    Lee, H., Grosse, R., Ranganath, R., Ng, A.Y.: Unsupervised learning of hierarchical representations with convolutional deep belief networks. Commun. ACM 54(10), 95–103 (2011)CrossRefGoogle Scholar
  5. 5.
    Ciresan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: Conference on Computer Vision and Pattern Recognition, pp. 3642–3649. IEEE (2012)Google Scholar
  6. 6.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, vol. 1, p. 4 (2012)Google Scholar
  7. 7.
    Bengio, Y.: Deep learning of representations for unsupervised and transfer learning. J. Mach. Learn. Res. Proc. Track 27, 17–36 (2012)Google Scholar
  8. 8.
    Cireşan, D.C., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Mitosis detection in breast cancer histology images with deep neural networks. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part II. LNCS, vol. 8150, pp. 411–418. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  9. 9.
    Erhan, D., Bengio, Y., Courville, A., Manzagol, P.A., Vincent, P., Bengio, S.: Why does unsupervised pre-training help deep learning? J. Mach. Learn. Res. 11, 625–660 (2010)MATHMathSciNetGoogle Scholar
  10. 10.
    Lee, H., Grosse, R., Ranganath, R., Ng, A.Y.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 609–616 (2009)Google Scholar
  11. 11.
    Brosch, T., Tam, R.: Manifold learning of brain MRIs by deep learning. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part II. LNCS, vol. 8150, pp. 633–640. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  12. 12.
    Coates, A., Ng, A.Y., Lee, H.: An analysis of single-layer networks in unsupervised feature learning. In: International Conference on Artificial Intelligence and Statistics, pp. 215–223 (2011)Google Scholar
  13. 13.
    Holmes III, D., Bartholmai, B., Karwoski, R., Zavaletta, V., Robb, R.: The lung tissue research consortium: an extensive open database containing histological, clinical, and radiological data to study chronic lung disease. In: The Insight Journal MICCAI Open Science Workshop (2006)Google Scholar
  14. 14.
    Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)CrossRefMATHMathSciNetGoogle Scholar
  15. 15.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  16. 16.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning, pp. 807–814 (2010)Google Scholar
  17. 17.
    Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., Bengio, Y.: Theano: a CPU and GPU math expression compiler. In: Proceedings of the Python for Scientific Computing Conference (SciPy), vol. 4 (2010)Google Scholar
  18. 18.
    Zavaletta, V.A., Bartholmai, B.J., Robb, R.A.: High resolution multidetector CT-aided tissue analysis and quantification of lung fibrosis. Acad. Radiol. 14(7), 772–787 (2007)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Computational Imaging Research Lab, Department of Biomedical Imaging and Image-guided TherapyMedical University ViennaViennaAustria

Personalised recommendations