# A Linear Optimal Transportation Framework for Quantifying and Visualizing Variations in Sets of Images

- 1.3k Downloads
- 31 Citations

## Abstract

Transportation-based metrics for comparing images have long been applied to analyze images, especially where one can interpret the pixel intensities (or derived quantities) as a distribution of ‘mass’ that can be transported without strict geometric constraints. Here we describe a new transportation-based framework for analyzing sets of images. More specifically, we describe a new transportation-related distance between pairs of images, which we denote as linear optimal transportation (LOT). The LOT can be used directly on pixel intensities, and is based on a linearized version of the Kantorovich-Wasserstein metric (an optimal transportation distance, as is the earth mover’s distance). The new framework is especially well suited for computing all pairwise distances for a large database of images efficiently, and thus it can be used for pattern recognition in sets of images. In addition, the new LOT framework also allows for an isometric linear embedding, greatly facilitating the ability to visualize discriminant information in different classes of images. We demonstrate the application of the framework to several tasks such as discriminating nuclear chromatin patterns in cancer cells, decoding differences in facial expressions, galaxy morphologies, as well as sub cellular protein distributions.

## Keywords

Optimal transportation Linear embedding## Notes

### Acknowledgements

The authors wish to thank the anonymous reviewers for helping significantly improve this paper. W. Wang, S. Basu, and G.K. Rohde acknowledge support from NIH grants GM088816 and GM090033 (PI GKR) for supporting portions of this work. D. Slepčev was also supported by NIH grant GM088816, as well as NSF grant DMS-0908415. He is also grateful to the Center for Nonlinear Analysis (NSF grant DMS-0635983 and NSF PIRE grant OISE-0967140) for its support.

## References

- Ambrosio, L., Gigli, N., & Savaré, G. (2008).
*Lectures in mathematics ETH zürich*.*Gradient flows in metric spaces and in the space of probability measures*(2nd ed.). Basel: Birkhäuser. zbMATHGoogle Scholar - Angenent, S., Haker, S., & Tannenbaum, A. (2003). Minimizing flows for the Monge-Kantorovich problem.
*SIAM Journal on Mathematical Analysis*,*35*(1), 61–97. (electronic). MathSciNetzbMATHCrossRefGoogle Scholar - Barrett, J. W., & Prigozhin, L. (2009). Partial
*L*^{1}Monge-Kantorovich problem: variational formulation and numerical approximation.*Interfaces and Free Boundaries*,*11*(2), 201–238. MathSciNetzbMATHCrossRefGoogle Scholar - Beg, M., Miller, M., Trouvé, A., & Younes, L. (2005). Computing large deformation metric mappings via geodesic flows of diffeomorphisms.
*International Journal of Computer Vision*,*61*(2), 139–157. CrossRefGoogle Scholar - Benamou, J. D., & Brenier, Y. (2000). A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem.
*Numerische Mathematik*,*84*(3), 375–393. MathSciNetzbMATHCrossRefGoogle Scholar - Bengtsson, E. (1999). Fifty years of attempts to automate screening for cervical cancer.
*Medical Imaging Technology*,*17*, 203–210. Google Scholar - Bishop, C. M. (2006).
*Pattern recognition and machine learning (information science and statistics)*. Berlin: Springer. Google Scholar - Blum, H., et al. (1967). A transformation for extracting new descriptors of shape. In
*Models for the perception of speech and visual form*(Vol. 19, pp. 362–380). Google Scholar - Boland, M. V., & Murphy, R. F. (2001). A neural network classifier capable of recognizing the patterns of all major subcellular structures in fluorescence microscope images of hela cells.
*Bioinformatics*,*17*(12), 1213–1223. CrossRefGoogle Scholar - do Carmo, M. P. (1992).
*Riemannian geometry. Mathematics: theory & applications*. Boston: Birkhäuser Boston. Translated from the second Portuguese edition by Francis Flaherty. Google Scholar - Chefd’hotel, C., & Bousquet, G. (2007). Intensity-based image registration using earth mover’s distance.
*Proceedings of SPIE*, vol.*6512*, p. 65122B. CrossRefGoogle Scholar - Delzanno, G. L., & Finn, J. M. (2010). Generalized Monge-Kantorovich optimization for grid generation and adaptation in
*L*_{p}.*SIAM Journal on Scientific Computing*,*32*(6), 3524–3547. MathSciNetzbMATHCrossRefGoogle Scholar - Dialynas, G. K., Vitalini, M. W., & Wallrath, L. L. (2008). Linking heterochromatin protein 1 (hp1) to cancer progression.
*Mutation Research*,*647*(1–2), 13–20. CrossRefGoogle Scholar - Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems.
*Annals of Eugenics*,*7*, 179–188. CrossRefGoogle Scholar - Gardner, M., Sprague, B., Pearson, C., Cosgrove, B., Bicek, A., Bloom, K., Salmon, E., & Odde, D. (2010). Model convolution: a computational approach to digital image interpretation.
*Cellular and Molecular Bioengineering*,*3*(2), 163–170. CrossRefGoogle Scholar - Grauman, K., & Darrell, T. (2004). Fast contour matching using approximate earth mover’s distance. In
*Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, CVPR 2004*. Google Scholar - Haber, E., Rehman, T., & Tannenbaum, A. (2010). An efficient numerical method for the solution of the
*L*_{2}optimal mass transfer problem.*SIAM Journal on Scientific Computing*,*32*(1), 197–211. MathSciNetzbMATHCrossRefGoogle Scholar - Haker, S., Zhu, L., Tennenbaum, A., & Angenent, S. (2004). Optimal mass transport for registration and warping.
*International Journal of Computer Vision*,*60*(3), 225–240. CrossRefGoogle Scholar - Kong, J., Sertel, O., Shimada, H., BOyer, K. L., Saltz, J. H., & Gurcan, M. N. (2009). Computer-aided evaluation of neuroblastoma on whole slide histology images: classifying grade of neuroblastic differentiation.
*Pattern Recognition*,*42*, 1080–1092. CrossRefGoogle Scholar - Ling, H., & Okada, K. (2007). An efficient earth mover’s distance algorithm for robust histogram comparison. In
*IEEE transactions on pattern analysis and machine intelligence*(pp. 840–853). Google Scholar - Lloyd, S. P. (1982). Least squares quantization in pcm.
*IEEE Transactions on Information Theory*,*28*(2), 129–137. MathSciNetzbMATHCrossRefGoogle Scholar - Loo, L., Wu, L., & Altschuler, S. (2007). Image-based multivariate profiling of drug responses from single cells.
*Nature Methods*,*4*(5), 445–454. Google Scholar - Methora, S. (1992). On the implementation of a primal-dual interior point method.
*SIAM Journal on Scientific and Statistical Computing*,*2*, 575–601. Google Scholar - Miller, M. I., Priebe, C. E., Qiu, A., Fischl, B., Kolasny, A., Brown, T., Park, Y., Ratnanather, J. T., Busa, E., Jovicich, J., Yu, P., Dickerson, B. C., & Buckner, R. L. (2009). Collaborative computational anatomy: an mri morphometry study of the human brain via diffeomorphic metric mapping.
*Human Brain Mapping*,*30*(7), 2132–2141. CrossRefGoogle Scholar - Moss, T. J., & Wallrath, L. L. (2007). Connections between epigenetic gene silencing and human disease.
*Mutation Research*,*618*(1–2), 163–174. CrossRefGoogle Scholar - Orlin, J. B. (1993). A faster strongly polynomial minimum cost flow algorithm.
*Operations Research*,*41*(2), 338–350. MathSciNetzbMATHCrossRefGoogle Scholar - Pele, O., & Werman, M. (2008). A linear time histogram metric for improved sift matching. In
*ECCV*. Google Scholar - Pele, O., & Werman, M. (2009). Fast and robust earth mover’s distances. In
*Computer vision, 2009 IEEE 12th international conference on*(pp. 460–467). New York: IEEE Press. CrossRefGoogle Scholar - Pincus, Z., & Theriot, J. A. (2007). Comparison of quantitative methods for cell-shape analysis.
*Journal of Microscopy*,*227*(2), 140–156. MathSciNetCrossRefGoogle Scholar - Rohde, G. K., Ribeiro, A. J. S., Dahl, K. N., & Murphy, R. F. (2008). Deformation-based nuclear morphometry: capturing nuclear shape variation in hela cells.
*Cytometry*,*73*(4), 341–350. CrossRefGoogle Scholar - Roweis, S. T., & Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding.
*Science*,*290*(5500), 2323–2326. doi: 10.1126/science.290.5500.2323. CrossRefGoogle Scholar - Rubner, Y., Tomassi, C., & Guibas, L. J. (2000). The earth mover’s distance as a metric for image retrieval.
*International Journal of Computer Vision*,*40*(2), 99–121. zbMATHCrossRefGoogle Scholar - Rueckert, D., Frangi, A. F., & Schnabel, J. A. (2003). Automatic construction of 3-d statistical deformation models of the brain using nonrigid registration.
*IEEE Transactions on Medical Imaging*,*22*(8), 1014–1025. CrossRefGoogle Scholar - Shamir, L. (2009). Automatic morphological classification of galaxy images.
*Monthly Notices of the Royal Astronomical Society*. Google Scholar - Shirdhonkar, S., & Jacobs, D. (2008). Approximate earth mover’s distance in linear time. In
*Proceedings of the 2008 IEEE computer society conference on computer vision and pattern recognition, CVPR 2008*. Google Scholar - Stegmann, M., Ersboll, B., & Larsen, R. (2003). Fame—a flexible appearance modeling environment.
*IEEE Transactions on Medical Imaging*,*22*(10), 1319–1331. CrossRefGoogle Scholar - Tenenbaum, J. B., de Silva, V., & Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction.
*Science*,*290*(5500), 2319–2323. doi: 10.1126/science.290.5500.2319. CrossRefGoogle Scholar - Vaillant, M., Miller, M., Younes, L., & Trouvé, A. (2004). Statistics on diffeomorphisms via tangent space representations.
*NeuroImage*,*23*, S161–S169. CrossRefGoogle Scholar - Villani, C. (2003).
*Graduate studies in mathematics: Vol.**58*.*Topics in optimal transportation*. Providence: Am. Math. Soc.. zbMATHGoogle Scholar - Villani, C. (2009).
*In Grundlehren der Mathematischen Wissenschaften [Fundamental principles of mathematical sciences]: Vol.**338*.*Optimal transport*. Berlin: Springer. doi: 10.1007/978-3-540-71050-9. zbMATHCrossRefGoogle Scholar - Wang, W., Mo, Y., Ozolek, J. A., & Rohde, G. K. (2011). Penalized fisher discriminant analysis and its application to image-based morphometry.
*Pattern Recognition Letters*,*32*(15), 2128–2135. CrossRefGoogle Scholar - Wang, W., Ozolek, J., & Rohde, G. (2010). Detection and classification of thyroid follicular lesions based on nuclear structure from histopathology images.
*Cytometry Part A*,*77*(5), 485–494. Google Scholar - Wang, W., Ozolek, J., Slepčev, D., Lee, A., Chen, C., & Rohde, G. (2011). An optimal transportation approach for nuclear structure-based pathology.
*IEEE Transactions on Medical Imaging*,*30*(3), 621–631. CrossRefGoogle Scholar - Yang, L., Chen, W., Meer, P., Salaru, G., Goodell, L., Berstis, V., & Foran, D. (2009). Virtual microscopy and grid-enabled decision support for large scale analysis of imaged pathology specimens.
*IEEE Transactions on Information Technology in Biomedicine*,*13*(4), 636–644. CrossRefGoogle Scholar