Skip to main content
SpringerLink
Log in

Assignment Flows

  • Chapter
  • First Online:
Handbook of Variational Methods for Nonlinear Geometric Data

Abstract

Assignment flows comprise basic dynamical systems for modeling data labeling and related machine learning tasks in supervised and unsupervised scenarios. They provide adaptive time-variant extensions of established discrete graphical models and a basis for the design and better mathematical understanding of hierarchical networks, using methods from information (differential) geometry, geometric numerical integration, statistical inference, optimal transport and control. This chapter introduces the framework by means of the image labeling problem and outlines directions of current and further research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    This includes spatio-temporal data—like e.g. videos—observed at points \((t_{i},x_{i}) \in [0,T] \times \Omega \subset \mathbb {R} \times \mathbb {R}^{d}\) in time and space.

  2. 2.

    The sizes of the components \(D_{i}^{j},\, j \in \mathcal {J}\) relative to each other only matter.

  3. 3.

    Here we overload the symbol Ω which denotes the Euclidean domain covered by the graph \(\mathcal {G}\), as mentioned after Eq. (8.2). Due to the subscripts Ωi and the context, there should be no danger of confusion.

  4. 4.

    Not to be confused with labels \(\mathcal {F}_{\ast }\)!

References

  1. Amari, S.I., Nagaoka, H.: Methods of Information Geometry. American Mathematical Society/Oxford University Press, Providence/Oxford (2000)

    MATH  Google Scholar 

  2. Antun, V., Renna, F., Poon, C., Adcock, B., Hansen, A.C.: On instabilities of deep learning in image reconstruction: does AI come at a cost? (2019). arXiv preprint arXiv:abs/1902.05300

    Google Scholar 

  3. Åström, F., Petra, S., Schmitzer, B., Schnörr, C.: Image labeling by assignment. J. Math. Imaging Vis. 58(2), 211–238 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  4. Ay, N., Jost, J., Lê, H.V., Schwachhöfer, L.: Information Geometry. Springer, Berlin (2017)

    Book  MATH  Google Scholar 

  5. Barndorff-Nielsen, O.E.: Information and Exponential Families in Statistical Theory. Wiley, Chichester (1978)

    MATH  Google Scholar 

  6. Basseville, M.: Divergence measures for statistical data processing—an annotated bibliography. Signal Proc. 93(4), 621–633 (2013)

    Article  MathSciNet  Google Scholar 

  7. Bergmann, R., Tenbrinck, D.: A graph framework for manifold-valued data. SIAM J. Imaging Sci. 11(1), 325–360 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  8. Berman, A., Shaked-Monderer, N.: Completely Positive Matrices. World Scientific, Singapore (2003)

    Book  MATH  Google Scholar 

  9. Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)

    Article  Google Scholar 

  10. Calin, O., Udriste, C.: Geometric Modeling in Probability and Statistics. Springer, Berlin (2014)

    Book  MATH  Google Scholar 

  11. Chan, T.F., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM J. Appl. Math. 66(5), 1632–1648 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  12. Cichocki, A., Zdunek, A., Phan, A.H., Amari, S.I.: Nonnegative Matrix and Tensor Factorizations. Wiley, London (2009)

    Book  Google Scholar 

  13. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20, 273–297 (1995)

    MATH  Google Scholar 

  14. Cover, T.M., Thomas, J.A.: Elements of Information Theory, 2nd edn. Wiley, London (2006)

    MATH  Google Scholar 

  15. Elad, M.: Deep, deep trouble: deep learning’s impact on image processing, mathematics, and humanity. SIAM News (2017)

    Google Scholar 

  16. Gary, R.M., Neuhoff, D.L.: Quantization. IEEE Trans. Inform. Theory 44(6), 2325–2383 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  17. Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6(6), 721–741 (1984)

    Article  MATH  Google Scholar 

  18. Graf, S., Luschgy, H.: Foundations of Quantization for Probability Distributions. Lecture Notes in Mathematics, vol. 1730. Springer, Berlin (2000)

    Google Scholar 

  19. Hairer, E., Lubich, C., Wanner, G.: Geometric Numerical Integration. Springer, Berlin (2006)

    MATH  Google Scholar 

  20. Har-Peled, S.: Geometric Approximation Algorithms. AMS, Providence (2011)

    Google Scholar 

  21. Hofbauer, J., Siegmund, K.: Evolutionary game dynamics. Bull. Am. Math. Soc. 40(4), 479–519 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  22. Hühnerbein, R., Savarino, F., Åström, F., Schnörr, C.: Image labeling based on graphical models using Wasserstein messages and geometric assignment. SIAM J. Imaging Sci. 11(2), 1317–1362 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  23. Hühnerbein, R., Savarino, F., Petra, S., Schnörr, C.: Learning adaptive regularization for image labeling using geometric assignment. In: Proceedings of the International Conference on Scale Space and Variational Methods in Computer Vision (SSVM). Springer, Berlin (2019)

    Google Scholar 

  24. Hummel, R.A., Zucker, S.W.: On the foundations of the relaxation labeling processes. IEEE Trans. Pattern Anal. Mach. Intell. 5(3), 267–287 (1983)

    Article  MATH  Google Scholar 

  25. Idel, M.: A Review of Matrix scaling and Sinkhorn’s normal form for matrices and positive maps (2016). arXiv preprint arXiv:abs/1609.06349

    Google Scholar 

  26. Iserles, A., Munthe-Kaas, H.Z., Nørsett, S.P., Zanna, A.: Lie-group methods. Acta Numer. 14, 1–148 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  27. Jost, J.: Riemannian Geometry and Geometric Analysis, 7th edn. Springer, Berlin (2017)

    Book  MATH  Google Scholar 

  28. Kappes, J., Andres, B., Hamprecht, F., Schnörr, C., Nowozin, S., Batra, D., Kim, S., Kausler, B., Kröger, T., Lellmann, J., Komodakis, N., Savchynskyy, B., Rother, C.: A comparative study of modern inference techniques for structured discrete energy minimization problems. Int. J. Comput. Vis. 115(2), 155–184 (2015)

    Article  MathSciNet  Google Scholar 

  29. Kleinberg, J., Tardos, E.: Approximation algorithms for classification problems with pairwise relationships: metric labeling and Markov random fields. J. ACM 49(5), 616–639 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  30. Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press, Cambridge (2009)

    MATH  Google Scholar 

  31. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems (NIPS), pp. 1097–1105. ACM, New York (2012)

    Google Scholar 

  32. Lauritzen, S.L.: Chapter 4: statistical manifolds. In: Gupta, S.S., Amari, S.I., Barndorff-Nielsen, O.E., Kass, R.E., Lauritzen, S.L., Rao, C.R. (eds.) Differential Geometry in Statistical Inference, pp. 163–216. Institute of Mathematical Statistics, Hayward (1987)

    Chapter  Google Scholar 

  33. Lauritzen, S.L.: Graphical Models. Clarendon Press, Oxford (1996)

    MATH  Google Scholar 

  34. Lee, J.M.: Introduction to Smooth Manifolds. Springer, Berlin (2013)

    MATH  Google Scholar 

  35. Lellmann, J., Schnörr, C.: Continuous multiclass labeling approaches and algorithms. SIAM J. Imag. Sci. 4(4), 1049–1096 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  36. Mézard, M., Montanari, A.: Information, Physics, and Computation. Oxford University Press, Oxford (2009)

    Book  MATH  Google Scholar 

  37. Munthe-Kaas, H.: High order Runge-Kutta methods on manifolds. Appl. Numer. Math. 29(1), 115–127 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  38. Pavan, M., Pelillo, M.: Dominant sets and pairwise clustering. IEEE Trans. Pattern Anal. Mach. Intell. 29(1), 167–172 (2007)

    Article  Google Scholar 

  39. Pelillo, M.: The dynamics of nonlinear relaxation labeling processes. J. Math. Imaging Vision 7, 309–323 (1997)

    Article  MathSciNet  Google Scholar 

  40. Peyré, G., Cuturi, M.: Computational Optimal Transport. CNRS, Paris (2018)

    MATH  Google Scholar 

  41. Phillips, J.: Coresets and sketches. In: Handbook of Discrete and Computational Geometry, chapter 48. CRC Press, Boca Raton (2016)

    Google Scholar 

  42. Povh, J., Rendl, F.: A copositive programming approach to graph partitioning. SIAM J. Optim. 18(1), 223–241 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  43. Rosenfeld, A., Hummel, R.A., Zucker, S.W.: Scene labeling by relaxation operations. IEEE Trans. Syst. Man Cybern. 6, 420–433 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  44. Ross, I.: A roadmap for optimal control: the right way to commute. Ann. N.Y. Acad. Sci. 1065(1), 210–231 (2006)

    Article  Google Scholar 

  45. Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  46. Rumelhart, D.E., McClelland, J.L.: Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations. MIT Press, Boca Raton (1986)

    Book  Google Scholar 

  47. Sandholm, W.H.: Population Games and Evolutionary Dynamics. MIT Press, Boca Raton (2010)

    MATH  Google Scholar 

  48. Sanz-Serna, J.: Symplectic Runge–Kutta schemes for adjoint equations, automatic differentiation, optimal control, and more. SIAM Rev. 58(1), 3–33 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  49. Savarino, F., Schnörr, C.: A variational perspective on the assignment flow. In: Proceedings of the International Conference on Scale Space and Variational Methods in Computer Vision (SSVM). Springer, Berlin (2019)

    Google Scholar 

  50. Savarino, F., Schnörr, C.: Continuous-domain assignment flows. Heidelberg University, October (2019). Preprint, submitted for publication

    Google Scholar 

  51. Wainwright, M.J., Jordan, M.I.: Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn. 1(1–2), 1–305 (2008)

    Article  MATH  Google Scholar 

  52. Wasserman, L.: All of Nonparametric Statistics. Springer, Berlin (2006)

    MATH  Google Scholar 

  53. Weinmann, A., Demaret, L., Storath, M.: Total variation regularization for manifold-valued data. SIAM J. Imag. Sci. 7(4), 2226–2257 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  54. Werner, T.: A linear programming approach to max-sum problem: a review. IEEE Trans. Pattern Anal. Mach. Intell. 29(7), 1165–1179 (2007)

    Article  Google Scholar 

  55. Yedidia, J.S., Freeman, W.T., Weiss, Y.: Constructing free-energy approximations and generalized belief propagation algorithms. IEEE Trans. Inform. Theory 51(7), 2282–2312 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  56. Zeilmann, A., Savarino, F., Petra, S., Schnörr, C.: Geometric numerical integration of the assignment flow. Inverse Problems, https://doi.org/10.1088/1361-6420/ab2772 (2019, in press)

  57. Zern, A., Zisler, M., Åström, F., Petra, S., Schnörr, C.: Unsupervised label learning on manifolds by spatially regularized geometric assignment. In: Proceedings of German Conference on Pattern Recognition (GCPR). Springer, Berlin (2018)

    Google Scholar 

  58. Zern, A., Zisler, M., Petra, S., Schnörr, C.: Unsupervised assignment flow: label learning on feature manifolds by spatially regularized geometric assignment (2019). arXiv preprint arXiv:abs/1904.10863

    Google Scholar 

  59. Zisler, M., Zern, A., Petra, S., Schnörr, C.: Unsupervised labeling by geometric and spatially regularized self-assignment. In: Proceedings of the Scale Space and Variational Methods in Computer Vision (SSVM). Springer, Berlin (2019)

    Google Scholar 

  60. Zisler, M., Zern, A., Petra, S., Schnörr, C.: Self-assignment flows for unsupervised data labeling on graphs. Heidelberg University, October (2019). Preprint, submitted for publication

    Google Scholar 

Download references

Acknowledgements

I thank my students Ruben Hühnerbein, Fabrizio Savarino, Alexander Zeilmann, Artjom Zern, Matthias Zisler and my colleague Stefania Petra for many discussions and collaboration. We gratefully acknowledge support by the German Science Foundation (DFG), grant GRK 1653.

This work has also been stimulated by the recently established Heidelberg STRUCTURES Excellence Cluster, funded by the DFG under Germany’s Excellence Strategy EXC-2181/1 - 390900948.

Author information

Authors and Affiliations

  1. Institute of Applied Mathematics, Heidelberg University, Heidelberg, Germany

    Christoph Schnörr

Authors
  1. Christoph Schnörr
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christoph Schnörr .

Editor information

Editors and Affiliations

  1. Faculty of Mathematics, University of Vienna, Wien, Austria

    Philipp Grohs

  2. Institute of Mathematics and Scientific Computing, University of Graz, Graz, Austria

    Martin Holler

  3. Department of Mathematics and Natural Sciences, Hochschule Darmstadt, Darmstadt, Germany

    Andreas Weinmann

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Schnörr, C. (2020). Assignment Flows. In: Grohs, P., Holler, M., Weinmann, A. (eds) Handbook of Variational Methods for Nonlinear Geometric Data. Springer, Cham. https://doi.org/10.1007/978-3-030-31351-7_8

Download citation

Publish with us

Policies and ethics

Search

Navigation