Journal of Mathematical Imaging and Vision

, Volume 46, Issue 2, pp 211–237 | Cite as

Lattice-Based High-Dimensional Gaussian Filtering and the Permutohedral Lattice

Article

Abstract

High-dimensional Gaussian filtering is a popular technique in image processing, geometry processing and computer graphics for smoothing data while preserving important features. For instance, the bilateral filter, cross bilateral filter and non-local means filter fall under the broad umbrella of high-dimensional Gaussian filters. Recent algorithmic advances therein have demonstrated that by relying on a sampled representation of the underlying space, one can obtain speed-ups of orders of magnitude over the naïve approach. The simplest such sampled representation is a lattice, and it has been used successfully in the bilateral grid and the permutohedral lattice algorithms. In this paper, we analyze these lattice-based algorithms, developing a general theory of lattice-based high-dimensional Gaussian filtering. We consider the set of criteria for an optimal lattice for filtering, as it offers a good tradeoff of quality for computational efficiency, and evaluate the existing lattices under the criteria. In particular, we give a rigorous exposition of the properties of the permutohedral lattice and argue that it is the optimal lattice for Gaussian filtering. Lastly, we explore further uses of the permutohedral-lattice-based Gaussian filtering framework, showing that it can be easily adapted to perform mean shift filtering and yield improvement over the traditional approach based on a Cartesian grid.

Keywords

Bilateral filtering High-dimensional filtering Non-local means Lattices Gaussian filtering Permutohedral lattice 

Notes

Acknowledgements

We would like to thank Marc Levoy for his advice and support, as well as Nokia Research.

Jongmin Baek acknowledges support from Nokia Research, as well as Lucent Technologies Stanford Graduate Fellowship; Andrew Adams is supported by a Reed-Hodgson Stanford Graduate Fellowship; Jennifer Dolson acknowledges support from an NDSEG Graduate Fellowship from the United States Department of Defense.

References

  1. 1.
    Adams, A., Gelfand, N., Dolson, J., Levoy, M.: Gaussian kd-trees for fast high-dimensional filtering. In: ACM Transactions on Graphics (Proc SIGGRAPH ’09), pp. 1–12 (2009) Google Scholar
  2. 2.
    Adams, A., Baek, J., Davis, A.: High-dimensional filtering with the permutohedral lattice. In: Proceedings of EUROGRAPHICS, pp. 753–762 (2010) Google Scholar
  3. 3.
    Arbelaez, P., Maire, M., Fowlkes, C.C., Malik, J.: From contours to regions: an empirical evaluation. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 2294–2301 (2009) Google Scholar
  4. 4.
    Aurich, V., Weule, J.: Non-linear Gaussian filters performing edge preserving diffusion. In: Mustererkennung 1995, 17. DAGM-Symposium, pp. 538–545 (1995) CrossRefGoogle Scholar
  5. 5.
    Bambah, R.P., Sloane, N.J.A.: On a problem of Ryskov concerning lattice coverings. Acta Arith. 42, 107–109 (1982) MathSciNetMATHGoogle Scholar
  6. 6.
    de Boor, C.: B-form basics. In: Geometric Modeling, pp. 131–148 (1987) Google Scholar
  7. 7.
    de Boor, C., Höllig, J., Riemenschneider, S.: Box splines, vol. 98. Springer, Berlin (1993) MATHCrossRefGoogle Scholar
  8. 8.
    Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 60–65 (2005) Google Scholar
  9. 9.
    Chen, J., Paris, S., Durand, F.: Real-time edge-aware image processing with the bilateral grid. In: ACM Transactions on Graphics (Proc SIGGRAPH ’07), p. 103 (2007) Google Scholar
  10. 10.
    Cheng, Y.: Mean shift, mode seeking, and clustering. IEEE Trans. Pattern Anal. Mach. Intell. 17(8), 790–799 (1995) CrossRefGoogle Scholar
  11. 11.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002) CrossRefGoogle Scholar
  12. 12.
    Conway, J.H., Sloane, N.J.A.: Sphere Packings, Lattices and Groups. Springer, Berlin (1998) Google Scholar
  13. 13.
    Delaunay, B.N., Ryskov, S.S.: Solution of the problem of least dense lattice covering of a four-dimensional space by equal spheres. Sov. Math. Dokl. 4, 1014–1016 (1963) Google Scholar
  14. 14.
    Dolbilin, N.P.: Minkowski’s theorems on parallelohedra and their generalizations. Russ. Math. Surv. 62, 793–795 (2007) MathSciNetMATHCrossRefGoogle Scholar
  15. 15.
    Dolson, J., Baek, J., Plagemann, C., Thrun, S.: Upsampling range data in dynamic environments. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 1141–1148 (2010) Google Scholar
  16. 16.
    Durand, F., Dorsey, J.: Fast bilateral filtering for the display of high-dynamic-range images. In: Proc. SIGGRAPH ’02, pp. 257–266 (2002) CrossRefGoogle Scholar
  17. 17.
    Eisemann, E., Durand, F.: Flash photography enhancement via intrinsic relighting. In: ACM Transactions on Graphics (Proc. SIGGRAPH 04), pp. 673–678 (2004) CrossRefGoogle Scholar
  18. 18.
    Entezari, A., Dyer, R., Möller, T.: Linear and cubic box splines for the body centered cubic lattice. In: IEEE Visualization, pp. 11–18 (2004) Google Scholar
  19. 19.
    Entezari, A., Ville, D.V.D., Möller, T.: Practical box splines for reconstruction on the body centered cubic lattice. IEEE Trans. Vis. Comput. Graph. 14, 313–328 (2008) CrossRefGoogle Scholar
  20. 20.
    Greengard, L.F., Strain, J.A.: The fast gauss transform. SIAM J. Sci. Stat. Comput. 12, 79–94 (1991) MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Grundmann, M., Kwatra, V., Han, M., Essa, I.: Efficient hierarchical graph-based video segmentation. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 2141–2148 (2010) Google Scholar
  22. 22.
    Kershner, R.: The number of circles covering a set. Am. J. Math. 61, 665–671 (1939) MathSciNetCrossRefGoogle Scholar
  23. 23.
    Kim, M.: Symmetric box-splines on root lattice. PhD thesis, University of Florida, Gainesville, FL (2008) Google Scholar
  24. 24.
    Kopf, J., Cohen, M.F., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. In: ACM Transactions on Graphics (Proc. SIGGRAPH 07), p. 96 (2007) Google Scholar
  25. 25.
    Kuhn, H.W.: Some combinatorial lemmas in topology. IBM J. Res. Dev. 45, 518–524 (1960) CrossRefGoogle Scholar
  26. 26.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proc. 8th Int. Conf. Computer Vision, vol. 2, pp. 416–423 (2001) Google Scholar
  27. 27.
    Mirzargar, M., Entezari, A.: Voronoi splines. IEEE Trans. Signal Process. 58, 4572–4582 (2010) MathSciNetCrossRefGoogle Scholar
  28. 28.
    Mood, R.V., Patera, J.: Voronoi and Delaunay cells of root lattices: classification of their faces and facets by Coxeter-Dynkin diagrams. J. Phys. A, Math. Gen. 25, 5089–5134 (1992) CrossRefGoogle Scholar
  29. 29.
    Paris, S.: Edge-preserving smoothing and mean-shift segmentation of video streams. In: Proc. European Conference on Computer Vision, pp. 460–473 (2008) Google Scholar
  30. 30.
    Paris, S., Durand, F.: A fast approximation of the bilateral filter using a signal processing approach. In: Proc. European Conference on Computer Vision, pp. 568–580 (2006) Google Scholar
  31. 31.
    Paris, S., Durand, F.: A topological approach to hierarchical segmentation using mean shift. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007) Google Scholar
  32. 32.
    Paris, S., Durand, F.: A fast approximation of the bilateral filter using a signal processing approach. Int. J. Comput. Vis. 81(1), 24–52 (2009) CrossRefGoogle Scholar
  33. 33.
    Perlin, K.: Noise. In: ACM SIGGRAPH ’02 Course Notes (2002) Google Scholar
  34. 34.
    Petersen, D.P., Middleton, D.: Sampling and reconstruction of wave-number-limited functions in N-dimensional Euclidean spaces. Inf. Control 5, 279–323 (1962) MathSciNetCrossRefGoogle Scholar
  35. 35.
    Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K.: Digital photography with flash and no-flash image pairs. In: ACM Transactions on Graphics (Proc. SIGGRAPH 04), pp. 664–672 (2004) CrossRefGoogle Scholar
  36. 36.
    Rahimi, A., Recht, B.: Random features for large-scale kernel machines. Adv. Neural Inf. Process. Syst. 20, 1177–1184 (2008) Google Scholar
  37. 37.
    Rogers, C.A.: Packing and Covering. Cambridge University Press, Cambridge (1964) MATHGoogle Scholar
  38. 38.
    Ryshkov, S.S.: The structure of primitive parallelohedra and Voronoi’s last problem. Russ. Math. Surv. 53, 403–405 (1998) MathSciNetMATHCrossRefGoogle Scholar
  39. 39.
    Ryskov, S.S., Baranovskii, E.P.: Solution of the problem of least dense lattice covering of five-dimensional space by equal spheres. Sov. Math. Dokl. 16, 586–590 (1975) Google Scholar
  40. 40.
    Samelson, H.: Notes on Lie Algebras. Van Nostrand Reinhold Company, Princeton (1969) MATHGoogle Scholar
  41. 41.
    Senechal, M.: Quasicrystals and Geometry. Cambridge University Press, Cambridge (1995) MATHGoogle Scholar
  42. 42.
    Serre, J.P.: Complex Semisimple Lie Algebras. Springer, Berlin (1987) MATHCrossRefGoogle Scholar
  43. 43.
    Smith, S., Brady, J.M.: SUSAN: a new approach to low level image processing. Int. J. Comput. Vis. 23, 45–78 (1997) CrossRefGoogle Scholar
  44. 44.
    Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proc. International Conference on Computer Vision, pp. 836–846 (1998) Google Scholar
  45. 45.
    Voronoi, G.F.: Studies of Primitive Parallelotopes, vol. 2, pp. 239–368. Ukrainian Academy of Sciences Press, Kiev (1952) Google Scholar
  46. 46.
    Wang, P., Lee, D., Gray, A., Rehg, J.M.: Fast mean shift with accurate and stable convergence. In: Melia, M., Shen, X. (eds.) Artificial Intelligence and Statistics (2007) Google Scholar
  47. 47.
    Weiss, B.: Fast median and bilateral filtering. In: ACM Transactions on Graphics (Proc. SIGGRAPH ’06), pp. 519–526 (2006) Google Scholar
  48. 48.
    Winnemöller, H., Olsen, S.C., Gooch, B.: Real-time video abstraction. ACM Trans. Graph. 25(3), 1221–1226 (2006) CrossRefGoogle Scholar
  49. 49.
    Witsenhausen, H.S.: Spiegelungsgruppen und aufzählung halbeinfacher liescher Ringe. Abh. Math. Sem. Univ. Hamburg 14 (1941) Google Scholar
  50. 50.
    Yang, C., Duraiswami, R., Gumerov, N.A., Davis, L.: Improved fast gauss transform and efficient kernel density estimation. In: Proc. International Conference on Computer Vision, vol. 1, pp. 664–671 (2003) CrossRefGoogle Scholar
  51. 51.
    Yang, Q., Yang, R., Davis, J., Nistér, D.: Spatial-depth super resolution for range images. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007) Google Scholar
  52. 52.
    Yaroslavsky, L.P.: Digital Picture Processing. An Introduction. Springer, Berlin (1985) MATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.Stanford UniversityStanfordUSA
  2. 2.Massachusetts Institute of TechnologyCambridgeUSA

Personalised recommendations