# Lattice-Based High-Dimensional Gaussian Filtering and the Permutohedral Lattice

## Abstract

High-dimensional Gaussian filtering is a popular technique in image processing, geometry processing and computer graphics for smoothing data while preserving important features. For instance, the bilateral filter, cross bilateral filter and non-local means filter fall under the broad umbrella of high-dimensional Gaussian filters. Recent algorithmic advances therein have demonstrated that by relying on a sampled representation of the underlying space, one can obtain speed-ups of orders of magnitude over the naïve approach. The simplest such sampled representation is a *lattice*, and it has been used successfully in the bilateral grid and the permutohedral lattice algorithms. In this paper, we analyze these lattice-based algorithms, developing a general theory of lattice-based high-dimensional Gaussian filtering. We consider the set of criteria for an optimal lattice for filtering, as it offers a good tradeoff of quality for computational efficiency, and evaluate the existing lattices under the criteria. In particular, we give a rigorous exposition of the properties of the permutohedral lattice and argue that it is the optimal lattice for Gaussian filtering. Lastly, we explore further uses of the permutohedral-lattice-based Gaussian filtering framework, showing that it can be easily adapted to perform mean shift filtering and yield improvement over the traditional approach based on a Cartesian grid.

## Keywords

Bilateral filtering High-dimensional filtering Non-local means Lattices Gaussian filtering Permutohedral lattice## Notes

### Acknowledgements

We would like to thank Marc Levoy for his advice and support, as well as Nokia Research.

Jongmin Baek acknowledges support from Nokia Research, as well as Lucent Technologies Stanford Graduate Fellowship; Andrew Adams is supported by a Reed-Hodgson Stanford Graduate Fellowship; Jennifer Dolson acknowledges support from an NDSEG Graduate Fellowship from the United States Department of Defense.

## References

- 1.Adams, A., Gelfand, N., Dolson, J., Levoy, M.: Gaussian kd-trees for fast high-dimensional filtering. In: ACM Transactions on Graphics (Proc SIGGRAPH ’09), pp. 1–12 (2009) Google Scholar
- 2.Adams, A., Baek, J., Davis, A.: High-dimensional filtering with the permutohedral lattice. In: Proceedings of EUROGRAPHICS, pp. 753–762 (2010) Google Scholar
- 3.Arbelaez, P., Maire, M., Fowlkes, C.C., Malik, J.: From contours to regions: an empirical evaluation. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 2294–2301 (2009) Google Scholar
- 4.Aurich, V., Weule, J.: Non-linear Gaussian filters performing edge preserving diffusion. In: Mustererkennung 1995, 17. DAGM-Symposium, pp. 538–545 (1995) CrossRefGoogle Scholar
- 5.Bambah, R.P., Sloane, N.J.A.: On a problem of Ryskov concerning lattice coverings. Acta Arith.
**42**, 107–109 (1982) MathSciNetzbMATHGoogle Scholar - 6.de Boor, C.: B-form basics. In: Geometric Modeling, pp. 131–148 (1987) Google Scholar
- 7.de Boor, C., Höllig, J., Riemenschneider, S.: Box splines, vol. 98. Springer, Berlin (1993) zbMATHCrossRefGoogle Scholar
- 8.Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 60–65 (2005) Google Scholar
- 9.Chen, J., Paris, S., Durand, F.: Real-time edge-aware image processing with the bilateral grid. In: ACM Transactions on Graphics (Proc SIGGRAPH ’07), p. 103 (2007) Google Scholar
- 10.Cheng, Y.: Mean shift, mode seeking, and clustering. IEEE Trans. Pattern Anal. Mach. Intell.
**17**(8), 790–799 (1995) CrossRefGoogle Scholar - 11.Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell.
**24**(5), 603–619 (2002) CrossRefGoogle Scholar - 12.Conway, J.H., Sloane, N.J.A.: Sphere Packings, Lattices and Groups. Springer, Berlin (1998) Google Scholar
- 13.Delaunay, B.N., Ryskov, S.S.: Solution of the problem of least dense lattice covering of a four-dimensional space by equal spheres. Sov. Math. Dokl.
**4**, 1014–1016 (1963) Google Scholar - 14.Dolbilin, N.P.: Minkowski’s theorems on parallelohedra and their generalizations. Russ. Math. Surv.
**62**, 793–795 (2007) MathSciNetzbMATHCrossRefGoogle Scholar - 15.Dolson, J., Baek, J., Plagemann, C., Thrun, S.: Upsampling range data in dynamic environments. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 1141–1148 (2010) Google Scholar
- 16.Durand, F., Dorsey, J.: Fast bilateral filtering for the display of high-dynamic-range images. In: Proc. SIGGRAPH ’02, pp. 257–266 (2002) CrossRefGoogle Scholar
- 17.Eisemann, E., Durand, F.: Flash photography enhancement via intrinsic relighting. In: ACM Transactions on Graphics (Proc. SIGGRAPH 04), pp. 673–678 (2004) CrossRefGoogle Scholar
- 18.Entezari, A., Dyer, R., Möller, T.: Linear and cubic box splines for the body centered cubic lattice. In: IEEE Visualization, pp. 11–18 (2004) Google Scholar
- 19.Entezari, A., Ville, D.V.D., Möller, T.: Practical box splines for reconstruction on the body centered cubic lattice. IEEE Trans. Vis. Comput. Graph.
**14**, 313–328 (2008) CrossRefGoogle Scholar - 20.Greengard, L.F., Strain, J.A.: The fast gauss transform. SIAM J. Sci. Stat. Comput.
**12**, 79–94 (1991) MathSciNetzbMATHCrossRefGoogle Scholar - 21.Grundmann, M., Kwatra, V., Han, M., Essa, I.: Efficient hierarchical graph-based video segmentation. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 2141–2148 (2010) Google Scholar
- 22.Kershner, R.: The number of circles covering a set. Am. J. Math.
**61**, 665–671 (1939) MathSciNetCrossRefGoogle Scholar - 23.Kim, M.: Symmetric box-splines on root lattice. PhD thesis, University of Florida, Gainesville, FL (2008) Google Scholar
- 24.Kopf, J., Cohen, M.F., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. In: ACM Transactions on Graphics (Proc. SIGGRAPH 07), p. 96 (2007) Google Scholar
- 25.Kuhn, H.W.: Some combinatorial lemmas in topology. IBM J. Res. Dev.
**45**, 518–524 (1960) CrossRefGoogle Scholar - 26.Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proc. 8th Int. Conf. Computer Vision, vol. 2, pp. 416–423 (2001) Google Scholar
- 27.Mirzargar, M., Entezari, A.: Voronoi splines. IEEE Trans. Signal Process.
**58**, 4572–4582 (2010) MathSciNetCrossRefGoogle Scholar - 28.Mood, R.V., Patera, J.: Voronoi and Delaunay cells of root lattices: classification of their faces and facets by Coxeter-Dynkin diagrams. J. Phys. A, Math. Gen.
**25**, 5089–5134 (1992) CrossRefGoogle Scholar - 29.Paris, S.: Edge-preserving smoothing and mean-shift segmentation of video streams. In: Proc. European Conference on Computer Vision, pp. 460–473 (2008) Google Scholar
- 30.Paris, S., Durand, F.: A fast approximation of the bilateral filter using a signal processing approach. In: Proc. European Conference on Computer Vision, pp. 568–580 (2006) Google Scholar
- 31.Paris, S., Durand, F.: A topological approach to hierarchical segmentation using mean shift. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007) Google Scholar
- 32.Paris, S., Durand, F.: A fast approximation of the bilateral filter using a signal processing approach. Int. J. Comput. Vis.
**81**(1), 24–52 (2009) CrossRefGoogle Scholar - 33.Perlin, K.: Noise. In: ACM SIGGRAPH ’02 Course Notes (2002) Google Scholar
- 34.Petersen, D.P., Middleton, D.: Sampling and reconstruction of wave-number-limited functions in N-dimensional Euclidean spaces. Inf. Control
**5**, 279–323 (1962) MathSciNetCrossRefGoogle Scholar - 35.Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K.: Digital photography with flash and no-flash image pairs. In: ACM Transactions on Graphics (Proc. SIGGRAPH 04), pp. 664–672 (2004) CrossRefGoogle Scholar
- 36.Rahimi, A., Recht, B.: Random features for large-scale kernel machines. Adv. Neural Inf. Process. Syst.
**20**, 1177–1184 (2008) Google Scholar - 37.Rogers, C.A.: Packing and Covering. Cambridge University Press, Cambridge (1964) zbMATHGoogle Scholar
- 38.Ryshkov, S.S.: The structure of primitive parallelohedra and Voronoi’s last problem. Russ. Math. Surv.
**53**, 403–405 (1998) MathSciNetzbMATHCrossRefGoogle Scholar - 39.Ryskov, S.S., Baranovskii, E.P.: Solution of the problem of least dense lattice covering of five-dimensional space by equal spheres. Sov. Math. Dokl.
**16**, 586–590 (1975) Google Scholar - 40.Samelson, H.: Notes on Lie Algebras. Van Nostrand Reinhold Company, Princeton (1969) zbMATHGoogle Scholar
- 41.Senechal, M.: Quasicrystals and Geometry. Cambridge University Press, Cambridge (1995) zbMATHGoogle Scholar
- 42.Serre, J.P.: Complex Semisimple Lie Algebras. Springer, Berlin (1987) zbMATHCrossRefGoogle Scholar
- 43.Smith, S., Brady, J.M.: SUSAN: a new approach to low level image processing. Int. J. Comput. Vis.
**23**, 45–78 (1997) CrossRefGoogle Scholar - 44.Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proc. International Conference on Computer Vision, pp. 836–846 (1998) Google Scholar
- 45.Voronoi, G.F.: Studies of Primitive Parallelotopes, vol. 2, pp. 239–368. Ukrainian Academy of Sciences Press, Kiev (1952) Google Scholar
- 46.Wang, P., Lee, D., Gray, A., Rehg, J.M.: Fast mean shift with accurate and stable convergence. In: Melia, M., Shen, X. (eds.) Artificial Intelligence and Statistics (2007) Google Scholar
- 47.Weiss, B.: Fast median and bilateral filtering. In: ACM Transactions on Graphics (Proc. SIGGRAPH ’06), pp. 519–526 (2006) Google Scholar
- 48.Winnemöller, H., Olsen, S.C., Gooch, B.: Real-time video abstraction. ACM Trans. Graph.
**25**(3), 1221–1226 (2006) CrossRefGoogle Scholar - 49.Witsenhausen, H.S.: Spiegelungsgruppen und aufzählung halbeinfacher liescher Ringe. Abh. Math. Sem. Univ. Hamburg
**14**(1941) Google Scholar - 50.Yang, C., Duraiswami, R., Gumerov, N.A., Davis, L.: Improved fast gauss transform and efficient kernel density estimation. In: Proc. International Conference on Computer Vision, vol. 1, pp. 664–671 (2003) CrossRefGoogle Scholar
- 51.Yang, Q., Yang, R., Davis, J., Nistér, D.: Spatial-depth super resolution for range images. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007) Google Scholar
- 52.Yaroslavsky, L.P.: Digital Picture Processing. An Introduction. Springer, Berlin (1985) zbMATHCrossRefGoogle Scholar