International Journal of Computer Vision

, Volume 81, Issue 1, pp 24–52 | Cite as

A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach

Article

Abstract

The bilateral filter is a nonlinear filter that smoothes a signal while preserving strong edges. It has demonstrated great effectiveness for a variety of problems in computer vision and computer graphics, and fast versions have been proposed. Unfortunately, little is known about the accuracy of such accelerations. In this paper, we propose a new signal-processing analysis of the bilateral filter which complements the recent studies that analyzed it as a PDE or as a robust statistical estimator. The key to our analysis is to express the filter in a higher-dimensional space where the signal intensity is added to the original domain dimensions. Importantly, this signal-processing perspective allows us to develop a novel bilateral filtering acceleration using downsampling in space and intensity. This affords a principled expression of accuracy in terms of bandwidth and sampling. The bilateral filter can be expressed as linear convolutions in this augmented space followed by two simple nonlinearities. This allows us to derive criteria for downsampling the key operations and achieving important acceleration of the bilateral filter. We show that, for the same running time, our method is more accurate than previous acceleration techniques. Typically, we are able to process a 2 megapixel image using our acceleration technique in less than a second, and have the result be visually similar to the exact computation that takes several tens of minutes. The acceleration is most effective with large spatial kernels. Furthermore, this approach extends naturally to color images and cross bilateral filtering.

Keywords

Image processing Bilateral filter Cross bilateral filter Color image filtering 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adalsteinsson, D., & Sethian, J. A. (1995). A fast level set method for propagating interfaces. Journal of Computational Physics, 118, 269–277. MATHCrossRefMathSciNetGoogle Scholar
  2. Aurich, V., & Weule, J. (1995). Non-linear Gaussian filters performing edge preserving diffusion. In Proceedings of the DAGM symposium. Google Scholar
  3. Bae, S., Paris, S., & Durand, F. (2006). Two-scale tone management for photographic look. ACM Transactions on Graphics, 250(3), 637–645. Proceedings of the ACM SIGGRAPH conference. CrossRefGoogle Scholar
  4. Barash, D. (2002). A fundamental relationship between bilateral filtering, adaptive smoothing and the nonlinear diffusion equation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(6), 844. CrossRefGoogle Scholar
  5. Barash, D., Schlick, T., Israeli, M., & Kimmel, R. (2003). Multiplicative operator splittings in non-linear diffusion: from spatial splitting to multiplicative timesteps. Journal of Mathematical Imaging and Vision, 19, 33–48. MATHCrossRefMathSciNetGoogle Scholar
  6. Bennett, E. P., & McMillan, L. (2005). Video enhancement using per-pixel virtual exposures. ACM Transactions on Graphics, 24, 845–852. Proceedings of the ACM SIGGRAPH conference. CrossRefGoogle Scholar
  7. Black, M. J., Sapiro, G., Marimont, D. H., & Heeger, D. (1998). Robust anisotropic diffusion. IEEE Transactions on Image Processing, 7(3), 421–432. CrossRefGoogle Scholar
  8. Blinn, J. F. (1996). Fun with premultiplied alpha. IEEE Computer Graphics and Applications, 16(5), 86–89. CrossRefGoogle Scholar
  9. Buades, A., Coll, B., & Morel, J.-M. (2005). Neighborhood filters and PDE’s (Technical Report 2005-04). CMLA. Google Scholar
  10. Chen, J., Paris, S., & Durand, F. (2007). Real-time edge-aware image processing with the bilateral grid. ACM Transactions on Graphics 26(3). Proceedings of the ACM SIGGRAPH conference. Google Scholar
  11. Choudhury, P., & Tumblin, J. E. (2003). The trilateral filter for high contrast images and meshes. In Proceedings of the Eurographics symposium on rendering. Google Scholar
  12. Durand, F., & Dorsey, J. (2002). Fast bilateral filtering for the display of high-dynamic-range images. ACM Transactions on Graphics, 21(3), 257–266. Proceedings of the ACM SIGGRAPH conference. CrossRefGoogle Scholar
  13. Eisemann, E., & Durand, F. (2004). Flash photography enhancement via intrinsic relighting. ACM Transactions on Graphics, 23(3), 673–678. Proceedings of the ACM SIGGRAPH conference. CrossRefGoogle Scholar
  14. Elad, M. (2002). On the bilateral filter and ways to improve it. IEEE Transactions on Image Processing, 11(10), 1141–1151. CrossRefMathSciNetGoogle Scholar
  15. Elad, M. (2005). Retinex by two bilateral filters. In Proceedings of the scale-space conference. Google Scholar
  16. Felsberg, M., Forssén, P.-E., & Scharr, H. (2006). Channel smoothing: Efficient robust smoothing of low-level signal features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(2), 209–222. CrossRefGoogle Scholar
  17. Fleishman, S., Drori, I., & Cohen-Or, D. (2003). Bilateral mesh denoising. ACM Transactions on Graphics, 22(3), 950–953. Proceedings of the ACM SIGGRAPH conference. CrossRefGoogle Scholar
  18. Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. M., & Stahel, W. A. (1986). Robust statistics—the approach based on influence functions. New York: Wiley–Interscience. ISBN 0-471-73577-9. MATHGoogle Scholar
  19. Huber, P. J. (1981). Robust statistics. Probability and statistics. New York: Wiley–Interscience. Google Scholar
  20. Jones, T. R., Durand, F., & Desbrun, M. (2003). Non-iterative, feature-preserving mesh smoothing. ACM Transactions on Graphics, 22(3), 943–949. Proceedings of the ACM SIGGRAPH conference. CrossRefGoogle Scholar
  21. Koenderink, J. J., & van Doorn, A. J. (1999). The structure of locally orderless images. International Journal of Computer Vision, 31(2–3), 159–168. CrossRefGoogle Scholar
  22. Liu, C., Freeman, T., Szeliski, R., & Kang, S. (2006). Noise estimation from a single image. In Proceedings of the computer vision and pattern recognition conference. New York: IEEE Press. Google Scholar
  23. Margulis, D. (2005). Photoshop LAB color: The canyon conundrum and other adventures in the most powerful colorspace. Berkeley: Peachpit. ISBN: 0321356780. Google Scholar
  24. Mrázek, P., Weickert, J., & Bruhn, A. (2006). Geometric properties from incomplete data. In On robust estimation and smoothing with spatial and tonal kernels. Berlin: Springer. Google Scholar
  25. Oh, B. M., Chen, M., Dorsey, J., & Durand, F. (2001) Image-based modeling and photo editing. In Proceedings of the ACM SIGGRAPH conference. New York: ACM. Google Scholar
  26. Osher, S., & Sethian, J. A. (1988). Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations. Journal of Computational Physics, 79, 12–49. MATHCrossRefMathSciNetGoogle Scholar
  27. Paris, S., & Durand, F. (2006). A fast approximation of the bilateral filter using a signal processing approach. In Proceedings of the European conference on computer vision. Google Scholar
  28. Paris, S., Briceño, H., & Sillion, F. (2004). Capture of hair geometry from multiple images. ACM Transactions on Graphics, 23(3), 712–719. Proceedings of the ACM SIGGRAPH conference. CrossRefGoogle Scholar
  29. Petschnigg, G., Agrawala, M., Hoppe, H., Szeliski, R., Cohen, M., & Toyama, K. (2004). Digital photography with flash and no-flash image pairs. ACM Transactions on Graphics, 23(3), 664–672. Proceedings of the ACM SIGGRAPH conference. CrossRefGoogle Scholar
  30. Pham, T. Q. (2006). Spatiotonal adaptivity in super-resolution of undersampled image sequences. Ph.D. thesis, Delft University of Technology. Google Scholar
  31. Pham, T. Q., & van Vliet, L. J. (2005). Separable bilateral filtering for fast video preprocessing. In International conference on multimedia and expo. New York: IEEE Press. Google Scholar
  32. Porter, T., & Duff, T. (1984). Compositing digital images. Computer Graphics, 18(3), 253–259. CrossRefGoogle Scholar
  33. Sand, P., & Teller, S. (2006). Particle video: Long-range motion estimation using point trajectories. In Proceedings of the computer vision and pattern recognition conference. Google Scholar
  34. Shannon, C. E. (1949). Communication in the presence of noise. Proceedings of the Institute of Radio Engineers, 37(1), 10–21. MathSciNetGoogle Scholar
  35. Smith, S. (2002). Digital signal processing. London: Newnes. ISBN: 075067444X. Google Scholar
  36. Smith, S. M., & Brady, J. M. (1997). SUSAN—a new approach to low level image processing. International Journal of Computer Vision, 23(1), 45–78. CrossRefGoogle Scholar
  37. Sochen, N., Kimmel, R., & Malladi, R. (1998). A general framework for low level vision. IEEE Transactions in Image Processing, 7, 310–318. MATHCrossRefMathSciNetGoogle Scholar
  38. Sochen, N., Kimmel, R., & Bruckstein, A. M. (2001). Diffusions and confusions in signal and image processing. Journal of Mathematical Imaging and Vision, 14(3), 237–244. CrossRefMathSciNetGoogle Scholar
  39. Tomasi, C., & Manduchi, R. (1998). Bilateral filtering for gray and color images. In Proceedings of the international conference on computer vision (pp. 839–846). New York: IEEE Press. Google Scholar
  40. van de Weijer, J., & van den Boomgaard, R. (2001). Local mode filtering. In Proceedings of the conference on computer vision and pattern recognition. Google Scholar
  41. Weickert, J., ter Haar Romeny, B. M., & Viergever, M. A. (1998). Efficient and reliable schemes for nonlinear diffusion filtering. IEEE Transactions on Image Processing, 7, 398–410. CrossRefGoogle Scholar
  42. Weiss, B. (2006). Fast median and bilateral filtering. ACM Transactions on Graphics, 25(3), 519–526. Proceedings of the ACM SIGGRAPH conference. CrossRefGoogle Scholar
  43. Willis, P. J. (2006). Projective alpha colour. Computer Graphics Forum, 25(3), 557–566. Proceedings of the Eurographics conference. CrossRefMathSciNetGoogle Scholar
  44. Winnemöller, H., Olsen, S. C., & Gooch, B. (2006). Real-time video abstraction. ACM Transactions on Graphics, 25(3), 1221–1226. Proceedings of the ACM SIGGRAPH conference. CrossRefGoogle Scholar
  45. Wong, W. C. K., Chung, A. C. S., & Yu, S. C. H. (2004). Trilateral filtering for biomedical images. In Proceedings of the international symposium on biomedical imaging. New York: IEEE Press. Google Scholar
  46. Xiao, J., Cheng, H., Sawhney, H., Rao, C., & Isnardi, M. (2006). Bilateral filtering-based optical flow estimation with occlusion detection. In Proceedings of the European conference on computer vision. Google Scholar
  47. Yaroslavsky, L. P. (1985). Digital picture processing. An introduction. Berlin: Springer. MATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Computer Science and Artificial Intelligence LaboratoryMassachusetts Institute of TechnologyCambridgeUSA

Personalised recommendations