Bayesian Estimation of Layers from Multiple Images
When estimating foreground and background layers (or equivalently an alpha matte), it is often the case that pixel measurements contain mixed colours which are a combination of foreground and background. Object boundaries, especially at thin sub-pixel structures like hair, pose a serious problem.
In this paper we present a multiple view algorithm for computing the alpha matte. Using a Bayesian framework, we model each pixel as a combined sample from the foreground and background and compute a MAP estimate to factor the two. The novelties in this work include the incorporation of three different types of priors for enhancing the results in problematic scenes. The priors used are inequality constraints on colour and alpha values, spatial continuity, and the probability distribution of alpha values.
The combination of these priors result in accurate and visually satisfying estimates. We demonstrate the method on real image sequences with varying degrees of geometric and photometric complexity. The output enables virtual objects to be added between the foreground and background layers, and we give examples of this augmentation to the original sequences.
KeywordsGround Truth Bayesian Estimation Multiple Image Virtual Object Foreground Object
Unable to display preview. Download preview PDF.
- 1.C. Barron. Matte painting in the digital age. In Proc. SIGGRAPH (sketches), 1998. http://www.matteworld.com/projects/siggraph01.html.
- 2.P. Beardsley, P. Torr, and A. Zisserman. 3D model acquisition from extended image sequences. In Proc. ECCV, LNCS 1064/1065, pages 683–695. Springer-Verlag, 1996.Google Scholar
- 3.M. J. Black and P. Anandan. A framework for the robust estimation of optical flow. In Proc. ICCV, pages 231–236, 1993.Google Scholar
- 4.P. T. Boggs and J. W. Tolle. Sequential quadratic programming. In Acta Numerica, pages 1–51. 1995.Google Scholar
- 5.Yung-Yu Chuang, Brian Curless, David Salesin, and Richard Szeliski. A bayesian approach to digital matting. In CVPR 2001, 2000.Google Scholar
- 6.P. J. Huber. Robust Statistics. John Willey and Sons, 1981.Google Scholar
- 7.M. Irani, B. Rousso, and S. Peleg. Detecting and tracking multiple moving objects using temporal integration. In G. Sandini editor, Proc. ECCV, pages 282–287. Springer-Verlag, 1992.Google Scholar
- 8.H. H. Nagel. Extending the’ oriented smoothness constraint’ into the temporal domain and the estimation of derivatives of optical flow. In O. D. Faugeras, editor, Proc. ECCV, pages 139–148. Springer-Verlag, 1990.Google Scholar
- 9.S. Peleg and J. Herman. Panoramic mosaics by manifold projection. In Proc. CVPR, 1997.Google Scholar
- 10.P. Perez, A. Blake, and M. Gangnet. Jetstream: Probabilistic contour extraction with particles. In International Conference on Computer Vision, pages 424–531, 2001.Google Scholar
- 11.P. Perona and J. Malik. Scale-space and edge detection using anisotropic diffusion. IEEE PAMI, 12(7):629–639, Jul 1990.Google Scholar
- 12.M. Shizawa and K. Mase. A unified computational theory for motion transparency and motion boundaries based on eigenenergy analysis. In CVPR91, pages 289–295, 1991.Google Scholar
- 13.A. R. Smith and J. F. Blinn. Blue screen matting. In Proc. SIGGRAPH, pages 259–268, 1996.Google Scholar
- 14.R. Szeliski, S. Avidan, and P. Anandan. Layer extraction from multiple images containing reflections and transparency. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’2000), volume 1, pages 246–253, 2000.Google Scholar
- 15.R. Szeliski and P. Golland. Stereo matching with transparency and matting. In Sixth International Conference on Computer Vision (ICCV’98), pages 517–524, 1998.Google Scholar