Video and Image Bayesian Demosaicing with a Two Color Image Prior

  • Eric P. Bennett
  • Matthew Uyttendaele
  • C. Lawrence Zitnick
  • Richard Szeliski
  • Sing Bing Kang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3951)


The demosaicing process converts single-CCD color representations of one color channel per pixel into full per-pixel RGB. We introduce a Bayesian technique for demosaicing Bayer color filter array patterns that is based on a statistically-obtained two color per-pixel image prior. By modeling all local color behavior as a linear combination of two fully specified RGB triples, we avoid color fringing artifacts while preserving sharp edges. Our grid-less, floating-point pixel location architecture can process both single images and multiple images from video within the same framework, with multiple images providing denser color samples and therefore better color reproduction with reduced aliasing. An initial clustering is performed to determine the underlying local two color model surrounding each pixel. Using a product of Gaussians statistical model, the underlying linear blending ratio of the two representative colors at each pixel is estimated, while simultaneously providing noise reduction. Finally, we show that by sampling the image model at a finer resolution than the source images during reconstruction, our continuous demosaicing technique can super-resolve in a single step.


Color Image Color Channel Multiple Image Bayer Color Bilinear Interpolation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bayer, B.: Color imaging array. U.S. Patent No. 3,971,065 (1976)Google Scholar
  2. 2.
    Cok, D.: Signal processing method and apparatus for producing interpolated chrominance values in a sampled color image signal. U.S. Patent No. 5,373,322 (1987)Google Scholar
  3. 3.
    Freeman, W.: Median filter for reconstructing missing color samples. U.S. Patent No 4,724,395 (1988)Google Scholar
  4. 4.
    Ramanath, R., Snyder, W., Bilbro, G., Sander, W.: Demosaicking methods for bayer color arrays. J. of Electronic Imaging 11(3), 306–315 (2002)CrossRefGoogle Scholar
  5. 5.
    Gupta, M., Chen, T.: Vector color filter array demosaicing. In: Procs. of SPIE, Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Apps. II., vol. 4306, pp. 374–382 (2001)Google Scholar
  6. 6.
    Laroche, C., Prescott, M.: Apparatus and method for adaptively interpolating a full color image utilizing chrominance gradients. U.S. Patent No. 5,629,734 (1994)Google Scholar
  7. 7.
    Hamilton, J., Adams, J.: Adaptive color plane interpolation in single sensor color electronic camera. U.S. Patent No. 5,629,734 (1997)Google Scholar
  8. 8.
    Chang, E., Cheung, S., Pan, D.: Color filter array recovery using a threshold-based variable number of gradients. In: Procs. of SPIE/IS and T EI., vol. 3650 (1999)Google Scholar
  9. 9.
    Kimmel, R.: Demosaicing, image reconstruction from color CCD samples. IEEE Trans. on Image Processing 8, 1221–1228 (1999)CrossRefGoogle Scholar
  10. 10.
    Malvar, H., He, L., Cutler, R.: High-quality linear interpolation for demosaicing of bayer-patterned color images. In: Procs. of ICASSP (2004)Google Scholar
  11. 11.
    Eastman Kodak Company: PhotoCD PCD0992,
  12. 12.
    Nayar, S.K., Narasimhan, S.G.: Assorted pixels: Multi-sampled imaging with structural models. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 636–652. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  13. 13.
    Hel-Or, Y.: The canonical correlations of color images and their use for demosaicing. Technical Report HPL-2003-164R1, Hewlett Packard Labs. Israel (2004)Google Scholar
  14. 14.
    Brainard, D.: Bayesian method for reconstructing color images from trichromatic samples. In: Soc. for Imaging Science Technology Conf., pp. 375–380 (1995)Google Scholar
  15. 15.
    Chuang, Y., Curless, B., Salesin, D., Szeliski, R.: A bayesian approach to digital matting. In: CVPR, vol. 2, pp. 264–271 (2001)Google Scholar
  16. 16.
    Zomet, A., Peleg, S.: Multi-sensor super-resolution. In: Procs. IEEE Workshop on Apps. of Computer Vision, pp. 27–31 (2002)Google Scholar
  17. 17.
    Freeman, W., Jones, T., Pasztor, E.: Example-based super-resolution. IEEE Computer Graphics and Applications 22(2), 56–65 (2002)CrossRefGoogle Scholar
  18. 18.
    Fung, J., Mann, S.: Projective demosaicing using multiple overlapping images. In: Procs. of the Int’l Symp. on Intelligent Multimedia, Video, and Speech Processing, pp. 190–193 (2004)Google Scholar
  19. 19.
    Gotoh, T., Okutomi, M.: Direct super-resolution and registration using raw CFA images. In: CVPR, vol. 2, pp. 600–607 (2004)Google Scholar
  20. 20.
    Zhang, X., Wandell, B.: A spatial extension of CIELAB for digital color image reproduction. In: Procs. of Soc. For Information Display, pp. 731–734 (1996)Google Scholar
  21. 21.
    Zhang, X., Wandell, B.: Color image fidelity metrics evaluated using image distortion maps. Signal Processing 70(3), 201–214 (1998)CrossRefMATHGoogle Scholar
  22. 22.
    Fairchild, M., Johnson, G.: The iCAM framework for image appearance, image differences, and image quality. J. of Electronic Imaging 13, 126–138 (2004)CrossRefGoogle Scholar
  23. 23.
    Pattanaik, S., Ferwerda, J., Fairchild, M., Greenberg, D.: A multiscale model of adaptation and spatial vision for realistic image display. In: ACM SIGGRAPH, pp. 287–298 (1998)Google Scholar
  24. 24.
    Daly, S.: The Visible Differences Predictor: An Algorithm for the Assessment of Image Fidelity. In: Digital Images and Computer Vision, pp. 179–206. MIT Press, Cambridge (1993)Google Scholar
  25. 25.
    Ramasubramanian, M., Pattanaik, S., Greenberg, D.: A perceptually based physical error metric for realistic image synthesis. In: ACM SIGGRAPH, pp. 73–82 (1999)Google Scholar
  26. 26.
    Walter, B., Pattanaik, S., Greenberg, D.: Using perceptual texture masking for efficient image synthesis. In: Procs. of Eurographics, pp. 393–400 (2002)Google Scholar
  27. 27.
    Lee, A., Mumford, D., Huang, J.: Occlusion models for natural images: A statistical study of a scale-invariant dead leaves model. IJCV 41, 35–59 (2001)CrossRefMATHGoogle Scholar
  28. 28.
    Lin, Z., Shum, H.: Fundamental limits of reconstruction-based superresolution algorithms under local translation. IEEE Trans. PAMI 26(1), 83–97 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Eric P. Bennett
    • 1
  • Matthew Uyttendaele
    • 2
  • C. Lawrence Zitnick
    • 2
  • Richard Szeliski
    • 2
  • Sing Bing Kang
    • 2
  1. 1.The University of North Carolina at Chapel HillChapel HillUSA
  2. 2.Microsoft ResearchRedmondUSA

Personalised recommendations