Joint Spatial-Temporal Color Demosaicking

  • Xiaolin Wu
  • Lei Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3540)


Demosaicking of the color CCD data is a key to the image quality of digital still and video cameras. Limited by the Nyquist frequency of the color filter array (CFA), color artifacts often accompany high frequency contents in the reconstructed images. This paper presents a general approach of joint spatial-temporal color demosaicking that exploits all three forms of sample correlations: spatial, spectral, and temporal. By motion estimation and statistical data fusion between multiple estimates obtained from adjacent mosaic frames, the new approach can significantly outperform the existing spatial color demosaicking techniques both in objective measure and subjective visual quality.


Motion Vector Green Sample Green Channel Adjacent Frame Blue Channel 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Adams, J.E.: Intersections between color plane interpolation and other image processing functions in electronic photography. In: Proceedings of SPIE, vol. 2416, pp. 144–151 (1995)Google Scholar
  2. 2.
    Adams, J.E.: Design of practical color filter array interpolation algorithms for digital cameras. In: Proceedings of SPIE, vol. 3028, pp. 117–125 (1997)Google Scholar
  3. 3.
    Bayer, B.E.: Eastman Kodak Company, Color Imaging Array, US patent 3 971 065 (1975)Google Scholar
  4. 4.
    Bedford, R., Wyszecki, G.: Wavelength discrimination for point sources. Journal of the Optical Society of America 48, 129 (1958)CrossRefGoogle Scholar
  5. 5.
    Chang, E., Cheung, S., Pan, D.Y.: Color filter array recovery using a threshold-based variable number of gradients. In: Proceedings of SPIE, vol. 3650, pp. 36–43 (1999)Google Scholar
  6. 6.
    Cok, D.R.: Eastman Kodak Company, Signal processing method and apparatus for producing interpolated chrominance values in a sampled color image signal, US patent 4 642 678 (1987)Google Scholar
  7. 7.
    Dufaux, F., Moscheni, F.: Motion estimation techniques for digital TV: a review and a new contribution. In: Proc. of IEEE, June 1995, vol. 83, pp. 858–876 (1995)Google Scholar
  8. 8.
    Gunturk, B.K., Altunbasak, Y., Mersereau, R.M.: Color plane interpolation using alternating projections. IEEE Trans. Image Processing 11, 997–1013 (2002)CrossRefGoogle Scholar
  9. 9.
    Hamilton Jr., J.F., Adams, J.E.: Adaptive color plane interpolation in single sensor color electronic camera. U. S. Patent, 5 629 734 (1997)Google Scholar
  10. 10.
    Hirakawa, K., Parks, T.W.: Adaptive homogeneity-directed demosaicing algorithm. IEEE Trans. on Image Processing 14, 360–369 (2005)CrossRefGoogle Scholar
  11. 11.
    Kakarala, R., Baharav, Z.: Adaptive demosaicing with the principal vector method. IEEE Trans. Consumer Electronics 48, 932–937 (2002)CrossRefGoogle Scholar
  12. 12.
    Kay, S.M.: Fundamentals of Statistical Signal Processing, 1st edn., March 1993. Estimation Theory, Pearson Education, vol. I (1993)Google Scholar
  13. 13.
    Kimmel, R.: Demosaicing: Image reconstruction from CCD samples. IEEE Trans. Image Processing 8, 1221–1228 (1999)CrossRefGoogle Scholar
  14. 14.
    Kuhn, P.: Algorithms, Complexity Analysis and VLSI Architectures for MPEG-4 Motion Estimation. Kluwer Academic Publishers, Boston (1999)zbMATHGoogle Scholar
  15. 15.
    Longère, P., Zhang, X., Delahunt, P.B., Brainard, D.H.: Perceptual assessment of demosaicing algorithm performance. Proc. of IEEE 90, 123–132 (2002)CrossRefGoogle Scholar
  16. 16.
    Lu, W., Tan, Y.-p.: Color filter array demosaicking: new method and performance measures. IEEE Trans. Image Processing 12, 1194–1210 (2003)CrossRefGoogle Scholar
  17. 17.
    Lukac, R., Plataniotis, K.N.: Normalized color-ratio modelling for CFA interpolation. IEEE Trans. Consumer Electronics 50, 737–745 (2004)CrossRefGoogle Scholar
  18. 18.
    Lukac, R., Plataniotis, K.N., Hatzinakos, D., Aleksic, M.: A novel cost effective demosaicing approach. IEEE Transactions on Consumer Electronics 50(1), 256–261 (2004)CrossRefGoogle Scholar
  19. 19.
    Pei, S.C., Tam, I.K.: Effective color interpolation in CCD color filter arrays using signal correlation. IEEE Trans. Circuits and Systems for Video Technology 13, 503–513 (2003)CrossRefGoogle Scholar
  20. 20.
    Ramanath, R., Snyder, W.E.: Adaptive demosaicking. Journal of Electronic Imaging 12(4), 633–642 (2003)CrossRefGoogle Scholar
  21. 21.
    Schultz, R.R., Meng, L., Stevenson, R.L.: Subpixel motion estimation for super-resolution image sequence enhancement. Journal of Visual Communication and Image Representation 9, 38–50 (1998)CrossRefGoogle Scholar
  22. 22.
    Schultz, R.R., Stevenson, R.L.: Extraction of high-resolution frames from video sequences. IEEE Trans. Image Processing 5, 996–1011 (1996)CrossRefGoogle Scholar
  23. 23.
    Stiller, C., Konrad, J.: Estimating motion in image sequence. IEEE Signal Processing Magazine 7, 70–91 (1999)CrossRefGoogle Scholar
  24. 24.
    Tom, B.C., Katsaggelos, A.K.: Resolution enhancement of monochrome and color video using motion compensation. IEEE Trans. Image Processing 10, 278–287 (2001)zbMATHCrossRefGoogle Scholar
  25. 25.
    Trussel, H.J., Hartwing, R.E.: Mathematics for demosaicking. IEEE Trans. Image Processing 11, 485–492 (2002)CrossRefGoogle Scholar
  26. 26.
    Wu, X., Choi, W.K., Bao, P.: Color restoration from digital camera data by pattern matching. In: Proceedings of SPIE, vol. 3018, pp. 12–17 (1997)Google Scholar
  27. 27.
    Wu, X., Zhang, N.: Primary-consistent soft-decision color demosaicking for digital cameras. IEEE Trans. Image Processing 13, 1263–1274 (2004)CrossRefGoogle Scholar
  28. 28.
    Zhang, L., Wu, X.: Color demosaicking via directional linear minimum mean square-error estimation. IEEE Trans. Image Processing (to appear)Google Scholar
  29. 29.
    Zhang, L., Wu, X.: On cross correlation function based discrete time delay estimation. In: ICASSP 2005, Philadelphia, PA, USA (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Xiaolin Wu
    • 1
  • Lei Zhang
    • 1
  1. 1.Department of Electrical and Computer EngineeringMcMaster UniversityHamiltonCanada

Personalised recommendations