Analysis of Processing Pipelines in Digital Raw Cameras

  • Michael SchöberlEmail author
  • Joachim KeinertEmail author
  • Andre KaupEmail author
  • Siegfried FoesselEmail author


Traditionally, image and video processing algorithms start from an RGB image. However, current image sensors deliver camera raw data that needs additional processing and interpolation for conversion into an RGB representation. While recent research delivers important improvements in image quality for processing and reconstruction of raw images, these algorithms come with a heavy computational complexity. Consequently, they are not suited for mobile solutions such as cameras for media production, mobile phones or surveillance. Offline processing on the other hand offers both higher computational power and better flexibility and is well suited for executing those algorithms. The workflow for utilizing this enhanced quality thus requires a shift from camera centric imaging to new off-camera processing strategies. This requires a novel infrastructure for transportation and interchange and enables the possibility for development of even more sophisticated algorithms for processing of camera raw data. This contribution highlights the challenges and possibilities arising from the above mentioned paradigm shift. We discuss algorithms that should stay within the camera and algorithms that benefit from offloading. Our research contributes to the increase in image quality of workflows for future video applications.


Image Sensor Calibration Data Lossy Compression Color Filter Array Correlate Double Sampling 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    CinemaDNG Image Data Format Specification (2009)Google Scholar
  2. 2.
    Bayer B (1976) Color imaging array. US Patent 3,971,065Google Scholar
  3. 3.
    Doutre C, Nasiopoulos P, Plataniotis KN (2008) H.264-based compression of bayer pattern video sequences. IEEE Trans. on Circuits & Systems for Video Techn 18(6):725–734CrossRefGoogle Scholar
  4. 4.
    Fowler B, Gamal AE, Yang D, Tian H (1998) A Method for Estimating Quantum Efficiency for CMOS Image Sensors. In: SPIE Solid State Sensor Arrays: Development and Applications II, 3301:178–185CrossRefGoogle Scholar
  5. 5.
    Franzen R (2011) Kodak lossless true color image suite.
  6. 6.
    Gamal AE, Fowler B, Min H, Liu X (1998) Modeling and estimation of FPN components in CMOS image sensors. In: SPIE Solid State Sensor Arrays: Development and Applications II, 3301:168–177CrossRefGoogle Scholar
  7. 7.
    Gunturk B, Altunbasak Y, Mersereau R (2002) Color plane interpolation using alternating projections. IEEE Trans. on Image Proc 11(9):997–1013CrossRefGoogle Scholar
  8. 8.
    Hirakawa K, Parks T (2005) Adaptive homogeneity-directed demosaicing algorithm. IEEE Trans on Image Proc 14(3):360–369CrossRefGoogle Scholar
  9. 9.
    Kovac M (1975) Removal of Dark Current Spikes from Image Sensor Output Signals. US Patent 3,904,818Google Scholar
  10. 10.
    Levine P (1986) Adaptive defect correction for solid-state imagers. US Patent 4,600,946Google Scholar
  11. 11.
    Lian NX, Chang L, Zagorodnov V, Tan YP (2006) Reversing demosaicking and compression in color filter array image processing: Performance analysis and modeling. IEEE Trans on Image Proc 15(11):3261–3278CrossRefGoogle Scholar
  12. 12.
    Lukac R (2008) Single-Sensor Imaging: Methods and Applications for Digital Cameras. CRC Press, Inc, Boca Raton, FL, USACrossRefGoogle Scholar
  13. 13.
    Malueg R (1976) Detector Array Fixed-Pattern Noise Compensation. US Patent 3,949,162Google Scholar
  14. 14.
    Menon D, Calvagno G (2011) Color image demosaicking: An overview. Signal Processing: Image Communication. In PressGoogle Scholar
  15. 15.
    Moghadam A, Aghagolzadeh M, Kumar M, Radha H (2010) Compressive demosaicing. In: IEEE Int Workshop on Multimedia Signal Proc, pp 105–110Google Scholar
  16. 16.
    Pape D, Reiss W (1991) Defect correction apparatus for solid state imaging devices including inoperative pixel detection. US Patent 5,047,863Google Scholar
  17. 17.
    Pillman B, Guidash R, Kelly S (2006) Fixed Pattern Noise Removal in CMOS Imagers Across Various Operational Conditions. US Patent 7,092,017Google Scholar
  18. 18.
    Schöberl M, Fößel S, Kaup A (2010) Fixed Pattern Noise Column Drift Compensation (CDC) for Digital Moving Picture Cameras. In: IEEE Int Conf on Image Proc, pp 573–576Google Scholar
  19. 19.
    Schöberl M, Seiler J, Kasper B, Fößel S, Kaup A (2011) Sparsity-Based Defect Pixel Compensation for Arbitrary Camera Raw Images. In: IEEE Int Conf on Acoustic, Speech, & Signal Proc, pp 1257–1260Google Scholar
  20. 20.
    Schöberl M, Senel C, Fößel S, Bloss H, Kaup A (2009) Non-linear Dark Current Fixed Pattern Noise Compensation for Variable Frame Rate Moving Picture Cameras. In: Europ Signal Proc Conf, pp 268–272Google Scholar
  21. 21.
    Seiler J, Kaup A (2010) Complex-valued frequency selective extrapolation for fast image and video signal extrapolation. IEEE Signal Processing Letters 17(11):949–952CrossRefGoogle Scholar
  22. 22.
    Tanbakuchi A, van der Sijde A, Dillen B, Theuwissen A, de Haan W (2003) Adaptive pixel defect correction. In: Proc. SPIE Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications IV, 5017:360–370Google Scholar
  23. 23.
    Tian H, Fowler B, Gamal AE (1999) Analysis of temporal noise in CMOS APS. In: SPIE Sensors, Cameras, & Systems for Scientific/Industrial Applications, 3649:177–185Google Scholar
  24. 24.
    Wang S, Yao S, Faurie O, Shi Z (2009) Adaptive defect correction and noise suppression module in the CIS image processing system. In: Proc. SPIE Int. Symp. on Photoelectronic Detection & Imaging, 7384:73842VGoogle Scholar
  25. 25.
    White M, Lampe D, Blaha F, Mack I (1974) Characterization of surface channel CCD image arrays at low light levels. IEEE J of Solid-State Circuits 9(1):1–12CrossRefGoogle Scholar
  26. 26.
    Zhang N, Wu X (2006) Lossless compression of color mosaic images. IEEE Trans on Image Proc 15(6):1379–1388CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  1. 1.Chair of Multimedia Communications and Signal ProcessingUniversity of Erlangen-NürnbergErlangenGermany
  2. 2.Fraunhofer-Institut für Integrierte Schaltungen IISErlangenGermany

Personalised recommendations