Digital Camera Image Formation: Processing and Storage

  • Aaron Deever
  • Mrityunjay Kumar
  • Bruce Pillman


This chapter presents a high-level overview of image formation in a digital camera, highlighting aspects of potential interest in forensic applications. The discussion here focuses on image processing, especially processing steps related to concealing artifacts caused by camera hardware or that tend to create artifacts themselves. Image storage format issues are also discussed.


Discrete Cosine Transform Noise Reduction Point Spread Function Processing Chain Color Channel 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The authors gratefully acknowledge many helpful discussions with James E. Adams, Jr., in the development and review of this chapter. Ken Parulski also provided a great deal of assistance for the discussion of file formats.


  1. 1.
    Adams JE Jr, Hamilton JF Jr (2003) Removing chroma noise from digital images by using variable shape pixel neighborhood regions. U.S. Patent 6621937Google Scholar
  2. 2.
    Adams JE Jr, Hamilton JF Jr (2008) Single-sensor imaging: methods and applications for digital cameras. Chapter, digital camera image processing chain design, 1st edn. CRC Press, Boca Raton, pp 67–103.Google Scholar
  3. 3.
    Adams JE Jr, Hamilton JF Jr, Hamilton JA (2004) Removing color aliasing artifacts from color digital images. U.S. Patent 6804392Google Scholar
  4. 4.
    Adams JE Jr, Hamilton JF Jr, Smith CM (2005) Reducing color aliasing artifacts from digital color images. U.S. Patent 6927804Google Scholar
  5. 5.
    Adams JE Jr, Hamilton JF Jr, Williams FC (2007) Noise reduction in color digital images using pyramid decomposition. U.S. Patent 7257271Google Scholar
  6. 6.
    Adobe RGB (1998) Color image encoding. Technical report Adobe Systems, Inc., San Jose.
  7. 7.
    Alleysson D, Susstrunk S, Herault J (2005) Linear demosaicing inspired by the human visual system. IEEE Trans Image Process 14(4):439–449CrossRefGoogle Scholar
  8. 8.
    Anderson M, Motta R, Chandrasekar S, Stokes M (1995) Proposal for a standard default color space for the internet: sRGB. Fourth IS&T/SID color imaging conference, In, pp 238–245Google Scholar
  9. 9.
    Andrews HC, Hunt BR (1977) Digital image restoration. Prentice Hall, Englewood CliffsGoogle Scholar
  10. 10.
    Bae S, Paris S, Durand F (2006) Two-scale tone management for photographic look. ACM Trans. Graph. 25(3):637–645 (2006). Google Scholar
  11. 11.
    Baer R (2007) Method and apparatus for removing flicker from images. U.S. Patent 7298401Google Scholar
  12. 12.
    Bando Y, Nishita T (2007) Towards digital refocusing from a single photograph. In: Proceedings of the 15th Pacific conference on computer graphics and applications, pp 363–372.Google Scholar
  13. 13.
    Bauschke HH, Hamilton CH, Macklem MS, McMichael JS, Swart NR (2003) Recompression of JPEG images by requantization. IEEE Trans Image Process 12(7):843–849CrossRefGoogle Scholar
  14. 14.
    Bawolek E, Li Z, Smith R (1999) Magenta-white-yellow (MWY) color system for digital image sensor applications. U.S. Patent 5914749Google Scholar
  15. 15.
    Bayer BE (1976) Color imaging array. U.S. Patent 3971065Google Scholar
  16. 16.
    Bean J (2003) Cyan-magenta-yellow-blue color filter array. U.S. Patent 6628331Google Scholar
  17. 17.
    Chang S, Yu B, Vetterli M (2000) Adaptive wavelet thresholding for image denoising and compression. IEEE Trans Image Process 9(9):1532–1546MathSciNetMATHCrossRefGoogle Scholar
  18. 18.
    CIE publ. no. 15.2 (1986) Colorimetry. Techical report, CIEGoogle Scholar
  19. 19.
    Debevec PE, Malik J (1997) Recovering high dynamic range radiance maps from photographs. ACM SIGGRAPH, In, pp 369–378Google Scholar
  20. 20.
    Durand F, Dorsey J (2002) Fast bilateral filtering for the display of high-dynamic-range images. In: SIGGRAPH ’02: Proceedings of the 29th annual conference on computer graphics and interactive techniques, pp 257–266. ACM, New York, NY, USA.
  21. 21.
    Elad M, Feuer A (1997) Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Trans Image Process 6(12):1646–1658CrossRefGoogle Scholar
  22. 22.
    Farbman Z, Fattal R, Lischinski D, Szeliski R (2008) Edge-preserving decompositions for multi-scale tone and detail manipulation. In: SIGGRAPH ’08: ACM SIGGRAPH 2008 papers, pp 1–10. ACM, New York, NY, USA.
  23. 23.
    Farsiu S, Robinson MD, Elad M, Milanfar P (2004) Fast and robust multiframe super resolution. IEEE Trans Image Process 13(10):1327–1344CrossRefGoogle Scholar
  24. 24.
    Finlayson G, Hordley S, HubeL P (2001) Color by correlation: a simple, unifying framework for color constancy. IEEE Trans. Pattern Anal. Mach. Intell. 23(11):1209–1221CrossRefGoogle Scholar
  25. 25.
    Gindele E, Adams JE Jr, Hamilton JF Jr, Pillman BH (2007) Method for automatic white balance of digital images. U.S. Patent 7158174Google Scholar
  26. 26.
    Gindele EB, Gallagher AC (2001) Method for adjusting the tone scale of a digital image. U.S. Patent 6275605Google Scholar
  27. 27.
    Giorgianni EJ, Madden TE (2008) Digital color management encoding solutions, 2nd edn. Wiley, New YorkGoogle Scholar
  28. 28.
    Glotzbach J, Schafer R, Illgner K (2001) A method of color filter array interpolation with alias cancellation properties. In: Proceedings of the IEEE International Conference Image Processing, vol 1. pp 141–144Google Scholar
  29. 29.
    Gonzalez RC, Woods RE (2007) Digital image processing, 3rd edn. Prentice Hall, Englewood CliffsGoogle Scholar
  30. 30.
    Goodwin RM, Gallagher A (1998) Method and apparatus for areas selective exposure adjustment. U.S. Patent 5818975Google Scholar
  31. 31.
    Gunturk B, Glotzbach J, Altunbasak Y, Schafer R, Mersereau R (2005) Demosaicking: color filter array interpolation. IEEE Signal Process Mag 22(1):44–54CrossRefGoogle Scholar
  32. 32.
    Gupta M, Chen T (2001) Vector color filter array demosaicing. In: Proceedings of the SPIE, vol 4306. pp 374–382Google Scholar
  33. 33.
    Hamilton JF Jr (2004) Correcting for defects in a digital image taken by an image sensor caused by pre-existing defects in two pixels in adjacent columns of an image sensor. U.S. Patent No. 6741754Google Scholar
  34. 34.
    Hamilton JF Jr (2005) Correcting defect in a digital image caused by a pre-existing defect in a pixel of an image sensor. U.S. Patent No. 6900836Google Scholar
  35. 35.
    Hamilton JF Jr, Adams JE Jr (2003) Correcting for chrominance interpolation artifacts. U.S. Patent 6542187Google Scholar
  36. 36.
    Hamilton JF Jr, Adams JE Jr, Orlicki D (2001) Particular pattern of pixels for a color filter array which is used to derive luminanance and chrominance values. U.S. Patent 6330029B1Google Scholar
  37. 37.
    Hirakawa K, Parks T (2005) Adaptive homogeneity-directed demosaicing algorithm. IEEE Trans Image Process 14(3):360–369CrossRefGoogle Scholar
  38. 38.
    Hirakawa K, Wolfe P (2007) Spatio-spectral color filter array design for enhanced image fidelity. In: Proceedings of the ICIP, pp II-81-II-84Google Scholar
  39. 39.
    Huffman DA (1952) A method for the construction of minimum-redundancy codes. Proc Inst Radio Eng 40(9):1098–1101Google Scholar
  40. 40.
    Hunt R (1987) The reproduction of colour. Fountain Press, EnglandGoogle Scholar
  41. 41.
    ISO 12234-2:2001 (2001) Electronic still picture imaging—removable memory—part 2: TIFF/EP image data formatGoogle Scholar
  42. 42.
    ISO/IEC 14496-10:2003 (2003) Information technology—coding of audio-visual objects—part 10: advanced video codingGoogle Scholar
  43. 43.
    ISO/IEC 14496-2:1999 (1999) Information technology—coding of audio-visual objects—part 2: visualGoogle Scholar
  44. 44.
    Jain A, Ranganath S (1981) Application of two dimensional spectral estimation in image restoration. In: IEEE International Conference Acoust., Speech, Signal Process (ICASSP), pp 1113–1116Google Scholar
  45. 45.
    Jain C, Sethuraman S (2008) A low-complexity, motion-robust, spatio-temporally adaptive video de-noiser with in-loop noise estimation. Proceedings of the ICIP, In, pp 557–560Google Scholar
  46. 46.
    JEITA CP 3451 (2002) Exchangeable image file format for digital still cameras: Exif version 2.2Google Scholar
  47. 47.
    JEITA CP 3451-1 (2003) Amendment 1 exchangeable image file format for digital still cameras: Exif version 2.21 (amendment to version 2.2)Google Scholar
  48. 48.
    Kaplinsky M, Subbotin I (2006) Method for mismatch detection between the frequency of illumination source and the duration of optical integration time for image with rolling shutter. U.S. Patent 7142234Google Scholar
  49. 49.
    Kim HS, Kim BG, Kim CY, Joo YH (2008) Digital photographing apparatus and method for detecting and correcting smear by using the same. U.S. Patent publication no. 2008/0084488 A1Google Scholar
  50. 50.
    Kim YJ, Oh JJ, Choi BT, Choi SJ, Kim ET (2009) Robust noise reduction using a hierarchical motion compensation in noisy image sequences. In Digest of Technical Papers of the ICCE, pp 1–2Google Scholar
  51. 51.
    Kimmel R (1999) Demosaicing: image reconstruction from color CCD samples. IEEE Trans Image Process 8(9):1221–1228CrossRefGoogle Scholar
  52. 52.
    Kondo K (2009) Image sensing apparatus, image sensing method, and program. U.S. Patent 7545420Google Scholar
  53. 53.
    Kumar M, Morales E, Adams JE Jr, Hao W (2009) New digital camera sensor architecture for low light imaging. Proceedings of the ICIP, In, pp 2681–2684Google Scholar
  54. 54.
    Kumar M, Ramuhalli P (2005) Dynamic programming based multichannel image restoration. In: IEEE International Conference Acoustics, Speech, Signal Process (ICASSP), vol 2. pp 609–612Google Scholar
  55. 55.
    Kundur D, Hatzinakos D (1996) Blind image deconvolution. IEEE Signal Process Mag 13(3):43–64CrossRefGoogle Scholar
  56. 56.
    Lee H (1986) Method for computing the scene-illuminant chromaticity from specular highlights. J Opt Soc Am A 3:1694–1699CrossRefGoogle Scholar
  57. 57.
    Lee J (1983) Digital image smoothing and the sigma filter. Comput Vis Graph Image Process 24:255–269CrossRefGoogle Scholar
  58. 58.
    Li X, Gunturk B, Zhang L (2008) Image demosaicing: a systematic survey. In: SPIE Conference on VCIP, vol 6822, pp. 68, 221J-68, 221J-15Google Scholar
  59. 59.
    Liu X, Gamal AE (2001) Simultaneous image formation and motion blur restoration via multiple capture. In: IEEE International conference acoustics, speech, signal process (ICASSP), vol 3. pp 1841–1844Google Scholar
  60. 60.
    Lu W, Tan YP (2003) Color filter array demosaicking: new method and performance measures. IEEE Trans Image Process 12(10):1194–1210CrossRefGoogle Scholar
  61. 61.
    Lukac R, Plataniotis K (2005) Color filter arrays: design and performance analysis. IEEE Trans Consum Electron 51(4):1260–1267CrossRefGoogle Scholar
  62. 62.
    McCann JJ, McKee SP, Taylor TH (1976) Quantitative studies in retinex theory. Vis Res 16:445–458Google Scholar
  63. 63.
    Miao L, Qi H, Snyder W (2004) A generic method for generating multispectral filter arrays. In: Proceedings of the ICIP, vol 5. pp 3343–3346Google Scholar
  64. 64.
    Miyano T (1997) Auto white adjusting device. U.S. Patent 5659357Google Scholar
  65. 65.
    Miyano T, Shimizu E (1997) Automatic white balance adjusting device. U.S. Patent 5644358Google Scholar
  66. 66.
    Moghadam A, Aghagolzadeh M, Radha H, Kumar M (2010) Compressive demosaicing. In: Proceedings of the MMSP, pp 105–110Google Scholar
  67. 67.
    Mukherjee J, Parthasarathi R, Goyal S (2001) Markov random field processing for color demosaicing. Pattern Recognit. Lett. 22(3–4):339–351MATHCrossRefGoogle Scholar
  68. 68.
    Multiple output sensors seams correction (2009) Technical report application note revision 1.0 MTD/PS-1149, Eastman Kodak image sensor solutionsGoogle Scholar
  69. 69.
    Ng MK, Bose NK (2003) Mathematical analysis of super-resolution methodology. IEEE Signal Process Mag 20(3):62–74CrossRefGoogle Scholar
  70. 70.
    Ohta YI, Kanade T, Sakai T (1980) Color information for region segmentation. Comput Graph Image Process 13(3):222–241CrossRefGoogle Scholar
  71. 71.
    Okura Y, Tanizoe Y, Miyake T (2009) CCD signal processing device and image sensing device. U.S. Patent publication no. 2009/0147108 A1Google Scholar
  72. 72.
    Park SG, Park MK, Kang MG (2003) Super-resolution image reconstruction: a technical overview. IEEE Signal Process Mag 20(3):21–36CrossRefGoogle Scholar
  73. 73.
    Parulski KA, Reisch R (2008) Single-sensor imaging: methods and applications for digital cameras. Chapter, digital camera image storage formats, 1st edn. CRC press, Boca Raton, pp 351–379Google Scholar
  74. 74.
    Patti AJ, Sezan MJ, Tekalp AM (1997) Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Trans Image Process 6(8):1064–1076CrossRefGoogle Scholar
  75. 75.
    Pennebaker WB (1993) Mitchell JL (1993) JPEG still image data compression standard. Van Nostrand Reinhold, New YorkGoogle Scholar
  76. 76.
    Petschnigg G, Szeliski R, Agrawala M, Cohen M, Hoppe H, Toyama K (2004) Digital photography with flash and no-flash image pairs. In: Proceedings of the ACM SIGGRAPH, vol 23. pp 664–672Google Scholar
  77. 77.
    Pillman B, Deever A, Kumar M (2010) Flexible readout image capture with a four-channel CFA. In: Proceedings of the ICIP, pp 561–564Google Scholar
  78. 78.
    Poplin D (2006) An automatic flicker detection method for embedded camera systems. IEEE Trans Consum Electron 52(2):308–311CrossRefGoogle Scholar
  79. 79.
    Raskar R, Agrawal A, Tumblin J (2006) Coded exposure photography: motion deblurring using fluttered shutter. ACM Trans Graph 25:795–804CrossRefGoogle Scholar
  80. 80.
    Rav-Acha A, Peleg S (2005) Two motion-blurred images are better than one. Pattern Recognit Lett 26(3):311–317CrossRefGoogle Scholar
  81. 81.
    Reference output medium metric RGB color space (ROMM RGB) white paper (1999) Technical report version 2.2, accession number 324122H, Eastman Kodak CompanyGoogle Scholar
  82. 82.
    Roddy J, Zolla R, Blish N, Horvath L (2006) Four color image sensing apparatus. U.S. Patent 7057654Google Scholar
  83. 83.
    Shan Q, Jia J, Agarwala A (2008) High-quality motion deblurring from a single image. ACM Trans Graph 27:1–10Google Scholar
  84. 84.
    Smith WJ (1966) Modern optical engineering. McGraw-Hill, San FranciscoGoogle Scholar
  85. 85.
    Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: Proceedings of the 1998 IEEE international conference on computer vision. IEEEGoogle Scholar
  86. 86.
    Triantafyllidis GA, Tzovaras D, Strintzis MG (2002) Blocking artifact detection and reduction in compressed data. IEEE Trans Circuits Syst Video Technol 12(10):877–890CrossRefGoogle Scholar
  87. 87.
    Trussell H, Hartwig R (2002) Mathematics for demosaicking. IEEE Trans Image Process 11(4):485–492MathSciNetCrossRefGoogle Scholar
  88. 88.
    Yamagami T, Sasaki T, Suga A (1994) Image signal processing apparatus having a color filter with offset luminance filter elements. U.S. Patent 5323233Google Scholar
  89. 89.
    Yamanaka S (1997) Solid state camera. U.S. Patent 4054906Google Scholar
  90. 90.
    Yoshida H (2004) Image pickup apparatus and method of correcting deteriorated pixel signal thereof. U.S. Patent 6809763Google Scholar
  91. 91.
    Yu W (2003) An embedded camera lens distortion correction method for mobile computing applications. IEEE Trans Consum Electron 49(4):894–901. doi: 10.1109/TCE.2003.1261171 CrossRefGoogle Scholar
  92. 92.
    Yuan L, Sun J, Quan L, Shum HY (2007) Image deblurring with blurred/noisy image pairs. ACM Trans Graph 26:1–10MATHCrossRefGoogle Scholar
  93. 93.
    Zhang F, Wu X, Yang X, Zhang W, Zhang L (2009) Robust color demosaicking with adaptation to varying spectral correlations. IEEE Trans Image Process 18(12):2706–2717MathSciNetCrossRefGoogle Scholar
  94. 94.
    Zhang L, Wu X (2005) Color demosaicking via directional linear minimum mean square-error estimation. IEEE Trans Image Process 14(12):2167–2178CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Corporate Research and EngineeringEastman Kodak CompanyRochesterUSA

Personalised recommendations