On Wide Dynamic Range Tone Mapping CMOS Image Sensor

Abstract

The dynamic range of a natural scene often covers over 6 decades of intensity from bright to dark areas. Typical image sensors, however, have limited ability to capture this dynamic range available in nature. Even after designing specific wide dynamic range (WDR) image sensors, displaying them on conventional media with limited ability requires computationally complex tone mapping. This paper proposed a novel CMOS pixel which can capture and perform tone mapping during data acquisition. The pixel requires a reference voltage to generate tone mapped response. A number of different reference signals are proposed and generated which can perform WDR operation. Nevertheless, fixed pattern noise (FPN) effects the performance of these pixel. A pixel model with simple parameter extraction procedure is described for a typical tone mapping operator. This model is then used to obtain a simple procedure for pixel calibration leading to reduced FPN. The new proposed pixel response is able to capture upto 6 decades of light intensity and reported FPN correction procedure produces 1% of FPN contrast error.

1 Introduction

CMOS image sensors are widely used in a number of instrumentation and scientific applications including cameras, mobile phones, auto-mobiles, security and biometrics [1]. However, most CMOS imagers are unable to capture the high dynamic range (HDR) available in nature and as would be visible by the human eye. The eyes can capture more than 6 decades of light intensities available in nature, which compares to 3–4 decades captured by most linear image sensors. This leads to generally disappointing, but often disastrous results when imaging high intensity scenes. A range of different pixel architectures have been suggested to overcome the low dynamic range of typical linear active pixel sensors. These include threshold comparing [2], well capacity adjustment [3], integration time control [4, 5], dual and/or multi-sampling architectures, linear-logarithmic pixel [6], logarithmic pixels [7] etc. Threshold comparing pixels integrate the photocurrent till the output reaches a known threshold value [2]. For example, Hyneceks’ pixel provides a provision for resetting the pixel only when the accumulated charge exceeds a certain threshold voltage and ignoring the reset at all other instances [2]. The number of resets can be stored in local memory inside each pixel. However, the pixel suffers due to a large number of compacters and memories required inside each pixel, thereby reducing the pixel fill factor and quantum efficiency.

Another way to increase the dynamic range of image sensor is to enhance the well capacity by changing the reset signal during the integration period. Decker and co-workers had proposed the similar technique to change the reset signal at a rate increasing with time [3]. This change in reset voltage changes the pixel charge. By increasing the pixel operating voltage, one can also adjust the well capacity. However, this will lead to increasing power consumption.

Integration time control technique can be used to capture wide dynamic range information by locally adapting the integration time for each pixel or group of pixels [4, 5]. One may then continuously compare the pixel output to one or many references [5]. However, these sensors suffer from high fixed pattern noise due to the mismatch of circuit elements as well as integration time mismatch between pixels.

Logarithmic pixel [7] provides another technique to capture wide dynamic range by operating the NMOS reset transistor of a classical 3-transistor image sensor in the weak inversion region with its gate shortened to the drain. This transistor then acts as logarithmic amplifier, which compresses the input photocurrent and converts into logarithmic voltage. However, they too suffer from high fixed pattern noise.

However, unless an image is compressed at the acquisition stage, like the logarithmic pixel, these HDR techniques often require more powerful data converters to get more number of bits per pixel and/or additional memory to store more than 1 frame per pixel. In addition, they often require post-processing to extract the dynamic range and reduce noise. Even when a digital camera can capture high dynamic range of visible information, it cannot be displayed on typical media, including screens, projectors or even paper. This is so as these media also have a limited dynamic range (LDR) of 2-3 decades [8].

In order to display high dynamic range images on a low dynamic range display, Tone Mapping algorithms are generally used to compress and fit the dynamic range of the image [8,9,10,11]. Once again, there is a large literature on different operators for tone mapping, each with its own advantages and disadvantages [12,13,14]. Some of them aim to retain key characteristics of real image such as contrast or details to be as close as possible to the original image. Others seek to achieve what would have been observed by a typical human observer. Irrespective of their nature, these algorithms require complex mathematical operations on the response of each pixel, which in turn makes them difficult to implement in real time and on low end imaging systems like mobile phones or in-field instrumentation systems. Hence, the current chain of imaging (which includes a high end-imager, high end-ADC, storage and significant processing) needs to be improved by reducing the required processing and improving the pixel design [15].

In this paper, a novel threshold comparing pixel is proposed to capture as well as display tone mapped images. Threshold comparing pixels have been proposed in literature primarily to act as logarithmic pixels or in time based pixel acquisition techniques neural pixels [16,17,18]. Using a comparative ramp or monotonically increasing reference, one can convert any of these pixels to produce tone mapped images. The new proposed pixel has an ability to capture dynamic range of upto 6 decades of light intensity. However, such images will suffer from fixed pattern noise (FPN), as typical FPN reducing mechanisms like correlated double sampling is generally not usable. FPN owes its origins to variations between the responses of individual pixels within an array and appears as a salt and pepper noise pattern on the image. In this paper, a simple procedure is also being proposed to produce an FPN free tone mapped image. While the results will be shown for a typical comparing pixel, the analysis is valid for any other voltage comparing pixel and can be adopted in other architectures.

Section 2 of the paper presents the design of the voltage comparing WDR pixel. Section 3 discusses how one can obtain any tone mapped response from such pixels. For a typical tone mapping operator, Sect.  4 then presents a pixel model, which can be used for fixed pattern noise correction. Section 5 presents fixed pattern noise simulation and correction technique. Section 6 presents the conclusion.

2 Wide Dynamic Range Pixel

A WDR pixel [19,20,21] which uses NMOS only transistors and can produce a tone mapped response is shown in Fig. 1a. It is similar to typical active pixel sensor with an addition of a comparative switch M2. The other transistors M1, M3 and M4 are reset device, source follower and row select switch. At start of a frame, transistor M1 is switched on by pulling the $V_{rst}$ signal high. This leads to a high voltage, typically around $V_{DD}-V{T,M1}$, on the diode node N1. After a brief reset period, this transistor is switched off by pulling $V_{rst}$ low. Simultaneously, a reference voltage, $V_{ref}$ is applied at the drain of M2. This reference signal is monotonically increasing. Initially, the gate of M2 is high compared to its source and drain. This means that this transistor is on and hence its source voltage which is also the gate voltage of M3 follows the reference voltage, $V_{ref}$. The light falling on the diode linearly discharges the voltage at node N1. Simultaneously, the reference voltage is increased with time. This means that at some point, the gate voltage of the transistor, M2 will become low enough to switch this transistor off. Its source will then not be able to track its drain voltage and the gate capacitance of M3, provided it is suitably designed, will hold this voltage level till the end of the integration frame. Figure 1b, shows an idealised response of the pixel for three different photocurrents and a monotonically increasing reference function.

The reference potential at which the transistor, M2 switches off is used as the pixel’s output. By comparing integrated voltage to a known reference voltage, one can capture high intensity information, otherwise lost in active pixel sensors. Herein, a typical voltage comparing pixel is described as this pixel will be used for rest of the papaer. However, a number of other architectures have been suggested which can be used. These include having a pMOS switch rather than an nMOS switch [22, 23], a comparator inside each pixel [24] . In all of these cases, one can use the reference function to generate any monotonically increasing pixel response. The reference function, however, has to be derived based on the required output transduction function. This can be obtained by comparing the integrated voltage at node N1 with that the output response. The integrating potential at node N1 can be expressed as

$$\begin{aligned} V_{N}(t) = V_{N}(t_0) - \frac{I_{ph}}{C_{PD}} (t - t_{0}). \end{aligned}$$

(1)

where $C_{PD}$ is photodiode capacitance and $I_{ph}$ is photocurrent. Assuming, the required output response is $V_{res} =f(I_{ph})$ , and $f'$ be the inverse function for this operator, such that $I_{ph} = f'(V_{res})$. Transistor M2 will remain on, till the potential at node N1 is higher than the reference voltage $V_{ref}$. However, when the transistor is on, the source of M2 follows Vref. Therefore, when the potential at node N1 becomes lower than Vref, it also means that the VGS has become less than 0, switching off the transistor. At the instance, we expect the reference voltage to be same as the tone mapped response for the given photocurrent, $V_{res} = V_{ref}(t)$. Therefore,

$$\begin{aligned} f'(V_{ref}(t))=C_{PD}\frac{V_{G,M2}(t)-V_{G,M2}(t_0)}{(t-t_0)} \end{aligned}$$

(2)

Fig. 1

a High dynamic range pixel schematic with adjustable reference function. b Operation of the pixel

3 Tone Mapping Reference Function

This technique can be used to extract any tone mapping monotonically increasing reference function. In this section, reference voltages for a number of tone mapping operators will be derived. These pixels enable generation of any tone mapped response and this paper does not concern itself with any particular tone mapping operator. It is also not concerned with individual merits of these operators and leaves the choice of any operator to the end-user. However, this analysis is undertaken to show that one can generate different transduction function from these operators. More importantly, one can change the reference voltage at the time of operation and hence can change the type of tone mapping operator being applied without changing the physical hardware of the pixel.

3.1 Reinhard Photographic Operator

A simple mapping operator by Reinhard [12] maps the scaled input luminance L(x, y) to the display luminance $L_{d}$ using following function

$$\begin{aligned} L_{d}(x,y)=\frac{L(x,y)}{1+L(x,y)} \end{aligned}$$

(3)

where L(x, y) is given as

$$\begin{aligned} L(x,y)=\frac{a}{L_{av}}\ L_{w}(x,y) \end{aligned}$$

(4)

$L_{w}(x,y)$ is input (world) luminance, a is the chosen exposure for developing the film in the photographic analogy and $L_{av}$ is the logarithm average luminance. Translating luminance values into pixel voltage and applying procedure discussed above, $V_{ref}$ for Reinhard simple operator can be obtained as

$$\begin{aligned} V_{ref}(t) = \frac{t}{C_{PD}\ g\ k}- V_{o} \end{aligned}$$

(5)

where $V_{o}$ is an offset related the threshold voltages, g is a/Lwh and k is a constant proportional to the photo-current and world luminance value.

This simple mapping operator, unfortunately is unable to burn out bright areas in a controllable fashion. Hence, in order to burn out bright areas, a more involved operator has been suggested [12]

$$\begin{aligned} L_{d}(x,y) = \frac{L(x,y)}{1+L(x,y)} {\left[ 1+\frac{L(x,y)}{L_{wh}^2}\right] } \end{aligned}$$

(6)

where $L_{wh}$ is the smallest luminance that will be mapped to white. Mapping luminance equation into pixel voltage and following the procedure discussed above, $V_{ref}$ for Reinhard complex operator is given as

$$\begin{aligned} V_{ref}(t) = V_{o}-\frac{t I_{w}^2 (t-aV_{dd})}{a (aV_{dd}-tI_{wh}^2)}. \end{aligned}$$

(7)

Where $V_{o}$ is an offset related the threshold voltages, a is user controllable value and $I_{wh}$ is the photocurrent corresponding to the value of $L_{wh}$. Figure 2a shows the monotonic increasing reference voltages for Reinhard operator.

3.2 Schlick Mapping Operator

Yet another tone mapping operator, due to Schlick uses the following expression [14]

$$\begin{aligned} L_{d}(x,y)= \frac{p\ L_{w}(x,y)}{(p-1)L_{w}(x,y)+L_{max}} \end{aligned}$$

(8)

where p is user controllable value and $L_{max}$ is maximum world luminance in scene. Translating luminance values into voltage current relationship and applying above procedure provides a reference function to obtain tone mapped response following the Schlick reference function

$$\begin{aligned} V_{ref}(t) = V_{dd}-V_{off}-\frac{p V_{dd}-\frac{I_{pmax}}{ck}}{p-1} \end{aligned}$$

(9)

where $V_{off}$ is the offset related to various threshold voltages in the readout chain of the pixel and $I_{pmax}$ is maximum photo-current in the frame.

3.3 Tumblin & Rushmier Mapping Operator

Tumblin & Rushmeier presented a mapping operator inspired by Stevens work on brightness [25] and given as

$$\begin{aligned}&L_{d}(x,y) =m L_{da} \left( \frac{L_w(x,y)}{L_{wa}}\right) ^\alpha \nonumber \\&\qquad { where}\ \alpha =\frac{\gamma (L_{wa})}{\gamma (L_{da})} \end{aligned}$$

(10)

$L_{da}$ and $L_{wa}$ are display and image adaptation luminance values respectively. Translating above equation into voltage-current relation and applying above procedure leads to Tumblin& Rushmier operator’s reference function as

$$\begin{aligned} V_{ref}=V_{dd}-V_{th}-V_{dd}(gk^e)\frac{cgkV_{dd}}{t}^f \end{aligned}$$

(11)

where $V_{th}$ is threshold voltage, g, k, e, f are user controllable values.

3.4 Drago Operator

The mapping operator presented by Drago is given as [13]

$$\begin{aligned} L_{d}(x,y)= \frac{L_{dmax}\ log_{10}(1+L_{w}(x,y))}{100log_{10}(1+L_{wmax})log_{10}\left[ 2+8\left( \frac{L_{w}(x,y)}{L_{wmax}}\right) ^{\frac{log_{10}(p)}{log_{10}(0.5)}}\right] } \end{aligned}$$

(12)

Transforming luminance into current voltage relationship and applying procedure discussed earlier, however, does not lead to a smooth solution. Nevertheless, once can use a look-up table to generate a reference voltage which can produce tone mapped response matching the Drago’s operator.

3.5 Exponential Reference Function

Exponential tone mapping operator is given as [26]

$$\begin{aligned} L_{d}(x,y)= 1-exp\left( \frac{-L_{w}(x,y))}{L_{av}}\right) \end{aligned}$$

(13)

Transforming luminance equation into pixel voltage-current relationship and following the procedure above, a reference function of exponential operator can be obtained as

$$\begin{aligned} t=\frac{v_{dd}cg(V_{ref}+V_{off}-1)}{log(V_{ref}+V_{off})} \end{aligned}$$

(14)

where g is constant and $V_{off}$ is offset related to threshold voltages in pixel.

3.6 Logarithmic Mapping Operator

Logarithmic mapping compress the high dynamic range information using logarithm function as

$$\begin{aligned} L_{d}(x,y)= \frac{log_{10}(1+L_{w}(x,y))}{log_{10}(1+L_{max})} \end{aligned}$$

(15)

where $L_{max}$ is maximum world luminance. Transforming luminance values into pixel voltages and comparing the pixel equations leads to the value of $V_{ref}$ that can be calculated using look up table generation. Figure 2 shows different reference functions required for wide dynamic range capture.

Fig. 2

a Monotonically increasing reference function for Reinhard Tone mapping operator. b Reference function for Schlick and exponential operators. c Reference function for Drago, Miller and Tumblin & Rushmeier

Fig. 3

High dynamic range pixel response covering upto 6 decades of intensity with Reinhard mapping operator

4 Pixel Model and Parameters Extraction

To further understand the implementation of these tone mapping operators, they were simulated with a typical pixel in a $0.35\mu$ CMOS process from Austria Microsystems using Cadence Virtuoso and Spectre simulator. The dimension of the reset transistor M1 was $1\mu m/0.7\mu m$ whereas that of the source follower and reset device was $2\mu m/0.7\mu m$. Figure 3 shows the response of the pixel over 6 decades of light intensity with Reinhard Tone Mapping operators. One can obtain other response functions as well.

Single pixel simulation show the ability to obtain any tone-mapped response. However, it is a well known that fixed pattern noise effects the performance of CMOS pixels. It arises due to variations between characteristics of each transistor in the pixels within an array. Typically, this is modelled as variations in the threshold voltage, $V_T$ and the current factor parameter $\beta$ of transistors.

For a good quality image to be obtained from these imagers, it is important to study the effect of the fixed pattern noise and remove it from the final image. Considering the complexity of the response, however, simple correction techniques like 2-parameter correlated double sampling will not be sufficient. One would need multi-parameter calibration. However, before doing so, one needs a simple model for the response of the tone mapped pixel. Herein, simple parametric models [27] for Reinhard simple and Reinhard complex will be derived to assist in FPN correction process to show case the ability to extract good quality images from these imagers. Reinhard Simple and Reinhard Complex operators are being selected as comparative studies of tone mapping operators have found them to be one of the better global tone mapping operators [28]. In addition, these pixel models also depicts the flexibility and proof of concept of general pixel behaviour. One can, nevertheless, derive pixel analytical models for other potential mapping operators, as well.

4.1 Reinhard Simple Operator

A simple parametric model for a pixel providing Reinhard simple tone mapping can be derived from the tone mapping operator shown in Equation 3

$$\begin{aligned} y_{s}=a-\frac{b(x+d)}{1+c(x+d)} \end{aligned}$$

(16)

This equation can be simplified by assuming that dark or leakage current is small enough to be neglected for bright light conditions. Hence the model response equation becomes

$$\begin{aligned} y_{s}=a-\frac{bx}{1+cx} \end{aligned}$$

(17)

In this equation, three parameters are unknown for each pixel. In an ideal condition, one would undertake curve fitting from the response of the pixel to a large number of photocurrents to identify the most optimum parameters. However, considering the simplicity of the equation, one can use the pixel’s response at three different calibration currents. Assuming three different currents $x_{1}$, $x_{2}$, $x_{3}$ and their respective response $y_{1}$, $y_{2}$ and $y_{3}$, such that

$$\begin{aligned} y_{s1}&=a-\frac{bx_{1}}{1+cx_{1}}\nonumber \\ y_{s2}&= a-\frac{bx_{2}}{1+cx_{2}} \nonumber \\ y_{s3}&= a-\frac{bx_{3}}{1+cx_{3}} \end{aligned}$$

(18)

Parameters value can be extracted by solving above equations as follows

$$\begin{aligned} c&=\frac{x_{1}y_{s2}-x_{2}y_{s1}-x_{1}y_{s3}+x_{3}y_{s1}+x_{2}y_{s3} -x_{3}y_{s2}}{x_{1}x_{2}y_{s1}-x_{1}x_{2}y_{s2}-x_{1}x_{3}y_{s1} +x_{1}x_{3}y_{s3}x_{2}x_{3}y_{s2}-x_{2}x_{3}y_{3}} \nonumber \\ b&=\frac{y_{s2}-y_{s3}}{ \frac{x_{3}}{1+cx_{3}} -\frac{x2}{1+cx_{2}}}\nonumber \\ a&=y_{s3}+ \frac{bx_{3}}{1+cx_{3}} \end{aligned}$$

(19)

The parameter d which represents the dark current can be extracted form Equation 16 by solving for an input photocurrent value of zero and hence,

$$\begin{aligned} d=\frac{a-y_{s}}{b-ac+cy_{s}} \end{aligned}$$

(20)

Despite apparent complexity of these equations, it is worth noting that these are computationally simple operators which only need to be applied at the time of calibration, generally once or with change in environment like the temperature.

4.2 Reinhard Complex Tone Mapping Operator

Similarly, pixel model for the Reinhard complex tone mapping operator can be rewritten in parametric form as

$$\begin{aligned} y_{c}=a-\frac{b(x+d)(1+e(x+d))}{1+c(x+d)} \end{aligned}$$

(21)

Once again a curve fit can be extracted from the response of the pixel to a large number of photocurrents. However, one can observe that at comparatively higher photocurrents, one can ignore the effect of the dark current and hence simplify the model as

$$\begin{aligned} y_{c}=a-\frac{bx(1+ex)}{1+cx} \end{aligned}$$

(22)

However, it is not easy to use this equation for simple parameters extraction. Rewriting the expression as

$$\begin{aligned} y_{c}=P+Qx-Rx^2-Sy_{c}x \end{aligned}$$

(23)

where $P=a$, $Q=ac-b$, $R=be$ and $S=c$. Now, there are 4 unknown parameters, hence four calibration currents $x_{1}$, $x_{2}$, $x_{3}$, $x_{4}$ with the responses $y_{c1}$, $y_{c2}$, $y_{c3}$, $y_{c4}$ can be used for parameters extraction. The model equations with calibration currents are given as

$$\begin{aligned} y_{c1}&=P+Qx_{1}-Rx_{1}^2-Sy_{c1}x_{1}\nonumber \\ y_{c2}&=P+Qx_{2}-Rx_{2}^2-Sy_{c2}x_{2} \nonumber \\ y_{c3}&=P+Qx_{3}-Rx_{3}^2-Sy_{c3}x_{3} \nonumber \\ y_{c4}&=P+Qx_{4}-Rx_{4}^2-Sy_{c4}x_{4} \end{aligned}$$

(24)

These equations can be solved for parameters by matrix method for simultaneous equation. Above equation can be rewritten as

$$\begin{aligned} \begin{pmatrix} 1 &{} x_{1} &{} -x_{1}^2 &{} -y_{c1}x_{1} \\ 1 &{} x_{2} &{} -x_{2}^2 &{} -y_{c2}x_{2} \\ 1 &{} x_{3} &{} -x_{3}^2 &{} -y_{c3}x_{3} \\ 1 &{} x_{4} &{} -x_{4}^2 &{} -y_{c4}x_{4} \\ \end{pmatrix} \cdot \begin{pmatrix} P \\ Q \\ R \\ S \\ \end{pmatrix} =\begin{pmatrix} y_{c1} \\ y_{c2} \\ y_{c3} \\ y_{c4} \\ \end{pmatrix} \end{aligned}$$

or as

$$\begin{aligned} A\cdot X=B \end{aligned}$$

(25)

Hence, parameters matrix X can be calculated by multiplication of matrix $A^{-1}$ with matrix B.

$$\begin{aligned} X=A^{-1} B \end{aligned}$$

With these parameters, one can then also extract the dark current related parameter, d. Hence, the calibration current used to extract parameters are one dark current, three mid-range photocurrents and one high photocurrent.

To validate both of these parameter extraction procedures, the pixel model response was compared with the pixel simulated response. Error between pixel model and simulated response is shown in Fig.  4. It can be seen that Reinhard simple exhibits the maximum model error of 15mV whereas Reinhard complex shows the error of 40mV. This absolute error compares well a full swing range 1.5V from the pixel, thereby corresponding to an error of less 1% for simple case. It is worth noting however, these the model error is an absolute error and does not effect fixed pattern noise, which is a relative error.

Fig. 4

Model error of Reinhard simple and complex operators

5 Fixed Pattern Noise Correction

In an array of pixels, the response of each pixel is different to uniform stimulus due to mismatch between transistors. This means that each parameter of the pixel is different to one another and hence should be calibrated for before a good quality image can be obtained. It has been observed that human eye has a contrast threshold of $1\%$ [29] which means the human eye is unable to perceive errors in misrepresented illuminations that are less than $1\%$ of the original illumination. Therefore, if a fixed pattern noise scheme leads to errors of less than $1\%$, human observers will not be able to distinguish the same.

To understand the behaviour of the fixed pattern noise in such pixels, Monte Carlo simulation of the pixel was undertaken with parameters provided by the foundry. The parameter calibration technique, described above was then used to extract the parameter of each pixel from its response to three photocurrents for simple Reinhard tone mapping and four currents for complex Reinhard tone mapping.

Once the parameters have been calculated, the actual photocurrent at any other illumination can be extracted by

$$\begin{aligned} I_{s}&=\frac{y_{s}-a-adc+y_{s}cd+bd}{ac-y_{s}c-b} \nonumber \\ I_{c}&=\frac{Q-2dR-Sy_{c}\pm \ (I)^{1/2}}{2R} \end{aligned}$$

(26)

where

$$\begin{aligned} I =(Q^2-2QSy_{c}+S^2y_{c}^2-4Ry_{c}+4PR) \end{aligned}$$

FPN was then corrected for each pixel for Reinhard simple as well as complex tone mapping operator. Figure  5 shows the percentage contrast error for Reinhard simple and complex operator after FPN correction at different photocurrents. After FPN correction, Reinhard simple exhibits the contrast error of more than $1\%$ at high and low photocurrents. At mid-range photocurrent, contrast error is less than $1\%$, which is very promising result. Reinhard complex performance is much better than Reinhard simple and contrast error is less than $1\%$ over the whole range of photocurrents. It may be noted that Reinhard simple operator has led to a higher contrast error at low photocurrents. This is so as the parameters are being extracted using 3 calibration currents and one dark level, assuming the dark current to be zero. This however, leads to errors as it does not represent actual photocurrent in the pixel. Nevertheless, the maximum error is 3$\%$ of contrast compared to 1$\%$ of contrast limit of the human eye. In Reinhard complex case, 4 calibration currents were used and hence a better fit to pixel response has been obtained leading to good error values. At high photocurrents, both techniques lead to large residual values; however, these photocurrents already exceed the 6 decades of intensity required from wide dynamic range pixels and the errors are still within 1$\%$ expectations in Reinhard complex case (Fig. 6).

Fig. 5

FPN expressed as relative contrast error of a Reinhard simple and b Reinhard complex operator

Fig. 6

Histogram comparison of a Reinhard simple extracted photocurrents before and b after FPN correction at middle-range current $10^{-11}A$

Fig. 7

a Test Image. b Test image with fixed pattern noise

To further understand the behaviour of the response, a simple test image shown in Fig. 7a¹ is used to check the effects of FPN and the suitability of the correction procedure on an image. FPN is added based from typical mismatch parameters of the transistors as provided by our foundry to test image to check the image quality degradation caused by FPN. FPN correction technique is later applied on an test image that already have FPN added. A test image with and without FPN is shown in Fig. 7. Figure 8 shows the image after FPN correction for Reinhard simple and complex operator. It can be seen that FPN after correction procedure shows some artifacts in dark regions of an image. The artifacts in an image are due to arbitrary selection of calibration currents $x_{1}$, $x_{2}$, $x_{3}$, $x_{4}$ and can be reduced by optimising the selection of calibration current. Similar fixed pattern noise correction can be performed for other operators by using similar technique.

Fig. 8

a Test image after FPN correction of Reinhard simple. b Test image after FPN correction of Reinhard complex

6 Conclusion and Discussions

This paper presents a pixel, its modelling and fixed pattern noise correction for wide dynamic range image capture providing real-time tone mapping operators. A pixel design is reported which uses NMOS only transistors. The reference functions extracted from different tone mapping operators were applied on pixel in order to capture wide dynamic range. WDR pixel model is also presented in terms of parameters that characterize the response of each pixel. In addition, parameters extraction routine is presented by assuming different calibration currents. Fixed pattern noise correction is performed. FPN comes from variations between the characteristics of individual pixel. A simple procedure has been adapted to reduce FPN in which parametric response of pixel is used. The parameters of simulated individual pixels are measured, recorded and then used to correct their response. The WDR pixel reported here can capture the dynamic range of upto 6 decades of light intensity. FPN correction result is presented with an overall relative contrast error of less than $1\%$ for Reinhard simple and Reinhard complex mapping operators.

Some system level design issues related to the pixel and technique are also worth discussing. The pixel uses three transistor in the signal path compared to two transistors in typical active pixel sensors. This means that the output signal swing would be less by one threshold voltage. This means that there would be additional stress on the data converter used in the column. However, typical high dynamic range schemes utilise a higher bit-count data converter to enable compression at a later stage. With focal plan compression, one may not need that level of data conversion and hence this reduction in signal swing will be compensated for.

Yet another aspect of the pixel is that it requires the same reference voltage, $V_{ref}$ to be applied on all pixels. This means that the signal source providing this waveform would need to drive a large number of drain connections over the entire area of the image sensor. This could lead to significant parasitic capacitance, thereby requiring very output current from this source. One way to reduce this requirement would be to place a signal buffer in each row. Finally, the pixel layout needs to be carefully undertaken to ensure minimal cross-talk between gates of devices M2 and M3.

Furthermore, the pixel operates as a partial global shutter pixel. All pixels are reset at the same time and the M3 gate capacitance holds the voltage at which M2 switches off. The gate of M3 is expected to hold this voltage till the readout. However, there will be a finite leakage at the gate and hence, the pixels which will be readout later in a frame could have a higher leakage than the ones which were readout earlier. This will manifests itself as another source of the fixed pattern noise, which can be corrected. Solution to both problems of capacitive leakage and coupling include careful layouts as well as additional capacitance at gate of device M3.

Finally, it is worth noting that any focal plane compression scheme will degrade the signal to noise ratio at the pixel output. However, being a continuously compared system, the SNR is not expected to see any dips as in double-capture or multiple capture schemes for wide dynamic range pixels. More importantly, in other wide dynamic range imaging schemes, the captured data has to be tone mapped at a later stage. Therefore, even if one has higher SNR at pixel stage, the final SNR will degrade due to tone mapping. Comparison of the final SNR between the two schemes of focal plane compression verses wide dynamic range capture followed by tone mapping will be part of our future work.