Evaluation of Structures and Methods for Resolution Determination of Remote Sensing Sensors

Abstract

Effective image resolution is an important image quality factor for remote sensing sensors and significantly affects photogrammetric processing tool chains. Tie points, mandatory for forming the block geometry, fully rely on feature points (i.e. SIFT, SURF) and quality of these points however is significantly correlated to image resolution. Spatial resolution can be determined in different ways. Utilizing bar test charts (e.g. USAF51), slanted edges (ISO 12233) and Siemens-Stars are widely accepted techniques. The paper describes these approaches and compares all in one joint experiment. Moreover, Slanted-Edge and Siemens-Star method is evaluated using (close to) ideal images convolved with known parameters. It will be shown that both techniques deliver conclusive and expected results.

1 Introduction

Ground resolved distance (GRD) or true ground sample distance (tGSD) is an essential parameter of imaging systems [4, 9], as it defines the detail of information in any image taken by remote sensing sensors. The effective geometric resolution significantly affects photogrammetric processing tool chains. Tie points, mandatory for forming the block geometry, fully rely on feature points (SIFT, SURF, etc.) and the quality parameters of these points however are significantly correlated to image resolution [7]. This is why resolution determination is of such importance to quantify the potential of a sensor-lens-combination.

Although acquisition of resolving power is a well-studied field of research, there are still some scientific questions to be answered when it comes to a standardized (eventually absolute) determination. This is also research object of a committee of the “German Institute for Standardization” and the given contribution outlines the current state of investigation concerning remote sensing sensors.

Orych [9] provided a description of calibration targets used for high-resolution remote sensing imaging equipment and concluded: “Based on a preliminary analysis, three types of test patterns were selected as possible choices for evaluating the quality of imagery acquired by UAV sensors: bar target, Slanted Edge Test and Siemens Star.” Extending the perspective from UAV-context to a general remote sensing perspective all three approaches must deliver similar or ideally the exact same results for identical images and image regions.

Furthermore, implementations of Slanted-Edge and Siemens-Star method can be tested for validity by using known (model) parameters for a Gaussian-kernel and subsequent convolution with (close to ideal) images. Then it must be possible to extract (resp. measure) the predefined parameters with both approaches.

Therefore, all techniques (USAF51, Slanted-Edge, Siemens-Star) will be described with mathematical detail in Sect. 2 followed by introducing a model-based approach to simulate distinct image resolution in Sect. 3. Related experiment description and obtained results are given in Sect. 4.

2 Structures and Techniques

Sharpness as an image property is characterized by the modulation transfer function (MTF) which is the spatial frequency response of an imaging system to a given illumination. “High spatial frequencies correspond to fine image detail. The more extended the response, the finer the detail - the sharper the image.” [8]. Inverse Fourier-transforming MTF, directly delivers the point spread function (PSF) [10]. The parameter $\sigma$ (standard deviation) of the PSF (assumed Gaussian-shape function) is one criterion. It directly relates to image space and can be seen as objective measure to compare different camera performances. Another criterion is the spatial frequency where the MTF reaches a certain (minimal-) value (i.e. 10%, MTF10). The reciprocal of that frequency is the approximation for size of the smallest line per pixel. The width of PSF at half the height of the maximum is another criterion (full width half maximum - FWHM) and is related to $\sigma$ of PSF as follows [14]. Starting by assuming a Gaussian-shape function (Eq. 1).

$$\begin{aligned} H\left( x \right) = \frac{1}{{\sigma \sqrt{2\pi }}} \cdot {e^{ - \frac{{{(x-\mu )^2}}}{{2 \cdot \sigma ^2}}}} \end{aligned}$$

(1)

The constant scaling factor $\frac{1}{{\sigma \sqrt{2\pi }}}$ can be ignored. Applying $H(x)=0.5$ leads to Eq. (2):

$$\begin{aligned} e^{ - \frac{{{x_0-\mu }}}{{2 \cdot \sigma ^2}}} = 2^{-1} \end{aligned}$$

(2)

Solving Eq. (2) and assuming function value $H(x_{max})$ occurs at $\mu =0$ half-maximum points $x_0$ are found (Eq. 3):

$$\begin{aligned} x_0 = \pm \sigma \sqrt{2\ln 2} \end{aligned}$$

(3)

The full width at half maximum is then given by:

$$\begin{aligned} FWHM=x_{+}-x_{-}=2\sqrt{2\ln 2} \sigma \approx 2.3548\sigma \end{aligned}$$

(4)

A similar measure exists in frequency domain. The effective instantaneous field-of-view (EIFOV) for MTF at $50\%$ contrast level [3]. Assuming a Gaussian-shape function for PSF (Eq. 1) the Fourier-transformed $\tilde{H}( \nu )$ (MTF, Eq. 5) is formulated as follows [5]. Again, the constant scaling factor can be ignored.

$$\begin{aligned} \tilde{H}\left( \nu \right) = {e^{ - 2 \cdot {\pi ^2} \cdot \sigma ^2 \cdot {\nu ^2}}} = 0.5 \end{aligned}$$

(5)

By setting $\tilde{H}( \nu )$ equal to $= 0.5$ Eq. (5) can be written as:

$$\begin{aligned} 2 \cdot {\pi ^2} \cdot \sigma ^2 \cdot {\nu ^2} = - \log (0.5) \end{aligned}$$

(6)

Subsequent transposing then gives:

(7)

Substituting with C

(8)

gives the formula for $\nu _\delta$ (Eq. 9) similar to Eq. (3) and $x_0$.

$$\begin{aligned} \nu \cdot \delta = {\nu _\delta } = \frac{C}{{\sigma ^\delta }} \end{aligned}$$

(9)

Finally EIFOV can be calculated with the following equation:

$$\begin{aligned} \mathrm{{EIFOV = }}\frac{{\sigma ^\delta }}{{2C}} = 2.67 \cdot \sigma ^\delta \end{aligned}$$

(10)

By comparing Eqs. (4) and (10) it is noticeable that both image quality parameters (FWHM & EIFOV) depend in their related domain (image- or frequency-domain) only on parameter $\sigma$ (PSF or MTF) and a similar constant factor.

Aforementioned image quality parameters can be determined with different structures of patterns and different techniques and will be described in the following sub sections.

2.1 Bar Target

A classic approach is to use defined test targets (e.g. USAF resolution test chart, see Fig. 1, left) with groups of bars [12]. “The resolving power target used on all tests shall be as follows: The target shall consist of a series of patterns decreasing in size as the $\sqrt{2}$, $\root 3 \of {2}$, $\root 6 \of {2}$, with a range sufficient to cover the requirements [...]. The standard target element shall consist of two patterns (two sets of lines) at right angles to each other. Each pattern shall consist of three lines separated by spaces of equal width. Each line shall be five times as long as it is wide.”

Images of test targets fulfilling these requirements are directly linked to object space metric resolution (see Fig. 1, right). There, the identified resolution corresponds to the distance between bars of the least discriminable group. The decision whether a group still is discriminable or not strongly relies on viewers’ perception. To diminish subjective influence statistically the number of viewers n is chosen to be significant (e.g. $n\ge 10$) and the resulting resolution $G_r$ (GRD, tGSD) is calculated (11) as mean of all independent observations $G_i$.

$$\begin{aligned} G_r = \frac{1}{n} \sum _{i=1}^n G_i \end{aligned}$$

(11)

With knowledge about interior camera parameters (focal length f, pixel size s) and distance between camera system and test target d the theoretical resolution $G_t$ is calculated as:

$$\begin{aligned} G_t = \frac{s}{f}\;d \end{aligned}$$

(12)

While $G_r$ is equivalent to GRD or tGSD, the quotient $G_q$ according to Eq. (13) provides another measure for image resolving power.

$$\begin{aligned} G_q = \frac{G_r}{G_t} \end{aligned}$$

(13)

Usually values for $G_q$ greater than 1 are expected to be calculated. In this case theoretical resolution $G_t$ is better than ultimately determined resolution $G_r$. Values $G_q\le 1$ either result due to loss-less transition from object space to image space or indicate image enhancement (e.g. edge-sharpening, color refinement or super resolution).

Besides the disadvantage of subjective influence included in this acquisition method values for resolving power are discrete instead of continuous.

Fig. 1.

Aerial image of USAF bar test target (left), corresponding ground resolution [cm] in object space (right)

Fig. 2.

ESF, LSF and normalized FFT of an edge.

2.2 Slanted-Edge

The presented approach uses an edge-step technique [1, 6]. It evaluates the transition between a very homogeneous dark area to a very homogeneous bright area along an extremely sharp, straight edge within the image. The most challenging part of the algorithm is to identify suitable horizontal and vertical edges [6] and to make sure that their position is known to sub-pixel accuracy [1]. Identification of the edges is done automatically either by using a line segment detector [13] or by using a Canny edge detector followed by a Hough transform. Each edge is refined to match the actual transition in the current image as closely as possible, using a custom-built refinement procedure.

After the edges have been located and confirmed to meet the quality standards, their complete profile, spanning their entire length, has to be derived. For each point on the edge, moving along the edge pixel by pixel, the profile following the image’s pixel grid is extracted and projected onto the perpendicular to the edge. An alternative approach is to scan and combine multiple perpendicular lines by applying bi-cubic or bi-linear interpolations methods [11].

The thus obtained projected edge profile is cleaned from blunders, filtered and approximated with a Sigmoid function. The resulting Edge Spread Function (ESF), i.e. the response of the system to this edge [1, 6], is shown in Fig. 2(a). The numerical derivative of the ESF yields the Line Spread Function (LSF), the response of the system to a line target [1, 6], an example of which is displayed in Fig. 2(b). Finally, a Fast Fourier Transform (FFT) is applied to the LSF (Fig. 2(c)) and the normalized magnitude of the result evaluated at the Nyquist frequency (0.5 cycles per pixel) yields the MTF.

2.3 Siemens-Star

Using a priori knowledge of the original scene (well-known Siemens-Star target) contrast transfer function CTF, MTF and PSF are approximated by a Gaussian shape function [7]. Coordinate axis X for CTF and MTF is the spatial frequency f (Eq. 14) and is calculated as target frequency ${f}_{s}$ divided by current scan radius r multiplied by $\pi$. Target frequency ${f}_{s}$ is constant and equivalent to the number of black-white segments of the well-known Siemens-Star.

$$\begin{aligned} f =\frac{{{f}_{s}}}{{\pi r}} \end{aligned}$$

(14)

Related (initially discrete) values for contrast transfer function ${C}_{d}\left( f\right)$ are derived using intensity maxima ${I}_{max}$ and minima ${I}_{min}$ for every scanned circle (Eq. 15). Simultaneously the function value is normalized to contrast level ${C}_{0}$ at spatial frequency equal to 0 (infinite radius).

$$\begin{aligned} {C}_{d}\left( f\right) =\frac{{{I}_{max}\left( f\right) -{I}_{min}\left( f\right) }}{{{I}_{max}\left( f\right) +{I}_{min}\left( f\right) }}*\frac{1}{{C}_{0}} \end{aligned}$$

(15)

Continuous function values C are derived by fitting a Gaussian function into discrete input data (Eq. 16).

$$\begin{aligned} {C} =\frac{1}{ \sigma \sqrt{2\pi }}e^{-\frac{1}{2}\left( \frac{x-\mu }{\sigma }\right) ^{2}} \end{aligned}$$

(16)

According to [2] the obtained CTF describes the system response to a square wave input while MTF is the system response to a sine wave input. The proposed solution is a normalization with $\frac{\pi }{4}$ followed by series expansion using odd frequency multiples (Eq. 17).

$$\begin{aligned} MTF\left( f\right) = \frac{\pi }{4}\left[ {C}\left( f\right) + \frac{ {C}\left( 3f\right) }{3} + \frac{ {C}\left( 5f\right) }{5} +\dots \right] \end{aligned}$$

(17)

MTF describes the effective resolving power in frequency domain while PSF is the image domain equivalent. For this reason both functions are linked directly by fourier transform (Eq. 18).

(18)

3 Model-Based PSF and MTF

A conclusive validation of Slanted-Edge (Sect. 2.2) and Siemens-Star technique (Sect. 2.3) is to apply predefined modulation (MTF) or spread parameters (PSF) to an ideal representation of resolving patterns (see Fig. 3). This can be done in both domains. In image-domain it can be done by forming a convolution of mathematical-ideal image-intensity values of an image (I), a Gaussian-shape model PSF ($H_{m}$) and a mathematical-ideal sensor PSF ($H_{s}$). Simulated PSF ($H_{sim}$) then can be formulated as follows:

$$\begin{aligned} H_{sim}(\rho ) = I(\rho ) * H_{m}(\rho ) * H_{s}(\rho ) \end{aligned}$$

(19)

In frequency-domain calculation gets simpler, only the product of image spectrum ($\tilde{I}$) with a predefined model-based MTF ($\tilde{H}_{M}$) and (mathematical-ideal) sensor MTF ($\tilde{H}_{S}$) has to be calculated. Therefore, simulated-image MTF ($\tilde{H}_{Sim}$) can be formulated as follows:

$$\begin{aligned} \tilde{H}_{Sim}( \nu ) = \tilde{I}( \nu ) \cdot \tilde{H}_{M}( \nu ) \cdot \tilde{H}_{S}( \nu ) \end{aligned}$$

(20)

Fig. 3.

Original image (upper left) continuous and discrete Gaussian PSF convolution kernel (upper mid) and convolution result (upper right), ideal MTF, close to ideal MTF and model-based MTF (lower left), related products in frequency domain (lower right)

The derived hypothesis is, if both algorithms (Slanted-Edge and Siemens-Star) described in Sect. 2 provide measurements of absolute value then model-MTF ($\tilde{H}_{M}$) respectively model-PSF ($H_{m}$) must directly be confirmed by measurement of simulated-image MTF ($\tilde{H}_{Sim}$) respectively PSF ($H_{sim}$).

Mathematical-ideal sensor-MTF $\tilde{H}_{S}( \nu )$ with $\nu \in \mathbb {R}$ is characterized as being equal to 1 for all frequencies (see Fig. 3, dotted line). However, when an ideal pattern is rendered to a pixel grid the resulting (Nyquist-limited) sensor-PSF and sensor-MTF unavoidably will differ from ideal shape. An example of (close to) ideal sensor-MTF can be seen in Fig. 3 (dashed-dotted line) with Nyquist-limit 1.0 line per pixel.

As a result, obtained MTF values ($\tilde{H}_{Sim}$) measure the product of (close to) ideal sensor-MTF ($\tilde{H}_{S}$) and model-MTF ($H_{M}$) and therefore are expected to be smaller than the product of ideal sensor-MTF ($\tilde{H}_{S}( \nu )=1$, $\nu \in \mathbb {R}$) and model-MTF ($H_{M}$) (see Fig. 3, magenta and cyan line).

Considering that, PSF and MTF are directly linked by (inverse) Fourier transformation (Eq. 18), it can be assumed that for increasing values $\sigma _m$ ($H_{s}$) respectively for decreasing values $\sigma _M$ ($\tilde{H}_{S}$) simulated images and corresponding measured quality parameter $\sigma _{SLE}$ and $\sigma _{Star}$ of $H_{sim}$ will be continuously less affected by the difference of ideal and (close to) ideal sensor- PSF or MTF. This assumption can be verified (empirically) by an experiment in Sect. 4.2.

4 Experiments

Algorithms for standardized (eventually absolute) determination of resolving power under consideration for norm-description (e.g. by a committee of the “German Institute for Standardization”) need to be validated with respect to conditions described in Sects. 2 and 3.

Every method to determine effective resolving power of remote sensing sensors described in Sect. 2 for itself has individual advantages. Slanted-Edge is a well studied approach and has been transferred to a norm-description (ISO 12233, [15]). Bar charts (e.g. USAF51) are very intuitive and responsive. Slanted-Edge uses the first derivative of ESF between intensity maxima and minima, in contrast the Siemens-Star approach uses exact those maxima and minima and calculates CTF (Eq. 14) and MTF. Empirical observations indicate that due to this difference measurements of the Siemens-Star approach are more robust against influence of widely used sharpening filters.

4.1 Simultaneous Resolving Power Determination

Given the variety of approaches and techniques it is consistently necessary to compare their respective results and answer the question if used techniques do or do not perform equivalently and what are reasons for particular observations. Therefore, all described approaches have been applied simultaneously for identical images and image regions (example Fig. 4). Used image quality parameter is ground resolved distance (GRD in cm). For USAF51 bar chart GRD is calculated according to Eq. (11) with number of observers $n\ge 10$. Reciprocal of MTF10 is the approximation for size of the smallest line per pixel. Multiplying reciprocal of MTF10-values from Slanted-Edge and Siemens-Star measurement with calculated ground sample distance (GSD, Eq. 12) delivers GRD for both algorithms. Seven images (example see Fig. 4), showing bar chart and Siemens-Star simultaneously and GSD between 1.24 cm and 1.27 cm, have been taken to obtain following results (Table 1):

Fig. 4.

Simultaneous determination of ground resolved distance (GRD) for USAF51 (left), Slanted-Edge (upper right) and Siemens-Star (lower right)

Table 1. Simultaneous determination of ground resolved distance (GRD) for USAF51, Slanted-Edge and Siemens-Star

The fourth column ($\varDelta$ SLE-Star) shows the absolute difference between Slanted-Edge and Siemens-Star approach in percent. Except for one outlier, both techniques seem to measure very similar at an overall mean difference of $3.3\%$.

Values obtained by independent human observers and USAF51 tend to be more static compared to the other methods. This effect could be caused by huge resolution steps between groups of bars. Rearranging the target, including more groups with finer descent, may weaken the effect.

Even when comparing all three approaches at once ($\varDelta$ Min-Max) the overall mean difference of $7.1\%$ still can be considered very low.

4.2 Measurement of Model-Based MTF and PSF

As described in Sect. 3, a conclusive validation of Slanted-Edge and Siemens-Star technique is to apply predefined modulation (MTF) or spread parameters (PSF). Then, the used model parameters $\sigma _m$ must be reproduced by both methods during measurement ($\sigma _{SLE}$ and $\sigma _{Star}$ of $H_{sim}$). For this reason, an image showing a Siemens-Star including (close to) ideal sensor PSF ($H_{s}$) has been convolved with different $\sigma _{m}$ starting at 0.500 and rising to 1.750. Subsequently, $\sigma$ of $H_{sim}$ has been calculated with both Slanted-Egde and Siemens-Star approach. Obtained results can be found in following table (Table 2):

Table 2. Model-PSF compared to measured PSF of Slanted-Edge and Siemens-Star

Values in column $\varDelta A$ show the difference between model parameter $\sigma _{m}$ and measured parameter $\sigma _{SLE}$ in absolute percentage [%]. Values of column $\varDelta B$ provide results for difference between $\sigma _{m}$ and measured $\sigma _{Star}$. Similar to the comparison of both techniques in preceding experiment Sect. 4.1 column $\varDelta C$ reflects the absolute difference between $\sigma _{SLE}$ and $\sigma _{Star}$ in absolute percentage [%].

Two observations can be emphasized. First, the difference between Slanted-Edge and Siemens-Star technique again is small. In contrast to experiment Sect. 4.1 overall mean difference of $1.6\%$ here is even smaller and measurements deliver no outliers. Second observation regards constructed hypothesis in Sect. 3: “...it can be assumed that for increasing values $\sigma _m$ (PSF) ...simulated images and corresponding quality parameter ...will be continuously less affected by the difference of ideal and (close to) ideal sensor- PSF”. Columns $\varDelta A$ and $\varDelta B$ indicate that this hypothesis is true. With rising $\sigma _{m}$ the absolute difference of both methods tend to approach zero.

5 Conclusion and Outlook

Mathematically detailed descriptions of three different techniques for determination of resolving power were presented. A model-based approach and its underlying theory has been introduced to verify two acquisition methods (Slanted-Edge and Siemens-Star). Moreover, two experiments have been conducted to verify similar and correct measurements of all techniques. It can be concluded that all methods deliver expected, similar and mathematical predictable results. In particular, experimental results for difference of Slanted-Edge (ISO 12233) and Siemens-Star deliver very similar output and thus both approaches can be considered for further evaluation regarding standardized norm-description. Presented results highly indicate that both methods can be seen complementary to each other.

Previous work [7] already described influence of used de-mosaicing methods on resolving power and related measurements. Future work and final contribution is going to conclude the investigation and thoroughly clarify further open issues as support to research of the “German Institute for Standardization”.

These open issues are: Siemens-Star center position (determination and associated confidence), normalization of contrast magnitude and related requirements of the test pattern layout, exposure time dependency, influence of motion blur, influence of used interpolation methods during signal-scan (e.g. nearest-neighbour, bi-linear, bi-cubic), different mathematical models for PSF/MTF (Gaussian-shape, polynomial-shape, piece-wise linear) and influence of test target inclination during acquisition.