DIC Challenge: Developing Images and Guidelines for Evaluating Accuracy and Resolution of 2D Analyses

Abstract

With the rapid spread in use of Digital Image Correlation (DIC) globally, it is important there be some standard methods of verifying and validating DIC codes. To this end, the DIC Challenge board was formed and is maintained under the auspices of the Society for Experimental Mechanics (SEM) and the international DIC society (iDICs). The goal of the DIC Board and the 2D–DIC Challenge is to supply a set of well-vetted sample images and a set of analysis guidelines for standardized reporting of 2D–DIC results from these sample images, as well as for comparing the inherent accuracy of different approaches and for providing users with a means of assessing their proper implementation. This document will outline the goals of the challenge, describe the image sets that are available, and give a comparison between 12 commercial and academic 2D–DIC codes using two of the challenge image sets.

Appendices

Appendix 1

In this section, example analyses and some comments are provided for Image Sets 1–13 and 16–17. These DIC results were performed by the lead author (not by the DIC Challenge participants), using a single, commercially-available DIC software. These results are intended to provide examples of analysis techniques and typical results expected from these images for others who intend to use the image sets to test their own DIC code.

A.1 Sample 1 and Sample 5 – Rigid-Body-Motion with Contrast Variation

Sample 1 and 5 are both rigid body shifted images with varying contrast and noise (1.5 counts) in an 8-bit image. The images are shifted in both x- and y-directions simultaneously in shifts of 0.05 pixels/step (Sample 1 – see Fig. 17) and 0.1 pixels/step (Sample 5 – see Fig. 18). Figure 17 shows the histogram for Sample 1 for the first and last image. Sample 1 provides a varying contrast throughout the image series. Most DIC codes cannot correlate throughout the entire series unless a ZNSSD algorithm is used because the contrast shift is different for each image. It is important for DIC codes to compensate for contrast shifts because in most DIC experiments, particularly for stereo-DIC, there will be changes in the contrast during the experiment.

Fig. 17

Bias and variance results for Sample 1. Inset in the figure are the DIC analysis parameters and the full-field results for file “trxy_s2_01.tif” with the subset size pictured on the speckle pattern. Bottom of figure is the histogram for the first and last image of the series as labeled. Error bars are based on +/− one standard deviation

Fig. 18

Bias and variance results for Sample 5. Inset color plot is for “Sample5-000 X0.00 Y0.00 N2 C75 R10.tif”. Bottom of figure is the histogram for the first and last image of the series as labeled. Error bars are based on +/− one standard deviation

A.2 Sample 2 and Sample 4 – Rigid-Body-Motion with High Image Noise

Sample 2 and Sample 4 were created to have rigid-body shifts with poor contrast and very high noise. Results for Sample 2 are shown in Fig. 19 and for Sample 4 are shown in Fig. 20.

Fig. 19

Sample 2 results. Insets show the DIC settings and the results from “trs2_b8_01.tif”. The DIC results are plotted over a reduced range of contrast to make the speckles visible. The subset is shown over the speckle pattern, note that the speckles are barely visible. Error bars are based on ± one standard deviation

Fig. 20

Sample 4 results for image “Sample4-002 X0.2 Y0.2 N8 C50 R0.tif”. Inset in the figure are the image (which appears nearly black due to the poor contrast) with the large subset (white square) required to get correlation to work as indicated in the DIC settings. Bottom of figure is the histogram for the first image of the series as labeled. Error bars are based on +/− one standard deviation

A.3 Sample 3 and Sample 3b – FFT Rigid-Body-Motion and Step Shift

Sample 3 is an FFT shifted image with a typical noise level and can be used to look at the variance and bias errors (shown in Fig. 21).

Fig. 21

Bias and variance results for Sample 3. Inset shows the DIC settings and results from “Sample3-002 X0.20 Y0.20 N2 C0 R0.tif”. Subset is shown over the speckle pattern. Bottom of figure is the histogram for the first image of the series as labeled. Error bars are based on +/− one standard deviation

Sample 3b shifts half of the image by 0.05, 0.1, 0.2, 0.25 and 0.5 pixels. Results are shown in Fig. 22 for all 5 shifts using the DIC settings shown in the inset. A study on the effect of changing the step size and the subset size are shown in Fig. 23. It can be seen that larger subsets and steps create a larger roll-off at the discontinuity.

Fig. 22

Results from all Sample 3b line cuts. Full-field results shown for “Sample3 Half01.tif” with the line cut shown. Histogram of the reference image and DIC settings also shown

Fig. 23

Close up of the shift profiles near the discontinuity for a subset and step size study using the “Sample3 Half01.tif” image and the line cut (in Fig. 22). Subset from smallest to largest are shown inset on the speckle pattern

A.4 Sample 6 and Sample 7 – Pseudo-Experimental Image Shifting

Sample 6 and Sample 7 were created using the pseudo-experimental approach. Note that the errors are much larger for Sample 7 than for the similar synthetically generated Samples 2 and 4. As the pseudo-experimental results mimic more realistically the errors that are witnessed in a typical DIC experiment, it prompts the question: What is being missed in the synthetic image creation process?

Fig. 24

Sample 6 results for low-noise pseudo-experimentally shifted images. Inset DIC results are for “Sample6-Pro Dot G0_Y00 X00.tif”. Inset shows speckle pattern with subset overlay. Histogram also shown. Error bars are based on +/− one standard deviation

Fig. 25

Sample 7 results for image “Sample7-LCG25-Y02-X02.tif” are shown in the inset with the subsite size illustrated. DIC parameters are inset as well. Bottom of figure is the histogram for the first image of the series as labeled. Error bars are based on +/− one standard deviation

A.5 Sample 8 and Sample 9 – Image Rotation

For Sample 8 and 9 the rotation mean and standard deviation were calculated at each subset step using the following equation:

$$ \theta ={\tan}^{-1}\frac{\sqrt{u^2+{v}^2}}{\sqrt{{\left(x-256\right)}^2+{\left(y-256\right)}^2}}\bullet \frac{180}{\pi } $$

(4)

The results are plotted in angle error, defined as the difference between the known angle and the measured angle: (θ_DIC – θ_synthetic). Figure 26 are the results for Sample 8 and Fig. 27 for Sample 9. The results are nearly identical, with the results from Sample 9 being noisier as indicated by the larger error bars representing one standard deviation (1σ) of the angle error. Sample 9 has a slightly larger angle noise due most likely to the sub-optimal painted speckle pattern verses the nearly ideal speckle pattern created by TexGen.

Fig. 26

Sample 8 TexGen rotation results. Inset DIC results for “rot_s1_05-tif”. Subset overlay on speckle pattern, histogram and DIC settings also displayed. Error bars are based on +/− one standard deviation

Fig. 27

Sample 9 FFT rigid body rotation results for image “Sample9–05.tif”. Inset in the figure are the full-field measured results for the 5° case, with the DIC software settings listed. Bottom of figure is the histogram for the first image of the series as labeled. Error bars are based on +/− one standard deviation

A.6 Samples 10, 11, 11b – Spatially Varying Strain Field

Sample 10 and Sample 11 used a spatially varying function from Fazzini and Laurent [25].

Sample 11b was created with a triangle shaped displacement field. This creates a step in the strain field that can be used to investigate the roll-off of the DIC filtering. Typical results with the DIC settings can be found in Fig. 28.

Fig. 28

Sample 11b results for the triangle wave function created using the FFT. Inset is the full-field strain results for image “Sample 11b Tri-0.5.tif”, which corresponds to a strain of 0.00195. Software settings for the analysis are inset as well. Commanded results are dotted lines, DIC reconstructed solid lines of the same color. Bottom of figure is the histogram for the first image of the series as labeled

A.7 Samples 12 and 13 – Experimental Image Series

Figure 29 shows a cross-sectional data cut through the center of the specimen for Sample 12 with the principal strain plotted. The subset, step and strain window (filter) were varied to demonstrate the importance of these parameters on the spatial resolution conducting a typical virtual strain gage size study. More details on the specimen and material may be found in [26].

A full-field result from Sample 13 is shown in Fig. 30. Incremental correlation was used with the DIC settings shown in the figure. Strain in the y-direction is plotted. More details on the experiment can be found in [27].

Fig. 29

Sample 12 results for image “oht_cfrp_11.tiff”. Inset full-field data is for subset = 15, step = 3, strain window = 5. Line cut plots shown taken vertically through the center of the specimen. Also shown the histogram for the first image of the series as labeled

Fig. 30

Sample 13 results for image “pw-0mil-p2-n14-660_0.tif”. Strain in the vertical direction is shown using the inset DIC software settings. Also shown the histogram for the first image of the series as labeled

A.8 Sample 16 – Experimental in-Plane Translation

Sample 16 was a rigid-body experimental translation data set. The experimental description and setup are detailed in the text and shown in Fig. 4. Displacement results for two steps, one early in the translation and one towards the end are shown in Fig. 31 and were analyzed using 3 common DIC interpolants (Fig. 35). Also reported in the figure are the stage position error and standard deviation. The interpolation bias error can be clearly seen in the DIC results (Fig. 35). The final complication is the lens distortions which began to grow as the sample was translated. Even with small lens distortions (±0.03 pixels at 1-pixel translation), they caused the displacement error to be incorrectly reported (see Fig. 34 image 00050). However, we assume that the mean value reported is still correct. Therefore, the error bars we report in Fig. 32 are the +/− one standard deviation noise from the first 0.01-pixel step before lens distortions become noticeable and biased the noise results.

Note the bias errors are extremely small. The ability to experimentally measure the bias errors (for a good interpolant) is extremely difficult. To our knowledge, there is only one other experiment reported in the literature where the bias errors can be clearly seen in an experiment using a translation stage [18]. To achieve this required control of the lab environment, a high-precision stage, and the image noise had to be reduced via binning and averaging the images. This is a cautionary tale regarding the relative importance of the bias errors versus the other experimental errors. For comparison we show a more typical experimental result in Fig. 33 where the identical setup was used with PointGrey cameras. We conducted the same shifting experiment but without averaging and binning of the images. Here the variance error can be seen to dominate and the interpolation bias error is no longer visible in the results.

Fig. 31

Experimental setup and stage specifications. DIC full-field results show the lens distortions at the last image

Fig. 32

Prosilica subpixel shifting results showing three different interpolants

Fig. 33

Comparison of a typical camera without binning and averaging versus the binned results. Note that the bias error for a typical camera setup is overwhelmed by the variance error (shown as error bars of ± one standard deviation)

A.9 Sample 17 (Out-of-Plane Phase Shifted)

Figure 34 shows the results from the DIC analysis of the out-of-plane shifted images. The u-displacement results shown are after fitting a best fit line to the u-displacement field and subtracting this best fit line from the raw u displacements. This removes the large u-expansion seen and highlights the interpolant error. Each blue to red “sine” wave indicates a linear shift of 1-pixel over the period. The overall expansion of the image then is about 15 pixels in the u-direction (same results in v, although not shown). Calculation of the strain immediately shows the bias error, without need of removing the uniform expansion. Note the marked improvement in switching from a linear interpolant to a bi-cubic spline interpolant in the bias error.

Fig. 34

Sample 17 results for a bi-linear interpolant (top) showing both the u-displacement results (left) and the strain ε_xx results (right) and cubic-spline interpolant (bottom)

Appendix 2

Appendix 2 contains some supplementary figures for Sample 14 that show the full-field results for all the codes.

Fig. 35

Image “Sample 14 L5 Amp0.1.tif” for all the codes as submitted by the participants and decoded and plotted by the LabVIEW code. Note some codes present results all the way to the edge of the FOV. White gaps in the displacement field indicate regions either above or below the scale limits rather than missing data

Fig. 36

Sample 14 L5 full-field strain plots. Again, some bias errors can be seen, and a feel for the relationship between smoothing and noise is intuitively obvious. White gaps in the strain field indicate regions either above or below the scale limits rather than missing data. Note the small vertical strain banding on the left side in the zero strain region in all of the results. The reason for this is unknown, but ubiquitous

Fig. 37

Displacement results for Sample 14 L5. Command, mean, bias and standard deviation curves are color coded in the plots. Typical results are shown