Based on the results of the three tests just shown, several observations can be made on the strengths and weaknesses of the SR algorithms investigated. In addition, benefits and disadvantages of using SR imaging instead of traditional LR techniques are also demonstrated. These characteristics shown by the SR initial comparison, rigid body translation, and mechanical deformation tests are discussed below.
SR Initial Comparison Test
The initial comparison effectively shows some of the qualitative differences between the SR algorithm results. Comparing the images in Fig. 7(b)-(f), all 3 SR algorithms show slight improvements over the LR, but the image quality of each SR image is poor compared to the HR.
In the case of RSR, some features show slight improvement over the LR image, such as recapturing some of the brighter colors on the lighted portions of the leaves. But many of the features become more pixelated than even the LR image, causing a loss of definition. This is evident in the screw at the top of the sign; the circular boundary between screw and sign is more distorted in the RSR image than in the LR image. Although some aspects are visually clearer, the algorithm still adds graininess that is not present in the LR.
In the case of PG, the outline of the blue gaps of sky between leaves is slightly more defined in the PG image. Additionally, the lettering has slightly heavier weight than the LR, making it more comparable to the HR. However, ‘trailing steps’ can be seen leading away to the left from the black-white diagonal border of the sign. This can also be seen in the LR image, but not in the HR image. Rather than smoothing out this boundary to approach the target HR image, the PG amplifies the trailing steps and emphasizes the artificial feature.
In the case of SANC, the appearance seems to fit most closely to the HR image. The lines and boundaries are smooth, and objects are easier to recognize than in either the RSR or the PG images. However, this strength of the SANC image is also its greatest shortcoming. Some details found in the HR image are lost to the smoothing effect, such as the blurring of letter corners.
This comparison test demonstrates that the most recent innovation of the three SR algorithms (SANC) performs best in visually obvious elements of reconstruction, but also highlights some of the potential downfalls in the new application to DIC. Because the SANC algorithm accounts for shape features, it follows that the features in the SANC-reproduced image more closely resemble those found in the HR image. The algorithm distorts the shape of the convolution kernel to match image geometries based on gradient fields. Thus, the SANC image best replicates the features and objects in an HR image in a qualitative ‘eye’ test. But this smoothing of feature boundaries could lead to decreased performance in reconstructing speckle patterns. This could be especially true when individual speckles are relatively small compared to the features that the SANC algorithm is built to search for. And while RSR and PG produce more pixelated shapes, they may still work for speckled patterns used in DIC.
Rigid Body Translation Test
In the rigid body translation test, images from all three of the SR algorithms measure an applied translation as well as or better than LR images. In Fig. 8, the average of all subsets in each image that successfully correlated was within the resolution uncertainty of the translation stage. However, some of the images produced by the PG algorithm failed to correlate, meaning that data was not extractable at all translation increments. Upon inspection of those images which failed to correlate, a checkerboard pattern of black pixels was interspersed in the image. This was also evidenced in the PG images from the mechanical deformation test as seen in Fig. 12. Three different types of this defect were found: A checkerboard of mostly black pixels, as seen on the far left of the figure, a series of black pixel vertical stripes, or a checkerboard of mostly non-altered pixels. This conversion of pixels to black did not appear in the LR original, shown in the right of the figure, or in the other SR images, as seen in the subsets of Fig. 11. This checkerboard pattern also did not appear in the PG images from the initial comparison test with the one-way sign. It may be caused by the PG algorithm, a bug in its implementation in this software , or a combination of both, but it can obviously affect the DIC results or make correlation impossible.
Figure 9 affords a clearer look at both the accuracy and shortcomings of the different methods, by focusing on the error between the measurement and applied translation. All six image sets produced averages that lie close to the line of perfect agreement and within the bounding lines marking the uncertainty of applied translation due to stage increment resolution. At each translation increment, the ring translation was the same for all methods, and all images are centered on the same point. Thus, the unknown error in applied translation due to stage resolution uncertainty is the same for each of the three SR methods and the LR method. Therefore, the spread of bias errors shown in the figure between the six methods at each increment reflects on the precision of the methods. Although all six show some variability in these bias errors from increment to increment, they all show acceptable precision well within the stage resolution uncertainty.
Because of this unknown error in the applied measurement, it is difficult to assess the three SR algorithms based solely on the average displacements in Fig. 9. To provide further clarity, the uncertainty bands can highlight the consistency within each image. Because there is no strain or deformation in this rigid body translation test, the applied motion is nominally uniform and ideal measurement would yield zero variation. At zero translation, the uncertainty bands show the noise floor. As expected, the LR image gives the largest spatial variation, followed by the LR Average and HR Interpolation benchmarks. The three SR algorithms show slight improvement over the benchmarks in their noise floors, with RSR showing the greatest difference.
A significant contribution to this measurement error is the temporal variation from image to image. In particular, the LR image sets are the most prone to temporal variation, or noise. This is because all other methods benefit from combining information from multiple images. To better understand the temporal variation of these inherently noisier LR images, a series of untranslated LR images was correlated, and a specific subset (seen in the left of Fig. 13) was followed through time. The fluctuation of the measured translation of that subset is shown at the right of Fig. 13. This temporal variation helps to quantify the error of the LR measurement associated with the random noise, which is mitigated through averaging in producing the images with the other methods.
At nearly every subsequent translation increment in Fig. 9, the LR also has the largest bands, demonstrating the greatest spatial variation amongst the subsets. The improvement in spatial variation each method offers over LR is given in Table 3. Generally, the size of the uncertainty bands remains relatively uniform from increment to increment for the LR Average and HR Interpolation benchmarks and SR images. However, the RSR varies greatly from increment to increment, showing the best and worst improvement over LR at different increments, as seen in the table. Because the data from the PG image dropped at several increments due to the loss of pixels as shown in Fig. 12, it also has consistency issues. The SANC and benchmarks demonstrate consistency and seems to show the greatest reliability in the accuracy of the measurement, with the SANC generally having uncertainty bands of a size equal to or smaller than the benchmarks.
The differences in spatial variation are further investigated as a function of subset size in Fig. 10. These data are taken from the measurements at the final ring translation increment of 0.782 mm, showing uncertainty band size plotted against the common physical subset size for the LR, benchmark, and SR images. At each comparable physical subset size, the SR algorithms all have smaller spatial variation, giving greater confidence in the measurement. Across all subset sizes, the LR Average benchmark showed a reduction in the band size by a factor of about 3, which aligns with the expectation that averaging N images can reduce noise by a factor of √N . Interestingly, the HR Interpolation benchmark data showed nearly identical band size as the LR Average at the same physical subset size, despite having 4 times more pixels per subset. This suggests that the added resolution through single image interpolation provides ‘empty magnification’ without adding useful information to the LR Average image. In contrast, the SR uncertainty bands are smaller than those of the benchmark data at the same subset sizes. This difference becomes more pronounced as subset size increases, with PG and RSR showing improvement by a factor of roughly 2 at the largest several subset sizes. This reduction of spatial variation seems to be due to more than just the averaging effect on image noise. As one of the main contributors to spatial variation errors is pattern-induced bias error , this may indicate that SR algorithms can improve the quality of the pattern captured.
Also interesting to note in Fig. 10 is the lower limit to subset size; data is left off the plot once the image no longer correlates. The RSR algorithm has a subset size lower limit of 21 pixels, which is the same as the LR has. However, this 21-pixel RSR subset covers a quarter of the area that the 21-pixel LR subset does, allowing much finer strain resolution. Although the LR Average offers a reduction in noise over the original LR images, the lower limit is only slightly improved from 21 to 19. When considering the actual physical subset size, the lower limit of the HR Interpolation is better than the PG and SANC algorithms but worse than the RSR. This superior range of subset size, combined with superior spatial variation error, clearly demonstrates the advantages the RSR algorithm holds over the other methods.
Mechanical Deformation Test
The mechanical deformation of the ring in this test demonstrated the capabilities of the LR and SR methods to measure heterogeneous strain fields. The results show similar displacement contours between the LR and SR images, as seen in Fig. 11. In each case, the ring exhibits a greater displacement gradient on the inside edge of the ring compared with the outer edge. This is consistent with higher strains at the inner edge as the curved ring is stretched and straightened, which is the expected behavior in this loading case. This distribution seems to match up well between the LR and SR. There are slight differences between the contours, however, as both the RSR and SANC appear to show slightly greater displacements on the top of the inner edge of the ring than the LR. The ‘higher resolution’ zoom lens image shows slightly higher displacements than the SR at the same location on the inner edge. However, the fact that the SR contours are slightly closer to the more accurate Zoom contour should not overshadow the reality that all methods produced very similar results.
The subsets shown in the bottom of Fig. 11 do give some insight into the differences between the various methods. The LR subset shows a typically pixelated speckle pattern. The RSR subset also looks pixelated, although there is a difference in the gray levels of the speckle borders. This seems to indicate a more gradual transition from the dark interior of the speckle to the lighter background. A similar effect is seen in the SANC subset, except that these transition zones are much smoother, leaving speckles that are more ‘bloblike’ than blocky. Neither of the SR methods approached the clarity of speckle offered by the Zoom lens with roughly 15 times the resolution of the LR images, as seen in the Zoom subset.
Pairing of Zoom Lens with Super Resolution
The use of the zoom lens in the mechanical deformation test served as a standard of comparison for the SR algorithms, demonstrating that SR is equally capable in deformation measurements, if not offering slight improvement. It follows that pairing the two methods of higher resolution, a zoom lens and SR techniques, could produce further improvement. That possibility was evaluated by combining multiple zoom lens images at each grip displacement increment using the RSR and SANC algorithms, and then using Vic-2D to produce DIC strain contours. The RSR contour failed to correlate, but the SANC contour is shown in Fig. 14.
The contour shows good agreement with the LR zoom lens results of Fig. 11, with similar distributions of displacements at comparable locations. However, it should be noted that the same speckle pattern is on the ring for the original LR images and these SR zoom images, meaning that the speckle size and length scale of the pattern is ill-suited for such an increase in resolution, as can be seen in the subset of Fig. 14. Because the zoom lens’ field of view is roughly 15 times smaller than the lower magnification lens, and because an interpolation factor of 1.5 was used for the SR zoom images, the pixel size of speckles in Fig. 14 is approximately 23 times larger than in the LR contours of Fig. 11. As such, the results of the SR zoom images are limited by the speckle pattern. This is expected to be the reason that the RSR zoom images failed to correlate in the DIC software. However, this does highlight that SR can be paired with lenses of varying magnification, as long as the length scale of the speckle pattern is appropriate for the image resolution.
Summary of Algorithm Performance
Through the three tests, differences in performance between traditional LR images and the 3 distinct SR alternatives became apparent. The initial comparison with the one-way sign showed clear improvement from the LR images for SANC, and to a lesser degree the PG and RSR also showed some improvement. However, none perfectly replicated the original HR image. Rigid body translation began to show problems with pairing PG with a DIC speckle pattern, dropping some of the translations. The minimal error in both RSR and SANC measurements showed good accuracy, although the uncertainty bands of SANC showed better consistency than was offered by the RSR algorithm. Conversely, RSR showed the smallest physical subset size, offering the best spatial resolution of the displacement measurement and the lowest spatial variation error. In the mechanical deformation, the issues with PG and DIC became very clear, causing all images to fail to correlate. In pairing SR with the zoom lens, the SANC image set was the only one of the three which was able to successfully correlate to the final deformation (Table 4).