Multi-view Digital Image Correlation Systems for In Vitro Testing of Arteries from Mice to Humans

Abstract

Background

Digital image correlation (DIC) methods are increasingly used for non-contact optical assessment of geometry and deformation in soft tissue biomechanics, thus providing the full-field strain estimates needed for robust inverse material characterization. Despite the well-known flexibility and ease of use of DIC, issues related to spatial resolution and depth-of-field remain challenging in studies of quasi-cylindrical biological samples such as arteries.

Objective

After demonstrating that standard surrounding multi-view DIC systems are inappropriate for such usage, we submit that both the optical setup and the data analysis need to be specifically designed with respect to the size of the arterial sample of interest. Accordingly, we propose novel and optimized DIC systems for two distinct ranges of arterial diameters: less than 2.5 mm (murine arteries) and greater than 10 mm (human arteries).

Methods

We designed, set up, and validated a four-camera panoramic-DIC system for testing murine arteries and a multi-biprism DIC system for testing human arteries. Both systems enable dynamic 360-deg measurements with refraction correction over the entire surface of submerged samples in their native geometries.

Results

Illustrative results for 3D shape and full-surface deformation fields were obtained for a mouse infrarenal aorta and a latex cylinder of size similar to the human infrarenal aorta.

Conclusion

Results demonstrated the feasibility and accuracy of both proposed methods in providing quantitative information on the regional behavior of arterial samples tested in vitro under physiologically relevant loading.

Introduction

The biomechanical properties of excised arteries have been studied intensely over many decades using vessels from humans as well as many different animal models [1, 2]. Such studies promote understanding of the alterations in biomechanical properties that result from microstructural changes during adaptations, disease, and injury, thus promoting improved diagnosis and prognosis. Moreover, connecting evolving biomechanical properties with changes in cell signaling is fundamental for understanding the associated mechanobiology, which promises to provide new insights into methods for preventing or treating disease, including regenerative medicine approaches. Notwithstanding the many advances in arterial material characterization to date [3], investigators have often focused on excised strips for uniaxial testing [4, 5], slabs for in-plane biaxial testing [6, 7], or cylindrical segments for distension-extension testing [8, 9]. Yet, many of the critical pathologies of interest, including aneurysm, stenosis, and tortuosity, are characterized by complex three-dimensional (3D) geometries and material heterogeneities that are not amenable to these standard methods of experimental biomechanics. There is, therefore, a pressing need for new methods to quantify regional material properties over complex domains [10,11,12,13].

Digital image correlation (DIC) is a flexible approach for measuring complex 3D shapes, motions, and deformations, with relatively straightforward data processing, that has become widespread in experimental mechanics [14, 15]. Because of its high spatial resolution and non-contacting nature, DIC is particularly well suited to collect full-field shape and deformation data during in vivo, in situ, or in vitro experiments in soft tissue biomechanics. Of particular note is stereo-DIC, which requires a pair of angled views of a region of interest (ROI) on the sample surface to reconstruct 3D positions of a given set of points. Using a temporal sequence of configurations of interest, the motion/deformation of the reconstructed point set can be tracked in 3D. To this end, the DIC algorithm finds the correspondence between homologous point pairs in the two stereo-images by comparing the grayscale distribution of square pixel subsets on the basis of a normalized cross-correlation coefficient [14]. The two sets of matched image points are then used within a perspective camera model to find the corresponding 3D object points, provided that the internal and external parameters of the two cameras are available through proper calibration [16, 17]. Finally, from the 3D position of the points, it is possible to extract the full-field 3D displacement and strain maps across the ROI with spatial resolution (at a given magnification) depending on the adopted pixel subset size and point grid spacing [14].

DIC-based testing of quasi-cylindrical arterial samples requires an enlarged, 360-deg surrounding field-of-view (FOV) that cannot be obtained with a standard two-camera stereo-DIC setup. In addition, arterial samples need to be immersed in a physiological solution to preserve their biomechanical properties. This latter requirement introduces a further challenge for optical measurements; all acquired images will be distorted by light refraction at the air–liquid interface. Moreover, the curvature of arterial samples reduces the actual angular extension of the ROI common to stereo-views and affects the accuracy of the DIC registration. Finally, the unavoidable conflict between spatial resolution and depth-of-field (DOF) makes measurement particularly cumbersome for small caliber samples. Whereas we ultimately seek to understand human disease and its treatment, thus necessitating tests on human samples, the many advantages of mice (including genetic manipulation, short gestational periods, and ease of longitudinal studies) often render it the animal model of choice in vascular biology and mechanics. There is, therefore, a need for consistent measurement over length scales characteristic of human and murine arteries.

In the following section, the main limitations of standard multi-view DIC systems are summarized to highlight the need to design size-specific optical systems for 360-deg measurements on quasi-cylindrical samples. Next, two different panoramic-digital image correlation (pDIC) systems developed specifically for testing murine and human arterial samples are described and analyzed, with illustrative results shown for a mouse aorta (~ 1 mm diameter) and a human vessel surrogate (~ 15 mm diameter thick-walled rubber tube). For both systems, the refraction error was corrected, though with two different strategies, thus confirming the major advantages of size-specific methods and data processing procedures.

Materials and Methods

Multi-camera DIC Measurements on Submerged Objects: Issues and Challenges

The most intuitive way to perform stereo-DIC measurements [14] over a large FOV is to arrange a cluster of cameras in a ‘wall’ [18] or a ‘surrounding’ [19] configuration (see [15] for a comprehensive review of multi-view DIC systems). For 360-deg measurements, n cameras are arranged in a surrounding configuration, namely, evenly spaced in a polar array to span the full panoramic view around the object of interest. Each pair of contiguous cameras acts as a conventional stereo-DIC system that acquires an image pair of a common ROI on the sample surface. Specific stereo-camera calibration procedures are then used to reconstruct the n partially overlapping 3D data patches relative to a single reference frame [19, 20]. To gather accurate full-surface information, the angular spacing between the cameras and the working distances need to be set properly with respect to the shape and the size of the sample [21] with the lens settings enabling in-focus acquisition of the ROI with sufficient spatial resolution [22]. The tight relationship between the camera angular spacing and the object shape can be inferred by reviewing the related literature; the required number of cameras is four for testing quasi-flat surfaces [19], twelve for quasi-cylindrical objects [20], and twenty-one for more complex shapes [23]. For the purpose of this study, however, we can narrow down the analysis based solely on the size of a straight cylindrical sample since it is representative of the case of interest.

Figure 1 illustrates two possible configurations for 3D measurements on a cylindrical surface with stereo-DIC. In particular, Fig. 1(a) depicts a stereo-system with the camera baseline perpendicular to the cylindrical axis and a 90° stereo-angle (same in Fig. 1(b) with the camera baseline parallel to the cylindrical axis). The region of the cylindrical surface acquired by each camera can be found by tracing the planes passing through the pin-hole and tangent to the cylinder. The intersections of these planes with the sensor plane delimit the sample silhouette that encloses the acquired cylindrical sector. Only a fraction of this sector is common to both views (delimited in Fig. 1(a, b) with a yellow line) and can potentially be used for the stereo-measurement. Another possible limitation to the extension of the common ROI is introduced, however, by the DOF of the lenses of the cameras. By considering the common ROI fully included within the volume in focus (delimited in Fig. 1 by a square prism), note that its size is comparable for the two configurations although with less extent along the circumferential direction of the sample in Fig. 1(a) and along the axial direction in Fig. 1(b).

Fig. 1

(a) Schematic view of a stereo-DIC system with camera parallax (baseline) perpendicular to the cylindrical sample axis and a 90° stereo angle. $h$ is the horizontal size of the sample image obtained by intersecting the two planes ${\pi }_{LT1}$ and ${\pi }_{LT2}$ tangent to the cylinder and passing through the pinhole ${PH}_{L}$ with the sensor plane (similarly for the right camera). The common ROI (in yellow on the sample surface and the sensors) is delimited by the two lines ${l}_{L}$ and ${l}_{R}$ that are the left (right) limits of the cylindrical sector acquired by the right (left) camera. The common ROI occupies only a fraction of the 360° cylindrical surface but is contained within the volume in focus delimited by the depth-of-field ${DOF}_{L}$ and ${DOF}_{R}$ of the left and right camera lens, respectively. (b) Schematic view of a stereo-DIC system with camera parallax parallel to the cylindrical sample axis and a 90° stereo angle. In this case, the common ROI is delimited by the two lines ${l}_{S}$ and ${l}_{I}$ (the latter is hidden) obtained as above. The common ROI now occupies a larger angular portion of the cylindrical surface but its length is not entirely contained within the volume in focus of the stereo-system. Moreover, corresponding areas of the common ROI (e.g. around points ${P}_{L}$ and ${P}_{R})$ may have significantly different spatial resolution. For clarity of representation, neither schema is to scale. Although this analysis focuses on a cylindrical sample for simplicity, the general method applies to quasi-cylindrical samples as seen in the experimental cases considered below

To simplify the analysis, without loss of generality, hereafter we refer to multi-view systems in which the cameras are arranged with their parallax perpendicular to the cylinder axis (as in Fig. 1(a)). To further reduce the complexity of the analysis, we also refer to characteristics of the industrial-grade camera used in our experimental setups (Sony-IMX179 sensor, 3264 x 2448 pixels with 1.4 µm pixel size, 5–50 mm varifocal lens). First, we constrain the analysis by imposing a sufficient lens magnification to ensure an accurate image correlation and 3D data reconstruction [22]. In particular, we impose a ratio between the focal length $f$ and the distance to the object $D$ so that the horizontal size of the image of the sample $h$ (see Fig. 1(a)) fills at least one-half of the sensor ($h=1632$ pixels, $M=f/D=0.18$). Increasing the magnification further would improve the spatial resolution within the ROI, but would be detrimental in terms of the DOF, which in this case is already just slightly larger than the cylinder radius.¹ Given these assumptions, and using the classical pinhole camera model of Fig. 1(a), it is straightforward to quantify the angular extension of the common ROI over the cylindrical surface as a function of the distance to object $D$ and the stereo-angle $\theta$ (Fig. 2(a)) within the allowed range of focal lengths, 5–50 mm, at a constant magnification $M$. Figure 2(b) shows this function for a 12.7 mm diameter cylinder since an accurate steel post of this size was later used for experimental validation and is representative of the typical size of a normal human female infrarenal aorta (~ 15 mm diameter). Unlike what might be thought, a four-camera configuration is inadequate for a 360-deg measurement because it does not cover the full cylindrical surface (common ROI < 90°, see also Fig. 1(a)).² Among other possible options with a common ROI ≥ 90°, if the overall dimensions of the optical system and the number of cameras need to be minimized, the optimal solution consists of five cameras evenly spaced at $D\cong$ 74 mm (point M in Fig. 2(b)), that is, the minimum distance at which five cameras can be placed together closely. A detailed step-by-step description of the procedure used to construct the curves in Fig. 2(b) and related equations is given in Supplementary Material.

Fig. 2

(a) Schematic drawing showing three different multi-view surrounding setups featuring the camera used in this study (pictured in the first panel). ${D}_{min}$ is the minimum distance to the object at which the cameras can be placed. Note that ${D}_{min}$ increases with the number of cameras. (b) Plots of the angular extent of the ROI common to the views of a stereo-DIC system as a function of the working distance and the stereo angle θ for a 12.7 mm diameter cylinder at a constant magnification. The minimum working distance (plotted as a dashed line) depends on the size of the camera-lens assembly (panel a) and the stereo angle (i.e., number of cameras n). M indicates the optimal solution for acquiring the entire 360-deg ROI (i.e. common ROI > 90°) with the minimum number of cameras and the smallest overall dimension of the multi-view system

Such an approach can be used to simulate a wide range of experimental conditions to identify the optimal setup and anticipate the achievable results. It must be noted, however, that the above theoretical analysis yet overestimates the required stereo-angle between cameras since it disregards the limitations entailed in a subset-based DIC analysis [24]. To illustrate the limitations of the subset-based DIC analysis used in this work, we will refer to the multi-biprism camera configuration presented later and pictured in Fig. 10 (see Fig. 3 caption for details on the optical settings and DIC parameters). Figure 3(a) shows a portion of the right and left views of a 12.7 mm diameter cylindrical post with a speckle pattern as seen by two contiguous cameras. By looking at the real image pairs in Fig. 3(a) and at the synthetic images with a regular dot pattern in Fig. 3(b), it is evident that the two image subsets with common ROIs (denoted in green and red) present large nonlinear inter-frame deformations. This effect is particularly evident for large curvatures (i.e., for small caliber samples) and it requires the adoption of second-order shape functions [25, 26] to enable accurate DIC image registration and 3D data reconstruction [27]. More importantly, due to the presence of object/background boundaries, image correlation approaches can yield erroneous or inaccurate results wherever the subset is not fully contained within the sample silhouette (e.g., red subset, Fig. 3(b)). Figure 3(c) shows the error on the identified radius over portions of the cylindrical sample reconstructed with our DIC code. Due to the above limitations, the actual common ROI that is valid in terms of an accurate DIC measurement (error on radius < 0.05 mm) is significantly narrower than its visible counterpart (e.g., for the image pairs in Fig. 3(a), the usable angular portion is only 26% of the common ROI). Even in this case, it is easy to infer how these detrimental effects become more evident for small ROI/subset-size ratios [26]. Namely, for small sized samples at a given magnification.

Fig. 3

(a) 350 × 350 pixels portions of the left and right images of a 12.7 mm diameter post as seen from a pair of contiguous cameras in the setup later reported in Fig. 10. Stereo-angle is 90°, working distance is 260 mm, and focal length is 7.6 mm. The common ROI has a horizontal size of 129 pixels and covers an angular portion of about 87° on the sample surface. Yet, due to the presence of borders, for the large relative deformation between image pairs, and to the finite size of the subset (here 42 × 42 pixels), DIC analysis provides accurate results only within a narrow portion of the visible common ROI (e.g. subset pairs in red cannot be accurately correlated). (b) Synthetic images highlight the large relative deformation between the two lateral views of a 12.7 mm diameter post with a regular dot pattern under the same experimental settings. Accurate image correlation can be obtained only over the ~ 23° portion of the cylindrical surface delimited by the dotted lines. (c) 3D shape portions of the 12.7 mm diameter post, as reconstructed with the four-camera configuration in Fig. 10, with the error on radius overlaid with respect to the nominal sized cylinder

From the arguments above, it follows that in order to acquire common ROIs with low relative deformation that are far from the boundaries and at the maximum magnification to ensure a sufficient DOF, the number of views should be increased. Yet, because of the physical constraints introduced by the size of the camera-lens (Fig. 2(a)) and/or by the experimental apparatus (e.g. columns of the loading frame in [28]), the optimal solution can often be obtained only by resorting to a pseudo multi-view arrangement (e.g. $n=14$ for a 10 mm diameter sample at $D\cong$ 150 mm in [28]). In such an approach, a single camera (or a stereo-camera set in the configuration of Fig. 1(b), [29]) is sequentially placed at $n$ angular positions around the sample of interest to acquire the required number of stereo-images. This solution permits more flexibility with respect to physical constraints, poses no limits on the number of views, and reduces the equipment costs (especially when highly expensive lenses are used, [29]). On the other hand, a calibration target may need to be included in the scene [28] and dynamic testing is no longer possible (one has to stop the deformation of the artery while moving the camera, [29]). As a final option, mirror-assisted multi-view DIC systems [30] can be employed for dynamic panoramic measurements, though only on large caliber samples because the entailed wide FOV and large DOF conflict with the high spatial resolution of the ROI required for an accurate DIC measurement [31].

All of the aforementioned issues should be considered when designing a pDIC system for testing arteries. Moreover, as an inherent consequence of submerging the arterial samples in a physiological solution, the important effect of light refraction [32] must be quantified and corrected. In fact, whenever an interface between two media having different refractive indices is interposed in the light ray path to the camera sensor, two important effects degrade the quality of the acquired image: image distortion and chromatic aberration. The latter effect can be minimized sufficiently by band-pass filters to illuminate and/or acquire within a narrow range of wavelengths. By contrast, the image distortion caused by refraction remains a challenging problem. Such a distortion cannot be corrected with the algorithms commonly used in camera calibration routines since they assume that image formation follows a perspective model. Indeed, one can demonstrate through ray tracing that, due the bending of light at an air–liquid interface, the hypothesis of the single-viewpoint (SVP) imaging system is no longer valid (see Fig. 2 in [33]).

Different approaches have been proposed by the computer-vision community to address the problem of refraction. They can be divided in two main categories: ray-tracing approaches that model the underlying physics described by Snell’s law [34, 35] and approaches relying on modified SVP camera calibration procedures (run in air or underwater) that can practically counteract the refraction error [29, 36,37,38]. In this work, we introduce two different methods from the latter category and discuss their related advantages and challenges.

Size-specific Multi-camera DIC Systems for Testing Arteries

From the discussion above, it is evident that relationships between the required camera-lens characteristics and the sample size are strong limiting factors against adoption of a universal multi-view DIC system for testing arteries. While a properly designed surrounding multi-camera system as depicted in Fig. 2 remains a viable approach for human arteries, it is not feasible to simply size it down for testing murine arteries because of the strict requirements of ROI spatial resolution and DOF. This problem motivated the development of the pDIC method, first proposed in [39] and successively improved until its most recent version [21]. In pDIC, the sample is placed coaxially within a concave conical mirror that transforms a conventional binocular stereo-DIC system, capturing the reflected scene from above, into an infinite-view surrounding system covering the full 360-deg sample surface (see [21] for an extensive description and related image-formation scheme). As detailed in [39], this optical arrangement allows a > 10 × amplification of the ROI spatial resolution that can be obtained with low magnification (and hence large DOF) lenses. Moreover, the subset-based DIC processing of the panoramic image is devoid of boundary effects aside from regions on the upper and lower ends of the sample that are beyond the securing ligatures and thus not used for extracting data significant for the analysis.

The pDIC method presents some important limitations though. Because of the large uneven image distortion introduced by the curved specular surface, the actual stereo angle $\varphi$ differs from the real angle $\theta$ and varies across the panoramic image. In particular, $\varphi$ is largely amplified with respect to $\theta$ everywhere within the panoramic image with the exception of the region in proximity to the line of intersection between the stereo-plane and image plane (hereafter referred as the stereo-line). The real stereo angle $\theta$ thus needs to be kept small to avoid generating large relative deformations across and between images, which implies that large uncertainty in the 3D data reconstruction is expected for image points close to the stereo-line, where $\varphi \cong \theta$ [21]. This inherent limitation of pDIC was previously addressed by implementing a modified image morphing (MIM) method that enables DIC registration between two image pairs with large heterogeneous deformations [21]. Using MIM processing, the stereo angle $\theta$ can be increased up to about 10° thus keeping the reconstruction uncertainty at the stereo-line below an acceptable threshold. As a direct consequence of this upgrade, the original pseudo multi-view pDIC system requiring quasi-static loading could be reduced to a conventional (i.e. fixed) two-camera system, thus capable of performing dynamic measurements.

Notwithstanding its enhanced capabilities, the two-camera pDIC system presented in [21] can still be improved and customized for testing murine arteries. Indeed, very small stereo-angles ($\theta <1^\circ$) are required for testing murine arteries (often 0.5–2.0 mm diameter) since the degree of amplification of $\theta$ into $\varphi$ (and related intra- and inter-image deformation) increases with decreasing sample size and gives rise to smeared regions in the most peripheral portion of the panoramic images. Keeping $\theta$ below 1° is thus a strict requirement, although this inevitably conflicts with the need to keep the value of uncertainty low for the points on the stereo-line [21]. This important issue is addressed herein by upgrading the two-camera pDIC system of [21] to a four-camera pDIC system specifically designed for dynamic testing of small caliber arteries. This upgraded version of the optical setup overcomes all the limitations entailed in the prior pDIC method while retaining the ease of use typical of conventional DIC methods.

Regarding the testing of human arteries (typically > 10 mm diameter), the two-camera pDIC system presented in [21] can be satisfactory. Nonetheless, development of a more conventional multi-view setup would facilitate integration of a 360-deg DIC measurement within already existing experimental settings that use universal testing machines and standard optomechanical components. As a practical alternative to the two-camera pDIC setup, we designed and validated a novel biprism-based four-camera setup for testing large arteries submerged in physiological solution.

A 4-camera pDIC System

Figure 4(a) shows the conceptual schematic of the four-camera pDIC system and its arrangement on the optical table (Fig. 4(b)). In both subfigures, the stereo angle $\theta$ between the four cameras and the conical mirror axis was exaggerated for the sake of clarity, but is actually slightly less than $1^\circ$ (though undetectable in the picture of the system in Fig. 5). Since the required stereo angle is very small, three beam splitters³ were used to arrange the four cameras to have near coincident axes (see schema in Fig. 4(a)). The simultaneous use of four cameras, instead of two [21], was chosen to implement a multi-parallax reconstruction scheme [40] that dramatically reduces measurement uncertainty for points lying on the two stereo-lines of the two camera pairs, 1–3 and 2–4. Figure 5 shows a picture of the experimental setup that, apart from the high precision manufactured conical mirror, uses off-the-shelf components (see Supplementary Material for a list of the optomechanical components and a technical drawing of the conical mirror). Although not strictly required, it is highly recommended to arrange the four cameras precisely in accordance with the polar symmetric layout of Fig. 4(a). In fact, if the four cameras have ‘identical’ settings (focal length, aperture, working distance, stereo-angle), the four acquired images show an evenly symmetric relative deformation that can be easily handled by the MIM data processing method [21]. Conversely, since the curved reflective surface of the conical mirror amplifies misalignments, small deviations from the ideal schema of Fig. 4(a) can lead to an uneven and/or significant deformation of the images, thus compromising the DIC registration. To prevent the latter, each component of the setup in Fig. 5 needs to be mounted on a multi-axis stage to allow fine tuning of the related pose.

Fig. 4

(a) Schematic of the four-camera pDIC system for testing murine arteries. Each camera axis forms an angle $\theta \cong 1^\circ$ with the axis of the conical mirror. (b) System layout allowing a clustering of the four cameras close together according to the optical schema of panel a. Each ray from object to camera is first reflected by a 45° mirror prior to encountering two beam splitters where it can be either transmitted or reflected before reaching the camera sensor (see e.g. path $A\to B\to C$ for camera 2)

Fig. 5

The actual four-camera pDIC setup for studying murine arteries: A- conical mirror, B- sample bath, C- two-axis stage, D- 45° mirror mounted on a kinematic mount, E- annular lamp, F,G,H- beam splitters, I- three-axis stage, L- kinematic mount

System alignment is performed as follows. The conical mirror is fixed onto a kinematic mount within the bath to adjust its $z$-axis perpendicular to the surface of the physiologic solution (the sample and tubing used for its pressurization are not shown in Fig. 5; see, pictures and description in Supplementary Material). The cone-bath assembly can then be moved to set the $x,y$ position of the cone vertex via a two-axis stage (Fig. 5; component C). A 45° mirror (Fig. 5; component D) mounted onto a kinematic mount rotates the conical mirror view by 90° thus allowing an easier and more stable horizontal layout of the cameras across the optical bench. As a first step, the four cameras are placed and adjusted with their axis coincident with the $z$-axis (dotted lines in Fig. 4(b)) via their individual six-axis stage (three-axis translations (Fig. 5; component I) plus two rotations through the kinematic mount (Fig. 5; component L) plus a further rotation around the mounting post. In this starting position, after setting correctly the focal lengths and working distances, the four cameras acquire identical scenes. In fact, the layout is designed to have the sample uniformly illuminated by an annular lamp (Fig. 5; component E) and the same number of reflecting/transmission interfaces on the light path of each camera. It follows that the four views possess the same level and quality of illumination. A simple digital subtraction between images can help ensure an optimal starting configuration. Once this configuration is set, each camera is firstly translated along its assigned parallax and then gently rotated about its mounting post (through the light blue knobs in Fig. 5) by the required angle $\theta$ thus obtaining the four angled views needed for the multi-parallax stereo-triangulation [40].

The cameras are calibrated prior to testing and again at the end of the experiment to avert possible system drifts. Refraction error can be corrected with any of the approaches presented and validated in [35] and [37]. Herein, we first used standard calibration routines in air [16] to estimate the intrinsic parameters of each camera needed to undistort the acquired images. Then, once the sample bath was filled with physiologic solution, a single image of the 3D calibration pattern on the outward-facing surface of the conical mirror (see Fig. 5; component A and a typical panoramic image in Supplementary Material with related description) was acquired, undistorted, and then processed with the Direct Linear Transformation (DLT) method [17]. This process allowed calculation of the external camera parameters, namely, estimation of the relative pose between the camera and the global reference system defined by the calibration pattern. Since a dot pattern can be engraved on the outer portion of the conical mirror and measured with high accuracy, the pose of each camera is also known with respect to the reflective surface and later used for stereo-triangulation [39]. Implementation of the DLT method in the presence of refraction is not justified theoretically since it refers to a linear SVP image formation model. We nonetheless verified with a calibrated cylindrical sample that DLT can accommodate refraction error in this application for the points within the ROI (1930 × 1930 pixels) with an overall 3D reconstruction accuracy of the order of ${10}^{-2}$ mm. This forgiving calculation is likely because i) the panoramic ROI and surrounding calibration target occupy the most central region of the image, ii) the focal length is relatively short (about 20 mm), iii) the physiologic bath is shallow (about 10 mm depth), and iv) the stereo-angle (hence the angle of the light ray at the air–water interface) is less than 1°. We later show, however, that the same approach applied to different experimental settings reported in the multi-biprism 360-deg DIC system results in reconstruction errors of at least one order of magnitude larger.

Following acquisition of the desired sequence of the deformed states of interest, the panoramic images are processed with the MIM method described in detail in [21]. Briefly, after an image sequence is obtained as the reference ${I}_{REF}$ (e.g., cam1 image of the undeformed state), the gross deformation is identified with a feature-based approach between ${I}_{REF}$ and the rest of images from the same configuration (cam2,3,4 images) and those from a given configuration of interest (e.g., cam1-4 images of the i^th deformed state). In particular, the Scale Invariant Feature Transform (SIFT) enables a sparse set of corresponding points to be retrieved even in highly deformed areas of the ROI [41, 42]. Maps of disparities ${u=x}_{TAR}-{x}_{REF}$ and ${v=y}_{TAR}-{y}_{REF}$ are obtained with SIFT between a given target image ${I}_{TAR}$ and the reference image ${I}_{REF}$ and are first filtered from outliers and then fit with a stiff non-uniform rational B-spline (NURBS) over the entire ROI. The value of disparities $u$ and $v$ is later extracted for each pixel of the ROI via interpolation from these approximate distributions. The pixel-wise estimated disparities are hence applied to ${I}_{TAR}$ to warp it into a ‘morphed’ image ${I}_{MORPH}$ more similar to ${I}_{REF}$ than the original target image. This latter pair of images is then registered with DIC to give the set of corresponding points ${(x}_{REF},{y}_{REF})$ and ${(x}_{MORPH},{y}_{MORPH})$. Hence, the inverse of the warping functions is applied to the set of points ${(x}_{MORPH},{y}_{MORPH})$ to map them back to the corresponding ${(x}_{TAR},{y}_{TAR})$ in the target image domain. Finally, the set of point pairs ${(x}_{REF},{y}_{REF})$ and ${(x}_{TAR},{y}_{TAR})$ are used again to calculate an updated estimate of the disparity maps $u$ and $v$ and the process is repeated until reaching convergence. Since $u$ and $v$ are calculated at each iteration with an increasing level of accuracy, the difference between the warped synthetic image ${I}_{MORPH}$ and the reference image ${I}_{REF}$ progressively decreases until theoretically vanishing. Actually, due to differences in illumination between views, and the image grey level quantization and interpolation, the above images could never be identical, thus the convergence criterion is set by defining an adequately small threshold for the distance between the points ${(x}_{REF},{y}_{REF})$ and ${(x}_{MORPH},{y}_{MORPH})$ that should theoretically be equal to zero.

Herein, we added a further block to the image processing flow that automated the MIM procedure. To remove outliers and poor accuracy matches (points with a correlation coefficient below a given threshold), we applied a ‘reverse’ correlation analysis. At the end of each correlation step between the images ${I}_{REF}$ and ${I}_{MORPH}$, the calculated points ${(x}_{MORPH},{y}_{MORPH})$ were considered for a new correspondence search between ${I}_{MORPH}$, taken this time as the reference image, and ${I}_{REF}$ treated as the target image. Out of the set of ‘reversed’ matches ${(x}_{MORPH},{y}_{MORPH})$ and ${(x}_{REF}\text{'},{y}_{REF}\text{'})$ only the subset for which the distance between ${(x}_{REF}\text{'},{y}_{REF}\text{'})$ and the original ${(x}_{REF},{y}_{REF})$ was below a given threshold was kept for calculating subsequent disparities maps. As expected, this automated procedure requires a larger number of iterations with respect to the case in which the user defines (subjectively) an appropriate set of matches to discard at each iteration [21]. Even if the disparity maps converge toward the final ones at smaller increments, this approach yet proved to be more robust with respect to isolated outliers in the disparity maps that can easily pass unnoticed and cause the warping functions to locally diverge.

Illustrative results of shape and deformation measurements performed with the above proposed four-camera pDIC method are presented for a non-cylindrical murine abdominal aortic segment relative to both an undeformed and a largely deformed (${p}_{def}$ = 120 mmHg) state. These two faraway states were chosen to demonstrate the ability of the MIM method to correlate images with a large relative deformation directly, that is, without the need to step through a series of small incremental deformations as most commonly found in DIC [43]. Figure 6(a) shows the external surface of the sample in its undeformed state from optical coherence tomography (OCT), its radial size (Fig. 6(c)), and the reconstructed undeformed and deformed shapes from pDIC for different views (Fig. 6(b) and Fig. 6(d), respectively). 3D data in Fig. 6 are raw data, that is, without filtering or smoothing of the DIC data (18 pixels subset size, 2 pixels grid spacing, NURBS-based shape functions, cubic grey level interpolation). A small percentage (3.4%) of invalid points were automatically removed from the original set of 38,766 measured points by reversing the correlation analysis as described above. Most invalid points were located at or beyond the securing ligatures due to the presence of loose ends of the threads and/or to a low quality speckle pattern (see data voids in Fig. 6). Note that the enhanced spatial resolution of the panoramic images enables resolution of the ligated branches, thus opening the possibility for insight into their possible effect on the arterial deformation.

Fig. 6

(a) Rendering of the external surface of a murine aortic sample in its undeformed state from OCT imaging, (b) pDIC reconstructed shape of the sample in its undeformed state under different views, (c) plot of the radius of the sample in its undeformed state; d) pDIC reconstructed shape of the sample at ${p}_{def}=$ 120 mmHg under different views

Figure 7 reports the full-surface distribution of the first and second principal Green strains (E₁ and E₂) for the artery at ${p}_{def}=$ 120 mmHg, with strains calculated as described in [39]. The distribution of these principal strains is similar to that for the circumferential and axial strains, respectively (not shown). Although no filtering was applied, the strain distributions were smooth, in part, due to the large size of the subset used for this analysis (42 pixels) and, mainly, to the hybrid local/global nature of the MIM method. In fact, although the DIC matching between the images ${I}_{MORPH}$ and ${I}_{REF}$ was performed with a local subset-based approach, the resulting disparity maps were later fitted with NURBSs, thus enforcing continuity of the deformation fields over the entire ROI, as is the case for global DIC.

Fig. 7

Distribution maps of the first and second principal Green strains (E₁ and E₂) for the murine aortic segment at ${p}_{def}=$ 120 mmHg under different views

Our novel pDIC method thus overcomes drawbacks of the previous version [40] while still providing full-field, dense sets of shape and deformation data across the entire surface of murine arterial samples with diameters in the range 0.5—2.5 mm and a maximum length of 10 mm. More specifically, there is no more need for: i) revolving the camera around the cone axis to acquire the required number ($\ge 4$) of stereo-images, ii) calibrating each image to extract the moving camera pose with respect to the reference frame, or iii) collecting image sequences of slowly evolving deformed states to cope with the large inter-frame deformation through a reference-updating scheme. It should be noted, of course, that careful set up and alignment of the optical system setup remains critical.

The Multi-biprism 360-deg DIC System

Although pDIC can be effectively used to test large sized arterial samples in air [21], in this section we propose a straightforward alternative for performing dynamic 360-deg measurements on quasi-cylindrical samples immersed in physiological solution. This method is a variant of the surrounding multi-camera system (Fig. 2(a)) with the dual purpose of reducing the required number of views (see discussion in Multi-camera DIC measurements on the submerged objects: issues and challenges) and correcting refraction. Figure 8 illustrates the rationale behind the proposed method. If an object is inserted at the center of a prismatic box filled with water and a camera acquires the scene with its optical axis bisecting the bath corner, a split image of the sample forms in the two halves of the sensor as a consequence of the refraction of light passing through the liquid wedge. This effect, called hereafter the ‘biprism-effect’, has previously been exploited in DIC to build pseudo-stereo systems to measure 3D surfaces using a single camera and a quartz wedge prism [36]. Following this rationale, we exploit the unavoidable presence of light refraction occurring at the bath walls to halve the number of cameras required to cover the full 360-deg sample surface with unaltered spatial resolution and DOF.

Fig. 8

Picture showing the rationale behind the multi-biprism method. If an object immersed within a transparent bath is viewed by a camera with its axis bisecting the liquid wedge, two split images of the sample are generated by refraction. By properly processing these stereo-images, it is possible to perform a 3D measurement with a single camera thus reducing the number of cameras in the array of a surrounding multi-view system

Figure 3 and prior discussion demonstrated that four cameras arranged evenly in polar array are insufficient for a full panoramic measurement on a 12.7 mm diameter cylindrical sample (only $\sim 23^\circ$ can be accurately processed with stereo-DIC for each camera pair). If, however, the same four equi-spaced cameras are arranged with respect to a square prismatic bath as shown in Fig. 9, the biprism-effect practically doubles the actual number of views by generating a pair of virtual cameras for each view. Ray-tracing in Fig. 9 further shows that the two virtual cameras form an acute angle meaning that the angular extension of the overlapping ROIs in the two virtual views is large (see curves in Fig. 2(b)) and the inter-frame deformation is moderate. In other words, the biprism effect not only ensures full coverage of the sample surface with a reduced number of cameras, it also improves the accuracy of the DIC measurement provided that image distortion caused by refraction is properly corrected.

Fig. 9

Schematic of the multi-biprism setup for testing large human arteries (not to scale). Ray tracing shows that, for each view, a pair of virtual cameras results from the light refraction at the liquid wedge-air interface

Because it shows only two refracted rays per view, the schematic of Fig. 9 does not reveal another important effect of the biprism system: the virtual camera is not a single perspective camera since the refracted rays do not pass through a single pin-hole before reaching the camera sensor but instead pass through a distributed set of pinholes. This has the important consequence that the image distortion caused by light refraction cannot be corrected with the algorithm embedded in common camera calibration routines since they rely on the SVP perspective model. Thus, as anticipated, the DLT-based refraction correction approach used for the four-camera pDIC system yielded unsatisfactory results in this case likely due to the large angle between the normal direction at the air–water interface and the direction of observation as well as due to the thickness of the liquid wedge. After testing different approaches, we developed and validated a new procedure that ensures an accurate panoramic reconstruction in the presence of light refraction.

To this end, two calibration frames were manufactured: a standard plate with a checkerboard pattern and a square prismatic target with a regular dot grid printed on its faces. First, all components of the multi-biprism setup in Fig. 10 (stereo-angle 90°, working distance ~ 260 mm, focal length ~ 8 mm) were finely aligned according to the schema in Fig. 9 to obtain four similar views of the sample when submerged in water. Then, the sample and its bath were removed and the four cameras were calibrated in air using the checkerboard plate as prescribed by common calibration procedures [16]. This first calibration stage was run separately for each camera to estimate the internal parameters needed to undistort all related images. Then, because the four cameras cannot view the checkerboard plate simultaneously, a 3D calibration frame (see lower inset in Fig. 10) was used to evaluate the relative camera position with respect to a unique reference system (depicted in Fig. 10) as described in [20]. In the subsequent stage, the calibrated 360-deg multi-view system was used to reconstruct four dense sets of points on the faces of a 25 mm × 25 mm × 80 mm square prism having a speckle pattern (camera 3 view in Fig. 11(a)). In particular, the square speckled prism was mounted so as to hang from above and was centered by acting on the three-axis stage (Fig. 10; component D). Then, the 100 mm × 100 mm × 100 mm acrylic bath was inserted from below and secured in its final position before gently raising the water level with the sample in place. As expected, each camera acquired an image with two angled views of the sample in the two halves of the sensor (Fig. 11(b)).

Fig. 10

Picture of the multi-biprism 360-deg DIC setup, A-sample, B- acrylic square bath, C- camera mounted on a kinematic stage, D- three-axis stage, E- syringe for sample pressurization. The lower inset shows the square prismatic calibration target as seen by camera 3

Fig. 11

Multi-biprism system calibration with refraction correction. The square speckle target as seen (a) in air and (b) within the bath filled with water. (c) Plot of the shape reconstruction error (0.36 ± 0.21 mm) before refraction correction

Results shown in Fig. 11(c) confirm that, if the calibration of the two sub-images of Fig. 11(b) is performed in water with DLT (as done for the four-camera pDIC), the 3D object reconstruction is affected by an unacceptably large error (here defined as the Euclidean distance between the positions of the same control data points measured in air and afterwards in water). A careful look at the error distribution also reveals that, notwithstanding the global barrel-like shape resembling the effect of a radial distortion, the error caused by refraction actually does not follow any common lens distortion parametric model [33, 38].

The problem of model-free distortions has been addressed previously by experimentally evaluating warping functions that, once applied to the acquired image, effectively remove the distortion [36, 38]. Herein, we closely followed the procedure in [36]. In particular, we disregarded the strategy of seeking a global rule to undistort the image, and instead estimated two unwarping functions (along horizontal and vertical directions of the sensor) on a local basis, that is, pixel-wise over the image ROI. Using a coarse set of matches, the two virtual views of Fig. 11(b) were calibrated with DLT and yielded two sets of parameters ${L}_{R}$ and ${L}_{L}$ [17]. Then, an optimization-based procedure found the disparities ${d}_{x}^{R}(i,j)$ and ${d}_{y}^{R}(i,j)$ to add to the point at position $(i,j)$ in the right view (and analogously ${d}_{x}^{L}(k,l)$ and ${d}_{y}^{L}(k,l)$ for the matched point at position $(k,l)$ in the left view) to minimize the distance between the 3D point reconstructed in water with the ${L}_{R}$ and ${L}_{L}$ parameters (Fig. 11(b)) and its counterpart reconstructed in air (Fig. 11(a)). The disparities calculated for a dense set of DIC matches in the two virtual views of Fig. 11(b) were fitted with NURBS surfaces to define two continuous warping functions that undistorted the images. The same procedure was applied to all pairs of virtual cameras of the setup in Fig. 10, thus mapping and correcting the refraction error over 360-deg.

To verify the feasibility of applying the evaluated warping functions to any object placed in the same working volume within the acrylic bath (of size comparable to the speckled square prism in Fig. 11(a)), we applied the distortion removal procedure to images of a 12.7 mm diameter cylindrical post painted with a speckle pattern (as shown in Fig. 3). Note that this size is representative of the human infrarenal aorta. To evaluate the error in strain calculation, we then slightly translated the post along three directions acting on the three-axis stage (Fig. 10; component D). Figure 12 shows the shape reconstruction error (radial deviation from the best fitting cylinder of nominal diameter) before (-0.087 ± 0.28 mm) and after (-0.003 ± 0.05 mm) refraction correction as well as the map of the total displacement imposed for the null strain analysis (1.12 ± 0.02 mm) and the related map of the first principal strain (0.0042 ± 0.008). No significant discontinuities were noticed across the four contiguous segments, and, interestingly, this was also the case for the uncorrected shape (Fig. 12(a)). Also for this test case, no filtering, smoothing, or re-meshing was performed on the raw DIC data. Strain peaks at the seams between views (Fig. 12(d)) occur because triangular facets with low aspect ratios are generated from the automatic tessellation of overlapping data points, and this heavily affects the results of the adopted triplet-based strain calculation [39]. Importantly, by comparing the 3D reconstructions and error distributions for the same cylindrical post in Fig. 3(c) and Fig. 12(b), it is possible to note how the adoption of the prismatic bath in the four-camera configuration of Fig. 10 increased the FOV to over 360-deg and improved the quality of the DIC reconstruction.

Fig. 12

Test on a 12.7 mm diameter post. (a) 3D reconstruction without refraction correction, (b) 3D shape after refraction correction, (c) total displacement map, and (d) strain in response to the imposed rigid body motion. Note that all plots report original raw data that were not filtered, smoothed, or re-meshed

Finally, to illustrate the potential use of the developed method for testing human arteries under reproduced physiological load, we speckled a latex tube of about 15 mm diameter and cannulated it for pressurization (as shown in Fig. 8). From the two sets of images acquired by the four cameras of the multi-biprism setup, the entire 360-deg surface was reconstructed in two configurations as shown in Fig. 13(a, b). Figure 13(c, d) show illustrative results for the first principal strain and the difference between the computed ${E}_{1}$ strain component (first principal strain) with and without refraction correction (0.0018 ± 0.006). Note the ability of the system to reconstruct regions of asymmetric bulging of the latex tube and to capture the associated inhomogeneous strain distribution. This highlights both the importance of local mechanical analysis and the strengths of the proposed setup. Once the setup is calibrated, and the procedure for refraction error correction is performed, the current system can be used for testing human (and/or porcine) arterial samples with diameters ranging from 10 to 25 mm and a maximum length of 80 mm under different combined loading conditions [21].

Fig. 13

Test on an initially 15 mm diameter latex tube under inflation (Fig. 8). (a) Undeformed shape, (b) deformed shape, (c) maximum principal strain map, and (d) map of the difference between strain maps calculated with and without refraction correction. Note the asymmetric inflation captured by the data analysis

Concluding Remarks

Although the search for universal testing methods must continue, the design of experimental methods that are specific to particularly demanding test cases often enables performance that is otherwise not possible with standard testing protocols. This is the case, for instance, in the field of vascular biomechanics where DIC has great potential. Here, we propose two sample-size dependent four-camera approaches to overcome these limitations by using additional reflective or refractive components. In particular, we propose a novel pDIC method for small arteries that uses a conical mirror to generate an infinite-view surrounding system, and we propose a multi-biprism panoramic method for large arteries that halves the number of required cameras with the aid of a refractive wedge of water. These two systems were specifically designed to enable dynamic 360-deg measurements on arteries within two ranges of diameters of primary interest for arterial biomechanics: less than 2.5 mm (murine arteries) and greater than 10 mm (human aorta). Illustrative results for 3D shape reconstruction and regional deformations, obtained for a representative mouse infrarenal aorta and a latex cylinder of similar size as the human infrarenal aorta, demonstrated the feasibility and accuracy of both proposed methods.

The major strength of both systems is the ability to perform dynamic 360-deg full-field measurements, even on very small samples, with four off-the-shelf cameras equipped with low magnification lenses. Both systems further enable the correction of refraction errors, thus allowing samples to be tested while immersed in a fluid bath reproducing physiological conditions. With regard to the system for testing murine arteries, similar measurement performance can be achieved with the experimental apparatus presented in [29]. In that work, two radially fixed and vertically stacked stereo-cameras (Fig. 1(b)) were equipped with long-distance high-magnification lenses and mounted onto a motor-controlled rotational stage. The imaging platform revolves around micro-aneurysmal surrogates submerged within an octagonal bath and collects images at 45° intervals thus realizing a pseudo multi-camera setup [15]. Unlike the fixed four-camera pDIC system presented here, the experimental apparatus in [29] used high-magnification lenses with shallow DOF and can be used only for quasi-static loading tests. To the best of our knowledge, there are no previously published DIC-based testing methods with measurement capabilities similar to setups proposed herein for in vitro testing of submerged human arterial samples.

Although both of the present systems use readily available components (complete parts lists are in Supplementary Material), a strong working knowledge of geometrical optics and DIC is essential to inform the setup and camera alignment as well as the optimal choice of camera parameters to maximize system performance [24, 31]. A further critical step, as in most arterial testing, is appropriate preparation of the sample ensuring appropriate hydration throughout the phases of cannulation, speckling, and mounting.

There are many potential biomedical applications for these novel pDIC systems. Pathologies such as aneurysms or dissections result in lesions with complex morphologies and heterogeneous material properties which have seldom been characterized properly from a mechanical point of view [44]. Accordingly, biomechanical mechanisms of lesion formation and the impact on the evolving biomechanics of the arterial wall remain unknown. We submit that our novel conical mirror-based approach will overcome the relative difficulty of studying the underlying biomechanical mechanisms in murine vessels, and thereby enable study of myriad mouse models that are relevant for developing future treatments of cardiovascular diseases. As lesions that arise in mouse models do not phenocopy human pathologies in a many regards, the biprism-based approach will enable human arteries to be studied similarly. Although their mechanical characterization through uniaxial tensile and biaxial tests has significantly advanced understanding, there is still a pressing need for understanding better the regional variations that manifest in most pathologies, with hopes that such variations can be correlated with the underlying regional biology that is becoming possible with single cell RNA sequencing and other methods. Our novel pDIC systems now provide a way to quantify regional behaviors from mouse models to human samples using similar approaches, thus facilitating the synthesis of information across these different, but essential, physical scales.