Main results of the 4th International PIV Challenge

PIV is a well-established technique even for the analysis of flows in systems whose effective dimensions are below 1 mm. However, the analysis of flows in microsystems differs from macroscopic flow investigations in many ways (Meinhart et al. 1999). Due to the volume illumination, the thickness of the measurement plane is mainly determined by the aperture of the imaging lens and therefore out-of-focus particle images are an inherent problem in microscopic PIV investigations. Consequently, digital image preprocessing algorithms are essential to enhance the spatial resolution by filtering out the unwanted signal caused by out-of-focus particles. Furthermore, the low particle image density achievable in microfluidic systems at large magnification and large particle image displacements as well as strong spatial gradients require sophisticated evaluation procedures, which are less common in macroscopic PIV applications. To bring the micro- and macro-PIV community together, a demanding microscopic PIV case, measured in a microdispersion systems, was considered in this PIV Challenge.

Microdispersion systems are very important in the field of process engineering, in particular food processing technology. These systems produce uniform droplets in the sub-micrometer range if driven with pressure around 100 bar (Kelemen et al. 2015a, b). The basic geometry consists of a straight microchannel (\(80 \,\upmu \hbox {m}\) in depth) with a sharp decrease in cross section and a sharp expansion thereafter as illustrated in Fig. 3. Due to this sudden change in width (from 500 to \(80 \,\upmu \hbox {m}\)), the velocity changes significantly by about 230 m/s within the field of view. The strong convective acceleration stretches large primary droplets into filaments. Due to the normal and shear forces in the expansion zone, the filaments break up into tiny droplets which persist in the continuous phase of the emulsion (fat in homogenized milk for instance). The flow includes cavitation at the constriction due to the large acceleration, strong in-plane and out-of-plane gradients (Scharnowski and Kähler 2016) in the boundary layers of the channel and shear layers of the jet flow and, in case of two-phase fluids, droplet-wall and droplet–droplet interaction. Thus, the flow characteristics are difficult to resolve due to the large range of spatial and velocity scales.

To characterize the single-phase flow, a \(\upmu \hbox {PIV}\) experiment was performed at the Bundeswehr University Munich within the DFG (German Research Foundation) research group 856 (Microsystems for Particulate Life-Science-Products). The channel was driven by a pressure of 200 bar. Fluorescent polystyrene particles with a diameter of \(1 \,\upmu \hbox {m}\) were added to the flow and imaged by a Zeiss Axio Observer microscope with a 20\(\times\) magnification lens. To capture the flow, a sCMOS camera with an interframing time of 120 ns was used. The particles were illuminated using a Litron Nano-PIV double-pulse Nd:YAG laser with 4 ns pulse length. 600 Double-frame images were recorded. The field of view covers approximately \(1400 \times 600\,\upmu {\text {m}}^2\) on \(2560 \times 1230\) pixels. The main challenges of this experiment are:

• Extremely large velocity gradients (Keane and Adrian 1990; Westerweel 2008)

• High dynamic velocity range (Adrian 1997)

• Depth of correlation

• Optical aberrations due to the thick window

• Low signal-to-noise ratio (SNR) and cross-talk

• Cavitation

Open image in new window

The time-averaged flow field is quite uniform and of small velocity prior to the inlet of the channel, see (1) in Fig. 3. Toward the inlet, the flow is highly accelerated (2) and a fast channel flow develops with very strong velocity gradients close to the walls (3). Cavitation occurs at different places, especially at the inlet of the small channel in regions of very high velocities (2). This can already be seen in the images. Taking an average image over images 1–250, cavitation bubbles can be seen at both sides of the inlet as shown in Fig. 4 (left). Later, these cavitation bubbles diminish as can be seen by averaging images 300–600, see Fig. 4 (right). Due to this effect, strong velocity changes are expected in this area as the flow state alternates between two modes, namely one with and without cavitation bubbles. It is evident that the velocity change will result in strong local Reynolds stresses on average, although both flow states are fully laminar. The Reynolds stresses are simply an effect of the change in flow state, which results in a mean flow measurement that does not exist at all in the experiment.

At the outlet of the small channel, a free jet with a very thin shear layer develops, indicated by (4). This jet tends to bend toward one wall, because of the 2D geometry. The preferred position might change for subsequent experiments. However, the jet remains attached to one wall for the rest of the experiment and a recirculation region forms (5). The flow field therefore features a high dynamic spatial range (DSR) and a very high dynamic velocity range (DVR) between the fast channel flow and the almost stagnant and reverse flow regions. Since the aim of the current experiment is to qualify the whole flow field at once, the magnification was chosen such that by using the smallest interframing time of 120 ns results in displacements that can still be processed. Increasing the magnification further was not possible, because the minimum interframing time of 120 ns would than have been to large and the velocity evaluation would not have been possible.

Open image in new window

In contrast to macroscopic PIV, the image quality for microscopic imaging is often poor due to low light intensities, small particles and optical aberrations. As typically for cameras with a CMOS architecture, the gain for different pixels might also be different, which results in so-called hot and cold pixels (Hain et al. 2007). Hot pixels show a very large intensity, which results in a bright spot in the image, whereas cold pixels have a low or even zero intensity, which results in a dark spot.

These hot and cold pixels must be taken into account during the evaluation process to avoid bias errors. If not removed, they cause strong correlation peaks at zero velocity due to their high intensity. Usually, this effect can be corrected by subtracting a dark image already during the recording. However, sometimes this does not work sufficiently well, especially when cameras are new on the market. Therefore, it was decided to keep these hot pixels (with constantly high intensity values) and cold pixels (with constantly low gain and low or zero intensity value) in the images and leave it up to the participant to correct for them, instead of using the internal camera correction. Furthermore, the signal-to-noise ratio (SNR) is very low, due to the use of small fluorescent particles as well as the large magnifications. In the current case, the ratio of the mean signal to background intensity was below two. In some images also blobs of high intensity can be seen that result from particle agglomerations.

An additional problem was the time delay between successive frames which was set to the minimal interframing time of 120 ns. Due to jitter in the timing, cross-talk between the frames can be observed in some images, i.e., particle images of both exposures appear in one frame.

The pairwise occurrence of particle images with the same distance in a single frame indicates a double exposure with almost the same intensity. These images can be identified by the average intensity of the different frames, which is, in the case of double exposure, significantly higher than in average. In total, only 6 out of the 600 images show significant cross-talk. Strategies to cope with this problem might include image preprocessing or simply the identification and exclusion of these images. However, if not taken into account, high correlation values at zero velocity bias the measurements.

In comparison with macroscopic PIV applications, the particle image density is due to the large magnification typically lower. In order to enhance the quality of the correlation peak and the resolution, ensemble correlation approaches are typically applied (Westerweel et al. 2004).

Another inherent problem for microfluidic investigations is the so-called depth of correlation (DOC) as already mentioned at the beginning of the section. In standard PIV, a thin laser light sheet is generated to illuminate a plane in the flow which is usually observed from large distances. The depth of focus of the camera is typically much larger than the thickness of the laser light sheet, and therefore, the depth of the measurement plane is determined by the thickness of the light sheet at a certain intensity threshold which depends also on the size of the particles, the camera sensitivity and the lens settings. In microfluidic applications, due to the small dimensions of the microchannels, the creation and application of a thin laser light sheet is not possible. Volume illumination is used instead. As a consequence of this, the thickness of the measurement plane is defined by the depth of field of the microscope objective lens. In \(\upmu \hbox {PIV}\) experiments, the DOC is commonly defined as twice the distance from the object plane to the nearest plane in which a particle becomes sufficiently defocused so that it no longer contributes significantly to the cross-correlation analysis (Meinhart et al. 2000). In the case of vanishing out-of-plane gradients within the DOC, the measurements are not biased, but the correlation peak can be deformed due to non-homogeneous particle image sizes, which can reduce the precision of the peak detection. Unfortunately in many cases, the DOC is on the order of the channel dimension and thus large out-of-plane gradients are present. Under certain assumptions, a theoretical value can be derived for the DOC based on knowledge of the magnification, particle diameter, numerical aperture, refractive index of the medium and wavelength of the light. The DOC can also be determined by an experiment, and very often the experimental values are larger since aberrations appear. In the current case, the theoretical value is \(\hbox {DOC}_{{\mathrm {theory}}}=17.6\,\upmu \hbox {m}\), whereas the experimentally obtained value gives \(\hbox {DOC}_{{\mathrm {exp}}}=31.5\,\upmu \hbox {m}\) (Rossi et al. 2012). Since the tiny channel had a squared cross section of \(80 \times 80\,\upmu \hbox {m}^2\) , this covers more than one-third of the channel and significant bias errors can be expected. However, adequate image preprocessing or non-normalized correlations can minimize this effect (Rossi et al. 2012). Alternatively, 3D3C methods can be applied to fully avoid bias errors due to the volume illumination (Cierpka and Kähler 2012).

In general, image preprocessing is very important for microscopic PIV to increase the signal-to-noise ratio and avoid large systematic errors due to the depth of correlation (Lindken et al. 2009). Consequently, most of the teams applied image preprocessing to enhance the quality of the images.

In total, 23 institutions requested the experimental images. Since the images were very challenging, not all of them succeeded in submitting results in time. Results were submitted by 13 institutions. The acronyms used in the paper and details about the data submitted are listed in Table 5. For the data sets named evaluation 1 (eval1), the participants that applied spatial correlation such as conventional window cross-correlation or sum of correlation by means of window correlation were forced to use a final interrogation window size of \(32 \times 32\) pixels. Multi-pass, window weighting, image deformation, etc. were allowed. The fixed interrogation windows size allows for a comparison of the different algorithms without major bias due to spatial resolution effects (Keane and Adrian 1992; Kähler et al. 2012a, b). However, the experience and attitude of the user has a very pronounced effect on the evaluation result, and different window sizes may have been favored by the different users to alter the smoothness of the results. Therefore, the teams were free to choose the interrogation window size in case of evaluation 2 (eval2). The evaluation parameters are summarized in Table 5. Some special treatments are described in the following if they differ significantly from the standard routines.

Dantec used close to the walls a special shaped wall window so that the contribution from particles further away can be excluded and thus the systematic velocity overestimation can be minimized. In addition, a N-sigma validation \((\sigma = 2.5)\) was used to detect and exclude outliers that may appear in groups due to the large overlap.

A feature tracking algorithm was applied by INSEAN (Miozzi 2004), which solves the optical flow equation in a local framework. The algorithm defines the best correlation measure as the minimum of the sum of squared differences (SSD) of pixel intensity corresponding to the same interrogation windows in two subsequent frames. After a linearization, the SSD minimization problem is iteratively solved in a least square style, by adopting two different models of a pure translational window motion and in the second step an affine window deformation. The deformation parameters are given directly by the algorithm solution (Miozzi 2005). The velocity for the individual images was only considered where the solutions of the linear system corresponding to the minimization problem exist. Subpixel image interpolation using a fifth-order B-Spline basis (Unser et al. 1993; Astarita and Cardone 2005) was performed. In-plane loss of pairs was avoided by adopting a pyramidal image representation and subpixel image interpolation using a bicubic scheme.

The peak detection scheme of IOT was based on the maximum width of the normalized correlation function at a value of 0.5. The assessment of the mean velocity and the velocity fluctuations started from the center of mass histogram method, and then it was used as an initial guess for an elliptic 2D Gaussian fitting with a nonlinear Levenberg–Marquardt optimization by the upper cap (0.5–1) of the correlation peak points. The displacement fluctuations were obtained by the shape of the ensemble correlation function following the approach of Scharnowski et al. (2012). A global displacement validation (\(-20\le DX\le 50\) px and \(-30 \le DY\le 30\) px and 53 px for vector length) was applied. Outliers after each iteration were replaced by \(3\times 3\) moving average.

For peak finding, TCD used a \(3\times 3\) 2D Gaussian estimator (Nobach and Honkanen 2005) that ranks peaks according to their volume instead of their height. For intermediate steps, a \(3\times 3\) spatial median filter was used to determine outliers (Westerweel and Scarano 2005). They were replaced by lower-ranked correlation peaks or local median of the neighbors. Two further median smoothing operations were applied.

An iterative multi-grid continuous window deformation interrogation algorithm was applied by TU Delft to individual images with decreasing window size starting with \(128\) pixels (Scarano and Riethmuller 2000) with bilinear interpolation for the velocity and \(8\times 8\) sinc interpolation for pixel intensities (Astarita and Cardone 2005). Close to the walls, a weighting function was applied to the non-masked region.

Case B of the 4th International PIV Challenge deals with a time-resolved image sequence. The data were measured in a water tunnel at the Technical University Munich within the EU project AFDAR (Advanced Flow Diagnostics for Aeronautical Research), see also Schröder et al. (2015) and Kähler et al. (2016) for more details. The test section of this facility is adopted to ERCOFTAC test case 81 “Flow over periodic hills” (see Fig. 12). For more details see ERCOFTAC QNET-CFD forum under case study UFR 3-30. and Rapp and Manhart (2011). The test section has 10 hills, and the measurements were performed downstream of the seventh hill summit as the flow is almost periodic at this location, e.g., the averaged flow quantities coincide half way between the neighboring hills.

Open image in new window

The seeding particles (glass hollow spheres with a mean diameter of \(10\,\upmu {\mathrm {m}}\)) were illuminated by a Spectra Physics Millennia Nd:YAG cw laser with a power of \(5\,{\mathrm {W}} \, @ \, 532\,{\mathrm {nm}}\). A Phantom v12 CMOS camera with a pixel size of \(20 \times 20\,\upmu {\mathrm {m}}^2\) at a recording rate of \(2000\,{\mathrm {Hz}}\) was used for image acquisition. In total, a sequence of 1044 single-frame single-pulse images with a resolution of \(1280 \times 800\,{\mathrm {px}}\) each was provided to the teams. An example image with the corresponding histogram (100 images have been considered) of the gray value distribution is shown in Fig. 13. As typically observed for this kind of high-speed image sequences, the SNR is rather low and the particle image diameter is small due to the large pixel size, see Hain et al. (2007). At the bottom and top wall, laser light reflections occur which complicate the evaluation near the wall. Furthermore, it can be seen in this figure that the particle image intensities depend on the x-location which is influenced by the laser light intensity distribution.

Open image in new window

The Kolmogorov length scale cannot be resolved using the experimental setup. This is typical for most of the PIV measurements since the field of view is usually relatively large in order to observe the large-scale flow structures. However, the spatial resolution is in the order of the Taylor scale which means that most of the vortices which occur in the turbulent spectrum can be resolved.

The participants had to perform two different evaluations. Evaluation 1 is a standard PIV evaluation, and evaluation 2 is more advanced.

In order to compare the evaluations of the teams, different plots are shown in the following. At first, instantaneous flow fields are presented. Afterward, PDFs of the displacements are shown in order to investigate the peak-locking effect and to see the differences between evaluation 1 and 2. Line plots of mean velocities and rms are shown in addition to see differences in the statistics from different results. A comparison of the temporal as well as spatial characteristics by means of Fourier transforms completes the analysis of case B.

Instantaneous flow fields represented by the displacement component \(\hbox {d}x\) for evaluation 1 are shown in Fig. 14. All teams, except INSEAN, used cross-correlation evaluation. Rather, INSEAN applied an optical flow approach. The flow fields of Dantec, DLR, INSEAN, IOT, TsU and UniMe look quite similar. In comparison, the displacement fields of IPP, LANL, MicroVec and MPI are much more noisy. Especially in the MicroVec evaluation, strong gradients are observed. TSI applied a filter which results in an unphysical strong smoothing of structures and gradients which is also visible in foregoing plots.

The estimated displacements at the upper wall show some differences. The high gradient does not seem to be captured in a meaningful physical sense. Depending on the mask and boundary condition, the displacements drop down to 0 at a location near to the wall. More details concerning the near-wall gradients are displayed in later line plots.

Open image in new window

In comparison with the displacement component \(\hbox {d}x\) of evaluation 1, the displacement component \(\hbox {d}x\) of evaluation 2 is smoother in most of the evaluations, see Fig. 15. This is a result of the noise reduction due to the advanced evaluation techniques which also will be visible in the results shown in the following.

Open image in new window

Histograms of the displacements \(\hbox {d}x\) are shown in Figs. 16 and 17.

Open image in new window

Open image in new window

It has to keep in mind that for, e.g., evaluation 1 the displacements obtained from the correlation of two images with a temporal distance of \(2 \cdot \Delta t\) has been divided by 2 in order to get the virtual displacement from one time step to the next one. Therefore, peak locking is observed in these figures for some teams at locations \(\Delta x = i \cdot 0.5\,{\mathrm {px}}\), with \(i \in {\mathbb {Z}}\). Concerning the peak locking, there is a strong difference between the participants. While some curves show nearly no peak locking, some other have a severe peak locking for evaluation 1 as well as evaluation 2. However, for many teams, the peak locking is strongly reduced or even vanishes by using the advanced evaluation technique. This results in an increased accuracy. The optical flow evaluation of INSEAN agrees well with the cross-correlation evaluations of, e.g., Dantec and DLR.

To evaluate statistical properties, line plots of the displacements in x-direction, \(\hbox {d}x\), are shown. The plots for the displacement \(\hbox {d}y\) in y-direction do not differ much so that they are not presented here.

Open image in new window

Open image in new window

In order to see whether there are differences between the participants regarding the statistical values, the mean displacements of \(\hbox {d}x\) along the line \(x=100\,{\mathrm {px}}\) are shown in Figs. 18 and 19 for evaluation 1 and evaluation 2, respectively. The dashed lines in these figures visualize the location of the walls. The hill is located at \(y=271\,{\mathrm {px}}\) and the upper wall at \(y=764\,{\mathrm {px}}\). The figures indicate that the mean values agree well between the participating teams, in contrast to case A. Only slight differences are observed near the wall. One important reason for these differences are the masks which have been applied. The comparison between evaluation 1 and evaluation 2 also shows only small differences. Due to the averaging process, stochastic errors do not play a role and no systematic errors occur. This result is quite encouraging as it means that all methods shown here are suited to obtain mean displacement fields with comparable accuracy.

Open image in new window

Open image in new window

The rms of the displacements \(\hbox {d}x\) along the line \(x=100\,{\mathrm {px}}\) is shown in Figs. 20 and 21. Compared with the mean displacement fields, the scatter of the data is higher. However, many teams provided quite similar results. Looking at subplot 1 in Figs. 20 and 21, it can be seen that the scatter between the teams for evaluation 2 is smaller than the scatter for evaluation 1 (not considering the IOT-PTV evaluation). The conclusion can be drawn that a proper evaluation by means of an advanced evaluation technique reduces the random error. However, not every advanced technique is able to do this, as can bee seen when comparing the subplots 3 in Figs. 20 and 21. In these cases, the scatter between the teams for evaluation 2 is larger than the scatter for evaluation 1, showing that the effect of the user is larger than the differences due to the specific evaluation techniques applied here.

A temporal Fourier transform has been performed in order to determine the frequency content in the evaluated signals. At every vector location 1024 time steps are available which have been acquired with an frequency of \(2000\,{\mathrm {Hz}}\). Such a temporal signal of, e.g., the displacement \(\hbox {d}x\) was used to perform a FFT. An average of the resulting amplitude spectrum was computed over the region \(x = 300 \ldots 700\,{\mathrm {px}}; y = 500 \ldots 700\,{\mathrm {px}}\) (20301 vectors/FFTs in total). The resulting mean FFTs are shown in Figs. 22 and 23 for evaluation 1 and evaluation 2, respectively.

Open image in new window

Open image in new window

For evaluation 1, the curves show a similar behavior, except for IOT. In the postprocessing, the teams performed a temporal filtering which leads to the suppression of high-frequency fluctuations. Comparing the results of the other participants shows that even at high frequencies the amplitude of \(\hbox {d}x\) does not significantly decrease. This is expected as these high-frequency fluctuations are caused by the noise. However, from the physical point of view such high frequency should not occur, cf. the discussion on the Kolmogorov scales in Sect. 6.1 and considering the averaging effect due to the final interrogation window size. The curves of evaluation 2 show a decreasing amplitude with increasing frequency for some teams who apply sophisticated evaluation techniques, which means that the measurement accuracy is increased. Typically the high-frequency oscillations are damped by locally increasing the particle image displacement and thus reducing the relative measurement error.

In order to determine the spatial frequency content in the evaluated signals, a spatial Fourier transform was performed at location \(x=572\,{\mathrm {px}}\) to calculate the amplitude spectrum. An average over the available 1024 time steps was computed in order to obtain a smooth spectrum. The resulting plots are shown in Figs. 24 and 25 for evaluation 1 and evaluation 2, respectively.

Open image in new window

Open image in new window

Using top-hat quadratic interrogation windows, the locations of the roots in these plots are theoretically determined from the sinc function which is the Fourier transform of the rectangular function. This leads to root locations at frequencies \(f=\frac{n}{IW}\) with \(n \in \{ 1, 2, 3,\ldots \}\) and IW being the size of the interrogation window in pixels. For many teams, strongly decreased amplitudes are observed at these frequencies. However, this is not the case for all teams for different reasons. One of these reasons is, using window weighting which decreases the mentioned effect. A second reason is the method of postprocessing which was applied by the participants. Depending on the kind of postprocessing, a filtering/smoothing effect is obtained in this step which leads to a modification of the spatial frequency spectrum. A third reason may be that some participants did not perform the evaluation on a grid with size \(2 \times 2 \,{\mathrm {px}}\). Rather they performed the evaluation on a coarser grid and interpolated the data on the fine grid. However, this effect should be negligible since as least the low spatial frequencies are not affected by this method.

In the 3D scenario, the spatial resolution is notoriously difficult to be predicted, as it depends on several parameters. The particles concentration and the reliability of the algorithm for the reconstruction of the 3D particles distribution play a key role. Generally, a compromise is required between dense particles distributions and sufficiently accurate reconstruction. To date, the trade-off is left to the personal experience of the experimenter. The purpose of the Test case C is to assess the performances of the reconstruction and processing algorithms in terms of final spatial resolution and accuracy.

The test case is performed on synthetic images of a complex flow field. The optimization of the particle image density in terms of maximum spatial resolution with minimum loss of accuracy is the main challenge of this test case.

The particles are randomly located in a \(48 \times 64 \times 16\,{\mathrm {mm}}^3\) region. Particles are also generated outside of the measurement region (in a volume extended by 24 and 32 mm on both sides on the x and y direction, respectively) in order to reproduce the experimental conditions of a laser slab of 16 mm thickness illuminating the measurement region. A small displacement is imposed to the outer particles in order to avoid trivial suppression by images subtraction. Projected images of 4 cameras (\(1280 \times 1600\) pixels at 16-bit discretization; pixel pitch 6.5 \(\upmu \hbox {m}\)) are provided with image density ranging between 0.01 ppp (particles per pixel) and 0.15 ppp. Sample images for low, medium and large density are reported in Fig. 26. The cameras are angled at \({\approx}{\pm} 35^\circ\) in both directions (deviations from these values are due to accommodation of the Scheimpflug angles while maintaining the imaged volume centered in the images). The magnification in the origin of the reference system is about 0.15 for all the cameras and images are generated with \(f_{\#} = 11\), thus resulting in a particle image diameter of about 2.5 pixels.

The projection images in this set are not contaminated by noise. Perfect calibration images (as well as the exact correspondence between space and image coordinates of the calibration markers in form of a lookup table) are provided. Since the calibration can be achieved with very small residual error, the effects of calibration uncertainty (vibrations or thermal expansion in the mechanical system for target displacement, loose connections, etc.) are not included in the analysis. Removing the influence of preprocessing on image quality restoring and of the effectiveness of the self-calibration (Wieneke 2008), in correcting calibration uncertainties within the camera set, might appear as an oversimplification with respect to the real experimental environment. This line has been a necessary choice given the early stage of volumetric PIV and the novelty of fully three-dimensional three-component test cases in the PIV Challenge. The interpretation of the results greatly benefits from this simplification. Effects of image corruption (distortion, background noise, reflections, etc.), calibration uncertainties and more realistic experimental conditions might be the object of test cases in future challenges.

Open image in new window

In order to have a fair comparison of the reconstruction algorithms, the participants have been asked to submit a reconstructed volume at a reference image density of 0.075 ppp. From this point on, unless otherwise stated, the reference resolution will be 20 vox/mm. The reconstruction has been performed by the participants on a larger volume; the depth of the volume is increased from 16 to 24 mm (320–480 voxels at 20 vox/mm).

Particles positions in the reconstructed volumes are determined by identification of local maxima on \(3 \times 3 \times 3\) voxel kernels. A sub-voxel accuracy position estimate is obtained by Gaussian interpolation on a 3-point stencil along the three spatial directions. A particle is identified as “true particle” if located at a distance smaller than 2 voxels (i.e., 0.1 mm at 20 vox/mm) from a particle of the exact distributions.

The number of true and ghost particles is reported in Fig. 27 for the full volume and for the central part, obtained by cutting 300 voxels in the x and y directions from the borders. The participants are sorted by increasing percentage of ghost particles in the full volume. For all the participants except LaVision and IoT, the particles are directly identified on the provided volumes. Since the volumes submitted by LaVision and IoT were saturated on the top intensity level, it was not possible to perform the same operation on the original volumes. For this reason, the analysis is conducted on the interpolated volumes used also for the Q factor calculation. The values are expressed in percentage of the total number of true particles in the exact distribution. In some of the provided reconstructions, there is a significant percentage of true particles which is lost in the process (see for instance ASU, LANL). This might be addressed by different reasons. ASU uses a direct method which tends to lose edge in case of overlapping particles. LANL uses the same direct method, but in addition particles are removed to match the image density. This procedure could be dangerous since the intensity distributions of true and ghost particles might not be statistically separated, and thus the weakest true particles might be erroneously removed. Additionally, note that the highest values of Q are achieved by the participants using a Gaussian smoothing during the reconstruction process, as proposed by Discetti et al. (2013). While this smoothing is not expected to change significantly the number of ghost and true particles, it regularizes the solution and redistributes the intensity from the reconstruction artifacts to the true particles, thus improving the quality of the reconstruction.

In almost all the cases, the number of ghost particles is larger than that of the true particles, with the exceptions of DLR and LaVision. There is little dependence on the used algorithm. For instance, ASU, which is using a direct method for the reconstruction, has nearly the same number of ghost particles of TUD, which is instead adopting an advanced algorithm with MTE-MART, and of BUAA, which is performing the reconstruction with a single-frame implementation of a MART-based method. It is remarkable, on the other hand, the difference between LaVision and IoT, which are using very similar processes for the reconstruction, the main differences being the adopted resolution (40 and 20 vox/mm, respectively) and the smoothing applied by LaVision between the iterations. However, since LaVision and IoT are both using MTE, there is an influence of the quality of the PIV interrogation on the reconstruction itself, which cannot be easily isolated from the provided volumes.

Open image in new window

The differences between the algorithms are more evident if the intensity of true and ghost particles is taken into account, as in the weighed levels or in the power ratio (see Fig. 28). These parameters are extremely relevant, as they indicate the relative importance of real and ghost particles in determining the estimated velocity field. In particular, the power ratio expresses in a synthetic form the “cross-correlation power” of the particles within the reconstructed volumes. Consequently, a high power ratio can be obtained with a small number of ghost particles, or with weak intensity ghost particles (when compared with the true ones), or both. If only the central portion of the volume is considered (i.e., the red columns in the histograms of Fig. 28), algebraic methods are superior in this sense with respect to direct methods. On the other hand, algebraic methods are more prone to border effects, as testified by the drop in weighed intensity levels and power ratio for BUAA, IPP, LaVision, TUD when considering the full volume. Very good results are also obtained by the StB method (DLR), which is an hybrid between PTV and Tomographic PIV. While traditional triangulation methods are severely challenged for image density larger than 0.005 ppp (Maas et al. 1993), the combination of 3D particles matching with iterative intensity-based correction can easily handle large image densities.

Open image in new window

The quality factor evaluated according to Eq. 5 is reported in Fig. 29. Participants using PTV-based methods (DLR) cannot be included. With only the exceptions of LaVision and ONERA, all the participants provided reconstructions with \(Q < 0.75\), which is commonly considered as the minimum value for an acceptable reconstruction. It is worth to stress that Q is evaluated on the volumes at 0.075 ppp, while some of the participants decided to use a lower image density to extract the velocity fields (ASU, IPP); thus, the quality factor might be larger. Again, algebraic methods suffer due to border effects while direct methods are relatively insensitive to this issue.

The comparison of the results in Figs. 27, 28, 29 poses a shade of doubt on the effectiveness of the quality factor in describing the accuracy of the reconstruction. For instance, Dantec has larger quality factor than TUD, but at the same time higher number of ghost particles and lower power ratio. Similarly, LANL has lower number of ghost particles than ASU, similar power ratio but lower Q; this might be related to true particles suppression in the LANL reconstruction algorithm. For this reason, when performing numerical assessment of reconstruction methods, the Q factor might be appealing for its intrinsic simplicity, but caution should be taken at the time of drawing conclusions.

Open image in new window

In Fig. 30, more insightful information on the reconstruction is reported for the teams ASU and TUD. The comparison is particularly meaningful, since ASU used a direct method for the reconstruction, while TUD used an algebraic method with the motion tracking enhancement. On the left column, a \(120 \times 120\) voxels slice (obtained by summing 20 adjacent planes in the center of the volume) is reported, with the reconstructed particles in gray levels and the true particles superimposed with green circles. The particles reconstructed by TUD appear sharper and better centered; the smoother particles shape in ASU’s results is due to the geometrical limitations of direct MLOS-like methods. The profiles obtained by summing the intensity along \(x-y\)-planes in the reconstructed volumes (normalized with the relative maximum value of the profile itself) are reported in the central column. A significant difference between direct and algebraic methods is that in the second case the intensity profile tends to drop in correspondence of the limits of the illuminated volume, while it increases again toward the boundaries. This peculiar feature might be addressed by the update mechanism of the iterative MART, which tends to accumulate ghost intensity in regions that are observed only by a subset of the cameras, or where the viewing angle becomes larger, thus leading to larger probability of particles overlapping and higher local image density (e.g., the volume corners). Finally, in the right column the histograms of ghost, true and exact particles distributions are reported (normalized with the mode of the intensity of the exact particles). While the number of ghost particles is similar (see Fig. 27), the intensity distribution of the true particles reconstructed in TUD is skewed toward larger intensity values, thus leading to the conclusions that iterative algebraic methods act significantly on the particles intensity but very weakly on the ghost particles number.

Open image in new window

The measurement of the underlying cascade turbulence mechanism at sufficiently high Reynolds number is a challenge for both computational and experimental fluid mechanics in terms of required spatial and dynamic velocity range. The establishment of accurate three-dimensional three-component anemometry techniques is of fundamental importance to both characterize the evolution and organization of the turbulent coherent structures and to provide a benchmark for numerical codes. In the test case D, a 3D PIV experiment is simulated imposing the velocity field of a direct numerical simulation (DNS) of an isotropic incompressible turbulent flow. The velocity fields and particle trajectories are directly extracted from the turbulence database of the group of Prof. Meneveau at John Hopkins University (http://turbulence.pha.jhu.edu/); details on the database and how to use it are provided by Perlman et al. (2007) and Li et al. (2008).

Case E aimed to evaluate the state-of-the-art for planar stereoscopic PIV measurements using real experimental data. The experiment that generated the case E data was performed at Virginia Tech’s Advanced Experimental Thermofluids Research Laboratory (AEThER Lab) and was based on time-resolved measurements of an impulsively started axisymmetric vortex ring flow. The experimental images acquired were purposefully subject to real-world sources of error that are representative of stereo experiments and that are difficult to model in simulated images. These error sources included focusing effects, diffraction and aberration artifacts, forward and backward scattering intensity variation, sensor noise, and calibration error.

Data were acquired simultaneously from five cameras in such a way that any combination of two out of the five cameras could yield a stereoscopic measurement data set. Hence, the experiment delivered a range of data sets with different levels of accuracy including a case for which an optimum stereoscopic configuration was approximated. In addition, the data set was processed as a three-dimensional tomographic PIV measurement that was then used as a reference for evaluating the accuracy of the stereo-PIV measurements. This comparison was based on the notion that a tomographic PIV system measures the three-dimensional velocity field within a laser sheet more accurately than traditional stereo-PIV measurements (Michaelis and Wieneke 2008; Wieneke and Taylor 2006; Elsinga et al. 2006a).

The case E flow field consisted of a laminar vortex ring with a Reynolds number based on diameter and piston velocity, of approximately \({Re} = 2300\). The vortex ring was generated using a hydraulic actuation to smoothly translate a piston-cylinder arrangement with a predetermined stroke distance and piston velocity program. The fluid was then ejected through a circular orifice. A schematic of the experimental arrangement is shown in Fig. 54a. Polystyrene fluorescent particles with a mean diameter of \(27 \, \upmu \,{\mathrm {m}}\) (Duke Scientific Corporation catalog number 36-5B) suspended in water were used as flow tracers. The field was illuminated using a dual head \(10 \, {\mathrm {mJ}}{\mathrm {pulse}} 527 \, {\mathrm {nm}}\) Nd:YLF laser that was synchronized with all cameras. The particle images were acquired as sequential single-pulse photographs recorded at \(1000 \, {\mathrm {Hz}}\). The camera array consisted of three Photron FASTCAM APX-RS cameras aligned vertically and two Photron FASTCAM SA4 cameras placed on either side of the three APX-RS cameras, as is shown in Fig. 54b. Cameras 2 and 4 used Scheimpflug adapters to account for the off normal angle between the cameras and the laser sheet. Since Cameras 3 and 5 were also not normal to the laser sheet and did not have Scheimpflug adapters, the lens aperture was decreased to ensure that the particles remained in focus. The optical axis of Camera 1 was normal to the laser sheet, and thus all the particles were in focus with the lens aperture fully open.

Open image in new window

A high resolution, low error estimate of the velocity field had to be created from the tomographic data to compare with the participant stereo-PIV velocity field submissions. This “ground truth” solution was used to complete the error analysis; however, since the data were collected from an experiment, the true velocity field within the physical fluid was unknown. Thus while the ground truth solution represents a more accurate measurement of the true field compared to what can be provided by stereo-PIV, it is still subject to measurement errors.

Calculating the ground truth solution from the five camera PIV data required several steps: First, a high accuracy tomographic intensity field reconstruction is calculated; second, a three-dimensional velocity field is calculated from the intensity field; third, the intensity field is used to identify the coordinates of the laser sheet; and 4th, the three-dimensional velocity field is interpolated onto the two-dimensional laser sheet coordinates. The diagram in Fig. 55 shows the full process involved in calculating the ground truth solution.

Open image in new window

The three-dimensional intensity field was calculated using LaVision DaVis 8.1.6. The images were preprocessed using background subtraction and particle intensity normalization. A third-order polynomial camera calibration (Soloff et al. 1997) was fit to the calibration grid images and then refined using volumetric self-calibration (Wieneke 2008). The three-dimensional intensity field was then reconstructed using 10 MART iterations (Elsinga et al. 2006b). Following this, the intensity field was used to calculate the three-dimensional velocity field and the position of the laser sheet.

The three-dimensional velocity field was calculated by performing volumetric pyramid robust phase correlations (Sciacchitano et al. 2012; Eckstein et al. 2008; Eckstein and Vlachos 2009) with iterative window deformation using code implemented in MATLAB. The pyramid correlation technique requires determining an optimum number of frames over which to take the correlations. The optimal number of frames is calculated by determining the largest frame separation that can be used to calculate correlations before the loss-of-correlation effects become too significant. To accomplish this, four loss-of-correlation sources were investigated: the material acceleration of the particles, the shear rate within the velocity field, the out-of-plane motion and the maximum particle image displacement within the correlation window (Sciacchitano et al. 2012). Graphs giving the optimal number of frames for the 50th image pair over the entire velocity field are shown in Fig. 56. The optimal frame number was then determined by finding the minimum number of frames over the full flow field for all four loss-of-correlation sources. Using these estimates, the optimal frame number was determined to be equal to 4 frames, with the material acceleration dominating the other three sources.

Open image in new window

The next step in calculating the ground truth velocity field was to determine the precise location of the laser sheet within the three-dimensional reconstruction. This is important for two reasons. First, the three-dimensional velocity field must be interpolated onto the two-dimensional coordinate grid to yield data comparable to stereo data. Second, stereo-PIV measurement error is decreased by performing self-calibration; however, the self-calibration process maps the velocity field coordinate system from the calibration grid coordinate system onto a different coordinate system based upon the laser sheet position. Thus the coordinate system used for measuring the velocity field in stereo-PIV and the coordinate system used for measuring the velocity field in tomographic PIV are different. Physically this difference is due to the fact that the coordinate plane of the calibration grid and the plane of the laser sheet may not be parallel and may not intersect along a line including the calibration grid origin. The transformation between these two coordinate systems corresponds to a rotation and a translation. This transformation changes the velocity components, but does not change the velocity magnitude nor any physical quantities derived from the velocity field such as forces or stresses. The effect of this transformation will typically be quite small, and in this case, applying the transformation caused the velocity vectors to change by a smaller magnitude than the expected PIV uncertainty. Thus the transformation negligibly changed the ground truth solution, but it was included for completeness.

The location of the laser sheet was calculated by summing the tomographically reconstructed intensity fields over all frames to produce a time-averaged intensity field and then fitting a plane to the peak of the average intensity field. Once the precise location of the laser sheet within the reconstruction was determined, the velocity field was transformed onto the laser sheet coordinate system and interpolated onto the specified coordinates given to the participants using a cubic interpolation algorithm to yield the ground truth solution. The velocity components of the ground truth solution are denoted by \(U^{*}_{{\mathrm {true}}}\), \(V^{*}_{{\mathrm {true}}}\), and \(W^{*}_{{\mathrm {true}}}\).

A set of 100 images from Camera 1 and Camera 3 was provided to the participants for evaluation. The images were saved in an uncompressed 16-bit TIFF format with a resolution of 1024\(\times\)1024 pixels. Furthermore, calibration images were provided showing a grid translated to seven different Z-axis positions with a spacing of \(Z = 1 \, {\mathrm {mm}}\). The grid consisted of two levels spaced \(3 \, {\mathrm {mm}}\) apart containing a rectilinear grid of \(3.2 \, {\mathrm {mm}}\) diameter dots. The dots on the grid were spaced \(15 \, {\mathrm {mm}}\) apart both horizontally and vertically.

The participants were required to submit a 2D3C (two-dimensional/planar, three-component) velocity field measured at 121 \(\times\) 121 vector locations spaced \(\Delta X = \Delta Y = 0.45\, {\mathrm {mm}}\) apart. Vector fields from the central 50 image pairs were requested allowing the participants to perform multi-frame processing methods; however, the participants were free to choose any processing method to produce the velocity vectors. In addition, the participants were asked to provide the vorticity field evaluated at the same coordinate grid as the velocity field as well as estimates of the vortex circulation calculated at the 50 time steps. The vorticity and circulation results showed negligible variation between the participants and provided little additional insight. Therefore, for the sake of brevity, a discussion of these results is omitted here.

Using the ground truth volumetric reconstruction velocity estimation, the absolute mean error for each of the measured velocity vector components was calculated as \(\Delta U^{*} = | U^{*} - U^{*}_{{\mathrm {true}}} |\), \(\Delta V^{*} = | V^{*} - V^{*}_{{\mathrm {true}}} |\) and \(\Delta W^{*} = | W^{*} - W^{*}_{{\mathrm {true}}} |\). The results are shown in Fig. 57a using bars to represent the mean absolute error and whiskers to show the corresponding standard deviation of the error. For all cases, the in-plane velocity errors were below 0.05 pixels—with DLR and IOT showing marginally lower errors compared to the other teams and approximately equal to the hacker cases. The other three teams (Dantec, LaVision and LANL) showed approximately equal in-plane errors with only the Dantec V-component appearing higher. The corresponding standard deviations of the in-plane velocity components were comparable for all cases.

The out-of-plane W component results exhibit more pronounced differences with errors varying from approximately 0.05 pixels for the HC24 and HG24 cases to approximately 0.12 pixels for Dantec and LANL cases. DLR and IOT produced the lowest errors approximately equal to 0.07 pixels while LaVision produced data with errors around 0.1 pixels. The amplification of the error of the out-of-plane component is anticipated and is captured by the calculation of the error ratios \(e_r \left( U^{*}, W^{*} \right) = \mu \left( \Delta W^{*} \right) / \mu \left( \Delta U^{*} \right)\), and \(e_r \left( V^{*}, W^{*} \right) = \mu \left( \Delta W^{*} \right) / \mu \left( \Delta V^{*} \right)\). The error ratios of the participant data are shown in Fig. 57b and compared against the theoretically expected ratios based upon the geometric configuration of the cameras. The theoretical error ratio for Cameras 2 and 4 equals 1.2 and for Cameras 1 and 3 equals 2.9. Considering the processing parameters employed by each participant (Table 16) and the results of Fig. 57, it appears that the in-plane accuracy was only affected by the choice of the multi-frame processing algorithm. The results also show that choice of the camera calibration model had relatively little impact on the error since the data from DLR and IOT produced nearly identical errors, although these teams used pinhole and polynomial calibration models, respectively. Moreover, the decision to use either the geometric or the generalized reconstruction algorithms did not have a noticeable impact on the results since neither method produced significant differences in the measured errors in either the participant or the hacker data.

The out-of-plane velocity is a function of the in-plane velocity, the stereo reconstruction method and the camera calibration function. The in-plane velocity error was found to be similar for all teams and the choice of the stereo reconstruction algorithm and camera calibration model appeared to not affect the out-of-plane velocity error. However, differences in the out-of-plane error and error ratios are still apparent among the teams. Thus it is possible that these differences can be attributed to the choice of the stereo self-calibration procedures employed by the teams.

Open image in new window

A more thorough characterization of the underlying statistics of the error magnitude is shown in Fig. 58 which shows plots of the error distributions and the statistical median, mode, and quartile values of the errors. The error distributions qualitatively follow a log-normal behavior with the IOT and DLR distributions showing a narrower spread and almost converging on the hacker cases. The spread in the error distribution, which scales with the RMS error, is wider for LaVision, Dantec and LANL, in increasing order. The mode of the error is the highest for Dantec. While in-plane errors contribute to the total error shown in Fig. 58, the differences in the error distributions are primarily driven by the increased error in the out-of-plane velocity component. The near-optimal stereo configuration given by the hacker cases HC24 and HG24 yielded the tightest error distributions with mode values approximately equal to 0.05 pixels and with 50% of the error population residing between 0.05 and 0.10 pixels. The error distributions produced by the DLR and IOT data resulted in similar modes and lower quartiles; however, the upper quartiles of the DLR and IOT data were increased to approximately 0.15 pixels. This increase in the error can possibly be attributed to the elevated errors observed within the vortex ring viscous core region where the high shear and particle rotation significantly impact the accuracy of both the in-plane and out-of-plane velocity estimations. This effect is shown in Fig. 59 which compares representative velocity error maps for both the in-plane and the out-of-plane velocity components between the DLR and the HG24 cases.

Open image in new window

Open image in new window

The results of the error analysis of the participant submissions and the hacker cases show several important considerations with respect to stereo-PIV experiments. The most pronounced observation that emerged from the analysis is that collecting and processing high-quality planar 2C2D velocity field data will yield high-quality 3C2D data. This process includes designing the experimental setups to collect PIV data in as optimal conditions as possible—for example, placing the cameras in an optimal stereo configuration was shown by the hacker cases HC24 and HG24 to produce lower out-of-plane errors. In addition, using the most advanced PIV algorithms available to process the two-dimensional data—in particular the multi-frame methods—yielded lower in-plane and out-of-plane errors.

The error analysis also showed that the choice of camera calibration model and the choice of the stereo reconstruction methods have relatively little effect on the quality of the data, as long as the in-plane velocity error was already low. There was relatively little optical distortion in the data collected for the case E experiment; however, in more complicated experiments, using a higher-order camera calibration model such as the third-order polynomial model may still be beneficial. Furthermore, while camera self-calibration could not be directly addressed by case E, the results show that performing an accurate self-calibration is vital to producing high-quality stereo reconstructions and thus it is important to ensure that collected calibration data are accurate. The result analyzed herein suggests that the self-calibration may be the most critical element in the stereo-PIV processing based on the current state of the method.

Images of particles with “known” displacement are important in estimating the error associated with PIV image analysis. While synthetic images are often used for this purpose (Kähler et al. 2012b), images of real particles are physically more accurate if the particle displacement field is precisely predetermined. For Case F images are obtained by recording the particles in a liquid column rotating at a steady uniform angular velocity. In this flow field, any velocity component in a Cartesian coordinate is proportional to the linear distance, and vorticity is uniform everywhere.

A plexiglass cylinder 10 mm deep \(\times\) 65 mm in diameter was filled with water/glycerin solution containing silver-coated hollow-glass particles (S-HGS-10, \(10\,\upmu {\mathrm {m}}\), Dantec). The cylinder was then covered with a glass plate (#62-606, BK7, 1/4” Edmund Scientific) to allow optical access through the glass without distortion of the image due to deformation of the liquid surface. The cylinder was placed on the turntable of an audio record player that spun at a constant speed of \(\omega = 33+1/3\) rpm (=\(10 \pi /9\) rad/s). After a few minutes, the flow inside the cylinder reached steady solid-body rotation. A laser beam produced by an Nd-YAG laser (EverGreen 70, 2 \(\times\) 70 mJ, 15 Hz, Quantel) was expanded by a cylindrical lens to form a laser light sheet approximately 4 mm thick. The light sheet illuminated a horizontal cross section of the fluid in the cylinder, and particles were imaged on a CMOS camera (Fastcam SA3, 1024 \(\times\) 1024 pixels, Photron) through a lens (AF Nikkor 28–105 mm, Nikon). The focal length and F-number were set at 180 mm and 22, respectively, and thus the dimensions of the field of view were 38 \(\times\) 38 mm. Trigger signals to determine the timing of camera exposure and laser pulse were sent from a timing generator (Type 9618, 8 channels, Quantum Composers). While exposure time of the camera was set at 8 ms, a frame-straddling scheme allowed to set time intervals between the two laser pulses fired during each frame period at \(\Delta t = 4\) ms. A total of 1000 image pairs were acquired in 100 s.

While flow inside the cylinder was expected to be stationary relative to the frame rotating around the center of the turntable, a negligible amount of convection remained. As a result of this convection, there was uncertainty in the displacement field involved in the particle image. It was identified as follows. The double-pulse laser was fired at \(t=0\) and \(t=\Delta t_{{\mathrm {rev}}}\), where \(\Delta t_{{\mathrm {rev}}} = 2 \pi / \omega\) was identical to the time period of one revolution of the cylinder, and particle images were exposed on two independent image frames. Particle displacement between the images would be zero everywhere if the convection were absent; actual displacement, however, was \(\delta _{{\mathrm {rev}}} = 0.53\) mean \(\pm 0.32\) s.t.d. pixels. Thus, the uncertainty of displacement in the particle images of \(\Delta t = 4\) ms interval used for this study was \(\delta = \delta _{{\mathrm {rev}}} \Delta t / \Delta t_{{\mathrm {rev}}} = 1.2 \cdot 10^{-3}\) mean \(\pm 0.7 \cdot 10^{-3}\) s.t.d. pixels.