Image-based appearance acquisition of effect coatings

Paint manufacturers strive to introduce unique visual effects to coatings in order to visually communicate functional properties of products using value-added, customized design. However, these effects often feature complex, angularly dependent, spatially-varying behavior, thus representing a challenge in digital reproduction. In this paper we analyze several approaches to capturing spatially-varying appearances of effect coatings. We compare a baseline approach based on a bidirectional texture function (BTF) with four variants of half-difference parameterization. Through a psychophysical study, we determine minimal sampling along individual dimensions of this parameterization. We conclude that, compared to BTF, bivariate representations better preserve visual fidelity of effect coatings, better characterizing near-specular behavior and significantly the restricting number of images which must be captured.


Introduction
Nowadays, digital appearances of materials play an important role in computer graphics and related industrial fields like gaming and the film industry. They are also crucial to many industrial sectors dealing with virtual product design and prototyping, as virtually designed products are often used for marketing even prior to the commencement of production. Therefore, the realism of material visualization and speed of the entire virtual product development cycle are major factors determining success in a market. Methods visualizing digital material appearance rely either on a mathematical model of materials or on captured data. In both cases, one requires measurements to identify the main features of material appearance; the amount of data required is always driven by the material appearance representation used.
A four-dimensional bidirectional reflectance distribution function (BRDF) [1] describes the distribution of energy reflected to some specific viewing direction when illuminated from some other. Its extension, a six-dimensional spatiallyvarying BRDF (SVBRDF), additionally captures spatial dependency of reflectance across a material surface. Whereas BRDF and SVBRDF impose restrictions on reciprocity, opacity, and a range of sample height variations, the six-dimensional bidirectional texture function (BTF) [2] generally does not meet these restrictions. This is due to non-local effects in rough material structure such as occlusions, masking, subsurface scattering, and inter-reflections. Regardless of the representation used, the measurement process faces the problem of high dimensionality due to the sampling of many combinations of incoming and outgoing directions. This in practice results in the requirement for time-and resource-heavy capturing of thousands of bidirectional samples. The main motivation of this paper is to reduce the necessary number of captured images while still preserving high fidelity of a material's visual features. Identification and minimization of the number of bidirectional samples required leads to an accelerated measurement process and consequently to the introduction of less complicated appearance capture devices, benefiting various industries.
Our analysis utilizes the half-difference parameterization [3]; it can be applied to materials with nearlyflat random structure represented by spatially-varying texture. A specific case of such material appearance is exhibited by effect coatings, e.g., metallic or pearlescent paint, where the distribution of flakes in a base coat causes globally isotropic appearance variations with local anisotropic high-frequency reflectance changes. It is often referred to as "sparkle", and is caused by randomly oriented, glittering particles. The structure of effect pigments is usually visible from several meters, so the visually correct representation of spatially-varying information is essential.
The main contributions of this paper are: • analysis of several spatially-varying appearance representations stemming from half-difference parameterization; • a perceptual experiment identifying minimal sampling along dimensions of this parameterization; • validation of the minimal sampling sets on groundtruth data for static and dynamic rendering scenarios. Our paper is structured as follows. Section 2 reviews prior work in effect coating appearance representation and capture. Section 3 discusses halfdifference representation and its properties. Section 4 describes material samples and acquisition devices. Section 5 introduces the appearance representations tested while Section 6 compares their visual performance with respect to ground-truth data, and analyzes their dynamic properties. Section 7 discusses method limitations and Section 8 concludes the paper.

Related work
This section overviews techniques related to material appearance data parameterization, capture, and modelling of effect coating appearance.

BRDF parameterizations
Instead of using a standard parameterization of incoming and outgoing directions based on spherical angles, Rusinkiewicz [3] introduced half-difference parameterization, which has better axis-alignment to typical variations in real materials. Romeiro et al. [4] suggested a method of BRDF dimensionality reduction in a half-difference parameterization giving a bivariate representation. An in depth analysis of BRDFs in half-difference parameterization was given by Burley et al. in Refs. [5] and [6], who introduced a characteristic slice, i.e., a θ h × θ d subset of BRDF space, and showed its convenience for analysis of physically related properties of the BRDFs in the MERL database. A parameterized non-linear sampling along θ h in the characteristic slice was applied in Ref. [7] for measurement of highly specular materials. Ward et al. [8] mentioned the possibility of using bilateral, quadrilateral, and radial symmetry to simplify the BSDF measurement process. While Helmholtz reciprocity and radial symmetry is widely used in many analytical BRDF models, its practical applicability to the spatial domain was not thoroughly studied due to the need for a goniometric acquisition device. Ferrero et al. [9] characterized the BRDF of special effect coatings using a half-difference parameterization of individual flakes, where half and difference elevation angles θ h , θ d are denoted by flake disorientation angle θ flake and flake-related illumination angle θ inc respectively. The same parameterization was applied in Ref. [10] for predicting the global appearances of effect-pigmented surfaces. Several methods exist for sparse acquisition of anisotropic BRDFs, either using minimal sampling in combination with a precomputed linear basis [11], or adaptive sampling along 1D slices in BRDF space [12].

Acquisition of spatially-varying reflectance
As effect coatings generally satisfy the assumptions of reciprocity and isotropy, a spatially-varying BRDF (SVBRDF) can be considered to be a suitable representation for such materials. Recently, several approaches for fast acquisition of SVBRDF have appeared. Dong et al. [13] estimated local anisotropic behavior using sparse sampling by near-field reflectometry and completed appearance reconstruction by assignment of these measured BRDF clusters to non-measured per-pixel locations. Aittala et al. [14] introduced a system of SVBRDF capture without moving parts, allowing two-shot acquisition by means of a specific illumination pattern and processing in the frequency domain. Aittala et al. [15] proposed an approach to single-shot SVBRDF acquisition relying on a texture descriptor based on a deep convolutional neural network. Although these advanced approaches perform well for a variety of real-world materials, their ability to correctly capture specular peaks and sparkling features of effect coatings is unknown. To this end, capable procedural approaches for the realistic rendering of glossy sparkling materials based on micro-facet theory exist [16]; however, a side-by-side comparison of their results to photographs of real materials has not been presented. Den Brok et al. [17] demonstrated the possibility of accurate BTF reconstruction from sparse multiplexed measurements. Spatially localized anisotropic effects of fabrics were fitted from BTF data in Ref. [18].

Acquisition and modelling of effect coatings
One of the first attempts to model effect coatings by means of the BRDF model was by Günther et al. [19], where the spatial sparkling effect was introduced using either physical [20] or statistical [21,22] models of sparkling flake sizes and orientations. Later, Mihalik and Durikovic [23] introduced a method of separately capturing and fitting BRDF data and spatially-varying sparkle effects. More advanced techniques extend to spatially-varying appearance representation using a BTF. Rump et al. [24] collected dense BRDF measurements from relatively large samples and fitted the Cook-Torrance model to measured BRDF data. The residual sparkling appearance was represented using BTF images compressed for individual viewing directions using PCA. During the visualization stage, the BRDF value was combined with a randomly sampled BTF patch. In follow up work, Rump et al. [25] considered detailed angular analysis of flake properties from video. The information obtained was used to guide sampling and example-based compression of the paint's measured BTF. Compression is based on histogram clustering and achieves significantly higher compression rates than previous approaches. An extended statistical representation for realtime rendering of effect coatings was presented in Refs. [26,27]. Kautz et al. [28] and Xu et al. [29] extended BTF by editing operations, allowing modification of its local geometrical and reflectance properties. Alternatively, anisotropic effects can be also introduced using statistical models. Yan et al. [30] presented a parametric model of anisotropic glints based on local normal distribution functions. An approach to scalable modelling of anisotropic effects using anisotropic spherical Gaussians was proposed in Ref. [31].

Psychovisual analysis of material appearance
Psychovisual analysis of material appearance aims to provide intuitive navigation BRDFs in parametric space [32][33][34]. Further work considered perceptually driven compression [35] and filtering [36] of bidirectional texture functions. We are unaware of any psychophysical analysis of low-dimensional effect coating representations operating in a spatio-angular domain. BRDF measurement and modelling methods disregard important spatial information. While BTFbased methods allow photorealistic visualization, they fall short due to their high demands during acquisition, compression, and visualization stages. Flake physical and empirical modelling approaches do not require extensive input data, but often suffer from limited descriptiveness of the model and inaccurate parameter fitting. Although the SVBRDF might seem as a good representation for effect coatings it has several disadvantages. Pigments exhibiting anisotropic effects have highly specular locally anisotropic behavior, and per-pixel appearance is often result of light interaction with multiple pigment flakes and coating layers rather than pure mirror-like reflection from a uniformly oriented flake. In contrast to the reviewed approaches, our method does not rely on any form of data fitting or support of BRDF models. Further, it requires capture of a lower number of images than BTF while allowing for a correct representation of near specular behavior.

Appearance parameterization
The BRDF is a four-dimensional function f r (θ i , A general four-dimensional function can describe anisotropic materials, i.e., those with variable reflectance when rotated about a surface normal. This property is common for many real-world materials, due to directional elements such as thread in fabric or grain in wood. A three-dimensional simplification of a BRDF is called an isotropic BRDF f r (θ i , θ v , Δϕ = ϕ v − ϕ i ), and it neglects anisotropic appearance.
A main limitation of standard parameterization using spherical angles is its inefficient representation of important features, such as specular highlights, not aligned with axes of the coordinate system. Therefore, locations of highlights depend on several variables which limit practical usage of the standard parameterization (such as BRDF compression). These limitations can be overcome by means of a half-difference (HD) parameterization proposed by Rusinkiewicz [3]. He suggested a change of variables from The half-vector parameterization is, due to its effective description of specular highlights, often used as a core of analytical BRDF models, e.g., Refs. [37,38]. The shape of specular highlights is represented by the elevation angle of the half vector θ h , and the specular reflection is obtained for θ h = 0 • . A non-linear sampling along θ h is often used to reproduce a highly specular material, allowing use of a lower number of samples [32]. Anisotropic behavior is controlled by the azimuthal angle of the half vector ϕ h , representing rotation of a sample around its normal. Reciprocity is introduced by symmetry of the azimuthal angle of the difference vector ϕ d = ϕ d + π. Therefore, angles range as:

BRDFs can be described by the three
Marschner [39] suggested how to reduce captured data by means of bilateral symmetry and an extension to the concept of isotropy. Rusinkiewicz [3] defined BRDF reciprocity in half-difference parameterization as folding data along angle ϕ d = ϕ d + π. Romeiro et al. [4] studied half-difference parameterization and concluded that reciprocity and bilateral symmetry impose periodicity for incoming and outgoing directions about a half-way vector: They further defined bivariate BRDFs as constant functions of ϕ d , understanding their properties as a direct generalization of isotropy, bilateral symmetry, and reciprocity. A simple experiment on the UTIA BRDF database shown in Fig. 2 demonstrates that bivariate representation using constant ϕ d at 90 • provides the lowest BRDF reconstruction error. This is due to the fact that for ϕ d = 90 • , the majority of the effective part of the hemisphere around half-direction H is above the plane defined by the material surface (see Fig. 1). Figure 2 also demonstrates symmetry around ϕ d = 90 • , suggesting that the majority of BRDFs are bilaterally symmetric [4]. Finally, our analysis revealed that using fixed ϕ d = 90 • introduced only a relatively low reconstruction error: 45-58 dB for PU PSNR [40].

Material specimens
Effect pigments can be, based on the principle of chroma and sparkling effect generation, roughly divided into three categories that in practice are often combined in coating systems [41]: (1) metallic pigments, relying mainly on geometrical properties of pigment flakes and their reflectance, (2) interference pigments, in which color effects are caused by light wave interference with a transparent substrate coated with materials of high refractive indices, and (3) diffractive pigments, decomposing light at a diffraction grating of a frequency close to the wavelength of the incoming light. We measured four kinds of coatings featuring state-of-the-art effect pigments in their base coat layers (see Fig. 3), introducing specific spatial behavior that plays a key role in the enhancement of material visual attractiveness: VMP (Decomet): a mirror-like, highly-reflective coating (category 1) based on well-aligned and oriented vacuum metallized pigments (VMPs) with mean particle size 12 μm, Silverdollar (AluMotion): an automotive coating including aluminum flakes of silverdollar morphology (category 1) with mean particle size 18 μm, UTP (Zenexo): a high-sparkle contrast, pearlescent coating including an ultra-thin, colored aluminium pigment (UTP) with a mean particle size 21 μm (category 1), and Polychromatic (MultiFlect): a polychromatic coating (category 3) including a holographic effect pigment with a mean particle size 150 μm. All but Decomet feature high sparkle contrast due to angularly dependent pigment reflectivity, and are highly-specular due to a clear coat used on top of the effect pigmented base coat layer. The polychromatic pigment also exhibits a unique, gonioapparent chromaticity due to a regular grating of pigment flakes introduced by means of optical lithography. Due to its specific appearance, this material represents a challenge to the both capture and visualization processes.

Appearance acquisition
We used a four-axis gonioreflectometer [42] to capture the angularly dependent appearances of coatings. The size of the captured sample was 6 cm×6 cm, camera distance was 2 m, and light distance was 1.5 m.
The spatial resolution of the images was 350 DPI. The device was geometrically and colorimetrically calibrated, and the measured flat sample normal was accurately aligned within the coordinate system of the device.
For all tested representations, a set of material surface images was taken with varying lighting and viewing directions, and small tiles of approximately 120 × 120 pixels, corresponding to 5 mm×5 mm, were cut [43]. Raw tiled image data were stored in uncompressed binary format and directly accessed using lookup functions in our OpenGL rendering application. Linear interpolation was used for values for non-measured directions: for BTF representation, this was barycentric interpolation using the 3 closest directions, while for half-difference parameterization we used linear interpolation from the two closest directions in each dimension.

Pigment angular behavior
We analyzed typical dynamic behavior and the angular lifetime of individual effect pigments. We fixed a camera at 45 • and captured the entire in-plane geometry of illumination elevations with sampling step 1 • . Analysis of individual pixels revealed that the angular pixel lifetime was between 5 • and 10 • . We selected a representative pixel with the highest contrast as shown in Fig. 4. One can observe different  behavior for different effect coatings. For a polychromatic coating with large flakes, we obtained wider and smoother peaks than for UTP and silverdollar with fine flakes. For fine flakes, the scattering over the peak can be caused by the contributions of several flakes to the same pixel location. Additionally, we show a non-linear sampling in θ h for steps 5 • (red circles), validating that this density should suffice to reasonably characterize typical pixel dynamics.

Capturing spatially-varying appearances of effect coatings
This section introduces variant representations of individually tested effect coating appearance, and compares renderings using them.

Uniform sampling using BTF
First, we captured the spatially-varying appearance of tested materials by means of a BTF [2] using 81 illumination and viewing directions distributed uniformly over a hemisphere [44]. This produced 6561 HDR images of the material surface. Measurement using a gonioreflectometer took approximately 20 hours; the resulting appearance of the captured BTF is shown in Fig. 8(a). As the BTF angular sampling is uniform, the shape of the specular highlight is not sampled properly, and intensity captured for ideal specular reflection leaks out to neighboring directions due to angular interpolation. To avoid these issues, we replaced values in specular reflections by the closest values captured nearby. In addition, we used a Phong model modulated by Schlick's approximation of Fresnel reflection to represent the highlights more properly where mix(.) is a linear blending function, c s is specular RGB color, k is an attenuation parameter, and α is a shininess parameter. H, V , R are half vector, viewing vector, and ideal mirror direction.
Although we fit intensity and RGB color to the discarded specular reflection data, the model is, due to lack of data, unable to correctly fit the unknown width of the specular highlight as well as the characteristic near-specular behavior of flakes. Moreover, physical parameters of the measured materials, e.g., refractive indices, are unavailable. Therefore, we fit the parameters manually to obtain the best perceptual results. Results are shown in the second row of Fig. 8(b). Although Fresnel reflection can improve the appearance of specular highlights in a BTF considerably, unless additional samples are taken near specular highlights, performance of automatic fitting is limited. Note that this process was not applied to the half-difference representationbased sampling approaches presented in the paper.

Sampling in half-difference parameterization
This section proposes several data reduction variants of half-difference parameterization. We use this representation as it allows for the capture of near specular reflections in more detail, while reducing the number of angular samples needed. If one were to use a sampling step of 5 • in all dimensions while using a standard square root non-linear sampling along elevation of half-vector θ h , and to rely on the reciprocity assumption, one would need to capture n θ h ×n θ d ×n ϕ h ×n ϕ d /2 = 18×18×72×72/2 = 839808 images. Finally, one could apply bilateral symmetry by additional folding along ϕ d = ϕ d +π/2 [4], arriving at 18 × 18 × 72 × 19 = 443232 images. As this number of images is still too high to be captured using the goniometer, we reduced them by resorting to assuming constant BRDF values along dimensions ϕ d and ϕ h . We tested four options.

Isotropic representation: constant
Our analysis of BRDFs of captured materials in a typical BTF angular sampling [44] (see the first row of Fig. 5) revealed that they do not globally exhibit any anisotropic behavior. This was expected as the data microstructure is arbitrary, due to a random distribution of flakes in the binder. In theory, the isotropic assumption works only for nearly flat materials that do not exhibit non-local shading and masking effects. However, it also can be extended to special effect paints exhibiting globally isotropic appearance when these effects are present but considerably suppressed due to relatively thin layers of binder and upper coatings. Unfortunately, this representation neglects local anisotropic effects of individual flakes and impacts its dynamics as discussed further. As local anisotropic behavior is neglected, the same values are replicated for all azimuthal directions of sample orientation. Applying isotropy, reciprocity, and bilateral symmetry assumptions reduces the number of images needed to 18×18×19 = 6156 images. However, some of the combinations are invalid as they position incoming or outgoing directions below the upper hemisphere, resulting in only 4951 valid samples. Further, since our goniometer has a technically limited range of incoming and outgoing grazing angles, we restricted the maximal elevations to 75 • . This further reduces the number of captured images to 3485 unique bidirectional samples. When we visualize appearance using such captured isotropic representations, we obtain results as shown in Fig. 8(c). We observe a considerable improvement in the capturing of specular highlights while still preserving important visual features.

Bivariate isotropic representation: constant
One can restrict the number of samples further by assuming radial symmetry in the material structure. This holds globally due to the random distribution of flakes and disregards the effects of mutual viewing and light rotations around the half-vector. Then we can extend the concept of bivariate BRDF [4] to bivariate SVBRDF and capture only images belonging to the characteristic slice depending on θ h and θ d [5] for fixed ϕ d = 90 • , which conveys most of the information. This restricts the number of images to n θ h × n θ d = 18 × 18 = 324; again due to the technical restriction on maximum elevation, only 243 images are captured. Renderings using this subset are shown in Fig. 8(d) and are barely visibly different to the complete isotropic representation in the third row.
Reduced bivariate isotropic representation. We captured the bivariate representation with a sampling step of 2 • along both dimensions. For incoming and outgoing elevations in [0 • , 75 • ], we collected a total of 1471 images. When the values across the images were averaged, we obtained characteristic BRDF slices as shown for three of the materials in Fig. 6. Apart from a typical specular highlight presented by white stripes at the tops of the images, for the polychromatic sample we observe a rainbow light diffraction effect of two subsequent diffraction orders [45]. We performed a psychophysical study analyzing maximal sampling steps in this representation (see Appendix) and we conclude that one can use 5 • sampling steps along θ h and up to 15 • sampling steps along θ d , resulting in just 83 samples. Distribution of samples in this minimal bivariate isotropic sampling is illustrated in Fig. 7.

Anisotropic representation: constant ϕ d (Δθ
One can extend bivariate representation to preserve local anisotropic effects (i.e., dynamic flake glittering) by acquiring the characteristic slice θ h and θ d for various azimuths ϕ h of material rotations under a fixed minimal measurement geometry specified by sampling along θ h and θ d (see Fig. 7). For azimuthal steps Δϕ h = 15 • , 83×n ϕ h = 83×24 = 1992 images are taken. Comparison of renderings to those from other representations in Fig. 9(e) shows similar performance for static scenes; however, this representation better captures dynamic behavior of flakes, as discussed in Section 6.4.

Bivariate anisotropic representation: constant
Finally, we further limited the anisotropic representation by keeping both ϕ d and θ d constant and thus acquiring only a data slice characterized by spherical angles defining the half-vector, i.e., θ h and ϕ h . The sampling points along θ h correspond to cameras and lights in the same direction (simultaneously red and blue points in Fig. 7). These sampling points are then replicated in different azimuthal angles specified by ϕ h . Note that sampling along θ h globally characterizes the coatings appearance, while sampling along ϕ h brings anisotropic sparkling effects. For a half-vector elevation step Δθ h = • and azimuthal step Δϕ h = 15 • , only 15 × 24 = 360 images are needed. As building such a coaxial setup might be impractical in many cases, we also tested positions of light and camera 15 • azimuthally apart. In fact, we obtained nearly identical results. A comparison of renderings to those from other representations in Fig. 9(f) shows similar performance for static scenes; however, this representation is superior in dynamic scenarios as discussed in Section 6.4. It preserves both dynamics of flakes as well as accurate near-specular behavior, while compared to BTF representation, it requires only a fraction of the samples as well as a less complicated acquisition procedure with mutually fixed lights and cameras rotating around the measured material.

Results
This section compares the results from the tested representations to photographs and ground-truth data using two independent experiments. Further, it also compares dynamic properties to a video of a real coated object.

Perceptual comparison of tested methods
In this psychophysical experiment we compared subjects' impressions from a rendering of static scenes using the tested representations. A total of 106 subjects responded as to whether they could observe differences between rendered images. The study was web-based, i.e., run in an uncontrolled environment using various screens, although subjects were advised to run it in a dim room environment. Finally, from yes/no (1/0) responses across all subjects, a mean opinion score was computed. We compared visualizations of BTF+Fresnel vs. isotropic SVBRDF representation (the second and third rows of Fig. 8) and of isotropic vs. isotropic bivariate SVBRDF representations (the third and fourth rows of Fig. 8).
The mean opinion scores in Fig. 10 show that the majority of subjects were able to clearly differentiate (91% of subjects) between images captured as a BTF and those captured as an isotropic SVBRDF as shown in the graph on the left, while many fewer (34% of subjects) were able to differentiate between isotropic and isotropic bivariate SVBRDFs as shown in the graph on the right. Note that for both representations a step of 5 • was used along all dimensions. We assume that the differences between the BTF and isotropic SVBRDF were biased by different shapes of specular highlight due to its modelling for BTF data by the Fresnel model. Another possible bias could be caused by subjects focusing on local flakes rather than on global visual similarity; we advise the reader to assess the methods in the video in the Electronic Supplementary Material (ESM) for themselves.

Validation using photographs
Probably the most straightforward way of assessing captured appearance quality is by comparing it to real appearance. To this end, we coated objects with two effect pigments: VMP and Polychromatic.
Moreover, Polychromatic was used both with and without a clear coat (CC). We took a photograph of the coated objects inside the acquisition setup and then compared it with renderings from several appearance representations. To achieve this, we captured the 3D geometry of the object and adjusted viewing and lighting conditions so as to render its appearance similar to that in the photograph.
Our results for Polychromatic with clear coat are shown in Fig. 9 (more results are shown in the ESM). BTF enhanced by Fresnel reflection shows  an unnaturally clear specular highlight without nearspecular sparkling. The representations based on the half-difference parameterization perform considerably better. Moreover, their results can be further improved by using parametric non-linear sampling along θ h as suggested in Ref. [7]. The elongated shape of the specular highlight in our renderings, when compared to the circular one in the photograph, is likely due to use of an ideal rather than real point light source with a non-zero area. We can eliminate possible impact of our visualization pipeline as these these elongated highlights caused by object geometry also appear in the per-pixel ground truth measurements in Fig. 11(a). An additional psychophysical evaluation of photographs and rendered images, shown in the ESM, concludes that, while their performance of isotropic and bivariate representations is comparable, they perform significantly better than BTF-based approaches. This supports our conclusion that bivariate representation is a promising approach for the effective capture of the appearance of effect coatings in static rendering scenarios.

Validation using ground-truth
In the second validation experiment, we selected a small area of 100 × 50 pixels around a specular highlight in a rendered image, as shown in the first row of Fig. 11. For each of these rasterized pixels, the underlying 3D model geometry was evaluated and illumination and viewing directions restored; then, exactly these directions were used for the bidirectional measurement of materials. In this way, we obtained a ground-truth data image of resolution 100 × 50 pixels as shown in Fig. 11(a) (intensity of the VMP material is reduced by a factor of 10 for better visibility). This data was used for a direct pixel-wise comparison with each tested appearance representation: (b) a raw BTF, (c) a BTF with Fresnel model for specular highlights, (d) isotropic measurements, (e, f) bivariate isotropic measurements for fixed ϕ d = 90 • and angular steps in Δθ h /Δθ d = 2 • /2 • and 5 • /15 • , respectively. Note that in this particular case, the parameters of the Fresnel model were manually tuned to provide results as close as possible to the groundtruth data, but one can still spot a considerable difference from the original. On the other hand, the appearances from isotropic and bivariate isotropic representations show a very close resemblance to the ground-truth data.

Analysis of dynamic behavior
This section compares dynamic properties of real effect coating appearance to renderings based on the tested variants of half-difference parameterizations. We analyzed this behavior in video of an object coated with polychromatic pigments alongside renderings of: a BTF+Fresnel model, isotropic measurements, minimal bivariate isotropic sampling for Δθ h /Δθ d = 5 • /15 • , and anisotropic sampling using minimal bivariate sampling for different sample orientations with Δϕ h = 15 • . To capture ground truth behavior of flakes, we followed the analysis of typical angular flake lifetime from Ref. [25] and varied both light and object positions in 0.5 • steps, taking 720 images for the full rotation of light/object. Videos showing the dynamic appearance of tested representations compared to real material are provided in the ESM.
We also compared dynamical behavior of the representations computationally by analyzing perpixel behavior at point positions across 720 frames. Figure 12 shows per-pixel variance for the tested methods; their mean values across an entire object area are shown in Fig. 13. First, one can see that isotropic and bivariate isotropic representations show significantly fewer active flakes and thus very low dynamics, due to replicating the same images for various ϕ d and/or ϕ h angles. On the contrary, BTF with Fresnel reflection modelling achieves significantly more excited flakes and thus preserves dynamics much better but cannot preserve the surroundings of specular highlights correctly as shown in Fig. 12. Finally, the anisotropic representation performs even slightly better than BTF as it captures azimuthal views for different sample orientations.
Although there are still some differences in videos due to natural phenomena not captured in the renderings such as light reflection from the table, we can conclude that while disregarding variation along ϕ d does not have a significant impact on visual quality, capturing anisotropic behavior along ϕ h is crucial for preserving local anisotropic behavior in dynamic scenes. In other words, disregarding effects of light and viewing direction rotations around a half-vector is significantly less visually apparent than disregarding effects of light and viewing direction rotations around a surface normal. This conclusion is also supported by the fact that the bivariate anisotropic representation  has the best dynamic behavior as shown in Figs. 12 and 13.

Recommendations on effect coating measurement
Our results suggest that only minimal bivariate isotropic sampling of a characteristic slice is needed for effect coating reproduction in static scenarios, while for dynamic ones, the best results can be achieved using a bivariate anisotropic slice, at the cost of slightly more samples. For bivariate isotropic representation one needs only to sample a quarter hemisphere over the material symmetrically along a plane orthogonal to that defined by a surface normal and half-way vector as shown in Fig. 7. This suggests the possibility of designing a measurement device with only a single mechanical degree of freedom consisting of an arc simultaneously holding lights and cameras rotating over the material surface around a horizontal axis. Non-linear uniform sampling along θ h could be realized by using various nonuniform rotation steps of the arc, while sampling along θ d would be determined by the number of lights and sensors and their distribution over the arc; only six are needed in the case of Δθ d = 15 • . As already mentioned, using a bivariate isotropic representation for capturing of effect pigments also imposes limitations on the dynamics of pigment flake behavior. This can be resolved by using bivariate anisotropic representation, where one needs to uniformly sample only the half angle over the hemisphere. To capture better dynamics of individual flakes using this representation, another setup with one mechanical degree of freedom would suffice. This could be achieved by using an arc outfitted with lights, and cameras would rotate around the normal of the measured sample. We have found that variation along ϕ h represents most of the dynamic behaviour, so to capture dynamic behavior more accurately, we must increase sampling density specifically along this dimension.

Extension to other materials
We conclude that while bivariate representation works well for the reproduction of effect coatings in static scenes, for correct reproduction of their dynamic behavior, one should capture bivariate isotropic sampling for different material orientations. This approach neglecting only data along ϕ d can also be extended to other nearly flat anisotropic materials such as fabric, wood, etc. Figure 14(left) shows an example of such an extension to highly anisotropic polyester fabric providing the same visual quality as BTF representation while requiring less than a third of of the data images.

Future work
The current form of our implementation does not consider light refraction at the interface between a clear coat and coating layer. An update of captured illumination directions taking into account refractive indices of layers might further optimize performance for clear-coated materials. Our methods were tested on four effect coatings only; however, we do not view this as a limitation since we purposely tested it on challenging diffraction effect pigments, and assume that the conclusions drawn in this paper are valid for a majority of typical effect coatings. The captured bivariate representations are publicly available for research purposes at http://btf.utia. cas.cz in the form of binary files with corresponding loading and look-up functions.

Conclusions
This paper analyzes approaches to capturing spatiallyvarying appearances of effect coatings. We compare uniform sampling using a bidirectional texture function (BTF) with several sampling variants relying on assumptions of data constancy along dimensions of half-difference parameterization. In total we compared five different appearance representations on four effect coatings. Using several psychophysical experiments, we identified minimal sampling steps along individual dimensions of the half-difference parameterization. Further, we have shown that for static scenes, a bivariate isotropic representation provides satisfactory results, while for dynamic scenes, local anisotropic behavior of coating flakes can be most effectively captured by bivariate anisotropic representation, completely neglecting light direction with regard to half-vector. Both representations require considerably fewer captured images than BTFbased approaches, while representing near-specular behavior more accurately.

Appendix: Perceptually optimized bivariate sampling
This section describes a perceptual experiment identifying appropriate sampling steps along the two dimensions of the characteristic slice (θ h × θ d ). We captured the bivariate SVBRDFs with a sampling step of 2 • and analyzed perceived differences as a function of the sampling step along θ h and θ d . In this experiment we gradually downsampled the original angular resolution in both dimensions to identify the minimal resolution still preserving all visually relevant features of tested materials. We designed an experiment as a two-alternative forced choice (2AFC) decision task, i.e., we prepared experimental stimuli consisting of two images and asking subjects to state whether they could observe differences between the images. The subjects were advised to judge based on global appearance and not only on a localized comparison of individual pixels. The order of the two images presented was random, and the remaining space on the screen was set to dark gray. Note that although we did not use a ground truth in our experiment as it was not available, we did attempt to obtain information on visual distance between individual methods or their settings.
The study was web-based, i.e., run in an uncontrolled environment on various screens, although subjects were advised to run it in a dim room environment. As Fig. 6 demonstrates, most of the changes in data are along θ h (the vertical axis); the tested resolution steps were 2 • , 4 • , 6 • , 8 • , and 10 • , while the variations along θ d (the horizontal axis) were significantly lower: the tested steps were 2 • , 8 • , and 14 • . For each combination of values θ h and θ d we rendered one image. In each test, we showed such an image side-by-side with a reference rendering using the shortest sampling step 2 • along both dimensions. This gave us 15 stimuli per set and 45 images in total. The orders of images in a pair, and also pairs in a session, were random. No resampling to rendered images was applied when images were produced. A total of 106 subjects participated in the experiment out of which 65 completed all trials. We recorded on average over 75 responses for each stimulus image. The average duration of stimulus observation was 5.1 s and total time of the entire session was 4.1 min. To evaluate the significance of the obtained data we used hypothesis testing. Due to the dichotomous nature of the data, we used a Cochran Q-test [46] for testing a null hypothesis that subject responses to a comparison of renderings with different sampling steps had the same number of positive and negative responses. We found that for all three test sessions, we could reject the null hypothesis at a 5% significance level with p val < 10 −14 .
When we average yes/no (1/0) responses across all subjects, psychometric data are generated. The 3D plots in Fig. 15 show psychometric data as a function of sampling density along θ h and θ d for three tested materials.
The data in each graph suggest that while subjects performed similarly for Silverdollar and UTP, they perceived higher differences for Polychromatic. The higher visual differences resulted from a disruption of the original holographic effect due to coarse sampling. Note that the guess rate was on average 0.2, i.e. 20% of subjects reported a difference although the images were the same. An even more important finding was that while subjects were sensitive to the increase in sampling step along θ h , their responses to step increase along θ d were almost negligible. The psychometric data averaged along θ d with corresponding variances across all subjects are shown in Fig. 16. One can observe a significant drop of variance after θ h = 4 • where the majority of subjects   Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.