Recent advances in glinty appearance rendering

The interaction between light and materials is key to physically-based realistic rendering. However, it is also complex to analyze, especially when the materials contain a large number of details and thus exhibit “glinty” visual effects. Recent methods of producing glinty appearance are expected to be important in next-generation computer graphics. We provide here a comprehensive survey on recent glinty appearance rendering. We start with a definition of glinty appearance based on microfacet theory, and then summarize research works in terms of representation and practical rendering. We have implemented typical methods using our unified platform and compare them in terms of visual effects, rendering speed, and memory consumption. Finally, we briefly discuss limitations and future research directions. We hope our analysis, implementations, and comparisons will provide insight for readers hoping to choose suitable methods for applications, or carry out research.


Introduction
In the real world, many materials exhibit visual appearances with glinty effects, e.g., car paint under strong sunlight, and tiny scratches on heavily used cutlery. These glinty visual appearances are often complex and unstructured, and they greatly enhance the richness of the visual world. However, in computer graphics, the glinty effects in visual appearances are often ignored, leading to an overly perfect and smooth appearance (see Fig. 1). In order to improve the realism of computer-generated imagery (CGI), it is very important for a photo-realistic renderer to take such glinty appearance into consideration.
However, glinty appearance rendering is difficult. In the real world, tiny bumps and dents on the surfaces of objects can be found everywhere. These geometric details introduce high-frequency variations in appearance and cause glinty effects noticeable to the human eye. Therefore, in order to incorporate glinty appearance in rendering, one must start from the causes of such appearancecomplex microgeometry and the way it interacts with light. However, two difficult problems arise naturally. The first is how to represent this complex microgeometry both exhaustively (without overly simplifying or ignoring any geometry) and compactly (without introducing significant storage or memory consumption overheads). The second is how to render such complex microgeometry efficiently, faithfully bringing out the glints without prohibitive computation.
Historically, people have used microfacet models [2] to describe optical properties of surfaces, and more specifically, bidirectional reflectance distribution functions (BRDFs). Microfacet theory assumes that a surface is composed of many microfacets. Each microfacet causes a perfect mirror-like reflection. Traditional methods prefilter glinty appearance and smoothly approximate BRDFs, which results in a smooth appearance which omits glinty effects. It is the distribution of the orientations (normals) of the microfacets that determine appearance. However, microfacet theory uses a statistical approach (e.g., using 2D Gaussians) to describe the microfacets' normal distribution function (NDF). Since statistical functions are usually smooth and only focus on Fig. 1 A complex indoor scene with four kinds of glinty appearance. Left: rendering with traditional microfacet theory. Right: rendering using method from Ref. [1]. The glinty features on the wooden sofa legs, the leather sofa cushion, the bumpy base of the ornament and scratched metals dramatically improve the image's realism.
overall distributions rather than details, they inevitably produce smooth appearances lacking glinty reflections.
Recently, various research has been devoted to extending microfacet theory, especially using actual NDFs, to produce glinty details on surfaces. We call this line of research glinty appearance rendering. As noted earlier, the key problems, which we focus on in this survey, are the representation and rendering of the complex microgeometry. We provide the first thorough summary of state-of-the-art glinty appearance rendering research.
Most related works concern offline methods, so we focus on offline pipelines, and discuss real-time methods as an extension to them. We begin by introducing background knowledge, and overview solutions in Section 2. We then divide the complex glinty appearance problem into representation and rendering issues, and discuss commonalities and differences of previous research work in Sections 3 and 4. In Section 5, we present our unified platform, which implements typical offline approaches and use it to compare these approaches in terms of visual effects, rendering efficiency, and memory consumption. Finally, we introduce extended works in Section 6, covering wave optics, machine learning, and real-time rendering. We believe our survey will not only help readers new to this area to quickly understand the high-level concepts and solutions, but also benefit experienced researchers in choosing suitable methods for different application scenarios, and discovering future research directions in this area.

Background and overview
In this section, we first introduce the microfacet-based BRDF, and then, we define what glinty appearance is and analyze deficiencies of naive rendering methods. Next, we recap pre-filtering techniques and analyze their limitations for glinty appearance rendering. Finally, we overview the glinty appearance methods that are discussed in detail in the following sections.

Statistical appearance models
In physically-based rendering, the microfacet BRDF is widely used to model surfaces with many tiny facets that reflect rays as perfect mirrors (see Fig. 2). The distribution of microfacet normals is generally defined by a normal distribution function (NDF) [4]. With the NDF, we can determine how many microfacets reflect light from the incident direction ω i to the outgoing direction ω o , or how many microfacets normals point exactly along the half vector direction h between the camera and light directions.
The microfacet BRDF can be defined as where F is the Fresnel term, G is the shadowingmasking term, and D is the NDF term. We focus on the NDF term throughout this survey because it is the dominant factor in glinty appearance rendering. The microfacet BRDF [2] is successfully used in practice. However, traditional methods work by statistically modeling the aggregate behavior of a collection of microfacets using a smooth NDF. This smooth NDF tends to eliminate glint features and result in smooth appearance (see Fig. 1). Kurt [5] reviews BRDF measurement and representation methods, and overviews the microfacet-based model very well. These methods focus on the naive microfacet BRDF which cannot effectively render complex glinty appearance.

What is glinty appearance?
Many surfaces in reality have rich, high-frequency variable reflections under intense light sources. Random, tiny facets on surfaces, with size from a few to hundreds of microns, can produce a glinty appearance. Human eyes are very sensitive to shiny reflections, under slight changes in lighting or viewing direction. In this survey, we identify materials that have microfacets and generate glinty visual effects as having glinty appearance.

Difficulties in glinty appearance rendering
The most straightforward approach to glinty appearance rendering is the path tracing [6] technique. However, neither the naive path tracing approach nor similar bidirectional approaches [7,8] can efficiently process glinty surfaces. The main problem is that the perfect mirror-like reflection behavior of individual facets prevents us from sampling the correct facets and finding valid light paths. As Fig. 3 shows, even if we spend more time to process more samples, and introduce a little roughness to make the facets not so smooth, it is still a challenge to shade glinty effects properly. Among tens of thousands of discrete tiny facets, only tens of them might contribute to a given pixel's highlight. We need to find them all to obtain a noise-free image containing glinty features. Doing so is extremely expensive and quite intractable. Recently, researchers have focused on these problems and provided various solutions. Fig. 3 (a, b) Path tracing with random pixel sampling has little chance to find a valid path because only a tiny minority of microfacets will contribute to the reflectance. (b, c) Brute force sampling will consume much more time but may still fail as the small proportion of contributing microfacets provide a large amount of energy, and brightness will be greatly reduce if some are missed. (d) Recent advances in glinty appearance rendering generate better results in less time. (b-d) are reproduced with permission from Ref. [3], c Owner/Author 2014.

Solutions: Overview
To solve the glinty appearance rendering problem, the basic idea is to process a surface patch P seen through a screen pixel all at once. Some researchers prefilter patch P to approximate the contribution over a surface patch (see Fig. 4). However, these methods tend to reduce variations and produce smooth results. To render glinty effects, some researchers take the actual patch as input and calculate the discrete NDF over P accurately (see Fig. 5). We briefly consider the former and discuss the latter in detail.  Rendering glinty appearance needs accurate evaluation of the P-NDF of the corresponding patch P on the surface which covers one pixel's projection.

Pre-filtering methods
Pre-filtering methods have been widely used for decades and provide partial solutions to glinty appearance rendering. These methods evaluate the outgoing radiance over a patch P and try to average the outgoing radiance from P (see Fig. 4). Bruneton and Neyret [12] present a thorough overview of such pre-filtering methods. Here we briefly introduce some typical pre-filtering methods and explain reasons why they cannot preserve glinty effects.
One pre-filtering strategy is to store a large number of pre-computed or measured reflectances for different viewing and lighting directions, and organize them according to P covering a coarse mesh. The values can be stored in a 6D table called a bidirectional texture function, or BTF [13]. Suykens et al. [14] create a BTF for Monte-Carlo simulation on a geometric surface model. Ma et al. [15] enable interactive BTF rendering by compressing the BTF into a manageable representation.
Assuming that surface colors, NDF, visibility, and shadow-masking are uncorrelated, another strategy pre-filters these properties separately; NDF prefiltering issues have been studied in many works.
The simplest approach is to pre-filter the NDF as a single lobe in P. Neyret [16] models the NDF with an ellipsoid function. Olano and Baker [10] model the NDF with a Gaussian-lobe defined by its mean normal and a covariance matrix. Their method supports anisotropic highlights efficiently. Heitz et al. [17] introduce the SGGX function to represent the spatially varying properties of anisotropic microflakes. However, these methods cannot deal with surfaces that have structured microfacets or ones organized into a pattern.
Multiple lobe pre-filtering methods provide the ability to fit a more complex NDF. Fournier [18] represents the NDF with a sum of Blinn-Phong lobes [19]. His method supports up to 7 lobes and 28 parameters in his implementations. Tan et al. [20,21] extend the approach by using a mixture of isotropic Gaussian lobes to represent the average NDF. Han et al. [11] use a convolution method to model the macroscopic NDF with its decomposition into spherical harmonics and VMFs. Wu et al. [22] define characteristic point maps and present a principal component analysis method to find principal lobes based on their data structure. They further present an efficient filtering algorithm to reconstruct bi-scale surfaces that contain both macro-scale and micro-scale information [23].
The above pre-filtering methods are helpful for some kinds of glinty appearance but have limitations in rendering glinty materials, as the NDF for a glinty appearance can be too complex to be fitted by several lobes. Figure 6 shows the effect of replacing one NDF by a single lobe or multiple lobes: note the loss of sharp features.

Glinty appearance rendering methods
Recently, researchers have taken actual microfacets as input and accurately describe the microfacets' distribution (mainly focusing on the NDF) in a Fig. 6 Approximating the actual P-NDF by an isotropic lobe [9], an anisotropic lobe [10], or several lobes [11] loses the sharp features that cause glints. Reproduced with permission from Ref. [3], c Owner/Author 2014. surface patch P, which preserves glinty effects. However, the distribution can be extremely complex, so the core problem of glinty appearance rendering is how to efficiently and accurately evaluate the NDF over the patch (P-NDF). We consider glinty appearance rendering methods in terms of representation and rendering.
To render the glinty appearance, the first step is represent the actual microstructures. This is because glinty appearances are generally complex and varied, and representing them is a prerequisite for rendering them. Furthermore, the representation determines the method of evaluation of the NDF, which affects overall performance and storage. The main issues to be considered include memory costs, computation costs, accuracy, and supported material types. In Section 3, we discuss existing representation methods and compare them in terms of the above factors.
In practice, how to render the glinty appearance is another challenge. In Section 4, we discuss different evaluation technologies including acceleration data structures. We also discuss the importance sampling and multiple scattering which have great impacts on the final results.

Microstructure representation
In the real world, most shiny surfaces have glinty appearance, which is noticeable under strong light. Typical high-frequency materials like bumps, flakes, scratches, leather grain, dimples, etc., contain various microstructures. Representing such features is not an easy task. First, the glint features are small and complex, and representing all details would require an extremely large amount of storage. Another difficulty is that we need area-integration methods to accurately evaluate the P-NDF of each patch, and the representation must effectively support evaluation. Different representations have their own advantages and limitations. In this section, we classify recent representations as explicit or implicit, and further discuss each kind in turn.

Explicit representation
Explicit representations store the original microstructure of a surface in various forms.
The naive form is a normal map. This well-known representation records surface details in terms of normals instead of geometry [24]. A high-resolution normal map allows the recreation of microstructure details. Unfortunately, in practice, a normal map based representation does not work well for highfrequency rendering. The reason is that we can hardly evaluate the P-NDF from normals that are sampled from the normal map with methods such as importance sampling.
To support area-integration based P-NDF evaluation, existing methods represent microstructure as piece-wise elements E such as discrete triangles, Gaussian lobes, or spherical histograms. Each form has its own advantages and drawbacks.
Yan et al. [3] discretize the high-resolution position-normal distribution as a large number of triangles. Each triangle contains position and normal information. They then evaluate the P-NDF by accumulating the contributions of triangles located in P (see Fig. 7). This representation is very accurate. However, since integration must be performed for each triangle element, evaluation is quite expensive.
To simplify evaluation, Yan et al. [1] define 4D Gaussian elements to describe the distribution of normals in one tiny area. They describe a glinty surface with a 4D position-normal distribution function, which is approximated by a mixture of millions of 4D Gaussians. The method requires a large amount of memory (1.7-1.9 GB for 2K×2K maps) to organize the Gaussians, but calculating Gaussians is much faster than integration over triangles. Many works [25][26][27] use similar elements to represent the microstructure.
Gamboa et al. [28] use a spherical histogram to represent microstructure. They discretize the microsurface as a 2D texture histogram. Each texel stores an element, which is an accumulated spherical histogram of normals. This method also requires a large amount of memory (2.3-2.7 GB for 2K×2K Fig. 7 Triangle representation compared to binning as a reference. The triangle representation keeps features very well. Reproduced with permission from Ref. [3], c Owner/Author 2014. maps). In their implementation, they equally divide the longitudinal and azimuthal space according to texture size. At run-time, they introduce a summed area table (SAT) to compute the P-NDF for an arbitrary range (see Fig. 8). Atanasov et al. [29] further introduce inverse bin maps (IBMs) which use constant memory (36 MB) to store the inverses of histograms.
An issue of concern is that, in order to define the microstructure details producing glinty appearance, we usually need extremely high-resolution normal maps. For example, the normal map on the snail's shell in Fig. 3 has a resolution of 200K×200K. One suggestion is to generate the high-resolution microstructure on the fly to reduce the memory cost. The inverse Fourier transform method [30] can generate tileable noise-like bumps. Texture synthesis methods can also turn normal maps into tileable patches, stitch patches, and obtain high-resolution representations. Texture synthesis methods can be categorized into three different kinds: expansion [31,32], blending [33], and tiling [34,35]. The first two have been used for explicit representations.
To be clear, even if synthesis methods generate representations on the fly, we still consider them to be explicit representations, as the input normal map is in an explicit form, and furthermore, the generation rules are general and do not rely on specific kinds of materials.
The expansion method [31,32] extends a small texture into a new larger texture dynamically. Zhu et al. [26] apply the expansion idea to high-frequency rendering. They take a small-sized microstructure as a sample, repeatedly select blocks from the sample using some generation rules, and stitch the selected blocks together into a high-resolution microstructure. For continuity, they additionally generate some new Gaussian elements to fill seams between the selected blocks. This method requires only about 1% of the storage needed by the method of Yan et al. (see Fig. 9).
Blending methods [33] assume that any pixel in the resulting texture is a blend of several blocks sampled from the input texture. Wang et al. [25] apply blending-based texture synthesis in their work. They take small microstructure samples as input and generate Gaussian elements on the fly, using constant storage. However, the glinty effects can be blurred in some cases, especially for materials with scratches (see Fig. 9).
Existing synthesis-based methods are effective in solving memory problems. However, a common limitation is that they usually fail to keep global features. For example, when scratches are long, synthesis-based methods do not keep the global distribution of scratches.

Implicit representation
Some studies try to generate glinty appearance through parameterized implicitly procedures. By using an implicit representation, one can generate infinitely large, non-repeating microstructures on the fly with little additional storage.
Procedural noise methods [39][40][41] use a few parameters to control the appearance of a noise function over an infinitely large space. These methods can generate random patterns controlled by noise, such as bumps. Guo et al. [42] propose an implicit representation method for procedural material parameter estimation. They introduce a Bayesian inference approach using Hamiltonian Monte Carlo methods to sample the space of plausible material parameters, and fit procedural models to a range of materials such as wall plaster, leather, Fig. 8 Querying a multi-scale NDF histogram for an arbitrary filtering patch P. The method accumulates the multi-scale NDF histogram and organizes it with a summed area table (below) for efficient querying. Reproduced with permission from Ref. [28], c ACM 2018. wood, anisotropic brushed metals, and metallic paints. However, for glinty appearance rendering, we need not only the microstructure, but also a corresponding acceleration method to prune noncontributing regions. Unfortunately, none of these methods currently support queries in an arbitrary range.
So far, only two kinds of glinty appearances can be represented implicitly: glittery materials and scratched materials. Mirror-like flakes cause glittery effects. Unlike a general microfacet model, a glittery surface contains a collection of tiny and discrete flakes which are supposed to be defined using a non-smooth, spatially varying BRDF. Some methods [43,44] regard the discrete flakes as random normals with positions. Günther et al. [45] store the normals and positions of flakes to avoid flickering between frames but requires much memory to do so. Jakob et al. [36] represent the glittery material by stochastically generating flakes on the surface. They assume surfaces to be a collection of a specific set of randomly oriented facets and use a random index to store the count of flakes in a specific area and solid angle. This method supports range queries on the microstructure. Zirr and Kaplanyan [37] also model the glittery appearance as a set of implicitly represented flake elements. They derive a stochastic bi-scale model based on flake elements and implement this model in real time.
For scratches, we need to consider two levels of microstructure distribution. The first level describes global scratch trajectories visible to the naked eye, defined as curves. The second level characterizes the microstructure profile of a single scratch considered as an element. Raymond et al. [38] model the microstructure profile for a single scratch as a multi-scale spatially varying BRDF. They use noise functions to generate the scratch distributions, with statistics determining the orientation and position of the scratch. For the range query, they adopt a simple idea, calculating the area occupied by a scratch element in a pixel. They further use the area and the BRDF of a single element to evaluate the contribution of a scratch element. However, their method does not take the cross section of two elements into consideration.
The above implicit representation methods can generate infinitely large parameterized microstructures of non-repeating patterns. However, a common challenge for such methods is how to efficiently integrate and evaluate the P-NDF. Another limitation is that they can only represent a few kinds of glinty appearances, using specifically designed generation methods.
We summarize explicit and implicit representation methods in Table 1. Explicit methods can represent all kinds of glinty appearances but require large storage and are generally slower than implicit methods. Implicit methods can represent nonrepeating large microstructures, require less computation, and incur little storage overhead, but so far can only represent glittery and scratched materials. Some methods [25,26] apply texture synthesis methods, which significantly reduces the storage. These methods can generate non-repeating large microstructures, but rendering is still slow.

Rendering solutions
In this section, we consider how to integrate glinty appearance representations into the classic Monte Carlo path tracing pipeline. We discuss the evaluation step in Section 4.1 and importance sampling in Section 4.2. Multiple scattering is also an important part of path tracing, discussed in Section 4.3.

Evaluation
The P-NDF integral can be defined in a general form as where s is a given direction to be queried. Within a pixel coverage G p , we visit each microfacet at position u and use G r to decide whether its normal n(u) is close enough to the direction s of the query. r is the intrinsic surface roughness, used to define the value of closeness. In this way, we can evaluate the density for any direction s on P.
In order to compute the integral and evaluate the P-NDF analytically, researchers use discrete piecewise elements to describe the microstructure. There may be millions of elements in total in query patch G p (u) but only a few of them make a non-negligible contribution to the given query vector s.
The P-NDF can be rewritten in discrete form as where m is the number of elements. E i is the contribution of the ith element at position u and direction s. The weighted sum of these elements gives the desired value. Evaluation in all methods with an explicit representation can be described by this equation.
Evaluation in methods with an implicitly represented NDF [36,38] cannot be simplified by Eq. (3) as the NDF does not store the correspondence between position and normal. Raymond et al. [38] directly calculate the ratio of the scratched area to the patch area P, and evaluate the NDF according to the ratio, scratch orientation, and measured BRDF.
Flake high-frequency material methods [36] generate a sequence of random numbers that represent the distribution of flakes. They then count particles that contribute to illumination without actually Table 1 Characteristics of representations. Element type: representation form in discrete elements. Storage: run-time memory for the material taking a 2K×2K normal map as input (or 512×512 for Refs. [25] and [26]). Appearance: representable appearance types. Multi-scale: whether the method supports multi-scale zoomed rendering. Tileable: whether the microstructure repeated when it is extremely large. Efficiency: element calculation speed, longer being more efficient. One dot: method is unlikely to finish in a reasonable time. Two dots: method is significantly slower than traditional BRDF methods. Three dots: method is slightly slower than traditional methods. Four dots: method is almost as fast as the traditional methods. generating them. In their stochastic approach, randomflake approximation replaces the P-NDF evaluation. In general, an acceleration hierarchies is used. Figures 10 and 11 illustrate two typical acceleration hierarchies for the query.
Yan et al. [3] and Wang et al. [46] create a 4D bounding box for each element and build a minmax structure to organize these bounding boxes in a top-down manner (see Fig. 11). Given a rectangle bounding the patch Gaussian G p (u) and a cone bounding direction s as input, the method queries the hierarchy tree by top-down traversal to find contributing Gaussians.
The evaluation process in Ref. [36] is performed on a 4D search tree as shown in Fig. 10. Each tree node also contains a 4D bounding box defined as the Cartesian product of a bounding box in texture space and a spherical triangle in direction space. Each branch node is further split both in texture and in direction space. In texture space, the bounding box is cut into four equal-sized sub-boxes, and in direction space, the spherical triangle is cut into four subtriangles by inserting vertices at the midpoints of the edges. The search proceeds alternately in spatial and in directional domains. Figure 12 shows a multiple importance sampling experiment in which a scratched surface is illuminated with a textured area light. The reflection from flat parts of the surface can easily be rendered by BRDF sampling (a), but rendering scratches relies on sampling and evaluating the P-NDF in the light direction (b). A combined image (c) has the benefits of both.

Importance sampling
We can sample the underlying normal map directly, or the discrete elements of P-NDF. Yan et al. [3] take the normal of a random surface point and perturb the normal by the intrinsic roughness value. This only works for normal map based representations.  Yan et al. [1] also sample from the discrete Gaussian elements. They select a Gaussian element proportional to its contribution to the patch and then pick a normal from that element. Their sampling method works well without the input normal maps. Raymond et al. [38] sample directions around ω i which are randomly generated within the reflection cone following a 1D probability distribution function based on the mirror scratch BRDF. Instead of sampling the actual P-NDF, Jakob et al. [36] define a smooth density function to describe the distribution of flakes and use for sampling. The advantage is that they do not generate all elements reducing time and space costs. However, the function is more capable of describing surfaces with discrete flakes.

Multiple scattering
By multiple scattering, in this survey, we mean self-scattering between microstructures. Modeling multiple scattering in glinty appearance rendering is considered to be an important open problem, since a non-negligible fraction of the energy leaving the surface occurs due to paths with multiple reflections. Figure 13 illustrates the principle of multiple scattering. Figure 14 shows how, for materials with scratches for example, the results will be less bright if multiple scattering is neglected.
While considering multiple scattering produces better results (see Figs. 13 and 14), it also requires much more computation and storage. Only a few investigations of glinty appearance rendering integrate multiple scattering into their solutions.
Raymond et al. [38] randomly distribute scratches on a surface and users can modify the profiles, orientations, and density of the scratches. They simulate multiple-scattering by pre-computing distributions of scratch profiles. Their method gives realistic results but only works for scratched surfaces. Chermain et al. [27] derive an energy-compensation BRDF to compensate for the energy lost by single  Scratched metal rendered with and without multiple scattering. Left: metal modeled with real geometry yields stronger contrasts and reflections due to multiple scattering. Right: the same metal modeled with NDF appears less bright, since scratches reflect little or none of the incoming radiance when higher-order bounces are neglected. Reproduced with permission from Ref. [38], c ACM 2016.
scattering. Their method leverages a local energy preserving BRDF by faking normal perturbations. Turquin [47] deduces a compensated P-BRDF for glinty appearance, and produces noticeably improved results.
Such methods are results-based, and deriving a physically-based analytical multiple scattering model is an open problem in glinty appearance rendering.

Experiments and analysis
In this section, we first define quality criteria for glinty appearance rendering methods. Then, we introduce our unified experimental platform that integrates dozens of recent glinty appearance rendering methods. We show some methods' results produced by the platform and compare them according to our criteria.

Quality criteria
Verifying complex surface rendering quality can be difficult due to the lack of mathematical criteria and the subject nature of human perception.
A direct approach to verifying the "correctness" of rendering results is to render texture-based glinty appearances with an extremely high number of samples per pixel (spp) and to consider the image result at convergence as "ground truth" or reference. Comparing the rendered results to the reference, we may estimate the degree of degradation. We render the reference images by brute force path tracing. How can we determine whether an image has converged or not? We can assume that the image converges when there is no significant change (when the RMSE of two images is smaller than 0.03) as we increase the number of samples. The time needed to compute such a converged image is about 14-20 h.
Another effective term to model glinty appearance is NDF. Thus, we adopt the correctness of the NDF into our quality criteria. Designing a metric suitable for comparing a model's NDF to the original normal distribution is necessary. In practice, we find comparing a visual comparison of the P-NDF to the reference is adequate to represent correctness.
In addition to rendering quality, rendering efficiency, memory consumption, and versatility are also useful criteria to assess the strengths and weaknesses of different methods. Energy conservation can be verified by a white furnace test, but since most existing techniques are not energy-conserving, we do not discuss this much further.
A further issue is that researchers and artists should not simply judge a method as good or bad based on the quality criteria alone. Instead, they should remember that different methods are applicable to different rendering requirements. Implicit rendering methods [36,38] are suitable when rendering scenes with glittery or scratched appearances, due to their good performance, but they have limited expressiveness. We need explicit rendering methods for other more complex scenes. Even so, there are big differences between different explicit rendering methods. For example, the method of Ref. [1] can accurately render scenes with complex glinty appearances. If the light sources are not sharp, it is more efficient to use methods in Refs. [28,48]. When there are many glinty appearances in a scene, the LOD method of Ref. [26] is efficient and storage-saving.

Unified experimental platform
To evaluate methods according to the quality criteria for glinty appearance rendering technologies defined above, we built a unified experimental platform based on the Mitsuba framework [49]. It: • implements more than a dozen advanced offline glinty appearance rendering methods, • provides RMSE and HDR visual difference maps to compare different rendering results, • provides NDF visualization and quantitative NDF statistics, • provides example scenes (leather shoes, a wooden ball, and a scratched kettle), including geometric models, scene files, glint features, and a converged reference for comparison, and • collects statistics of memory and rendering time.
Using the platform, researchers can easily compare their methods with previous work, and artists can quickly select methods that meet their needs. Our platform only supports comparison of offline methods.

Results
In Fig. 15, we compare typical explicit glinty appearance rendering methods on our unified experimental platform in terms of rendering effects, accuracy of P-NDF, and rendering time. All timings in this section were measured on a 2.20 GHz Intel Xeon with 22 cores and 128 GB of memory. We consider four glinty appearances: leather, a scratched coating, brushed metal, and bumpy plastic, on the simple bent quad geometry, illuminated by an environment map and four tiny area lights (which can be considered to be point lights). All timings are for pictures with 1024 pixels×1024 pixels.
First, let us observe the rendering results. Compared to the reference, the traditional microfacet model and the pre-filtering method [11] fail to render the glinty effect, while results from methods in Refs. [1,3] are very close to the reference. Atanasov et al. [29] capture glint effects, but compared to the reference, their results differ somewhat in close up. For the texture synthesis methods, except for slight discontinuities on the brushed metal, the results of Zhu et al. [26] are visually identical to the reference; the results of Wang et al. [25] look blurry for the scratched coating, brushed metal, and the bump plastic scenes.
In terms of rendering speed, the method of Ref. [3] is significantly slower than other methods. Yan et al. [1] take only about 1.4× as long as standard microfacet BRDF rendering. Refs. [25,26] represent the microstructure as Gaussian elements like Ref. [1]. Zhu et al. [26] use a clustering method, so their method is slightly faster than that of Ref. [1]. Wang et al. [25] generate the Gaussian elements on the fly, so their method is slower than that of Ref. [1]. Ref. [29] takes a longer time than Ref. [25] because their methods have higher computational complexity. Fig. 15 Four glinty appearances rendered on the Bent Quad scene: leather, a scratched coating, brushed metal, and bumpy plastic, using seven different methods on our unified experimental platform; the reference is also shown. Tables to the right of each scene compare rendering time and correctness of the P-NDF for different methods. The sizes of microstructures are as given in Table 1.
We also compare the correctness of the P-NDF. In practice, when the RMSE is smaller than 0.01, the P-NDF can be considered correct. We can observe that all glinty appearance rendering methods compared correctly evaluate the P-NDF. Correctness of the P-NDF is also an essential requirement of a glinty appearance rendering method. Figure 16 illustrates rendering results of the method developed by Jakob et al. [36]. These methods are efficient and require no additional storage space; rendering speed is correlated with the number of flakes, and they only need to store a few parameters.

Extensions
So far, we have presented practical methods for rendering surfaces with glint features accurately and efficiently, and showed they are able to render credible results. In theory, using these methods, the rendering results should exactly match the actual appearance, but we still require better speed and realism.
Several techniques and applications leverage or extend the idea of glinty appearance rendering for broader applications and more realistic rendering. This section provides a brief overview of these areas.

Wave optics
When we look at actual photographs (see Fig. 17), we can observe that there may be colored glint features even when a white light source illuminates the object. This is an interesting phenomenon, and is not possible in traditional geometric optics, which only produces white highlights from a white light source. This phenomenon is explained by wave optics.
In wave optics, light is described by complexvalued fields. Scalar diffraction models, such as those proposed by Harvey-Shack [50,51] or Kirchhoff [52,53], can be used to estimate the reflected field from a rough surface. For scratches, Werner et al. [54] derive a wave-optical and analytical shading model based on Harvey-Shack theory [50], where the surface is represented as a collection of randomly oriented scratches over a smooth BRDF. This work is further extended to real time by Velinov et al. [55]. For mirror flakes, Guo et al. [56] extend the stochastic model [36] to take wave-optical effects due to thinfilm interference into account, reproducing iridescent reflection. Yan et al. [57] present a solution to derive a wave effect-aware BRDF model on surfaces described as a discretized height field. The BRDF model is estimated by simulating diffraction effects of coherent light over the corresponding area on the height field, allowing their method to support arbitrary glint features. While capturing wave effects accurately, wave optics glinty appearance rendering methods are about one order of magnitude slower than those using geometric optics; further acceleration should be explored.

Machine learning methods
There are some glinty appearance-related studies in the field of machine learning. They are more concerned with the representation of details than improving the actual rendering process. We introduce them in this section as a complement to previous representation approaches.
Again, we classify machine learning-based methods in two categories: one uses an inverse model to provide an explicit representation, and the other uses a procedural model for an implicit one.
The inverse model basically estimates spatial varying material parameters (diffuse albedo, roughness, and normal for a microfacet model) from measured data like images and derives per-pixel BRDFs inversely, as an SVBRDF. These parameters are stored explicitly in a texture. Chandraker [58] utilizes motion cues to jointly recover shapes and BRDFs of objects from images. Hui and Sankaranarayanan [59] recover SVBRDFs and shape from multiple images taken with a fixed view-point and varying illumination. Riviere et al. [60], Hui et al. [61], and Li et al. [62] propose SVBRDF recovery algorithms under a simpler setup consisting of a camera and a flashlight, as commonly found in mobile devices. Li et al. [63] improve the approach to handle larger inputs.
Gao et al. [64] present a unified framework for estimating SVBRDFs.
Generally, the inverse model heavily relies on a specific reflection model, which, in most cases, is the microfacet model, and which is not suitable for glinty appearance rendering as explained in Section 2.1. In order to obtain an SVBRDF at microscale resolution, Nam et al. [65] propose a microscopic material acquisition system. They use machine learning to compress the vast amounts of data. While different reflection models could be used, this approach cannot handle advanced complex models like the wave optical one introduced in Section 6.1.
The procedural model learns glint features by fitting an implicit pattern distribution instead of explicitly estimating parameters everywhere on the surface. Guo et al. [42] focus on procedural material parameter estimation by employing a Bayesian framework and using an image classification neural network (usually VGG) as a descriptor. Their procedural material models generally consist of a microfacet SVBRDF and an explicitly constructed procedural height field (usually a noise texture). Kuznetsov et al. [66] propose a procedural model and use a conditional generative adversarial network to learn to generate P-NDF for every location with a fixed patch P. Using a relatively lightweight GAN network, their procedural model can reproduce a P-NDF close to the ground truth with lower storage due to its implicit nature, while supporting traditional geometric optics and wave optics. Moreover, this serves as a precomputation for evaluating the P-NDF integral (Eq. (2)) and speeds up evaluation when rendering with wave optics compared to the method proposed by Yan et al. [57]. However, their work is limited to a fixed patch size and may introduce artifacts in the near-field and grazing angles where the size of P varies dramatically.

Real-time rendering
We have mainly discussed offline glinty appearance rendering methods above.
It is not easy to apply offline glinty appearance models to real-time rendering. The first reason is that GPU and CPU architectures differ, so offline acceleration structures cannot be applied to real-time rendering. The second reason is the small video memory of the GPU: the storage required for explicitly represented materials is large.
Current real-time research work mainly deals with implicitly represented materials.
Zirr and Kaplanyan [37] propose a stochastic bi-scale microfacet model for real-time rendering of multi-scale glint features including discrete flakes and brushed marks. Wang et al. [25] propose a pre-filtering method for the stochastic discrete microfacet model to simulate glints under both environment maps and point light sources in real time. Velinov et al. [55] treat scratches under wave optics. Chermain et al. [67] propose a method of rendering flakes in real time. They use mip-map structures to speed up rendering. They further propose an anti-aliasing method [68] in real time. However, no existing real-time solutions can consistently process glinty appearances explicitly defined by highresolution normal maps, limiting the variety of glinty appearances in real-time rendering. The limited video memory available in real-time rendering makes it difficult to store complete mapping-based glinty appearance information. Also, texture synthesis methods are challenging to implement and accelerate reasonably well in real-time rendering.

Conclusions and future work
Glinty appearance methods have made promising steps toward modeling and rendering visual appearances with real-world complexity. In this survey, we have discussed recent advances in glinty appearance rendering by broadly categorizing approaches based on representation and practical rendering.
Glinty appearances can be represented in explicit or implicit form, but both explicit and implicit representation have in common that they take discrete forms. Explicit representation methods have been intensively studied because they can represent all kinds of glinty appearances like bumps, flakes, scratches, leather patterns, and dents. However, a common problem for explicit representations is that they require large additional storage to store all microstructure information. Texture synthesisbased methods reduce the storage to a certain extent by generating the large microstructure from small samples. Implicit representations can represent microstructures of unlimited size while requiring little additional storage. As a downside, such methods are not general: existing implicit representations can only represent glittery and scratched materials.
For practical rendering, we have distinguished three sub-problems, evaluation, importance sampling, and multiple scattering. Acceleration hierarchies are used to quickly prune non-contributing parts to efficiently evaluate the glinty appearance. During evaluation, methods with explicit representations are generally slower than those with implicit representations. Because the P-NDF is known, all methods can easily perfectly importance sample the P-NDF. Sometimes, multiple scattering has non-negligible effects on glinty appearance. Existing methods precompute the multiple-scattered BRDF or adopt energy compensation methods. However, these methods are experience-based and usually oversimplify multiple scattering evaluation.
We have also discussed several open problems and potential future research directions in this area. The conflict between representation ability, rendering efficiency, and memory is still the core problem in glinty appearance rendering. Explicitly represented methods require more storage and longer rendering time. Implicitly represented methods are not general and can only represent a few appearance types. One solution is to deduce a general implicit model which could represent all types of appearances while incurring no storage, and efficiently evaluating the P-NDF during rendering. Although it is far from easy to settle the range query problem of implicit representation methods, we believe this to be an area that provides potential advantages for further practical applications. Another practical solution is to find a better discrete element form providing easy calculation in explicitly represented methods.
Apart from solving existing problems, new problems are to be found. For example, the glinty appearance rendering of volumes is a more difficult integral problem than glinty surface appearance rendering. Another problem is denoising algorithms. How can we distinguish glint features from noise? All these problems and challenges provide new research opportunities and may stimulate deeper observations and explorations.