Introduction

Surface form measurement is important in engineering applications [1]. Fringe projection technologies are widely used for diffused surface form measurement [2,3,4]. For object surfaces which exhibit special properties, such as being specular and transparent, fringe projection methods cannot be implemented. For measuring transparent objects, Mériaudeau provides a brief on the latest advancements [5]. Measurement of 3D surface form of specular objects remains a challenge because of the reflecting properties of their surfaces. From all the existing technologies [6, 7], interferometry [8,9,10,11,12] and deflectometry [13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52] are the two competitive methods for specular object measurement because of their none-contact measurement nature and high measurement resolution. Interferometry is a superior technique for measuring simple surfaces such as spheres, planar surfaces, and weakly aspheric surfaces, but has difficulty in measuring complicated aspheric mirrors or free-form specular objects because a reference surface is required during measurement [8,9,10,11,12]. Phase measuring deflectometry (PMD) is a famous deflectometry technique that is based on phase calculation [23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49, 51, 52]. With the advancement of displaying, imaging and computing technologies, PMD becomes an effective three-dimensional (3D) form measurement method for specular surfaces. Compared with interferometry, PMD has many advantages such as having a high dynamic range, full field of view and non-null measurement. As an alternative technology for measuring surface form of specular objects, PMD has been studied by many researchers. Huang et al. [53] and Zhang et al. [54] reviewed history of PMD and key techniques applied in PMD. They also made a comparison with other measurement techniques, and discussed major challenges and application of PMD. In this short review we present the development of PMD with the advancement of related technologies and discuss challenges and further developments.

Advancements of PMD

PMD measures specular surfaces by utilising the reflective property of surfaces, which is very similar to Moire deflectometry [15,16,17,18]. PMD displays phase-shifting sinusoidal fringe patterns on displaying screens and captures the deformed reflected patterns through imaging sensors. Phase information is extracted from the deformed patterns to calculate the 3D shapes of surfaces under test (SUT). Due to the application of sinusoidal fringe patterns and phase techniques, PMD has a high dynamic range, and high measurement accuracy and solution compared to other deflectometry technologies [13,14,15,16,17,18,19,20,21,22]. According to system setup, PMD can be classified into the following three categories.

Single screen and sensor based PMD

The single screen and sensor based PMD is illustrated in Fig. 1(a). A reference surface shown in red is assumed to be parallel to the displaying screen with a distance d. The camera ray is perpendicular to the screen and the reference. With known phase difference ∆φ between Q0 and Q1, the gradient of the measured surface can be obtained according to Eq. (1).

$$ \mathit{\tan}2\alpha =\mathrm{p}\bullet \Delta \varphi /\left(2\pi \mathrm{d}\right) $$
(1)

where p represents physical distance per fringe period. Several PMD systems have been developed based on this method. Häusler et al. introduced a microdeflectometry system to measure microscope features [23]. Liu et al. established a system with an illuminated photographic film for the measurement of small specular objects [24]. However, the measurement principle of this method is based on paraxial approximation, which means it can only measure objects with small size.

Fig. 1
figure 1

Illustration of PMD based on single screen and sensor. a Model based on paraxial approximation; b model based on flat reference; c model based on an estimated shape and positon of SUT

Full size image

Compared with the strict geometry requirements of Fig. 1(a), a model with a flexible component arrangement is described in Fig. 1(b). Assuming the measured specular surface is very close to a reference with known positon, gradient of a surface point S can be obtained based on a triangular relationship composed of P, S and Q. Huang et al. developed a fast measurement PMD system based on this model [25]. Li et al. analysed the influence of position error of the reference plane and explored increasing the plane’s positional accuracy by applying two visible laser sources and a confocal white-light distance sensor [26]. However, PMD based on this model can only measure near-plane surfaces.

An advanced model shown in Fig. 1(c) is quite similar to that in Fig. 1(b) but can measure objects with high curvature change. Rough shape and positon of the measured surface is required to be estimated to replace the reference plane in Fig. 1(b). Objects with different surface properties were detected by Bothe et al. based on this model, such as metal surfaces, transparent plastic surfaces, glass, and water surfaces [27]. A software-configurable optical test system (SCOTS) is developed by Su et al. to measure telescope mirrors based on the advanced model [28]. The accuracy of the estimated rough surface directly affects the measurement accuracy of this model. Generally, a positon sensor such as a coordinate measuring machine (CMM) is used to estimate the rough surface. However, it is a challenge to combine the coordinate systems of the position sensor and PMD with high accuracy. Xu et al. studied a calibration method to obtain the rough surface with a manufacturing positioning system and establish the relationship between the manufacturing system and PMD measuring system directly [29, 30]. However, this calibration method is specially designed for in-situ measurement with PMD and still needs to be improved for off-line measurement.

Multi-screen based PMD

A simple model shown in Fig. 2(a) is used to describe the principles of PMD based on multiple screens [31,32,33,34,35]. Camera ray PS is reflected by surface point S and intersects two parallel screens at Q1, Q2. The normal of S can be calculated based on incident ray PS and reflected ray Q1Q2 with known camera imaging parameters and location of Q1 and Q2. Since Screen 1 blocks light from reaching Screen 2, a common solution is to move a screen vertically from one position to another during the measurement process. However, screen movement increases complexity of system structure and measurement time of the system. With the development of display techniques, Li et al. improved multi-screen based PMD by using a transparent screen [36]. The camera ray can pass through the transparent screen to a general displaying screen. The two screens can be fixed in a relatively free position in Li’s system.

Fig. 2
figure 2

Illustration of PMD based on multiple screens. a Model based on screen movement; b model of DPMD

Full size image

Direct phase measuring deflectometry (DPMD) [37,38,39] is another type of multi-screen based PMD system, as shown in Fig. 2(b). Depth data of the measured surface can be obtained according to Eq. (2) with known phase values φ1, φ2, \( {\varphi}_1^{\prime } \) and \( {\varphi}_2^{\prime } \).

$$ h=\frac{\mathrm{d}\left[\left({\varphi}_1-{\varphi}_2\right)-\left({\varphi}_1^{\prime }-{\varphi}_2^{\prime}\right)\right]-\Delta d\left({\varphi}_1^{\prime }-{\varphi}_1\right)}{\left({\varphi}_2-{\varphi}_1\right)+\left({\varphi}_2^{\prime }-{\varphi}_1^{\prime}\right)} $$
(2)

where d is the distance between Screen 1 and the reference, and ∆d is the distance between two parallel screens. The camera ray reflected by the reference intersects the screens at phase points φ1 and φ2. \( {\varphi}_1^{\prime } \) and \( {\varphi}_2^{\prime } \) are the intersections of the screens and the reflected camera ray through the measured surface. In an actual DPMD system, a beam splitter is applied to generate a virtual screen that is parallel to a real screen. However, it is different to guarantee an absolute parallel between the virtual screen and the real screen. Compared with PMD based on gradient integration, DPMD can measure discontinuous specular surfaces. However, its accuracy is not as good as PMD based on gradient integration for continuous specular surface measurement, since the measured surface cannot be reconstructed based on gradient information in DPMD.

Multi-sensor based PMD

Stereo deflectometry is a PMD technique based on multiple imaging sensors, which is proposed by Knauer et al. [40] in 2004. The principle of stereo deflectometry is shown in Fig. 3. Space points are searched along the rays of a master camera. Another camera is used to assist the master camera in determining points on the measured surface by matching gradients. For a chosen space point S1, its image point P1 on CCD (Charge Coupled Device) of Camera 1 can be obtained based on the result of camera calibration. A point Q1 can be located on screen based on the corresponding phase values of P1. An equivalent normal can be calculated based on points Q1, S1 and P1 according to the law of reflection. P2 is the image point of S1 on CCD of Camera 2. A screen point Q2 can be acquired based on phase values at P2. Another equivalent normal can be obtained based on P2, S1 and Q2. Theoretically, the normal vectors calculated from these two cameras should be the same for a surface point. Therefore, the gradient and 3D data of the measured surface can be determined by matching normal vectors calculated from the two cameras. Stereo deflectometry can achieve a nano-scale relative depth accuracy [41, 42] and relatively low absolute depth accuracy. However, the search process of space points takes a large amount of time. The algorithms of stereo deflectometry are required to be improved for faster measurement.

Fig. 3
figure 3

Illustration of stereo deflectometry

Full size image

In order to clearly demonstrate advantages and disadvantages of the three categories of PMD, a comparison is listed in Table 1.

Table 1 Comparison of three categories of PMD
Full size table

Challenges and further development

Figure 4 illustrates the techniques applied in the measurement process of stereo deflectometry. Phase shifting and phase unwrapping techniques are applied to obtain phase information [43, 44]. Two groups of perpendicular fringe patterns are required to establish a phase rectangular coordinate system [29,30,31,32,33,34,35,36, 45]. In order to suppress phase noise, each fringe pattern is captured several times to calculate the average [41, 42, 47,48,49]. Phase shifting algorithms with high steps are usually applied in PMD, due to the same reason [41, 42, 47,48,49]. In the process of measurement, displaying and capturing a large number of patterns can improve measurement accuracy but reduce speed [46]. How to increase the accuracy and speed of phase extraction is a challenge. Calibration of PMD includes screen calibration, camera calibration, and system calibration [40]. Screen calibration is to acquire the relationship between phase and physical distance in the displaying screen. Early PMD system [40] applied a projector to generate fringe patterns on a rough flat plate. Though the light source of the projector can be extended to infrared light or ultraviolet light to test surfaces with different reflectivity [50], it is difficult to accurately calibrate the corresponding physical position of the phase on the plate. As the displaying technologies advances currently stereo deflectometry systems [41, 47, 51] mainly use TFT (Thin Film Transistor) screens to display fringe patterns, since the manufacturing of TFT screens ensures good flatness and accurate pixel spacing of the screen [42, 48, 49, 52]. Camera calibration is to establish the relationship between a space point and the image on camera pixel, which is an essential task for camera based optical 3D measurement systems [55, 56]. A popular camera calibration method is based on a chessboard, proposed by Zhang [55]. Zhang’s method is easy to operate and improves the flexibility of camera calibration. In recent years, camera calibration methods based on phase targets have been investigated by researchers [57,58,59]. The calibration target of this type of method is an absolute phase map extracted from sinusoidal fringe patterns displayed on an LCD screen. Compared with Zhang’s method, a phase target-based method can achieve better calibration accuracy and resolution [57,58,59]. In addition, the displaying screen in PMD systems can be utilised as a phase target, which can avoid the use of additional calibration tools and can be combined with the subsequent system calibration. Therefore, camera calibration methods based on phase targets are widely applied in the calibration of PMD [41, 42, 47,48,49]. PMD is based on the principle of geometrical optics, therefore the relative positions of each component in the system must be obtained accurately before measurement. The process of obtaining the position of components is called system calibration. Traditional calibration methods complete the process with the assistance of calibration tools such as flat mirrors with markers [60, 61], extra cameras and/or screens [62] and calibration targets [63]. In order to avoid errors caused by additional calibration tools, self-calibration methods are studied to utilize the displaying screen to calibrate PMD systems [41, 42, 64]. Ren et al. improved system calibration accuracy by adding constraints in the process of parameter iterative optimization [41]. Xu et al. enhanced the compensation accuracy of the error between the real imaging process and the ideal imaging model by studying an iterative distortion compensation method [42]. Due to the imaging model being based on a pinhole camera model, which is far from a true camera model, accurate calibration of the imaging system remains a challenge to the PMD system. Once phase and system geometry is obtained, initial gradient and/or 3D data of the measured surface can be calculated through a gradient calculation process. The calculation process is affected by the phase error and calibration error in the system. Xu et al. investigated methods to decrease the influence on stereo deflectometry of phase error and calibration error by optimising component parameters [48] and system assignment [49]. Since iterative computation is required during the gradient calculation process for most PMD systems [27,28,29,30, 40,41,42, 45,46,47,48,49], it is a challenge to reduce the time spent on the iterative process. Reconstruction is the final step for the measurement of gradient interpolation based PMD [27,28,29,30, 40,41,42, 45,46,47,48,49]. 3D shape of the measured surface is reconstructed based on the obtained gradient data. Methods have been studied to improve reconstruction accuracy [47, 65,66,67]. Ren et al. improved the high-order least-squares integration method to increase the reconstruction accuracy of stereo deflectometry [47]. Huang et al. studied a least squares integration method based on splines for wavefront reconstruction [65]. However, a reconstruction method with both high accuracy and fast speed is still required to be investigated [67]. In addition to the problems in the specific technologies mentioned above, challenges exist in the future development of PMD.

Fig. 4
figure 4

Measurement process of stereo deflectometry

Full size image

Measurement of low reflective surfaces

Rough surfaces and specular surfaces can be measured by fringe projection technique and PMD separately. However, there is no clear boundary between rough surfaces and specular surfaces. A number of surfaces are between rough and smooth, or partly rough and partly smooth. One solution for measuring such surfaces is to combine fringe projection techniques and PMD [61]. Another solution is to utilise the principle that the reflection rate of surface changes with light sources with different wavelengths. PMD with infrared light has been tested to measure surfaces with low reflection rate [50].

Fast measurement

Most current PMD systems apply TFT screens as a fringe-displaying screen, since calibration for the relationship between phase and physical distance on the screen can be avoided. Multiple fringe patterns are required to be displayed and captured during the measurement process in order to obtain absolute phase maps. However, display and switching between patterns takes plenty of time, which results in low capture speed. Several methods such as a cross pattern based on Fourier transform [25] and colour fringe techniques [68] have been studied for fast measurement, but have their problems. Fourier transform techniques cannot measure surfaces with large gradient change and discontinuous surfaces. Crosstalk between colour channels affects measurement accuracy seriously. One possible solution would be to use a projector to replace the TFT screen since it is easy to achieve high speed synchronization of the projector and camera. A new generation of display screens with storage and synchronization functions is needed for this development.

Measurement of high curvature changing surfaces

While PMD can achieve high measurement accuracy for surfaces with small curvature change [41, 42], it is difficult to maintain the same accuracy for the measurement of high curvature changing surfaces. This is because the increase of curvature will blur the reflected fringes and magnify phase noise. In addition, high curvature convex surfaces will make the reflected camera ray through the measured surface exceed the physical range of the display screen, which causes measurement failure. Graves et al. located a convex surface on a rotating platform, and stitched segments of the surface [69]. This method can measure convex surfaces successfully but is time-consuming. Carvalho et al. [70] measured roundness of a cylindrical surface by using a conical mirror to transfer the cylindrical surface into a planar image. It is expected that high curvature surfaces will be one of the research focuses of PMD.

Portable and online measurement

Miniaturization and portability are one of the development trends of PMD systems. As a result, PMD systems can be embedded into a manufacturing system and realise online measurement. Butel et al. [71] and Willomitzer et al. [72] explored to establish a PMD system based on mobile devices. It is expected that with the development of mobile technology and 5G [73], portable PMD systems with improved performance will emerge.

Conclusions

Measurement principles and development of PMD are presented in this paper. Single screen and sensor based PMD has a simple structure, however requires the measured surface to be small in size and low in curvature, or have known shape and position. Multi-screen based PMD has the burden of using an additional screen, which increases the volume of the system, and is not suitable for embedded measurement. Stereo deflectometry applies a search algorithm to determine the measured surface. This algorithm is time-consuming and not fit for fast measurement. Further studies are required to miniaturise the system, increase the measurement speed and range, so as to realise online and embedded measurement.