Rapid and accurate reverse engineering of geometry based on a multi-sensor system

The reduction of the lead time in measurement and reverse engineering, and the increased requirements in terms of accuracy and flexibility, have resulted in a great deal of research effort aimed at developing and implementing multi-sensor systems. This paper describes an effective competitive approach for using a tactile probe to compensate the data from a laser line scanner to perform accurate reverse engineering of geometric features. With the data acquired using laser scanning, intelligent feature recognition and segmentation algorithms can be exploited to extract the global surface information of the object. The tactile probe is used to re-measure the geometric features with a small number of sampling points and the obtained information can be subsequently used to compensate the point data patches which are measured by laser scanning system. Then, the compensated point data can be exploited for accurate reverse engineering of a CAD model. The limitations of each measurement system are compensated by the other. Experimental results on three parts validate the rapidity and accuracy of this multi-sensor data fusion approach.


Introduction
Even though tactile and optical sensing technologies are widely used in data acquisition in measurement or reverse engineering (RE), it has been shown that each technique has its own characteristics and limitations, which lend them to particular applications [1].Existing tactile coordinate measuring machine (CMM) methods are widely used for industrial dimensional metrology [2], but the digitisation process on such systems is very time-consuming for the acquisition of the initial set of points on complex or freeform surfaces if applied to RE.Another disadvantage in the context of RE is that a predefined path must be specified in advance to cover all features of a workpiece that are to be probed.This requires a prior knowledge of the part, presenting an obstacle to future automatic RE strategies.An alternative approach is represented by non-contact digitisation of surfaces based on optical techniques, for example laser scanning [3][4][5].A laser line sensor is a non-contact probe using the triangulation method to obtain point cloud data on a measured surface.Laser line scanning can capture dense point clouds efficiently in terms of speed and required human intervention.Additionally, it offers the possibility of measuring surface points from multiple features via a single and relatively simple probing path.This makes it a common choice in RE applications and quality control methods of freeform surfaces.However, laser line scanning is sensitive to issues that are almost irrelevant to tactile-based methods such as the high influence of surface colour, shininess, transparency, etc. [6].It also suffers from problems of digitization of any non-surface features such as slots or holes, due to occlusions and obscuration of these artefacts.In general, laser line scanning is considered to be less accurate in comparison with tactile probing [7].
Multi-sensor systems allow the selection of discrete probing or scanning methods to measure different features.The decision is often based on the principle that tight tolerance elements should be measured by high-precision contact methods, while elements with looser tolerance can be scanned via optical techniques.This combination of sensors enables the efficient measurement of a wider range of objects than any of the individual sensors alone and the limitations of each system are compensated by the other.
Durrant-Whyte [8] classifies physical sensor configuration in a multi-sensor data fusion system into three categories as follows: & A competitive sensor configuration is one where the sensors are configured to measure the same feature independently in order to reduce the measurement uncertainty and to avoid erroneous measurements.& A complementary sensor configuration is one where the sensors do not directly depend on each other but can be combined in order to give more complete information about the object.& A cooperative sensor configuration uses the information provided by two or more independent inhomogeneous sensors to derive data that would not be available from any sensor individually.
In this paper, we propose an effective competitive integration approach for the compensation of a laser line scanner by using a tactile probe to perform the reverse engineering of geometric features.With the coordinate data acquired using the laser scanning, intelligent feature recognition algorithms can be applied to extract the geometric elements of the object.These key features of elements can be re-measured by the slower tactile probe with a small number of points, and then the geometric elements can be described by mathematical and numerical methods.The obtained information can subsequently be used to compensate the comprehensive, loweraccuracy point cloud data measured by the laser line sensor.
The remainder of this paper is organized as follows: Section 2 surveys the prior work in multi-sensor coordinate measurement, Section 3 gives a brief introduction on data segmentation methods and Section 4 introduces the least squares best fitting methods for the four most common geometric features.The proposed method in this paper is described in Section 5 while Section 6 presents the experimental results.Finally, the conclusions of this piece of work and discussion of possible future direction is presented in Section 7.

Literature review
To meet the requirement of both high-speed and highaccuracy 3D measurement for quality control or RE applications, multi-sensor measuring systems have been developed to measure, analyse and rebuild a CAD model of the objects.When using RE methods to reproduce a given shape, the tolerance distribution of the scanned part must be considered [9].Bradley and Chan [10] and Zexiao et al. [11] each present a complementary sensor approach for RE.The optical sensor is applied to scan the profile of a part from different views, while the tactile probe is used to measure the edge and key features.
Hybrid contact-optical coordinate measuring systems are each designed by Chan [12], Carbone [13], Tzung-Sz [14] and Sladek et al. [15] where the obtained information digitized by an optical sensor is used to guide the touch probe for remeasuring the sample surfaces.The presented systems are cooperative integrations where optical sensors acquire the global shape information of objects to guide the touch probes for automatic point sensing.They are, however, limited to dealing with workpieces with relatively simple features.
Only limited research on competitive integration of hybrid contact-optical sensors has been found.Huang and Qian [16] develop a dynamic approach for integrating a laser scanner and a touch probe to improve the measurement speed and quality.A part is first scanned by the laser scanner to capture the overall shape.It is then probed by a tactile sensor where the probing positions are determined dynamically to reduce the measurement uncertainty according to the scanned data.They use a Kalman filter to fuse the data and to incrementally update the surface model based on the dynamically probed points.More recently, Bešić et al. [17] introduce a method for improving the output of a CMM-mounted laser line scanner for measurement applications.The improvement is achieved by using a median filter to reduce the laser scanner's random error and by simultaneously combining with the reliable but slow tactile probing.The filtered point data is used to estimate the form deviation of the inspected elements while a few points obtained by the tactile probe are used to compensate for errors in the point cloud position.Current commercial systems or software often only focus on processing points data from individual sensors or techniques; the issue of where and how to effectively and efficiently improve the accuracy of fused data is still a challenge.In particular, to the authors' best knowledge, no relevant research has provided a method to efficiently handle integrated measurement data in RE to use sparse accurate measurement information to improve the overall measurement accuracy for RE applications.

Point cloud data segmentation
After a part is scanned, the acquired point cloud data should be divided into several smooth regions for surface fitting purposes.This is called the segmentation process.Segmentation is the problem of grouping the points in the original dataset into subsets, each of which logically belong to a single primitive surface.A segmentation that extracts the edges and partitions the 3D point data plays an important role in fitting surface patches and applying the measured data to the RE process.Considerable research activities in shape segmentation have been explored in recent years.The methods for segmenting 3D data in engineering applications can be generally classified into three types: edge-based [18][19][20][21], regiongrowing [22][23][24] and hybrid-based [25][26][27] methods.The data sets are segmented into point-based data patches or polygonbased data patches by using different methods.As the discrete point data is much easier to modify in comparison with polyhedral surface, this work only considers the segmentation methods that are able to generate the outputs for point-based data patches.
Woo et al. [26] developed an octree approach for segmenting the scan data.First, the 3D non-uniform grids are generated by calculating the normal of each point.Then points are assigned in the subdivided cells with different levels in size.The edge points are extracted by selecting the points contained in the smallsized cells.Finally, the segmented point-based data patches are obtained after these edge points have been removed.This method is able to effectively extract edge neighbourhood points and group data points and was therefore selected for performing the data segmentation in this paper.

Least squares best fit geometric elements
After the segmentation process, the original point set is divided into subsets which can be broadly classified into two categories: geometric elements and freeform surfaces.The various geometries that are used to reconstruct a CAD model for RE applications and studied in this paper are planes, spheres, cylinders and cones.

Parameterization (a) Parameterization of plane
A space plane can be specified by a point (x o ,y o ,z o ) on the plane and the direction cosines (a,b,c) of the normal to the plane.
x o , y o , z o , a, b and c are the desired parameters.
(b) Parameterization of sphere A sphere is specified by its centre (x o ,y o ,z o ) and radius r.Any point (x i ,y i ,z i ) on the sphere satisfies the equation Equation ( 2) can be simplified as where a=2x o , b=2y o , c=2z o and ρ=x o 2 +y o 2 +z o 2 −r 2 .a, b, c and ρ are the desired parameters.

(c) Parameterization of cylinder
A cylinder can be specified by a point (x o ,y o ,z o ) on its axis; a vector (a,b,c) pointing along the axis and its radius, r.
For a near vertical axis, we can set c=1.Also, if x o and y o are known, then z o can be determined as (d) Parameterization of cone A cone can be specified by a point (x o ,y o ,z o ) on its axis; a vector (a,b,c) pointing along the axis and the apex semi-angle ϕ.
For a nearly vertical cone, we set c=1 and then axis

Least squares best fitting methods
The algorithms for fitting using a least squares method for various geometric elements have been studied by several authors [28][29][30].For linear geometries such as lines and planes, the parameters can be solved by linear least squares methods directly.For nonlinear geometries such as spheres, cylinders and cones, the Levenberg-Marquardt algorithm [30] and Gauss-Newton algorithm [31] have been widely used.Since the least squares best fit geometric elements algorithms are fully detailed by Forbes [28], his algorithms are exploited as fitting methods in this paper.
When applying the Levenberg-Marquardt algorithm or Gauss-Newton algorithm to fit cylinders and cones, good starting values are necessary for fast convergence of the algorithm and for obtaining the global optimal solution.In this paper, the tactile probe on CMM is used to scan two circles perpendicular to the axis (see Fig. 1).By least squares best fitting both 3D circles, the initial estimates for fitting cylinders and cones can be obtained.Then the data set is rotated by applying a 3×3 matrix, U, to align the cylinder or cone along the z-axis (see Fig. 1) to find the desired parameters.

Description of the proposed method
When digitizing an object, all surface geometry is captured including imperfections caused by the manufacturing process and any damage the part may have suffered as well as noise introduced by the measurement process.Typically, the part will be manually remodelled to capture the design intent and to disregard imperfections.There are some reasons for this.Firstly, modelling every defect could be time consuming and therefore expensive.Secondly, one of the main goals of RE is to reconstruct a CAD model of the workpiece.Therefore, the aim is to create a 'more perfect' part representing true design intent rather than simply copying the original product.This may require a detailed understanding of the function, depending on the part being modelled, because only then can the design intent be correctly interpreted.
After the multi-sensor coordinate system calibration, the optical scanner and tactile probe measure in nominally the same absolute coordinate system.However, two data sets measured by different sensors are unlikely to coincide absolutely, which means there will be a measurement difference between the tactile and optical sensor.The final aim of RE is to obtain a comprehensive, accurate CAD reconstruction model.To achieve this goal, a data compensation method is proposed to enhance the measurement accuracy of the point cloud data from the optical scanner.The proposed method is targeted at manufacturing problems where a reverse engineered model with accuracy better than 50 μm is needed.Naturally, the scanned data points must be representative of the geometric elements concerned.

Proposed method
After data segmentation, the data points are grouped into two types of data sets: geometric elements and freeform surfaces.
The elements which include planes, spheres, cylinders and cones can represent 85 % of machined objects [32].Due to their simple mathematical description and ability to model a large percentage of manufacture objects, they are widely used in various modelling systems [33,34].In RE, the accuracy of the final CAD model depends on the measured point data.A small amount of discrete point data measured by the high accuracy, but relatively slow tactile probe, can be used to compensate the densely scanned data patches that have been measured by the fast, but relatively low accuracy optical method.The specific method follows these logical steps:  1. Use the laser scanner to digitize the entire surface of part, and then exploit a segmentation algorithm, as described in Section 3, to group the point data patches each belonging to a different surface patches; these data patches will be compensated in step 3.

2.
Use the tactile probe to re-measure tight-toleranced geometric features with a small number of points to minimize the temporal cost.Then use the least squares method to best fit these geometric elements to derive the parameters based on mathematical and numerical principles.According to our proposed method, the new z coordinate can be derived by: To compare the normal of the plane, we let d=1.
(b) Sphere compensation According to Eq. ( 2), the parameters to be solved are x o , y o , z o and r.First, we translate a copy of the data so that the centre of the sphere is at the origin.
Then the value of the new z coordinate can be derived by Here, two values of z N are obtained that will fit the sphere.To determine which one is needed, the value of z(i) is determined to be positive or negative: In some cases, the solving results of z N includes imaginary parts, which means that the original points are beyond the scope of the sphere to be fitted.We have to delete those points in the new point data coordinates.In fact, this is an effective way to exclude noisy data.
Finally, the origin is translated by an amount equal and opposite to the vector in Eq. 6, above.

(c) Cylinder compensation
For cylinder and cone compensation, the proposed method is inverse shifting of the data so that the point (x o ,y o ,z o ) on the axis lies at the origin.Inverse rotation of the data set is then performed using the transpose of rotation matrix U which rotates (a, b, c) to coincide with z-axis.Again, we will translate and transform the data back after compensation.Then the value of the new y N coordinate can be obtained by The method for determining the positive and negative of y Ni is much the same as that for spheres.

(d) Cone compensation
After data set translation and rotation, the value of the new y Ni coordinate of cone can be calculated by where k=tan(ϕ).In this section, the feasibility and robustness of the proposed approach are first examined.Then a part with common geometric features is selected to test the introduced method after data segmentation.Finally, a workpiece that contains typical geometric features and freeform characteristics is investigated to further verify the proposed method.The multi-sensor measurement system used for data acquisition is a Nikon LK V 15.10.8HA high-accuracy ceramic bridge CMM (see Fig. 3a).It operates in a temperature-controlled room, typical of normal CMM applications, with the environmental temperature controlled at 20±0.2 °C.
The multi-sensor system integrates two different sensors: a Renishaw SP25M tactile probe (Fig. 3b) and a Nikon LC15Dx single-stripe laser scanner (Fig. 3c).The two sensors integrate with the CMM via the Renishaw PH10MQ articulating motorized probe head; only one sensor can operate at any given time.The software platform CAMIO7 multi-sensor CMM metrology software is used for measurement planning and data acquisition.
The specifications of the Renishaw SP25M scanning probe and Nikon LC15Dx laser scanner are listed in Table 1.

Example one
In example one, a sphere (part A) with nominal radius of 12.7 mm is used to test the robustness and feasibility of the introduced method.The sphere is made of solid polypropylene with a matt finish and has good roundness and sphericity.Part A is scanned by the LC15Dx laser scanner five times and SP25M touch probe three times, separately.During laser scanning, the distance between the surface of the artefact and the laser scanner in various orientations is kept constant by using the optimal distance to minimize influence of the scan depth to measurement error.The original point data scanned by the laser sensor is compensated by using data  measured by the tactile probe.Figure 4 shows the coordinates of the sphere centre, radii and standard deviations of part A separately calculated from the laser, tactile probe and compensated data.The fitting results show high stability of the data measured from tactile probe.The first set of data measured by the tactile probe is used to compensate the data scanned by laser sensor separately.We can see systematic errors between laser sensor and tactile sensor significant decrease after the data set is compensated.

Example two
Part B (see Fig. 5a) is made of aluminium alloy 5083 and designed with only geometric features.After data segmentation, these geometric elements are exploited to prove the concept proposed in this paper.The elements on the parts include most typical of geometric features: plane, sphere, cylinder and cone.The workpiece is located on the bed of the CMM and fixed by clamps when performing the measurement.
The laser sensor is used for surface global information acquisition.Because of the reflection of the surface, the intensity of the laser is changed to scan the shiny part.The views of the laser scanner are adjusted by the Renishaw PH10MQ probe head to cover the full surface.A trial version of the commercial software, Geomagic Wrap 2013, is used for performing the data pre-process tasks of data denoising and reduction, etc.The data scanned by laser and tactile probe are shown in Fig. 5b, c, respectively.Figure 5d displays merged and organized point cloud data.
The data measured by the laser scanner is segmented by the described feature recognition algorithm (see Fig. 6).Then the large amounts of unordered points that belong to different geometric element patches can be compensated by a small amount of point data using the tactile probe.
There is a desire to determine how many points must be captured by the tactile probe to achieve the desired representation of geometric elements.Traditionally, the number of sampled points is required to be ten times the number of parameters in the model [35].However, in this experiment, we take more sampled points and compare the fitting results.The fitting results for different features using different methods and their standard deviations are listed in Table 2.
All the computing tasks are operated on a desktop computer with an AMD Phenom II×4970 3.5 GHz processor and 8 GB RAM.Table 3 presents the computing time of data updating for different features.
The meshed surfaces before and after compensation are illustrated in Fig. 7a, b, respectively.From above fitting results we can draw several conclusions: 1.It has been shown that ten times the number of parameters is a sufficient number of measurement points using the tactile probe, considering its measurement uncertainty.This result is consistent with the conclusion introduced in literature [35].2. After compensation, the quality of the point data measured from the optical sensor has been greatly improved (see Table 2 and Fig. 7).In theory, the compensated data has the same accuracy as the data measured from the tactile probe.3. The compensated data is more robust and more likely to be identified by recognition algorithms, and is therefore more conducive to process in the next step of model reconstruction.

Example three
Part C is a more complex housing (Fig. 8a) which is modified from literature [36].This workpiece has a shiny metal surface.The laser sensor is used to scan the global information of the surface.Two poses of the part are required in order to acquire entirely the point cloud data from the top and the bottom of the workpiece.The datum plane and tight tolerance features need to be measured more accurately, and the inner holes are difficult to scan by laser scanner because of occlusion.Therefore, an SH25-1 stylus holder with a M3 40 mm stylus carried by the SP25M tactile probe is exploited to measure these features (see Fig. 8a).First of all, the top and bottom surface point data were registered to the same coordinate system; the method proposed in reference [37] is exploited to align the two pieces of data.The points scanned by the laser scanner and tactile probe are shown in Fig. 8b, c, respectively.All point cloud data is scanned manually by the operator.Table 4 presents the numbers of points of raw data, scanning views and measurement time.
After data segmentation and compensation, a reconstructed CAD model was rebuilt in the SolidWorks 2013 environment as shown in Fig. 8d (both uncompensated and compensated models are visually the same).

(a) Measurement speed comparison for sensors
In the digitizing process for part C, the actual scanning time is 19 min 19 s and the tactile probing time is 14 min 55 s, therefore the total measuring time is 34 min 14 s (Table 4).Finally 105,122 points scanned by laser sensor are retained after data reduction to generate satisfactory mesh surface for CAD model reconstruction, and 2,019 points measured from tactile probe are exploited to compensate the laser scanned data.
In comparison, if we assume that the tactile probe senses the part at the speed of 4-5 points per second, based on the scanning speed presented in Table 4, the time to digitize the part using only a tactile method would be a minimum of 6 h.The integration of the laser scanner and CMM therefore leads to much faster measurement than the tactile method alone.

(b) CAD model accuracy comparison
To validate the final reconstructed CAD model quality, we use the root mean square (RMS) error distance to check the accuracy of the constructed CAD model S CAD .However, in practice, the actual surface may be difficult or even impossible to obtain.A large number of accurate CMM probing points can be exploited as the reference points on the actual surface.The RMS can then be obtained by where p i T is the sampled point using CMM tactile probe, N is the number of those sampled points, p i S is the projected point to the constructed surface of CAD model S CAD and d(p i T ,p i S ) is the distance between p i T and p i S .Both uncompensated and compensated models are used as the nominal geometry for CMM measurement using a tactile probe.In each case, the same scanning path was used.In the first instance, the Nikon CMM with Renishaw SP25M probe used to perform the RE measurements was used to evaluate the accuracy of both models.Figure 9 presents the comparison between probed points and final reconstructed CAD models before and after compensation.To further examine performance of the reconstructed CAD model, an independent, high-accuracy Zeiss PRISMO CMM (MPE E = 1.9 + L/ 300 μm (ISO 10360-2:2009)) was used.Table 5 shows the RMS comparisons of reconstructed CAD models for both systems.We can see that both results indicate that the accuracy of the geometric elements of reconstructed CAD model has been greatly improved after compensation.information of the object and then guide the slower tactile probe to digitize the In the presented method, the data sets are acquired by laser scanning and tactile probing in both competitive and complementary multi-sensor configurations.With the point cloud data acquired using a laser scanner, intelligent feature recognition and segmentation algorithms can be applied to extract and segment the point cloud data.Then, the tactile probe is used to re-measure the holes, which are difficult to measure by laser sensor, and tight tolerance geometric features with a small number of sampling points.The obtained information can be subsequently used to compensate the point data patches which are measured by laser sensor.
The results of the three case-study experiments show that the algorithms provide satisfactory performance and the vast amounts of unordered points measured from optical sensor are converted to orderly and more accurate point data after compensation.In addition, it usually needs several hours to fully digitize a workpiece by using a tactile sensing device.However, in the presented approach, the total measurement time is drastically reduced.A case strictly (part C) would take 6 h to digitize using a tactile probe, but only about 30 min using the proposed method, an improvement of 90 % which greatly improves measurement efficiency without losing accuracy.
These results compare well to other competitive methods.Huang and Qian's approach [16] can effectively save measurement time and be able to deal with shiny surfaces, but according to the experimental results displayed in the literature, this approach does not significantly improve accuracy of the fused data.In Bešić et al.'s work [17], the introduced method is very intuitive and understandable.However, only a plane is considered in the literature and, because each point cloud data must be filtered before shifting, the usability is adversely affected.In general, the method introduced in this paper reveals a better accuracy of data fusion than above approaches.
The proposed multi-data compensation method has been proven to work for geometric elements.Multi-sensor data fusion for freeform dimensional measurement or reverse engineering applications is another promising avenue for research.In addition, a cooperative and competitive integration of the proposed multi-sensor measuring system would realize the benefits of automatic digitization and would be especially valuable for reconstruction of complex surfaces.

Fig. 1
Fig. 1 Points data translation and rotation: a cylinder, b cone

Fig. 3
Fig. 3 Nikon LK V 15.10.8HA high accuracy ceramic bridge CMM. a Overview of Nikon LK V 15.10.8HA CMM, b Renishaw SP25M probe and SH25-1 stylus holder and c Nikon LC15Dx laser scanner

Fig. 4 3 .
Fig. 4 Fitting results of different methods.a X coordinates, b Y coordinates, c Z coordinates, d sphere radius and f standard deviations

2 pFig. 5
Fig. 5 Points data measured using multi-sensor system.a Part B to be measured, b data measured from laser scanner, c points measured from tactile probe and d merged data

Fig. 6
Fig. 6 Point cloud data segmentation: a plane, b sphere, c cylinder and d cone

Fig. 7
Fig. 7 Mesh surface.a Mesh surface before compensation and b mesh surface after compensation

Fig. 8
Fig. 8 RE of part C. a Part C and tactile probing plan, b points data scanned from laser, c data captured by tactile probe, d CAD model constructed in SolidWorks

7Fig. 9
Fig. 9 The comparison between probed points and reconstructed CAD model.a CAD model before compensation and b CAD model after compensation

Table 1
Specifications of the Renishaw SP25M probe and Nikon

Table 3
Computational time

Table 4
Numbers of points and scanning time