Huber\(L_1\)based nonisometric surface registration
Abstract
Nonisometric surface registration is an important task in computer graphics and computer vision. It, however, remains challenging to deal with noise from scanned data and distortion from transformation. In this paper, we propose a Huber\(L_1\)based nonisometric surface registration and solve it by the alternating direction method of multipliers. With a Huber\(L_1\)regularized model constrained on the transformation variation and position difference, our method is robust to noise and produces piecewise smooth results while still preserving fine details on the target. The introduced assimilaraspossible energy is able to handle different size of shapes with little stretching distortion. Extensive experimental results have demonstrated that our method is more accurate and robust to noise in comparison with the stateofthearts.
Keywords
Surface registration Huber\(L_1\) Nonisometric1 Introduction
Surface registration has drawn intensive attention in computer graphics and computer vision with massive applications such as film production and computer games. With the development of geometry acquisition technology, 3D scanning systems allow us to capture highresolution and highly detailed 3D geometries even for dynamic scenes. However, these scanned data often contain noise, especially from commodity depth sensors such as Microsoft Kinect. To tackle this problem, surface registration is introduced to deform a high quality model (template) so that it aligns with the scanned shape (target). Often, the size and details of the template differ from that of the target. Hence, registration methods which are robust to noise and capable of handling different sizes and details have been investigated.
According to the type of deformation mapping [6], the surface registration is generally categorized into two groups: rigid registration and nonrigid registration. The rigid registration [2, 3] aims to find a rigidbody transformation between two shapes, and thus, it cannot handle deformable (nonrigid) shapes. The nonrigid registration, including isometric registration and similar registration, is to find a set of local transformations that align two shapes. The isometric registration which deforms the shape in an asrigidaspossible (ARAP) manner has been widely used between isometric shapes [7, 10, 25]. However, it is not capable of handling shapes with different sizes since it tries to preserve the length of edges. To address this limitation, similar registration approaches [8, 29] introduce a scale factor into each local transformation and formulate as an assimilaraspossible (ASAP) energy, which allows local scalability to handle the size difference.
For the accuracy and robustness of registration, the transformation variation and position difference constraints are usually formulated as a smoothness term and a data term, respectively, to measure the smoothness of the neighboring transformation and the closeness of registration shapes, respectively. Most works [1, 8, 10] use the classic squared \(L_2\)norm on both constraints (\(L_2\)\(L_2\)). However, the smoothness term in \(L_2\)norm tends to penalize large transformation variation. It is not suitable for articulated models where large deformation variations exist at their joints. This could also be seen in image processing where discontinuities are allowed to highlight the sharp edges in image denoising [5]. The classic model in image denoising is ROF model [18], where the total variation (TV) term is an \(L_1\)normbased regularization and the data term is in squared \(L_2\)norm (TV\(L_2\)). Based on ROF model, Yang et al. [27] propose a sparse nonrigid registration method with an \(L_1\)norm regularized on the smoothness term. However, ROF tends to produce overregularized results as the \(L_2\)norm strives to distribute errors evenly, thus fitting the result evenly on the noisy parts. To tackle this issue, TV\(L_1\) model [30] is proposed to efficiently remove the outliers while preserving fine details. Based on TV\(L_1\) model, Yang et al. [11] propose a dual sparsity registration approach on both position and transformation sparsity, allowing the positional error to concentrate on small regions. However, TV\(L_1\) model tends to produce piecewise constant results as shown in Werlberger et al. [26]. We propose a Huber\(L_1\)based nonisometric surface registration to reduce the staircasing artifacts known from TV. A Hubernorm is applied on the transformation variation, which significantly reduces artificial discontinuities and produces piecewise smooth results. Meanwhile, an \(L_1\)norm is applied on the position difference, solving the overregularized issue that appeared in \(L_2\)norm. The ASAP energy is introduced to handle shapes of different sizes and poses. To reduce the risks of surface foldover, we adopt Laplacian energy to smooth the template on both primal and dual domains. We compare Huber\(L_1\) to other stateoftheart models in the experiments on clean, noisy and real scanned data, demonstrating the advantage and robustness of our method.

We propose a Huber\(L_1\)based nonisometric registration method regularized on transformation variation and position difference. The Huber\(L_1\) model is solved by the alternating direction method of multipliers (ADMM) with each energy term being represented in matrix form. The proposed model is robust to noise and produces piecewise smooth results with the target’s fine details being well preserved.

We incorporate ASAP energy in the registration method, which is not only able to handle shapes with different sizes but can also reduce local stretch and distortion. Since ASAP energy preserves the overall geometry of the template while allowing local scaling, it demands a few user efforts to provide a good initial shape estimation.

We introduce Laplacian energy to relax the template on both primal and dual domains, improving the template mesh quality and decreasing the foldover occurrence.
2 Related work
Over last two decades, surface registration has been deeply researched. A complete survey can be referred in [24]. Here, we briefly review the related works on three categories: rigid registration, isometric registration and nonisometric registration.
Rigid registration The most classic algorithm in rigid registration is iterative closest point (ICP) [2]. It alternates between closest point searching and optimal transformation solving. To improve the converge rate of ICP, Low et al. [13] used a pointtoplane metric in which the energy function is the sum of the squared distance between a point and the tangent plane at its correspondence point. One main issue for ICP and its variants is that they are sensitive to outliers and missing data. To tackle this issue, Bouaziz et al. [3] propose a new formulation of ICP using sparsity inducing norms. However, methods in this category cannot deal with deformable shapes registration.
Isometric registration Huang et al. [7] present a nonrigid registration of a pair of partially overlapping surfaces, constraining transformation locally asrigidaspossible (ARAP). Rouhani et al. [17] propose a nonrigid registration between two clouds of points, where the source set is clustered into small patches and then deformed rigidly to align with the target. Sussmuth et al. [23] slide the template mesh along the time–space surface in ARAP manner to reconstruct animated meshes from a series of timedeforming point clouds. Wand et al. [25] take a set of timevarying unstructured sample points as input and reconstruct a single shape and a local deformation field. Li et al. [10] introduce a registration algorithm for partial range scans of deforming shapes. They deform local features as rigidly as possible to avoid shearing and stretching artifacts. Yang et al. [11] propose a dualsparsitybased nonrigid registration, which adds orthogonality constraints on the local transformation to preserve local rigidity.
3 Surface registration
In this paper, we adopt a coarsetofine fitting strategy to implement the whole registration process in three steps: coarse fitting, midscale fitting and fine fitting. For each step, different energy terms are used and combined, which will be introduced at their first appearance. The comparison results with and without some energy terms will be illustrated to stress the significance of these energies.
3.1 Notations
Suppose the template mesh is composed of n vertices \({\mathcal {P}} \triangleq \{{\mathbf {p}}_1, \ldots ,{\mathbf {p}}_n\}\), where \({\mathbf {p}}_i \triangleq [x_i, y_i, z_i]^\top \) is a 3D vertex position in Euclidean coordinate. In coarse fitting step, n is the vertex number of coarse template mesh. The vertices of the target are denoted as \({\mathcal {Q}} = \{{\mathbf {q}}_1, \ldots ,{\mathbf {q}}_m\}\). For nonrigid registration, a \(3 \times 4\) affine transformation matrix \({\mathbf {A}}_i \triangleq [{\mathbf {X}}_i, {{\mathbf {t}}}_i]\) is associated with each vertex \({\mathbf {p}}_i\) of the template, where \({\mathbf {X}}_i\) is a \(3 \times 3\) linear transformation matrix and \({{\mathbf {t}}}_i\) is a \(3 \times 1\) translation vector. For simplification, we concatenate \({\mathbf {p}}_i, {\mathbf {q}}_i, {\mathbf {X}}_i, {{\mathbf {t}}}_i\) into a \(n \times 3\) matrix \({\mathbf {P}} \triangleq [{\mathbf {p}}_1 \ldots {\mathbf {p}}_n]^\top \), a \(m \times 3\) matrix \({\mathbf {Q}} \triangleq [{\mathbf {q}}_1 \ldots {\mathbf {q}}_m]^\top \), a \(3n \times 3\) matrix \({\mathbf {X}} \triangleq [{\mathbf {X}}_1 \ldots {\mathbf {X}}_n]^\top \) and a \(n \times 3\) matrix \({\mathbf {T}} \triangleq [{{\mathbf {t}}}_1 \ldots {{\mathbf {t}}}_n]^\top \), respectively. Similarly, the vertices on the template dual mesh \({\mathcal {P}}^*\) are denoted by \({\mathbf {P}}^* \triangleq [{\mathbf {p}}^*_1 \ldots {\mathbf {p}}^*_{n^*}]^\top \), where \(n^*\) is the number of the vertices on the template dual mesh, which is also equal to the number of triangle faces on the template primal mesh. Again, a translation vector will be assigned to each dual vertex, all of which can be concatenated as \({\mathbf {T}}^* \triangleq [{{\mathbf {t}}}^*_1 \ldots {{\mathbf {t}}}^*_{n^*}]^\top \).
3.2 Coarse fitting
3.2.1 Feature point constraints
3.2.2 ASAP energy
3.2.3 Regularization
3.2.4 Laplacian energy
3.3 Midscale fitting
3.4 Data constraint

\({\mathbf {c}}_i\) is inside a triangle of the target.

The distance between \({\mathbf {c}}_i\) and \({\mathbf {p}}_i\) is under a threshold \(\alpha \).

The angle between the normals at \({\mathbf {c}}_i\) and \({\mathbf {p}}_i\) is under a threshold \(\Theta \).
3.5 Fine fitting
3.6 Consistency constraint
If the resolution of the template mesh is insufficient to fit the target tightly, a uniform or adaptive subdivision approach can be employed. Here, we adopt 1–4 uniform subdivision method [9] to subdivide the template (Fig. 1j), and then, the dualdomain relaxation algorithm is performed on the subdivided template mesh again.
4 Optimization
5 Experiments
The number of feature points (#FP), vertices (#V), faces (#F) of the template and the target models in the examples
Name  #FP  Template  Target  

#V  #F  #V  #F  
Bouncing  9  12,500  24,996  10,002  20,000 
Camel  24  6608  13,200  9469  18,934 
Crane  11  12,500  24,996  10,002  20,000 
Dog  0  25,290  50,528  25,290  50,528 
Gorilla  0  25,438  50,868  25,438  50,868 
Head  10  1669  3298  281,581  562,554 
5.1 Parameters and weights
Quantitative evaluation in the bouncing and camel examples. D, I and H indicate distance error [%], intersection error and Hausdorff error [%], respectively
Huber\(L_1\)  CASAP  ACAP  SMASAP  PDS  

Bouncing  
D  2.1556e \(\) 05  4.3619e\(\)05  4.9460e\(\)05  0.0010  1.8525e04 
I  0  242  494  909  5639 
H  0.0113  0.0273  0.0784  0.7279  0.1619 
Camel  
D  7.1209e \(\) 05  1.8185e\(\)04  1.8074e\(\)04  9.8233e\(\)04  2.9304e\(\)04 
I  0  0  76  85  3884 
H  0.0172  0.0704  0.0710  0.3222  0.1238 
5.2 Results on clean data
Iteration steps and time (in seconds) in the bouncing and camel example. #O and #I indicate the number of outer iteration steps and total inner iteration steps, respectively. “Inner” indicates the average time required for each inner iteration step. “Total” represents the total fitting time
Huber\(L_1\)  CASAP  ACAP  SMASAP  PDS  

Bouncing  
#O  24  54  73  9  500 
#I  2064  3477  577  37  1608 
Inner  0.068  0.034  0.277  0.252  0.035 
Total  140.199  118.044  159.687  9.346  56.281 
Camel  
#O  24  54  73  9  500 
#I  2037  3352  562  34  1582 
Inner  0.065  0.032  0.272  0.261  0.034 
Total  132.405  107.264  152.864  8.874  54.328 
Quantitative evaluation in the crane example
Huber\(L_1\)  CASAP  ACAP  SMASAP  PDS  

Crane  
D  4.7397e−05  5.2738e−05  5.6536e−05  9.4177e−04  1.6583e−04 
I  0  143  453  8965  5862 
H  0.0227  0.0729  0.0469  0.5691  0.1258 
5.3 Results on noisy data
In this subsection, we set up two different experiments to demonstrate the robustness of our method. In both experiments, the targets are polluted with noise along the normal direction of each vertex by multiplying the standard deviation of the average length of the edges in the target.
First, we compare our method with the startofthearts on the noisy data in Fig. 6. The quantitative evaluation is shown in Table 4. Affected by the noise, CASAP and ACAP get poor initial shape estimation and regard some noise as correspondence, which makes parts of template fitted to noise as shown at the right waist of CASAP and the left arm of ACAP. SMASAP and PDS still produce poor results as on the clean data. Thanks to the dual relaxation and Huber\(L_1\) regularization, our method is robust against noise and achieves more accurate results than other methods without any foldover generated.
Quantitative evaluation in the dog and gorilla examples
Huber\(L_1\)  TV\(L_1\)  TV\(L_2\)  \(L_2\)\(L_2\)  

Dog  
D  1.0441e−06  5.2783e−06  1.0682e−05  1.5101e−05 
H  1.6409  2.9771  3.9666  2.6419 
Gorilla  
D  2.3474e−06  2.8365e−06  1.6977e−05  2.3565e−05 
H  4.1795  4.5067  5.5522  5.1793 
5.4 Results on real scans
6 Conclusions
Quantitative evaluation in the head example
Huber\(L_1\)  CASAP  ACAP  SMASAP  PDS  

Head  
D  3.6954e−06  4.0682e−04  3.1503e−04  0.0016  0.0048 
I  0  1287  2345  103  185 
H  3.1624  3.2413  3.2244  3.2463  3.2516 
In the future, we will detect the inversion of the elements to prevent the occurrence of foldover completely during registration. In addition, if the feature points can be detected automatically from deep learning, the whole registration process will be implemented without any user intervention. This would be another interesting research topic.
Notes
Acknowledgements
We would like to thank Gabriel Peyre and Alec Jacobson for their help in rendering experimental results in MATLAB.
Funding
This study was funded by EU H2020 under the REA Grant Agreement (Grant Number 691215)
Compliance with ethical standards
Conflicts of interest
The authors declare that they have no conflict of interest.
References
 1.Amberg, B., Romdhani, S., Vetter, T.: Optimal step nonrigid ICP algorithms for surface registration. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007, CVPR’07, pp. 1–8. IEEE (2007)Google Scholar
 2.Besl, P.J., McKay, N.D.: A method for registration of 3D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)Google Scholar
 3.Bouaziz, S., Tagliasacchi, A., Pauly, M.: Sparse iterative closest point. In: Computer graphics forum, vol. 32, pp. 113–123. Wiley (2013)Google Scholar
 4.Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J., et al.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends® Mach. Learn. 3(1), 1–122 (2011)zbMATHGoogle Scholar
 5.Chambolle, A., Pock, T.: A firstorder primaldual algorithm for convex problems with applications to imaging. J. Math. Imag. Vis. 40(1), 120–145 (2011)MathSciNetzbMATHGoogle Scholar
 6.Floater, M.S., Hormann, K.: Surface parameterization: a tutorial and survey. In: Dodgson, N., Floater, M.S., Sabin, M. (eds.) Advances in Multiresolution for Geometric Modelling, pp. 157–186. Springer, Berlin (2005)Google Scholar
 7.Huang, Q.X., Adams, B., Wicke, M., Guibas, L.J.: Nonrigid registration under isometric deformations. In: Computer Graphics Forum, vol. 27, pp. 1449–1457. Wiley (2008)Google Scholar
 8.Jiang, T., Qian, K., Liu, S., Wang, J., Yang, X., Zhang, J.: Consistent assimilaraspossible nonisometric surface registration. Vis. Comput. 33(6–8), 891–901 (2017)Google Scholar
 9.Lee, A., Moreton, H., Hoppe, H.: Displaced subdivision surfaces. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 85–94. ACM Press/AddisonWesley Publishing Co. (2000)Google Scholar
 10.Li, H., Sumner, R.W., Pauly, M.: Global correspondence optimization for nonrigid registration of depth scans. In: Computer Graphics Forum, vol. 27, pp. 1421–1430. Wiley (2008)Google Scholar
 11.Li, K., Yang, J., Lai, Y., Guo, D.: Robust nonrigid registration with reweighted position and transformation sparsity. IEEE Trans. Vis. Comput. Graphics 25(6), 2255–2269 (2019)Google Scholar
 12.Liao, M., Zhang, Q., Wang, H., Yang, R., Gong, M.: Modeling deformable objects from a single depth camera. In: IEEE 12th International Conference on Computer Vision, 2009, pp. 167–174. IEEE (2009)Google Scholar
 13.Low, K.L.: Linear LeastSquares Optimization for PointtoPlane ICP Surface Registration, vol. 4. University of North Carolina, Chapel Hill (2004)Google Scholar
 14.Moenning, C., Dodgson, N.A.: Fast marching farthest point sampling. Technical report, University of Cambridge, Computer Laboratory (2003)Google Scholar
 15.Müller, M., Heidelberger, B., Teschner, M., Gross, M.: Meshless deformations based on shape matching. ACM Trans. Gr. 24, 471–478 (2005)Google Scholar
 16.Papazov, C., Burschka, D.: Deformable 3D shape registration based on local similarity transforms. In: Computer Graphics Forum, vol. 30, pp. 1493–1502. Wiley (2011). https://doi.org/10.1111/j.14678659.2011.02023.x
 17.Rouhani, M., Boyer, E., Sappa, A.D.: Nonrigid registration meets surface reconstruction. In: 2014 2nd International Conference on 3D Vision (3DV), vol. 1, pp. 617–624. IEEE (2014)Google Scholar
 18.Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D Nonlinear Phenom. 60(1–4), 259–268 (1992)MathSciNetzbMATHGoogle Scholar
 19.Sorkine, O., Alexa, M.: Asrigidaspossible surface modeling. In: Symposium on Geometry Processing, vol. 4, p. 30 (2007)Google Scholar
 20.Sumner, R.W., Popović, J.: Deformation transfer for triangle meshes. ACM Trans. Gr. 23, 399–405 (2004)Google Scholar
 21.Sumner, R.W., Schmid, J., Pauly, M.: Embedded deformation for shape manipulation. ACM Trans. Gr. 26, 80 (2007)Google Scholar
 22.Surazhsky, V., Surazhsky, T., Kirsanov, D., Gortler, S.J., Hoppe, H.: Fast exact and approximate geodesics on meshes. ACM Trans. Gr. 24, 553–560 (2005)Google Scholar
 23.Süßmuth, J., Winter, M., Greiner, G.: Reconstructing animated meshes from timevarying point clouds. In: Computer Graphics Forum, vol. 27, pp. 1469–1476. Wiley (2008)Google Scholar
 24.Tam, G.K., Cheng, Z.Q., Lai, Y.K., Langbein, F.C., Liu, Y., Marshall, D., Martin, R.R., Sun, X.F., Rosin, P.L.: Registration of 3d point clouds and meshes: a survey from rigid to nonrigid. IEEE Trans. Vis. Comput. Gr. 19(7), 1199–1217 (2013)Google Scholar
 25.Wand, M., Adams, B., Ovsjanikov, M., Berner, A., Bokeloh, M., Jenke, P., Guibas, L., Seidel, H.P., Schilling, A.: Efficient reconstruction of nonrigid shape and motion from realtime 3d scanner data. ACM Trans. Gr. 28(2), 15 (2009)Google Scholar
 26.Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., Bischof, H.: Anisotropic huberl1 optical flow. In: BMVC, vol. 1, p. 3 (2009)Google Scholar
 27.Yang, J., Li, K., Li, K., Lai, Y.K.: Sparse nonrigid registration of 3D shapes. In: Computer Graphics Forum, vol. 34, pp. 89–99. Wiley (2015). https://doi.org/10.1111/cgf.12699
 28.Yeh, I.C., Lin, C.H., Sorkine, O., Lee, T.Y.: Templatebased 3d model fitting using dualdomain relaxation. IEEE Trans. Vis. Comput. Gr. 17(8), 1178–1190 (2011)Google Scholar
 29.Yoshiyasu, Y., Ma, W.C., Yoshida, E., Kanehiro, F.: Asconformalaspossible surface registration. In: Computer Graphics Forum, vol. 33, pp. 257–267. Wiley (2014). https://doi.org/10.1111/cgf.12451
 30.Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TVl 1 optical flow. In: Joint Pattern Recognition Symposium, pp. 214–223. Springer (2007)Google Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.