Skip to main content
SpringerLink
Log in

3D Reconstruction Supported by Gaussian Process Latent Variable Model Shape Priors

  • Original Article
  • Published:
PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science Aims and scope Submit manuscript

Abstract

This article presents a method that removes outliers, reduces noise and fills holes in a point cloud using a learned shape prior. The shape prior is learned from a set of training objects using the Gaussian process latent variable model. All training objects are represented by a signed distance function. To improve the time performance, the signed distance functions are compressed using the discrete cosine transform. As input data the method uses the estimated object pose from an object detector and a segmented point cloud. It is shown that the estimated shape prior is capable of modelling fine details to a certain degree. The limitations and difficulties of this method are also investigated. It is shown that by applying this method the accuracy and completeness of the reconstructed models can be significantly increased.

Zusammenfassung

Bereinigung von 3D-Punktwolken mit Hilfe von a-priori-Formen, geschätzt mit dem Gaussian Process Latent Variable Model. Dieser Artikel präsentiert ein Verfahren, mit dem Ausreißer und Rauschen aus einer Punktwolke entfernt und Löcher gefüllt werden können. Das Verfahren verwendet dabei eine angelernte a-priori-Form. Diese Form wird aus mehreren Trainingsformen mit dem Gaussian Process Latent Variable Model angelernt. Alle Trainingsobjekte werden durch eine vorzeichenbehaftete Abstandsfunktion repräsentiert. Zur Erhöhung der Performanz werden die Abstandsfunktionen mit der Diskreten Kosinustransformation komprimiert. Als Eingabedaten verwendet das Verfahren die vom Objekt-Detektor geschätzte Pose und eine segmentierte Punktwolke. Es wird gezeigt, dass die geschätzte a-priori-Form in der Lage ist, Details bis zu einem bestimmten Grad zu modellieren. Die Grenzen dieses Verfahrens werden ebenfalls untersucht. Es wird gezeigt, dass durch Anwendung des Verfahrens die Genauigkeit und Vollständigkeit der rekonstruierten Objekte signifikant verbessert werden kann.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  • Ahmed N, Natarajan T, Rao KR (1974) Discrete cosine transform. IEEE Trans Comput 23(1):90–93

    Article  Google Scholar 

  • Allen B, Curless B, Popović Z (2003) The space of human body shapes: reconstruction and parameterization from range scans. ACM Trans Graph (SIGGRAPH Conf Proc) 22(3):587–594

    Article  Google Scholar 

  • Anguelov D, Srinivasan P, Koller D, Thrun S, Rodgers J, Davis J (2005) Scape: shape completion and animation of people. ACM Trans Graph (SIGGRAPH Conf Proc) 24(3):408–416

    Article  Google Scholar 

  • Bao SYZ, Chandraker M, Lin Y, Savarese S (2013) Dense object reconstruction with semantic priors. In: IEEE Conference on computer vision and pattern recognition, pp 1264–1271

  • Berger M, Tagliasacchi A, Seversky LM, Alliez P, Levine JA, Sharf A, Silva C (2014) State of the art in surface reconstruction from point clouds. In: Eurographics 2014—State of the Art Reports, pp 161–185

  • BlendSwap (2016) http://www.blendswap.com/. 17 Nov 2016

  • Broyden CG (1970) The convergence of a class of double-rank minimization algorithms 1. General considerations. J Inst Math Appl 6(1):76–90

    Article  Google Scholar 

  • Dame A, Prisacariu VA, Ren CY, Reid ID (2013) Dense reconstruction using 3d object shape priors. In: IEEE conference on computer vision and pattern recognition, pp 1288–1295

  • Felzenszwalb PF, Girshick RB, McAllester D, Ramanan D (2010) Object detection with discriminatively trained part based models. IEEE Trans Pattern Anal Mach Intell 32(9):1627–1645

    Article  Google Scholar 

  • Furukawa Y, Ponce J (2010) Accurate, dense, and robust multi-view stereopsis. IEEE Trans Pattern Anal Mach Intell 32(8):1362–1376

    Article  Google Scholar 

  • Furukawa Y, Curless B, Seitz SM, Szeliski R (2010) Towards internet-scale multi-view stereo. In: IEEE conference on computer vision and pattern recognition, pp 1434–1441

  • Gal R, Shamir A, Hassner T, Pauly M, Cohen-Or D (2007) Surface reconstruction using local shape priors. In: Fifth Eurographics symposium on geometry processing, pp 253–262

  • GrabCAD (2016) https://grabcad.com/. 17 Nov 2016

  • Heimann T, Meinzer HP (2009) Statistical shape models for 3d medical image segmentation: a review. Med Image Anal 13(4):543–563

    Article  Google Scholar 

  • Hotelling H (1933) Analysis of a complex of statistical variables into principal components. J Educ Psychol 24:417–441

    Article  Google Scholar 

  • Huang J, You S (2014) Segmentation and matching: towards a robust object detection system. In: IEEE winter conference on applications of computer vision, pp 325–332

  • Jensen R, Dahl A, Vogiatzis G, Tola E, Aanæs H (2014) Large scale multi-view stereopsis evaluation. In: IEEE conference on computer vision and pattern recognition, pp 406–413

  • Jones MW, Baerentzen JA, Sramek M (2006) 3d distance fields: a survey of techniques and applications. IEEE Trans Vis Comput Graph 12(4):581–599

    Article  Google Scholar 

  • Kazhdan M, Hoppe H (2013) Screened poisson surface reconstruction. ACM Trans Graph (SIGGRAPH Conf Proc) 32(3):29:1–29:13

  • Kraevoy V, Sheffer A (2005) Template-based mesh completion. In: Proceedings of the third Eurographics symposium on geometry processing, Eurographics Association, SGP ’05, pp 13–22

  • Krenzin J, Hellwich O (2016) Reduction of point cloud artifacts using shape priors estimated with the gaussian process latent variable model. In: Pattern recognition. GCPR. Lecture notes in computer science. Springer International Publishing, Cham, pp 273–284 (2016)

  • Lawrence ND (2004) Gaussian process latent variable models for visualisation of high dimensional data. Adv Neural Inf Process Syst 16(3):329–336

    Google Scholar 

  • Lawrence ND (2005) Probabilistic non-linear principal component analysis with Gaussian process latent variable models. J Mach Learn Res 6(Nov):1783–1816

  • Lévy B (2003) Dual domain extrapolation. ACM Trans Graph (SIGGRAPH Conf Proc) 22(3):364–369

    Article  Google Scholar 

  • Lorensen WE, Cline HE (1987) Marching cubes: a high resolution 3d surface construction algorithm. ACM Trans Graph (SIGGRAPH Conf Proc) 21(4):163–169

    Article  Google Scholar 

  • Matlab (2015) MATLAB and optimization toolbox release R2015a. The MathWorks Inc., Natick. https://www.mathworks.com/

  • Maurer CR Jr, Qi R, Raghavan V (2003) A linear time algorithm for computing exact Euclidean distance transforms of binary images in arbitrary dimensions. IEEE Trans Pattern Anal Mach Intell 25(2):265–270

    Article  Google Scholar 

  • Moons T, Van Gool L, Vergauwen M (2009) 3d reconstruction from multiple images: part 1—principles. Found Trends Comput Graph Vis 4(4):287–404

  • Nan L, Xie K, Sharf A (2012) A search-classify approach for cluttered indoor scene understanding. ACM Trans Graph (SIGGRAPH Conf Proc) 31(6):137:1–137:10

  • Pauly M, Mitra NJ, Giesen J, Gross MH, Guibas LJ (2005) Example-based 3d scan completion. In: Symposium on geometry processing, Eurographics Association, ACM transactions on graphics (SIGGRAPH conference proceedings), vol 255, pp 23–32

  • Pearson K (1901) On lines and planes of closest fit to systems of points in space. Philos Mag 2(6):559–572

    Article  Google Scholar 

  • Prisacariu VA, Reid ID (2011a) Nonlinear shape manifolds as shape priors in level set segmentation and tracking. In: IEEE conference on computer vision and pattern recognition, pp 2185–2192

  • Prisacariu VA, Reid ID (2011b) Shared shape spaces. In: International conference on computer vision. IEEE Computer Society, pp 2587–2594

  • Prisacariu VA, Segal AV, Reid ID (2013) Simultaneous monocular 2d segmentation, 3d pose recovery and 3d reconstruction. In: Asian conference on computer vision, pp 593–606

  • Quirion S, Duchesne C, Laurendeau D, Marchand M (2008) Comparing GPLVM approaches for dimensionality reduction in character animation. J Int Conf Central Eur Comput Graph Vis Comput Vis 16(1–3):41–48

    Google Scholar 

  • Ren CY, Prisacariu V, Reid I (2013) Regressing local to global shape properties for online segmentation and tracking. Int J Comput Vis 106(3):269–281

  • Rusu RB, Cousins S (2011) 3D is here: Point Cloud Library (PCL). In: IEEE international conference on robotics and automation, pp 1–4

  • Schölkopf B, Smola A, Müller KR (1998) Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput 10(5):1299–1319

    Article  Google Scholar 

  • Seitz S, Curless B, Diebel J, Scharstein D, Szeliski R (2006) A comparison and evaluation of multi-view stereo reconstruction algorithms. In: IEEE conference on computer vision and pattern recognition, vol 1, pp 519–526

  • Shao T, Xu W, Zhou K, Wang J, Li D, Guo B (2012) An interactive approach to semantic modeling of indoor scenes with an RGBD camera. ACM Trans Graph (SIGGRAPH Conf Proc) 31(6):136:1–136:11

  • Sheffer A, Praun E, Rose K (2006) Mesh parameterization methods and their applications. Found Trends Comput Graph Vis 2(2):105–171

    Article  Google Scholar 

  • Shen CH, Fu H, Chen K, Hu SM (2012) Structure recovery by part assembly. ACM Trans Graph (SIGGRAPH Conf Proc) 31(6):180:1–180:11

  • Teutsch C, Trostmann E, Berndt D (2011) A parallel point cloud clustering algorithm for subset segmentation and outlier detection. Proc SPIE 8085(808):509

    Google Scholar 

  • Wang Q, Wang F, Li D, Wang X (2014) Clustering-based latent variable models for monocular non-rigid 3d shape recovery. In: 10th International conference on intelligent computation 2014, pp 162–172

  • Wu C (2016) VisualSFM: a visual structure from motion system. http://ccwu.me/vsfm/. 17 Nov 2016

  • Zhang Z (1994) Iterative point matching for registration of free-form curves and surfaces. Int J Comput Vis 13(2):119–152

    Article  Google Scholar 

  • Zheng S, Prisacariu VA, Averkiou M, Cheng MM, Mitra NJ, Shotton J, Torr PHS, Rother C (2015) Object proposals estimation in depth image using compact 3d shape manifolds. In: Pattern recognition. GCPR, Lecture notes in computer science. Springer International Publishing, Cham, pp 196–208

Download references

Author information

Authors and Affiliations

  1. Computer Vision and Remote Sensing, TU Berlin, Sekr. MAR 6-5, Marchstraße 23, 10587, Berlin, Germany

    Jens Krenzin & Olaf Hellwich

Authors
  1. Jens Krenzin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Olaf Hellwich
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jens Krenzin.

Additional information

This paper is an extended version of the paper Reduction of Point Cloud Artifacts Using Shape Priors Estimated with the Gaussian Process Latent Variable Model published at GCPR 2016.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Krenzin, J., Hellwich, O. 3D Reconstruction Supported by Gaussian Process Latent Variable Model Shape Priors. PFG 85, 97–112 (2017). https://doi.org/10.1007/s41064-017-0009-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41064-017-0009-0

Keywords

Search

Navigation