International Journal of Computer Vision

, Volume 92, Issue 1, pp 32–52 | Cite as

Model-Based Multiple Rigid Object Detection and Registration in Unstructured Range Data

  • Dirk BreitenreicherEmail author
  • Christoph Schnörr


We present a two-stage approach to the simultaneous detection and registration of multiple instances of industrial 3D objects in unstructured noisy range data. The first non-local processing stage takes all data into account and computes in parallel multiple localizations of the object along with rough pose estimates. The second stage computes accurate registrations for all detected object instances individually by using local optimization.

Both stages are designed using advanced numerical techniques, large-scale sparse convex programming, and second-order geometric optimization on the Euclidean manifold, respectively. They complement each other in that conflicting interpretations are resolved through non-local convex processing, followed by accurate non-convex local optimization based on sufficiently good initializations.

As input data a sparse point sample of the object’s surface is required exclusively. Our experiments focus on industrial applications where multiple 3D object instances are randomly assembled in a bin, occlude each other, and unstructured noisy range data is acquired by a laser scanning device.


Range data Point set registration Multiple object detection Geometric optimization Sparse convex programming 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Adler, R. L., Dedieu, J.-P., Margulies, J. Y., Martens, M., & Shub, M. (2002). Newton’s method on Riemannian manifolds and a geometric model for the human spine. IMA Journal of Numerical Analysis, 22(3), 359–390. MathSciNetzbMATHCrossRefGoogle Scholar
  2. Ballard, D. H. (1981). Generalizing the Hough transform to detect arbitrary shapes. Pattern Recognition, 13, 111–122. zbMATHCrossRefGoogle Scholar
  3. Belongie, S., Malik, J., & Puzicha, J. (2002). Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 509–522. CrossRefGoogle Scholar
  4. Benhimane, S., & Malis, E. (2006). A new approach to vision-based robot control with omni-directional cameras. In Proc. of IEEE int. conf. on robotics and automation (pp. 526–531). Google Scholar
  5. Bennett, K., & Parrado-Hernández, E. (2006). The interplay of optimization and machine learning research. Journal of Machine Learning Research, 7, 1265–1281. Google Scholar
  6. Bertsimas, D., & Weismantel, R. (2005). Optimization over Integers. Dynamic Ideas. ISBN-13: 978-0975914625 Google Scholar
  7. Besl, P. J., & Jain, R. C. (1985). Three-dimensional object recognition. ACM Computing Surveys, 17(1), 75–145. CrossRefGoogle Scholar
  8. Besl, P. J., & McKay, N. D. (1992). A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14, 239–256. CrossRefGoogle Scholar
  9. Birgin, E. G., Martínez, J. M., & Raydan, M. (2000). Nonmonotone spectral projected gradient methods on convex sets. SIAM Journal of Optimization, 10, 1196–1211. zbMATHCrossRefGoogle Scholar
  10. Boykov, Y., & Funka-Lea, G. (2006). Graph cuts and efficient n-d image segmentation. International Journal of Computer Vision, 70(2), 109–131. CrossRefGoogle Scholar
  11. Breitenreicher, D., & Schnörr, C. (2009). Intrinsic second-order geometric optimization for robust point set registration without correspondence. In 7th int. workshop on energy minimization methods in comp. vision and pattern recogn. Google Scholar
  12. Breitenreicher, D., & Schnörr, C. (2009). Robust 3D object registration without explicit correspondence using geometric integration. Machine Vision and Applications. doi: 10.1007/s00138-009-0227-6 Google Scholar
  13. Chan, T., Esedoglu, S., & Nikolova, M. (2006). Algorithms for finding global minimizers of image segmentation and denoising models. SIAM Journal on Applied Mathematics, 66(5), 1632–1648. MathSciNetzbMATHCrossRefGoogle Scholar
  14. Chen, S., Donoho, D., & Saunders, M. (2001). Atomic decomposition by basis pursuit. SIAM Review, 43(1), 129–159. MathSciNetzbMATHCrossRefGoogle Scholar
  15. Chin, R. T., & Dyer, C. R. (1986). Model-based recognition in robot vision. ACM Computing Surveys, 18(1), 67–108. CrossRefGoogle Scholar
  16. Chua, C. S., & Jarvis, R. (1997). Point signatures: A new representation for 3D object recognition. International Journal of Computer Vision, 25(1), 63–85. CrossRefGoogle Scholar
  17. Chui, H., & Rangarajan, A. (2000). A feature registration framework using mixture models. In IEEE workshop math. methods in biomed. image anal. (pp. 190–197). Google Scholar
  18. Cover, T., & Thomas, J. (1991). Elements of information theory. New York: Wiley. zbMATHCrossRefGoogle Scholar
  19. do Carmo, M. P. (1992). Riemannian geometry. Cambridge: Birkhäuser Boston. zbMATHGoogle Scholar
  20. Donoho, D. (2006). Compressed sensing. IEEE Transactions on Information Theory, 52, 1289–1306. MathSciNetCrossRefGoogle Scholar
  21. Donoho, D. L., Elad, M., & Temlyakov, V. N. (2006). Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Transactions on Information Theory, 52, 6–18. MathSciNetCrossRefGoogle Scholar
  22. Drummond, T., & Cipolla, R. (2002). Real-time tracking of complex structures with on-line camera calibration. Image Vision Computing, 20(5–6), 427–433. CrossRefGoogle Scholar
  23. Edelman, A., Arias, T. A., & Smith, S. T. (1999). The geometry of algorithms with orthogonality constraints. SIAM Journal on Matrix Analysis and Applications, 20, 303–353. MathSciNetCrossRefGoogle Scholar
  24. Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395. MathSciNetCrossRefGoogle Scholar
  25. Fitzgibbon, A. W. (2003). Robust registration of 2D and 3D point sets. Image Vision Computing, 21(13–14), 1145–1153. CrossRefGoogle Scholar
  26. Frome, A., Huber, D., Kolluri, R., Bulow, T., & Malik, J. (2004). Recognizing objects in range data using regional point descriptors. In Proc. Europ. conf. comp. vision. Google Scholar
  27. Gelfand, N., Mitra, N. J., Guibas, L. J., & Pottmann, H. (2005). Robust global registration. In Proc. symp. geom. processing. Google Scholar
  28. Golub, G., & Van Loan, C. (1996). Matrix computations (3rd edn.). Baltimore: The John Hopkins University Press. zbMATHGoogle Scholar
  29. Greenspan, M. (2002). Geometric probing of dense range data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 495–508. CrossRefGoogle Scholar
  30. Hartley, R. I., & Kahl, F. (2009). Global optimization through rotation space search. International Journal of Computer Vision, 82(1), 64–79. CrossRefGoogle Scholar
  31. Jian, B., & Vemuri, B. C. (2005). A robust algorithm for point set registration using mixture of Gaussians. In Proc. int. conf. comp. vision. Google Scholar
  32. Johnson, A., & Hebert, M. (1999). Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21, 433–449. CrossRefGoogle Scholar
  33. Krishnan, S., Lee, P. Y., Moore, J. B., & Venkatasubramanian, S. (2007). Optimisation-on-a-manifold for global registration of multiple 3D point sets. International Journal of Intelligent Systems Technologies and Applications, 3(3/4), 319–340. CrossRefGoogle Scholar
  34. Lavva, I., Hameiri, E., & Shimshoni, I. (2008). Robust methods for geometric primitive recovery and estimation from range images. IEEE Transactions on Systems, Man and Cybernetics, Part B, Cybernetics, 38(3), 826–845. CrossRefGoogle Scholar
  35. Li, H., & Hartley, R. (2007). The 3D-3D registration problem revisited. In Proc. int. conf. comp. vision. Google Scholar
  36. Matsushima, Y. (1972). Differentiable manifolds. New York: Dekker. zbMATHGoogle Scholar
  37. Mitra, N. J., Gelfand, N., Pottmann, H., & Guibas, L. (2004). Registration of point cloud data from a geometric optimization perspective. In Proc. sym. geom. process. Google Scholar
  38. Natarajan, K. (1995). Sparse approximate solutions to linear systems. SIAM Journal on Computing, 24, 227–234. MathSciNetzbMATHCrossRefGoogle Scholar
  39. Nesterov, Y. (2005). Smooth minimization of non-smooth functions. Mathematical Programming, 103(1), 127–152. MathSciNetzbMATHCrossRefGoogle Scholar
  40. Olson, C. (1997). Efficient pose clustering using a randomized algorithm. International Journal of Computer Vision, 23(2), 131–147. CrossRefGoogle Scholar
  41. Olsson, C., Kahl, F., & Oskarsson, M. (2009). Branch-and-bound methods for Euclidean registration problems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(5), 783–794. CrossRefGoogle Scholar
  42. Pottmann, H., Huang, Q.-X., Yang, Y.-L., & Hu, S.-M. (2006). Geometry and convergence analysis of algorithms for registration of 3D shapes. International Journal of Computer Vision, 67(3), 277–296. CrossRefGoogle Scholar
  43. Rangarajan, A., Chui, H., & Bookstein, F. L. (1997). The softassign procrustes matching algorithm. In Proc. int. conf. inf. process. med. imaging. Google Scholar
  44. Reyes, L., Medioni, G., & Bayro, E. (2007). Registration of 3D points using geometric algebra and tensor voting. International Journal of Computer Vision, 75(3), 351–369. CrossRefGoogle Scholar
  45. Rockafellar, R., & Wets, R.-B. (1998). Grundlehren der math. Wissenschaften : Vol. 317. Variational analysis. Berlin: Springer. zbMATHCrossRefGoogle Scholar
  46. Rusinkiewicz, S., & Levoy, M. (2001). Efficient variants of the ICP algorithm. In Proc. 3rd int. conf. on 3D digital imaging and modeling (pp. 145–152). CrossRefGoogle Scholar
  47. Salvi, J., Matabosch, C., Fofi, D., & Forest, J. (2007). A review of recent range image registration methods with accuracy evaluation. Image and Vision Computing, 25, 578–596. CrossRefGoogle Scholar
  48. Shang, L., & Greenspan, M. (2007). Pose determination by potential well space embedding. In Proc. 6th int. conf. 3-D digital imaging and modeling (pp. 297–304). Los Alamitos: IEEE Comput. Soc.. Google Scholar
  49. Shi, Q., Xi, N., Chen, Y., & Sheng, W. (2006). Registration of point clouds for 3D shape inspection. In Int. conf. intell. robots syst. Google Scholar
  50. Subbarao, R., & Meer, P. (2009). Nonlinear mean shift over Riemannian manifolds. International Journal of Computer Vision, 84, 1–20. CrossRefGoogle Scholar
  51. Taylor, C. J., & Kriegman, D. J. (1994). Minimization on the Lie group SO(3) and related manifolds. Technical Report 9405, Center for Systems Science, Dept. of Electrical Engineering, Yale University. Google Scholar
  52. Teboulle, M. (2007). A unified continuous optimization framework for center-based clustering methods. The Journal of Machine Learning Research, 8, 65–102. MathSciNetGoogle Scholar
  53. Tropp, J. A. (2006). Just relax: Convex programming methods for identifying sparse signals in noise. IEEE Transactions on Information Theory, 52, 1030–1051. MathSciNetCrossRefGoogle Scholar
  54. Tsin, Y., & Kanade, T. (2004). A correlation-based approach to robust point set registration. In Proc. Europ. conf. comp. vision (Vol. III, pp. 558–569). Google Scholar
  55. Wainwright, M., & Jordan, M. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning, 1(1–2), 1–305. zbMATHGoogle Scholar
  56. Wang, F., Vemuri, B. C., Rangarajan, A., Schmalfuss, I. M., & Eisenschenk, S. J. (2006). Simultaneous nonrigid registration of multiple point sets and atlas construction. In Proc. Europ. conf. comp. vision. Google Scholar
  57. Wiedemann, C., Ulrich, M., & Steger, C. (2008). Recognition and tracking of 3D objects. In Pattern recogn. Google Scholar
  58. Wolfson, H. J., & Rigoutsos, I. (1997). Geometric hashing: An overview. Computing in Science and Engineering, 4, 10–21. Google Scholar
  59. Zhu, L., Barhak, J., Shrivatsan, V., & Katz, R. (2007). Efficient registration for precision inspection of free-form surfaces. International Journal of Advanced Manufacturing Technology, 32, 505–515. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Image & Pattern Analysis Group (IPA), Heidelberg Collaboratory for Image Processing (HCI)University of HeidelbergHeidelbergGermany

Personalised recommendations