Skip to main content
SpringerLink
Log in

Objects Can Move: 3D Change Detection by Geometric Transformation Consistency

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13693))

Included in the following conference series:

  • 2746 Accesses

Abstract

AR/VR applications and robots need to know when the scene has changed. An example is when objects are moved, added, or removed from the scene. We propose a 3D object discovery method that is based only on scene changes. Our method does not need to encode any assumptions about what is an object, but rather discovers objects by exploiting their coherent move. Changes are initially detected as differences in the depth maps and segmented as objects if they undergo rigid motions. A graph cut optimization propagates the changing labels to geometrically consistent regions. Experiments show that our method achieves state-of-the-art performance on the 3RScan dataset against competitive baselines. The source code of our method can be found at https://github.com/katadam/ObjectsCanMove.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ambrus, R., Folkesson, J., Jensfelt, P.: Unsupervised object segmentation through change detection in a long term autonomy scenario. In: 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), pp. 1181–1187. IEEE (2016)

    Google Scholar 

  2. Armeni, I., et al.: 3d semantic parsing of large-scale indoor spaces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1534–1543 (2016)

    Google Scholar 

  3. Bai, X., et al.: PointDSC: robust point cloud registration using deep spatial consistency. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15859–15869 (2021)

    Google Scholar 

  4. Barron, J.T.: A generalization of Otsu’s method and minimum error thresholding. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12350, pp. 455–470. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58558-7_27

    Chapter  Google Scholar 

  5. Bolles, R.C., Fischler, M.A.: A RANSAC-based approach to model fitting and its application to finding cylinders in range data. In: IJCAI, vol. 1981, pp. 637–643. Citeseer (1981)

    Google Scholar 

  6. Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)

    Article  MATH  Google Scholar 

  7. Brasch, N., Bozic, A., Lallemand, J., Tombari, F.: Semantic monocular slam for highly dynamic environments. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 393–400. IEEE (2018)

    Google Scholar 

  8. Cui, L., Ma, C.: SOF-SLAM: a semantic visual slam for dynamic environments. IEEE Access 7, 166528–166539 (2019)

    Article  Google Scholar 

  9. Fehr, M., et al.: TSDF-based change detection for consistent long-term dense reconstruction and dynamic object discovery. In: 2017 IEEE International Conference on Robotics and automation (ICRA), pp. 5237–5244. IEEE (2017)

    Google Scholar 

  10. Finman, R., Whelan, T., Kaess, M., Leonard, J.J.: Toward lifelong object segmentation from change detection in dense RGB-D maps. In: 2013 European Conference on Mobile Robots, pp. 178–185. IEEE (2013)

    Google Scholar 

  11. Glocker, B., Izadi, S., Shotton, J., Criminisi, A.: Real-time RGB-D camera relocalization. In: 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 173–179. IEEE (2013)

    Google Scholar 

  12. Halber, M., Shi, Y., Xu, K., Funkhouser, T.: Rescan: inductive instance segmentation for indoor RGBD scans. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2541–2550 (2019)

    Google Scholar 

  13. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)

    Google Scholar 

  14. Herbst, E., Henry, P., Ren, X., Fox, D.: Toward object discovery and modeling via 3-d scene comparison. In: 2011 IEEE International Conference on Robotics and Automation, pp. 2623–2629. IEEE (2011)

    Google Scholar 

  15. Johnson, J., Douze, M., Jégou, H.: Billion-scale similarity search with GPUs. arXiv preprint arXiv:1702.08734 (2017)

  16. Karpathy, A., Miller, S., Fei-Fei, L.: Object discovery in 3d scenes via shape analysis. In: 2013 IEEE International Conference on Robotics and Automation, pp. 2088–2095. IEEE (2013)

    Google Scholar 

  17. Katsura, U., Matsumoto, K., Kawamura, A., Ishigami, T., Okada, T., Kurazume, R.: Spatial change detection using normal distributions transform. ROBOMECH J. 6(1), 1–13 (2019). https://doi.org/10.1186/s40648-019-0148-8

    Article  Google Scholar 

  18. Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951)

    Article  MathSciNet  MATH  Google Scholar 

  19. Landrieu, L., Obozinski, G.: Cut pursuit: Fast algorithms to learn piecewise constant functions on general weighted graphs. SIAM J. Imag. Sci. 10(4), 1724–1766 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  20. Langer, E., Patten, T., Vincze, M.: Robust and efficient object change detection by combining global semantic information and local geometric verification. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 8453–8460. IEEE (2020)

    Google Scholar 

  21. Langer, E., Ridder, B., Cashmore, M., Magazzeni, D., Zillich, M., Vincze, M.: On-the-fly detection of novel objects in indoor environments. In: 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 900–907. IEEE (2017)

    Google Scholar 

  22. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

  23. Mason, J., Marthi, B.: An object-based semantic world model for long-term change detection and semantic querying. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3851–3858. IEEE (2012)

    Google Scholar 

  24. Palazzolo, E., Stachniss, C.: Change detection in 3d models based on camera images. In: 9th Workshop on Planning, Perception and Navigation for Intelligent Vehicles at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2017)

    Google Scholar 

  25. Palazzolo, E., Stachniss, C.: Fast image-based geometric change detection given a 3d model. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 6308–6315. IEEE (2018)

    Google Scholar 

  26. Palma, G., Cignoni, P., Boubekeur, T., Scopigno, R.: Detection of geometric temporal changes in point clouds. In: Computer Graphics Forum, vol. 35, pp. 33–45. Wiley Online Library (2016)

    Google Scholar 

  27. Papon, J., Abramov, A., Schoeler, M., Worgotter, F.: Voxel cloud connectivity segmentation-supervoxels for point clouds. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2027–2034 (2013)

    Google Scholar 

  28. Phan, A.V., Le Nguyen, M., Nguyen, Y.L.H., Bui, L.T.: DGCNN: a convolutional neural network over large-scale labeled graphs. Neural Netw. 108, 533–543 (2018)

    Article  Google Scholar 

  29. Runz, M., Buffier, M., Agapito, L.: MaskFusion: real-time recognition, tracking and reconstruction of multiple moving objects. In: 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 10–20. IEEE (2018)

    Google Scholar 

  30. Rusu, R.B., Blodow, N., Beetz, M.: Fast point feature histograms (FPFH) for 3d registration. In: 2009 IEEE International Conference on Robotics and Automation, pp. 3212–3217. IEEE (2009)

    Google Scholar 

  31. Steinhauser, D., Ruepp, O., Burschka, D.: Motion segmentation and scene classification from 3d lidar data. In: 2008 IEEE Intelligent Vehicles Symposium, pp. 398–403. IEEE (2008)

    Google Scholar 

  32. Taneja, A., Ballan, L., Pollefeys, M.: Image based detection of geometric changes in urban environments. In: 2011 International Conference on Computer Vision, pp. 2336–2343. IEEE (2011)

    Google Scholar 

  33. Taneja, A., Ballan, L., Pollefeys, M.: City-scale change detection in cadastral 3d models using images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 113–120 (2013)

    Google Scholar 

  34. Tombari, F., Salti, S., Di Stefano, L.: Unique signatures of histograms for local surface description. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6313, pp. 356–369. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15558-1_26

    Chapter  Google Scholar 

  35. Ulusoy, A.O., Mundy, J.L.: Image-based 4-d reconstruction using 3-d change detection. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 31–45. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10578-9_3

    Chapter  Google Scholar 

  36. Wald, J., Avetisyan, A., Navab, N., Tombari, F., Nießner, M.: Rio: 3d object instance re-localization in changing indoor environments. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7658–7667 (2019)

    Google Scholar 

  37. Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph CNN for learning on point clouds. ACM Trans. Graph. (TOG) 38(5), 1–12 (2019)

    Article  Google Scholar 

  38. Wong, Y.S., Li, C., Niessner, M., Mitra, N.J.: RigidFusion: RGB-D scene reconstruction with rigidly-moving objects. Comput. Graph. Forum 40(2), 511–522 (2021)

    Article  Google Scholar 

  39. Xiao, W., Vallet, B., Paparoditis, N.: Change detection in 3d point clouds acquired by a mobile mapping system. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 1(2), 331–336 (2013)

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by projects EU RDF IMPACT No. CZ.02.1.01/0.0/0.0/15_003/0000468, EU H2020 ARtwin No. 856994 and the EU Horizon 2020 project RICAIP (grant agreement No 857306).

Author information

Authors and Affiliations

  1. Czech Institute of Informatics, Robotics and Cybernetics, CTU in Prague, Prague, Czechia

    Aikaterini Adam, Torsten Sattler & Tomas Pajdla

  2. National Technical University of Athens, Athens, Greece

    Aikaterini Adam & Konstantinos Karantzalos

Authors
  1. Aikaterini Adam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Torsten Sattler
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Konstantinos Karantzalos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tomas Pajdla
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aikaterini Adam .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Shai Avidan

  2. University College London, London, UK

    Gabriel Brostow

  3. Google AI, Accra, Ghana

    Moustapha Cissé

  4. University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Facebook (United States), Menlo Park, CA, USA

    Tal Hassner

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 4558 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Adam, A., Sattler, T., Karantzalos, K., Pajdla, T. (2022). Objects Can Move: 3D Change Detection by Geometric Transformation Consistency. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13693. Springer, Cham. https://doi.org/10.1007/978-3-031-19827-4_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19827-4_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19826-7

  • Online ISBN: 978-3-031-19827-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation