Skip to main content

Advertisement

SpringerLink
Log in

A robust registration algorithm based on salient object detection

  • 1168: Deep Pattern Discovery for Big Multimedia Data
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Point cloud registration plays an important role in 3D computer vision. A challenge in this field is the presence of small salient objects with huge flat backgrounds in point clouds, which may result in poor registration. Despite substantial approaches for point cloud registration have been proposed, few attempts have been made to address this problem. In this paper, we present an approach which fully leverages not only geometric information but also texture information presented by RGB images to tackle with this problem. To mitigate the influence of background, we introduce a superior 2D salient object detection method to highlight the role of the salient objects. The color supported generalized iterative closest points algorithm is the state-of-the-art approach in iterative closest points (ICP) variations, which can efficiently exploit the color information. However, it cannot deal with the mentioned problem. On this basis, we further propose a joint objective to align both salient color points and background points. The registration and reconstruction experiments demonstrate the robustness and accuracy of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Andreas N, Lingemann K, Hertzberg J (2007) Cached k-d tree search for ICP algorithms. In: IEEE Sixth international conference on 3-D digital imaging and modeling, pp 21–23

  2. Besl PJ, Hasegawa O (1992) A method for registration 3-D shapes. IEEE Trans Pattern Anal Mach Intell 14(2):193–200

    Article  Google Scholar 

  3. Bing J, Vemuri BC (2005) A robust algorithm for point set registration using mixture of gaussians. IEEE International Conference on Computer Vision 2:1246–1251

    Google Scholar 

  4. Breitenreicher D, Schnörr C (2011) Model-Based multiple rigid object detection and registration in unstructured range data. Int J Comput Vis 92(1):32–52

    Article  Google Scholar 

  5. Chen Y, Medioni G (1991) Object modeling by registration of multiple range images. Image Vis Comput 10(3):145–155

    Article  Google Scholar 

  6. Chen Y, Medioni G (1992) Object modeling by registration of multiple range images. Image Vis Comput 10(3):145–155

    Article  Google Scholar 

  7. Chen Y, Lin Z, Zhao X, Wang G, Gu Y (2017) Deep learning-based classification of hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 7(6):2094–2107

    Article  Google Scholar 

  8. Christopher C, Wei D, Vladlen K (2020) Deep global registration. In: IEEE conference on computer vision and pattern recognition, pp 2514–2523

  9. Dai A, Nießner M, Zollhöfer M, Izadi S, Theobalt C (2017) Bundlefusion: Real-timeglobally consistent 3D reconstruction using on-the-fly surface reintegration. ACM Trans Graph 36(3):76–95

    Article  Google Scholar 

  10. Druon S, Aldon MJ, Crosnier A (2007) Color constrained ICP for registration of large unstructured 3D color data sets. In: IEEE international conference on information acquisition, pp 249–255

  11. Du S, Liu J, Zhang C, Zhu J, Li K (2015) Probability iterative closest point algorithm for m-D point set registration with noise. Neurocomputing 157:187–198

    Article  Google Scholar 

  12. Du S, Xu G, Zhang S, Zhang X, Gao Y, Chen B (2020) Robust rigid registration algorithm based on pointwise correspondence and correntropy. Pattern Recognition Letters 132:91–98

    Article  Google Scholar 

  13. El-Gamal EZA, Elmogy M, Atwan A (2016) Current trends in medical image registration and fusion. Egyptian Informatics Journal 17(1):99–124

    Article  Google Scholar 

  14. Ferrante E, Paragios N (2017) Slice-to-volume medical image registration: a survey. Medical Image Analysis 39:101–123

    Article  Google Scholar 

  15. Hou Q, Cheng M, Hu X, Borji A, Tu Z, Torr PHS (2019) Deeply supervised salient object detection with short connections. IEEE Trans Pattern Anal Mach Intell 41(4):815–828

    Article  Google Scholar 

  16. Itti L, Koch C, Niebur E (2002) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  17. Jian B, Vemuri BC (2011) Robust point set registration using Gaussian Mixture Models. IEEE Trans Pattern Anal Mach Intell 33(8):1633–1645

    Article  Google Scholar 

  18. Klein DA, Frintrop S (2011) Center-surround divergence of feature statistics for salient object detection. In: IEEE international conference on computer vision, pp 2214–2219

  19. Korn M, Holzkothen M, Pauli J (2014) Color supported generalized-ICP. In: International conference on computer vision theory and applications, pp 592–599

  20. Lee G, Tai YW, Kim J (2016) Deep saliency with encoded low level distance map and high level features. In: IEEE conference on computer vision and pattern recognition, pp 660–668

  21. Ma G, Chen C, Li S, Peng C, Hao A, Qin H (2020) Salient object detection via multiple instance joint re-learning. IEEE Transactions on Multimedia 22(2):324–336

    Article  Google Scholar 

  22. Men H, Gebre B, Pochiraju K (2011) Color point cloud registration with 4D ICP algorithm. In: IEEE international conference on robotics and automation, pp 1511–1516

  23. Myronenko A, Song X (2009) Point set registration: Coherent point drift. IEEE Trans Pattern Anal Mach Intell 32(12):2262–2275

    Article  Google Scholar 

  24. Park J, Zhou QY, Koltun V (2017) Colored point cloud registration revisited. In: IEEE international conference on computer vision, pp 143–152

  25. Paschos G (2001) Perceptually uniform color spaces for color texture analysis: an empirical evaluation. IEEE Trans Image Process 10(6):932–937

    Article  Google Scholar 

  26. Phillips JM, Liu R, Tomasi C (2006) Outlier robust ICP for minimizing fractional RMSD. In: Sixth international conference on 3-D digital imaging and modeling, pp 427–434

  27. Ren MJ, Sun LJ, Liu MY, Cheung CF, Yin YH (2017) A reconstruction-registration integrated data fusion method for measurement of multiscaled complex surfaces. IEEE Trans Instrum Meas 66(99):414–423

    Article  Google Scholar 

  28. Segal A, Hähnel D, Thrun S (2009) Generalized-ICP. Robotics: Science and Systems 2(4):435–442

    Google Scholar 

  29. Wan T, Du SY, Cui WT, Yao RZ, Ge YY, Li C, Gao Y, Zheng NN (2021) RGB-D point cloud registration based on salient object detection. IEEE Trans Neural Netw Learn Syst, pp 1–13

  30. Wang Y, Solomon MJ (2019) Deep closest point: Learning representations for point cloud registration. In: IEEE international conference on computer vision, pp 3523–3532

  31. Wang L, Lu H, Ruan X, Yang MH (2015) Deep networks for saliency detection via local estimation and global search. In: IEEE conference on computer vision and pattern recognition, pp 3183–3192

  32. Xie S, Tu Z (2017) Holistically-nested edge detection. Int J Comput Vis 125(5):3–18

    Article  MathSciNet  Google Scholar 

  33. Yang J, Li H, Jia Y (2013) Go-ICP: Solving 3D registration efficiently and globally optimally. In: IEEE international conference on computer vision, pp 1457–1464

  34. Ye M, Wang X, Yang R, Ren L, Pollefeys M (2011) Accurate 3D pose estimation from a single depth image. In: IEEE transactions on pattern analysis and machine intelligence, pp 731–738

  35. Yew ZJ, Lee GH (2020) Rpm-net: Robust point matching using learned features. In: IEEE conference on computer vision and pattern recognition, pp 11824–11833

  36. Zhu L, Song J, Zhu X, Zhang C, Zhang S, Yuan X (2020) Adversarial learning based semantic correlation representation for cross-modal retrieval. IEEE MultiMedia 7(6):2094–2107

    Google Scholar 

  37. Zhu X, Zhang S, Hu R, Zhu Y, Song J (2018) Local and global structure preservation for robust unsupervised spectral feature selection. IEEE Trans Knowl Data Eng 30(3):517–529

    Article  Google Scholar 

  38. Zhao R, Ouyang W, Li H, Wang X (2015) Saliency detection by multi-context deep learning. In: IEEE conference on computer vision and pattern recognition, pp 1265–1274

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China under Grant Nos. 61790562, 61971343 and 62088102, the Fundamental Research Funds for the Central Universities under Grant No.xzy022020052 and the Clinical Research Center of Shaanxi Province for Dental and Maxillofacial Diseases, College of Stomatology, Xi’an Jiaotong University under Grant No. 2021YHJB04.

Author information

Authors and Affiliations

  1. Institute of Artificial Intelligence and Robotics, College of Artificial Intelligence, Xi’an Jiaotong University, Xi’an, Shaanxi Province, 710049, People’s Republic of China

    Runzhao Yao, Shaoyi Du, Teng Wan & Wenting Cui

  2. Clinical Research Center of Shaanxi Province for Dental and Maxillofacial Diseases, College of Stomatology, Xi’an Jiaotong University, Xi’an, Shaanxi Province, 710049, People’s Republic of China

    Runzhao Yao, Shaoyi Du, Teng Wan & Wenting Cui

  3. Shunan Academy of Artificial Intelligence, Ningbo, Zhejiang Province, 315000, People’s Republic of China

    Shaoyi Du

Authors
  1. Runzhao Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shaoyi Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Teng Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenting Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shaoyi Du.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yao, R., Du, S., Wan, T. et al. A robust registration algorithm based on salient object detection. Multimed Tools Appl 81, 34387–34400 (2022). https://doi.org/10.1007/s11042-022-13194-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13194-3

Keywords

Search

Navigation