Superpixel-Guided Two-View Deterministic Geometric Model Fitting

Abstract

Geometric model fitting is a fundamental research topic in computer vision and it aims to fit and segment multiple-structure data. In this paper, we propose a novel superpixel-guided two-view geometric model fitting method (called SDF), which can obtain reliable and consistent results for real images. Specifically, SDF includes three main parts: a deterministic sampling algorithm, a model hypothesis updating strategy and a novel model selection algorithm. The proposed deterministic sampling algorithm generates a set of initial model hypotheses according to the prior information of superpixels. Then the proposed updating strategy further improves the quality of model hypotheses. After that, by analyzing the properties of the updated model hypotheses, the proposed model selection algorithm extends the conventional “fit-and-remove” framework to estimate model instances in multiple-structure data. The three parts are tightly coupled to boost the performance of SDF in both speed and accuracy, and SDF has the deterministic nature. Experimental results show that the proposed SDF has significant advantages over several state-of-the-art fitting methods when it is applied to real images with single-structure and multiple-structure data.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Notes

  1. 1.

    We download the codes from their websites.

  2. 2.

    http://www.cse.usf.edu/~sarkar/BLOGS/.

  3. 3.

    http://www.robots.ox.ac.uk/~vgg/data/.

  4. 4.

    http://cs.adelaide.edu.au/.

References

  1. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., & Susstrunk, S. (2012). Slic superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11), 2274–2282.

    Article  Google Scholar 

  2. Brahmachari, A. S., & Sarkar, S. (2013). Hop-diffusion monte carlo for epipolar geometry estimation between very wide-baseline images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(3), 755–762.

    Article  Google Scholar 

  3. Chin, T. J., Purkait, P., Eriksson, A., & Suter, D. (2016). Efficient globally optimal consensus maximisation with tree search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1(1), 1–14.

    Google Scholar 

  4. Chin, T. J., Yu, J., & Suter, D. (2012). Accelerated hypothesis generation for multistructure data via preference analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(4), 625–638.

    Article  Google Scholar 

  5. Chum, O., & Matas, J. (2005). Matching with prosac-progressive sample consensus. In: Proceedings of IEEE conference on computer vision and pattern recognition (pp. 220–226).

  6. Chum, O., Matas, J., & Kittler, J. (2003). Locally optimized ransac. Pattern Recognition, 25(1), 236–243.

    Article  Google Scholar 

  7. Enqvist, O., Ask, E., Kahl, F., & Åström, K. (2015). Tractable algorithms for robust model estimation. International Journal of Computer Vision, 112(1), 115–129.

    MathSciNet  Article  MATH  Google Scholar 

  8. Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.

    MathSciNet  Article  Google Scholar 

  9. Fragoso, V., Sen, P., Rodriguez, S., & Turk, M. (2013). Evsac: Accelerating hypotheses generation by modeling matching scores with extreme value theory. In: Proceedings of IEEE international conference on computer vision (pp. 2472–2479).

  10. Fragoso, V., & Turk, M. (2013). Swigs: A swift guided sampling method. In: Proceedings IEEE conference on computer vision and pattern recognition (pp. 2770–2777).

  11. Fredriksson, J., Larsson, V., & Olsson, C. (2015). Practical robust two-view translation estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2684–2690).

  12. Hart, P. E., Nilsson, N. J., & Raphael, B. (1968). A formal basis for the heuristic determination of minimum cost paths. IEEE Transactions on Systems Science and Cybernetics, 4(2), 100–107.

    Article  Google Scholar 

  13. Isack, H., & Boykov, Y. (2014). Energy based multi-model fitting & matching for 3d reconstruction. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1146–1153).

  14. Kanazawa, Y., & Kawakami, H. (2004). Detection of planar regions with uncalibrated stereo using distributions of feature points. In: Proceedings of British machine vision conference (pp. 247–256).

  15. Lai, T., Wang, H., Yan, Y., Xiao, G., & Suter, D. (2017). Efficient guided hypothesis generation for multi-structure epipolar geometry estimation. Computer Vision and Image Understanding, 154, 152–165.

    Article  Google Scholar 

  16. Lee, K. H., & Lee, S. W. (2013). Deterministic fitting of multiple structures using iterative maxfs with inlier scale estimation. In: Proceedings of IEEE international conference on computer vision (pp. 41–48).

  17. Li, H. (2009). Consensus set maximization with guaranteed global optimality for robust geometry estimation. In: Proceedings of IEEE international conference on computer vision (pp. 1074–1080).

  18. Litman, R., Korman, S., Bronstein, A., & Avidan, S. (2015). Inverting ransac: Global model detection via inlier rate estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5243–5251).

  19. Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.

    MathSciNet  Article  Google Scholar 

  20. Magri, L., & Fusiello, A. (2014). T-linkage: A continuous relaxation of j-linkage for multi-model fitting. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3954–3961).

  21. Magri, L., & Fusiello, A. (2016). Multiple model fitting as a set coverage problem. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3318–3326).

  22. Mittal, S., Anand, S., & Meer, P. (2012). Generalized projection-based m-estimator. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(12), 2351–2364.

    Article  Google Scholar 

  23. Pham, T. T., Chin, T. J., Schindler, K., & Suter, D. (2014). Interacting geometric priors for robust multimodel fitting. IEEE Transactions on Image Processing, 23(10), 4601–4610.

    MathSciNet  Article  MATH  Google Scholar 

  24. Poling, B., & Lerman, G. (2014). A new approach to two-view motion segmentation using global dimension minimization. International Journal of Computer Vision, 108(3), 165–185.

    MathSciNet  Article  MATH  Google Scholar 

  25. Serradell, E., Özuysal, M., Lepetit, V., Fua, P., & Moreno-Noguer, F. (2010). Combining geometric and appearance priors for robust homography estimation. In: Proceedings of European conference on computer vision (pp. 58–72).

  26. Shen, J., Du, Y., Wang, W., & Li, X. (2014). Lazy random walks for superpixel segmentation. IEEE Transactions on Image Processing, 23(4), 1451–1462.

    MathSciNet  Article  MATH  Google Scholar 

  27. Tennakoon, R., Bab-Hadiashar, A., Cao, Z., Hoseinnezhad, R., & Suter, D. (2016). Robust model fitting using higher than minimal subset sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2), 350–361.

    Article  Google Scholar 

  28. Tordoff, B. J., & Murray, D. W. (2005). Guided-mlesac: Faster image transform estimation by using matching priors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(10), 1523–1535.

    Article  Google Scholar 

  29. Tran, Q. H., Chin, T. J., Chojnacki, W., & Suter, D. (2014). Sampling minimal subsets with large spans for robust estimation. International Journal of Computer Vision, 106(1), 93–112.

    MathSciNet  Article  MATH  Google Scholar 

  30. Wand, M., & Jones, M. (1994). Kernel smoothing. Boca Raton: CRC Press.

    Book  MATH  Google Scholar 

  31. Wang, H., Chin, T. J., & Suter, D. (2012). Simultaneously fitting and segmenting multiple-structure data with outliers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(6), 1177–1192.

    Article  Google Scholar 

  32. Wang, H., Xiao, G., Yan, Y., & Suter, D. (2015). Mode-seeking on hypergraphs for robust geometric model fitting. In: Proceedings of the IEEE international conference on computer vision (pp. 2902–2910).

  33. Wong, H. S., Chin, T. J., Yu, J., & Suter, D. (2011). Dynamic and hierarchical multi-structure geometric model fitting. In: Proceedings of the IEEE international conference on computer vision (pp. 1044–1051).

  34. Woodford, O. J., Pham, M. T., Maki, A., Perbet, F., & Stenger, B. (2014). Demisting the hough transform for 3d shape recognition and registration. International Journal of Computer Vision, 106(3), 332–341.

    Article  Google Scholar 

  35. Xiao, G., Wang, H., Lai, T., & Suter, D. (2016). Hypergraph modelling for geometric model fitting. Pattern Recognition, 60(1), 748–760.

    Article  Google Scholar 

  36. Xiao, G., Wang, H., Yan, Y., & Suter, D. (2016). Superpixel-based two-view deterministic fitting for multiple-structure data. In: Proceedings of European conference on computer vision (pp. 517–533).

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grants U1605252, 61702431, 61472334 and 61571379, by the China Postdoctoral Science Foundation 2017M620272 and by the Fujian Province Education-Science Project for Middle-aged and Young Teachers JAT170024. This work was carried out when David Suter was with The University of Adelaide. This work was partially supported by ARC Grant DP130102524.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Hanzi Wang.

Additional information

Communicated by J. Kosecka.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Xiao, G., Wang, H., Yan, Y. et al. Superpixel-Guided Two-View Deterministic Geometric Model Fitting. Int J Comput Vis 127, 323–339 (2019). https://doi.org/10.1007/s11263-018-1100-8

Download citation

Keywords

  • Model fitting
  • Superpixel
  • Deterministic algorithm
  • Multiple-structure data