Skip to main content

Segmentation by Continuous Latent Semantic Analysis for Multi-structure Model Fitting

Abstract

In this paper, we propose a novel continuous latent semantic analysis fitting method, to efficiently and effectively estimate the parameters of model instances in data, based on latent semantic analysis and continuous preference analysis. Specifically, we construct a new latent semantic space (LSS): where inliers of different model instances are mapped into several independent directions, while gross outliers are distributed close to the origin of LSS. After that, we analyze the data distribution to effectively remove gross outliers in LSS, and propose an improved clustering algorithm to segment the remaining data points. On the one hand, the proposed fitting method is able to achieve excellent fitting results; due to the effective continuous preference analysis in LSS. On the other hand, the proposed method can efficiently obtain final fitting results due to the dimensionality reduction in LSS. Experimental results on both synthetic data and real images demonstrate that the proposed method achieves significant superiority over several state-of-the-art model fitting methods on both fitting accuracy and computational speed.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Notes

  1. 1.

    That is, among the generated model hypotheses, there is at least one model hypothesis (generated by a minimal subset) corresponding to a true model instance in data.

  2. 2.

    CPA is a soft generalization of the binary preference analysis.

  3. 3.

    The elements of the preference set \(\varvec{PS}_i\) are computed according to the preference function \(f(\cdot )\) in Eq. (1), based on the corresponding data point \(x_i\) and all the generated model hypotheses. Similarly, the elements of \(\varvec{PS}_j\) can be computed.

  4. 4.

    For MSHF, we use the code provided by the authors. For RansaCov, T-linkage and RPA, we download the codes from the following website: http://www.diegm.uniud.it/fusiello/demo.

  5. 5.

    For the CPU time, we exclude the time used for sampling minimal subsets and generating model hypotheses for all the competing fitting methods in all the following experiments.

  6. 6.

    http://cs.adelaide.edu.au/~hwong/doku.php?id=data

References

  1. Amayo, P., Pinis, P., Paz, L. M., & Newman, P. (2018). Geometric multi-model fitting with a convex relaxation algorithm. In Proceedings of the IEEE conference on computer vision and pattern recognition, 8138–8146.

  2. Arthur, D., & Vassilvitskii, S. (2007). K-means++: The advantages of careful seeding. In Proceedings of the eighteenth annual ACM-SIAM symposium on discrete algorithms, 1027–1035.

  3. Barath, D., Matas, J., & Noskova, J. (2019). Magsac: Marginalizing sample consensus. In Proceedings of the IEEE conference on computer vision and pattern recognition, 10197–10205.

  4. Barath, D., Noskova, J., Ivashechkin, M., & Matas, J. (2020). Magsac++, a fast, reliable and accurate robust estimator. In Proceedings of the IEEE conference on computer vision and pattern recognition, 1304–1312.

  5. Chin, T. J., Wang, H., & Suter, D. (2009). Robust fitting of multiple structures: The statistical learning approach. In Proceedings of the IEEE international conference on computer vision, 413–420.

  6. Chin, T. J., Yu, J., & Suter, D. (2012). Accelerated hypothesis generation for multistructure data via preference analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(4), 625–638.

    Article  Google Scholar 

  7. Chum, O., & Matas, J. (2005). Matching with prosac-progressive sample consensus. In Proceedings of the IEEE conference on computer vision and pattern recognition, 220–226.

  8. Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6), 391–407.

    Article  Google Scholar 

  9. Ester, M., Kriegel, H. P., Sander, J., & Xu, X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. In IEEE Transactions on Pattern Analysis and Machine Intelligence pp. 226–231.

  10. Ferraz, L., Felip, R., Martínez, B., & Binefa, X. (2007). A density-based data reduction algorithm for robust estimators. In Proceedings of Iberian conference on pattern recognition and image analysis, 355–362.

  11. Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.

    MathSciNet  Article  Google Scholar 

  12. Isack, H., & Boykov, Y. (2012). Energy-based geometric multi-model fitting. International Journal of Computer Vision, 97(2), 123–147.

    Article  Google Scholar 

  13. Jiang, X., Ma, J., Jiang, J., & Guo, X. (2020). Robust feature matching using spatial clustering with heavy outliers. IEEE Transactions on Image Processing, 29, 736–746.

    MathSciNet  Article  Google Scholar 

  14. Kanazawa, Y., & Kawakami, H. (2004). Detection of planar regions with uncalibrated stereo using distributions of feature points. In Proceedings of British machine vision conference, 2701–2710.

  15. Le, H. M., Chin, T. J., Eriksson, A., Do, T. T., & Suter, D. (2019). Deterministic approximate methods for maximum consensus robust fitting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1, 1–14.

    Google Scholar 

  16. Lee, J. K., & Yoon, K. J. (2019). Joint estimation of camera orientation and vanishing points from an image sequence in a non-manhattan world. International Journal of Computer Vision, 127(10), 1426–1442.

    Article  Google Scholar 

  17. Ma, J., Jiang, X., Fan, A., Jiang, J., & Yan, J. (2021). Image matching from handcrafted to deep features: A survey. International Journal of Computer Vision, 129(1), 23–79.

    MathSciNet  Article  Google Scholar 

  18. Ma, J., Zhao, J., Jiang, J., Zhou, H., & Guo, X. (2019). Locality preserving matching. International Journal of Computer Vision, 127(5), 512–531.

    MathSciNet  Article  Google Scholar 

  19. Maaten, L. V. D., & Hinton, G. (2008). Visualizing data using t-sne. Journal of Machine Learning Research, 9(11), 2579–2605.

    MATH  Google Scholar 

  20. MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, 281–296.

  21. Magri, L., & Fusiello, A. (2014). T-linkage: A continuous relaxation of j-linkage for multi-model fitting. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3954–3961.

  22. Magri, L., & Fusiello, A. (2016). Multiple model fitting as a set coverage problem. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3318–3326.

  23. Magri, L., & Fusiello, A. (2017). Multiple structure recovery via robust preference analysis. Image and Vision Computing, 67, 1–15.

    Article  Google Scholar 

  24. Magri, L., & Fusiello, A. (2019). Fitting multiple heterogeneous models by multi-class cascaded t-linkage. In Proceedings of the IEEE conference on computer vision and pattern recognition, 7460–7468.

  25. Meer, P. (2020). Robust estimation techniques. Computer Vision: A Reference Guide, pp. 1–8.

  26. Mittal, S., Anand, S., & Meer, P. (2012). Generalized projection-based m-estimator. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(12), 2351–2364.

    Article  Google Scholar 

  27. Pham, T. T., Chin, T. J., Yu, J., & Suter, D. (2014). The random cluster model for robust geometric fitting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(8), 1658–1671.

    Article  Google Scholar 

  28. Purkait, P., Chin, T. J., Ackermann, H., & Suter, D. (2017). Clustering with hypergraphs: The case for large hyperedges. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(9), 1697–1711.

    Article  Google Scholar 

  29. Rodriguez, A., & Laio, A. (2014). Clustering by fast search and find of density peaks. Science, 344(6191), 1492–1496.

    Article  Google Scholar 

  30. Tanimoto, T. (1957). An elementary mathematical theory of classification and prediction. IBM: Internal technical report.

  31. Tennakoon, R., Bab-Hadiashar, A., Cao, Z., Hoseinnezhad, R., & Suter, D. (2016). Robust model fitting using higher than minimal subset sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2), 350–362.

    Article  Google Scholar 

  32. Toldo, R., & Fusiello, A. (2008). Robust multiple structures estimation with j-linkage. In Proceedings of European conference on computer vision, 537–547.

  33. Torr, P. H., & Murray, D. W. (1997). The development and comparison of robust methods for estimating the fundamental matrix. International Journal of Computer Vision, 24(3), 271–300.

    Article  Google Scholar 

  34. Tran, Q. H., Chin, T. J., Chojnacki, W., & Suter, D. (2014). Sampling minimal subsets with large spans for robust estimation. International Journal of Computer Vision, 106(1), 93–112.

    MathSciNet  Article  Google Scholar 

  35. Wang, H., Chin, T. J., & Suter, D. (2012). Simultaneously fitting and segmenting multiple-structure data with outliers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(6), 1177–1192.

    Article  Google Scholar 

  36. Wang, H., Xiao, G., Yan, Y., & Suter, D. (2019). Searching for representative modes on hypergraphs for robust geometric model fitting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(3), 697–711.

    Article  Google Scholar 

  37. Wong, H. S., Chin, T. J., Yu, J., & Suter, D. (2011). Dynamic and hierarchical multi-structure geometric model fitting. In Proceedings of the IEEE international conference on computer vision, 1044–1051.

  38. Wu, Y., Li, Y., & Hu, Z. (2008). Detecting and handling unreliable points for camera parameter estimation. International Journal of Computer Vision, 79(2), 209–223.

    Article  Google Scholar 

  39. Xiao, G., Wang, H., Lai, T., & Suter, D. (2016). Hypergraph modelling for geometric model fitting. Pattern Recognition, 60(1), 748–760.

    Article  Google Scholar 

  40. Xiao, G., Wang, H., Yan, Y., & Suter, D. (2019). Superpixel-guided two-view deterministic geometric model fitting. International Journal of Computer Vision, 127(4), 323–329.

    Article  Google Scholar 

  41. Xu, L., Oja, E., & Kultanen, P. (1990). A new curve detection method: randomized hough transform (rht). Pattern Recognition Letter, 11(5), 331–338.

    Article  Google Scholar 

  42. Xu, X., Cheong, L. F., & Li, Z. (2019). 3d rigid motion segmentation with mixed and unknown number of models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1, 1–14.

    Google Scholar 

  43. Yang, X., & Meer, P. M. J. (2020). A new approach to robust estimation of parametric structures. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1, 1–14.

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grants 62072223, 61872307, and by the Natural Science Foundation of Fujian Province under Grants 2020J01829.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Hanzi Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Communicated by Zhouchen Lin.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Xiao, G., Wang, H., Ma, J. et al. Segmentation by Continuous Latent Semantic Analysis for Multi-structure Model Fitting. Int J Comput Vis 129, 2034–2056 (2021). https://doi.org/10.1007/s11263-021-01468-6

Download citation

Keywords

  • Latent semantic analysis
  • Preference analysis
  • Geometric model fitting
  • Multi-structure data