A speedup scheme for MRF stereo using local label hierarchy

  • Sangsan Leelhapantu
  • Thanarat H. Chalidabhongse
Original Paper
  • 17 Downloads

Abstract

Random field formulation has proven to be a powerful framework for solving stereo correspondence problems because of its ability to intuitively incorporate global smoothness constraint with local matching costs. However, solving such problems for cases where large number of pixel variables and possible disparity labels are common can be impractical as the computational complexity grows fast with the number of labels. We proposed a speedup scheme using local label hierarchy in which we exploit characteristics of stereo vision problem to obtain a hierarchical energy minimization technique. In doing so, we give definitions and notations for local label hierarchy as well as approaches for label-wise grouping. We also generalize the definition of energy function to include sets of labels and present heuristics for assigning group potentials. Our approach builds different “local” hierarchy for each variable using information from the energy function which enables us to achieve better performance when compared to using the same hierarchy for every variable. The added processing steps have significantly less theoretical computational complexity than the overall process. Our method was tested with different combinations of cost functions, and our experiment has shown that our heuristics can assist in speeding up the computation time while providing comparable energy and error.

Keywords

Stereo vision Energy minimization Markov random fields QPBO 

Notes

Acknowledgements

The Scholarship from the Graduate School, Chulalongkorn University to commemorate the 72nd anniversary of his Majesty King Bhumibol Aduladeja is gratefully acknowledged.

References

  1. 1.
    Ahmadvand, A., Kabiri, P.: Multispectral MRI image segmentation using Markov random field model. Signal Image Video Process. 10(2), 251–258 (2016)CrossRefGoogle Scholar
  2. 2.
    Birchfield, S., Tomasi, C.: A pixel dissimilarity measure that is insensitive to image sampling. IEEE Trans. Pattern Anal. Mach. Intell. 20(4), 401–406 (1998)CrossRefGoogle Scholar
  3. 3.
    Boros, E., Hammer, P.L.: Pseudo-Boolean optimization. Discret. Appl. Math. 123(13), 155–225 (2002)MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  5. 5.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)CrossRefGoogle Scholar
  6. 6.
    Conejo, B., Komodakis, N., Leprince, S., Avouac, J.P.: Inference by learning: speeding-up graphical model optimization via a coarse-to-fine cascade of pruning classifiers. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 2105–2113. Curran Associates Inc., New York (2014)Google Scholar
  7. 7.
    Dahlhaus, E., Johnson, D.S., Papadimitriou, C.H., Seymour, P.D., Yannakakis, M.: The complexity of multiterminal cuts. SIAM J. Comput. 23(4), 864–894 (1994)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Delong, A., Veksler, O., Osokin, A., Boykov, Y.: Minimizing sparse high-order energies by submodular vertex-cover. Adv. Neural Inf. Process. Syst. 25, 971–979 (2012)Google Scholar
  9. 9.
    Douini, Y., Riffi, J., Mahraz, A.M., Tairi, H.: An image registration algorithm based on phase correlation and the classical LucasKanade technique. Signal Image Video Process. 11(7), 1321–1328 (2017)CrossRefGoogle Scholar
  10. 10.
    Hammer, P., Hansen, P., Simeone, B.: Roof duality, complementation and persistency in quadratic 01 optimization. Math. Program. 28(2), 121–155 (1984).  https://doi.org/10.1007/BF02612354 CrossRefMATHGoogle Scholar
  11. 11.
    Hirschmuller, H.: Accurate and efficient stereo processing by semi-global matching and mutual information. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 2, pp. 807–814. (2005)Google Scholar
  12. 12.
    Hirschmuller, H., Scharstein, D.: Evaluation of cost functions for stereo matching. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR ’07, pp. 1–8 (2007)Google Scholar
  13. 13.
    Ishikawa, H.: Exact optimization for Markov random fields with convex priors. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1333–1336 (2003)CrossRefGoogle Scholar
  14. 14.
    Kappes, J.H., Andres, B., Hamprecht, F.A., Schnrr, C., Nowozin, S., Batra, D., Kim, S., Kausler, B.X., Krger, T., Lellmann, J., Komodakis, N., Savchynskyy, B., Rother, C.: A comparative study of modern inference techniques for structured discrete energy minimization problems. Int. J. Comput. Vis. (2015).  https://doi.org/10.1007/s11263-015-0809-x MathSciNetGoogle Scholar
  15. 15.
    Kohli, P., Shekhovtsov, A., Rother, C., Kolmogorov, V., Torr, P.: On partial optimality in multi-label MRFs. In: Proceedings of the 25th International Conference on Machine learning, ICML ’08, pp. 480–487. ACM, New York, (2008)Google Scholar
  16. 16.
    Kolmogorov, V., Rother, C.: Comparison of energy minimization algorithms for highly connected graphs. In: Leonardis A., Bischof H., Pinz A. (eds.) Computer Vision ECCV 2006, No. 3952 in Lecture Notes in Computer Science, pp. 1–15. Springer, Berlin (2006)Google Scholar
  17. 17.
    Kolmogorov, V., Rother, C.: Minimizing nonsubmodular functions with graph cuts-A review. IEEE Trans. Pattern Anal. Mach. Intell. 29(7), 1274–1279 (2007)CrossRefGoogle Scholar
  18. 18.
    Kumar, M.P., Koller, D.: MAP estimation of semi-metric MRFs via hierarchical graph cuts. In: Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI ’09, pp. 313–320. AUAI Press, Arlington, (2009)Google Scholar
  19. 19.
    Leelhapantu, S., Chalidabhongse, T.H.: Stereo correspondence using multi-label QPBO method. In: 2014 19th International Conference on Digital Signal Processing (DSP), pp. 173–178 (2014)Google Scholar
  20. 20.
    Lempitsky, V., Rother, C., Blake, A.: LogCut-efficient graph cut optimization for Markov random fields. In: IEEE 11th International Conference on Computer Vision, 2007. ICCV 2007, pp. 1–8 (2007), 00069Google Scholar
  21. 21.
    Lempitsky, V., Rother, C., Roth, S., Blake, A.: Fusion moves for Markov random field optimization. IEEE Trans. Pattern Anal. Mach. Intell. 32(8), 1392–1405 (2010)CrossRefGoogle Scholar
  22. 22.
    Rother, C., Kolmogorov, V., Lempitsky, V., Szummer, M.: Optimizing binary MRFs via extended roof duality. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR’07. pp. 1–8 (2007)Google Scholar
  23. 23.
    Roy, S.: Stereo without epipolar lines: a maximum-flow formulation. Int. J. Comput. Vision 34(2–3), 147–161 (1999)CrossRefGoogle Scholar
  24. 24.
    Roy, S., Cox, I.: A maximum-flow formulation of the N-camera stereo correspondence problem. In: Sixth International Conference on Computer Vision, 1998, pp. 492–499. (1998)Google Scholar
  25. 25.
    Scharstein, D., Pal, C.: Learning conditional random fields for stereo. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR ’07, pp. 1–8. (2007)Google Scholar
  26. 26.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vision 47(1), 7–42 (2002)CrossRefMATHGoogle Scholar
  27. 27.
    Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings, vol. 1, pp. I-195–I-202. (2003)Google Scholar
  28. 28.
    Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., Rother, C.: A comparative study of energy minimization methods for Markov random fields with smoothness-based priors. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 1068–1080 (2008)CrossRefGoogle Scholar
  29. 29.
    Tian, J., Ma, K.K.: A survey on super-resolution imaging. Signal, Image and Video Process. (2011).  https://doi.org/10.1007/s11760-010-0204-6 Google Scholar
  30. 30.
    Veksler, O.: Reducing search space for stereo correspondence with graph cuts. In: BMVC, pp. 709–718. (2006). 00017Google Scholar
  31. 31.
    Zach, C.: A principled approach for coarse-to-fine MAP inference. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1330–1337. (2014)Google Scholar
  32. 32.
    Zhang, Y., Hartley, R., Wang, L.: Fast multi-labelling for stereo matching. In: Computer Vision ECCV 2010, pp. 524–537. Springer, Berlin (2010)Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  • Sangsan Leelhapantu
    • 1
  • Thanarat H. Chalidabhongse
    • 1
  1. 1.Department of Computer EngineeringChulalongkorn UniversityBangkokThailand

Personalised recommendations