Skip to main content
SpringerLink
Log in

Support vector machines based stereo matching method for advanced driver assistance systems

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Stereo vision is a measurement method for finding correspondence between two or more input images in order to obtain a detailed 3D representation of a scene. This paper presents an approach for matching stereo sequences acquired by a stereo sensor mounted on an intelligent vehicle. The approach uses machine learning alongside spatio-temporal information to predict the matching results. This means that the obtained matching results during the previous frame are used as a training samples for a support vector machine classifier, as well as to derive disparity ranges for each scan-line, which are then used to predict the matching of the current frame. The distance to the hyper-plan computed by the SVM is used as a cost function to fill a 2D search space. Then, the dynamic programming algorithm is performed for matching edge points in the stereo pair. Experiments on both virtual and real stereo image sequences have been conducted, demonstrating satisfactory performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Ansari ME, Mazoul A, Bensrhair A, Bebis G (2011) A real-time spatio-temporal stereo matching for road applications. In: 14th International IEEE conference on intelligent transportation systems, ITSC 2011, Washington, DC, USA, October 5–7, 2011. [Online]. Available: https://doi.org/10.1109/ITSC.2011.6082875, pp 1483–1488

  2. Boser BE, Guyon IM, Vapnik VN (1992) A training algorithm for optimal margin classifiers. In: Proceedings of the fifth annual workshop on computational learning theory. ACM, pp 144–152

  3. Boykov Y, Veksler O, Zabih R (2001) Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell 23(11):1222–1239

    Article  Google Scholar 

  4. Canny J (1987) A computational approach to edge detection. In: Readings in computer vision. Elsevier, pp 184–203

  5. Čech J, Sanchez-Riera J, Horaud R (2011) Scene flow estimation by growing correspondence seeds. In: CVPR 2011. IEEE, pp 3129–3136

  6. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  7. Cruz J, Pajares G, Aranda J, Vindel J (1995) Stereo matching technique based on the perceptron criterion function. Pattern Recognit Lett 16(9):933–944

    Article  Google Scholar 

  8. Davis J, Ramamoorthi R, Rusinkiewicz S (2003) Spacetime stereo: a unifying framework for depth from triangulation. In: 2003 IEEE computer society conference on computer vision and pattern recognition. Proceedings, vol 2. IEEE, pp II–359

  9. Dobias M, Sara R (2011) Real-time global prediction for temporally stable stereo. In: 2011 IEEE international conference on computer vision workshops (ICCV Workshops). IEEE, pp 704–707

  10. El Ansari M, Mousset S, Bensrhair A (2010) Temporal consistent real-time stereo for intelligent vehicles. Pattern Recognit Lett 31(11):1226–1238

    Article  Google Scholar 

  11. El Jaafari I, El Ansari M, Koutti L, Mazoul A, Ellahyani A (2016) Fast spatio-temporal stereo matching for advanced driver assistance systems. Neurocomputing 194:24–33

    Article  Google Scholar 

  12. El Jaafari I, El Ansari M, Koutti L (2017) Fast edge-based stereo matching approach for road applications. Signal Image Video Process 11(2):267–274

    Article  Google Scholar 

  13. Ellahyani A, El Ansari M (2017) Mean shift and log-polar transform for road sign detection. Multimed Tools Appl 76(22):24495–24513

    Article  Google Scholar 

  14. Gong M (2006) Enforcing temporal consistency in real-time stereo estimation. In: European conference on computer vision. Springer, Berlin, pp 564–577

  15. Hirschmuller H (2008) Stereo processing by semiglobal matching and mutual information. IEEE Trans Pattern Anal Mach Intell 30(2):328–341

    Article  Google Scholar 

  16. Hosni A, Rhemann C, Bleyer M, Rother C, Gelautz M (2013) Fast cost-volume filtering for visual correspondence and beyond. IEEE Trans Pattern Anal Mach Intell 35(2):504–511

    Article  Google Scholar 

  17. Huang T, Yang G, Tang G (1979) A fast two-dimensional median filtering algorithm. IEEE Trans Acoust Speech Signal Process 27(1):13–18

    Article  Google Scholar 

  18. Jiang J, Cheng J, Chen B, Wu X (2014) Disparity prediction between adjacent frames for dynamic scenes. Neurocomputing 142:335–342

    Article  Google Scholar 

  19. Jiao J, Wang R, Wang W, Dong S, Wang Z, Gao W (2014) Local stereo matching with improved matching cost and disparity refinement. IEEE MultiMedia 21(4):16–27

    Article  Google Scholar 

  20. Kendall A, Martirosyan H, Dasgupta S, Henry P, Kennedy R, Bachrach A, Bry A (2017) End-to-end learning of geometry and context for deep stereo regression. In: Proceedings of the IEEE international conference on computer vision, pp 66–75

  21. Kerkaou Z, Alioua N, El Ansari M, Masmoudi L (2018) Edge points-based stereo matching approach for omnidirectional images. J Electron Imaging 27(5):053015

    Article  Google Scholar 

  22. Kerkaou Z, Alioua N, El Ansari M, Masmoudi L (2018) A new dense omnidirectional stereo matching approach. In: 2018 International conference on intelligent systems and computer vision (ISCV). IEEE, pp 1–8

  23. Kerkaou Z, El Ansari M, Masmoudi L (2018) Fast spatio-temporal stereo matching method for omnidirectional images. In: 2018 6th International conference on wireless networks and mobile communications (WINCOM). IEEE, pp 1–6

  24. Klaus A, Sormann M, Karner K (2006) Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In: 18th International conference on pattern recognition, 2006. ICPR 2006, vol 3. IEEE, pp 15–18

  25. Kogler J, Eibensteiner F, Humenberger M, Sulzbachner C, Gelautz M, Scharinger J (2014) Enhancement of sparse silicon retina-based stereo matching using belief propagation and two-stage postfiltering. J Electron Imaging 23(4):043011

    Article  Google Scholar 

  26. Kolmogorov V, Zabih R (2001) Computing visual correspondence with occlusions using graph cuts. In: IEEE international conference on computer vision, 2001. ICCV 2001. Proceedings. Eighth, vol 2. IEEE, pp 508–515

  27. Kong D, Tao H (2004) A method for learning matching errors for stereo computation. In: BMVC, vol 1, p 2

  28. Labayrade R, Aubert D, Tarel J-P (2002) Real time obstacle detection in stereovision on non flat road geometry through “v-disparity” representation. In: Intelligent vehicle symposium, 2002. IEEE, vol 2. IEEE, pp 646–651

  29. Lahmyed R, El Ansari M, Tremeau A, Kerkaou Z (2018) Camera-light detection and ranging data fusion-based system for pedestrian detection. J Electron Imaging 27(6):063011

    Google Scholar 

  30. Lahmyed R, El Ansari M, Ellahyani A (2019) A new thermal infrared and visible spectrum images-based pedestrian detection system. Multimed Tools Appl 78(12):15861–15885

    Article  Google Scholar 

  31. Lew MS, Huang TS, Wong K (1994) Learning and feature selection in stereo matching. IEEE Trans Pattern Anal Mach Intell 16(9):869–881

    Article  Google Scholar 

  32. Li Y, Fang S (2019) Removal-based multi-view stereo using a window-based matching method. Optik 178:1318–1336

    Article  Google Scholar 

  33. Liu Y, Cheng M-M, Fan D-P, Zhang L, Bian J, Tao D (2018) Semantic edge detection with diverse deep supervision. arXiv preprint arXiv:1804.02864

  34. Luo W, Schwing AG, Urtasun R (2016) Efficient deep learning for stereo matching. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5695–5703

  35. Ma Y, Chen W, Ma X, Xu J, Huang X, Maciejewski R, Tung AK (2017) Easysvm: a visual analysis approach for open-box support vector machines. Comput Vis Media 3(2):161–175

    Article  Google Scholar 

  36. Mazoul A, El Ansari M, Zebbara K, Bebis G (2014) Fast spatio-temporal stereo for intelligent transportation systems. Pattern Anal Appl 17(1):211–221

    Article  MathSciNet  Google Scholar 

  37. Mei X, Sun X, Dong W, Wang H, Zhang X (2013) Segment-tree based cost aggregation for stereo matching. In: IEEE conference on computer vision and pattern recognition (CVPR), 2013. IEEE, pp 313–320

  38. Menze M, Geiger A (2015) Object scene flow for autonomous vehicles. In: Conference on computer vision and pattern recognition (CVPR)

  39. Motten A, Claesen L, Pan Y (2012) Trinocular disparity processor using a hierarchic classification structure. In: 2012 IEEE/IFIP 20th international conference on VLSI and System-on-Chip (VLSI-SoC). IEEE, pp 247–250

  40. Nie G-Y, Cheng M-M, Liu Y, Liang Z, Fan D-P, Liu Y, Wang Y (2019) Multi-level context ultra-aggregation for stereo matching. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3283–3291

  41. Ohta Y, Kanade T (1985) Stereo by intra-and inter-scanline search using dynamic programming. IEEE Trans Pattern Anal Mach Intell 2:139–154

    Article  Google Scholar 

  42. Pham CC, Jeon JW (2013) Domain transformation-based efficient cost aggregation for local stereo matching. IEEE Trans Circ Syst Video Technol 23(7):1119–1130

    Article  Google Scholar 

  43. Raghavendra U, Makkithaya K, Karunakar A (2015) Anchor-diagonal-based shape adaptive local support region for efficient stereo matching. Signal Image Video Process 9(4):893–901

    Article  Google Scholar 

  44. Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vis 47(1–3):7–42

    Article  MATH  Google Scholar 

  45. Song X, Zhao X, Fang L, et al (2020) Edgestereo: An Effective Multi-task Learning Network for Stereo Matching and Edge Detection. Int J Comput Vis 128, 910–930

  46. Spyropoulos A, Komodakis N, Mordohai P (2014) Learning to detect ground control points for improving the accuracy of stereo matching. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1621–1628

  47. Sun J, Zheng N-N, Shum H-Y (2003) Stereo matching using belief propagation. IEEE Trans Pattern Anal Mach Intell 25(7):787–800

    Article  MATH  Google Scholar 

  48. Tao H, Sawhney HS, Kumar R (2001) Dynamic depth recovery from multiple synchronized video streams. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, 2001. CVPR 2001, vol 1. IEEE, pp I–I

  49. van der Mark W (2008) Synthetic stereo image data for algorithm evaluation. http://stereodatasets.wvandermark.com/

  50. Wang L, Gong M, Gong M, Yang R (2006) How far can we go with local optimization in real-time stereo matching. In: Third international symposium on 3D data processing, visualization, and transmission. IEEE, pp 129–136

  51. Yang Q (2012) A non-local cost aggregation method for stereo matching. In: 2012 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1402–1409

  52. Yang G, Zhao H, Shi J, Deng Z, Jia J (2018) Segstereo: exploiting semantic information for disparity estimation. In: Proceedings of the European conference on computer vision (ECCV), pp 636–651

  53. Yang G, Song X, Huang C, Deng Z, Shi J, Zhou B (2019) Drivingstereo: a large-scale dataset for stereo matching in autonomous driving scenarios. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 899–908

  54. Yu W, Chen T, Hoe JC (2009) Real time stereo vision using exponential step cost aggregation on gpu. In: 2009 16th IEEE international conference on image processing (ICIP). IEEE, pp 4281–4284

  55. žbontar J, LeCun Y (2016) Stereo matching by training a convolutional neural network to compare image patches. J Mach Learn Res 17(1):2287–2318

    MATH  Google Scholar 

  56. Zhang L, Curless B, Seitz SM (2003) Spacetime stereo: shape recovery for dynamic scenes. In: 2003 IEEE computer society conference on computer vision and pattern recognition, 2003. Proceedings, vol 2. IEEE, pp II–367

  57. Zhang G, Jia J, Wong T-T, Bao H (2009) Consistent depth maps recovery from a video sequence. IEEE Trans Pattern Anal Mach Intell 31(6):974–988

    Article  Google Scholar 

  58. Zhang J, Skinner KA, Vasudevan R, Johnson-Roberson M (2019) Dispsegnet: leveraging semantics for end-to-end learning of disparity estimation from stereo imagery. IEEE Robot Autom Lett 4(2):1162–1169

    Article  Google Scholar 

  59. Zhao J-X, Liu J-J, Fan D-P, Cao Y, Yang J, Cheng M-M (2019) Egnet: edge guidance network for salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 8779–8788

  60. Zhu S, Gao R, Li Z (2017) Stereo matching algorithm with guided filter and modified dynamic programming. Multimed Tools Appl 76(1):199–216

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. LabSIV, Department of Computer Science, Faculty of Science, Ibn Zohr University, BP 8106, 80000, Agadir, Morocco

    Zakaria Kerkaou & Mohamed El Ansari

Authors
  1. Zakaria Kerkaou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed El Ansari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zakaria Kerkaou.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kerkaou, Z., El Ansari, M. Support vector machines based stereo matching method for advanced driver assistance systems. Multimed Tools Appl 79, 27039–27055 (2020). https://doi.org/10.1007/s11042-020-09260-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09260-3

Keywords

Search

Navigation