Advertisement

Multimedia Tools and Applications

, Volume 77, Issue 17, pp 22923–22952 | Cite as

Dense matching for multi-scale images by propagation

  • Mohammed Laraqui
  • Abderrahim Saaidi
  • Ali Mouhib
  • Mustapha Abarkan
Article

Abstract

This article puts forward a new algorithm of dense matching between two images of the same scene, not necessarily stereoscopic and which can be of different scales. This algorithm is based on points of interest identified by a multi-scale detector and later matched according to a very high threshold to keep only the most reliable points. Afterwards, these points serve as germs for the next iteration. This propagation process is guided according to geometric constraints, and repeated until all the possible correspondents between the two images are obtained. The results of the experiments obtained on test images and those of the real world are very satisfactory even in difficult cases of great displacements and changes in appearance between the two captures.

Keywords

Stereovision Multi-scale Propagation Dense matching 

References

  1. 1.
    Barnes C, Shechtman E, Finkelstein A, Goldman D (2009) PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics-TOG 28(3).  https://doi.org/10.1145/1531326.1531330
  2. 2.
    Barnes C, Shechtman E, Goldman DB, Finkelstein A (2010) The generalized PatchMatch correspondence algorithm, European Conference on Computer Vision (ECCV):29–43.  https://doi.org/10.1007/978-3-642-15558-1_3
  3. 3.
    Bay H, Tuytelaars T, Van Gool L (2006) Surf: Speeded up robust features. European Conference on Computer Vision (ECCV):404–417.  https://doi.org/10.1007/11744023_32
  4. 4.
    Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Surf: Speeded Up Robust Features. Comput Vis Image Underst 110(3):346–359.  https://doi.org/10.1016/j.cviu.2007.09.014 CrossRefGoogle Scholar
  5. 5.
    Chen Q, Medioni G (1999) A volumetric stereo matching method: application to image-based modeling. Computer Vision and Pattern Recognition (CVPR) 1:29–34.  https://doi.org/10.1109/CVPR.1999.786913 Google Scholar
  6. 6.
    Chen D, Ardabilian M, Chen L (2015) A fast trilateral filter based adaptive support weight method for stereo matching. IEEE Transactions on Circuits and Systems for Video Technology 25(5):730–743.  https://doi.org/10.1109/TCSVT.2014.2361422 CrossRefGoogle Scholar
  7. 7.
    Colodro-Conde C, Toledo-Moreo FJ, Toledo-Moreo R, Martınez-Alvarez JJ, Garrigos Guerrero J, Ferrandez-Vicente JM (2014) Evaluation of stereo correspondence algorithms and their implementation on FPGA. J Syst Archit 60(1):22–31.  https://doi.org/10.1016/j.sysarc.2013.11.006 CrossRefGoogle Scholar
  8. 8.
    Ding G, Guo Y, Zhou J, Gao Y (2016) Large-Scale Cross-Modality Search via Collective Matrix Factorization Hashing. IEEE Trans. Image Processing 25(11):5427–5440.  https://doi.org/10.1109/TIP.2016.2607421 MathSciNetCrossRefGoogle Scholar
  9. 9.
    Ekstrand F, Ahlberg C, Ekstrom M, Spampinato G (2015) High-speed segmentation-driven high-resolution matching. Seventh International Conference on Machine Vision (ICMV 2014), pp. 94451Y-94451Y, International Society for Optics and PhotonicsGoogle Scholar
  10. 10.
    El hazzat S, Saaidi A, Karam A, Satori K (2015) Incremental multi-view 3D reconstruction starting from two images taken by a stereo pair of cameras. 3D Res 6(1):1–18.  https://doi.org/10.1007/s13319-015-0041-z CrossRefGoogle Scholar
  11. 11.
    Fang J, Varbanescu AL, Shen J, Sips H, Saygili G, Van Der Maaten L (2012) Accelerating cost aggregation for real-time stereo matching. International Conference on Parallel and Distributed Systems (ICPADS12):472–481.  https://doi.org/10.1109/ICPADS.2012.71
  12. 12.
    Fischler MA, Bolles RC (1981) Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395.  https://doi.org/10.1145/358669.358692 MathSciNetCrossRefGoogle Scholar
  13. 13.
    Fowers J, Brown G, Wernsing J, Stitt G (2013) A performance and energy comparison of convolution on GPUs, FPGAs, and multicore processors. ACM Transactions on Architecture and Code Optimization (TACO) 9(4).  https://doi.org/10.1145/2400682.2400684
  14. 14.
    Gales G, Chambon S, Crouzil A, McDonald J (2012) Reliability measure for propagation-based stereo matching. International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS) 7:1–4.  https://doi.org/10.1109/WIAMIS.2012.6226761 Google Scholar
  15. 15.
    Guo Y, Ding G, Han J, Gao Y (2017) Zero-Shot Learning With Transferred Samples. IEEE Trans Image Processing 26(7):3277–3290.  https://doi.org/10.1109/TIP.2017.2696747 MathSciNetCrossRefGoogle Scholar
  16. 16.
    HaCohen Y, Shechtman E, Goldman DB, Lischinski D (2011) Non-rigid dense correspondence with applications for image enhancement. ACM transactions on graphics (TOG) 30(4).  https://doi.org/10.1145/2010324.1964965
  17. 17.
    Hartley R, Zisserman A (2003) Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge, pp 257–259Google Scholar
  18. 18.
    Hassner T, Mayzels V, Zelnik-Manor L (2012) On SIFTs and their scales, Computer Vision and Pattern Recognition (CVPR):1522–1528.  https://doi.org/10.1109/CVPR.2012.6247842
  19. 19.
    Hirschmuller H (2008) Stereo processing by semiglobal matching and mutual information. IEEE Trans Pattern Anal Mach Intell 30(2):328–341.  https://doi.org/10.1109/TPAMI.2007.1166 CrossRefGoogle Scholar
  20. 20.
    Jin M, Maruyama T (2014) Fast and accurate stereo vision system on FPGA. ACM Transactions on Reconfigurable Technology and Systems (TRETS) 7(1).  https://doi.org/10.1145/2567659
  21. 21.
    Kalarot R, Morris J (2010) Comparison of FPGA and GPU implementations of real-time stereo vision. Computer Vision and Pattern Recognition Workshops (CVPRW):9–15.  https://doi.org/10.1109/CVPRW.2010.5543743
  22. 22.
    Kim J, Liu C, Sha F, Grauman K (2013) Deformable spatial pyramid matching for fast dense correspondences. Computer Vision and Pattern Recognition (CVPR):2307–2314.  https://doi.org/10.1109/CVPR.2013.299
  23. 23.
    Kokkinos I, Yuille A (2008) Scale invariance without scale selection. Computer Vision and Pattern Recognition (CVPR):1–8.  https://doi.org/10.1109/CVPR.2008.4587798
  24. 24.
    Korman S, Avidan S (2011) Coherency sensitive hashing. IEEE International Conference on Computer Vision (ICCV):1607–1614.  https://doi.org/10.1109/TPAMI.2015.2477814
  25. 25.
    Kowalczuk J, Psota ET, Perez LC (2012) Real-time stereo matching on CUDA using an iterative refinement method for adaptive support-weight correspondences. IEEE Transactions on Circuits and Systems for Video Technology 23(1):94–104.  https://doi.org/10.1109/TCSVT.2012.2203200 CrossRefGoogle Scholar
  26. 26.
    Laraqui M, Saaidi A, Mouhib A, Abarkan M (2015) Images Matching Using Voronoï Regions Propagation. 3D Res 6(3):1–16.  https://doi.org/10.1007/s13319-015-0056-5 CrossRefGoogle Scholar
  27. 27.
    Lee Z, Juang J, Nguyen TQ (2013) Local disparity estimation with three-moded cross census and advanced support weight. IEEE Transactions on Multimedia 15(8):1855–1864.  https://doi.org/10.1109/TMM.2013.2270456 CrossRefGoogle Scholar
  28. 28.
    Lhuillier M, Long Q (2002) Match propagation for image-based modeling and rendering. Pattern Analysis and Machine Intelligence (PAMI) 24(8):1140–1146.  https://doi.org/10.1109/TPAMI.2002.1023810 CrossRefGoogle Scholar
  29. 29.
    Lhuillier M, Quan L (2004). Reconstruction quasi-dense et modèles 3d à partir d’une séquence d’images. Congrès francophone AFRIF-AFIA de reconnaissance des formes et d'intelligence artificielle. AFRIF-AFIAGoogle Scholar
  30. 30.
    Lin W-Y, Liu S, Matsushita Y, Ng T-T, Cheong L-F (2011) Smoothly varying affine stitching. Computer Vision and Pattern Recognition (CVPR):345–352.  https://doi.org/10.1109/CVPR.2011.5995314
  31. 31.
    Lin Y, Lu N, Lou X, Zou F, Yao Y, Du Z (2013) Matching cost filtering for dense stereo correspondence. Math Probl Eng 2013.  https://doi.org/10.1155/2013/654139
  32. 32.
    Liu C, Yuen J, Torralba A (2011) SIFT flow: Dense correspondence across scenes and its applications. IEEE transactions on Pattern Analysis and Machine Intelligence TPAMI 33(5):978–994.  https://doi.org/10.1109/TPAMI.2010.147 CrossRefGoogle Scholar
  33. 33.
    Ma Y, Liu W (2005) Progressive matching based on segmentation for 3d reconstruction. Computer and Information Technology (CIT):575–579.  https://doi.org/10.1109/CIT.2005.157
  34. 34.
    Ma L, Li J, Ma J, Zhang H (2013) A modified census transform based on the neighborhood information for stereo matching algorithm. Image and Graphics (ICIG13):533–538.  https://doi.org/10.1155/2013/654139
  35. 35.
    Ma Z, He K, Wei Y, Sun J, Wu E (2013) Constant time weighted median filtering for stereo matching and beyond. International Conference on Computer Vision (ICCV13):49–56.  https://doi.org/10.1109/ICCV.2013.13
  36. 36.
    Matsuo T, Fujita S, Fukushima N, Ishibashi Y (2015). Efficient edge-awareness propagation via single-map filtering for edge-preserving stereo matching. Three-Dimensional Image Processing, Measurement (3DIPM), and Applications, pp. 93930S-8. doi:10.1117/12.2083087Google Scholar
  37. 37.
    Mattoccia S (2013) Stereo vision algorithms for FPGAs. Computer Vision and Pattern Recognition Workshops (CVPRW13):636–641.  https://doi.org/10.1109/CVPRW.2013.96
  38. 38.
    Megyesi Z, Chetverikov D (2004) Affine propagation for surface reconstruction in wide baseline stereo. Pattern Recognition (ICPR2004) 4:76–79.  https://doi.org/10.1109/ICPR.2004.1333709 Google Scholar
  39. 39.
    Miled W, Pesquet JC, Parent M (2009) A convex optimization approach for depth estimation under illumination variation. IEEE Trans Image Process 18(4):813–830.  https://doi.org/10.1109/TIP.2008.2011386 MathSciNetCrossRefMATHGoogle Scholar
  40. 40.
    Miron A, Ainouz S, Rogozan A, Bensrhair A (2014) A robust cost function for stereo matching of road scenes. Pattern Recogn Lett 38(1):70–77.  https://doi.org/10.1016/j.patrec.2013.11.009 CrossRefGoogle Scholar
  41. 41.
    Mota V, Falcao G, Antunes M, Barreto J, Nunes U (2014) Using the GPU for fast symmetry-based dense stereo matching in high resolution images. Acoustics, Speech, and Signal Processing (ICASSP14):7520–7524.  https://doi.org/10.1109/ICASSP.2014.6855062
  42. 42.
    Perez JM, Sánchez P (2011) Real-time stereo matching using memory-efficient Belief Propagation for high-definition 3D telepresence systems. Pattern Recogn Lett 32(16):2250–2253.  https://doi.org/10.1016/j.patrec.2011.06.016 CrossRefGoogle Scholar
  43. 43.
    Pham CC, Jeon JW (2013) Domain transformation-based efficient cost aggregation for local stereo matching. IEEE Transactions on Circuits and Systems for Video Technology 23(7):1119–1130.  https://doi.org/10.1109/TCSVT.2012.2223794 CrossRefGoogle Scholar
  44. 44.
    Ploumpis S, Amanatiadis A, Gasteratos A (2015) A stereo matching approach based on particle filters and scattered control landmarks. Image Vis Comput 38:13–23.  https://doi.org/10.1016/j.imavis.2015.04.001 CrossRefGoogle Scholar
  45. 45.
    Sabater N, Morel J-M, Almansa A (2011) How Accurate Can Block Matches Be in Stereo Vision? SIAM Journal on Imaging Sciences 4(1):472–500.  https://doi.org/10.1137/100797849 MathSciNetCrossRefMATHGoogle Scholar
  46. 46.
    Salmen J, Schlipsing M, Edelbrunner J, Hegemann S, Luke S (2009) Real-time stereo vision: making more out of dynamic programming. International Conference on Computer Analysis of Images and Patterns:1096–1103.  https://doi.org/10.1007/978-3-642-03767-2_133
  47. 47.
    Taime A, Riffi J, Saaidi A, Satori K (2017) Robust point matching via corresponding circles. Multimedia Tools and Applications, pp:1–20.  https://doi.org/10.1007/s11042-017-5086-y
  48. 48.
    Tippetts B, Lee DJ, Lillywhite K, Archibald JK (2014) Hardware-efficient design of real-time profile shape matching stereo vision algorithm on FPGA. International Journal of Reconfigurable Computing 2014.  https://doi.org/10.1155/2014/945926
  49. 49.
    Trulls E, Kokkinos I, Sanfeliu A, Moreno-Noguer F (2013) Dense segmentation-aware descriptors. Computer Vision and Pattern Recognition (CVPR):2890–2897.  https://doi.org/10.1109/CVPR.2013.372
  50. 50.
    Vedaldi A, Fulkerson B (2010). VLFeat: An open and portable library of computer vision algorithms. Proceedings of the 18th ACM international conference on Multimedia. ACM, pp. 1469–1472. doi: 10.1145/1873951.1874249Google Scholar
  51. 51.
    Vijayanagar KR, Loghman M, Kim J (2014) Real-time refinement of kinect depth maps using multi-resolution anisotropic diffusion. Mobile Networks and Applications 19(3):414–425.  https://doi.org/10.1007/s11036-013-0458-7 CrossRefGoogle Scholar
  52. 52.
    Wang W, Yan J, Xu N, Wang Y, Hsu FH (2013). Real-time high-quality stereo vision system in FPGA. 12th International Conference on Field-Programmable Technology (FPT 13), doi: 10.1109/FPT.2013.6718387Google Scholar
  53. 53.
    Wang Y, Zhong F, Peng Q, Qin X (2014) Depth map enhancement based on colour and depth consistency. Vis Comput 30(10):1157–1168.  https://doi.org/10.1007/s00371-013-0896-z CrossRefGoogle Scholar
  54. 54.
    Wang D, Wang B, Zhao S, Yao H, Liu H (2017) View-based 3D object retrieval with discriminative views. Neurocomputing 252(c):58–66.  https://doi.org/10.1016/j.neucom.2016.06.095 CrossRefGoogle Scholar
  55. 55.
    Wei Y, Quan L (2004) Region-based progressive stereo matching. Computer Vision and Pattern Recognition.  https://doi.org/10.1109/CVPR.2004.1315020
  56. 56.
    C. Wu (2011) VisualSFM: A Visual Structure from Motion System, http://ccwu.me/vsfm/
  57. 57.
    Wu C Towards Linear-time Incremental Structure From Motion. 3DTV-Conference (IEEE):127–2013, 134.  https://doi.org/10.1109/3DV.2013.25
  58. 58.
    Wu C, Agarwal S, Curless B, Seitz SM (2011) Multicore Bundle Adjustment. Computer Vision and Pattern Recognition (CVPR):3057–3064.  https://doi.org/10.1109/CVPR.2011.5995552
  59. 59.
    Xu L, Au OC, Sun W, Fang L, Tang K, Li J, Guo Y (2013) Stereo matching by adaptive weighting selection based cost aggregation. Circuits and Systems (ISCAS13):1420–1423.  https://doi.org/10.1109/ISCAS.2013.6572122
  60. 60.
    Xu T, Cockshott P, Oehler S (2014) Acceleration of stereo-matching on multi-core CPU and GPU. High Performance Computing and Communications(HPCC):108–115.  https://doi.org/10.1109/HPCC.2014.22
  61. 61.
    Yang Q, Wang L, Yang R, Stewenius H, Nistér D (2009) Stéreo matching with colour-weighted correlation, hierarchical belief propagation, and occlusion handling. IEEE Trans Pattern Anal Mach Intell 31(3):492–504.  https://doi.org/10.1109/TPAMI.2008.99 CrossRefGoogle Scholar
  62. 62.
    Yang Q, Ji P, Li D, Yao S, Zhang M (2014) Fast stereo matching using adaptive guided filtering. Image Vis Comput 32(3):202–211.  https://doi.org/10.1016/j.imavis.2014.01.001 CrossRefGoogle Scholar
  63. 63.
    Yang H, Lin W-Y, Lu J (2014) Daisy filter flow: A generalized discrete approach to dense correspondences. Computer Vision and Pattern Recognition (CVPR):3406–3413.  https://doi.org/10.1109/CVPR.2014.435
  64. 64.
    Yao S, Wang L, Lin C, Zhang M (2015) Real-time stereo to multi-view conversion system based on adaptive meshing. J Real-Time Image Proc:1–19.  https://doi.org/10.1007/s11554-015-0490-x
  65. 65.
    Zhang Z, Shan Y (2001) A progressive scheme for stereo matching. 3D Structure from Multiple Images of Large-Scale Environments (SMILE):68–85.  https://doi.org/10.1007/3-540-45296-6_5
  66. 66.
    Zhang N, Wang H, Cr J (2013) A near real-time colour stereo matching method for GPU. 3rd International Conference on Advanced Communications and Computation, Lisbon, Portugal, pp 27–32Google Scholar
  67. 67.
    Zhang J, Nezan JF, Pelcat M, Cousin JG (2013) Real-time GPU-based local stereo matching method. Design and Architectures for Signal and Image Processing (DASIP13), Cagliari, Italy, pp 209–214Google Scholar
  68. 68.
    Zhang S, Wang C, Chan SC (2013) A new high resolution depth map estimation system using stereo vision and depth sensing device. Signal Processing and Its Applications (CSPA13):49–53.  https://doi.org/10.1109/CSPA.2013.6530012
  69. 69.
    Zhao S, Chen L, Yao H, Zhang Y, Sun X (2015) Strategy for dynamic 3D depth data matching towards robust action retrieval. Neurocomputing 151(2):533–543.  https://doi.org/10.1016/j.neucom.2014.03.092 CrossRefGoogle Scholar
  70. 70.
    Zhao S, Yao H, Zhang Y, Wang Y, Liu S (2015) View-based 3D object retrieval via multi-modal graph learning. Signal Process.  https://doi.org/10.1016/j.sigpro.2014.09.038

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Mohammed Laraqui
    • 1
  • Abderrahim Saaidi
    • 1
    • 2
  • Ali Mouhib
    • 1
  • Mustapha Abarkan
    • 1
  1. 1.LSI, Laboratory of Engineering Sciences, Polydisciplinary Faculty of TazaSidi Mohamed Ben Abdellah University Fez MoroccoFesMorocco
  2. 2.LIIAN, Department of Mathematics and Computer Science, Faculty of Sciences Dhar El MahrazSidi Mohamed Ben Abdellah University Fez MoroccoFesMorocco

Personalised recommendations