Journal of Real-Time Image Processing

, Volume 9, Issue 4, pp 621–633 | Cite as

Phase-correlation guided area matching for realtime vision and video encoding

  • Alfonso Alba
  • Edgar Arce-Santana
  • Ruth M. Aguilar-Ponce
  • Daniel U. Campos-Delgado
Original Research Paper

Abstract

In computer vision and video encoding applications, one of the first and most important steps is to establish a pixel-to-pixel correspondence between two images of the same scene obtained at slightly different times or points of view. One of the most popular methods to find these correspondences, known as Area Matching, consists in performing a computationally intensive search for each pixel in the first image, around a neighborhood of the same pixel in the second image. In this work we propose a method which significantly reduces the search space to only a few candidates, and permits the implementation of real-time vision and video encoding algorithms which do not require specialized hardware such as GPU’s or FPGA’s. Theoretical and experimental support for this method is provided. Specifically, we present results from the application of the method to the realtime video compression and transmission, as well as the realtime estimation of dense optical flow and stereo disparity maps, where a basic implementation achieves up to 100 fps in a typical dual-core PC.

Keywords

Phase correlation Video encoding Realtime vision Optical flow Stereo vision 

References

  1. 1.
    Aguilar-Ponce, R.M., Tecpanecatl-Xihuitl, J.L., Alba-Cadena, A., Arce-Santana, E.: Poc-guided area matching algoritm for video encoding. In: Proceedings of the 53rd IEEE International Midwest Symposium on Circuits and Systems, Seattle, Washington, pp. 503–508 (2010)Google Scholar
  2. 2.
    Alba, A., Arce-Santana, E.: Phase-correlation guided search for realtime stereo vision. In: 13th International Workshop on Combinatorial Image Analysis. Lecture Notes in Computer Science, vol. 5852, pp. 212–223 (2009)Google Scholar
  3. 3.
    Alba, A., Arce-Santana, E., Rivera, M.: Optical flow estimation with prior models obtained from phase correlation. In: 6th International Symposium on Visual Computing. Lecture Notes in Computer Science, vol. 6453, pp. 417–426 (2010)Google Scholar
  4. 4.
    Arce, E., Marroquin, J.L.: High-precision stereo disparity estimation using HMMF models. Image Vis. Comput. 25, 623–636 (2007)CrossRefGoogle Scholar
  5. 5.
    Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. Tech. Rep. MSR-TR-2009-179. Microsoft Research (2009)Google Scholar
  6. 6.
    Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. Int. J. Comput. Vis. 92(1), 1–31 (2011)CrossRefGoogle Scholar
  7. 7.
    Barron, J., Fleet, D., Beauchemin, S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994)CrossRefGoogle Scholar
  8. 8.
    Bruhn, A., Weickert, J., Feddern, C., Kohlberger, T., Schnörr, C.: Variational optical flow computation in real time. IEEE Trans. Image Process. 14(5), 608–615 (2005)CrossRefMathSciNetGoogle Scholar
  9. 9.
    Chien, L.H., Aoki, T.: Robust motion estimation for video sequences based on phase-only correlation. In: 6th IASTED International Conference on Signal and Image Processing, pp. 441–446 (2004)Google Scholar
  10. 10.
    Darabiha, A., MacLean, W.J., Rose, J.: Reconfigurable hardware implementation of a phase-correlation stereo algorithm. Mach. Vis. Appl. 17(2), 116–132 (2006)CrossRefGoogle Scholar
  11. 11.
    Di Stefano, L., Marchionni, M., Mattoccia, S.: A fast area-based stereo matching algorithm. Image Vis. Comput. 22, 983–1005 (2004)CrossRefGoogle Scholar
  12. 12.
    Díaz, J., Ros, E., Mota, S., Carrillo, R., Agis, R.: Real time optical flow processing system. Field programmable logic and application. Lecture Notes in Computer Science, vol. 3203, pp. 617–626 (2004)Google Scholar
  13. 13.
    Ernst, I., Hirschmüller, H.: Mutual information based semi-global stereo matching on the GPU. In: 4th International Symposium on Visual Computing. Lecture Notes in Computer Science, vol. 5358, pp. 228–239 (2008)Google Scholar
  14. 14.
    Essannouni, F., Haj Thami, R.O., Salam, A., Aboutajdine, D.: An efficient fast full search block matching algorithm using FFT algorithms. Int. J. Comput. Sci. Network Security 6(3B), 130–133 (2006)Google Scholar
  15. 15.
    Frigo, M., Johnson, S.G.: FFTW Web Page. http://www.fftw.org (2009)
  16. 16.
    Gong, M.: Enforcing temporal consistency in real-time stereo estimation. In: 9th European Conference on Computer Vision. Lecture Notes in Computer Science, vol. 3953, pp. 564–577 (2006)Google Scholar
  17. 17.
    Hirschmüller, H., Innocent, P.R., Garibaldi, J.: Real-time correlation-based stereo vision with reduced border errors. Int. J. Comput. Vis. 47(1-3), 229–246 (2002)CrossRefMATHGoogle Scholar
  18. 18.
    Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)CrossRefGoogle Scholar
  19. 19.
    Keller, Y., Averbuch, A., Moshe, I.: Pseudopolar-based estimation of large translations, rotations, and scalings in images. IEEE Trans. Image Process. 14(1), 12–22 (2005)CrossRefMathSciNetGoogle Scholar
  20. 20.
    Kim, H., Min, D.B., Sohn, K.: Real-time stereo using foreground segmentation and hierarchical disparity estimation. In: 6th Pacific-Rim Conference on Multimedia. Lecture Notes in Computer Science, vol. 3767, pp. 384–395 (2005)Google Scholar
  21. 21.
    Koga, T., Iinuma, K., Hirano, A., Iijima, Y.: Motion compensated interframe coding for video conferencing. NTC’81 Conference Record, pp. G5.3.1–G5.3.5 (1981)Google Scholar
  22. 22.
    Kollnig, H., Nagel, H.H., Otte, M.: Association of motion verbs with vehicle movements extracted from dense optical flow fields. In: Proceedings of 3rd European Conference on Computer Vision (ECCV ’94). Lecture Notes in Computer Science, vol. 801/II, pp. 338–347 (1994)Google Scholar
  23. 23.
    Kopp, M., Purgathofer, W.: Efficient 3 × 3 Median Filter Computations. Technical University, Vienna (1994)Google Scholar
  24. 24.
    Li, R., Zeng, B., Liou, M.L.: A new three-step search algorithm for block motion estimation. IEEE Trans. Circuits Syst. Video Technol. 4(4), 438–442 (1994)CrossRefGoogle Scholar
  25. 25.
    López, S., Callico, G.M., López, J.F., Roberto, S.: A high quality/low computational cost technique for block matching motion estimation. In: Design, Automation and Test in Europe Conference and Exhibition (DATE 05), pp. 2–7 (2005)Google Scholar
  26. 26.
    Lu, J., Liou, M.L.: A simple and efficient search algorithm for block matching motion estimation. IEEE Trans. Circuits Syst. Video Technol. 7(2), 429–433 (1997)CrossRefGoogle Scholar
  27. 27.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of Imaging Understanding Workshop, pp. 121–130 (1981)Google Scholar
  28. 28.
    Marroquin, J.L., Mitter, S., Poggio, T.: Probabilistic solution of ill-posed problems in computational vision. J. Am. Stat. Assoc. 82, 76–89 (1987)CrossRefMATHGoogle Scholar
  29. 29.
    Marzat, J., Dumortier, Y., Ducrot, A.: Real-time dense and accurate parallel optical flow using cuda. In: WSCG 2009 Proceedings, pp. 105–111 (2009)Google Scholar
  30. 30.
    Nie, Y., Ma, K.K.: Adaptive rood pattern search for fast block-matching motion estimation. IEEE Trans. Image Process. 11(12), 1442–1448 (2002)CrossRefGoogle Scholar
  31. 31.
    Pauwels, K., Van Hulle, M.M.: Realtime phase-based optical flow on the GPU. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2008Google Scholar
  32. 32.
    Po, L., Ma, W.: A novel four-step search algorithm for fast block motion estimation. IEEE Trans. Circuits Syst. Video Technol. 6(3), 313–317 (1996)CrossRefGoogle Scholar
  33. 33.
    Rannacher, J.: Realtime 3D motion estimation on graphics hardware. Bachelor thesis, Heidelberg University (2009)Google Scholar
  34. 34.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 7–42 (2002)CrossRefMATHGoogle Scholar
  35. 35.
    Shimizu, M., Okutomi, M.: Sub-pixel estimation error cancellation on area-based matching. Int. J. Comput. Vis. 63(3), 207–224 (2005)CrossRefGoogle Scholar
  36. 36.
    Takita, K., Muquit, M.A., Aoki, T., Higuchi, T.: A sub-pixel correspondence search technique for computer vision applications. IECIE Trans. Fundam. E 87-A(8), 1913–1923 (2004)Google Scholar
  37. 37.
    Toyama, K., Krumm, J., Brummit, B., Meyers, B.: Wallflower: principles and practice of background maintenance. In: Seventh International Conference on Computer Vision, pp. 255–261 (1999)Google Scholar
  38. 38.
    Veksler, O.: Reducing search space for stereo correspondence with graph cuts. In: British Machine Vision Conference, vol. 2, pp. 709–719 (2006)Google Scholar
  39. 39.
    Wei, Z., Lee, D.J., Nelson, B., Martineau, M.: A fast and accurate tensor-based optical flow algorithm implemented in FPGA. In: Proceedings of the IEEE Workshop on Applications of Computer Vision (WACV’07) (2007)Google Scholar
  40. 40.
    Zhu, S., Ma, K.: A new diamond search algorithm for fast block-matching motion estimation. IEEE Trans. Image Process. 9(2), 287–290 (2000)CrossRefMathSciNetGoogle Scholar
  41. 41.
    Zinner, C., Humenberger, M., Ambrosch, K., Kubinger, W.: An optimized software-based implementation of a census-based stereo matching algorithm. In: 4th International Symposium on Visual Computing. Lecture Notes in Computer Science, vol. 5358, pp. 216–227 (2008)Google Scholar

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  • Alfonso Alba
    • 1
  • Edgar Arce-Santana
    • 1
  • Ruth M. Aguilar-Ponce
    • 1
  • Daniel U. Campos-Delgado
    • 1
  1. 1.Facultad de Ciencias, Universidad Autónoma de San Luis PotosíSan Luis PotosíMexico

Personalised recommendations