Journal of Real-Time Image Processing

, Volume 5, Issue 4, pp 275–289 | Cite as

Integrated video motion estimator with Retinex-like pre-processing for robust motion analysis in automotive scenarios: algorithmic and real-time architecture design

  • Stefano Marsi
  • Sergio SaponaraEmail author
Special Issue


The paper presents a novel technique for robust motion analysis in real automotive scenarios based on integrated Retinex-like pre-processing algorithm with block matching video motion estimator. Both algorithmic and real-time hardware design issues are discussed. The benefits of the proposed technique are manifold: the entire system is more robust; the estimated motion vectors are more reliable and less dependent on critical ambient conditions like shadows or flashes; the proposed algorithm may allow to perform motion estimation using very few bits and running as a 2- or 1-bit transform, still maintaining good performances. Real-time hardware implementation is achieved by design and synthesis in 65 nm CMOS standard-cells technology of an Application Specific Instruction-set Processor. Design optimizations for both the processing core and the memory organization are presented. With respect to the state of the art the proposed hardware implementation ensures bounded circuit complexity, low power consumption and reprogrammability of the technique.


Robust motion analysis Video processing for automotive applications Motion estimation Retinex filters Application Specific Instruction-set Processor (ASIP) Hardware for real-time processing 



This research was partially supported by grants of the University of Trieste and of the Regione Friuli-Venezia Giulia within the “Eladin 2” project.


  1. 1.
    Yamada, K., Soga, M.: A compact integrated visual motion sensor for ITS applications. IEEE Trans. Intell. Transp. Syst. 4(1), 35–42 (2003)CrossRefGoogle Scholar
  2. 2.
    Gillner, W.: Motion based vehicle detection on motorways. IEEE Int. Veh. Symp. pp. 483–487 (1995)Google Scholar
  3. 3.
    McCall, J.: Video-based lane estimation and tracking for driver assistance: survey, system, evaluation. IEEE Trans. Intell. Transp. Syst. 7(1), 20–37 (2006)CrossRefGoogle Scholar
  4. 4.
    Soga, M., Kato, T., Ohta, M., Ninomiya, Y.: Pedestrian detection with stereo vision. ICDE Workshops 2005Google Scholar
  5. 5.
    He, Z., Qin, Z., Wen, H.: Video-based measure of traffic volume parameter. In: IEEE International Conference on Automation and Logistics, pp. 421–425 (2007)Google Scholar
  6. 6.
    Boltz, S., Wolsztynski, E., Debreuve, E., Thierry, E., Barlaud, M., Pronzato, L.: A minimum-entropy procedure for robust motion estimation. In: IEEE ICIP 2006, pp. 1249–1252 (2006)Google Scholar
  7. 7.
    Barjatya, A.: Block matching algorithms for motion estimation. Matlab central: (2005)
  8. 8.
    Cheung, H.-K., Siu, W.-C., Feng, D., Wang, Z.: Retinex based motion estimation for sequences with brightness variations and its application to H.264. In: IEEE international conference on acoustics, speech and signal processing, 2008 (ICASSP 2008), pp. 1161–1164 (2008)Google Scholar
  9. 9.
    Cheung, H.-K., Siu, W.-C., Feng, D., Wan, Z.: Constrained one-bit transform for Retinex based motion estimation for sequences with brightness variations. In: International Conference on Neural Networks and Signal Processing, 2008, pp. 682–685 (2008)Google Scholar
  10. 10.
    Cheung, H.-K., Siu, W.-C., Feng, D., Wang, Z.: Windowing technique for the DCT based Retinex algorithm to handle videos with brightness variations coded using the H.264. In: IEEE ICIP 2008, pp. 2860–2863 (2008)Google Scholar
  11. 11.
    Urhan, O., Erturk, S.: Constrained one-bit transform for low complexity block motion estimation. IEEE Trans. Circ. Syst. Video Technol. 17(4), 478–482 (2007)CrossRefGoogle Scholar
  12. 12.
    Lee, S., Kim, J.-M., Chae, S.-I.: New motion estimation algorithm using adaptively quantized low bit-resolution image and its VLSI architecture for MPEG2 video encoding. IEEE Trans. Circ. Syst. Video Technol. 8(6), 734–744 (1998)CrossRefGoogle Scholar
  13. 13.
    Feng, J., Lo, K.-T., Mehrpour, H., Karbowiak, A.E.: Adaptive block matching motion estimation algorithm using bit-plane matching. IEEE ICIP 2005 3, 496–499 (1995)Google Scholar
  14. 14.
    Erturk, A., Erturk, S.: Two-bit transform for binary block motion estimation. IEEE Trans. Circ. Syst. Video Technol. 15(7), 938–946 (2005)CrossRefGoogle Scholar
  15. 15.
    Celebi, A., Urhan, O., Hamzaoglu, I., Erturk, S.: Efficient hardware implementations of low bit depth motion estimation algorithms. IEEE Signal Process. Lett. 6(6), 513–516 (2009)CrossRefGoogle Scholar
  16. 16.
    Erturk, S.: Multiplication-free one-bit transform for low-complexity block-based motion estimation. IEEE Signal Process. Lett. 14(2), 109–112 (2007)CrossRefGoogle Scholar
  17. 17.
    Land, E.H., McCann, J.: Lightness and Retinex theory. J. Opt. Soc. Am. 61(1), 1–11 (1971)Google Scholar
  18. 18.
    Funt, B., Ciurea, F., McCann, J.: Retinex in Matlab. In: Proceedings of IS&T/SID 8th Color Imaging Conference, pp. 112–121 (2000)Google Scholar
  19. 19.
    Jobson, D.J., Rahman, Z., Woodell, G.A.: Properties and performance of a center/surround Retinex. IEEE Trans. Image Process. 6(3), 451–462 (1997)CrossRefGoogle Scholar
  20. 20.
    Orsini, G., Ramponi, G., Carrai, P., Di Federico, R.: A modified Retinex for image contrast enhancement and dynamics control. In: IEEE ICIP 2003, Barcelona, Spain (2003)Google Scholar
  21. 21.
    Marsi, S., Impoco, G., Ukovich, A., Ramponi, G., Carrato, S.: Using a recursive rational filter to enhance color images. IEEE Trans. Instrum. Meas. 57, 1230–1236 (2008)CrossRefGoogle Scholar
  22. 22.
    Ramponi, G.: Polynomial and rational operators for image processing and analysis. In: Mitra, S.K., Sicuranza, G.L. (eds.) Nonlinear Image Processing. Academic Press, New York (2000)Google Scholar
  23. 23.
    Jain, A.K.: Fundamentals of Digital Image Processing. Prentice-Hall International, Englewood Cliffs, NJ (1989)Google Scholar
  24. 24.
    Saponara, S., Fanucci, L., Ramponi, G., Marsi, S.: Algorithmic and architectural design for real-time and power-efficient Retinex image/video processing. J. Real Time Image Process. 1(4), 267–283 (2007)CrossRefGoogle Scholar
  25. 25.
    Saponara, S., Fanucci, L., Ramponi, G., Marsi, S., Kammler, D., Witte, E.: Application-specific instruction-set processor for Retinex-like image and video processing. IEEE Trans. Circ. Syst II 54(7), 596–600 (2007)CrossRefGoogle Scholar
  26. 26.
    Gilge, M.: Motion estimation by scene adaptive block matching (SABM) and illumination correction. In: SPIE Conference on Image Processing Algorithms and Techniques, pp. 355–366 (1990)Google Scholar
  27. 27.
    Schliebusch, O., Chattopadhyay, A., Witte, E.M., Kammler, D., Ascheid, G., Leupers, R., Meyr, H.: Optimization techniques for ADL-driven RTL processor synthesis. In: Proceedings of IEEE Workshop on Rapid Prototyping Systems, Montreal, pp. 165–171 (2005)Google Scholar
  28. 28.
    Schliebusch, O., Chattopadhyay, A., Kammler, D., Ascheid, G., Leupers, R., Meyr, H., Kogel, T.: A framework for automated and optimized ASIP implementation supporting multiple hardware description languages. IEEE ASP-DAC 1, 280–285 (2005)CrossRefGoogle Scholar
  29. 29.
    Dinoi, L., Martini, R., Masera, G., Quaglio, F., Vacca, F.: ASIP design for partially structured LDPC codes. Electron. Lett. 42(18), 49–50 (2006)CrossRefGoogle Scholar
  30. 30.
    Momcilovic, S. et al.: Application specific instruction set processor for adaptive video motion estimation. In: Proceedings of IEEE Euromicro DSD, pp 160–167 (2006)Google Scholar
  31. 31.
    Bajot, Y., Mehrez, H.: Customizable DSP architecture for ASIP core design. In: Proceedings of ISCAS’01, pp. 302–305 (2001)Google Scholar
  32. 32.
    Kappen, G., Noll, T.: Application specific instruction processor based implementation of a GNSS receiver on an FPGA. In: IEEE DATE’06, vol. 2, pp. 1–6 (2006)Google Scholar
  33. 33.
    Lee, J., Moon, J., Heo, K., Sunwoo, M., Oh, S., Kim, I.: Implementation of application-specific DSP for OFDM systems. In: Proceedings of IEEE International Conference on Circuits and Systems (ISCAS), pp. 665–668 (2004)Google Scholar
  34. 34.
    Yue, H., Lai, M.-C., Dai, K., Wang, Z.-Y., Design of a configurable embedded processor architecture for DSP functions. In: Proceedings of IEEE ICPADS’05, pp. 27-31 (2005)Google Scholar
  35. 35.
    Chattopadhyay, A., Ahmed, W., Karuri, K., Kammler, D., Leupers, R., Ascheid, G., Meyr, H.: Design space exploration of partially reconfigurable embedded processors. In: IEEE DATE’07, pp. 1–6 (2007)Google Scholar
  36. 36.
    Peters, H., Sethuraman, R., Beric, A., Meuwissen, P., Balakrishnan, S., Pinto, C., Kruijtzer, W., Ernst, F., Alkadi, G., van Meerbergen, J., de Haan, G.: Application specific instruction-set processor template for motion estimation in video applications. IEEE Trans. Circ. aSyst. Video Technol 15, 508–527 (2005)CrossRefGoogle Scholar
  37. 37.
    Vogt, T., When, N.: A reconfigurable application specific instruction set processor for Viterbi and log-map decoding. In: IEEE Workshop on Signal Proceedings Systems Design and Implementation, pp. 142–147 (2006)Google Scholar
  38. 38.
    Goossens, G., Lanneer, D., Geurts, W., Van Praet, J.: Design of ASIPs in multi-processor SoCs using the chess/checkers retargetable tool suite. In: IEEE International Symposium on System-on-Chip, 2006, pp. 1–4Google Scholar
  39. 39.
    Saponara, S., Casula, M., Fanucci, L.: ASIP-based reconfigurable architectures for power-efficient and real-time image/video processing. J. Real Time Image Process. 3(3), 201–216 (2008)CrossRefGoogle Scholar
  40. 40.
    Fanucci, L., Saponara, S., Bertini, L.: A parametric VLSI architecture for video motion estimation. Integration VLSI J. 31(1), 79–100 (2001)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  1. 1.DEEI, University of TriesteTriesteItaly
  2. 2.Department of Information EngineeringUniversity of PisaPisaItaly

Personalised recommendations