Journal of Real-Time Image Processing

, Volume 9, Issue 1, pp 171–185 | Cite as

On-chip semidense representation map for dense visual features driven by attention processes

  • Sara GranadosEmail author
  • Francisco Barranco
  • Sonia Mota
  • Javier Díaz
  • Eduardo Ros
Special Issue


We describe an intelligent scheme to condense dense vision features, efficiently reducing the size of representation maps and keeping relevant information for further processing during subsequent stages. We have integrated our condensation algorithm in a low-level-vision system that obtains several vision-features in real-time working on an FPGA. Within this framework, our condensation algorithm allows for the transfer of information from the FPGA device (or processing chip) to any co-processor (from embedded ones to external PCs or DSPs) under technological constraints (such as bandwidth, memory and performance ones). Our condensation core processes 1024 × 1024 resolution images at up to 90 fps. Hence, our condensation module performs this process introducing an insignificant delay in the vision system. A hardware implementation usually implies a simplified version of the vision-feature extractor. Therefore, our condensation process inherently regularizes low-level-vision features, effectively reducing discontinuities and errors. The semidense representation obtained is compatible with mid-/high-level-vision modules, usually implemented as software components. In addition, our versatile semidense map is ready to receive feedback from attention processes, integrating task-driven attention (i.e. top-down information) in real time. Thus, the main advantages of this core are: real-time throughput, versatility, inherent regularization, scalability and feedback from other stages.


Visual features Sparse representation  FPGA Real time  Image processing 



The authors thank A. L. Tate for revising their English text.


  1. 1.
    Anguita. M., Diaz, J., Ros, E., Fernandez-Baldomero, F. Optimization strategies for high-performance computing of optical-flow in general-purpose processors. IEEE Trans on CSVT 19(10):1475–1488 (2009). doi: 10.1109/TCSVT.2009.2026821 Google Scholar
  2. 2.
    Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M., Szeliski, R. A database and evaluation methodology for optical flow. Int. J. Comput. Vis. 92(1):1–31 (2011). doi: 10.1007/s11263-010-0390-2 Google Scholar
  3. 3.
    Barranco, F., Tomasi, M., Diaz, J., Vanegas, M., Ros, E. Pipelined architecture for real-time cost-optimized extraction of visual primitives based on fpgas. Digital Signal Process. (2012). doi: 10.1016/j.dsp.2012.09.017
  4. 4.
    Bay, H., Ess, A,, Tuytelaars, T., Van-Gool, L. Speeded-up robust features (surf). Comput. Vis. Image Underst. 110:346–359 (2008). doi: 10.1016/j.cviu.2007.09.014 Google Scholar
  5. 5.
    Benoit, A., Caplier, A., Durette, B., Herault, J. Using human visual system modeling for bio-inspired low level image processing. Comput. Vis. Image Underst. 114(7):758–773 (2010). doi: 10.1016/j.cviu.2010.01.011 Google Scholar
  6. 6.
    Boahen, K. A burst-mode word-serial address-event linki: Transmitter design. IEEE Trans Circuits Syst I, Reg Papers, pp. 1269–1280 (2004)Google Scholar
  7. 7.
    Camus, T. Calculating time-to-contact using real-time quantized optical flow. In: National Institute of Standards and Technology NISTIR 5609 (1995)Google Scholar
  8. 8.
    Canny, J. A computational approach to edge detection. IEEE Trans PAMI 8(6):679–698 (1986). doi: 10.1109/TPAMI.1986.4767851 Google Scholar
  9. 9.
    Chessa, M., Solari, F., Sabatini, S.P., Bisio, G.M. Motion interpretation using adjustable linear models. In: BMVC (2008)Google Scholar
  10. 10.
    core OCJH (2012), jpeg online
  11. 11.
    Daz, J., Ros, E., Mota, S., Carrillo, R. Local image phase, energy and orientation extraction using FPGAs. Int. J. Electron. 95:743–760 (2008). doi: 10.1080/00207210801941200 Google Scholar
  12. 12.
    Dong, F., Ye, X. Multiscaled texture synthesis using multisized pixel neighborhoods. IEEE Comput. Graph Appl. 27:41–47 (2007). doi: 10.1109/MCG.2007.66 Google Scholar
  13. 13.
    Gai, J., Stevenson, R. Optical flow estimation with p-harmonic regularization. In: Image Processing (ICIP), 2010 17th IEEE International Conference on, pp. 1969–1972 (2010)Google Scholar
  14. 14.
    Galić, I., Weickert, J., Welk, M., Bruhn, A., Belyaev, A., Seidel, H. Image compression with anisotropic diffusion. J. Math. Imag. Vis. 31:255–269 (2008). doi: 10.1007/s10851-008-0087-0 Google Scholar
  15. 15.
    Granados, S., Chumerin, N., Mota, S., Diaz, J. Obstacle detection using semidense representation maps. J. Vis. Commun. Image Represent. (2012) (submitted)Google Scholar
  16. 16.
    Itti, L., Koch, C. Computational modelling of visual attention. Nat. Rev. Neurosci. 2(3):194–203 (2001)Google Scholar
  17. 17.
    Kolb, H., Lipetz, L. The anatomical basis for colour vision in the vertebrate retina. Vis. Vis. Dysfunct. Percept. colour 6:128–145 (1991)Google Scholar
  18. 18.
    Kruger, N., Felsberg, M. A continuous formulation of intrinsic dimension. In: Proceedings of the British Machine Vision Conference (2003)Google Scholar
  19. 19.
    Kruger, N., Potzsch, M., Peters, G. Principles of Cortical Processing Applied to and Motivated by Artificial Object Recognition. Cambridge University Press, Cambridge (2000)Google Scholar
  20. 20.
    Lowe, D. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60:91–110 (2004). doi: 10.1023/B:VISI.0000029664.99615.94 Google Scholar
  21. 21.
    Luettgen, M., Clem-Karl, W., Willsky, A. Efficient multiscale regularization with applications to the computation of optical flow. IEEE Trans. Image Process. 3(1):41–64 (1994). doi: 10.1109/83.265979 Google Scholar
  22. 22.
    Mikolajczyk, K., Schmid, C.A performance evaluation of local descriptors. IEEE Trans. PAMI. 27(10), 1615–1630. doi: 10.1109/TPAMI.2005.188
  23. 23.
    Ortigosa, E., Cañas, A., Ros, E., Martínez-Ortigosa, P., Mota, S., Díaz, J. Hardware description of multi-layer perceptrons with different abstraction levels. Microprocess. Microsyst. 30(7):435–444 (2006)Google Scholar
  24. 24.
    Pauwels, K., Kruger, N., Lappe, M., Worgotter, F., Van-Hulle, M. A cortical architecture on parallel hardware for motion processing in real time. J. Vis. 10(10), Art No 18, 1–21 (2010). doi: 10.1167/10.10.18 Google Scholar
  25. 25.
    Pauwels, K., Tomasi, M., Diaz-Alonso, J., Ros, E., Van-Hulle, M. A comparison of fpga and gpu for real-time phase-based optical flow, stereo, and local image features. IEEE Trans. Comput. 61(7):999–1012 (2012). doi: 10.1109/TC.2011.120
  26. 26.
    Pugeault, N., Wörgötter, F., Krüger, N. Visual primitives: local, condensed, semantically rich visual descriptors and their applications in robotics. Int. J. Humanoid Robotics. 7(3):379–405 (2010)Google Scholar
  27. 27.
    Ralli, J., Díaz, J., Ros, E. A method for sparse disparity densification using voting mask propagation. J. Vis. Comput. Image Rep. 21(1):67–74 (2009). doi: 10.1016/j.jvcir.2009.08.005 Google Scholar
  28. 28.
    Sabatini, S., Gastaldi, G., Solari, F., Pauwels, K., Van-Hulle, M., Diaz, J., Ros, E., Pugeault, N., Krüger, N. A compact harmonic code for early vision based on anisotropic frequency channels. Comput. Vis. Image Underst. 114:681–699 (2010). doi:  10.1016/j.cviu.2010.03.008 Google Scholar
  29. 29.
    Sanger, T. Stereo disparity computation using gabor filters. Biol. Cybern. 59:405–418 (1988). doi: 10.1007/BF00336114 Google Scholar
  30. 30.
    Sayood, K. Introduction to data compression. Morgan Kaufmann Publishers Inc., San Francisco (2000)Google Scholar
  31. 31.
    Shi, J., Tomasi, C. Good features to track. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp. 593–600 (1994). doi: 10.1109/CVPR.1994.323794
  32. 32.
    Solutions S (2012) online
  33. 33.
    Tomasi, M., Vanegas, M., Barranco, F., Diaz, J., Ros, E. Massive parallel-hardware architecture for multi-scale stereo, optical flow and image-structure computation. IEEE Trans. CSVT 22, 282–294 (2011a). doi: 10.1109/TCSVT.2011.2162260
  34. 34.
    Tomasi, M., Vanegas, M., Barranco, F., Diaz, J., Ros, E. Real-time architecture for a robust multi-scale stereo engine on fpga. IEEE Trans VLSI Systems 20, 2208–2219 (2011b). doi: 10.1109/TVLSI.2011.2172007
  35. 35.
    Vanegas, M., Tomasi, M., Díaz, J., Vidal, E.R. Multi-port abstraction layer for fpga intensive memory exploitation applications. J. Syst. Architect. Embedded Syst. Des. 56(9):442–451 (2010)Google Scholar
  36. 36.
    Webpage D (2012) DRIVSCO European Project.
  37. 37.
    Weems, C. Jr. Real-time considerations in the design of the image understanding architecture. Real Time Imag. 2:341–350 (1996). doi: 10.1006/rtim.1996.0035
  38. 38.
    Xilinx (2012) online

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Sara Granados
    • 1
    Email author
  • Francisco Barranco
    • 1
  • Sonia Mota
    • 2
  • Javier Díaz
    • 1
  • Eduardo Ros
    • 1
  1. 1.Department of Computer Architecture and Computer Technology, CITIC-UGRUniversity of GranadaGranadaSpain
  2. 2.Department of Signal Theory, Telematics and Communications, CITIC-UGRUniversity of GranadaGranadaSpain

Personalised recommendations