Skip to main content
SpringerLink
Log in

Low-power DSP system for real-time correction of fish-eye cameras in automotive driver assistance applications

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The development of an embedded system for real-time correction of fish-eye effect is presented. The fish-eye lens is applied to driver assistance video systems because of its wide-angled view. A large field of view can reduce the number of cameras needed for video system and their cost, installation, maintenance and wiring issues. On the other hand, this lens causes inherent radial distortion to image that has to be corrected in real-time with a low-cost and low-power processing platform. This paper proposes a solution that can be easily adapted to different types of lens and camera, and meets real-time constraints with a power budget within 100 mW and a board size of few cm2. Starting from mathematical equations, given by the geometrical optics, a state-of-the-art correction method is presented, then optimizations are introduced at different levels: algorithmic level, where a real-time correction parameter calculation avoids extra non-volatile off-chip memory cards; data transfer level, where a new pixel pair management reduces memory access and storage burden; HW-SW implementation level, where a low-power board has been developed and tested in real automotive scenarios. Other applications of the developed system, such as multi-camera and multi-dimensional video systems, are finally presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

References

  1. Yamada, K., Soga, M.: “A compact integrated visual motion sensor for ITS applications”. In: IEEE transactions on intelligent transportation systems, vol. 4, no. 1, pp. 35–42 (2003)

  2. McCall, J.: “Video-based lane estimation and tracking for driver assistance: survey, system, evaluation” In: IEEE transactions on intelligent transportation systems, vol. 7, no. 1, pp. 20–37 (2006)

  3. Soga, M., Kato, T., Ohta, M., Ninomiya, Y.: “Pedestrian detection with stereo vision”, ICDE Workshops 2005

  4. He, Z. et al.: “Video-based measure of traffic volume parameter”, IEEE International conference automation and logistics, pp. 421–425 (2007)

  5. Saponara, S., et al.: Algorithmic and architectural design for real-time and power-efficient Retinex image/video processing. J. Real-Time Image Process. 1(4), 267–283 (2007)

    Article  Google Scholar 

  6. Marsi, S., et al.: Integrated video motion estimator with Retinex-like pre-processing for robust motion analysis in automotive scenarios: algorithmic and real-time architecture design. J. Real-Time Image Process. 5(4), 275–289 (2010)

    Article  Google Scholar 

  7. Maddalena, S., Darmon, A., Diels, R.: “Automotive CMOS image sensors”, VDI-Buch, advanced microsystems for automotive applications, Part 6, pp. 401–412 (2005)

  8. Römer, M., Heimann, T.: “Real-time camera link for driver assistance applications”, VDI-Buch, advanced microsystems for automotive applications, Part 3, pp. 299–310 (2009)

  9. Azzopardi, M., et al.: A high speed tri-vision system for automotive applications. Eur. Transp. Res. Rev. 2, 31–51 (2010)

    Article  Google Scholar 

  10. Manipal Dot Net with Altera Corporation: “A flexible architecture for fisheye correction in automotive rear view cameras”, white paper (2008)

  11. Manipal dot Net and Altera: “generating panoramic views by stitching multiple fisheye images”, white paper (2009)

  12. Dhane, P., et al.: “A generic non-linear method for fisheye correction”, Int. J. Comput. Appl., vol. 51, no. 10, August (2012)

  13. Wei, J., et al.: “Fisheye video correction”, IEEE Trans. Vis. Comput. Graph. 18(10), pp. 1771–1783 (2012)

    Google Scholar 

  14. Kun, B., et al.: “An image correction method of fisheye lens base on bilinear interpolation”, In: fourth international conference on intelligent computation technology and automation (2011)

  15. Hughes, C., Glavin, M., Jones, E., Denny, P.: Wide-angle camera technology for automotive applications: a review. IET Intell. Transp. Syst. 3(1), 19–31 (2009)

    Article  Google Scholar 

  16. Friel, M., Hughes, C., Denny, P., Jones, E., Glavin, M.: Automatic calibration of fish-eye cameras from automotive video sequences. IET Intell. Transp. Syst. 4(2), 136–148 (2010)

    Article  Google Scholar 

  17. Thomas, B., et al.: “Development of a cost effective bird’s eye view parking assistance system”. In: International journal of advanced research in computer science and software engineering (IJARCSSE) 2011, pp. 461–466

  18. Bellas, N.: “Real-time fisheye lens distortion correction using automatically generated streaming accelerators”. In: 17th IEEE symposium on field programmable custom computing machines (2009)

  19. Hughes, C., et al.: “Review of geometric distortion compensation in fish-eye cameras”, ISSC, pp. 162–167 (2008)

  20. Bangadkar, S., et al.: “Mapping matrix for perspective correction from fish eye distorted images”, IEEE ICRTIT, pp. 1288–1292 (2011)

  21. Saito, M., et al.: “People detection and tracking from fish-eye image based on probabilistic appearance model”, IEEE SICE, pp. 435–440 (2011)

  22. La Hung Manh et al.: “A small-scale research platform for intelligent transportation systems”, IEEE ROBIO, pp. 1373–1378 (2011)

  23. Panorama Tools [Online]. Available: http://panotools.sourceforge.net

  24. Image Trends Fisheye-Hemi Plug-In [Online]. Available: http://www.imagetrendsinc.com/products/prodpage_hemi.asp

  25. Intel Core 2 processor datasheet http://download.intel.com/design/processor/datashts/318732.pdf

  26. Xylon logiBrics Technology, logiVIEW [Online] Available: http://www.logicbricks.com/Products/logiVIEW.aspx

  27. Intersil Fisheye Image Correction Technology [Online]. Available: http://www.intersil.com/video/fisheye.asp

  28. Salomon, D.: Transformations and projections in computer graphics, Springer, Berlin, (2006)

  29. ITU Recommendation BT.601 [Online] http://www.itu.int/rec/R-REC-BT.601-7-201103-I/en

  30. TI, “TMS320DM642 Video/Imaging Fixed-Point DSP”, Oct. 2010, [Online] http://focus.ti.com/docs/prod/folders/print/tms320dm642.html

  31. TI, “TMS320DM365 Digital Media System-on-Chip”, June 2011 [Online] http://focus.ti.com/docs/prod/folders/print/tms320dm365.html

  32. TI, “TMS320DM368 Digital Media System-on-Chip”, June 2011 [Online] http://focus.ti.com/docs/prod/folders/print/tms320dm368.html

Download references

Acknowledgments

We would like to thank Prof. M. Diani who lent us the EvmDM642 board and R.I.Co. srl that supported the research.

Author information

Authors and Affiliations

  1. Department of Information Engineering, University of Pisa, Via G. Caruso 16, 56122, Pisa, PI, Italy

    Mauro Turturici, Sergio Saponara & Luca Fanucci

  2. R.I.Co. srl, Via Adriatica 17, 60022, Castelfidardo, AN, Italy

    Emilio Franchi

Authors
  1. Mauro Turturici
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sergio Saponara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luca Fanucci
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Emilio Franchi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mauro Turturici.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Turturici, M., Saponara, S., Fanucci, L. et al. Low-power DSP system for real-time correction of fish-eye cameras in automotive driver assistance applications. J Real-Time Image Proc 9, 463–478 (2014). https://doi.org/10.1007/s11554-013-0330-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-013-0330-9

Keywords

Search

Navigation