Skip to main content
SpringerLink
Log in

On the design and implementation of a dual fisheye camera-based surveillance vision system

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image processing and computer vision have been a focus of researchers for decades in various application domains. This research is continuously rising with the rise of Artificial Intelligence in the fourth industrial revolution. One of the important digital image processing applications is to produce panorama images. The wide range of view a panorama image provides can be used in a variety range of applications which may include surveillance applications and remote robot operations. A panorama image is a combination of several individual natural looking images into a composite one to provide a wide field of view that may reach 360 degrees horizontally without any distortion. Wide-angle lenses provide a wide field of view, but using them alone does not necessarily make a panorama image. In this work the design and implementation of a wide-angle stereo vision system that suites many real-time applications is proposed. The system makes use of two wide-angle fisheye cameras where each camera covers around 170 degrees field of view. The horizontal angle between the cameras is 140 degrees. The cameras acquire the instantaneous overlapping images continuously and transmits them to a base station via a communication link. The base station calibrates, corrects, correlates and stitches the non-overlapping corrected images to a composite one. The resultant final image covers 310 degrees field of view. The system is of low computational complexity compared with previously implemented systems. It is tested on a laptop and on a standalone embedded computing device. The processing speed for the panorama image stitching including the correction of the fisheye barrel distortion on the laptop computer and the embedded computer is 11 fps, and 6 fps, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

References

  1. Abughalieh KM, Sababha BH, Rawashdeh NA (2018) A video-based object detection and tracking system for weight sensitive UAVs. Multimed Tools Appl:1–19. https://doi.org/10.1007/s11042-018-6508-1

  2. Al-Harasis R, Al-Zmaily E, Al-Bishawi H, Abu Shash J, Shreim M, Sababha BH (2015) Design and Implementation of an Autonomous UGV for the Twenty Second Intelligent Ground Vehicle Competition. The International Conference on Software Engineering, Mobile Computing and Media Informatics (SEMCMI 2015), Kuala Lumpur

  3. Bechar A, Vigneault C (2017) Agricultural robots for field operations. Part 2: Operations and systems. Biosyst Eng 153:110–128

    Article  Google Scholar 

  4. Bellas N, Chai SM, Dwyer M, Linzmeier D (2009) Real-time fisheye lens distortion correction using automatically generated streaming accelerators. In: Field Programmable Custom Computing Machines, 2009. FCCM'09. 17th IEEE Symposium on, Napa

  5. Ben-Tzvi P, Xu X (2010) An embedded feature-based stereo vision system for autonomous mobile robots. In: Robotic and Sensors Environments (ROSE), 2010 IEEE International Workshop on, Phoenix

  6. Chira IM, Chibulcutean A, Danescu RG (2010) Real-time detection of road markings for driving assistance applications. In: Computer Engineering and Systems (ICCES), 2010 International Conference on, Cairo

  7. Driscoll Jr E, Morrow H, Steinhauer AJ, Lomax WC (2002) Method and apparatus for a panoramic camera to capture a 360 degree image. U.S. Patent Patent 6,459,451

  8. Drulea M, Szakats I, Vatavu A, Nedevschi S (2014) Omnidirectional stereo vision using fisheye lenses. In: Intelligent Computer Communication and Processing (ICCP), 2014 IEEE International Conference on, Cluj Napoca, Romania

  9. Fan D-P, Cheng M-M, Liu Y, Li T, Borji A (2017) Structure-measure: A new way to evaluate foreground maps. In: Proceedings of the IEEE International Conference on Computer Vision, Venice

  10. Fan D-P, Gong C, Cao Y, Ren B, Cheng M-M, Borji A (2018) Enhanced-alignment Measure for Binary Foreground Map Evaluation. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18), Stockholm

  11. Gao L-F, Gai Y-X, Fu S (2007) Simultaneous localization and mapping for autonomous mobile robots using binocular stereo vision system. In: Mechatronics and Automation, 2007. ICMA 2007. International Conference on

  12. Gao W, Shen S (2017) Dual-Fisheye Omnidirectional Stereo. IEEE/ RSJ International Conference on Intelligent Robots and system (IROS), Vancouver

    Book  Google Scholar 

  13. Ho T, Schizas ID, Rao KR, Budagavi M (2017) 360-Degree Video Stitching for Dual-Fisheye Lens Cameras Based on Rigid Moving Least Squares. arXiv:1708.05922v1

  14. Kim H, Chae E, Jo G, Paik J (2015) Fisheye lens-based surveillance camera for wide field-of-view monitoring. In: Consumer Electronics (ICCE), 2015 IEEE International Conference on, Las Vegas

  15. Liu T-M, Ju C-C, Huang Y-H, Chang T-S, Yang K-M, Lin Y-T (2017) A 360-degree 4Kx2K Panorama Video Processing Over Smart-phones. IEEE International Conference on Consumer Electronics (ICCE)

  16. Mathworks (2009) Interpolation-methods. Available: https://www.mathworks.com/help/vision/ug/interpolation-methods.html. [Accessed 6 July 2017]

  17. National Instruments (2015) myRIO Student Embedded Device. Available: http://www.ni.com/en-lb/shop/select/myrio-student-embedded-device

  18. National Instruments. LabVIEW. Available: http://www.ni.com/en-lb/shop/labview.html

  19. T. Nishimoto and J. Yamaguchi (2007) Three dimensional measurement using fisheye stereo vision. In: SICE Annual Conference 2007, Takamatsu

  20. Ohashi A, Tanaka Y, Masuyama G, Umeda K, Fukuda D, Ogata T, Narita T, Kaneko S, Uchida Y, Irie K (2016) Fisheye stereo camera using equirectangular images. In: Mechatronics (MECATRONICS)/17th International Conference on Research and Education in Mechatronics (REM), 2016 11th France-Japan & 9th Europe-Asia Congress on, Compiegne

  21. Polydoros AS, Nalpantidis L (2017) Survey of model-based reinforcement learning: Applications on robotics. J Intell Robot Syst 86(2):153–173

    Article  Google Scholar 

  22. Rawashdeh NA, Rawashdeh OA, Sababha BH (2017) Vision-based sensing of UAV attitude and altitude from downward in-flight images. J Vib Control 23(5):827–841

    Article  Google Scholar 

  23. Rawashdeh O, Sababha B (2011) An Image-Processing-Based Gimbal System Using Fisheye Video. Computer Technology and Application Journal 2(2):85–93

    Google Scholar 

  24. Rawashdeh OA, Yang HC, AbouSleiman RD, Sababha BH (2009) Microraptor: A low-cost autonomous quadrotor system. In: ASME 2009 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference

  25. Ray LA, Schauffele CN, VanSprewenburg B (2000) Method and apparatus for capturing panoramic images with range data. U.S. Patent Patent 6,023,588

  26. Sababha BH, Al Zu'bi HM, Rawashdeh OA (2015) A rotor-tilt-free tricopter UAV: design, modelling, and stability control. International Journal of Mechatronics and Automation 5(2–3):107–113

    Article  Google Scholar 

  27. Sawalmeh HK, Bjanthala HE, Al-Lahham MM, Sababha BH (2015) A Surveillance 3D Hand-Tracking-Based Tele-Operated UGV. In: The 6th International Conference on Information and Communication Systems (ICICS2015), Amman

  28. Semwal VB, Bhushan A, Nandi GC (2013) Study of humanoid Push recovery based on experiments. In: Control, Automation, Robotics and Embedded Systems (CARE), 2013 International Conference on

  29. Semwal VB, Chakraborty P, Nandi GC (2015) Less computationally intensive fuzzy logic (type-1)-based controller for humanoid push recovery. Robot Auton Syst 63:122–135, 2015

    Article  Google Scholar 

  30. Semwal VB, Gaud N, Nandi G (2017) Human Gait State Prediction Using Cellular Automata and Classification Using ELM. In: MISP-2017

  31. Semwal VB, Katiyar SA, Chakraborty R, Nandi GC (2015) Biologically-inspired push recovery capable bipedal locomotion modeling through hybrid automata. Robot Auton Syst 70:181–190, 2015

    Article  Google Scholar 

  32. Semwal VB, Mondal K, Nandi GC (2017) Robust and accurate feature selection for humanoid push recovery and classification: deep learning approach. Neural Comput & Applic 28(3):565–574

    Article  Google Scholar 

  33. Semwal VB, Nandi GC (2016) Generation of joint trajectories using hybrid automate-based model: a rocking block-based approach. IEEE Sensors J 16(12):5805–5816

    Article  Google Scholar 

  34. Semwal VB, Raj M, Nandi GC (2015) Biometric gait identification based on a multilayer perceptron. Robot Auton Syst 65:65–75, 2016

    Article  Google Scholar 

  35. Silva LC, Petraglia MR, Petraglia A (2004) A robust method for camera calibration and 3-D reconstruction for stereo vision systems. In: Proceedings of the 12th European Signal Processing Conference, Vienna

  36. Sinclair MJ, Simmons LE (2003) Panoramic Digital Camera System and Method. US Patent Patent 6,545,701 B2

  37. Takano T, Ono S, Matsushita Y, Kawasaki H, Ikeuchi K (2015) Super resolution of fisheye images captured by on-vehicle camera for visibility support. In: Vehicular Electronics and Safety (ICVES), 2015 IEEE International Conference on, Yokohama

  38. Wachtel RA, Keable J, Paulson R (2007) Digital imaging system using overlapping images to formulate a seamless composite image and implemented using either a digital imaging sensor array. U.S. Patent Patent 7,215,364

  39. Wang W, Yan J, Xu N, Wang Y, Hsu F-H (2015) Real-time high-quality stereo vision system in FPGA. IEEE Transactions on Circuits and Systems for Video Technology 25(10):1696–1708

    Article  Google Scholar 

  40. Wei J, Li C-F, Hu S-M, Martin RR, Tai C-L (2012) Fisheye video correction. IEEE Trans Vis Comput Graph 18(10):1771–1783

    Article  Google Scholar 

  41. Zhang B, Liu N, Jiao Y, Li Y, Zhu J (2015) Research on image matching technology for the spherical stereo vision. In: Mechatronics and Automation (ICMA), 2015 IEEE International Conference on, Beijing

  42. Zhao S, Zhang B, Li L, Cao Z (2010) Fisheye lens camera system calibration and localization error analysis. In: Computer Design and Applications (ICCDA), 2010 International Conference on, Qinhuangdao

Download references

Author information

Authors and Affiliations

  1. Electrical Engineering Department, Princess Sumaya University for Technology, Amman, 11941, Jordan

    Raghad Al-Harasis

  2. Computer Engineering Department, Princess Sumaya University for Technology, Amman, 11941, Jordan

    Belal H. Sababha

Authors
  1. Raghad Al-Harasis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Belal H. Sababha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Belal H. Sababha.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Al-Harasis, R., Sababha, B.H. On the design and implementation of a dual fisheye camera-based surveillance vision system. Multimed Tools Appl 78, 22667–22689 (2019). https://doi.org/10.1007/s11042-019-7501-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7501-z

Keywords

Search

Navigation