Skip to main content
SpringerLink
Log in

Camera Calibration and Video Stabilization Framework for Robot Localization

  • Chapter
  • First Online:
Control Engineering in Robotics and Industrial Automation

Part of the book series: Studies in Systems, Decision and Control ((SSDC,volume 371))

Abstract

Two major issues in robot localization are Camera Calibration (CC) and Video Stabilization (VS). The effectiveness of CC is highly provisional based on adjusting settings, image quality, and image gradient. Recent breakthrough methods employ fixed threshold to calculate pixel difference between frames and preset variables, and neglect slope information causing blurring effect for image frame selected in CC phase. Additionally, contemporary optical flow requires expert manual setting of Gaussian pyramid parameters such as sigma, down scale factor, and number of levels, which consume a lot of time and efforts to train and measure. Apart from that, the localization key challenges of humanoid stereo vision are large motion, motion blur, and defocus blurs of image. Though state-of-the-art approaches used landmark recognition and probabilistic models to overcome those issues, yet localization accuracy is still poor due to image distortion. This work proposed a framework for robot localization via CC and VS methods and triangulation concept. The framework with Fuzzy Camera Calibration (FCC) achieved better results in re-projection error compared to s about 0.85 and 2.62 in pairs based on self-collected dataset, whereas FCC versus Ferstl scored approximately 0.21 and 0.24 in pairs using Time-of-flight camera dataset. For VS, this framework with Fuzzy Optical Flow (FOF) method achieved second rank compared to the state-of-the-art methods such as Farneback, Brox (GPU), LK (GPU), Farneback (GPU), Dual_TVL1, and Simple Flow tested on SINTEL benchmark datasets. Finally, our proposed stereo vision, localization framework also outperformed Mono Vision method vision about 4.07 cm and 61.07 cm subsequently of distance errors.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Alcantarilla, P., Stasse, O., Druon, S., Bergasa, L., Dellaert, F.: How to localize humanoids with a single camera? Autonom. Robots 34(1–2), 47–71 (2013)

    Article  Google Scholar 

  2. Amanatiadis, A., Gasteratos, A., Papadakis, S., Kaburlasos, V.: Image Stabilization in Active Robot Vision Ed.: INTECH Open Access Publisher (2010)

    Google Scholar 

  3. Audet, S., Okutomi, M.: A user-friendly method to geometrically calibrate projector-camera systems. Computer Vision and Pattern Recognition Workshops, 2009. CVPR Workshops 2009. IEEE Computer Society Conference on. pp. 47–54 (2009)

    Google Scholar 

  4. Banks, J., Corke, P.: Quantitative evaluation of matching methods and validity measures for stereo vision. Int. J. Robot. Res. 20(7), 512–532 (2001)

    Article  Google Scholar 

  5. Chen, S.Y.: Kalman filter for robot vision: a survey. IEEE Trans. Industr. Electron. 59(11), 4409–4420 (2012)

    Article  Google Scholar 

  6. Eruhimov, V.: OpenCV: Camera calibration and 3D reconstruction (2016). http://docs.opencv.org/master/d4/d94/tutorial_camera_calibration.htm1#gsc.tab=0. Accessed October 2016

  7. Feng, Y., Ren, J., Jiang, J., Halvey, M., Jose, J.: Effective venue image retrieval using robust feature extraction and model constrained matching for mobile robot localization. Mach. Vis. Appl. 23(5), 1011–1027 (2012)

    Article  Google Scholar 

  8. Feng, Y., Zoubir, A.M., Fritsche, C., Gustafsson, F.: Robust cooperative sensor network localization via the EM criterion in LOS/NLOS environments. Signal Processing Advances in Wireless Communications (SPAWC), 2013 IEEE 14th Workshop on. pp. 505–509 (2013)

    Google Scholar 

  9. Ferstl, D., Reinbacher, C., Riegler, G., Rüther, M., Bischof, H.: Learning depth calibration of time-of-flight cameras. In: Proceedings of the British Machine Vision Conference (BMVC), pp. 1–12 (2015)

    Google Scholar 

  10. Fuentes-Pacheco, J., Ruiz-Ascencio, J., Rendón-Mancha, J.: Visual simultaneous localization and mapping: a survey. Artif. Intell. Rev. 43(1), 55–81 (2015)

    Article  Google Scholar 

  11. Goncalves Lins, R., Givigi, S.N., Gardel Kurka, P.R.: Vision-based measurement for localization of objects in 3-D for robotic applications. Instrume. Measur. IEEE Trans. 64(11), 2950–2958 (2015)

    Article  Google Scholar 

  12. Gueaieb, W., Miah, M.S.: An intelligent mobile robot navigation technique using RFID technology. Instrum. Measur. IEEE Trans. 57(9), 1908–1917 (2008)

    Article  Google Scholar 

  13. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision Ed.: Cambridge University Press (2003)

    Google Scholar 

  14. Heikkila, J., Silven, O.: A four-step camera calibration procedure with implicit image correction. Computer Vision and Pattern Recognition, 1997. Proceedings, 1997 IEEE Computer Society Conference on, pp. 1106–1112 (1997)

    Google Scholar 

  15. Jia, C., Evans, B.L.: Constrained 3D rotation smoothing via global manifold regression for video stabilization. Signal Process. IEEE Trans. 62(13), 3293–3304 (2014)

    Article  MathSciNet  Google Scholar 

  16. Ko, N.Y., Kuc, T.-Y.: Fusing range measurements from ultrasonic beacons and a laser range finder for localization of a mobile robot. Sensors 15(5), 11050–11075 (2015)

    Article  Google Scholar 

  17. Koch, H., Konig, A., Weigl-Seitz, A., Kleinmann, K., Suchy, J.: Multisensor contour following with vision, force, and acceleration sensors for an industrial robot. Instrum. Measur. IEEE Trans. 62(2), 268–280 (2013)

    Article  Google Scholar 

  18. Kumar, S., Hegde, R.M.: An efficient compartmental model for real-time node tracking over cognitive wireless sensor networks. Signal Process. IEEE Trans. 63(7), 1712–1725 (2015)

    Article  MathSciNet  Google Scholar 

  19. Leitner, J., Harding, S., Frank, M., Forster, A., Schmidhuber, J.: Learning spatial object localization from vision on a humanoid robot. Int. J. Adv. Rob. Syst. 9, 1–10 (2012)

    Article  Google Scholar 

  20. Lins, R.G., Givigi, S.N., Kurka, P.R.G.: Vision-based measurement for localization of objects in 3-D for robotic applications. IEEE Trans. Instrum. Meas. 64(11), 2950–2958 (2015)

    Article  Google Scholar 

  21. Liu, S., Yuan, L., Tan, P, Sun, J.: Steadyflow: spatially smooth optical flow for video stabilization. Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pp. 4209–4216 (2014)

    Google Scholar 

  22. Murray, D., Jennings, C.: Stereo vision based mapping and navigation for mobile robots. Robotics and Automation, 1997. Proceedings., 1997 IEEE International Conference on, 2 pp. 1694–1699 (1997)

    Google Scholar 

  23. Orghidan, R., Danciu, M., Vlaicu, A., Oltean, G., Gordan, M., Florea, C.: Fuzzy versus crisp stereo calibration: a comparative study. Image and Signal Processing and Analysis (ISPA), 2011 7th International Symposium on, pp. 627–632 (2011)

    Google Scholar 

  24. PirahanSiah, F., Abdullah, S.N.H.S., Sahran, S.: Simultaneous localization and mapping trends and humanoid robot linkages. Asia-Pacific J. Inform. Technol. Multimedia 2(2), 12 (2013)

    Article  Google Scholar 

  25. PirahanSiah, F., Abdullah, S.N.H.S., Sahran, S.: Peak signal-to-noise ratio based on threshold method for image segmentation. J. Theoret. Appl. Inform. Technol. 57(2) (2013)

    Google Scholar 

  26. PirahanSiah, F., Abdullah, S.N.H.S., Sahran, S.: Adaptive Image Thresholding based on the peak signal-to-noise ratio. Res. J. Appl. Sci. Eng. Technol. (2014)

    Google Scholar 

  27. PirahanSiah, F., Abdullah, S.N.H.S., Sahran, S.: Augmented optical flow methods for video stabilization. 4th Artificial Intelligence Technology Postgraduate Seminar (CAITPS 2015). Faculty of Information Science and Technology (FTSM)—UKM on 22 and 23 December 2015. pp. 47–52 (2015)

    Google Scholar 

  28. PirahanSiah, F., Abdullah, S.N.H.S., Sahran, S.: Camera calibration for multi-modal robot vision based on image quality assessment. Control Conference (ASCC), 10th Asian, pp. 1–6 (2015)

    Google Scholar 

  29. Sarunic, P., Evans, R.: Hierarchical model predictive control of UAVs performing multitarget-multisensor tracking. Aerospace and Electron. Syst. IEEE Trans. 50(3), 2253–2268 (2014)

    Article  Google Scholar 

  30. Sciacca, L.: Distributed Electronic Warfare Sensor Networks. Association of Old Crows Convention (2002)

    Google Scholar 

  31. Shirmohammadi, S., Ferrero, A.: Camera as the instrument: the rising trend of vision-based measurement. IEEE Instrum. Meas. Mag. 17(3), 41–47 (2014)

    Google Scholar 

  32. Stein, G.P.: Accurate internal camera calibration using rotation, with analysis of sources of error. Computer Vision. Proceedings., Fifth International Conference on, pp. 230–236 (1995)

    Google Scholar 

  33. Sudin, M.N., Nasrudin, M.F., Abdullah, S.N.H.S.: Humanoid localisation in a robot soccer competition using a single camera. IEEE 10th International Colloquium on Signal Processing and Its Applications, CSPA 2014. IEEE Computer Society,  6805724, pp. 77–81 (2014)

    Google Scholar 

  34. Tsai, R.Y.: A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. Robot. Autom. IEEE J. 3(4), 323–344 (1987)

    Article  Google Scholar 

  35. Vijay, G., Ben Ali Bdira, E., Ibnkahla, M.: Cognition in wireless sensor networks: a perspective. Sensors J. IEEE 11(3), 582–592 (2011)

    Google Scholar 

  36. Walton, L., Hampshire, A., Forster, D.M.C., Kemeny, A.A.: Stereotactic localization with magnetic resonance imaging: a phantom study to compare the accuracy obtained using two-dimensional and three-dimensional data acquisitions. Neurosurgery 41(1), 131–139 (1997)

    Article  Google Scholar 

  37. Wang, J., Shi, F., Zhang, J., Liu, Y.: A new calibration model of camera lens distortion. Pattern Recogn. 41(2), 607–615 (2008)

    Article  Google Scholar 

  38. Wang, Q., Fu, L., Liu, L.: Review on camera calibration. Chinese Control and Decision Conference (CCDC), pp. 3354–3358 (2010)

    Google Scholar 

  39. Yoo, J.K., Kim, J.H.: Gaze control-based navigation architecture with a situation-specific preference approach for humanoid robots. IEEE-ASME Trans. Mechatron. 20(5), 2425–2436 (2015)

    Article  Google Scholar 

  40. Zhang, Z., Zhu, D., Zhang, J., Peng, Z.: Improved robust and accurate camera calibration method used for machine vision application. Opt. Eng. 47(11), 117201–117211 (2008)

    Google Scholar 

  41. Zhengyou, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)

    Google Scholar 

Download references

Acknowledgements

This research was initially funded by the Ministry of Education through FRGS/1/2012/SG05/UKM/02/8 entitled “Generic Object Localization Algorithm for Image Segmentation” and UKM Prime Impact Fund DIP-2015-023 entitled “Object descriptor via optimized unsupervised learning approaches.”

Author information

Authors and Affiliations

  1. Center for Artificial Intelligence Technology, Faculty of Information Science & Technology, Universiti Kebangsaan Malaysia, 43600, Bangi, Selangor, Malaysia

    Farshid Pirahansiah, Shahnorbanun Sahran & Siti Norul Huda Sheikh Abdullah

Authors
  1. Farshid Pirahansiah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shahnorbanun Sahran
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Siti Norul Huda Sheikh Abdullah
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shahnorbanun Sahran .

Editor information

Editors and Affiliations

  1. Faculty of Engineering, Universiti of Malaysia Sabah, Kota Kinabalu, Malaysia

    Muralindran Mariappan

  2. Faculty of Electrical Engineering Technology, Universiti Malaysia Perlis, Perlis, Malaysia

    Mohd Rizal Arshad

  3. School of Mechanical Engineering, The University of Adelaide, Adelaide, Australia

    Rini Akmeliawati

  4. Faculty of Electrical Engineering, Universiti Teknikal Malaysia Melaka, Durian Tunggal, Malaysia

    Chong Shin Chong

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Pirahansiah, F., Sahran, S., Abdullah, S.N.H.S. (2022). Camera Calibration and Video Stabilization Framework for Robot Localization. In: Mariappan, M., Arshad, M.R., Akmeliawati, R., Chong, C.S. (eds) Control Engineering in Robotics and Industrial Automation. Studies in Systems, Decision and Control, vol 371. Springer, Cham. https://doi.org/10.1007/978-3-030-74540-0_12

Download citation

Publish with us

Policies and ethics

Search

Navigation