Skip to main content
SpringerLink
Log in

Rapid self-localization of robot based on omnidirectional vision technology

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

In this paper, we propose a self-localization method for a soccer robot using an omnidirectional camera. Based on the projective geometry of the omnidirectional visual system, the image distortion from the original omnidirectional image can be completely corrected, so the robot can quickly localize itself on the playing field. First, we transform the distorted omnidirectional image to a distortion-free unwrapped image of the soccer field by projective geometry. The obtained image makes the sequent field recognizable and the self-localization of the robot more convenient and accurate. Then, by geometric invariants, the correspondence between the unwrapped image and the model of the playing field is constructed. Next, the homography theory is applied to get the precise location and orientation of the robot. The simulation and experimental results show that the proposed method can quickly and accurately determine the position and azimuth of the soccer robot and the distance between two objects on the playing field.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27

Similar content being viewed by others

References

  1. Chiang, S.-Y., Lin, K.-Y., Chia, T.-L.: Self-localization for an autonomous mobile robot based on an omnidirectional vision system. In: Proceedings of the SPIE-IS&T Electronic Imaging, SPIE, Vol. 9025 (2014)

  2. Yagi, Y., Yachida, M.: Real-time omnidirectional image sensors. Int. J. Comput. Vis. 58(3), 173–207 (2001)

    Google Scholar 

  3. Abdul, B., Robert, S., Gregor, N.: Line-based landmark recognition for self-localization of soccer robots. In: Proceedings of the 2005 IEEE International Conference on Emerging Technologies, Islamabad, pp. 132–137 (2005)

  4. Tan, K.H., Hua, H., Ahuja, N.: Multi-view panoramic cameras using mirror pyramids. IEEE Trans. Pattern Anal. Mach. Intell. 26(7), 941–946 (2004)

    Google Scholar 

  5. Kang, S.B., Szeliski, R.: 3-D scene data recovery using omnidirectional multi-baseline stereo. Int. J. Comput. Vis. 25(2), 167–183 (1997)

    Google Scholar 

  6. Ishiguro, H., Yamamoto, M., Tsuji, S.: Omnidirectional stereo. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 257–262 (1992)

    Google Scholar 

  7. Spacek, L., Burbridge, C.: Instantaneous robot self-localization and motion estimation with omnidirectional vision. Robot. Auton. Syst. 55(2007), 667–674 (2007)

    Google Scholar 

  8. Gaspar, J., Winters, N., Santos-Victor, J.: Vision-based navigation and environmental representations with an omnidirectional camera. IEEE Trans. Robot. Autom. 16(6), 890–898 (2000)

    Google Scholar 

  9. Luan, X., Chen, M., Qi, W.W., Ge, L.H., Song, D.L.: A target localization method for soccer robot based on omnidirectional vision. In: Proceedings of the 2nd International Conference on Information Science and Engineering, pp. 1803–1806 (2010)

  10. Joko, H., Vavilin, A., Jo, K.H.: A geometry-based 3D reconstruction from a single omnidirectional image. In: Proceedings of the 19th Korea–Japan Joint Workshop on Frontiers of Computer Vision, pp. 295–299 (2013)

  11. Geyer C., Daniilidis, K.: A unifying theory for central panoramic systems and practical implications. In: Proceedings of the European Conference on Computer Vision, pp. 445–461 (2000)

  12. Censi, A., Scaramuzza, D.: Calibration by correlation using metric embedding from non-metric similarities. IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2357–2370 (2013)

    Google Scholar 

  13. Deng, X.M., Wu, F.C., Wu, Y.H.: An easy calibration method for central catadioptric cameras. Acta Autom. Sin. 33(8), 801–808 (2007)

    Google Scholar 

  14. Cheng, W., Gu, S., Chen, Q.: A novel parameter estimation algorithm based on RANSAC for precise omnidirectional image unwarping. In: Proceedings of the International Conference on Information Science and Technology, Nanjing, Jiangsu, China, pp. 717–720 (2011)

  15. Scaramuzza, D., Martinelli, A., Siegwart, R.: A flexible technique for accurate omnidirectional camera calibration and structure from motion. In: Proceedings of the IEEE International Conference on Computer Vision Systems, pp. 45–52 (2006)

  16. Puig, L., Bermúdez, J., Sturm, P., Guerrero, J.J.: Calibration of omnidirectional cameras in practice: a comparison of methods. Comput. Vis. Image Underst. 116(1), 120–137 (2012)

    Google Scholar 

  17. Puig, L., Bastanlar, Y., Sturm, P., Guerrero, J.J., Barreto, J.: Calibration of central catadioptric cameras using a DLT-like approach. Int. J. Comput. Vis. 93(2011), 101–114 (2011)

    Google Scholar 

  18. Mei, C., Rives, P.: Single view point omnidirectional camera calibration from planar grids. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 3945–3950 (2007)

  19. Rameau, F., Sidibe, D.D., Demonceaux, C., Fofi, D.: Visual tracking with omnidirectional cameras: an efficient approach. IET Electron. Lett. 47(21), 1183–1184 (2011)

    Google Scholar 

  20. Arican, Z., Frossard, P.: Joint registration and super-resolution with omnidirectional images. IEEE Trans. Image Process. 20(11), 3151–3162 (2011)

    MathSciNet  MATH  Google Scholar 

  21. Baker, S., Nayar, S.K.: A theory of single-viewpoint catadioptric image formation. Int. J. Comput. Vis. 35(2), 175–196 (1999)

    Google Scholar 

  22. Wong, W.K., Liew, J.T.Y., Loo, C.K., Wong, W.K.: Omnidirectional surveillance system for digital home security. In: Proceedings of the International Conference on Signal Acquisition and Processing, pp. 8–12 (2009)

  23. Neves, A.J.R, Corrente, G.A., Pinho, A.J.: An omnidirectional vision system for soccer robots. In: Neves, J., Santos, M.F., Machado, J.M. (Eds.) Proceedings of the 13th Portuguese Conference on Artificial Intelligence, Progress in Artificial Intelligence (EPIA 2007), Lecture Notes in Computer Science, Vol. 4874 (2007)

  24. Neves, A.J.R., Pinho, A.J., Martins, D.A., Cunha, B.: An efficient omnidirectional vision system for soccer robots: from calibration to object detection. Mechatronics 21(2), 399–410 (2011)

    Google Scholar 

  25. Lu, H., Yang, S., Zhang, H., Zheng, Z.: A robust omnidirectional vision sensor for soccer robots. Mechatronics 21(2011), 373–389 (2011)

    Google Scholar 

  26. Lima, P., Bonarini, A., Machado, C., Marchese, F., Marques, C., Ribeiro, F., Sorrenti, D.: Omni-directional catadioptric vision for soccer robots. Robot. Auton. Syst. 36, 87–102 (2001)

    MATH  Google Scholar 

  27. Kasaei, S.H., Kasaei, S.M., Kasaei, S.A., Monadjemi, S.A.H., Taheri, M.: Modeling and implementation of a fully autonomous soccer robot based on omni-directional vision system. Ind. Robot Int. J. 37(3), 279–286 (2010)

    Google Scholar 

  28. Lu, H., Li, X., Zhang, H., Hu, M., Zheng, Z.: Robust and real-time self-localization based on omnidirectional vision for soccer robots. Adv. Robot. 27(10), 799–811 (2013)

    Google Scholar 

  29. Sturm, P., Ramalingam, S., Tardif, J.-P., Gasparini, S., Barreto, J.: Camera models and fundamental concepts used in geometric computer vision. Found. Trends Comput. Graph. Vis. 6(1–2), 1–183 (2011)

    Google Scholar 

  30. Kao, Y.-F., Chien, Y.-H., Li, I-H., Wang, W.-Y., Lee, T.-T.: Design and implementation of adaptive dynamic controllers for wheeled mobile robots. In: IEEE International Conference on System Science and Engineering, pp. 195–199 (2013)

  31. Direkoglu, C., Sah, M., O’Connor, N.E.: Player detection in field sports. Mach. Vis. Appl. 29(2), 187–206 (2018)

    Google Scholar 

  32. Drawil, N.M., Amar, H.M., Basir, O.A.: GPS localization accuracy classification: a context-based approach. IEEE Trans. Intell. Transp. Syst. 14(1), 262–273 (2013)

    Google Scholar 

  33. Kalman, R.E.: A new approach to linear filtering and prediction problems. J. Basic Eng. 82(1), 35–45 (1960)

    MathSciNet  Google Scholar 

  34. Lu, W., Zhang, J., Zhao, X., Wang, J., Dang, J.: Multimodal sensory fusion for soccer robot self-localization based on long short-term memory recurrent neural network. J. Ambient Intell. Humaniz. Comput. 8(6), 885–893 (2017)

    Google Scholar 

  35. Dellaert, F., Fox, D., Burgard, W., Thrun, S.: Monte Carlo localization for mobile robots. In: IEEE International Conference on Robotics and Automation, pp. 1322–1328 (1999)

  36. Kim, D.O., Lee, D.Y., Oh, J.I., Kang, T.H., Kang, T.K.: Local obstacle avoidance using obstacle-dependent Gaussian potential field for robot soccer. Robot Intell. Technol. Appl. 4, 539–549 (2017)

    Google Scholar 

  37. Vivacqua, R.P.D., Bertozzi, M., Cerri, P., Martins, F.N., Vassallo, R.F.: Self-localization based on visual lane marking maps: an accurate low-cost approach for autonomous driving. IEEE Trans. Intell. Transp. Syst. 19(2), 582–597 (2017)

    Google Scholar 

  38. Cai, H., Hu, Z., Huang, G., Zhu, D., Su, X.: Integration of GPS, monocular vision, and high definition (HD) map for accurate vehicle localization. Sensors 18(10), 3270–3285 (2018)

    Google Scholar 

  39. Luo, R., Min, H.: A new omni-vision based self-localization method for soccer robot. In: 2009 WRI World Congress on Software Engineering, Vol. 1, pp. 126–130 (2009)

  40. Fox, D., Burgard, W., Thrun, S.: Markov localization for mobile robots in dynamic environments. J. Artif. Intell. Res. 11, 391–427 (1999)

    MATH  Google Scholar 

  41. Hsu, C.-C., Wong, C.-C., Teng, H.-C., Li, N.-J., Ho, C.-Y.: Localization of mobile robots via an enhanced particle filter. In: Proceedings of the IEEE on Instrumentation and Measurement Technology Conference (2010)

  42. Sobreira, H., Costa, C.M., Sousa, I., Rocha, L., Lima, J., Farias, P.C.M.A., Costa, P., Moreira, A.P.: Map-matching algorithms for robot self-localization: a comparison between perfect match, iterative closest point and normal distributions transform. J. Intell. Robot. Syst. 93(3–4), 533–546 (2019)

    Google Scholar 

  43. Yu, H., Fu, Q., Yang, Z., Tan, L., Sun, W., Sun, M.: Robust robot pose estimation for challenging scenes with an RGB-D camera. IEEE Sens. J. 19(6), 2217–2229 (2018)

    Google Scholar 

  44. Wong, K., Javanmardi, E., Javanmardi, M., Kamijo, S.: Estimating autonomous vehicle localization error using 2D geographic information. ISPRS Int. J. Geo-Inf. 8(6), 288–310 (2019)

    Google Scholar 

  45. Aria, M.: Real-time 2D mapping and localization algorithms for mobile robot applications. IOP Conf. Ser. Mater. Sci. Eng. 662(2), 1–7 (2019)

    Google Scholar 

  46. Konečný, J., Krömer, P., Prauzek, M., Musilek, P.: Scan matching by cross-correlation and differential evolution. Electronics 8, 856–875 (2019)

    Google Scholar 

  47. Wu, P., Li, W., Yan, M.: 3D scene reconstruction based on improved ICP algorithm. Microprocess. Microsyst. 75, 1–8 (2020)

    Google Scholar 

  48. Ye, C., Chen, G., Qu, S., Yang, Q., Chen, K., Du, J., Hu, R.: Self-localization of parking robots using square-like landmarks. In: 2018 IEEE International Conference on Robotics and Biomimetics, pp. 1987–1992 (2018)

  49. Loesch, A., Bourgeois, S., Gay-Bellile, V., Gomez, O., Dhome, M.: Localization of 3D objects using model-constrained SLAM. Mach. Vis. Appl. 29(7), 1041–1068 (2018)

    Google Scholar 

  50. Belter, D., Nowicki, M., Skrzypczyński, P.: Modeling spatial uncertainty of point features in feature-based RGB-D SLAM. Mach. Vis. Appl. 29(5), 827–844 (2018)

    Google Scholar 

  51. Wang, M.L., Wu, H.S., He, C.H., Huang, Q.T., Lin, H.Y.: Geometric constraints for robot navigation using omnidirectional camera. In: Proceedings of the 2012 IEEE International Conference on Systems, Man, and Cybernetics, Seoul, Korea, pp. 1724–1729 (2012)

  52. Liu, B., Fan, J., Zhou, J.: A self-localization method through pose point matching for autonomous soccer robot based on omni-vision. In: Proceedings of the 9th International Conference on Electronic Measurement and Instruments, Hohai University, Beijing, pp. 4-246–4-249 (2009)

  53. Liu, J., Yin, B., Liao, X.: Robot self-localization with optimized error minimizing for soccer contest. J. Comput. 6(7), 1485–1492 (2011)

    Google Scholar 

  54. Kim, H., Oh, T., Lee, D. Myung, H.: Image-based localization using prior map database and Monte Carlo localization. In: The 11th International Conference on Ubiquitous Robots and Ambient Intelligence, pp. 308–310 (2014)

  55. Hsu, C.-C., Wong, C.-C., Teng, H.-C., Ho, C.-Y.: Dual-circle self-localization for soccer robots with omnidirectional vision. J. Chin. Inst. Eng. 35(6), 619–631 (2012)

    Google Scholar 

  56. Jang, G., Kim, S., Kim, J., Kweon, I.: Metric localization using a single artificial landmark for indoor mobile robots. In: Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2857–2862 (2005)

  57. Luo, R., Min, H.: A new omni-vision based self-localization method for soccer robot. In: Proceedings of the WRI World Congress on Software Engineering, Xiamen, pp. 126–130 (2009)

  58. Yuan, P.H., Yang, K.F., Tsai, W.H.: Real-time security monitoring around a video surveillance vehicle with a pair of two-camera omni-imaging devices. IEEE Trans. Veh. Technol. 60(8), 3603–3614 (2011)

    Google Scholar 

  59. Mundy, J.L., Zisserman, A.: Geometric Invariance in Computer Vision. MIT Press, London (1992)

    Google Scholar 

  60. Lei, G.: Recognition of planar objects in 3-D space from single perspective views using cross ratio. IEEE Trans. Robot. Autom. 6(4), 432–437 (1990)

    MathSciNet  Google Scholar 

  61. Griminisi, A.: Accurate Visual Metrology from Single and Multiple Uncalibrated Images. Springer, Berlin (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer and Communication Engineering, Ming Chuan University, Taoyuan City, 333, Taiwan

    Tsorng-Lin Chia

  2. Department of Information and Telecommunications Engineering, Ming Chuan University, Taoyuan City, 333, Taiwan

    Shu-Yin Chiang

  3. College of Artificial Intelligence, Yango University, Fuzhou, China

    Chaur-Heh Hsieh

Authors
  1. Tsorng-Lin Chia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shu-Yin Chiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chaur-Heh Hsieh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shu-Yin Chiang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chia, TL., Chiang, SY. & Hsieh, CH. Rapid self-localization of robot based on omnidirectional vision technology. Machine Vision and Applications 31, 74 (2020). https://doi.org/10.1007/s00138-020-01129-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-020-01129-7

Keywords

Search

Navigation