Skip to main content
SpringerLink
Log in

Sensor fusion of camera, GPS and IMU using fuzzy adaptive multiple motion models

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

A tracking system that will be used for augmented reality applications has two main requirements: accuracy and frame rate. The first requirement is related to the performance of the pose estimation algorithm and how accurately the tracking system can find the position and orientation of the user in the environment. Accuracy problems of current tracking devices, considering that they are low-cost devices, cause static errors during this motion estimation process. The second requirement is related to dynamic errors (the end-to-end system delay, occurring because of the delay in estimating the motion of the user and displaying images based on this estimate). This paper investigates combining the vision-based estimates with measurements from other sensors, GPS and IMU, in order to improve the tracking accuracy in outdoor environments. The idea of using Fuzzy Adaptive Multiple Models was investigated using a novel fuzzy rule-based approach to decide on the model that results in improved accuracy and faster convergence for the fusion filter. Results show that the developed tracking system is more accurate than a conventional GPS–IMU fusion approach due to additional estimates from a camera and fuzzy motion models. The paper also presents an application in cultural heritage context running at modest frame rates due to the design of the fusion algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. http://www.phidgets.com/.

References

  • Agrawal M, Konolige K, Bolles R (2007) Localization and mapping for autonomous navigation in outdoor terrains: a stereo vision approach. In: IEEE workshop on applications of computer vision, pp 7–12

  • Almagbile A, Wang J, Ding W (2010) Evaluating the performances of adaptive kalman filter methods in GPS/INS integration. J Glob Position Syst 9(1):33–40

    Article  Google Scholar 

  • Armesto L, Tornero J, Vincze M (2007) Fast ego-motion estimation with multi-rate fusion of inertial and vision. Int J Robot Res 26:577–589

  • Azuma R (1997) A survey of augmented reality. Presence Teleoperators Virtual Environ 6:355–385

    Article  Google Scholar 

  • Bleser G (2009) Towards visual-inertial slam for mobile augmented reality. Ph.D. thesis, Fachbereich Informatik der Technischen Universitt Kaiserslautern

  • Bostanci E, Clark A, Kanwal N (2012) Vision-based user tracking for outdoor augmented reality. In: 2012 IEEE symposium on computers and communications (ISCC), pp 566–568

  • Bostanci E, Kanwal N, Ehsan S, Clark AF (2013) User tracking methods for augmented reality. Int J Comput Theory Eng 5(1):93–98

    Article  Google Scholar 

  • Broll W, Lindt I, Herbst I, Ohlenburg J, Braun A, Wetzel R (2008) Toward next-gen mobile are games. IEEE computer graphics and applications, pp 40–48

  • Chen J, Pinz A (2004) Structure and motion by fusion of inertial and vision-based tracking. Austrian Assoc Pattern Recognit 179:75-62

    Google Scholar 

  • Chroust SG, Vincze M (2004) Fusion of vision and inertial data for motion and structure estimation. J Robotic Syst 21(2):73–83

    Article  Google Scholar 

  • Civera J, Davison A, Montiel JMM (2008) Interacting multiple model monocular slam. In: International conference on robotics and automation, pp 3704–3709

  • Groves P (2008) Principles of GNSS, inertial, and multi-sensor integrated navigation systems. GNSS technology and applications series, Artech House

  • Hong SK (2003) Fuzzy logic based closed-loop strapdown attitude system for unmanned aerial vehicle (uav). Sens Actuators A Phys 107(2):109–118

    Article  Google Scholar 

  • Kanatani K (2004) Uncertainty modeling and model selection for geometric inference. IEEE Trans Pattern Anal Mach Intell 26(10):1307–1319

  • Kaplan E, Hegarty C (2005) Understanding GPS: principles and applications. Artech house mobile communications series, Artech house

  • Kleeman L (2013) Understanding and applying kalman filtering. http://www.ecse.monash.edu.au/centres/irrc/LKPubs/Kalman.PDF. Accessed Sep 2013

  • Konolige K, Agrawal M, Bolles R, Cowan C, Fischler M, Gerkey B (2008) Outdoor mapping and navigation using stereo vision. Exp Robot 39:179–190

    Article  Google Scholar 

  • Kramer J, Kandel A (2012) On accurate localization and uncertain sensors. Int J Intell Syst 27(5):429–456

    Article  Google Scholar 

  • Lang P, Kusej A, Pinz A, Brasseur G (2002) Inertial tracking for mobile augmented reality. In: IEEE instruments and measurement technology conference

  • Madgwick S, Vaidyanathan R, Harrison A (2010) An efficient orientation filter for inertial measurement units (IMUs) and magnetic angular rate and gravity (MARG) sensor arrays. Technical report, Department of Mechanical Engineering

  • Ojeda L, Borenstein J (2002) Flexnav: fuzzy logic expert rule-based position estimation for mobile robots on rugged terrain. In: IEEE international conference on robotics and automation, 2002. Proceedings. ICRA ’02, vol 1, pp 317–322

  • Oskiper T, Samarasekera S, Kumar R (2012) Multi-sensor navigation algorithm using monocular camera, imu and gps for large scale augmented reality. In: 2012 IEEE international symposium on mixed and augmented reality (ISMAR), pp 71–80

  • Read P, Meyer M (2000) Restoration of motion picture film. Elsevier Science, Butterworth-Heinemann series in conservation and museology

  • Reid I (2012) Estimation II. http://www.robots.ox.ac.uk/~ian/Teaching/Estimation/LectureNotes2.pdf. Accessed Sep 2013

  • Ribo M, Lang P, Ganster H, Brandner M, Stock C, Pinz A (2002) Hybrid tracking for outdoor augmented reality applications. IEEE computer graphics and applications, pp 54–63

  • Ross T (2009) Fuzzy logic with engineering applications. Wiley, New York

    Google Scholar 

  • Schindler K, Suter D (2006) Two-view multibody structure-and-motion with outliers through model selection. IEEE Trans Pattern Anal Mach Intell 28(6):983–995

  • Schleicher D, Bergasa LM, Ocana M, Barea R, Lopez M (2009) Real-time hierarchical outdoor slam based on stereovision and gps fusion. IEEE Trans Intell Transp Syst 10:440–452

    Article  Google Scholar 

  • Seo J, Lee JG, Park C (2005) Leverarm compensation for integrated navigation system of land vehicles. In: Proceedings of 2005 IEEE conference on control applications, 2005. CCA 2005, pp 523–528

  • Thrun S, Burgard W, Fox D (2006) Probabilistic robotics. MIT Press, New York

    MATH  Google Scholar 

  • Tornqvist D, Schon TB, Karlsson R, Gustafsson F (2009) Particle filter slam with high dimensional vehicle model. J Intell Robot Syst 55:249–266

    Article  MATH  Google Scholar 

  • Torr P (2002) Bayesian model estimation and selection for epipolar geometry and generic manifold fitting. Int J Comput Vision 50(1):35–61

    Article  MATH  Google Scholar 

  • Tseng C, Chang C, Jwo D (2011) Fuzzy adaptive interacting multiple model nonlinear filter for integrated navigation sensor fusion. Sensors 11:2090–2111

    Article  Google Scholar 

  • You S, Neumann U, Azuma R (1999) Orientation tracking for outdoor augmented reality registration. IEEE virtual reality, pp 36–42

  • Zadeh LA (1994) Fuzzy logic, neural networks, and soft computing. Commun ACM 37(3):77–84

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering Department, Ankara University, Golbasi, Ankara, Turkey

    Erkan Bostanci

  2. HAVELSAN Inc., Ankara, Turkey

    Betul Bostanci

  3. Lahore College for Women University, Lahore, Pakistan

    Nadia Kanwal

  4. School of Computer Science and Electronic Engineering, University of Essex, Essex, UK

    Adrian F. Clark

Authors
  1. Erkan Bostanci
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Betul Bostanci
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nadia Kanwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Adrian F. Clark
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Erkan Bostanci.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bostanci, E., Bostanci, B., Kanwal, N. et al. Sensor fusion of camera, GPS and IMU using fuzzy adaptive multiple motion models. Soft Comput 22, 2619–2632 (2018). https://doi.org/10.1007/s00500-017-2516-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-017-2516-8

Keywords

Search

Navigation