Soft Computing

, Volume 22, Issue 8, pp 2619–2632 | Cite as

Sensor fusion of camera, GPS and IMU using fuzzy adaptive multiple motion models

  • Erkan Bostanci
  • Betul Bostanci
  • Nadia Kanwal
  • Adrian F. Clark
Methodologies and Application


A tracking system that will be used for augmented reality applications has two main requirements: accuracy and frame rate. The first requirement is related to the performance of the pose estimation algorithm and how accurately the tracking system can find the position and orientation of the user in the environment. Accuracy problems of current tracking devices, considering that they are low-cost devices, cause static errors during this motion estimation process. The second requirement is related to dynamic errors (the end-to-end system delay, occurring because of the delay in estimating the motion of the user and displaying images based on this estimate). This paper investigates combining the vision-based estimates with measurements from other sensors, GPS and IMU, in order to improve the tracking accuracy in outdoor environments. The idea of using Fuzzy Adaptive Multiple Models was investigated using a novel fuzzy rule-based approach to decide on the model that results in improved accuracy and faster convergence for the fusion filter. Results show that the developed tracking system is more accurate than a conventional GPS–IMU fusion approach due to additional estimates from a camera and fuzzy motion models. The paper also presents an application in cultural heritage context running at modest frame rates due to the design of the fusion algorithm.


Sensor fusion Fuzzy adaptive motion models Camera GPS IMU 


Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. Agrawal M, Konolige K, Bolles R (2007) Localization and mapping for autonomous navigation in outdoor terrains: a stereo vision approach. In: IEEE workshop on applications of computer vision, pp 7–12Google Scholar
  2. Almagbile A, Wang J, Ding W (2010) Evaluating the performances of adaptive kalman filter methods in GPS/INS integration. J Glob Position Syst 9(1):33–40CrossRefGoogle Scholar
  3. Armesto L, Tornero J, Vincze M (2007) Fast ego-motion estimation with multi-rate fusion of inertial and vision. Int J Robot Res 26:577–589Google Scholar
  4. Azuma R (1997) A survey of augmented reality. Presence Teleoperators Virtual Environ 6:355–385CrossRefGoogle Scholar
  5. Bleser G (2009) Towards visual-inertial slam for mobile augmented reality. Ph.D. thesis, Fachbereich Informatik der Technischen Universitt KaiserslauternGoogle Scholar
  6. Bostanci E, Clark A, Kanwal N (2012) Vision-based user tracking for outdoor augmented reality. In: 2012 IEEE symposium on computers and communications (ISCC), pp 566–568Google Scholar
  7. Bostanci E, Kanwal N, Ehsan S, Clark AF (2013) User tracking methods for augmented reality. Int J Comput Theory Eng 5(1):93–98CrossRefGoogle Scholar
  8. Broll W, Lindt I, Herbst I, Ohlenburg J, Braun A, Wetzel R (2008) Toward next-gen mobile are games. IEEE computer graphics and applications, pp 40–48Google Scholar
  9. Chen J, Pinz A (2004) Structure and motion by fusion of inertial and vision-based tracking. Austrian Assoc Pattern Recognit 179:75-62Google Scholar
  10. Chroust SG, Vincze M (2004) Fusion of vision and inertial data for motion and structure estimation. J Robotic Syst 21(2):73–83CrossRefGoogle Scholar
  11. Civera J, Davison A, Montiel JMM (2008) Interacting multiple model monocular slam. In: International conference on robotics and automation, pp 3704–3709Google Scholar
  12. Groves P (2008) Principles of GNSS, inertial, and multi-sensor integrated navigation systems. GNSS technology and applications series, Artech HouseGoogle Scholar
  13. Hong SK (2003) Fuzzy logic based closed-loop strapdown attitude system for unmanned aerial vehicle (uav). Sens Actuators A Phys 107(2):109–118CrossRefGoogle Scholar
  14. Kanatani K (2004) Uncertainty modeling and model selection for geometric inference. IEEE Trans Pattern Anal Mach Intell 26(10):1307–1319Google Scholar
  15. Kaplan E, Hegarty C (2005) Understanding GPS: principles and applications. Artech house mobile communications series, Artech houseGoogle Scholar
  16. Kleeman L (2013) Understanding and applying kalman filtering. Accessed Sep 2013
  17. Konolige K, Agrawal M, Bolles R, Cowan C, Fischler M, Gerkey B (2008) Outdoor mapping and navigation using stereo vision. Exp Robot 39:179–190CrossRefGoogle Scholar
  18. Kramer J, Kandel A (2012) On accurate localization and uncertain sensors. Int J Intell Syst 27(5):429–456CrossRefGoogle Scholar
  19. Lang P, Kusej A, Pinz A, Brasseur G (2002) Inertial tracking for mobile augmented reality. In: IEEE instruments and measurement technology conferenceGoogle Scholar
  20. Madgwick S, Vaidyanathan R, Harrison A (2010) An efficient orientation filter for inertial measurement units (IMUs) and magnetic angular rate and gravity (MARG) sensor arrays. Technical report, Department of Mechanical EngineeringGoogle Scholar
  21. Ojeda L, Borenstein J (2002) Flexnav: fuzzy logic expert rule-based position estimation for mobile robots on rugged terrain. In: IEEE international conference on robotics and automation, 2002. Proceedings. ICRA ’02, vol 1, pp 317–322Google Scholar
  22. Oskiper T, Samarasekera S, Kumar R (2012) Multi-sensor navigation algorithm using monocular camera, imu and gps for large scale augmented reality. In: 2012 IEEE international symposium on mixed and augmented reality (ISMAR), pp 71–80Google Scholar
  23. Read P, Meyer M (2000) Restoration of motion picture film. Elsevier Science, Butterworth-Heinemann series in conservation and museologyGoogle Scholar
  24. Reid I (2012) Estimation II. Accessed Sep 2013
  25. Ribo M, Lang P, Ganster H, Brandner M, Stock C, Pinz A (2002) Hybrid tracking for outdoor augmented reality applications. IEEE computer graphics and applications, pp 54–63Google Scholar
  26. Ross T (2009) Fuzzy logic with engineering applications. Wiley, New YorkGoogle Scholar
  27. Schindler K, Suter D (2006) Two-view multibody structure-and-motion with outliers through model selection. IEEE Trans Pattern Anal Mach Intell 28(6):983–995Google Scholar
  28. Schleicher D, Bergasa LM, Ocana M, Barea R, Lopez M (2009) Real-time hierarchical outdoor slam based on stereovision and gps fusion. IEEE Trans Intell Transp Syst 10:440–452CrossRefGoogle Scholar
  29. Seo J, Lee JG, Park C (2005) Leverarm compensation for integrated navigation system of land vehicles. In: Proceedings of 2005 IEEE conference on control applications, 2005. CCA 2005, pp 523–528Google Scholar
  30. Thrun S, Burgard W, Fox D (2006) Probabilistic robotics. MIT Press, New YorkzbMATHGoogle Scholar
  31. Tornqvist D, Schon TB, Karlsson R, Gustafsson F (2009) Particle filter slam with high dimensional vehicle model. J Intell Robot Syst 55:249–266CrossRefzbMATHGoogle Scholar
  32. Torr P (2002) Bayesian model estimation and selection for epipolar geometry and generic manifold fitting. Int J Comput Vision 50(1):35–61CrossRefzbMATHGoogle Scholar
  33. Tseng C, Chang C, Jwo D (2011) Fuzzy adaptive interacting multiple model nonlinear filter for integrated navigation sensor fusion. Sensors 11:2090–2111CrossRefGoogle Scholar
  34. You S, Neumann U, Azuma R (1999) Orientation tracking for outdoor augmented reality registration. IEEE virtual reality, pp 36–42Google Scholar
  35. Zadeh LA (1994) Fuzzy logic, neural networks, and soft computing. Commun ACM 37(3):77–84CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  • Erkan Bostanci
    • 1
  • Betul Bostanci
    • 2
  • Nadia Kanwal
    • 3
  • Adrian F. Clark
    • 4
  1. 1.Computer Engineering DepartmentAnkara UniversityGolbasiTurkey
  2. 2.HAVELSAN Inc.AnkaraTurkey
  3. 3.Lahore College for Women UniversityLahorePakistan
  4. 4.School of Computer Science and Electronic EngineeringUniversity of EssexEssexUK

Personalised recommendations