Fusing Intertial Data with Vision for Enhanced Image Understanding

Conference paper

DOI: 10.1007/978-3-319-29971-6_11

Part of the Communications in Computer and Information Science book series (CCIS, volume 598)
Cite this paper as:
Haines O., Bull D.R., Burn J.F. (2016) Fusing Intertial Data with Vision for Enhanced Image Understanding. In: Braz J. et al. (eds) Computer Vision, Imaging and Computer Graphics Theory and Applications. VISIGRAPP 2015. Communications in Computer and Information Science, vol 598. Springer, Cham


In this paper we show that combining knowledge of the orientation of a camera with visual information can be used to improve the performance of semantic image segmentation. This is based on the assumption that the direction in which a camera is facing acts as a prior on the content of the images it creates. We gathered egocentric video with a camera attached to a head-mounted display, and recorded its orientation using an inertial sensor. By combining orientation information with typical image descriptors, we show that segmentation of individual images improves in accuracy compared with vision alone, from 61 % to 71 % over six classes. We also show that this method can be applied to both point and line based features from the image, and that these can be combined together for further benefits. Our resulting system would have applications in autonomous robot locomotion and guiding visually impaired humans.


Vision guided locomotion Segmentation Image interpretation Scene understanding Inertial sensors Oculus Rift Mobile robotics 

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.University of BristolBristolUK

Personalised recommendations