## Abstract

This paper introduces a new indoor way-finding method for the visually impaired person (VIP) by utilizing the naturally-generated inertial and geomagnetic information. Reliable and accurate indoor orientation and localization are provided by newly designed sensor fusion algorithms, which take advantage of inertial and geomagnetic information and overcome the inherent problems of the naturally-generated signals, such as low signal-to-noise ratio (SNR) and high environmental sensitivity. Geomagnetic information compensates the sensor drift and accumulative error of the inertial sensors whereas the inertial sensors help to correct the orientation-related errors and drift of the magnetic fields. A parameter derived from the magnetic tensor is introduced for obstacle avoidance and object/destination approach, especially when the relatively large localization uncertainty exists. With a prototype developed based on the system design, several experiments under different indoor scenarios demonstrate that the proposed indoor-way finding method can guide the VIPs and avoid obstacles indoor efficiently and accurately.

### Similar content being viewed by others

## References

Aitor, A., López-Nicolás, G., Puig, L., Guerrero, J.J.: Navigation assistance for the visually impaired using RGB-D sensor with range expansion. IEEE Syst. J.

**10**(3), 922–932 (2014)Bourne, R.R.A., et al.: Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: a systematic review and meta-analysis. Lancet Glob. Health

**5**(9), e888–e897 (2017)Cechowicz, R.: Bias drift estimation for MEMS gyroscope used in inertial navigation. Acta Mechanica et Automatica

**11**(2), 104–110 (2017)Daniele, C., Giarré, L., Pascucci, F., Tinnirello, I., Galioto, G.E., Garlisi, D., Valvo, A.L.: An indoor and outdoor navigation system for visually impaired people. IEEE Access

**7**, 170406–170418 (2019)Flores, J.V.Z., Rasseneur, L., Galani, R., Rakitic, F., Farcy, R.: Indoor navigation with smart phone IMU for the visually impaired in university buildings. J. Assist. Technol.

**10**(3), 133–139 (2016)Fourati, H.: Heterogeneous data fusion algorithm for pedestrian navigation via foot-mounted inertial measurement unit and complementary filter. IEEE Trans. Instrum. Meas.

**64**(1), 221–229 (2015)Global Data on Visual Impairment (2020) [Online]. https://www.who.int/blindness/publications/globaldata/en/. Accessed 2 Sept 2020.

He, Z., Ye, C.: An indoor wayfinding system based on geometric features aided graph SLAM for the visually impaired. IEEE Trans. Neural Syst. Rehabil. Eng.

**25**(9), 1592–1604 (2017)Javed, Y., Khan, Z., Asif, S.: Evaluating indoor location triangulation using Wi-Fi signals. Advances in Internet, Data and Web Technologies Lecture Notes on Data Engineering and Communications Technologies, pp. 180–186 (2019).

Katzschmann, R.K., Araki, B., Rus, D.: Safe local navigation for visually impaired users with a time-of-flight and haptic feedback device. IEEE Trans. Neural Syst. Rehabil. Eng.

**26**(3), 583–593 (2018)Kietlinski-Zaleski, J., Yamazato, T.: UWB positioning using known indoor features—environment comparison. International Conference on Indoor Positioning and Indoor Navigation, pp. 1–9 (2010).

Kok, M., Hol, J.D., Schön, T.B.: Using inertial sensors for position and orientation estimation. Found Trends Signal Process

**11**(1–2), 1–153 (2017)Kulyukin, V., Gharpure, C., Nicholson, J., Pavithran, S.: RFID in robot-assisted indoor navigation for the visually impaired. IEEE/RSJ Int. Conf. Intell. Robot. Syst. (IROS)

**2**, 1979–1984 (2004)Lee, K.M., Li, M.: Magnetic Field Localization Method for Guiding Visually Impaired Applications. IEEE/ASME Advanced Intelligent Mechatronics (AIM), Jul. 9–12, pp. 542-547. Wollongong, NSW (2013).

Lee, K.M., Li, M.: Magnetic tensor sensor for gradient-based localization of ferrous object in geomagnetic field. IEEE Trans. Magn.

**52**(8), 1–10 (2016)Lee, K.M., Li, M., Lin, C.-Y.: A novel way-finding method based on geomagnetic field effects and magnetic tensor measurements for visually impaired users. In: 2015 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), Jul. 7–11, pp. 232–237. Busan, Korea (2015).

Lee, K.M., Li, M., Lin, C.Y.: Magnetic tensor sensor and way-finding method based on geomagnetic field effects with applications for visually impaired users. IEEE/ASME Trans. Mechatron.

**21**(6), 2694–2704 (2016)Madgwick, S.O., Harrison, A.J., Vaidyanathan, R.: Estimation of IMU and MARG orientation using a gradient descent algorithm. In: 2011 IEEE International Conference on Rehabilitation Robotics, Jun. 29–Jul, pp. 1–7. ETH Zurich Science City, Switzerland (2011).

Mahony, R., Hamel, T., Pflimlin, J.M.: Nonlinear complementary filters on the special orthogonal group. IEEE Trans. Autom. Control

**53**(5), 1203–1218 (2008)Martinez-Sala, A., Losilla, F., Sánchez-Aarnoutse, J., García-Haro, J.: Design, implementation and evaluation of an indoor navigation system for visually impaired people. Sensors

**15**(12), 32168–32187 (2015)Riehle, T.H., Anderson, S.M., Lichter, P.A., Giudice, N.A., Sheikh, S.I., Knuesel, R.J., Kollmann, D.T.: Indoor magnetic navigation for the blind. IEEE Annual International Conference of Engineering in Medicine and Biology Society (EMBS), Aug. 28-Sep. 1, pp. 1972–1975. San Diego, CA, USA (2012).

Yun, X., Bachmann, E.R., Moore, H., Calusdian, J.: Self-contained position tracking of human movement using small inertial/magnetic sensor modules. IEEE International Conference on Robotics and Automation (ICRA), Apr. 10–14, pp. 2526–2533. Roma, Italy (2007).

Zhang, X., et al.: A slam based semantic indoor navigation system for visually impaired users. In: 2015 IEEE International Conference on Systems, Man, and Cybernetics, pp. 1458–1463. Kowloon (2015).

## Acknowledgements

This work was supported by Faculty Research Grant (FRG) of Minnesota State University Mankato.

## Author information

### Authors and Affiliations

### Corresponding author

## Additional information

### Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Appendices

### Appendix 1: Orientation representation with quaternion

Quaternions, which consist of four numbers as shown in Eq. (17), can uniquely determine a 3D rotation or the orientation of coordinate *B* relative to coordinate *A*.

The inverse (denoted by subscript ^{−1}) or conjugate (denoted by subscript *) of the rotation quaternion represents the opposite rotation or swapped relative orientation, which is mathematically expressed in Eq. (18a, b).

where **q**^{AB} represents the orientation of coordinate *A* with respect coordinate *B*. To represent a sequential orientation and coordinate transformation, the Hamilton product (denoted by \(\otimes\)) of the quaternion is introduced in Eq. (19).

Assume another coordinate *C* is introduced and its orientation **q**^{CB} with respect to coordinate *B* is given. The orientation of *C* relative to *A* is represented with the quaternion product in Eq. (20).

Assume **u**^{A} is a vector described in coordinate *A*. A 0(zero) is inserted to this vector to make it a row vector containing 4 elements. Given the relative orientation of coordinate *B* represented with **q**^{AB}, the same vector described in coordination *B* is expressed in Eq. (21).

It can also be represented in a rotation matrix form.

### Appendix 2: MTS calibration

Assume the MTS is placed outside (no external magnetic anomalies), the measurements of the magnetic sensors in the MTS **T** consists of two components, MFD of the geomagnetic field **B**^{G} and internal magnetic anomalies **B**^{A} generated by the magnetic solder materials.

In order to estimate **B**^{A}, the MTS is placed outside with orientation shown in Fig.

18. P2 and P3 can the positions when rotating 180° about *z* and *y* axis of the sensor frame. Assume internal magnetic anomalies \({\mathbf{B}}_{1}^{G} = \left[ {\begin{array}{*{20}c} {B_{x}^{G} } & {B_{y}^{G} } & {B_{z}^{G} } \\ \end{array} } \right]^{T}\) at P1, internal magnetic anomalies at P2 and P3 can be estimated with \({\mathbf{B}}_{2}^{G} = \left[ {\begin{array}{*{20}c} { - B_{x}^{G} } & { - B_{y}^{G} } & {B_{z}^{G} } \\ \end{array} } \right]^{T}\) and \({\mathbf{B}}_{3}^{G} = \left[ {\begin{array}{*{20}c} { - B_{x}^{G} } & {B_{y}^{G} } & { - B_{z}^{G} } \\ \end{array} } \right]^{T}\). Assume MFD measurements at P1, P2 and P3 are **T**_{1}, **T**_{2} and **T**_{3} respectively, after applying Eq. (22) for each position, the internal magnetic anomalies BA can be estimated with Eq. (23a, b, c, d).

## Rights and permissions

## About this article

### Cite this article

Li, M., Ammanabrolu, J. Indoor way-finding method using IMU and magnetic tensor sensor measurements for visually impaired users.
*Int J Intell Robot Appl* **5**, 264–282 (2021). https://doi.org/10.1007/s41315-021-00163-6

Received:

Accepted:

Published:

Issue Date:

DOI: https://doi.org/10.1007/s41315-021-00163-6