Autonomous Robots

, Volume 40, Issue 2, pp 343–359 | Cite as

Estimation of a nonvisible field-of-view mobile target incorporating optical and acoustic sensors

  • Kuya Takami
  • Tomonari Furukawa
  • Makoto Kumon
  • Daisuke Kimoto
  • Gamini Dissanayake


This paper presents a nonvisible field-of-view (NFOV) target estimation approach that incorporates optical and acoustic sensors. An optical sensor can accurately localize a target in its field-of-view whereas the acoustic sensor could estimate the target location over a much larger space, but only with limited accuracy. A recursive Bayesian estimation framework where observations of the optical and acoustic sensors are probabilistically treated and fused is proposed in this paper. A technique to construct the observation likelihood when two microphones are used as the acoustic sensor is also described. The proposed technique derives and stores the interaural level difference of observations from the two microphones for different target positions in advance and constructs the likelihood through correlation. A parametric study of the proposed acoustic sensing technique in a controlled test environment, and experiments with an NFOV target in an actual indoor environment are presented to demonstrate the capability of the proposed technique.


Nonvisible field-of-view target estimation Recursive Bayesian estimation Interaural level difference Acoustic localization 


  1. Bahl, P., & Padmanabhan, V. N. (2000). Radar: An in-building rf-based user location and tracking system. In INFOCOM 2000. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings IEEE, (Vol. 2, pp. 775–784). IEEE.Google Scholar
  2. Bertinato, M., Ortolan, G., Maran, F., Marcon, R., Marcassa, A., Zanella, F., Zambotto, M., Schenato, L., & Cenedese, A. (2008). RF localization and tracking of mobile nodes in wireless sensors networks: Architectures, algorithms and experiments.Google Scholar
  3. Chan, Y., Tsui, W., So, H., & Ching, P. (2006). Time-of-arrival based localization under nlos conditions. IEEE Transactions on Vehicular Technology, 55(1), 17–24.CrossRefGoogle Scholar
  4. Chen, P. (1999). A non-line-of-sight error mitigation algorithm in location estimation. In Wireless Communications and Networking Conference, 1999. WCNC. 1999 IEEE, (pp. 316–320). IEEE.Google Scholar
  5. Dai, H., Zhu, Z., & Gu, X. (2012). Multi-target indoor localization and tracking on video monitoring system in a wireless sensor network. Journal of Network and Computer Applications, 36(1), 228–234.CrossRefGoogle Scholar
  6. Even, J., Morales, Y., Kallakuri, N., Ishi, C., & Hagita, N. (2014). Audio ray tracing for position estimation of entities in blind regions. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), 2014 , (pp. 1920–1925). IEEE.Google Scholar
  7. Furukawa, T., Bourgault, F., Lavis, B., & Durrant-Whyte, H. (2006). Recursive bayesian search-and-tracking using coordinated uavs for lost targets. In Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006 (pp. 2521–2526). IEEE.Google Scholar
  8. Furukawa, T., Mak, L. C., DurrantWhyte, H., & Madhavan, R. (2012). Autonomous bayesian search and tracking, and its experimental validation. Advanced Robotics, 26(5–6), 461–485.CrossRefGoogle Scholar
  9. Gao, P., Shi, W., Zhou, W., Li, H., & Wang, X. (2013). A location predicting method for indoor mobile target localization in wireless sensor networks. International Journal of Distributed Sensor Networks. doi:10.1155/2013/949285.
  10. Gezici, S. (2008). A survey on wireless position estimation. Wireless Personal Communications, 44(3), 263–282.CrossRefGoogle Scholar
  11. Guvenc, I., & Chong, C. (2009). A survey on toa based wireless localization and nlos mitigation techniques. IEEE Communications Surveys & Tutorials, 11(3), 107–124.CrossRefGoogle Scholar
  12. Jankovic, N. D., & Naish, M. D. (2005). Developing a modular active spherical vision system. In Proceedings of the 2005 IEEE International Conference on Robotics and Automation, 2005. ICRA 2005 (pp. 1234–1239). IEEE.Google Scholar
  13. Jung, J., & Myung, H. (2011). Indoor localization using particle filter and map-based nlos ranging model. In 2011 IEEE International Conference on Robotics and Automation (ICRA) (pp. 5185–5190).Google Scholar
  14. Khoury, H. M., & Kamat, V. R. (2009). Evaluation of position tracking technologies for user localization in indoor construction environments. Automation in Construction, 18(4), 444–457.CrossRefGoogle Scholar
  15. Kimoto, D., & Kumon, M. (2011). On sound direction estimation by binaural auditory robots with pinnae. 35th Meeting of Special Interest Group on AI Challenges.Google Scholar
  16. Kimoto, D., & Kumon, M. (2012). Optimization of the ear canal position for sound localization using interaural level difference. 36th Meeting of Special Interest Group on AI Challenges.Google Scholar
  17. Kobilarov, M., Sukhatme, G., Hyams, J., & Batavia, P. (2006). People tracking and following with mobile robot using an omnidirectional camera and a laser. In Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006 (pp. 557–562). IEEE.Google Scholar
  18. Ladd, A. M., Bekris, K. E., Rudys, A. P., Wallach, D. S., & Kavraki, L. E. (2004). On the feasibility of using wireless ethernet for indoor localization. IEEE Transactions on Robotics and Automation, 20(3), 555–559.CrossRefGoogle Scholar
  19. Ledwich, L., & Williams, S. (2004). Reduced SIFT features for image retrieval and indoor localisation. In Australian conference on robotics and automation (Vol. 322, p. 3).Google Scholar
  20. Liu, H., Darabi, H., Banerjee, P., & Liu, J. (2007). Survey of wireless indoor positioning techniques and systems. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 37(6), 1067–1080.CrossRefGoogle Scholar
  21. Lu, Y., & Cooke, M. (2010). Binaural estimation of sound source distance via the direct-to-reverberant energy ratio for static and moving sources. IEEE Transactions on Audio, Speech, and Language Processing, 18(7), 1793–1805.CrossRefGoogle Scholar
  22. Mak, L. C., & Furukawa, T. (2009). Non-line-of-sight localization of a controlled sound source. In IEEE/ASME International Conference on Advanced Intelligent Mechatronics, 2009. AIM 2009 (pp. 475–480). IEEE.Google Scholar
  23. Mauler, R. (2003). Recent developments in cooperative control and optimizatio, chapter Objective functions for Bayesian control-theoretic sensor management, II: MHC-Like approximation (pp. 273–316). Kluwer Academic Publishers, Norwell, MA.Google Scholar
  24. Nakadai, K., Nakajima, H., Murase, M., Kaijiri, S., Yamada, K., Nakamura, T., Hasegawa, Y., Okuno, H. G., & Tsujino, H. (2006). Robust tracking of multiple sound sources by spatial integration of room and robot microphone arrays. In 2006 IEEE International Conference on Acoustics, Speech and Signal Processing, 2006. ICASSP 2006 Proceedings (Vol. 4, pp. IV–IV). IEEE.Google Scholar
  25. Narang, G., Nakamura, K., & Nakadai, K. (2014). Auditory-aware navigation for mobile robots based on reflection-robust sound source localization and visual slam. In 2014 IEEE International Conference on Systems, Man and Cybernetics (SMC) (pp. 4021–4026). IEEE.Google Scholar
  26. Nayar, S. K. (1997). Catadioptric omnidirectional camera. In Proceedings., 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1997 (pp. 482–488). IEEE.Google Scholar
  27. Ni, L. M., Liu, Y., Lau, Y. C., & Patil, A. P. (2004). Landmarc: Indoor location sensing using active rfid. Wireless networks, 10(6), 701–710.CrossRefGoogle Scholar
  28. Noda, Y., & Kumon, M. (2012). Sound source direction estimation in the median plane by two active pinnae. 13th SICE System Integration Division Annual Conference.Google Scholar
  29. Prigge, E. A. (2004). A positioning system with no line-of-sight restrictions for cluttered environments. PhD thesis, Stanford University.Google Scholar
  30. Riba, J., & Urruela, A. (May 2005). A non-line-of-sight mitigation technique based on ml-detection. In Proceedings of IEEE International Conference on Acoustic, Speech and Signal Processing (Vol. 2, pp. 153–156).Google Scholar
  31. Sasaki, Y., Kagami, S., & Mizoguchi, H. (2009). Online short-term multiple sound source mapping for a mobile robot by robust motion triangulation. Advanced Robotics, 23(1–2), 145–164.CrossRefGoogle Scholar
  32. Sederberg, T. W., Zheng, J., Bakenov, A., & Nasri A. (2003). T-splines and t-nurccs. In ACM transactions on graphics (TOG) (Vol. 22, pp. 477–484). ACM.Google Scholar
  33. Seow, C. K., & Tan, S. Y. (2008). Non-line-of-sight localization in multipath environments. IEEE Transactions on Mobile Computing, 7(5), 647–660.CrossRefGoogle Scholar
  34. Svaizer, P., Brutti, A., & Omologo, M. (2012). Environment aware estimation of the orientation of acoustic sources using a line array. In 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO) (pp. 1024–1028). IEEE.Google Scholar
  35. Valin, J., Michaud, F., Rouat, J., & Létourneau, D. (2003). Robust sound source localization using a microphone array on a mobile robot. In Proceedings. 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2003.(IROS 2003) (Vol 2, pp. 1228–1233). IEEE.Google Scholar
  36. Wang, J., Gao, Q., Yu, Y., Wang, H., & Jin, M. (2012). Toward robust indoor localization based on bayesian filter using chirp-spread-spectrum ranging. IEEE Transactions on Industrial Electronics, 59(3), 1622–1629.CrossRefGoogle Scholar
  37. Zhang, D., Yang, Y., Cheng, D., Liu, S., & Ni, L. M. (2010). Cocktail: An rf-based hybrid approach for indoor localization. In 2010 IEEE International Conference on Communications (ICC) (pp. 1–5). IEEE.Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Kuya Takami
    • 1
  • Tomonari Furukawa
    • 1
    • 3
  • Makoto Kumon
    • 2
  • Daisuke Kimoto
    • 2
  • Gamini Dissanayake
    • 3
  1. 1.Department of Mechanical EngineeringVirginia TechBlacksburgUSA
  2. 2.Department of Mechanical System EngineeringKumamoto UniversityKumamotoJapan
  3. 3.Center for Autonomous SystemsUniversity of Technology, SydneyUltimoAustralia

Personalised recommendations