Skip to main content
SpringerLink
Log in

Perception of Audio Visual Information for Mobile Robot Motion Control Systems

  • Chapter
  • First Online:
Computer Vision in Control Systems-2

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 75))

Abstract

Motion is the main characteristic of intelligent mobile robots. There exist a lot of methods and algorithms for mobile robots motion control. These methods are based on different principles, but the results from these methods must leads to one final goal—to provide a precise mobile robot motion control with clear orientation in the area of robot perception and observation. First, in the proposed chapter the mobile robot audio and visual systems with the corresponding audio (microphone array) and video (mono, stereo or thermo cameras) sensors, accompanied with laser rangefinder sensor, are outlined. The audio and video information captured from the sensors is used in the perception audio visual model proposed to perform joint processing of audio visual information and to determine the current mobile robot position (current space coordinates) in the area of robot perception and observation. The captured from audio visual sensors information is estimated with the suitable algorithms developed for speech and image quality estimation to apply the preprocessing methods for increasing the quality and to minimizing the errors of mobile robot position calculations. The current space coordinates determined from laser rangefinder are used as supplementary information of mobile robot position, for error calculation and for comparison with the results from audio visual mobile robot motion control. In the development of the mobile robot perception audio visual model, some methods are used: method RANdom SAmple Consensus (RANSAC) for estimation of parameters of a mathematical model from a set of observed audio visual coordinate data; method Direction Of Arrival (DOA) for sound source direction localization with microphone array of speaker sending voice commands to the mobile robot; method for speech recognition of the voice command sending from the speaker to the robot. The current mobile robot position calculated from joint usage of perceived audio visual information is used in appropriate algorithms for mobile robot navigation, motion control, and objects tracking: map based or map less methods, path planning and obstacle avoidance, Simultaneous Localization And Mapping (SLAM), data fusion, etc. The error, accuracy, and precision of the proposed mobile robot motion control with perception of audio visual information are analyzed and estimated from the results of the numerous experimental tests presented at the end of this chapter. The experiments are carried out mainly with simulations of the algorithms listed above, but are trying also parallel computing methods in implementation of the developed algorithms to reach real time robot navigation and motion control using perceived audio visual information from the mobile robot audio visual sensors.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Suh II. H. Editor-in-Chief. Intelligent Service Robotics. Springer, ISSN Print: 1861-2776, ISSN Online: 1861-2784

    Google Scholar 

  2. Jarvis R (2008) Intelligent robotics: past, present and future. Int J Comput Sci Appl Technomathematics Res Found 5(3):23–35

    Google Scholar 

  3. Bittermann MS, Sariyildiz IS, Ciftcioglu Ö (2007) Visual perception in design and robotics. J Int Comput Aided Eng Inform Control Autom Robot 14(1):73–91

    Google Scholar 

  4. Bigun J (2006) Vision with direction. Springer, Berlin

    Google Scholar 

  5. Adams B, Breazeal C, Brooks RA, Scassellati B (2000) Humanoid robots: a new kind of tool. Int Syst Appl IEEE 15(4):25–31

    Article  Google Scholar 

  6. Eckmiller R, Baruth O, Neumann D (2006) On human factors for interactive man-machine vision: requirements of the neural visual system to transform objects into percepts. In: IEEE world congress on computational intelligence—international joint conference on neural networks (WCCI 2006), pp 99–703

    Google Scholar 

  7. Demmel J, Lafferriere G, Schwartz J, Sharir M (1988) Theoretical and experimental studies using a multifinger planar manipulator. In: IEEE international conference on robotics and automation, pp 390–395

    Google Scholar 

  8. Murray RM, Sastry SS (1990) Grasping and manipulation using multi fingered robot hands. In: Brockett RW (ed) Robotics: proceedings of symposia in applied mathematics, American Mathematical Society, Providence, Rhode Island

    Google Scholar 

  9. Manocha D, Canny JF (1992) Real time inverse kinematics for general 6R manipulators. Technical report ESRC 92-2, University of California, Berkeley

    Google Scholar 

  10. Jarvis R, Ho N, Byrne JB (2007) Autonomous robot navigation in cyber and real worlds. In: International conference on cyberworlds (CW’2007), pp 66–73

    Google Scholar 

  11. Siegwart R, Nourbakhsh IR (2004) Introduction to autonomous mobile robots. a bradford book. The MIT Press Cambridge, Massachusetts

    Google Scholar 

  12. Adams M (1999) Sensor modelling, design and data processing for autonomous navigation. World scientific series in robotics and intelligent systems. World Scientific Publishing Co Ltd, Singapore

    Google Scholar 

  13. Borenstein J, Everet HR, Feng L (1996) Navigating mobile robots, systems and techniques. AK Peters Ltd, Wellesley

    MATH  Google Scholar 

  14. de Pina Filho AC (ed) (2010) Humanoid Robots. New Developments. Advanced Robotic Systems International and I-Tech, Vienna

    Google Scholar 

  15. Kuffner JJ, Nishiwaki K, Kagami S, Inaba M (2001) Motion planning for humanoid robots under obstacle and dynamic balance constraints. IEEE Int Conf Robot Autom 1:692–698

    Google Scholar 

  16. Favorskaya M (2012) Recognition of dynamic visual images based on group transformations. J Pattern Recognit Image Anal 22(1):180–187

    Article  Google Scholar 

  17. Favorskaya M, Pyankov D, Popov A (2013) Motion Estimations based on invariant moments for frames interpolation in stereovision. Procedia Comput Sci 22:1102–1111

    Article  Google Scholar 

  18. Wit C (1998) Trends in mobile robot and vehicle control. In: Siciliano B, Valavanis KP (eds) Control problems in robotics. Springer, London

    Google Scholar 

  19. Bekiarski A, Pleshkova S (2009) Microphone array beamforming for mobile robot. In: 8th WSEAS International Conference on Circuits, systems, electronics, control and signal processing (CSECS’2009), pp 146–149

    Google Scholar 

  20. Favorskaya M (2009) Detection of moving objects by using local 3D structural tensors. Vestnik of Siberian State Aerospace Univ 2:141–146 (in Russian)

    Google Scholar 

  21. Dehkharghani SS (2014) Development of methods and algorithms for audio-visual mobile robot motion control (Doctoral dissertation) Technical University of Sofia, Bulgaria

    Google Scholar 

  22. Dehkharghani SS, Pleshkova S (2014) Geometric thermal infrared camera calibration for target tracking by a mobile robot. Comptes rendus de l’Academie bulgare des Sciences 67(1):109–114

    Google Scholar 

  23. Heikkila J, Olli S (1997) A four-step camera calibration procedure with implicit image correction. In: IEEE computer society conference on computer vision and pattern recognition, pp 1106–1112

    Google Scholar 

  24. Abdel-Aziz YI, Karara HM (1971) Direct linear transformation into object space coordinates in close-range photogrammetry. In: Symposium on close-range photogrammetry, pp 1–18

    Google Scholar 

  25. Levenberg K (1994) A method for the solution of certain nonlinear problems in least squares. Q Appl Math 2(2):164–168

    Google Scholar 

  26. Benjamin JR, Cornell CA (1970) Probability, statistics, and decision for civil engineers. McGraw-Hill, New York

    Google Scholar 

  27. Dehkharghani SS, Bekiarski A, Pleshkova S (2012) Application of probabilistic methods in mobile robots audio visual motion control combined with laser range finder distance measurements. In: Advances in circuits, systems, automation and mechanics, pp 91–98

    Google Scholar 

  28. Dehkharghani SS, Bekiarski A, Pleshkova S (2013) Method and algorithm for precise estimation of joined audio visual robot control. In: Iran’s 3rd international conference on industrial automation

    Google Scholar 

  29. Hough PVC (1962) Method and means for recognizing complex patterns. U.S. Patent No. 3,069,654. 18.12.1962

    Google Scholar 

  30. Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell PAMI 8(6):679–698

    Google Scholar 

  31. Venkov P, Bekiarski A, Dehkharghani SS, Pleshkova S (2010) Search and tracking of targets with mobile robot by using audio-visual information. In: International conference on automation and informatics (CAI’2010), pp 463–469

    Google Scholar 

  32. Omologo M, Svaizer P (1994) Acoustic event localization using a crosspower-spectrum phase based technique. IEEE Int Conf Acoust Speech Signal Process 2:273–276

    Google Scholar 

  33. Valin JM, Michaud F, Rouat J, Létourneau D (2003) Robust sound source localization using a microphone array on a mobile robot. IEEE/RSJ Int Conf Intell Robot Systs (IROS 2003) 2:1228–1233

    Google Scholar 

  34. Ji P, Benyuan L, Towsley D, Kurose J (2002) Modeling frame-level errors in gsm wireless channels. IEEE Globecom Internet Perform Symp 3:2483–2487

    Google Scholar 

  35. Mohamed S, Rubino G, Varela M (2004) A method for quantitative evolution of audio quality over packet networks and its comparison with existing techniques. In: Measurement of speech and audio quality in networks (MESAQIN’2004)

    Google Scholar 

  36. Hu Y, Loizou P (2006) Subjective comparison of speech enhancement algorithms. IEEE Int Conf Acoust Speech Signal Process 1:153–156

    Google Scholar 

  37. Kondo K (2012) Subjective quality measurement of speech: its evaluation, estimation and applications. Springer, Berlin

    Book  Google Scholar 

  38. ITU-T Rec. P.862 (2001) Perceptual evaluation of speech quality (pesq), an objective method for end-to-end speech quality assessment of narrowband telephone networks and speech codecs. International Telecommunication Union, Geneva, Switzerland

    Google Scholar 

  39. ITU-T Rec. P.835 (2003) Subjective test methodology for evaluating speech communication systems that include noise suppression algorithm. International Telecommunication Union, Geneva, Switzerland

    Google Scholar 

  40. Malfait L, Berger J, Kastner M (2006) P.563-the ITU-T standard for single-ended speech quality assessment. IEEE Trans Audio Speech Lang Process 14(6):1924–1934

    Google Scholar 

  41. ITU-T Rec. P.800 (1996) Methods for subjective determination of transmission quality. International Telecommunication Union, Geneva, Switzerland

    Google Scholar 

  42. ITU-R Rec. BS.1116 (1994) Methods for the subjective assessment of small impairments in audio systems including multichannel sound systems. International Telecommunication Union, Geneva, Switzerland

    Google Scholar 

  43. Thorpe L (1999) Subjective evaluation of speech compression codes and other non-linear voice-path devices for telephony applications. Int J Speech Technol 2:273–288

    Article  Google Scholar 

  44. Pleshkova S, Peeva K (2013) Simulation of different types of voice communication systems used for speech quality estimation with applying speech to text as objective criterion of audio quality. Int J of Emerging Technol Comput Appl Sci Issue 2(6):107–111

    Google Scholar 

  45. Pleshkova S, Peeva K (2013) Application of speech to text as criterion of audio quality estimation in multimedia communication systems, Sofia, (CEMA’2013), pp 92–95

    Google Scholar 

  46. Pleshkova-Bekjarska S (2013) Simulation analysis of speech quality dependence from communication channel type and channel coding methods. Int J Emerging Technol Comput Appl Sci 1(6):8–12

    Google Scholar 

  47. Surveyor SRV-1 Blackfin Robot Surveyor Corporation. http://www.surveyor.com/SRV_info.html. Accessed 15 June 2014

  48. Surveyor SRV-1 Blackfin Camera Surveyor Corporation. http://www.surveyor.com/blackfin/. Accessed 15 June 2014

  49. Scanning range finder (SOKUIKI sensor): URG-04LX-UG01 Hokuyo Corporation. http://www.hokuyo-aut.jp/02sensor/07scanner/urg_04lx_ug01.html. Accessed 15 June 2014

  50. Omni Vision. http://www.surveyor.com/blackfin/OV9655-datasheet.pdf. Accessed 15 June 2014

  51. Bouguet J. Camera Calibration Toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/. Accessed 15 June 2014

Download references

Author information

Authors and Affiliations

  1. Department of Telecommunications, Technical University of Sofia, 8 Bld. Kliment Ohridski Sofia, 1000, Sofia, Bulgaria

    Snejana Pleshkova & Alexander Bekiarski

  2. French Language Faculty of Electrical Engineering, Technical University of Sofia, 8 Bld. Kliment Ohridski Sofia, 1000, Sofia, Bulgaria

    Shima Sehati Dehkharghani & Kalina Peeva

Authors
  1. Snejana Pleshkova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexander Bekiarski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shima Sehati Dehkharghani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kalina Peeva
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander Bekiarski .

Editor information

Editors and Affiliations

  1. Department of Informatics and Computer Techniques, Siberian State Aerospace University, Krasnoyarsk, Russia

    Margarita N. Favorskaya

  2. Faculty of Education, Science, Technology and Mathematics, University of Canberra, Canberra, Australia

    Lakhmi C. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Pleshkova, S., Bekiarski, A., Dehkharghani, S.S., Peeva, K. (2015). Perception of Audio Visual Information for Mobile Robot Motion Control Systems. In: Favorskaya, M., Jain, L. (eds) Computer Vision in Control Systems-2. Intelligent Systems Reference Library, vol 75. Springer, Cham. https://doi.org/10.1007/978-3-319-11430-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-11430-9_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-11429-3

  • Online ISBN: 978-3-319-11430-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation