Skip to main content
SpringerLink
Log in
Book cover

Encyclopedia of Sustainability Science and Technology pp 46–64Cite as

Active Multifocal Vision System, Adaptive Control of

  • Reference work entry

  • 277 Accesses

Definition of the Subject

The perception of the environment is crucial for autonomous systems and visual perception in particular is a substantial source of information for intelligent vehicles. For a visual perception system of an intelligent vehicle high measurement accuracy combined with a large field of view is essential. Since these are contradictory requirements for a single camera, multifocal vision systems – equipped with tele- and wide-angle cameras – are frequently used. However, due to their small aperture angles high resolution telecameras have to be actively directed toward scene regions of interest. Moreover active inertial gaze stabilization is required since telecamera s are highly sensitive to rotational vehicle motions. Gaze stabilization as well as adjusting the orientation of the telecamera requires control of the vision system. The use of adaptive control allows for varying camera...

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   6,999.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   549.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Abbreviations

Adaptive control:

The adaptation of the control mechanism to unknown or changing plant characteristics.

Multifocal vision system:

A set of cameras with different focal length. Potentially the orientation of the cameras can be actively changed. Equivalent to camera platform.

Saccade:

Rapid, abrupt sensor movement to bring an interesting object into the sensor focus.

Selective attention:

Selecting the most informative parts within the sensor’s field of view for further exploration.

Smooth-pursuit movement:

Slow sensor movement to keep an object centered in the field of view.

Vestibulo-ocular reflex:

Sensor movement to compensate disturbing motions of the sensor carrier.

Visual odometry:

Determining the position and orientation of a moving camera by analyzing the camera images.

Bibliography

  1. Palmer SE (1999) Vision science. MIT Press, Cambridge, MA

    Google Scholar 

  2. Kolb B, Whishaw IQ (2009) Fundamentals of human neuropsychology, 6th edn. Worth, New York

    Google Scholar 

  3. Kandel ER, Schwartz JH, Jessell TH (1995) Essentials of neural science behavior, 1st edn. Prentice Hall International, London

    Google Scholar 

  4. Henderson JM (2003) Human gaze control during real-world scene perception. Trends Cogn Sci 7(11):498–504, November

    Article  Google Scholar 

  5. Treue S (2003) Visual attention: the where, what, how and why of saliency. Curr Opin Neurobiol 13:428–432

    Article  CAS  Google Scholar 

  6. Chun MM, Marois R (2002) The dark side of visual attention. Curr Opin Neurobiol 12:184–189

    Article  CAS  Google Scholar 

  7. Krauzlis RJ (2005) The control of voluntary eye movements: new perspectives. Neuroscientist 11(2):124–137

    Article  Google Scholar 

  8. Knudsen EI (2007) Fundamental components of attention. Annu Rev Neurosci 30:57–78

    Article  CAS  Google Scholar 

  9. Summerfield C, Egner T (2009) Expectation (and attention) in visual cognition. Trends Cogn Sci 13(9):403–409

    Article  Google Scholar 

  10. Land MF, Lee DN (1994) Where we look when we steer. Nature 369:742–744

    Article  CAS  Google Scholar 

  11. Chapman PR, Underwood G (1998) Visual search of driving situations: danger and experience. Perception 27:951–964

    Article  CAS  Google Scholar 

  12. Olson PL, Battle DS, Aoki T (1989) Driver eye fixations under different operating conditions, University of Michigan, Ann Arbor, Transportation Research Institute, Technical Report

    Google Scholar 

  13. Wann JP, Wilkie RM (2004) How do we control high speed steering. In: Optic flow and beyond. Kluwer, Norwell, pp 401–419, ch 18

    Google Scholar 

  14. Falkmer T, Gregersen NP (2005) A comparison of eye movement behavior of inexperienced and experienced drivers in real traffic environments. Optom Vis Sci 82(8):732–739

    Article  Google Scholar 

  15. Metz B (2009) Worauf achtet der Faherer? steuerung der aufmerksamkeit beim fahren mit visuellen nebenaufgaben. Ph.D. dissertation, Julius-Maximilians-Universität Würzburg

    Google Scholar 

  16. Doshi A, Trivedi MM (2009) On the roles of eye gaze and head dynamics in predicting driver’s intent to change lanes. IEEE Trans Intell Transp Syst 10(3):453–462

    Article  Google Scholar 

  17. Kühnlenz K (2007) “Aspects of multi-focal vision,” ser. Mess-, Steuerungs- und Regelungstechnik.VDI Verlag 8(1129)

    Google Scholar 

  18. Kühnlenz K, Bachmayr M, Buss M (2006) A multi-focal high-performance vision system. In: Proceedings of the 2006 IEEE international conference on robotics and automation, Orlando, pp 150–155

    Google Scholar 

  19. Ude A, Gaskett C, Cheng G (2006) Foveated vision systems with two cameras per eye. In: Proceedings of the 2006 IEEE international conference on robotics and automation (ICRA 2006), Orlando

    Google Scholar 

  20. Roca X, Vitrià J, Vanrell M, Villanueva JJ (1999) Gaze control in a binocular robot systems. In: Proceedings of the 1999 IEEE international conference on emerging technologies and factory automation (ETFA 1999), Spain

    Google Scholar 

  21. Peng J, Srikaew A, Wilkes M, Kawamura K, Peters A (2000) An active vision system for mobile robots. In: Proceedings of the 2000 IEEE international conference on systems, man and cybernetics (ICSMC 2000), Nashville

    Google Scholar 

  22. Yamato J (1999) A layered control system for stereo vision head with vergence. In: Proceedings of the 1999 IEEE international conference on systems, man and cybernetics (ICSMC 1999), Tokyo

    Google Scholar 

  23. Panerai F, Metta G, Sandini G (2000) Adaptive image stabilization: a need for vision-based active robotic agents. In: Proceedings of the international conference on the simulation of adaptive behavior (SAB 2000), Paris

    Google Scholar 

  24. Lobo J, Ferreira JF, Dias J (2008) Robotic implementation of biological Bayesian models towards visuo-inertial image stabilization and gaze control. In: Proceedings of the 2008 IEEE international conference on robotics and biomimetics, Bangkok

    Google Scholar 

  25. Suppa M, Kielhöfer S, Langwald J, Hacker F, Strobl KH, Hirzinger G (2007) The 3D-modeller: a multi-purpose vision platform. In: Proceedings of the 2007 IEEE international conference on robotics and automation (ICRA 2007), Roma

    Google Scholar 

  26. Vilgrattner T, Ulbrich H (2011) Design and control of a compact high-dynamic camera-orientation system. IEEE-ASME T Mech 16(2):221–231

    Article  Google Scholar 

  27. Lenz A, Balakrishnan T, Pipe AG, Melhuish C (2008) An adaptive gaze stabilization controller inspired by the vestibulo-ocular reflex. Bioinspir Biomim 3(3):035001

    Article  CAS  Google Scholar 

  28. Zhang X, Tay ALP (2009) A physical system for binocular vision through saccade generation and vergence control. Cybernet Syst 40:549–568

    Article  Google Scholar 

  29. Dickmanns ED (2007) Dynamic vision for perception and control of motion. Springer, London

    Google Scholar 

  30. Dickmanns ED (2003) An advanced vision system for ground vehicles. In: Proceedings of the 1st international workshop on in-vehicle cognitive computer vision systems, Graz

    Google Scholar 

  31. Schiehlen J, Dickmanns ED (1994) A camera platform for intelligent vehicles. In: Proceedings of the 1994 IEEE intelligent vehicles symposium (IV 1994), Paris

    Google Scholar 

  32. Schiehlen J, Dickmanns ED (1994) Design and control of a camera platform for machine vision. In: Proceedings of the 1994 IEEE international conference on intelligence in robotics and autonomous systems (IROS 1994), Neubiberg

    Google Scholar 

  33. Schiehlen J (1995) Kameraplattformen für aktiv sehende fahrzeuge ser. Mess-, Steuer- und Regelungstechnik. VDI Verlag 8(514)

    Google Scholar 

  34. Günthner W, Wagner P, Ulbrich H (2006) An inertially stabilised vehicle camera system – hardware, algorithms, test drives. In: Proceedings of the 2006 IEEE international conference on industrial electronics (IECON 2006), Paris

    Google Scholar 

  35. Günthner W, Glasauer S, Wagner P, Ulbrich H (2006) Enhancing inertial measurement units by multi-sensor fusion with visual data for active camera stabilization. In: Proceedings of joint conference of the 37th international symposium on robotics ISR 2006 and the German conference on robotics, Munich

    Google Scholar 

  36. Guenthner W, Glasauer S, Wagner P, Ulbrich H (2007) Biomimetic control for adaptive camera stabilization in driver-assistance systems. J Mech Sci Technol 21(6):930–934

    Article  Google Scholar 

  37. Günthner W (2008) Enhancing cognitive assistance systems with inertial measurement units. Springer, Berlin

    Book  Google Scholar 

  38. Ducrocq Y, Bahrami S, Duvieubourg L, Cabestaing F (2008) An effective active vision system for gaze control. In: Advances in visual computing, ser. Lecture notes in computer science, vol 5359. Springer, Berlin, pp 267–277

    Chapter  Google Scholar 

  39. Koch C, Ullman S (1985) Shifts in selective visual attention: towards the underlying neural circuitry. Hum Neurobiol 4:219–227

    CAS  Google Scholar 

  40. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  41. Driscoll JA, Peters RA II, Cave KR (1998) A visual attention network for a humanoid robot. In: Proceedings of the 1998 IEEE/RSJ international conference on intelligent robotic systems (IROS 1998), Victoria

    Google Scholar 

  42. Backer G, Mertsching B, Bollmann M (2001) Data- and model-driven gaze control for an active-vision system. IEEE Trans Pattern Anal Mach Intell 23(12):1415–1429

    Article  Google Scholar 

  43. Breazeal C, Scassellati B (1999) A context-dependent attention system for a social robot. In: Proceedings of the sixteenth international joint conference on artificial intelligence (IJCAI 1999), Stockholm

    Google Scholar 

  44. Vijayakumar S, Conradt J, Shibata T, Schaal S (2001) Overt visual attention for a humanoid robot. In: Proceedings of the 2001 IEEE international conference on intelligence in robotics and autonomous systems (IROS 2001), Hawaii

    Google Scholar 

  45. Sprague N, Ballard D (2003) Eye movements for reward maximization. In: Advances in neural information processing systems 15. MIT Press, Cambridge, MA

    Google Scholar 

  46. Choi SB, Jung BS, Ban SW, Lee M (2004) Trainable attention model based vergence control for active stereo vision system. In: Proceedings of the 2004 IEEE international conference on intelligent sensors, sensor networks and information processing (ISSNIP 2004), Melbourne

    Google Scholar 

  47. Navalpakkam V, Itti L (2005) Modeling the influence of task on attention. Vis Res 45:205–231

    Article  Google Scholar 

  48. Orabona F, Metta G, Sandini G (2005) Object-based visual attention: a model for a behaving robot. In: Proceedings of the 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR 2005), San Diego

    Google Scholar 

  49. Ude A, Wyart V, Lin LH, Cheng G (2005) Distributed visual attention on a humanoid robot. In: Proceedings of the 2005 IEEE-RAS international conference on humanoid robots, Tsukuba

    Google Scholar 

  50. Frintrop S (2006) VOCUS: a visual attention system for object detection and goal-directed search. Lecture notes in computer sciences ed., ser. (Lecture notes in artificial intelligence), vol 3899. Springer, Heidelberg

    Google Scholar 

  51. Begum M, Mann GKI, Gosine RG (2006) A biologically inspired Bayesian model of visual attention for humanoid robots. In: Proceedings of the 2006 IEEE-RAS international conference on humanoid robots (HUMANOIDS 2006), Genova

    Google Scholar 

  52. Yu Y, Mann GKI, Gosine RG (2008) An object-based visual attention model for robots. In: Proceedings of the 2008 IEEE international conference on robotics and automation (ICRA 2008), Pasadena

    Google Scholar 

  53. Vogel J, de Freitas N (2007) Target-directed attention: sequential decision-making for gaze planning. In: Proceedings of the 2007 IEEE international conference on robotics and automation (ICRA 2007), Pasadena

    Google Scholar 

  54. Borji A, Ahmadabadi MN, Araabi BN (2009) Learning sequential visual attention control through dynamic state space discretization. In: Proceedings of the 2009 IEEE international conference on robotics and automation (ICRA 2009), Kobe

    Google Scholar 

  55. Xu T, Kühnlenz K, Buss M (2008) Information-based gaze control adaptation to scene context for mobile robots. In: Proceedings of the 2008 IEEE international conference on pattern recognition (ICPR 2008), Tampa

    Google Scholar 

  56. Xu T, Wu H, Zhang T, Kühnlenz K, Buss M (2009) Environment adapted active multi-focal vision system for object detection. In: Proceedings of the 2009 IEEE international conference on robotics and automation (ICRA 2009), Kobe

    Google Scholar 

  57. Itti L, Baldi P (2009) Bayesian surprise attracts human attention. Vis Res 49(10):1295–1306, June

    Article  Google Scholar 

  58. Xu T, Zhang T, Kühnlenz K, Buss M (2010) Attentional object detection with an active multi-focal vision system. Int J Hum Robot 7(2):223–243

    Article  Google Scholar 

  59. Michalke T, Kastner R, Adamy J, Bone S, Waibel F, Gayko MKA, Gepperth A, Fritsch J, Goerick C (2008) An attention-based system approach for scene analysis in driver assistance. Automatisierungstechnik 56(11):575–584

    Article  Google Scholar 

  60. Aloimonos JY, Weiss I, Bandyopadhyay A (1988) Active vision. Int J Comput Vision 7:333–356

    Article  Google Scholar 

  61. Bajcsy R (1988) Active perception. Proc IEEE 76(8):966–1005, Invited Paper

    Article  Google Scholar 

  62. Aloimonos JY (1990) Purposive and qualitative active vision. In: Proceedings of the 1990 IEEE international conference on pattern recognition, Atlantic

    Google Scholar 

  63. Ballard DH (1991) Animate vision. Artif Intell 48:57–86

    Article  Google Scholar 

  64. Aloimonos Y (ed) (1993) Active perception. Lawrence Erlbaum, Hillsdale

    Google Scholar 

  65. Dickmanns ED (1988) An integrated approach to feature based dynamic vision. In: Proceedings of the 1988 IEEE computer society conference on computer vision and pattern recognition (CVPR 1988), Ann Arbor

    Google Scholar 

  66. Davison AJ (1998) Mobile robot navigation using active vision. PhD Thesis, University of Oxford

    Google Scholar 

  67. Davision AJ, Murray DW (1998) Mobile robot localisation using active vision. In: Proceedings of the 5th European conference on computer vision (ECCV 1998), Freiburg

    Google Scholar 

  68. Seara JF, Strobl KH, Schmidt G (2002) Information management for gaze control in vision guided biped walking. In: Proceedings of the 2002 IEEE international conference on intelligent robots and systems (IROS 2002), Lausanne

    Google Scholar 

  69. Seara JF, Strobl H, Schmidt G (2003) Path-dependent gaze control for obstacle avoidance in vision guided humanoid walking. In: Proceedings of the 2003 IEEE international conference on robotics and automation (ICRA 2003), Taipei

    Google Scholar 

  70. Seara JF, Strobl KH, Martín E, Schmidt G (2003) Task-oriented and situation-dependent gaze control for vision guided humanoid walking. In: Proceedings of the 2003 IEEE-RAS international conference on humanoid robots (HUMANOIDS 2003), Munich

    Google Scholar 

  71. Seara JF (2004) Intelligent gaze control for vision-guided humanoid walking, ser. Mess-, Steuer- und Regelungstechnik.VDI Verlag 8(1044)

    Google Scholar 

  72. Seara JF, Schmidt G (2005) Gaze control strategy for vision-guided humanoid walking. Autmatisierungstechnik 53(2):49–58

    Article  Google Scholar 

  73. Lidoris G, Kühnlenz K, Wollherr D, Buss M (2007) Combined trajectory planning and gaze direction control for robotic exploration. In: Proceedings of the 2007 IEEE international conference on robotics and automation (ICRA 2007), Rome

    Google Scholar 

  74. Kühnlenz K, Lidoris G, Wollherr D, Buss M (2007) On foveated gaze control and combined gaze and locomotion planning. In: de Pina ACF (ed) Humanoid robots: new developments. I-Tech Education and Publishing, Vienna

    Google Scholar 

  75. Kühnlenz K, Buss M (2008) Multi-focal vision and gaze control improve navigation performance. J Hum 1(1):33–44

    Google Scholar 

  76. Xu T, Kühnlenz K, Buss M (2008) A view direction planning strategy for a multi-camera vision system. In: Proceedings of the 2008 IEEE international conference on information and automation (ICIA 2008), Zhangjiajie

    Google Scholar 

  77. Davison AJ, Kita N (2001) 3D simultaneous localisation and map-building using active vision for a robot moving on undulating terrain. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition (CVPR 2001), Lihue

    Google Scholar 

  78. Davison AJ, Murray DW (2002) Simultaneous localization and map-building using active vision. IEEE Trans Pattern Anal Mach Intell 24:865–880

    Article  Google Scholar 

  79. Lidoris G, Kühnlenz K, Wollherr D, Buss M (2006) Information-based gaze direction planning algorithm for slam. In: Proceedings of the 2006 IEEE-RAS international conference on humanoid robots (HUMANOIDS 2006), Genova

    Google Scholar 

  80. Pellkofer M, Dickmanns ED (2000) EMS-vision: gaze control in autonomous vehicles. In: Proceedings of the 2000 IEEE intelligent vehicles symposium (IV 2000), Dearborn

    Google Scholar 

  81. Pellkofer M (2003) Verhaltensentscheidung für autonome fahrzeuge mit blickrichtungssteuerung. Dissertation, Universität der Bundeswehr München

    Google Scholar 

  82. Dickmanns ED (2004) Three-stage visual perception for vertebrate-type dynamic machine vision. In: Proceedings of the conference on engineering of intelligent systems (EIS 2004), Madeira

    Google Scholar 

  83. Unterholzner A, Wuensche HJ (2009) Adaptive state space control of a camera platform for an autonomous ground vehicle. In: Proceedings of the 2009 IEEE intelligent vehicles symposium (IV 2009), Xi’an

    Google Scholar 

  84. Unterholzner A, Wuensche HJ (2010) Hybrid adaptive control of a multi-focal vision system. In: Proceedings of the 2010 IEEE intelligent vehicles symposium (IV 2010), San Diego

    Google Scholar 

  85. Canudas C, Åström KJ, Braun K (1987) Adaptive friction compensation in DC-motor drives. IEEE J Robotic Autom RA-3(6):681–685

    Article  Google Scholar 

  86. Goodwin GC, Sin KS (1984) Adaptive filtering prediction and control, Information and system sciences. Prentice-Hall, Englewood Cliffs

    Google Scholar 

  87. Ramasubramanian A, Ray LR (2007) Comparison of EKBF-based and classical friction compensation. J Dyn Syst Meas Contr 129:236–242

    Article  Google Scholar 

  88. Franklin GF, Powell JD, Emami-Naeini A (2002) Feedback control of dynamic systems, 4th edn. Prentice Hall, Upper Saddle River

    Google Scholar 

  89. Athans M, Falb PL (1966) Optimal control: an introduction to the theory and its applications. McGraw-Hill, New York

    Google Scholar 

  90. Utkin VI (1977) Variable structure systems with sliding modes. IEEE Trans Autom Control AC-22(2):212–222

    Article  Google Scholar 

  91. Hung JY, Gao W, Hung JC (1993) Variable structure control: a survey. IEEE Trans Ind Electron 40(1):2–22, February

    Article  Google Scholar 

  92. Chatchanayuenyong T, Parnichkun M (2008) Time optimal hybrid sliding mode-PI control for an autonomous underwater robot. Int J Adv Rob Syst 5(1):91–98

    Google Scholar 

  93. You KH, Lee EB (2000) Robust, near time-optimal control of nonlinear second order systems with model uncertainty. In: Proceedings of the 2000 IEEE international conference on control applications, Anchorage, Alaska

    Google Scholar 

  94. Iliev B, Kalaykov I (2004) Minimum-time sliding mode control for second-order systems. In: Proceedings of the 2004 American control conference, Boston

    Google Scholar 

  95. Dean P, Porrill J, Stone JV (2002) Decorrelation control by the cerebellum achieves oculomotor plant compensation in simulated vestibulo-ocular reflex. Proc R Soc Lond Biol Sci 269(1503):1895–1904, August

    Article  Google Scholar 

  96. Unterholzner A, Rohland M, Schweitzer M, Wuensche HJ (2010) Vision based online-calibration of inertial gaze stabilization. In: Proceedings of the 2010 IEEE intelligent vehicles symposium (IV 2010), San Diego

    Google Scholar 

  97. Julier SJ, Uhlmann JK (2004) Unscented filtering and nonlinear estimation. Proc IEEE 92(3):410–422, Invited Paper

    Article  Google Scholar 

  98. Schweitzer M, Wuensche HJ (2010) Real-time visual odometry for ground moving robots using GPUs. In: Proceedings of the international conference on computer vision theory and applications (VISAPP 2010), Angers

    Google Scholar 

  99. Scaramuzza D, Fraundorfer F, Pollefeys M, Siegwart R (2009) Absolute scale in structure from motion from a single vehicle mounted camera by exploiting nonholonomic constraints. In: Proceedings of the IEEE international conference on computer vision (ICCV 2009), Kyoto

    Google Scholar 

  100. Schweitzer M, Wuensche HJ (2009) Efficient keypoint matching for robot vision using GPUs. In: Proceedings of the fifth IEEE workshop on embedded computer vision at ICCV 2009, Kyoto

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Autonomous Systems Technology, Department of Aerospace Engineering, University of the Bundeswehr Munich, Werner-Heisenberg-Weg 39, Neubiberg, 85577, Germany

    Alois Unterholzner & Dr. Hans-Joachim Wuensche

Authors
  1. Alois Unterholzner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dr. Hans-Joachim Wuensche
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alois Unterholzner .

Editor information

Editors and Affiliations

  1. RAMTECH LIMITED, Larkspur, CA, USA

    Robert A. Meyers

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer Science+Business Media, LLC

About this entry

Cite this entry

Unterholzner, A., Wuensche, HJ. (2012). Active Multifocal Vision System, Adaptive Control of. In: Meyers, R.A. (eds) Encyclopedia of Sustainability Science and Technology. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-0851-3_472

Download citation

Publish with us

Policies and ethics

Search

Navigation