Encyclopedia of Sustainability Science and Technology

2012 Edition
| Editors: Robert A. Meyers

Active Multifocal Vision System, Adaptive Control of

Reference work entry
DOI: https://doi.org/10.1007/978-1-4419-0851-3_472

Definition of the Subject

The perception of the environment is crucial for autonomous systems and visual perception in particular is a substantial source of information for intelligent vehicles. For a visual perception system of an intelligent vehicle high measurement accuracy combined with a large field of view is essential. Since these are contradictory requirements for a single camera, multifocal vision systems – equipped with tele- and wide-angle cameras – are frequently used. However, due to their small aperture angles high resolution telecameras have to be actively directed toward scene regions of interest. Moreover active inertial gaze stabilization is required since telecamera s are highly sensitive to rotational vehicle motions. Gaze stabilization as well as adjusting the orientation of the telecamera requires control of the vision system. The use of adaptive control allows for varying camera configurations and thereby further enhances the applicability of multifocal...

This is a preview of subscription content, log in to check access

Bibliography

  1. 1.
    Palmer SE (1999) Vision science. MIT Press, Cambridge, MAGoogle Scholar
  2. 2.
    Kolb B, Whishaw IQ (2009) Fundamentals of human neuropsychology, 6th edn. Worth, New YorkGoogle Scholar
  3. 3.
    Kandel ER, Schwartz JH, Jessell TH (1995) Essentials of neural science behavior, 1st edn. Prentice Hall International, LondonGoogle Scholar
  4. 4.
    Henderson JM (2003) Human gaze control during real-world scene perception. Trends Cogn Sci 7(11):498–504, NovemberCrossRefGoogle Scholar
  5. 5.
    Treue S (2003) Visual attention: the where, what, how and why of saliency. Curr Opin Neurobiol 13:428–432CrossRefGoogle Scholar
  6. 6.
    Chun MM, Marois R (2002) The dark side of visual attention. Curr Opin Neurobiol 12:184–189CrossRefGoogle Scholar
  7. 7.
    Krauzlis RJ (2005) The control of voluntary eye movements: new perspectives. Neuroscientist 11(2):124–137CrossRefGoogle Scholar
  8. 8.
    Knudsen EI (2007) Fundamental components of attention. Annu Rev Neurosci 30:57–78CrossRefGoogle Scholar
  9. 9.
    Summerfield C, Egner T (2009) Expectation (and attention) in visual cognition. Trends Cogn Sci 13(9):403–409CrossRefGoogle Scholar
  10. 10.
    Land MF, Lee DN (1994) Where we look when we steer. Nature 369:742–744CrossRefGoogle Scholar
  11. 11.
    Chapman PR, Underwood G (1998) Visual search of driving situations: danger and experience. Perception 27:951–964CrossRefGoogle Scholar
  12. 12.
    Olson PL, Battle DS, Aoki T (1989) Driver eye fixations under different operating conditions, University of Michigan, Ann Arbor, Transportation Research Institute, Technical ReportGoogle Scholar
  13. 13.
    Wann JP, Wilkie RM (2004) How do we control high speed steering. In: Optic flow and beyond. Kluwer, Norwell, pp 401–419, ch 18Google Scholar
  14. 14.
    Falkmer T, Gregersen NP (2005) A comparison of eye movement behavior of inexperienced and experienced drivers in real traffic environments. Optom Vis Sci 82(8):732–739CrossRefGoogle Scholar
  15. 15.
    Metz B (2009) Worauf achtet der Faherer? steuerung der aufmerksamkeit beim fahren mit visuellen nebenaufgaben. Ph.D. dissertation, Julius-Maximilians-Universität WürzburgGoogle Scholar
  16. 16.
    Doshi A, Trivedi MM (2009) On the roles of eye gaze and head dynamics in predicting driver’s intent to change lanes. IEEE Trans Intell Transp Syst 10(3):453–462CrossRefGoogle Scholar
  17. 17.
    Kühnlenz K (2007) “Aspects of multi-focal vision,” ser. Mess-, Steuerungs- und Regelungstechnik.VDI Verlag 8(1129)Google Scholar
  18. 18.
    Kühnlenz K, Bachmayr M, Buss M (2006) A multi-focal high-performance vision system. In: Proceedings of the 2006 IEEE international conference on robotics and automation, Orlando, pp 150–155Google Scholar
  19. 19.
    Ude A, Gaskett C, Cheng G (2006) Foveated vision systems with two cameras per eye. In: Proceedings of the 2006 IEEE international conference on robotics and automation (ICRA 2006), OrlandoGoogle Scholar
  20. 20.
    Roca X, Vitrià J, Vanrell M, Villanueva JJ (1999) Gaze control in a binocular robot systems. In: Proceedings of the 1999 IEEE international conference on emerging technologies and factory automation (ETFA 1999), SpainGoogle Scholar
  21. 21.
    Peng J, Srikaew A, Wilkes M, Kawamura K, Peters A (2000) An active vision system for mobile robots. In: Proceedings of the 2000 IEEE international conference on systems, man and cybernetics (ICSMC 2000), NashvilleGoogle Scholar
  22. 22.
    Yamato J (1999) A layered control system for stereo vision head with vergence. In: Proceedings of the 1999 IEEE international conference on systems, man and cybernetics (ICSMC 1999), TokyoGoogle Scholar
  23. 23.
    Panerai F, Metta G, Sandini G (2000) Adaptive image stabilization: a need for vision-based active robotic agents. In: Proceedings of the international conference on the simulation of adaptive behavior (SAB 2000), ParisGoogle Scholar
  24. 24.
    Lobo J, Ferreira JF, Dias J (2008) Robotic implementation of biological Bayesian models towards visuo-inertial image stabilization and gaze control. In: Proceedings of the 2008 IEEE international conference on robotics and biomimetics, BangkokGoogle Scholar
  25. 25.
    Suppa M, Kielhöfer S, Langwald J, Hacker F, Strobl KH, Hirzinger G (2007) The 3D-modeller: a multi-purpose vision platform. In: Proceedings of the 2007 IEEE international conference on robotics and automation (ICRA 2007), RomaGoogle Scholar
  26. 26.
    Vilgrattner T, Ulbrich H (2011) Design and control of a compact high-dynamic camera-orientation system. IEEE-ASME T Mech 16(2):221–231CrossRefGoogle Scholar
  27. 27.
    Lenz A, Balakrishnan T, Pipe AG, Melhuish C (2008) An adaptive gaze stabilization controller inspired by the vestibulo-ocular reflex. Bioinspir Biomim 3(3):035001CrossRefGoogle Scholar
  28. 28.
    Zhang X, Tay ALP (2009) A physical system for binocular vision through saccade generation and vergence control. Cybernet Syst 40:549–568CrossRefGoogle Scholar
  29. 29.
    Dickmanns ED (2007) Dynamic vision for perception and control of motion. Springer, LondonGoogle Scholar
  30. 30.
    Dickmanns ED (2003) An advanced vision system for ground vehicles. In: Proceedings of the 1st international workshop on in-vehicle cognitive computer vision systems, GrazGoogle Scholar
  31. 31.
    Schiehlen J, Dickmanns ED (1994) A camera platform for intelligent vehicles. In: Proceedings of the 1994 IEEE intelligent vehicles symposium (IV 1994), ParisGoogle Scholar
  32. 32.
    Schiehlen J, Dickmanns ED (1994) Design and control of a camera platform for machine vision. In: Proceedings of the 1994 IEEE international conference on intelligence in robotics and autonomous systems (IROS 1994), NeubibergGoogle Scholar
  33. 33.
    Schiehlen J (1995) Kameraplattformen für aktiv sehende fahrzeuge ser. Mess-, Steuer- und Regelungstechnik. VDI Verlag 8(514)Google Scholar
  34. 34.
    Günthner W, Wagner P, Ulbrich H (2006) An inertially stabilised vehicle camera system – hardware, algorithms, test drives. In: Proceedings of the 2006 IEEE international conference on industrial electronics (IECON 2006), ParisGoogle Scholar
  35. 35.
    Günthner W, Glasauer S, Wagner P, Ulbrich H (2006) Enhancing inertial measurement units by multi-sensor fusion with visual data for active camera stabilization. In: Proceedings of joint conference of the 37th international symposium on robotics ISR 2006 and the German conference on robotics, MunichGoogle Scholar
  36. 36.
    Guenthner W, Glasauer S, Wagner P, Ulbrich H (2007) Biomimetic control for adaptive camera stabilization in driver-assistance systems. J Mech Sci Technol 21(6):930–934CrossRefGoogle Scholar
  37. 37.
    Günthner W (2008) Enhancing cognitive assistance systems with inertial measurement units. Springer, BerlinCrossRefGoogle Scholar
  38. 38.
    Ducrocq Y, Bahrami S, Duvieubourg L, Cabestaing F (2008) An effective active vision system for gaze control. In: Advances in visual computing, ser. Lecture notes in computer science, vol 5359. Springer, Berlin, pp 267–277CrossRefGoogle Scholar
  39. 39.
    Koch C, Ullman S (1985) Shifts in selective visual attention: towards the underlying neural circuitry. Hum Neurobiol 4:219–227Google Scholar
  40. 40.
    Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259CrossRefGoogle Scholar
  41. 41.
    Driscoll JA, Peters RA II, Cave KR (1998) A visual attention network for a humanoid robot. In: Proceedings of the 1998 IEEE/RSJ international conference on intelligent robotic systems (IROS 1998), VictoriaGoogle Scholar
  42. 42.
    Backer G, Mertsching B, Bollmann M (2001) Data- and model-driven gaze control for an active-vision system. IEEE Trans Pattern Anal Mach Intell 23(12):1415–1429CrossRefGoogle Scholar
  43. 43.
    Breazeal C, Scassellati B (1999) A context-dependent attention system for a social robot. In: Proceedings of the sixteenth international joint conference on artificial intelligence (IJCAI 1999), StockholmGoogle Scholar
  44. 44.
    Vijayakumar S, Conradt J, Shibata T, Schaal S (2001) Overt visual attention for a humanoid robot. In: Proceedings of the 2001 IEEE international conference on intelligence in robotics and autonomous systems (IROS 2001), HawaiiGoogle Scholar
  45. 45.
    Sprague N, Ballard D (2003) Eye movements for reward maximization. In: Advances in neural information processing systems 15. MIT Press, Cambridge, MAGoogle Scholar
  46. 46.
    Choi SB, Jung BS, Ban SW, Lee M (2004) Trainable attention model based vergence control for active stereo vision system. In: Proceedings of the 2004 IEEE international conference on intelligent sensors, sensor networks and information processing (ISSNIP 2004), MelbourneGoogle Scholar
  47. 47.
    Navalpakkam V, Itti L (2005) Modeling the influence of task on attention. Vis Res 45:205–231CrossRefGoogle Scholar
  48. 48.
    Orabona F, Metta G, Sandini G (2005) Object-based visual attention: a model for a behaving robot. In: Proceedings of the 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR 2005), San DiegoGoogle Scholar
  49. 49.
    Ude A, Wyart V, Lin LH, Cheng G (2005) Distributed visual attention on a humanoid robot. In: Proceedings of the 2005 IEEE-RAS international conference on humanoid robots, TsukubaGoogle Scholar
  50. 50.
    Frintrop S (2006) VOCUS: a visual attention system for object detection and goal-directed search. Lecture notes in computer sciences ed., ser. (Lecture notes in artificial intelligence), vol 3899. Springer, HeidelbergGoogle Scholar
  51. 51.
    Begum M, Mann GKI, Gosine RG (2006) A biologically inspired Bayesian model of visual attention for humanoid robots. In: Proceedings of the 2006 IEEE-RAS international conference on humanoid robots (HUMANOIDS 2006), GenovaGoogle Scholar
  52. 52.
    Yu Y, Mann GKI, Gosine RG (2008) An object-based visual attention model for robots. In: Proceedings of the 2008 IEEE international conference on robotics and automation (ICRA 2008), PasadenaGoogle Scholar
  53. 53.
    Vogel J, de Freitas N (2007) Target-directed attention: sequential decision-making for gaze planning. In: Proceedings of the 2007 IEEE international conference on robotics and automation (ICRA 2007), PasadenaGoogle Scholar
  54. 54.
    Borji A, Ahmadabadi MN, Araabi BN (2009) Learning sequential visual attention control through dynamic state space discretization. In: Proceedings of the 2009 IEEE international conference on robotics and automation (ICRA 2009), KobeGoogle Scholar
  55. 55.
    Xu T, Kühnlenz K, Buss M (2008) Information-based gaze control adaptation to scene context for mobile robots. In: Proceedings of the 2008 IEEE international conference on pattern recognition (ICPR 2008), TampaGoogle Scholar
  56. 56.
    Xu T, Wu H, Zhang T, Kühnlenz K, Buss M (2009) Environment adapted active multi-focal vision system for object detection. In: Proceedings of the 2009 IEEE international conference on robotics and automation (ICRA 2009), KobeGoogle Scholar
  57. 57.
    Itti L, Baldi P (2009) Bayesian surprise attracts human attention. Vis Res 49(10):1295–1306, JuneCrossRefGoogle Scholar
  58. 58.
    Xu T, Zhang T, Kühnlenz K, Buss M (2010) Attentional object detection with an active multi-focal vision system. Int J Hum Robot 7(2):223–243CrossRefGoogle Scholar
  59. 59.
    Michalke T, Kastner R, Adamy J, Bone S, Waibel F, Gayko MKA, Gepperth A, Fritsch J, Goerick C (2008) An attention-based system approach for scene analysis in driver assistance. Automatisierungstechnik 56(11):575–584CrossRefGoogle Scholar
  60. 60.
    Aloimonos JY, Weiss I, Bandyopadhyay A (1988) Active vision. Int J Comput Vision 7:333–356CrossRefGoogle Scholar
  61. 61.
    Bajcsy R (1988) Active perception. Proc IEEE 76(8):966–1005, Invited PaperCrossRefGoogle Scholar
  62. 62.
    Aloimonos JY (1990) Purposive and qualitative active vision. In: Proceedings of the 1990 IEEE international conference on pattern recognition, AtlanticGoogle Scholar
  63. 63.
    Ballard DH (1991) Animate vision. Artif Intell 48:57–86CrossRefGoogle Scholar
  64. 64.
    Aloimonos Y (ed) (1993) Active perception. Lawrence Erlbaum, HillsdaleGoogle Scholar
  65. 65.
    Dickmanns ED (1988) An integrated approach to feature based dynamic vision. In: Proceedings of the 1988 IEEE computer society conference on computer vision and pattern recognition (CVPR 1988), Ann ArborGoogle Scholar
  66. 66.
    Davison AJ (1998) Mobile robot navigation using active vision. PhD Thesis, University of OxfordGoogle Scholar
  67. 67.
    Davision AJ, Murray DW (1998) Mobile robot localisation using active vision. In: Proceedings of the 5th European conference on computer vision (ECCV 1998), FreiburgGoogle Scholar
  68. 68.
    Seara JF, Strobl KH, Schmidt G (2002) Information management for gaze control in vision guided biped walking. In: Proceedings of the 2002 IEEE international conference on intelligent robots and systems (IROS 2002), LausanneGoogle Scholar
  69. 69.
    Seara JF, Strobl H, Schmidt G (2003) Path-dependent gaze control for obstacle avoidance in vision guided humanoid walking. In: Proceedings of the 2003 IEEE international conference on robotics and automation (ICRA 2003), TaipeiGoogle Scholar
  70. 70.
    Seara JF, Strobl KH, Martín E, Schmidt G (2003) Task-oriented and situation-dependent gaze control for vision guided humanoid walking. In: Proceedings of the 2003 IEEE-RAS international conference on humanoid robots (HUMANOIDS 2003), MunichGoogle Scholar
  71. 71.
    Seara JF (2004) Intelligent gaze control for vision-guided humanoid walking, ser. Mess-, Steuer- und Regelungstechnik.VDI Verlag 8(1044)Google Scholar
  72. 72.
    Seara JF, Schmidt G (2005) Gaze control strategy for vision-guided humanoid walking. Autmatisierungstechnik 53(2):49–58CrossRefGoogle Scholar
  73. 73.
    Lidoris G, Kühnlenz K, Wollherr D, Buss M (2007) Combined trajectory planning and gaze direction control for robotic exploration. In: Proceedings of the 2007 IEEE international conference on robotics and automation (ICRA 2007), RomeGoogle Scholar
  74. 74.
    Kühnlenz K, Lidoris G, Wollherr D, Buss M (2007) On foveated gaze control and combined gaze and locomotion planning. In: de Pina ACF (ed) Humanoid robots: new developments. I-Tech Education and Publishing, ViennaGoogle Scholar
  75. 75.
    Kühnlenz K, Buss M (2008) Multi-focal vision and gaze control improve navigation performance. J Hum 1(1):33–44Google Scholar
  76. 76.
    Xu T, Kühnlenz K, Buss M (2008) A view direction planning strategy for a multi-camera vision system. In: Proceedings of the 2008 IEEE international conference on information and automation (ICIA 2008), ZhangjiajieGoogle Scholar
  77. 77.
    Davison AJ, Kita N (2001) 3D simultaneous localisation and map-building using active vision for a robot moving on undulating terrain. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition (CVPR 2001), LihueGoogle Scholar
  78. 78.
    Davison AJ, Murray DW (2002) Simultaneous localization and map-building using active vision. IEEE Trans Pattern Anal Mach Intell 24:865–880CrossRefGoogle Scholar
  79. 79.
    Lidoris G, Kühnlenz K, Wollherr D, Buss M (2006) Information-based gaze direction planning algorithm for slam. In: Proceedings of the 2006 IEEE-RAS international conference on humanoid robots (HUMANOIDS 2006), GenovaGoogle Scholar
  80. 80.
    Pellkofer M, Dickmanns ED (2000) EMS-vision: gaze control in autonomous vehicles. In: Proceedings of the 2000 IEEE intelligent vehicles symposium (IV 2000), DearbornGoogle Scholar
  81. 81.
    Pellkofer M (2003) Verhaltensentscheidung für autonome fahrzeuge mit blickrichtungssteuerung. Dissertation, Universität der Bundeswehr MünchenGoogle Scholar
  82. 82.
    Dickmanns ED (2004) Three-stage visual perception for vertebrate-type dynamic machine vision. In: Proceedings of the conference on engineering of intelligent systems (EIS 2004), MadeiraGoogle Scholar
  83. 83.
    Unterholzner A, Wuensche HJ (2009) Adaptive state space control of a camera platform for an autonomous ground vehicle. In: Proceedings of the 2009 IEEE intelligent vehicles symposium (IV 2009), Xi’anGoogle Scholar
  84. 84.
    Unterholzner A, Wuensche HJ (2010) Hybrid adaptive control of a multi-focal vision system. In: Proceedings of the 2010 IEEE intelligent vehicles symposium (IV 2010), San DiegoGoogle Scholar
  85. 85.
    Canudas C, Åström KJ, Braun K (1987) Adaptive friction compensation in DC-motor drives. IEEE J Robotic Autom RA-3(6):681–685CrossRefGoogle Scholar
  86. 86.
    Goodwin GC, Sin KS (1984) Adaptive filtering prediction and control, Information and system sciences. Prentice-Hall, Englewood CliffsGoogle Scholar
  87. 87.
    Ramasubramanian A, Ray LR (2007) Comparison of EKBF-based and classical friction compensation. J Dyn Syst Meas Contr 129:236–242CrossRefGoogle Scholar
  88. 88.
    Franklin GF, Powell JD, Emami-Naeini A (2002) Feedback control of dynamic systems, 4th edn. Prentice Hall, Upper Saddle RiverGoogle Scholar
  89. 89.
    Athans M, Falb PL (1966) Optimal control: an introduction to the theory and its applications. McGraw-Hill, New YorkGoogle Scholar
  90. 90.
    Utkin VI (1977) Variable structure systems with sliding modes. IEEE Trans Autom Control AC-22(2):212–222CrossRefGoogle Scholar
  91. 91.
    Hung JY, Gao W, Hung JC (1993) Variable structure control: a survey. IEEE Trans Ind Electron 40(1):2–22, FebruaryCrossRefGoogle Scholar
  92. 92.
    Chatchanayuenyong T, Parnichkun M (2008) Time optimal hybrid sliding mode-PI control for an autonomous underwater robot. Int J Adv Rob Syst 5(1):91–98Google Scholar
  93. 93.
    You KH, Lee EB (2000) Robust, near time-optimal control of nonlinear second order systems with model uncertainty. In: Proceedings of the 2000 IEEE international conference on control applications, Anchorage, AlaskaGoogle Scholar
  94. 94.
    Iliev B, Kalaykov I (2004) Minimum-time sliding mode control for second-order systems. In: Proceedings of the 2004 American control conference, BostonGoogle Scholar
  95. 95.
    Dean P, Porrill J, Stone JV (2002) Decorrelation control by the cerebellum achieves oculomotor plant compensation in simulated vestibulo-ocular reflex. Proc R Soc Lond Biol Sci 269(1503):1895–1904, AugustCrossRefGoogle Scholar
  96. 96.
    Unterholzner A, Rohland M, Schweitzer M, Wuensche HJ (2010) Vision based online-calibration of inertial gaze stabilization. In: Proceedings of the 2010 IEEE intelligent vehicles symposium (IV 2010), San DiegoGoogle Scholar
  97. 97.
    Julier SJ, Uhlmann JK (2004) Unscented filtering and nonlinear estimation. Proc IEEE 92(3):410–422, Invited PaperCrossRefGoogle Scholar
  98. 98.
    Schweitzer M, Wuensche HJ (2010) Real-time visual odometry for ground moving robots using GPUs. In: Proceedings of the international conference on computer vision theory and applications (VISAPP 2010), AngersGoogle Scholar
  99. 99.
    Scaramuzza D, Fraundorfer F, Pollefeys M, Siegwart R (2009) Absolute scale in structure from motion from a single vehicle mounted camera by exploiting nonholonomic constraints. In: Proceedings of the IEEE international conference on computer vision (ICCV 2009), KyotoGoogle Scholar
  100. 100.
    Schweitzer M, Wuensche HJ (2009) Efficient keypoint matching for robot vision using GPUs. In: Proceedings of the fifth IEEE workshop on embedded computer vision at ICCV 2009, KyotoGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.Institute for Autonomous Systems Technology, Department of Aerospace EngineeringUniversity of the Bundeswehr MunichNeubibergGermany