Summary
In this chapter we discuss challenging control problems in multi-modal telepresence and teleaction systems. The typical system structure consists of a human operator, the possibly distant (and differently scaled) teleoperator and various types of barriers between them. Fundamental control modes to organize and distribute control between human and system are discussed. Potential applications and enabling technologies are sampled. A subjective listing of central control issues is given including multi-modal control, impact of time-delays, local control measures, real world, virtual environment (VE) and augmented environment interaction, sensory substitution, etc. An experimental prototype system for multi-modal telepresence and teleaction developed in our laboratory is presented and some experiments further illustrating typical control challenges are highlighted.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Anderson, R. (1996). Autonomous, Teleoperated, and Shared Control of Robot Systems. In Proceedings of the IEEE International Conference on Robotics and Automation, Minneapolis, Minnesota, pp. 2025–2032.
Anderson, R. and M. Spong (1989). Bilateral Control of Teleoperators with Time Delay. IEEE Transaction on Automatic Control 34, 494–501.
Arimoto, S. (1991). Learning for Skill Refinement. In Proceedings of the International Workshop on Intelligent Robots and Systems IROS, pp. 10–13.
Baier, H., M. Buss, F. Freyberger, J. Hoogen, P. Kammermeier, and G. Schmidt (1999). Distributed PC-Based Haptic, Visual and Acoustic Telepresence System—Experiments in Virtual and Remote Environments. In Proceedings of the IEEE Virtual Reality Conference VR’99, Houston, TX, pp. 118–125.
Baier, H., M. Buss, and G. Schmidt (1997). Control Mode Switching for Teledrilling Based on a Hybrid System Model. In Proceedings of the IEEE/ASME International Conference on Advanced Intelligent Mechatronics AIM’97, Tokyo, Japan, Paper No. 50.
Baier, H. and G. Schmidt (1998). Control Aspects of Closed-Loop Haptic Tele-operation. In Proceedings of the 7th International Workshop on Robotics in Alpe-Adria-Danube Region (RAAD’98), Smolenice, Slovakia, pp. 151–156.
Barfield, W., C. Hendrix, O. Bjorneseth, K. Kaczmarek, and W. Lotens (1995, Fall). Comparison of Human Sensory Capabilities with Technical Specifications of Virtual Environment Equipment. Presence 4(4), 329–356.
Basdogan, C., C. Ho, and M. Srinivasan (1997). Haptic Rendering: Point- and Ray-Based Interactions. In Proceedings of the Second PHANToM Users Group Workshop, Dedham, MA.
Bejczy, A., W. Kim, and S. Venema (1990). The phantom robot: Predictive displays for teleoperation with time delay. In Proceedings of the IEEE International Conference on Robotics and Automation, Cincinnati, Ohio, pp. 546–551.
Brooks, T. (1990). Telerobotic response requirements. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, pp. 113–120.
Burdea, G. (1996). Force and Touch Feedback for Virtual Reality. New York: John Wiley & Sons.
Burdea, G. and J. Zhuang (1991). Dexterous telerobotics with force feedback — an overview, part 1: Human factors, part 2: Control and implementation. Robotica 9, 171–178.
Burdea, G. and J. Zhuang (1991). Dexterous telerobotics with force feedback — an overview, part 1: Human factors, part 2: Control and implementation. Robotica 9, 291–298.
Buss, M. (1994, June). Study on Intelligent Cooperative Manipulation. Ph. D. thesis, University of Tokyo, Tokyo.
Buss, M. and H. Hashimoto (1993a). Human manipulative skill analysis and acquisition for the intelligent assisting system. In Proceedings 12th IFAC World Congress, International Federation of Automatic Control, Volume 9, Sydney, Australia, pp. 321–326.
Buss, M. and H. Hashimoto (1993b). Skill acquisition and transfer system as approach to the intelligent assisting system—ias. In Proceedings of the 2nd IEEE Conference on Control Applications, Vancouver, British Columbia, Canada, pp. 451–456.
Buss, M. and H. Hashimoto (1994). Manipulation skill modeling for dexterous hands. In Proceedings of the IEEE International Conference on Robotics and Automation, San Diego, California, pp. 3403–3408.
Buss, M. and H. Hashimoto (1995). Motion scheme for dextrous manipulation in the intelligent cooperative manipulation system—icms. In V. Graefe (Ed.), Intelligent Robots and Systems, pp. 279–294. Amsterdam: Elsevier Science.
Caldwell, D., S. Lawther, and A. Wardle (1996). Tactile Perception and its Application to the Design of Multi-modal Cutaneous Feedback Systems. In Proceedings of the IEEE International Conference on Robotics and Automation, Minneapolis, Minnesota, pp. 3215–3221.
Card, S., T. Moran, and A. Newell (1983). The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates.
Colgate, J., P. Grafing, M. Stanley, and G. Schenkel (1993). Implementation of stiff virtual walls in force-reflecting interfaces. In Proceedings of the IEEE Virtual Reality Annual International Symposium VRAIS, pp. 20–26.
Delson, N. and H. West (1993). Robot programming by human demonstration: Subtask compliance controller identification. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS, pp. 33–41.
Denk, J. (1997). Teleoperation of the ROMAN-Gripper over the Internet. Internal Report, Institute of Automatic Control Engineering, Technical University of Munich.
Deussen, O. and Kuhn T (1995). Echtzeitsimulation deformierbarer Objekte über nodale Modelle. In Proc. Integration von Bild, Modell und Text, ASIM Mitteilungen No. 46, Universität Magdeburg.
Doel, K. and D. Pai (1998, August). The Sounds of Physical Shapes. Presence 7(4), 382–395.
Fischer, C., M. Buss, and G. Schmidt (1996). HuMan-Robot-Interface for Intelligent Service Robot Assistance. In Proceedings of the IEEE International Workshop on Robot and Human Communication (ROMAN), Tsukuba, Japan, pp. 177–182.
Friedrich, H., J. Holle, and R. Dillmann (1998). Interactive Generation of Flexible Robot Programs. In Proceedings of the IEEE International Conference on Robotics and Automation, Leuven, Belgium, pp. 538–543.
Hall, A., S. Kolssouzidis, and T. Schickinger (1998, June). Modeling of Deformable Objects in Virtual Worlds. Internal Report, Institute of Automatic Control Engineering, Technical University of Munich.
Handwerker, H. (1989). General Sensory Physiology. In R. Schmidt and G. Thews (Eds.), Human Physiology, pp. 176–195. Berlin Heidelberg New York London Paris Tokyo Hong Kong: Springer-Verlag.
Hannaford, B. (1989, August). A design framework for teleoperators with kinesthetic feedback. IEEE Transactions on Robotics and Automation 5(4), 426–434.
Hendrix, C. and W. Barfield (1996, Summer). The Sense of Presence within Auditory Virtual Environments. Presence 5(3), 290–301.
Hirzinger, G. (1993). Rotex-the first robot in space. In Proceedings of the ICAR International Conference on Advanced Robotics, pp. 9–33.
Hollander, A. (1994). An exploration of virtual auditory shape perception. Master’s thesis, Human Interface Technology Laboratory, University of Washington, Seattle, WA.
Hoogen, J. (1998). Development of an Experimental System for Kinesthetic Feedback in Telepresence Systems. Internal Report, Institute of Automatic Control Engineering, Technical University of Munich.
Johannsen, G. (1993). Mensch-Maschine-Systeme. Springer.
Kaczmarek, K., M. Tyler, and P. Bach-y-Rita (1994). Electrotactile Haptic Displays on the Fingertips. In Proceedings of the IEEE International Conference on Engineering in Medicine and Biology, New York.
Kammermeier, P. (1997). Development of a Haptic Display for Tactile Feedback in Telepresence Systems. Internal Report, Institute of Automatic Control Engineering, Technical University of Munich.
Kazerooni, H., T.-I. Tasy, and K. Hollerbach (1993). A Controller Design Framework for Telerobotic Systems. IEEE Transaction Control Systems Technology 11, 105–116.
Kim, W. (1996, Spring). Virtual Reality Calibration and Preview/Predictive Displays for Telerobotics. Presence 5(2), 173–190.
Kontarinis, D. and R. Howe (1995, Fall). Tactile Display of Vibratory Information in Teleoperation and Virtual Environments. Presence 4(4), 387–402.
Kosuge, K., Y. Fujisawa, and T. Fukuda (1993). Control of robot directly maneuvered by operator. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS, pp. 49–54.
Kosuge, K., T. Itoh, T. Fukuda, and M. Otsuka (1995). Scaled Telemanipulation System Using Semi-Autonomous Task-Oriented Virtual Tool. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS, Pittsburgh, PA, pp. 124–129.
Kosuge, K., H. Murayama, and K. Takeo (1996). Bilateral Feedback Control of Telemanipulators via Computer Network. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS, Osaka, Japan, pp. 1380–1385.
Kunii, Y., M. Buss, and H. Hashimoto (1995). Force feedback using dynamic force simulator. In Video Proceedings of the IEEE International Conference on Robotics and Automation, Nagoya, Japan, pp. 15.
Kunii, Y. and H. Hashimoto (1996). Dynamic force simulator for haptic interface. In J. Gertler, J. Cruz, and M. Peshkin (Eds.), Preprints of the 13th World Congress, Volume L, pp. 403–408. San Francisco: International Federation of Automatic Control—IFAC.
Kunii, Y. and H. Hashimoto (1997). Tele-teaching by human demonstration in virtual environment for robotic network system. In Proceedings of the IEEE International Conference on Robotics and Automation, Albuquerque, New Mexiko, pp. 405–410.
Lawrence, D. (1993, October). Stability and Transparency in Bilateral Teleoperation. IEEE Transactions on Robotics and Automation 9(5), 624–637.
Leung, G., B. Francis, and A. Apkarian (1993). Bilateral controller for teleoperators with time delay via μ-synthesis. IEEE Transactions on Robotics and Automation 10, 105–116.
Mann, S. (1998, November). Humanistic Computing: “WearComp” as a New Framework and Application for Intelligent Signal Processing. Proceedings of the IEEE 86(11), 2123–2151.
Massie, T. and J. Salisbury (1994). The PHANToM Haptic Interface: A Device for Probing Virtual Objects. In Proc. of the ASME Winter Annual Meeting, Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Chicago.
Massimino, M. and T. Sheridan (1994). Sensory Substitution for Force Feedback in Teleoperation. Presence 2(4).
Mitsuishi, M., T. Hori, and T. Nagao (1994). Predictive Information Display for Tele-Handling/Machining System. In Proceedings of the IEEE/RSJ/GI International Conference on Intelligent Robots and Systems IROS, Munich, Germany, pp. 260–267.
Nahvi, A., D. Nelson, J. Hollerbach, and D. Johnson (1998). Haptic Manipulation of Virtual Mechanisms from Mechanical CAD Designs. In Proceedings of the IEEE International Conference on Robotics and Automation, Leuven, Belgium, pp. 375–380.
Niemeyer, G. and J. E. Slotine (1998). Towards Force-Reflecting Teleoperation Over the Internet. In Proceedings of the IEEE International Conference on Robotics and Automation, Leuven, Belgium, pp. 1909–1915.
Rest, E. L., J.-L. Fleureau, and L. Marcé (1997). Semantics and implementation of a language for telerobotics. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS, Grenoble, France, pp. 1807–1812.
Rosenberg, L. and B. Adelstein (1993). Perceptual decomposition of virtual haptic surfaces. In Proceedings of the IEEE Symposium on Research Frontiers in Virtual Reality, San Jose, CA, pp. 46–53.
Salcudean, S. (1998). Control for Teleoperation and Haptic Interfaces. In B. Siciliano and K. Valavanis (Eds.), Lecture Notes in Control and Information Sciences 230: Control Problems in Robotics and Automation, pp. 51–66. London: Springer.
Salcudean, S. and T. Vlaar (1994). On the emulation of stiff walls and static friction with a magnetically levitated input/output device. In Proceedings of ASME WAM, New York, pp. 303–309.
Satava, R. and S. Jones (1997). Virtual environments for medical training and education. Presence 6, 139–146.
Schloerb, D. (1995, Winter). A Quantitative Measure of Telepresence. Presence 4(1), 64–80.
Sheridan, T. (1992). Telerobotics, Automation, and Human Supervisory Control. Cambridge, Massachusetts: MIT Press.
Sheridan, T. (1996, Spring). Further Musings on the Psychophysics of Presence. Presence 5(2), 241–246.
Shimoga, K. (1993a). A survey of perceptual feedback issues in dexterous telemanipulation: Part i. In Proceedings of the IEEE Virtual Reality Annual International Symposium VRAIS, pp. 263–270.
Shimoga, K. (1993b). A survey of perceptual feedback issues in dexterous telemanipulation: Part ii. In Proceedings of the IEEE Virtual Reality Annual International Symposium VRAIS, pp. 271–279.
Sitti, M. and H. Hashimoto (1998). Tele-Nanorobotics using Atomic Force Microscope. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS, Vancouver, Canada, pp. 1739–1746.
Stein, M. and R. Paul (1994). Operator interaction, for time-delayed teleoperation, with a behavior-based controller. In Proceedings of the IEEE International Conference on Robotics and Automation, San Diego, California, pp. 231–236.
Thompson, T., D. Johnson, and E. Cohen (1997). Direct haptic rendering of sculptured models. In Proceedings Symposium on Interactive 3D Graphics, Providence, pp. 167–176.
Thompson, T., D. Nelson, E. Cohen, and J. Hollerbach (1997). Maneuverable NURBS models within a haptic virtual environment. In Proceedings ASME Dynamic Systems and Control Division, Dallas, pp. 37–44.
Welch, R., T. Blackmon, A. Liu, B. Meilers, and L. Stark (1996, Summer). The Effects of Pictorial Realism, Delay of Visual Feedback, and Observer Interactivity on the Subjective Sense of Presence. Presence 5(3), 263–273.
Wrenzel, E., F. Wightman, D. Kistler, and S. Foster (1988). The Convolvotron: Realtime synthesis of out-of-head localization. In Second Joint Meeting Acoustic Societies of America and Japan, Honolulu, HI.
Wermuth, S. (1998, Mai). Quantitative Investigations of Multi-Modal Telepresence Quality in Visual-Haptic Telepresence Systems. Internal Report, Institute of Automatic Control Engineering, Technical University of Munich.
Yan, J. and S. Salcudean (1993). Teleoperation controller design using h ∞ optimization. IEEE Transaction Control Systems Technology 4, 244–258.
Yang, B. and H. Asada (1990). Skill acquisition for robot grinding: Learning of high-level feedback laws from teaching data. In Proceedings of the 1990 JAPAN- U.S.A Symposium on Flexible Automation, pp. 649–655.
Yang, J., Y. Xu, and C. Chen (1993). Hidden markov model approach to skill learning and its application in telerobotics. In Proceedings of the IEEE International Conference on Robotics and Automation, Atlanta, Georgia, pp. 396–402.
Zelek, J. (1997). Human-robot interaction with a minimal spanning natural language template for autonomous & tele-operated control. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS, Grenoble, France, pp. 299–305.
Zilles, C. and J. Salisbury (1995). A Constraint-based God-object Method For Haptic Display. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag London Limited
About this paper
Cite this paper
Buss, M., Schmidt, G. (1999). Control Problems in Multi-Modal Telepresence Systems. In: Frank, P.M. (eds) Advances in Control. Springer, London. https://doi.org/10.1007/978-1-4471-0853-5_3
Download citation
DOI: https://doi.org/10.1007/978-1-4471-0853-5_3
Publisher Name: Springer, London
Print ISBN: 978-1-4471-1216-7
Online ISBN: 978-1-4471-0853-5
eBook Packages: Springer Book Archive