Abstract
This paper introduces a novel concept of visual-haptic display for situational awareness improvement for crowded and low altitude airspace situations. The visual augmentation display that constitutes of Virtual Fences delimiting no-fly zones, and a specific tri-dimensional highlight graphics that enhances visibility of other remotely piloted or autonomous agents, as well as conventional manned aircraft operating in the area is presented first. Then the Shared Control paradigm and the Haptic Force generation mechanism, based on a Proportional-Derivative-like controller applied to repulsive forces generated by the Virtual Fences and other UAVs are introduced and discussed. Simulations with 26 pilots were performed in a photo-realistic synthetic environment showing that the combined use of Visual-haptic feedback outperforms the Visual Display only in helping the pilot keeping a safe distance from no-fly zones and other vehicles.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sachs, G.: Perspective predictor/flight-path display with minimum pilot compensation. J. Guid. Control Dyn. 23(3), 420–429 (2000)
Mulder, M., Veldhuijzen, A.R., van Paassen, M.M., Mulder, J.A.: Integrating fly-by-wire controls with perspective flight-path displays. J. Guid. Control Dyn. 28(6), 1263–1274 (2005)
Giulietti, F., Pollini, L., Avanzini, A.: Visual aids for safe operation of remotely piloted vehicles in the controlled air space. In: Proceedings of Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering, Published online before print 7 March 2016 (2016). doi:10.1177/0954410016632014
Innocenti, M., Pollini, L., Giulietti, F.: Visual tools for man-machine interface real time simulation. In: Proceedings of IEEE Symposium on Information Technology in Mechatronics, ITM 2001, Istanbul, Turkey, October 2001 (2001)
Avanzini, G.: Frenet-based algorithm for trajectory prediction. J. Guid. Control Dyn. 27(1), 127–135 (2004)
Lam, T.M., Boschloo, H.W., Mulder, M., van Paassen, M.M.: Artificial force field for haptic feedback in UAV teleoperation. IEEE Trans. Syst. Man Cybern. Part A: Syst. Hum. 39(6), 1316–1330 (2009)
Lam, T.M., Mulder, M., van Paassen, M.M., Mulder, J.A., van Der Helm, F.C.T., Force-stiffness feedback in UAV tele-operation with time delay. In: AIAA Guidance, Navigation, and Control Conference, Chicago, Illinois, August 2009
Diolaiti, N., Melchiorri, C.: Tele-operation of a mobile robot through haptic feedback. In: IEEE International Workshop on Haptic Virtual Environments and Their Applications (HAVE 2002), Ottawa, Ontario, Canada, November (2002)
Barnes, D.P., Counsell, M.S.: Haptic communication for remote mobile manipulator robot operations. In: Proceedings of 8th Topical Meeting on Robotics and Remote Systems. American Nuclear Society, Pittsburgh (1999)
Farkhatdinov, I., Ryu, J,-H., An, J.: A preliminary experimental study on haptic teleoperation of mobile robot with variable force feedback gain. In: Haptics Symposium, 25–26 March 2010. IEEE (2010)
Horan, B., Crelghton, D., Nahavandi, S., Jamshidi, M., Bilateral haptic teleoperation of an articulated track mobile robot. In: IEEE International Conference on System of Systems Engineering, SoSE 2007, 16–18 April 2007 (2007)
Mitsou, N.C., Velanas, S.V., Tzafestas, C.S.: Visuo-haptic interface for teleoperation of mobile robot exploration tasks. In: The 15th IEEE International Symposium on Robot and Human Interactive Communication, ROMAN 2006, 6–8 September 2006, pp. 157–163 (2006)
Rsch, O.J., Schilling, K., Roth, H.: Haptic interfaces for the remote control of mobile robots. Control Eng. Pract. 10(11), 1309–1313 (2002)
Lee, S., Sukhatme, G.S., Kim, G.J., Park, C.-M.: Haptic control of a mobile robot: a user study. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 3, pp. 2867–2874 (2002)
De Stigter, S., Mulder, M., van Paassen, M.M.: Design and evaluation of a haptic flight director. J. Guid. Control Dyn. 30(1), 35–46 (2007)
Abbink, D.A., Boer, E.R., Mulder, M.: Motivation for continuous haptic gas pedal feedback to support car following. In: Proceedings of IEEE Intelligent Vehicles Symposium, pp. 283–290 (2008)
Abbink, D., Mulder, M., Boer, E.R.: Haptic shared control: smoothly shifting control authority? Cogn. Technol. Work 14(1), 19–28 (2012)
Goodrich, K., Schutte, P., Williams, R.: Haptic-multimodal flight control system update. In: Proceedings of 11th AIAA Aviation Technology, Integration, and Operations (ATIO) Conference, pp. 1–17 (2011)
Olivari, M., Nieuwenhuizen, F.M., Buelthoff, H.H., Pollini, L.: Pilot adaptation to different classes of haptic aids in tracking tasks. J. Guid. Control Dyn. 37(6), 1741–1753 (2014)
Maimeri, M., Olivari, M., Buelthoff, H.H., Pollini, L.: On effects of failures in haptic and automated pilot support systems. In: Proceedings of AIAA Modeling and Simulation Technologies Conference, pp. 1–12 (2016)
D’Intino, G., Olivari, M., Geluardi, S., Venrooij, J., Innocenti, M., Buelthoff, H.H., Pollini, L.: Evaluation of haptic support system for training purposes in a tracking task. In: IEEE Systems, Men, and Cybernetics Society Conference (2016)
Petermeijer, S., Abbink, D., Mulder, M., de Winter, J.: The effect of haptic support systems on driver performance: a literature survey. IEEE Trans. Haptics 8(4), 467–479 (2015)
Alaimo, S.M.C., Pollini, L., Bresciani, J.P., Blthoff, H.H.: A comparison of direct and indirect haptic aiding for remotely piloted vehicles. In: Proceedings of 19th IEEE International Symposium in Robot and Human Interactive Communication (IEEE Ro-Man 2010)
Alaimo, S.M.C., Pollini, L., Bresciani, J.-P., Blthoff, H.H.: Experiments of direct and indirect haptic aiding for remotely piloted vehicles with a mixed wind gust rejection/obstacle avoidance task. In: AIAA Modeling and Simulation Technologies Conference 2011 (MST-2011), Portland, Oregon (2011)
Profumo, L., Pollini, L., Abbink, D.A.: Direct and indirect haptic aiding for curve negotiation. In: IEEE International Conference on Systems, Man, and Cybernetics (2013)
Alaimo, S.M.C., Pollini, L., Innocenti, M., Bresciani, J.P., Buelthoff, H.H.: Experimental comparison of direct and indirect haptic aids in support of obstacle avoidance for remotely piloted vehicles. J. Mech. Eng. Autom. 2(10), 2159–5275 (2012)
Di Corato, F., Innocenti, M., Pollini, L.: Combined vision-inertial navigation for improved robustness. In: IEEE Israel Itzhack Y. Bar-Itzhack Memorial Symposium on Estimation, Navigation, and Spacecraft Control, Haifa, Israel, 14–17 October 2012
Di Corato, F., Innocenti, M., Pollini, L.: Robust vision-aided inertial navigation algorithm via entropy-like relative pose estimation. Gyroscopy Navig. 4(1), 1–13 (2013)
Pollini, L., Metrangolo, A.: Simulation and robust backstepping control of a quadrotor aircraft. In: AIAA Modeling and Simulation Technologies Conference, Honolulu, Hawaii (2008)
Pollini, L., Innocenti, M.: A synthetic environment for dynamic systems control and distributed simulation. IEEE Control Syst. Mag. 20(2), 49–61 (2000)
Hutton, J., Mostafa, M.M.R.: 10 years of direct georeferencing for airborne photogrammetry. GIS Bus. (GeoBit) 11(1), 33–41 (2005)
Nebiker, S., Eugster, H., Flckiger, K., Christen, M.: Planning and management of real-time geospatial UAS missions within a virtual globe environment. In: UAV-g 2011, Conference on Unmanned Aerial Vehicle in Geomatics (2011)
Pollini, L., Razzanelli, M., Olivari, M., Brandimarti, A., Maimeri, M., Pazzaglia, P., Pittiglio, G., Nuti, R., Innocenti, M., Buelthoff, H.H.: Design, realization and experimental evaluation of a haptic stick for shared control studies. In: The 13th IFAC/IFIP/IFORS/IEA Symposium on Analysis, Design, and Evaluation of Human-Machine Systems, Kyoto, Japan (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Razzanelli, M. et al. (2016). A Visual-Haptic Display for Human and Autonomous Systems Integration. In: Hodicky, J. (eds) Modelling and Simulation for Autonomous Systems. MESAS 2016. Lecture Notes in Computer Science(), vol 9991. Springer, Cham. https://doi.org/10.1007/978-3-319-47605-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-47605-6_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-47604-9
Online ISBN: 978-3-319-47605-6
eBook Packages: Computer ScienceComputer Science (R0)