Abstract
In this paper we present the preliminary analysis and results of a novel robotic system that allows a user to paint with eyes, using an eye tracker device and a collaborative robot. Eye tracking is a sensing technology that allows a computer to detect where a person is looking (point of gaze). This technology is used to control the motion of a collaborative robot that paints on a canvas based on the detected input. Results show the feasibility of the robotic system and its possible future application as a tool for artistic and creative painting by people with disabilities who cannot use their hands, or that have lost the control of all or part of their muscles.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Al-Rahayfeh, A., Faezipour, M.: Eye tracking and head movement detection: a state-of-art survey. IEEE J. Trans. Eng. Health Med. 1, 2100212–2100212 (2013)
Beltramello, A., Scalera, L., Seriani, S., Gallina, P.: Artistic robotic painting using the palette knife technique. Robotics 9(1), 15 (2020)
De Boor, C.: A Practical Guide to Splines, vol. 27. Springer, New York (1978)
El Helou, M., Mandt, S., Krause, A., Beardsley, P.: Mobile robotic painting of texture. In: International Conference on Robotics and Automation (ICRA), pp. 640–647. IEEE (2019)
Gêgo, D., Carreto, C., Figueiredo, L.: Teleoperation of a mobile robot based on eye-gaze tracking. In: 12th International Conference on Information System and Technologies (CISTI), pp. 1–6. IEEE (2017)
Fink, G.: Eye drawings (2020). https://grahamfink.com/eye-drawings
Igno-Rosario, O., Hernandez-Aguilar, C., Cruz-Orea, A., Dominguez-Pacheco, A.: Interactive system for painting artworks by regions using a robot. Robot. Auton. Syst. 121, 103263 (2019)
Karimov, A.I., Kopets, E.E., Rybin, V.G., Leonov, S.V., Voroshilova, A.I., Butusov, D.N.: Advanced tone rendition technique for a painting robot. Robot. Auton. Syst. 115, 17–27 (2019)
Lindemeier, T., Metzner, J., Pollak, L., Deussen, O.: Hardware-based non-photorealistic rendering using a painting robot. Comput. Graph. Forum 34, 311–323 (2015)
Maimon-Mor, R.O., Fernandez-Quesada, J., Zito, G.A., Konnaris, C., Dziemian, S., Faisal, A.A.: Towards free 3d end-point control for robotic-assisted human reaching using binocular eye tracking. In: 2017 International Conference on Rehabilitation Robotics (ICORR), pp. 1049–1054. IEEE (2017)
Majaranta, P., Bulling, A.: Eye tracking and eye-based human–computer interaction. In: Advances in Physiological Computing, pp. 39–65. Springer (2014)
Quintero, C.P., Dehghan, M., Ramirez, O., Ang, M.H., Jagersand, M.: Flexible virtual fixture interface for path specification in tele-manipulation. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 5363–5368. IEEE (2017)
Salvucci, D.D., Goldberg, J.H.: Identifying fixations and saccades in eye-tracking protocols. In: Proceedings of the 2000 Symposium on Eye Tracking Research & Application, pp. 71–78 (2000)
Santella, A., DeCarlo, D.: Abstracted painterly renderings using eye-tracking data. In: Proceedings of the 2nd International Symposium on Non-Photorealistic Animation and Rendering, pp. 75–82 (2002)
Scalera, L., Mazzon, E., Gallina, P., Gasparetto, A.: Airbrush robotic painting system: experimental validation of a colour spray model. In: International Conference on Robotics in Alpe-Adria Danube Region, pp. 549–556. Springer (2017)
Scalera, L., Seriani, S., Gasparetto, A., Gallina, P.: Busker robot: a robotic painting system for rendering images into watercolour artworks. In: IFToMM Symposium on Mechanism Design for Robotics, pp. 1–8. Springer (2018)
Scalera, L., Seriani, S., Gasparetto, A., Gallina, P.: Non-photorealistic rendering techniques for artistic robotic painting. Robotics 8(1), 10 (2019)
Scalera, L., Seriani, S., Gasparetto, A., Gallina, P.: Watercolour robotic painting: a novel automatic system for artistic rendering. J. Intell. Robot. Syst. 95(3–4), 871–886 (2019)
Song, D., Lee, T., Kim, Y.J., Sohn, S., Kim, Y.J.: Artistic pen drawing on an arbitrary surface using an impedance-controlled robot. In: IEEE International Conference on Robotics and Automation (ICRA) (2018)
This person does not exist (2020). https://www.thispersondoesnotexist.com/
Tobii Technology, Sweden. This is eye-tracking (2020). https://www.tobii.com/
Tramonte, S., Sorbello, R., Guger, C., Chella, A.: Acceptability study of A3–K3 robotic architecture for a neurorobotics painting. Front. Neurorobot. 12, 81 (2019)
Tresset, P., Leymarie, F.F.: Portrait drawing by paul the robot. Comput. Graph. 37(5), 348–363 (2013)
Trigatti, G., Boscariol, P., Scalera, L., Pillan, D., Gasparetto, A.: A look-ahead trajectory planning algorithm for spray painting robots with non-spherical wrists. In: IFToMM Symposium on Mechanism Design for Robotics, pp. 235–242. Springer (2018)
Trigatti, G., Boscariol, P., Scalera, L., Pillan, D., Gasparetto, A.: A new path-constrained trajectory planning strategy for spray painting robots-rev. 1. Int. J. Adv. Manuf. Technol. 98(9–12), 2287–2296 (2018)
Vempati, A.S., Khurana, H., Kabelka, V., Flueckiger, S., Siegwart, R., Beardsley, P.: A virtual reality interface for an autonomous spray painting UAV. IEEE Robot. Autom. Lett. 4(3), 2870–2877 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Scalera, L., Seriani, S., Gasparetto, A., Gallina, P. (2021). A Novel Robotic System for Painting with Eyes. In: Niola, V., Gasparetto, A. (eds) Advances in Italian Mechanism Science. IFToMM ITALY 2020. Mechanisms and Machine Science, vol 91. Springer, Cham. https://doi.org/10.1007/978-3-030-55807-9_22
Download citation
DOI: https://doi.org/10.1007/978-3-030-55807-9_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-55806-2
Online ISBN: 978-3-030-55807-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)