Abstract
Robots are usually operated through text-based inputs made on an external computer or through an associated human machine interface (HMI). This requires skill and expert knowledge to take full advantage of the robot as machine, tool and extension of the human operator, thus limiting applications for users who hold manual skills but not machine knowledge. Consequently, this research aims to identify processes that allow a non-specialist to operator a robot with a similar ease as a specialist. This paper presents research into minimizing (or fully negating) text-based programming for robotic fabrication, thereby opening a potential for adopting robotic fabrication by users with a broader level of skills. This can be achieved by introducing a process for non-specialists to use a semantic drawn language, whereby manual instructions are drawn on a workpiece before being robotically processed. The language can be extended by the operator through interaction with a machine learning (ML) system operated on an HMI, which parses the language and informs the robot what to do. The paper discusses further research into a previously developed tablet interface framework that manages this process, and specifically details the process of adding ML functionalities that can continuously improve the framework. It describes the development process of a data gathering method; provides an overview use cases for classification results and choice of training system; and discusses results and limitations, with discussion of future work.
Similar content being viewed by others
Data availability
The datasets will be placed on GitHub.
Code availability
Not available.
References
Andraos S (2015) DMR: a semantic robotic control language. In: Martens B, Wurzer G, Grasl T, Lorenz WE, Schaffranek R (Eds), Real Time–Proceedings of the 33rd ECAADe Conference–Vol 2, Vienna University of Technology, Vienna, Austria, 16–18 September 2015, pp. 261–268. CUMINCAD, http://papers.cumincad.org/cgi-bin/works/paper/ecaade2015_246
Apolinarska AA, Ralph B, Reto F, Fabio G, Matthias K (2016) Mastering the “Sequential Roof” computational methods for integrating design, structural analysis, and robotic fabrication. Adv Archit Geom. https://doi.org/10.3218/3778-4_17 (vdf Hochschulverlag AG)
Associates McNeel &. ‘Grasshopper—New in Rhino 6’. www.rhino3d.com. Accessed 29 Mar 2022. https://www.rhino3d.com/6/new/grasshopper/
Bauer W, Bender M, Martin B, Peter R, Oliver S. (20160 Lightweight robots in manual assembly–best to start simply! Examining companies' initial experiences with lightweight robots
Bishop CM (2006) Pattern recognition and machine learning. Softcover reprint of the original, 1st edn. Springer, New York, NY
FastForestBinaryTrainer Class (Microsoft.ML.Trainers.FastTree). https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.trainers.fasttree.fastforestbinarytrainer. Accessed 17 Aug 2022
Food4Rhino (2011a) ‘HAL|Robot Programming & Control’. Text, 23 November. https://www.food4rhino.com/en/app/hal-robot-programming-control
Food4Rhino (2011b) ‘KUKA|prc—Parametric Robot Control for Grasshopper’. Text, 21 October. https://www.food4rhino.com/en/app/kukaprc-parametric-robot-control-grasshopper
Food4Rhino (2016) ‘Taco ABB’. Text, 2 March. https://www.food4rhino.com/en/app/taco-abb
Fragkia V, Isak WF, Anke P (2021) Predictive information modeling: machine learning strategies for material uncertainty. Technol Archit Design 5(2):163–76. https://doi.org/10.1080/24751448.2021.1967057
Gomez C, Carlos MP, Valeria R, Elizabeth J (2021) The robot is present: creative approaches for artistic expression with robots. Front Robotics AI, 8. https://doi.org/10.3389/frobt.2021.662249. https://www.frontiersin.org/articles/
Green T, Blackwell A (1998) Cognitive dimensions of information artefacts: a tutorial. BCS HCI Conf 98:1–75
Heidegger M (2008) Being and time. Reprint edition. Harper Perennial Modern Classics, New York
Johns RL, Axel K, Nicholas F (2022) Design approaches through augmented materiality and embodied computation. springerprofessional.de. https://www.springerprofessional.de/en/design-approaches-through-augmented-materiality-and-embodied-com/2090894. Accessed 21 Nov 2022
LickLider JCR (1960) Man-Computer symbiosis. IRE Trans Human Factors in Electronics, (n.d.): 4–11
LinearSvmTrainer Class (Microsoft.ML.Trainers) (2022) https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.trainers.linearsvmtrainer. Accessed 17 Aug 2022
Microsoft (2022) ‘ML.NET|Machine Learning Made for .NET’. Accessed 21 Jan 2022. https://dotnet.microsoft.com/en-us/apps/machinelearning-ai/ml-dotnet. ‘MNIST Handwritten Digit Database, Yann LeCun, Corinna Cortes and Chris Burges’. http://yann.lecun.com/exdb/mnist/. Accessed 16 Aug 2022
OneVersusAllTrainer Class (Microsoft.ML.Trainers) (2022) https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.trainers.oneversusalltrainer. Accessed 17 Aug 2022
Papers with code—an ensemble of simple convolutional neural network models for MNIST digit recognition (2022) https://paperswithcode.com/paper/an-ensemble-of-simple-convolutional-neural. Accessed 16 Aug 2022
Pedersen J, Narendrakrishnan N, Jay H, Asbjørn S, Dagmar R (2020) Augmented drawn construction symbols: a method for Ad Hoc robotic fabrication. Int J Archit Computing 18(3):254–69. https://doi.org/10.1177/1478077120943163 (Redacted for anonymity)
Pedersen J, Asbjørn S, Dagmar R (2021) Hand-drawn digital fabrication: calibrating a visual communication method for robotic on-site fabrication’. Construction Robotics 5(2):159–73. https://doi.org/10.1007/s41693-020-00049-2 (Redacted for anonymity)
Rhinoceros 3D (2022) www.rhino3d.comhttps://www.rhino3d.com/. Accessed 29 Mar 2022
Rossi G, Paul N (2019) Haptic learning: ECAADe. In: José Pedro S, Joäo PX, Goncalo CH (eds). Architecture in the Age of the 4th Industrial Revolution, ECAADE SIGRADI 2019 Architecture in the age of the 4th Industrial revolution, pp 201–10
Sennett R (2009) The craftsman, 1st edn. Yale University Press, New Haven, Conn
Smigielska M (2018) Application of machine learning within the integrative design and fabrication of robotic rod bending processes, 523–36. https://doi.org/10.1007/978-981-10-6611-5_44
Technologies, Unity (2022) ‘Get Your Unity Pro Subscription Today|Unity’. https://unity.com/pages/unity-pro-buy-now. Accessed 29 Mar 2022
Thoma A, Arash A, Matthias H, Thomas W, Fabio G, Matthias K (2018) Robotic fabrication of bespoke timber frame modules. https://doi.org/10.1007/978-3-319-92294-2_34
Wu K, Axel K (2019) Designing natural wood log structures with stochastic assembly and deep learning. In: Jan W, Philippe B, Marco H, Kendra B, Tim S (eds) In robotic fabrication in architecture, art and design 2018. Springer International Publishing, Cham, pp 16–30. https://doi.org/10.1007/978-3-319-92294-2_2
Funding
The research is partly funded by the Danish Innovation fund.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Not applicable.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Pedersen, J., Reinhardt, D. Robotic drawing communication protocol: a framework for building a semantic drawn language for robotic fabrication. Constr Robot 6, 239–249 (2022). https://doi.org/10.1007/s41693-022-00089-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41693-022-00089-w