1 Introduction

Within the field of architecture there exist many domain-specific challenges to the development of large-scale customized production and construction. The building scale, in most cases, exceeds the working space of standard production tools, i.e. industrial 6-axis robots commonly used in fabrication. Additionally, tolerances in the realm of construction engage a different magnitude of scale than those found in manufacturing. Other industries requiring similar large-scale production, including the aerospace or marine industries, benefit from the economy-of-scale implicit in serialized mass production. Most buildings, however, are unique, custom-built or one-off constructions that are assembled on-site, reducing the applicability of such strategies. Moreover, an additional barrier hindering large-scale automation in construction is the complexity and diversity of physical tasks associated with typical building and assembly processes.

Many existing approaches have been developed in recent years to enable large-scale continuous robotic fabrication to surpass the inherent limitations of single robots or machines in a scenario such as a construction site. Predominantly, these approaches are based on hybridizing or integrating appropriated technologies with other systems, including mobile platforms or digitally controlled construction equipment, achieving an enhanced workspace through mobility or integration. Though many other research endeavors have begun to investigate multi-robot collaboration, the extent of these investigations in the domain of fabrication and construction are limited and in most cases the diversity of the machines, robots, and sensors has been minimized. In contrast, this research considers the benefits of distributed, heterogeneous and multi-robotic systems, which can offer a more scalable and adaptable solution to the various constraints of construction automation. Here, a heterogeneous system is defined as a system made up of robots with varying specifications, tooling, and control logics which may be from different manufacturers or may be customized from off-the-shelf components and controllers.

To demonstrate the potential of a distributed multi-robotic fabrication setup, a demonstrator was developed exploring lightweight fiber composite construction for long-span architectural applications (Fig. 1). Fiber composite construction processes, particularly those utilizing fiber winding strategies, can exhibit a high degree of automation. These highly efficient fabrication systems are commonly used to make compression tanks, wind turbine blades and aircraft fuselages due to their high strength to weight ratio and ease of formability. Composite fabrication offers an opportunity to integrate a wide range of architectural systems such as structure, form, enclosure and connections while requiring only a limited range of materials, processes and tasks. Despite the inherent potential of this material system, there is no standard industrialized processes for fiber composite production that is commonly used to automate large-scale architectural construction.

Fig. 1
figure 1

Fabrication setup enabling a continuous long span lament winding process, achieved through physical collaboration between a custom UAV and two industrial robots

Full size image

The presented research explores the possibilities of this apparent gap in construction automation through the development and utilization of a custom UAV to supplement the limited working space of industrial, high-payload, six-axis robots in a collaborative fabrication process for the production of a lightweight, materially efficient, long-span fiber composite cantilever. This paper goes into extensive technical detail regarding the communication infrastructure, task distribution and coordination strategies, integrated sensing, and adaptive robotic routines required for achieving a large-scale interactive and collaborative production process. For additional information concerning the project's design, material and structural development, see the previously presented research: (Felbrich et al. 2017; Solly et al. 2018a, b).

2 Context

Large-scale building construction requires the execution of a diverse set of tasks and processes across many different scales. Many of the early construction automation systems, often categorized in literature as "single-task construction robots", or STCRs, emerging primarily in Japan in the 1970s and 90 s as described in (Bock and Linner 2016a), were developed to automate single limited tasks directly on the construction site. These robots were developed to apply fireproofing and finishes to steel construction and facades, handle and position large components, to move or deliver materials, among other tasks. Early iterations of STCRs were found to be insufficient and uneconomical. The integration of these systems was a significant hurdle, as there was no exchange of information for upstream or downstream processes, (Bock and Linner 2016a). Historically, single-task construction robots that performed independently or functioned in stages could not address the diversity of processes within typical construction scenarios.

In a research context, the development of single-task or customized robotic solutions necessitates significant time and cost investments to achieve the necessary functionality, accuracy, payload, and specifications. Additionally, such robots are inherently inflexible: their tailored functionality hinders the ability to re-use or re-purpose them towards other tasks or material processes.

In contrast, current fabrication research is predominantly based on utilizing generic industrial robots, engineered for speed and repeatability in structured factory conditions. On one hand, industrial robots have several attributes that facilitate their deployment towards flexible and customized production, such as their ability to be programmed to execute computationally generated motions accurately in Cartesian space through their machine control languages. Such a characteristic has enhanced their ability to be integrated into design-to-production work flows, and multiple open-source software tools are available to make their constraints and control logics accessible to researchers and practitioners. Due to their generic nature, industrial robots also imply an advantage in terms of economics and flexibility: through custom tooling and programming, they can easily execute multiple tasks within a single workflow, or be deployed in multiple fabrication processes or construction processes.

However, as appropriated technologies, there are also several limitations preventing industrial robots from fulfilling a role as a universal and multi-functional automation tool within the domain of construction. In particular, typical industrial robots have highly unidirectional kinematics, a high weight and stiffness, and a limited working area compared to the size of buildings. In summary, neither single-task, customized robots or generic industrial robots are fully capable of handling the range of diverse construction processes necessary for large-scale building production. However, a strategic combination of these two types of robots could pave the way towards establishing flexible and robust networked fabrication systems that can work at the scale of building construction.

2.1 On-site precedents for large-scale continuous construction

Several precedent research efforts have focused on overcoming the inherent scalar restrictions of industrial robots. Research at the ETH Zürich has addressed these limitations by developing a mobile robotic system for onsite construction. Dim-Rob was the first iteration of this mobile set-up, and was deployed in various additive fabrication processes. Recent research has extended this approach to other building processes and systems (Helm et al. 2014). The in-situ fabricator was developed to install a bespoke metal mesh serving as the in-place formwork for the subsequent in-place concrete casting process in the Mesh Mould. These precedents have illustrated the main challenges of mobile construction platforms, including achieving the necessary accuracy for the building industry, and the relative difficulty of developing a platform that can freely navigate on uneven terrain or around obstacles, which is especially important in on-site contexts. Critical technical features of the in-situ fabricator system included geometric closed-loop control, enabling a target geometry to be closely followed despite local inconsistencies (Dörfler et al. 2019). Localization of the robot in this system was enabled through a visual fiducial system consisting of AprilTags, while closed-loop control decoupled the movement of the base from the kinematics of the robot, enabling autonomous localization and re-positioning (Sandy et al. 2016; Lussi et al. 2018).

Researchers in the Mediated Matter Group at the Massachusetts Institute for Technology have similarly sought to extend the limited kinematic reach of six-axis robot arms by combining a six-axis industrial robot arm with standard construction equipment. Their Digital Construction Platform (DCP) was constructed by mounting an industrial manipulator on a compound hydraulic arm, (Keating et al. 2014). This project extended scale by functionally extending both the degrees of freedom of the system and its inherent reach.

2.2 Scaled production machinery for large-scale continuous construction

In contrast, a rather obvious strategy in overcoming the limited work range of construction machinery is to increase the size of machinery or robots to fit the task. The first iteration of this concept was developed in the 1990’s, based on the principal of creating a large-scale automated factory for implementation directly on a construction site. One pre-eminent example was the Automated Building Construction System or ABCS by the Obayashi Corporation, which was successfully deployed towards constructing a 28-storey building in 2000, (Bock and Linner 2016b). This concept presents huge obstacles in terms of costs and logistics, and provided little ability for customization on the global scale.

More recently, large gantry systems have emerged in many contexts. Large scale 3D printing has been an ongoing research topic, with a variety of different printer setups and materials being used. Notable approaches include the use of a large-scale indoor concrete printer (Khoshnevis et al. 2006), multiple on-site systems with a nozzle on a gantry (Sakin and Kiroglu 2017), or even a multi-story high cable robot for low-cost mud printing (WASP) (Nematollahi et al. 2017). Another notable example is the brick laying robot, Hadrian, which is a custom long-arm robot developed for deployment on the construction site (FBR 2020).

These experiments often suffer from a trade-off between construction speed and the granularity in the construction detail. Moreover, such solutions are extremely expensive, provide significant logistical challenges in terms of transportation, and cannot be further scaled up.

2.3 Multi-robot systems for large-scale continuous construction

The above research has mainly addressed extending the potential scale and complexity of parts through solutions based on robotic mobility, scaled setups, or integration with other equipment. The goal of the presented research, however, is to establish a physically distributed process enabling teams of robots to communicate and physically collaborate towards the fabrication of large-scale structures. Thus, a final category of relevant research has overcome the scalar limitations of single robots through multi-robot systems.

Several different research groups have already investigated the potentials of using teams of unmanned aerial vehicles (UAVs) for applications in construction and assembly, though as of now the payload capacity of the drones have been a major limitation. Lindsey et al. demonstrated an automated assembly process of a cubic truss system through autonomous teams of quadcopters (Lindsey et al. 2011). Within the context of architecture and fabrication, Flight Assembled Architecture, developed at the ETH Zürich, was among the first to explore the scalar design potentials of unconstrained UAVs (Willmann et al. 2012). In this project, drones were used to stack lightweight polystyrene bricks to build a bespoke undulating tower. The same research groups later used a similar drone model to weave a rope bridge that spanned 7.4 m and could support the weight of humans (Augugliaro et al. 2013; Mirjan et al. 2016). A more recent project investigated utilizing UAVs to spray mud on a substructure towards the realization of earthen mud shells (Bravo et al. 2018; Chaltiel et al. 2018). In this case, the UAV primarily acted as an extended arm of a human operator, while the degree of automation was relatively small.

While many of these previous projects demonstrated advanced UAV control and the architectural potentials therein, in most cases the low payload severely limited the developed material system. In addition, some aspects of the technological implementation, including external motion tracking systems, would limit future applications, particularly in variable on-site conditions. These novel UAV projects exclusively make use of one or multiple similar UAVs. To our knowledge the presented project is the first research incorporating the collaboration of UAVs and with other means of robotic fabrication.

Several investigations have explored fabrication systems composed of collaborative teams of robots with similar specifications. In the ICD/ITKE Research Pavilion 2013–2014, two 6-axis robotic arms were synchronized during production of a modular fiber composite pavilion with double-layered hyperbolic composite parts, (Prado et al. 2014). The synchronization of the 12-axis robotic setup in this case was platform specific, utilizing Kuka’s propriety Roboteam software add-on. This system, though adaptable to various component geometries, was perhaps still limited in terms of scale and kinematics; the increase in scalar design freedom was only achieved by overlapping two limited workspaces. Workcells composed of multiple self-similar robots have also been deployed in many other processes. At the ETH Zürich, a workcell composed of four ceiling-mounted industrial robots has been deployed for many collaborative processes, including coordinated additive assembly for non-standard timber framing systems, spatial metal welding, and brick stacking (Thoma et al. 2018; Gandia et al. 2018; Piskorec et al. 2018). In most precedent works, the similarity of the robots simplified the complexity of the communication network, and gantries or other permanently installed technological infrastructure facilitated collaboration.

Recent investigations have recognized the value in heterogeneous robotic systems, where multiple robots with diverse tooling and specifications can correspond to specialized roles within an overall production chain. Researchers at IAAC in Barcelona developed a series of autonomous “mini-builders”, each with their own unique task and application, that could collaborate on the construction of a large-scale 3D printed structure (Minibuilders 2015; Bock and Linner 2016a). These robots each functioned in stages, thus their physical interaction was minimal or non-existent. Similarly, Maria Yablonina from the ICD in Stuttgart has developed a heterogeneous robot team for a long-span fiber filament winding processes (Yablonina and Menges 2018); (Yablonina and Menges 2019). These examples demonstrate the benefits of system diversity, and also illustrate that the production of large-scale structures might be possible through a distributed system of robots that are much smaller than the architecture itself. Similar to precedent UAV examples, in these cases the inaccuracy and lower payload of custom developed robots limited the design and structural possibilities of the resulting systems. An additional challenge these systems encounter is that coordination between dissimilar robots becomes more difficult, as the robots might have different control, communication, or programming logics.

2.4 Motivation

The presented research sought to address the limitations of precedent work and expand the possible solutions for continuous large-scale fabrication through multi-robot collaborative processes based on hardware exchange. In particular, an aim of the developed system was to establish a scalable and platform-agnostic communication infrastructure for connecting the actions of a diverse and heterogeneous team of robots, machines, and auxiliary sensors. Task diversity is addressed within the developed workflow through digitally mediated strategic task trading between the mobile custom-developed UAV and generic high-payload industrial robots.

The specialized UAV, developed for the single purpose of transporting a fiber from one station to the other under a dynamic amount of tension, expands the working space of the industrial robotic arms through physical interaction and hardware exchange within a collaborative process. The developed system also addresses integration between robotics middleware (ROS) and a CAD design environment: enabling coordinated construction tasks to be directly derived from a customizable fiber syntax, with digital mechanisms and a custom interface in place for the flexible and adaptive execution of these construction tasks. Additional sensors are integrated into the process and communication framework in order to enable UAV localization and handle tolerances emerging from process inconsistencies.

3 Materials and methods

Though this project was produced in a structured environment, the experimental setup was structured to have lessons that could be extrapolated and reused in other scenarios. In particular, the goal was to use only the UAV for fiber transport, and minimize any dependency on large-scale permanent infrastructure, negating the use of a gantry-system or an integrated linear axis. The developed system was also meant to address flexibility and scalability, where the location and the number of robots could potentially increase, and the physical range of the localization system could also increase.

3.1 Construction setup

The general construction strategy was to extend the scale of robotic coreless filament winding for fiber composite production (Fig. 2). Coreless filament winding is a fabrication process where a continuous impregnated fiber filament is wound around a minimal or temporary scaffold without the need for an interior mold (Prado et al. 2014). The developed automated process primarily utilized pre-impregnated spools of carbon and glass fiber with an epoxy resin matrix. The pre-impregnation process, whereby dry spools pre-wound with glass or carbon fiber were infused with an epoxy resin, was developed externally by project partners.

Fig. 2
figure 2

The experimental setup consisted of two stationary industrial robots, a custom UAV, drone landing platforms, and a tension control mechanism with integrated sensing. AprilTags were placed in the ceiling to facilitate UAV localization

Full size image

A resin bath infusion system was also used during some portions of production to minimize costs and also reduce fabrication time. In this device, the fiber is pulled through a system of guide-rollers and temporarily submerged in a bath of resin. Due to the resistance of pulling the fibers through a resin bath, such systems automatically exert a high amount of tension on the fiber line. For additional information about resin bath systems, we redirect the reader to (Shibley 1982) (Fig. 3).

Fig. 3
figure 3

A custom quadcopter was developed for the specific purpose of fiber transportation. The quadcopter included a Pixhawk flight controller, an Odroid XU4 single board computer, a 90-fps monochrome global-shutter camera (IDS Imaging UI-1221), and other on-board electronics

Full size image

In the developed process, two high pay-load industrial six-axis robotic arms take turns winding the epoxy pre-impregnated carbon or glass fiber roving around fiber anchors that are mounted to a rigid metal or carbon fiber frame. During construction, an iterative exchange of a pre-impregnated fiber filament enables the robots to wind a shell-like fiber structure connecting both frames, which then cures to become a self-supporting cantilever structure. Throughout the process, the custom developed UAV is used to transport the fiber between both robots, thus extending the area of operation and the size of the buildable structure. Two landing platforms for the UAV facilitated the fiber exchange process between drone and robot. Furthermore, the fiber was smoothly redirected through a fiber end-effector consisting of a semi-flexible tube connected via caster wheel to a metal plate with mounted connection details.

Due to the low payload of the UAV and the demand for high fiber tension in robotic fiber winding, the active tension in the fiber filament had to be controlled throughout the construction process. Thus the fiber spools were integrated within a tension mechanism for active braking, extrusion, and slack removal. This tension mechanism needed to achieve multiple functions: removing slack from the system, and varying the amount of tension on the fiber line depending on whether the robot or the drone was active. Slack in the system would be caused whenever the robot utilized indirect paths, moving away from the fiber source and then back towards the fiber source, a somewhat unavoidable consequence of the developed experimental setup and construction strategy. The mechanics of the tension mechanism is explained in more detail in Sect. 3.1.3.

To minimize fabrication time, fiber had to be transported manually in some stages of production. In these cases, the resin-bath infusion system was sometimes used. Thus, the three modes of operation were:

  • Full automation mode, where robots, UAV and tension control were automated to wind pre-impregnated spools;

  • Semi-automated mode without the UAV, where the end-effector was passed manually between robots utilizing pre-impregnated spools, and digital tension control could maintain a high tension in the system and also remove slack;

  • Semi-automated mode where the dry roving was led through a resin bath and manually passed to the robots without tension control. In this case, there was no possibility to remove slack from the system.Footnote 1

3.1.1 Robotic hardware

Two Kuka KR 210 R3100 Ultra were used during fabrication. They were each equipped with a Festo pneumatic parallel gripper to hold and release the winding effector. Each gripper-mount further contained an infrared camera to localize IR LEDs on the UAV and compensate for landing imprecision during the process of end-effector exchange. Robotic control and programming will be further discussed in Sect. 3.2.2.

3.1.2 Custom UAV development

The quadcopter was the result of four successive prototypes and gradually tailored towards stable flight, low self-weight, a payload of at least 2 kg, and scalable self-localization. Special focus during its development was put on a robust automated flight behavior with little turbulences under varying angular loads imposed by an active tensile thread. Its hardware consisted of off-the-shelf CFRP components (arms, body plates), custom made CFRP parts (protective propeller cage) and various electronic components.

At its core it used a (Pixhawk Meier et al. 2012) Autopilot microcontroller running PX4 flight control software for low level control of the brushless propeller motors and stabilization. This setup allowed for manual control of the UAV via a remote control or automated flight towards dynamically defined target points passed to the controller through a MAVLink interface. A mechanical toggle on the remote enabled immediate switching between automated and manual mode. In auto-mode the autopilot received instructions in the form of velocity vectors from an onboard single board computer—an OdroidXU4—running ROS and MavROS. This main computation unit was further used for communication with ground control and sensor processing. For meaningful velocity control, a reliable self-localization through onboard sensors was crucial. A 90-fps monochrome global-shutter camera (IDS Imaging UI-1221) was used along with the visual fiducial system AprilTags (Olson 2011) and the accompanying ROS package. Thus the estimated relative poses of unique visual markers, distributed throughout the fabrication space and surveyed using a Leica multi-station, could be found. Through the markers’ fixed, known position in global world space, the drone could back-solve for its own pose. However, due to the high computational load a frequency of only about four Hertz could be achieved with this comparatively accurate pose estimation technique. Furthermore, in some instances no AprilTags could be located in the camera’s field of view. To fill these potentially long gaps between measurements, Pixhawk’s internal high-frequency pose estimation in conjunction with another onboard optical flow camera sensor was used, (Honegger et al. 2013). Due to inaccuracies of the onboard compass caused by external magnetic interference this localization method alone was susceptible to degrading accuracy and resulted in UAV drift. Thus it was corrected whenever fresh measurements from the AprilTag system became available (Figs. 4, 5).

Fig. 4
figure 4

The UAV was the result of four successive prototypes and gradually tailored towards stable flight, low self-weight, a payload of at least 2 kg, and scalable self-localization

Full size image
Fig. 5
figure 5

Localization system on the UAV. AprilTags are used as the dominant localization strategy, while an optical flow camera was used to track the relative transformation in between AprilTag localizations

Full size image

The drone was further equipped with an onboard electrostatic magnet to pick up and release the winding effector. It also had two downward facing always-on infrared LEDs with which its position could be identified with the IR camera on the robot’s gripper, with a precision of approximately 5 mm. For task planning and visualization a continuous connection between the onboard ROS system and the CAD software running on a ground control laptop was made through ROS-Bridge and a custom-developed WebSocket communication plugin for Grasshopper. Thus the precise drone path could be recorded and target points could be set from within the Rhino/Grasshopper interface. As the drone approached a specified target point it was replaced by the next one along the flight path and the UAV’s velocity vector was updated towards the new target. Although arbitrary flight paths could be achieved with this device, a typical path would lead vertically upward from the landing platform on which the drone rested and then in a straight line or flat arc towards a point right in front of the opposite landing platform. For the last phase of the flight the landing platform's curved V-shaped rails acted as a physical guiding system for the drone to slide into its new resting position.

As opposed to exterior localization systems like motion-capturing, this physically scalable tag-based on-board localization system is potentially applicable to much larger fabrication scenarios both indoors and outdoors. Although the main purpose of the UAV in this particular setup was to alternate between two fixed landing positions, its operational area could easily be extended on-site by adding additional tags to the environment. The deployment of this localization system would require modification for the case of outdoors, as the AprilTags would have to be repositioned on the ground, similar to projects presented in the context.

3.1.3 Sensor integrated tension control mechanism

Within this fabrication process, special care had to be given to develop a system that would enable the amount of tension in the fiber to be manipulated in response to the current status of production. The UAV could handle only a very low amount of tension, (2 to 3 kg online), and thus required mechanical assistance to unspool the pre-impregnated fibers. The tension mode when the drone was flying was known as “low tension mode.” In contrast, the robotic fiber winding process necessitated a “high tension mode,” so that the fibers would be pulled tightly around previously wound layers. Because many of the robotic motions were indirect, moving away and then towards the fiber source, the spools would be unspooled, causing slack in the system and a lower tension. Thus the tension system needed to have the capability of unwinding and rewinding while maintaining a high tension, while causing minimal fiber damage. While both mechanical and electrical solutions exist for each of these functions acting in isolation, the challenge was to integrate all necessary functions into a single functioning mechatronic system.

To achieve adaptive tension control, a custom tension mechanism with integrated sensing was developed. It featured multiple components: a spool station consisting of a set of spools interlinked through a friction-based gear belt, controlled by a DC motor (Figs. 6, 7). A hanging weight dancing bar mechanism was developed to maintain a weight on the fiber when the tension mechanism was in high tension mode. A drum in between the spools and the tension mechanism served as a braking and extruding mechanism, preventing the fibers from being unspooled while the dancing bar was active. When the UAV was active, the drum brake would actively extrude, helping to pull the fibers through the system to minimize the tension force on the line while the UAV carried the fiber end-effector. In this case the dancer would be lowered, and the UAV just had to pull already extruded fibers through a system of rollers.

Fig. 6
figure 6

Sensor integrated tension mechanism. A hanging weight dancer applies weight on the fiber line. Two distance sensors detect when the weight has fallen. A motorized drum extruder can rotate either forward or backwards, either raising the dancer weight, or actively extruding fiber for the case of drone flight. The individual pre-impregnated spools are also rotated by a motorized belt to enable slack removal

Full size image
Fig. 7
figure 7

The tension mechanism is high tension mode: the dancer provides high consistent tension on the fiber line, and the rotation of the drum can lift the dancer further if necessary

Full size image

The effective length of the dancing bar tension mechanism was extended through integrated sensing: two distance sensors measured the relative height of the dancer mechanism. If the dancer was low, some of the extra length in the system could be removed by the drum brake extruding backwards, essentially lifting the dancer higher to avoid letting it hit the ground.

Though the tension mechanism could change tension mode autonomously in response to an instruction delivered through ROS, some of the functionality was exposed through custom buttons in a user interface, developed utilizing Windows forms.

3.1.4 Physical collaboration through hardware exchange

An interactive and collaborative process was achieved through an iterative physical exchange of the fiber effector between the robot and the UAV (Figs. 8, 9). After the UAV had landed on the landing platform, a process was enacted where an IR sensor measured the relative transformation between the robotic end-effector and the UAV. The robot then executed a sub-routine to move the correct transform in relative coordinates. The pneumatic gripper on the robot was then activated, and the electro-magnets on the UAV were released. After the exchange through magnet deactivation was confirmed, the tension mechanism switched to high tension mode by lifting a dancer bar with the drum brake. The robot retracted, and moved to a neutral position before traveling around the frame to the next anchor point.

Fig. 8
figure 8

Overview of the fabrication sequence. The UAV flies from one platform to the other. The industrial robot moves to the platform and uses the IR sensor to measure the relative transformation between its position and the UAV. The effector is exchanged through pneumatic actuation and electronic release of a magnet by the drone. The robot travels around the frame and winds the target anchor and then returns the fiber effector to the UAV

Full size image
Fig. 9
figure 9

Physical exchange process. Infared (IR) LEDs act in combination with a IR camera to locate the precise position of the end effector with an accuracy of approximately 5 mm. Beveled details enable the effector to be exchanged through electromagnetic and pneumatic actuation despite tolerances

Full size image

To wind the next target fiber anchor, the robot travels around the frame, keeping the fiber high above the composite before winding it on around an anchor point. It then returns the winding effector back to the drone.

3.2 Communication and digital coordination

Rather than output individual instructions for each individual machine and robot, the project necessitated coordinated and sequence-dependent instructions for all of the robots and machines in the scene that would allow one machine to be situationally aware of whether the previous machine or robot had completed its task, with inbuilt flexibility for changing and adapting instructions at run-time. This communication network needed to consider the stability of communication and flexibility in being able to handle a variety of protocols.

To meet these needs, a decentralized ROS communication system based on a publish and subscribe architecture was developed (Fig. 10). The main instance of a ROSCore Server established a communication infrastructure, where unique topics would be published from a web client interface to a second set of processes (Fig. 11). These ROS nodes included “KukaA”, “KukaB”, “Gripper”, “Drone”, “Drone Magnet”, and “Tension Control Mechanism”. This system was connected over WebSocket with an instance of the CAD modeling environment Rhino with Grasshopper running at the same time, controlling the micro-controllers for the IR sensor. The CAD environment also stored critical parameters that varied over the course of construction, including the current position of the fiber anchors.Footnote 2

Fig. 10
figure 10

Overview of the communication infrastructure

Full size image
Fig. 11
figure 11

Overview of the communication between the web-browser user interface and the ROS nodes utilizing ROSBridge for WebSocket communication

Full size image

A web (UI) application was developed to allow an operator to iteratively move through a list of fabrication instructions, called a “task list”, in either an automated, semi-automated, or manual mode.

A computational library enabled any desired set of fiber paths to be compiled into this XML task list, with templates providing flexibility if the level of automation needed to be manipulated.

3.2.1 Task list

A custom compilation tool was developed to compile a target syntax of fibers in the design environment into the coordinated XML list of high level instructions for each robot and machine (Fig. 12). To execute the code, this XML file would be loaded into the web client interface, which would then iteratively dispatch instructions utilizing the ROSBridge package for WebSocket communication. The web client subscribed to messages back, enabling the process to be situationally aware of when a process had completed. In semi-automated mode, the web client would wait for confirmation from an operator before dispatching the next task. In full automated mode, the next task would automatically be dispatched.

Fig. 12
figure 12

A given fiber syntax in the CAD environment is compiled into a task list of sequential XML instructions. Any reoccurring sequence could be grouped together as a “template”. To recompile the task list, a different set of task templates would need to be utilized, corresponding to a different level of automation or to a different hardware setup. This strategy enabled the automation and coordination system to be both modular and flexible

Full size image

This communication workflow had several different opportunities for modification during the actual fabrication process. Each set of repeatable XML instructions was modularized into a template. A “template” included the typical set of instructions that was executed in the same sequence for a given fabrication task. For example, “Wind frame A” would include instructions for getting the effector from the drone, activating the gripper, retracting from the platform, traveling around the frame, winding the anchor, winding back to the platform, and again re-delivering the effector to the drone.

In contrast to the typical file-to-factory workflows for robotic construction, these tasks could be considered high-level tasks rather than geometrically static tasks.

There were three sets of templates corresponding to three hardware setups: one corresponding to full automation mode including tasks output to the UAV, one corresponding to half automation mode with active tension control, and one corresponding to a resin bath setup with only mechanical tension control. For example, in the case that the UAV drone was not being utilized, and the end-effector would be passed from industrial robot to another through a manual process, the XML instructions could be recompiled without instructions for the UAV. In this case, the robot would move to a neutral position where the gripper could be inserted manually.

The use of modular templates enabled a given fiber syntax to be recompiled quickly depending on the level of automation, and enabled a high degree of flexibility in regards to the hardware set-up or level of automation at any given time.

3.2.2 Adaptive robotic programming

For the developed collaborative workflow, a flexible, sub-routine based approach was implemented for adaptive execution of robotic control code, in which each task that the robot would potentially execute was pre-programmed as a sub-procedure in the robotics control language with a finite set of expected input parameters. The signal processing was configured utilizing the software add-on Robot-Sensor Interface, (RSI), coupled with Kuka RSI Visual for sensor mapping. The developed workflow enabled iterative data packages to be sent to the robot controller and converted into system variables (Figs. 13, 14). The industrial robots executed the exact same machine control code file throughout the fabrication process.

Fig. 13
figure 13

Flexible sub-routine approach for adaptive and sequence dependent execution of robotic control code. A structured XML data package is sent every 12 ms to the robot controller. This package would indicate which sub-procedure to run. Additional data is remapped onto system variables serving as sub-routine input parameters, including inputs such as the robot orientation, the target frame, and the winding behavior

Full size image
Fig. 14
figure 14

High-level flow chart of the flexible sub-routine based approach within the robot control code. The inputs being sent through signal processing are remapped onto process parameters

Full size image

The robot control code included the following sub-routines: “return to platform”, “exist platform”, “travel” (forward or backward), and “wind anchor.” During fabrication, the robot iteratively received packages from the robot server at a frequency of 12 ms, but would allow the new task to run and the newly received signals to affect its stored parameters only when the received task identifier was different than the previously received task identifier. This prevented a task from being sent twice.

Upon receipt of an XML data package different than the previous package, the first identifier signaled which sub-procedure to execute. Each sub-procedure uses other additional packets within the XML data package as process inputs, including process speeds, anchor planes and hooking orientation expressed as a plane, and robot orientation expressed as a separate plane, allowing the code to be executed flexibly and adaptively.

At the conclusion of each sub-routine, a robot system variable indicating task completion was set to 100, and the robot would then return to the main control loop to wait for the next instruction. When an error occurred, the robot control program could be merely restarted, which effectively would empty the buffer of previously received signal values, and the next sub-routine for returning to the previous step could be executed.

A flexible sub-routine based approach had several advantages: the algorithms for motion control on the robot could be utilized and did not need to be handled through external computational processes. This approach was also highly flexible, meaning that the entire solution set of possible fiber paths could be compiled at run time, sent directly from the design environment, or dispatched through the user interface. In addition, parameters which varied throughout production, including the relative displacement of anchors, could be easily updated in response to tolerances, without the need of regenerating control code.

The main disadvantage of this technique was the discontinuity between sub-routines, which would cause the robot to partially decelerate before beginning the next motion command, resulting in somewhat fragmented motion that could have been smoother if the robotic control code was compiled in a continuous set of motions.

3.3 Results

The presented research successfully demonstrated a collaborative fabrication process between short-range high-precision robots and a long-range low-precision UAV. During development, the described automation system was brought to a fully functioning state.

However, the developed automated workflow was significantly less efficient than the semi-automated workflows. Due to external time constraints, about 60 percent of the fabrication process was executed in semi-automation mode with a resin bath infusion system, while just under 40 percent was executed in semi-automation mode with manual fiber transportation of pre-impregnated fibers. The fully automated workflow with UAV transport was tested and executed only during dedicated automation days.

This process and experimental setup successfully enabled the production of a large scale 12-m long composite cantilever. The calculated weight of the cantilever demonstrator is approximately 1000 kg and covers an area of 26.50 m2 with a width of 2.60 m (Fig. 15).

Fig. 15
figure 15

Final 12 m long composite cantilever

Full size image

The developed process demonstrated that it is possible to wind long span fibers through a fully automated approach based on digitally mediated hardware exchange. Furthermore, there was no evident detriment to the fiber placement or fiber quality in the fully automated workflow.Footnote 3 Thus the developed workflow extended the design space and surpassed former scalar limitations imposed by stationary setups. It should be acknowledged however that the full solution space of necessary fiber paths, in particular wrapping fibers around and through the bent frame, could not be achieved with the given automation setup.

3.3.1 Achievements

One of the most significant accomplishments was the overall communication strategy. This distributed communication system was established towards connecting the actions of a diverse set of industrial robots, customized machines and devices. The described automation system consisted of five different sensor systems in the categories optical, sonar and haptic. It successfully integrated five different actuator types and two industrial robots. These devices were controlled through twelve individual microcontrollers of varying complexity. In contrast to precedent projects in sensor integrated and multi-robot fabrication, a variety of protocols and communication types were handled, and the communication network was not specific to the control logics or proprietary language of any singular machine (Fig. 16).

Fig. 16
figure 16

All tasks are coordinated, monitored, and dispatched through the ROS communication system

Full size image

The process was informed by and tightly integrated with a CAD model for task planning and production monitoring. The actions of each robot and device were successfully output with sequence dependency, based on compiling a desired syntax in the CAD design environment into a task list. Significantly, instead of outputting geometric instructions from the CAD environment, the task list was composed of high level instructions. When loaded into a web-based interface, these sequence dependent instructions could be published to each of the entities and robots in the scene. The development of these tailored interfaces made the process accessible and easy to use for multiple users.

The adaptive robotic programming scheme based on flexible sub-routines was utilized throughout the production process. It proved especially useful when correcting robot paths in reaction to deformations of the winding frame and when reaching for the winding effector attached on the UAV after landing.

The tension control mechanism successfully removed the amount of slack in the system autonomously, and also successfully augmented the UAV through automated extrusion. The main shortcoming was that in order for it to function correctly, the redirected fiber path needed to be completely frictionless. In some cases rollers had to be added to the system to redirect the fiber above the structure, and even the filament effector had edges and other features which the fiber could catch the fibers. In addition to developing a smoother fiber feeding system, an alternate system which could be considered in future iterations may have individual tension control on each filament spool, which would increase the number of motorized actuators but potentially minimize mechanical complexity.

The described UAV sensor and control mechanism enabled a well-controlled automated flight whenever localization tags were in the sensor camera’s field of view. During these flights the UAV did not diverge further than around 10 cm from the predefined flight path. Even when the impregnated fiber pulled it in an angular direction the UAV maintained its position and constantly pulled towards its next target point. Flight episodes of more than approximately five seconds without a visible tag, which could occur due to sudden load changes or too fast take off sequences, occasionally led to undesirable drift and in the worst case even uncontrolled drone behavior. In such cases a human operator could take over control immediately.

3.3.2 Shortcomings

The most significant shortcoming was that the fully automated workflow was less efficient than a manually assisted process. Due to the limited UAV payload only a single filament from one spool could be transported between landing stations at a time. Thus the entire production cycle including effector exchange and robotic fiber winding would typically take approximately 6.5 min per fiber. In contrast, the half-automated process in which the winding effector was manually transported between the robotic stations, but all other aspects were automated, enabled the simultaneous placement of four to six filament strands from juxtaposed spools in only about two minutes, effectively speeding up production by a factor of 15 to 20. The main reason for the slow drone flight was a generally high but also inconsistent amount of friction of the fiber inside the winding effector’s aluminum tube. This friction varied due to resin curing time, resin amount, and the fluctuating angle between fiber direction and effector. Thus to ensure a secure procedure, the flight speed was deliberately kept low. Furthermore, the high energy consumption, especially under load only allowed for continuous flight of around 120 s before the six-cell LiPo battery had to be exchanged.

The fully automated mode further showed inconsistencies in many fabrication steps and required a high number of team members for operating and debugging. Thus the developed workflow can best be understood as a proof-of-concept workflow that would need further development. Overall the workflow reinforces the need to keep humans-in-the-loop through user interfaces and other physical and digital mechanisms, in order to execute physical tasks that resist automation, and cognitive tasks that benefit from human observation.

3.4 Discussion and outlook

In contrast to robots mounted on top of mobile platforms, systems based on the physical exchange of material expand the possible hardware setups and solutions for large scale, continuous material fabrication in a scenario such as a construction site.

Potentially, a digitalized construction site would make use of multiple approaches, keeping some equipment and robots stationary, and strategically utilizing mobile platforms only when necessary. Thus many of the strategies for collaboration and hardware exchange were very relevant towards the goal of establishing networked and digitally mediated collaborative systems, surpassing the inherent scalar restrictions of any single robot. Collaborative multi-robot systems based on hardware exchange and task trading are conceptually infinitely scalable.

Similarly, the high level compiler integrated with the design environment for directly outputting coordinated instructions for the construction process was very successful and used throughout the project. Such a system could be developed further and deployed in future research. A particularly interesting next step would be to consider how to generate a task list when different tasks depended on each other for completion, as in a critical path diagram. Similarly, while the project exemplified geometric flexibility, an interesting outlook would be to develop a coordinated control strategy for only dispatching high level goals to each robotic agent towards achieving a greater level of system autonomy. In such a case, the robots would only receive their goal, and may be capable of autonomously modifying their behavior within their situated local context to achieve that goal.