Towards controlled semi-autonomous deconstruction

Abstract

The automation of deconstruction processes presents unique difficulties due to the harsh working conditions involved. In this research, we aim to address these limitations by advancing the teleoperated machine, BROKK 170, toward a developed semi-autonomous concept. We propose a framework for robot-assisted deconstruction, exploring the communication and sensing systems of the prototype deconstruction robot, along with its capabilities. Additionally, field tests are conducted to evaluate the performance of the proposed approach in real-world scenarios. The potentials and limitations drawn from these initial results are discussed.

1 Introduction

The (de-)construction industry is the most significant waste stream of any other business Directive (2013). There is a clear need to increase resource efficiency and decrease the environmental effect of the (de-)construction industry. This growing interest in deconstruction has captured the attention of industry and researchers alike. However, the process of deconstruction faces significant challenges in managing dust and hazardous pollutants in a cost-effective manner that also prioritizes employee safety. The (de-)construction industry has been associated with a poor track record in employee retention partly due to the sector’s high incidence of catastrophic accidents. In fact, among all sectors, (de-)construction has the third-highest incidence of such accidents, with 10 out of every 100,000 (de-)construction workers affected US Bureau (2020).

Despite societal issues, there is currently limited automation for the deconstruction process Lee et al. (2022b), which is challenging due to multiple variables, including changing weather conditions in outdoor activities. Heavy-duty operations require actuators with robustness, high reliability, and the ability to withstand substantial force. Significant efforts are required to develop a robotic system for these kinds of operations in outdoor environments. Consequently, (de-)construction machines are still directly or remotely controlled by human operators, with the operator standing near the machine Brell-Cokcan and Lee (2022).

Fig. 1

Conventional teleoperation, where the human operator directly stands on the deconstruction scene

Fig. 2

BROKK 170 used in this work (left) and its joint geometric representation (right)

1.1 Related works

Construction machine manufacturers have traditionally focused on improving deficiencies, comfort, and safety to enable machine actuators to perform various tasks under different conditions. However, human operators still predominantly control these machines directly or remotely, with the operator positioned near the machine, see Fig. 1. Control is typically achieved using individual joysticks or levers at the joint level. Operating these machines effectively requires significant training due to their multiple degrees of freedom and complex coordination of multiple joints Lee et al. (2023a). Even experienced operators require months of training, leading to reduced local accuracy and decreased work efficiency Rosenberg (1993). Furthermore, the proximity to the machine exposes workers to risks and hazards associated with operating heavy machinery on a construction site (Fig. 1).

To introduce automation in deconstruction sites, researchers have proposed integrating industrial robots. One such system is the Robotic Facade Disassembly and Refurbishment System (RFDRS) from Lublasser et al. (2017), which employs a robot-assisted deconstruction approach with motion programming to automate the dismantling of multi-layered facade structures. However, traditional industrial robots with electrically actuated manipulators face limitations, such as a low payload/weight ratio and vulnerability to vibration and impact damage Merckaert et al. (2018). Consequently, the application of such systems is restricted to specific types of work due to these inherent limitations of industrial robots.

Fig. 3

Schematic system overview of the core modules of BROKK 170. The green components are executed from a safe control container, whereas the others are executed onboard BROKK 170. The components added to the original design are depicted in blue

Here, the idea of enhancing the level of machine autonomy in existing heavy-duty construction machinery has captured the interest of researchers Lee and Brell-Cokcan (2021, 2023b). One notable study by Lampinen et al. focuses on upgrading an industrial breaker boom with a visual perception system, enabling it to break rocks autonomously Lampinen et al. (2021). However, the system’s effectiveness relies heavily on autonomous components, particularly the vision system, which required 4733 rock photos from a field test for rock detection. They reported a success rate of 34% for rock cracking.

Given the existing challenges and needs in the deconstruction process, there is a growing demand for alternative approaches to enhance machine autonomy. Currently, no robotic platform fully meets the requirements for controlled deconstruction, to the best of our knowledge. While autonomous solutions in similar construction tasks have demonstrated various potential benefits Jud et al. (2019); Giftthaler et al. (2017), we argue that adopting a systematic approach toward a semi-autonomous approach is a necessary step toward further development of the deconstruction process. Considering the dynamic nature of the task, where the hammering target continuously changes over time, a fully autonomous robot should be capable of understanding its surroundings and reasoning the best suitable hammering target accordingly. By integrating human operators into the control process and allowing them to interactively define the target positions for hammering, the cognitive load on robots can be reduced on deconstruction sites. At the same time, the robot can convert the target hammering position into precise control and coordination of multiple joints ensuring both the safety and efficiency of operations. This work builds upon this aspect of the semi-autonomous approach.

1.2 Contributions of this paper

We propose a semi-autonomous deconstruction concept as a progressive approach to ensure safe and controlled deconstruction processes. Unlike conventional teleoperation methods that require the operator to control each joint continuously, our proposed semi-autonomous concept allows the operator to control the deconstruction machine using a high-level goal, i.e., a desired hammering point. By specifying this desired point, the appropriate joint motions are generated according to different states, such as moving the hammer toward the target or activating the hammering action. In this way, this approach eliminates the need for operators to micromanage the deconstruction machine. At the same time, controllability and safety are ensured by pre-visualizing the planned motion and eliminating the operators from the direct proximity of the deconstruction machine and scene.

We present each component of our proposed system, along with the limitations we encountered during the development process and their corresponding solutions. The paper is organized as follows: First, we propose a list of basic hardware adaptations to convert an off-the-shelf teleoperated deconstruction machine into a programmable robot. Second, we provide a systematic overview of the information flow between each component. Next, we introduce the planning and control algorithms employed to achieve the required capabilities for robot-assisted deconstruction. We highlight the capabilities of our proposed system through a demonstrator in Sect. 4 and outline the lessons learned from the initial field test and plans for addressing the identified limitations. In Sect. 5, we draw essential conclusions.

2 System overview

In the following subsections, we describe the overall concept of the semi-autonomous approach, followed by the hardware description and the required modification.

2.1 Concept of operations

The main goal of developing a semi-autonomous deconstruction system is to enhance operator safety by minimizing their direct exposure to hazardous deconstruction scenes and reducing the need for constant manual control of the machine at the joint space level. This objective aligns with the concept of autonomous robots, where tasks are performed without human intervention. However, the challenging nature of construction sites, which are dynamic and complex environments with numerous variables like changing conditions and human presence, presents additional complexities compared to controlled manufacturing facilities. As a result, our proposed solution focuses on allowing the human operator to define high-level goals for the deconstruction robot. By doing so, we aim to alleviate the cognitive burden on the robot while improving safety and overall system robustness. The control concept in detail is further explained in Sect. 3.

Fig. 4

System architecture. Based on the captured visual information of the workspace, the operator defines the desired pose for the hammer, which is converted to the corresponding control inputs by the trajectory controller. The planned motion gets visualized to the operator and is only executed when the operator acknowledges it

2.2 System description

This section provides a description of the hardware configuration of the BROKK 170 construction machinery, along with the necessary modifications for the proposed work. The BROKK 170 (see Fig. 2), like other modern construction machines, utilizes components such as actuators and sensors, which are connected to the machine control unit (MCU) through a Controller Area Network (CAN) bus. When the operator sends commands using the control device, such as joysticks, specific CAN messages are generated and forwarded to the Programmable Logic Controller (PLC) controller, where they are then translated into corresponding Pulse-Width Modulation (PWM) signals with specific voltage levels. The power electronics receive these signals and drive the mechanical components of the machine. Figure 3 illustrates the original hardware architecture (in gray) and the additional hardware added (in blue).

To enable operators to program the deconstruction machine for the semi-autonomous deconstruction process interactively, a communication link is established using an embedded PC, Jetson AGX Xavier. This embedded PC is equipped with a CAN bus controller and acts as a bridge, saving filters and forwarding incoming CAN bus messages to the controller of the BROKK 170. This setup expands the original connection between the control device (i.e., radio receiver) and the controller, as shown in Fig. 3. With this configuration, the machine can be teleoperated using the original joystick signals and also receive signals computed by algorithms deployed on the Host PC in program mode. These computed signals enable optimized and controlled motions. The embedded PC communicates wirelessly with the host PC, exchanging machine states and visual information of the remote workspace. This wireless communication facilitates the interactive programming of the deconstruction machine and enables semi-autonomous operation by exchanging information between the embedded PC and the host PC (Fig. 4).

Fig. 5

Captured visual information of the workspace

To ensure operator safety, this study avoids direct proximity between the operator and the deconstruction machine and scene. Instead, the operator observes the remote workspace using visual sensors, which capture 3D point clouds and 2D images, respectively. The lack of depth perception in 2D visual information presents challenges in accurately perceiving the spatial relationships and distances within the scene, see Fig. 5. To address this issue, we employ the use of 3D visual sensor. By leveraging 3D visual information, operators can infer depth information, reducing the cognitive load they experience and minimizing potential errors in their understanding of the environment.

The captured data is transmitted wirelessly to the operator using WLAN with the 802.11ac standard. Due to data size limitations and the constraints of the wireless network, the frame rate is limited to 2 Hz for 3D information and 25 Hz for 2D information. The overall system architecture, shown in Fig. 4, provides an overview of the components and their interactions in the proposed semi-autonomous deconstruction system.

Fig. 6

a Interactive hammer model; b according to the desired pose of the hammer, the trajectory planner generates the corresponding joint motions; c only after acknowledgment from the operator the planned motion is forwarded to the real machine

3 Control methods

This study introduces a flexible interface enabling the operator to dynamically set or adjust various task parameters during runtime, ensuring successful execution. These parameters may include goals or trajectories for the controller, object locations, or poses. The proposed approach follows a shared control methodology, where both the operator and the robot can simultaneously control different signals rather than relying solely on teleoperation or full autonomy. By employing shared control, the operator can define higher-level goals, thereby reducing their workload for reasoning. This approach is particularly well suited for dynamic work environments such as construction sites, where the work conditions continuously change. This work mainly focuses on empowering the operator to control the robot using high-level goals, such as the desired hammering point. As opposed to manually controlling every joint-level command of the robot, the defined high-level goals are automatically converted into different joint motions based on the state machine aligning with the principles of shared control.

3.1 High-level control

The high-level planner is responsible for converting these task-level robot commands into the corresponding operation modes based on an event-triggered state machine. Here, the current state of the machine, i.e., the pressure value, the current, and the desired task space pose of the hammer, are considered to define the transition between the states.

3.1.1 Defining the hammering target

In this study, we utilized the RVIz program to implement interactive markers. These markers serve as visual controls that enable users to engage with a robot in a fully immersive 3D environment. This environment encompasses real-time feedback on the robot’s status and utilizes various 3D sensory data, such as laser scans and RGB-D point clouds. Users can easily interact with these markers by manipulating them with the mouse. Each marker offers the ability to control up to six degrees of freedom, including both position and orientation. Additionally, developers have the flexibility to associate custom shapes with these markers, allowing for representations like tools or the robot’s end-effector. For instance, Fig. 6 showcases an interactive marker exemplified by a hammer-shaped model. Notably, we specifically focus on the position and orientation of the robot, denoted as ${\textbf {x}}^D=x, y, z, \theta$, while disregarding roll and yaw angles. This decision is driven by the robot’s limited motion within a fixed-base position, eliminating the need for base movement. As depicted in Fig. 6, the operator defines the desired pose for the hammer by actively interacting with its corresponding model.

Fig. 7

A snippet of the step response from 0.6 rad to 0.8 rad and the corresponding PWM signals

Fig. 8

Desired poses 1 and 2 for the tracking test in the task space level and the snapshots from the real machine

Fig. 9

Tracking result in the task space level for the desired poses 1 and 2

Fig. 10

The operator works in a control container monitoring the remote workplace using the 2D and 3D visual information (Left). The state of the machine is also provided to the operator together with the interactive hammer model for defining the desired hammer pose; Experimental Setup (Right)

3.1.2 Planning the hammering

In our study, we utilize a predefined order logic consisting of four states: distance adjustment, tracking, approach, and hammering. It is important to note that the base of the system remains fixed throughout the task.

The BROKK 170 manipulator is a hydraulic serial link manipulator with five revolute joints. Among these joints, the second joint, denoted as $q_2$, is mechanically coupled to $q_3$. The manufacturer has designed $q_2$ to modify the manipulator’s reach and implemented a communication protocol to ensure its independent control from the other joints. This raises the question of when to utilize $q_2$ and when to utilize the other joints, based on a desired goal pose $x^D = [x^D, y^D, z^D, \theta ^D]^T$ defined by the user.

In our approach, we first check the distance between $q_2$ and $x^D$ for the desired goal pose $x^D$. Through experimental analysis, we determined that if this distance falls within the range of 1.7 m, 2.4 m, the goal position is within the reachability of the BROKK 170. In the initial distance adjustment mode, only $q_2$ is actuated until the distance to the goal falls within this range. Once the distance criterion is met, the system transitions to the next mode, tracking.

In the tracking mode, a trajectory is planned that includes a constraint on $q_2$ to maintain its current position and prevent its utilization. This constraint ensures that a trajectory in the joint space $q^D_{i...N}$ is defined, which brings the hammer to the desired $x^D$ without involving $q_2$. The planned trajectory is first visualized to the operator and then tracked by the low-level controller in the tracking phase. Monitoring the states of $q_{1,3,4,5}$ and comparing them with the final trajectory waypoint $q^D_N$ allows for a smooth transition from the tracking phase to the approach phase.

The hammer mechanism of the BROKK 170 is hydraulically powered, with piston movement in the cylinder generating the hammering action. During the hammering operation, the chisel attached to the hammer moves back and forth for approximately 15 cm, indicating its ability to press against a surface. The objective of the approach state is to position the hammer in close proximity to the target object’s surface and press the chisel against it, ensuring effective impact during hammering. The pressure in the main hydraulic pump increases when the hammer encounters resistance from the surface while pressing down. Therefore, the pressure value from the main pump serves as a trigger to transition to the next stage, hammering. Specifically, when the pressure exceeds 100 bar while the hammer is moving in the negative z direction, the movement is halted, and the subsequent hammering state is initiated.

Maintaining an appropriate distance between the chisel and the target object’s surface is crucial for successful hammering. During the hammering process, the surface is demolished, causing the distance between the chisel and the surface to automatically increase. However, if the distance becomes too large, the chisel may not effectively impact the surface. To address this, the joint $q_5$ of the robot manipulator is gradually rotated toward the surface’s direction (assuming the surface is located below the hammer) to maintain a small distance during the hammering state.

In the current setup, the decision on when to start and stop the hammering operation is made intuitively by a human operator who observes the remote workplace through captured visual information. Once the target point is properly hammered, the operator can define a new goal pose by repositioning the interactive hammer model, which subsequently triggers the distance adjustment state to resume the task.

3.1.3 Trajectory planner

In the context of trajectory planning, we further elaborate on the tracking aspect. We aim to plan and accurately track a trajectory to position the hammer at the desired task space goal $x^D$. To achieve this, we first convert the defined desired pose $x^D$ into the corresponding joint angle values $q^D$ using an inverse kinematic solver Lee and Brell-Cokcan (2021). Using the computed $q^D$, the trajectory planner generates waypoints from the current joint configuration. The desired changes between these waypoints are then converted into control inputs, specifically PWM values, through the implementation of a low-level controller.

To generate collision-free trajectories, we employ the motion planning framework MoveIt! Coleman et al. (2014). This framework utilizes the rapidly exploring random tree (RRT) method from the open motion planning library (OMPL) Sucan et al. (December 2012) to sample the current and desired end poses and construct search trees. The generated trajectories are created and smoothly interpolated using a quintic-polynomial spline. To facilitate closed-loop control, we implement a control system based on the ROS (Robot Operating System) control framework. The main control loop of the system operates at a predefined interval of 20 Hz.

Within each control loop iteration, the control system updates sensor information, such as joint states from the robot, and computes commands for each joint to guide the robot toward the desired pose defined by the operator. Before the computed joint commands are forwarded to the low-level controller for conversion into actual motions, the operator has the opportunity to observe the defined hammer goal position and the corresponding manipulator motions. Once the operator is satisfied with the planned motions, they can be executed as actual motions.

3.1.4 Low-level control

The objective of the low-level controller in this research is to determine suitable control inputs, specifically PWM values for the valve system, based on the given actual and desired joint states. While analytical control techniques, such as sliding mode control Jin et al. (2009) and adaptive controllers Ahn et al. (2014), have been proposed for similar hydraulic applications, their implementation can be challenging due to the lack of accurate machine models. Most manufacturers do not provide comprehensive descriptions of their machines and the underlying system dynamics, making it difficult to generate an analytical model.

In this work, a Proportional-Integral-Derivative (PID) controller is employed for the low-level controller. The controller gains are adjusted primarily to prevent joint overshoot and avoid unintended collisions with the environment. The PID controller is chosen due to its simplicity and robustness, as it does not rely on accurate system models. By appropriately tuning the controller gains, the control inputs are optimized to achieve the desired joint movements.

4 Experimental results

The experiments are designed first to evaluate the performance of the low-level controller using pulse tests. Then the trajectory tracking performance in the task space level is evaluated, where a position trajectory is computed by considering the desired hammer pose from the operator. After that, we show the performance of the implemented framework within a deconstruction task.

Fig. 11

Pressure value profile from an individual hammering attempt. First, the joints are actuated in a disjointed manner to adjust the distance ($q_2$) and track the desired hammer pose ($q_{1,3,4,5}$). The manipulator moves down to press the chisel (green). Afterward, the hammering follows (gray)

4.1 Pulse test

The pulse tests are conducted to assess the performance of the low-level controller under various conditions, such as movements toward and against gravity in different directions. In these tests, the pulse time is set to 8 s. The PID gains are manually tuned by gradually increasing their values and comparing the system’s performance. During this manual tuning process, special attention is given to avoid overshoot, as safety is a crucial concern when the machine’s movement exceeds the target and encounters unexpected structures.

The overall tracking performance can be observed in Fig. 7. While the implemented PID controller produces satisfactory results in terms of overshoot and rise time during the pulse tests, the differences in settling and rise time observed in two different rotation directions, i.e., toward and against gravity, indicate the nonlinear behavior of the system. This nonlinearity should be further investigated in future research to better understand its impact and refine the control approach.

4.2 Tracking performance at the task level

The objective of this experiment is to assess the tracking performance of the implemented framework at the task level. To accomplish this, the BROKK 170 starts from a given initial pose $x_s$ = [$p_{xs}, p_{ys}, p_{zs}, \theta _s$]$^T$ = [1.96, $-$1.17, 1.69, 1.04]$^T$ and moves toward the predefined end-effector pose $x_{d1}$ = [2.47, 0.01, 2.12, 0.38]$^T$. Subsequently, the robot proceeds from $x_{d1}$ toward the desired pose $x_{d2}$ = [1.84, 1.16, 1.78, 0.11]$^T$. This trajectory is repeated five times to evaluate consistency.

The outcomes of the experiment are depicted in Figs. 8 and 9. For the first desired TCP pose $x_{d1}$, the proposed system achieves a position error of 2.5 cm and an orientation error of 0.02 rad. The tracking error decreases during the second posture $x_{d2}$, particularly when the joints largely move toward gravity. As demonstrated in the previous pulse test, the joints rotate at different speeds depending on the direction. For $x_{d1}$, the joints rotate from $q_s$ = [2.10, 0.64, 1.13, 0.60, 2.01] to $q_{d1}$ = [1.55, 0.64, 1.06, 1.30, 1.56] along the generated joint space trajectories. Similarly, for the second desired TCP pose $x_{d2}$, the joints rotate from $q_{d1}$ = [1.55, 0.64, 1.06, 1.30, 1.56] to $q_{d2}$ = [1.01, 0.64, 0.60, 0.78, 0.35]. It is noteworthy that joint $q_4$ rotates in the positive direction for $x_{d1}$, while it rotates in the negative direction for $x_{d2}$. The pulse test results reveal that joint $q_4$ exhibits faster rising and settling times when rotating in the negative direction, which accounts for the slightly larger tracking error observed for $x_{d1}$ where $q_4$ rotates positively. Nonetheless, Fig. 9 illustrates that the tracking error remains reasonably small for both poses.

Fig. 12

A snippet of task space level tracking performance during the deconstruction

Fig. 13

Snapshots of the manual teleoperation (top) and the semi-autonomous approach (bottom) for deconstruction

4.3 Field deployment

In order to demonstrate the system’s performance, the BROKK 170’s base is positioned in front of a concrete structure, and the manipulator carries out the deconstruction process using the proposed method. During this process, the operator interacts with the interactive hammer model, utilizing the visual information provided, see Fig. 10 to define the desired hammering target. The control container is situated at a distance of approximately 20 ms from the deconstruction site.

4.4 Experimental results

The proposed system exhibits promising capabilities in achieving accurate deconstruction of structures in a semi-autonomous manner while prioritizing human operator oversight and safety throughout the process. Figure 11 provides a depiction of the pressure profile, offering insights into the transitions between different states. It is worth noting that the human operator retains the ability to modify planned movements or acknowledge actions, leading to idle states between each pre-defined state. For example, in the approach state, the pressure value gradually increases until the hammer tip makes contact with the object’s surface, resulting in a sharp rise in pressure. Once the pressure value surpasses the threshold of 100 bar, the state transitions from approach to hammering.

Figure 12 illustrates that the tracking error remains minimal throughout the deconstruction experiment, indicating accurate tracking performance. As illustrated in Figs. 13 and 14, the proposed semi-autonomous deconstruction method offers a solution to mitigate the dust and risks associated with human proximity to heavy machinery in deconstruction sites while simultaneously increasing safety and accuracy by integrating the human operator into the control loop. Notably, the integration of the human operator’s reasoning capability in defining the deconstruction target along with the pre-visualization of the planned motion for the operator, contributes to enhancing the controllability and safety of the deconstruction process.

4.5 Lessons learned from the field test

In this demonstration, we created a hardware and software framework to build and test the system. The successful completion of this demonstration was essential in evaluating the chosen framework and gaining valuable knowledge to enable robots to assist in deconstruction tasks in challenging environments like construction sites, which can be unstructured, cluttered, and dynamic. The conducted experiments demonstrate that it is possible to enhance the level of machine autonomy and improve controllability during the deconstruction process. However, as expected in any field test, we have identified several areas that need improvement and further investigation.

In this work, a fine-tuned PID controller was utilized to convert the desired joint states into the corresponding PWM values. Although the experiments show promising results, the performance can not be guaranteed if the disparity between nominal and actual system behavior increases during the dynamic deconstruction process. Here, more advanced control approaches need to be investigated.

Fig. 14

Deconstruction results

The semi-autonomous approach presented in this study exhibits a limited depth of deconstruction, as illustrated in Fig. 14. This limitation arises from the dynamic nature of the deconstruction process. With each hammering attempt, the structure’s surface undergoes continuous changes, necessitating dynamic adaptation of the hammer’s orientation and position to maximize its impact. While a human operator positioned directly in front of the workspace can observe and promptly adjust the manipulator configuration during hammering in conventional teleoperation, the proposed method lacks this adaptiveness. Investigating and enhancing the adaptiveness of the approach in future research work is thus planned.

The integration of robotics technology in the construction industry presents promising opportunities for improving on-site operations and the collaboration between human workers and robots, as demonstrated in the conducted experiments. However, a crucial requirement for the successful implementation of such emerging technologies is stable and low-latency communication between the human operator and the robot, ensuring minimal data transmission delays. In our experiments, the remote workplace was captured using visual sensors and streamed to the operator through WLAN. Here, the continuous streaming of captured visual information over WLAN was found to be a limitation due to the resulting point cloud data size, even after resolution and scan area reduction, reaching several tens of megabytes. This highlights the need for a more robust wireless communication solution. While established compression techniques exist Rusu and Cousins (2011), their application often requires computation time, particularly when performed on embedded PCs. This poses a critical challenge, considering that the human operator heavily relies on spatial awareness of the remote workplace. One potential solution to address this challenge is leveraging the 5 G mobile networking standard. However, the utilization of 5 G for wireless communication in construction sites remains largely unexplored. One significant obstacle in existing research gaps is installing a 5 G network on-site. Unlike many industrial production environments, the workspace in construction sites undergoes continuous infrastructure changes. For example, the workspace evolves in building projects as the structure is constructed. Therefore, the efficient deployment of a 5 G network on-site and its effective integration with the control of construction machines are currently under investigation in a separate line of research Lee et al. (2022a).

5 Conclusions

This paper introduces a semi-autonomous approach for deconstruction tasks, wherein a commercially available machine is further enhanced for the proposed work. Equipped with visual sensors, the machine enables the human operator to monitor the workspace remotely. The operator can define the desired hammering point through the interactive placement of a dummy hammer. A trajectory-tracking controller is developed to achieve precise control at both the joint and task levels. Prior to execution, the planned motion is visualized to the operator, allowing for the possibility of pre-planning and further optimization. Experimental results demonstrate the capability of the proposed approach for semi-autonomous deconstruction. However, the initial field test reveals both potential benefits and limitations.

The presented approach offers the potential to enhance worker safety by reducing the need for direct proximity between human operators and the machine or workspace, thus mitigating associated risks and hazards on the construction site. Additionally, verifying and adjusting the planned motion as needed can significantly improve the controllability of the deconstruction process. Traditional teleoperation techniques struggle with precise control of construction machines due to their complex degrees of freedom, requiring extensive training and experience. Visualizing the planned motion and enabling motion optimization for heavy-duty construction machines enhances overall process safety and efficiency, particularly when precision is required in the task.

Future research directions include investigating methods to enhance adaptiveness during hammering attempts, considering the continuously changing surface of the structure throughout the process. Additionally, exploring the potential of leveraging 5 G technology will be pursued. Furthermore, the current work focuses on the hammer as the primary tool, and thus the aspect of material reuse has not been extensively addressed. Future investigations will involve replacing the hammer with a wall saw to explore the controlled dismantling of structures while preserving elements and materials for potential reuse.