Complete area-coverage path planner for surface cleaning in hospital settings using mobile dual-arm robot and GBNN with heuristics

Abstract

Complete area-coverage path planners are essential for robots performing tasks like cleaning, inspection, and surveying. However, they often involve complex calculations, mapping, and determining movement directions, leading to high computational or processing overheads and the risk of deadlocks. This paper proposes an approach for cleaning, i.e., by linear wiping of generic and discontinuous surfaces in hospital settings using inhouse assembled mobile dual-arm (MDA) robotic system. The proposed framework introduces key features: (a) a less resource-intensive approach for MDA positioning and cleaning surface mapping, (b) Modified Glasius Bioinspired Neural Network through use of heuristics (GBNN+H) to optimize surface linear wiping while obstacle avoidance, and traversal across discontinuous surfaces. The advantages of the proposed algorithm are highlighted in simulation with GBNN+H significantly reduces the number of steps and flight time required for complete coverage compared to existing algorithms. The proposed framework is also experimentally demonstrated in a hospital setting, paving the way for improved automation in cleaning and disinfection tasks. Overall, this work presents a generic and versatile, applicable to various surface orientations and complexities.

Introduction

Complete area-coverage path planners (CAPP) are important for robots that are used for cleaning or inspection tasks, as they help to efficiently and effectively cover a given space. These path planners use algorithms to calculate the most efficient path for the robot to follow. The robot architecture used for cleaning applications depends upon the nature of the surface to be accessed. For example, autonomous solutions like mobile robots for indoor cleaning are served by iRoomba [1], outdoor settings like pavement by robotic architecture proposed in [2, 3], and vertical surfaces like glass façade [4]. Hence, a suitable selection of CAPP complementing the robotic system is important for cleaning a given surface area so as to minimise resources used, i.e., the number of sensors, computation or processing overheads, steps for area coverage, and ease of deployment are essential factors.

Challenges in surface cleaning in hospital settings

Hospitals in past years faced several challenges in cleaning and disinfecting surfaces in hospital settings, including beds, tables, floors and walls. The method of cleaning performed by trained hospital staff is a zig–zag, or s-shaped manner [5]. Automated cleaning is dominated by cleaning robots and are designed specifically for floor cleaning [6, 7], wall cleaning [8], and multi-purpose cleaning [9]. Floor cleaning is approaching maturity in its technological readiness, while wall cleaning is only widely established as facade cleaners [8, 10]. By automating some of the cleaning tasks, hospital staff can focus on other important tasks, such as patient care. This can help to reduce the risk of staff becoming infected with germs from patients or contaminated surfaces. Human-sized anthropomorphic robotic systems are suitable to mimic the human action of accessing surfaces. Humanoid robots [11,12,13] might be among the best tools for meeting society’s requirements, but the computation cost and complexity of the mechanical analysis increase as the system’s degrees of freedom (DoF) increase. Dual-arm robots, also known as bimanual arm robots, can perform surface cleaning tasks. When mounted on a mobile base [9, 14], it has a high level of accessibility to navigate desirable cleaning areas. Dual-arm robots were favorably discussed to be superior to single-arm robots [15, 16]. However, path planning for cleaning remains challenging for dual-arm robots due to complex environments with obstacles, gaps, and discontinuous surfaces. The objective of this work is to tackle the challenges of complex environment with discontinuous surfaces, and to have an efficient complete area-coverage path planner for surface cleaning using Mobile Dual-arm Robot. Next, literature review of existing complete coverage algorithms for surface are detailed.

Literature review

Various coverage planners have been developed for diverse applications. For instance, a multi-layer-based trajectory planning approach for inspection has been proposed in [17], and an intelligent speed-suggestion planner is discussed in [18]. In the field of agriculture, a coverage path planner is designed to adapt to dynamically changing geometric shapes of enclosures [19], enabling sustainable farming and soil preservation through the use of smaller autonomous vehicles [20]. Notably, complete coverage path planning (CCPP) is rapidly advancing in cleaning applications with reconfigurable robots, as highlighted in [21]. Voronoi diagrams [22] divide the area into cells, with the robot visiting each cell’s center, but generating the diagram is computationally expensive. These algorithms often employ techniques like potential fields or artificial potentials to guide the robot and ensure complete coverage. Common CCPP approaches include random walk [23, 24], where the robot moves randomly, but may be slow and inefficient due to revisiting areas. Spiral planners start from the center and move outward in a spiral and is referred as Backtracking Spiral Algorithm (BSA) [25, 26]. Moreover, the Predator–prey methods create zig-zag patterns, effectively handling coverage area changes reported as Decentralized Predator–Prey-based approach to adaptive Coverage Path Planning Dec-(PPCPP) in [27].

The Glasius bio-inspired neural network (GBNN) is increasingly recognized among complete coverage path planners (CCPP), especially in autonomous robots focusing on area coverage tasks. An instance of its application is seen in a study [28], where GBNN is employed to plan the path for an autonomous underwater vehicle (AUV) covering a grid-mapped 2D underwater environment, showcasing its capability for complete coverage and deadlock avoidance. In the domain of cleaning, GBNN is utilized to optimize energy costs for ship hull cleaning in [29] and pavement sweeping robot in [30]. Despite its versatility, GBNN exhibits suboptimal performance in complex environments, leading to modifications [31]. However, no prior work, to the best of the author’s knowledge, has tailored GBNN for a cleaning and disinfecting task using a dual-arm manipulator mounted on a mobile base. This work fills this gap by demonstrating the adaptability of GBNN with heuristics (GBNN+H), ensuring deadlock prevention and effective traversal across discontinuous surfaces. In this paper we have used these state of the art CCPP, namely Backtracking Spiral Algorithm and Predator-Prey-based approach to adaptive Coverage Path Planning to compare it with our porposed GBNN+H. Complexity of coverage algorithms with mobile dual-arm is described next.

Complexity in current approaches

Despite significant advancements in coverage algorithms, manipulators, and mobile robots, the prevalence of mobile dual-arm cleaning robots remains limited in modern cleaning technologies [11]. The main impediments to widespread adoption are the computational expense associated with coverage planners [32] and the computational demands in controlling the total degrees of freedom (DOF) of a dual-arm robot [33]. To mitigate computational complexity, manipulation planning dimensionalities can be streamlined to focus on surface area-coverage path planning using classical control methodologies. Cleaning control methods fall into two primary classes [11]: classical and learning-based control. For instance, control approaches for an industrial manipulator arm leverage its identified model [34, 35]. While learning-based controls like iterative learning control [36] offer versatility and optimality, their implementation is often expensive due to the need for well-defined training sets. On the other hand, classical control is mature and easily deployable, with cascade controllers exhibiting high performance in complex robot systems such as parallel robot platforms [37, 38]. Consequently, classical control methods are preferred in this work, typically involving movement along a targeted cleaning surface. The paper’s distinct contributions arising from the introduction are listed next.

Contributions

Based on the above discussion and gaps highlighted, the objectives and contribution of this work are listed below:

• Mobile Dual-Arm Cleaning Framework: Design of a novel framework for cleaning, i.e., linear wiping of hospital surfaces using a mobile dual-arm robot, addressing complex scenarios like obstacles, gaps, and discontinuous surfaces.

• Propose Algorithm: propose Glasius Bioinspired Neural Network with Heuristics (GBNN+H) algorithm, specifically designed to address scenarios listed in above points.

• Simulation Validation: Through simulations, demonstrate GBNN+H superior performance, achieving significant reductions in cleaning steps compared to existing algorithms.

• Real-World Experiments: Validation of GBNN+H in real-world cleaning tasks on hospital beds and side tables, showcasing its effectiveness in achieving complete coverage for efficient disinfection.

The contribution on path planner can be likened to the evolution of Dijkstra’s algorithm into the well-known Astar algorithm. Similar to how Astar enhanced an uninformed search algorithm into an informed one, our proposed heuristic function enhances the capabilities of GBNN for more efficient surface planning. Moreover, the benefit lies in the automation’s adaptability for cleaning unconventional surfaces, unlike walls or floors. Our study introduces the use of mobile robotic arms in hospital settings, extending the potential application of mobile biarms to broader urban contexts. The structure of the paper is as follows. Sections “System description and framework” and “Complete coverage path planning” describe a typical environmental settings with the robot description and the framework is proposed for surface cleaning using CCPP respectively. Sections “Simulation studies” and “Experiments” show the simulation and experiment results and discussion of the proposed algorithm performance relating to the algorithm behaviour. Finally, Sect. “Conclusion” concludes the paper with future work recommendations. The simulation video and experimental studies is also shared in the supporting video link¹.

System description and framework

In this section, we highlight the environmental settings inside the hospital where the area coverage task is desired using the mobile dual-arm (MDA) robotic systems. These settings require a special area coverage algorithm which is the main objective of the manuscript, detailed in Sect. “Complete coverage path planning”.

Environmental settings

Fig. 1

General setting and the surface area to be covered or cleaned, namely, Scenario (S1) with the vertical surface, Scenario (S2) with the horizontal surface and gaps, and Scenario (S3) with the objects kept on the table with obstacle and discontinuous surface to be cleaned by robots

Figure 1 highlights the typical settings found inside a given hospital ward. The surfaces considered in this work are mainly vertical and horizontal, with discontinuity and gaps. An anthropomorphic ability similar to the human hand is essential to access these surfaces for cleaning or disinfecting tasks. However, the mobile dual-arm (MDA) system is a useful candidate for these tasks. Although the main objective of this manuscript is to present the global path planning approach considering the environment with the use of the dual arm, for brevity, this paper presents the in-house designed MDA along with framework and a simple vision based method for approaching the region of interest using a mobile base.

Mobile Dual-Arm robotics system

Fig. 2

a Mobile dual-arm (MDA) system with anthropomorphic ability designed to access the surfaces highlighted in Fig. 1, b MDA inside a grid for the workspace estimation and c Frame of references for the robot as $\{R\}$ and two end-effectors $\{EE_L\}$ and $\{EE_R\}$

The MDA system used in this work is shown in Fig. 2a. The mobile base has holonomic locomotion ability, which enables it to access tight spaces with its sideways movement. The dual-arm manipulation module is mounted on top of the mobile base. The module is designed to carry and support two off-the-shelf manipulator arms, i.e., Kinova 7 DOF Gen 3 arm. The dual arm mounting frames positions are indicated with $\{A_1\}$ and $\{A_2\}$ with 150 mm apart. Each arm end-effector (EE) is mounted with a microfiber cleaning cloth. The two EE frames are indicated with $\{EE_L\}$ and $\{EE_R\}$ respectively as shown in Fig. 2c. This work targets to provide low computational systems; hence the system is equipped with one unit of ZED 2 stereo camera mounted between the two arms as shown in Fig. 2c. The workspace of the robot is depicted in Fig. 2b. The blue area shows the workspace envelopment achievable by the reach of the arms. The brown area excludes the area as part of the allowable workspace. The geometric parameters of the Kinova 7 DOF Gen 3 arm and control features of the arm can be found at [39] and were also checked and updated using kinematic parameter identification technique proposed in [40]. The arm provides both the position and torque control which is utilized in this work for the contact and non-contact locomotion.

Fig. 3

a Top–down view of a typical hospital setting and b Framework for the execution of the cleaning task in hospital settings

Framework for the execution of cleaning tasks

The framework presents an intuitive flow of how the robot is made to carry out cleaning operations in a given hospital setting, as shown in Fig. 3a. Although the navigation of the mobile base is in an unstructured environment, it is simplified using fiducial markers being placed on the region of interest (RoI) to be accessed or cleaned. Moreover, the waypoints of the robots are shown in Fig. 3a to access the required region with the use of an MDA robot.

Figure 3b highlights the process flow, which is divided into the preprocessing required for executing the task and the steps focusing on the complete area coverage path planner. The preprocessing consists of the MDA robot mapping the area with its onboard 2D Lidar, and the approximate waypoints are laid on the generated map for the robot’s navigation. The waypoints are positioned on the map such that the markers are visible to the camera placed near the neck of the MDA robot. The robot waypoint teaching process is based on the length and width of the effective area to be cleaned $A_{EC}$. It is divided based on the workspace of the robot ($WS_A$) and the field of view of the camera ($FoV_C$) using the intersection of them as the task space, $TS_c = FoV_C \cap WS_A$.

Fig. 4

a Alignment of the MDA robot using RGBD camera and the marker placed inside the RoI near the waypoints along with the algorithm and b The given surface area of hospital bed and its occupancy grid generation with its flowchart indicated below the image

In the next subsection, the alignment scheme of the robot w.r.t. the marker and the arm workspace is presented along with the algorithm for generating the cost map of the RoI to be accessed.

Alignment of robot and cost map generation

In this section, the steps for orienting the robot near the RoI in Fig. 4a and generating the cost map generated from the point cloud obtained from the RGBD camera in Fig. 4b are discussed. The robot self-aligning algorithm solves the three requirements are listed below:

Req1::

The robot is required to orient the normal to the virtual line connecting two neighboring markers.

Req2::

The robot is to place itself in the middle of the line between two neighboring markers.

Req3::

The robot has the maximum effective cleaning coverage area.

Generating the cost map can be achieved using the fiducial markers/AR tags that easily give the pose of AR relative to the origin of the camera frame [41]. Assuming that two markers are visible to the camera, the boundary of the workspace will be defined. The boundary information can be constructed into a bounding box, with the third axis being the depth. The noise tolerance is assigned to the coordinates of the point cloud data. The 3D points are usually available using an RGBD camera, which is constructed with the k-dimensional tree as seen in Fig. 4b. The accuracy of the generated occupancy grid depends on the grid size and sensor imperfections, including noise, lighting variations, and quantization error (particularly relevant for depth sensors). As seen in Fig. 4b, the generated grid (black: obstacles, white: cleaning surfaces) shows rough edges due to these uncertainties among the other uncertainties in the system which is out of the scope of this study.

Complete coverage path planning

This section introduces the proposed CCPP solution. First the Glasius Bio-inspired Neural Network (GBNN) for 3D exploration and its limitations for surface area coverage tasks are highlighted. To overcome these limitations, GBNN with heuristics (GBNN+H), a novel variant designed specifically for efficient area coverage with reduced computational demands and steps are proposed. The symbols and nomenclature used throughout this section are listed in Table 1.

Fig. 5

Visual representation of the i th neuron and its neighboring j th neurons within the receptive field in the neural network

Glasius bio-inspired neural network (GBNN) Algorithm

Grid maps are used to create the GBNN algorithm. However, the neural network is discretized, whereas the external information is typically continuous. As a result, a 3D workspace must be discretized, and a grid map must be created, as seen in Fig. 5b. GBNN in 3D is described by the following equations:

$$\begin{aligned} N_{i}(t+1) = f\left( \sum \limits _{k=1}^{m}w_{ij}N_k^+ + \alpha _{i}\right) \end{aligned}$$

(1)

where $N_{i}(t+1)$ represents the $i^{th}$ neuron’s neural activity levels, $N_k^+$ is the $k^{th}$ neural activity levels where it equals the output of the function $max(N_k,0)$. As it only takes neurons with positive neural activity in the summation, neurons with negative neural activity does not propagate, allowing neurons with positive neural activity to propagate through the workspace. The parameter $\alpha _{i}$ in the following equation:

$$\begin{aligned} \alpha _{i} = {\left\{ \begin{array}{ll} + \mu \,\,=\, ~Surface:~Uncleaned \\ \,\,\,\,\,\,0 \, \,=\, ~Surface:~Cleaned/covered \\ - \mu \, \,=\, ~~Surface~Gap~/~Discontinuous~Surface\\ \end{array}\right. } \end{aligned}$$

(2)

is positive when it is uncleaned, 0 when it is cleaned/covered and negative when it is a gap or a discontinuous surface, where $\mu$ is a large positive constant. The parameter $w_{ij}$ is the connection weight coefficient between neuron i and j within the receptive field R, b is a positive constant and $||i-j||$ is the Euclidean distance between neuron i and j as $\sqrt{\left( x_i-x_j\right) ^2+\left( y_i-y_j\right) ^2+\left( z_i-z_j\right) ^2}$.

$$\begin{aligned} w_{ij} = {\left\{ \begin{array}{ll} \exp ^{-b||i-j||^2}\,,\, 0 < ||i - j|| \le R \\ 0 \,, \,||i - j|| > R.\\ \end{array}\right. } \end{aligned}$$

(3)

Figure 5a shows the ith and jth neuron in a receptive field in a GBNN mesh where $R = 2$. Since Eq. 1 only accepts positive neural activity, only neurons with positive neural activity propagate throughout the GBNN mesh, and negative neural activity does not. The next equation:

$$\begin{aligned} f(u) = {\left\{ \begin{array}{ll} -1 \,,\, u< 0 \\ a u\,,\, 0 \le u < 1\,,\quad where\, a> 0\\ 1 \,,\, u > 1\\ \end{array}\right. } \end{aligned}$$

(4)

bounds the neural activity levels to between 1 and $-1$, where a is a positive constant. Lastly, GBNN selects the next position, $wp_{i+1}$, based on the following equation:

$$\begin{aligned} p_{i+1} = max(N_k+O) \end{aligned}$$

(5)

where O is a function that influences the next direction selection. In the original GBNN [28], O represents the normalized magnitude of the change of direction based on possible next positions. Typically in GBNN, a neuron with a negative neural activity represents an obstacle-obstructed zone that the robot’s EE should avoid. Similar intentions in this representation are used, but also for areas on the cleaning surface where a “hole”, “gap” or a different surface is detected; gaps and obstacles are prohibited or not allowed to cover, as visualized in Fig. 5b. Figure 5b shows how neighboring nodes such as obstacles and gaps, are classified as occupied.

Fig. 6

Variants of GBNN suitable for dual-arm robot operation towards area coverage of the bottom 2D surface. Voxel grid is represented by the cubes in 3D space

GBNN described above is visualized in Fig. 6a, where the algorithm will cover all uncovered areas in the 3D space (i.e. 5 layers grids of depth). GBNN for dual-arm robots can be reduced to 2 cascaded layers of grids to optimize GBNN for dual-arm robots instead of a full 3D mesh which reduces the computation in autonomy and control. Solving the area coverage using the bimanual arm in two states (touching the surface and suspended) uses two layers of GBNN and is referred to as GBNN (2 L). The region of interest for area coverage for the dual-arm robot is the plan surface. A two-layer GBNN is seen in Fig. 6b, which can be applied for dual-arm robots where one layer represents the area of interest and the other layer represents a free space of the cleaned area.

As positive neural activity propagates and negative neural activities are local, GBNN (2 L) is able to efficiently generate the paths for a dual-arm robot to navigate through discontinuous free space in the area of interest by moving through the cleaned layer. Although it is possible to perform CCPP for dual-arm robots using GBNN (2 L), it is computationally intensive. The proposed CCPP, GBNN with heuristics (GBNN+H) in Fig. 6c, is a less computationally intensive CCPP for dual-arm robots which is only applied on a single 2D layer of GBNN grid and will be described next.

GBNN with heuristics

There are many successful cases where heuristics have benefited path planning algorithms in optimizing their performance. For example, Dijkstra’s algorithm is the work of a greedy function. With the addition of heuristics, the $A^{*}$ algorithm is derived. This is also seen in many use cases in the literature found even in coverage planning [42, 43]. It is noted that without the heuristics, 2D complete coverage path planners, such as GBNN, are unable to plan a path across a discontinuous surface. The nearest uncovered neuron G is depicted in Fig. 5c GBNN+H. The equation to derive the heuristic, $H_k$, is as follows:

$$\begin{aligned} H_k= & {} \sqrt{(x_i-x_G)^2+(y_i-y_G)^2} \nonumber \\{} & {} - \sqrt{(x_{k}-x_{G})^2+(y_{k}-y_{G})^2} \end{aligned}$$

(6)

Fig. 7

Example of nearest uncovered neuron G based on the Euclidean distance between the current position and uncovered neurons

where ($x_i, y_i$), ($x_k,y_k$) and ($x_G,y_G$) represents the x and y coordinates of neuron i, k and G respectively. The Euclidean distance from the the $i^{th}$ neuron to nearest uncovered neuron, G, is expressed as $\sqrt{(x_i-x_G)^2+(y_i-y_G)^2}$ or $h_r$ as seen in Fig. 7. Similarly, the potential next position, $k^{th}$ neuron distance to G is expressed as $\sqrt{(x_k-x_{G})^2+(y_k-y_{G})^2}$ or $h_k$. The position of i, k and G is denoted in terms of x and y indices seen in Eq. 6. The difference between each $h_k$ to $h_r$ will be computed as the base heuristic of each neighbor as a possible selection for the next waypoint as seen in Eq. 6. The base heuristic can provide a leading bias towards uncleaned surfaces when the cleaning EE is trapped on a surface segregated by $-\mu$ nodes. When selecting the next waypoint, $h_r$ is held constant while $h_k$ changes between neighbors. For neighboring nodes with larger $h_k$, $H_r$ will be smaller than the neighboring nodes with lower $h_k$ values. The search for the nearest uncovered neuron G coordinates is given as:

$$\begin{aligned}{} & {} x_G, y_G =min(f(\chi )) \end{aligned}$$

(7)

$$\begin{aligned}{} & {} f(x) = \sqrt{(x_i-x_l)^2+(y_i-y_l)^2} \end{aligned}$$

(8)

$$\begin{aligned}{} & {} \chi = \bigcup _{l=1}^{p-1} D_l \end{aligned}$$

(9)

$$\begin{aligned}{} & {} D_l = {\left\{ \begin{array}{ll} \{ x, y\},if~~~~N_{x,y} = 1 \\ Null~~~~~otherwise \\ \end{array}\right. } \end{aligned}$$

(10)

where $\chi$ represents the set of coordinates of neurons in the GBNN grid, p represents the total number of available neurons in the GBNN mesh, and f(x) represents the Euclidean distance between the $i^{th}$ neuron to the $l^{th}$ neuron. The $l^{th}$ neuron refers to all uncovered neurons in the grid. Equation 7 derives the position of the neuron G with the minimum Euclidean distance from the $i^{th}$ neuron. The weighted heuristics, $H_{val}$, is the product of a linear incremental coefficient of C, and H will grow and eventually outweigh the predominant neural activities of a typical GBNN as characterized in Eq. 11. The incremental step of the coefficient is a preset value. Equation 12 shows that the neighboring nodes with the highest selection value is selected as the next waypoint, $P_{i+1}$. The “selection value" is calculated using the neural activity of the neighbor ($N_k$) and its corresponding values from an optimizing function and heuristics values (O, $H_{val}$). This selection will be iterated until all nodes on the grid are less than 1.

$$\begin{aligned}{} & {} H_{val} = C\cdot H_k \end{aligned}$$

(11)

$$\begin{aligned}{} & {} wp_{i+1} = max(N_k+O+H_{val}) \end{aligned}$$

(12)

The proposed function enables a heuristic behavior of the path planning model to be derived. This behavior will be activated over iterations where the planner is successively unable to obtain a selected $wp_{i+1}$ where the neural activity is not equivalent to 1. Upon selecting $P_{i+1}$ neural activity to be at 1, the value of C will be reset as a form to 0. After the reset, the algorithm will restore the full behaviors of a typical GBNN path planner.

In case of the MDA robot, it can move in position and torque control, specifically, impedance control mode [35]. When the path is indicated as an occupied grid, the robot arm’s impedance control (IC) will be switched to positional control. The position control mode moves the end-effector (EE) offset in the normal direction of the surface plane. Therefore, this approach results in the 3D movement of the EE with the cleaning of the surface performed in a 2D grid using impedance control mode. The impedance control mode maintains a constant force with the surface (here 30 N) and moves along the 2D grid path. The proposed algorithm is implemented in the pseudo-code described and presented in Algorithm 1.

Algorithm 1

Pseudocode for GBNN using mobile dual-arm with position and impedance Control.

In the pseudo-code, previous step position for the two end-effectors are, $EE_{lp}$ and $EE_{rp}$. Correspondingly, the generated waypoint of both effectors is $wp_l$ and $wp_r$. The input map is denoted as grid indexed by i and j to represent x and y spatial dimensions. When GetNeighbors is run as seen in line 5 and 7 in Algorithm 1, the selection of neighbor is operated by Eq. 12 on all its identified neighbors, $N_k$, of $EE_{L}$ and $EE_{R}$. As mentioned earlier, the neighbor with the highest value among neighbors will be selected. This implementation of the proposed CCPP is implemented for a dual-arm cleaning operation. In the next Section, the simulation for the proposed GBNN+H is discussed.

Simulation studies

In this section, we evaluate the proposed method against the state-of-the-art CCPP, by comparing it to GBNN without heuristics, the Backtracking Spiral Algorithm (BSA) [25] and the Predatory-Prey Complete Coverage Algorithm (PPCPP) [27]. The simulation is performed by dividing the cost map for a given arbitrary surface as shown in Fig. 8 accounting for obstacles and discontinuity in the surface. Note that the division of the given cost map into two equal halves is simplified based on the symmetric design and reachability of each arm in an MDA robot. The simulation was carried out to demonstrate the efficiency of computation. The efficiency is based on the steps taken for the complete coverage of the test cost map and the computing time of the algorithm. Upon a deadlock situation, the simulation will consider the computation time infinitely high with undefined time complexity. Moreover, each algorithm is run for 500 times and logged with their respective computation time as part of their performances.

Two sets of comparative studies have been undertaken are namely. (a) The initial comparative analysis contrasts the proposed algorithm with its earlier developmental versions, as depicted in Fig. 8. Within this first study, the illustration showcases the performances of GBNN, GBNN(2 L), and GBNN+H on the cleaning surface, presenting a total of 3 simulations on the graph. The empirical results for time are statistically presented, including average time and its standard deviation. The efficiency is measured through the flight of each end effector (EE), and the path is quantified by steps when the EE is not in contact with the cleaning surface. (b) In the second comparative study, the proposed algorithm is compared against the Backtracking Spiral Algorithm (BSA), a widely utilized CCPP, and the Predator-Prey-Based Approach to Adaptive Coverage Path Planner (PPCPP), an alternative state-of-the-art algorithm.

Comparison with existing GBNN

Fig. 8

Surface area coverage (SAC) using the existing GBNN, GBNN(2 L), i.e., two layers, and GBNN with the heuristics. a Highlights the steps taken by the end-effectors ($EE_l$, $EE_r$) at the instance when the area covered is approximately 20, 40, 60, 80, and $99\%$. Note that only the 2D surface which has to be covered is shown, avoiding the layers in 3D. The state of the EE is highlighted with the color code (i.e., red and blue for on the surface and above the surface or suspended). The region of the area coverage for $EE_l$ and $EE_r$ is also demarcated

The first simulation demonstrates the complete coverage function and performance of GBNN. GBNN is known to ensure that the robot path does not get trapped in deadlocks. In the context of cleaning surface applications, GBNN is required to be operated in a 3D grid (x, y, and z). Each dimension represents a point in space. This is similar to how it is applied in exploration, surveillance, search, and rescue missions. To the best of the knowledge of the research group in this study, complete coverage is widely done for cleaning, but there is none that provides cleaning surface applications that operate in 3D. Hence, GBNN is applied in 3D as a direct comparison. In this simulation study, GBNN will only be applied to a mesh consisting of the test graph with 4 additional layers of unoccupied grids. The algorithm’s performance is seen in Fig. 8 and had reached complete coverage with 564 steps in 169.212 ms and a standard deviation of 5.968 ms.

The second simulation demonstrates the reduction of the computational grid by replacing the axis normal to the cleaning surface into a binary control mode (CM), where $\text {CM} = {I, P}$. When $\text {CM} = I$, it denotes that the EE is in contact with the surface. When $\text {CM} = P$, it denotes that the EE is offset from the surface, controlled by a fixed position control. Logically, this would directly reduce the amount of computation required as compared to the originally proposed GBNN set to compute in a full 3D space. In the same comparison of progress against the original GBNN, the algorithm’s performance is seen in Fig. 8. The algorithm obtained full coverage with 346 steps in 105.74 ms and a standard deviation of 3.51.

Numerical results are discussed in Sects. “Comparison with existing GBNN” and “Comparison with other State-of-the-art coverage algorithm” to observe changes from modifications and a comparative study with other existing solutions. The analytical interpretations of the results and additional observations will be discussed in Sect. “Simulation discussion”.

Table 1 Simulation results based on the coverage task shown in Fig. 8. Flight refers to the number of steps taken when the EE is not touching the surface

The final simulation presents the proposed algorithm. It is computed purely in 2D as compared to the GBNN in the contextualized problem. When the EE is on an obstructed area, it will assume to be in an offset position, otherwise engaged in impedance mode. This will further reduce the computational steps as compared to the examples from earlier. The proposed algorithm achieves full coverage with 85 steps in 31.309 ms and a standard deviation of 0.974 ms.

All results mentioned above are results described in general for both EE computed by the proposed algorithm. In detail, the executed time, the standard deviation of performance in 500 runs, and amount of steps taken for complete coverage and flight are recorded in Table 1.

Comparison with other State-of-the-art coverage algorithm

Fig. 9

Surface area coverage using the existing BSA, PPCPP (2D), PPCPP (3D) and GBNN with the heuristics. Highlights the steps taken by the end-effectors ($EE_l$, $EE_r$) at the instance when the area covered is approximately 20, 40, 60, 80, and $99\%$. Note that the workspace shown above and later in the experiments are divided equally in this paper for the dual-arm, i.e., with $EE_l$ and $EE_r$ (as done in Fig. 8) to access the given grid map

The second set of simulation presents the proposed algorithm compared against other complete coverage path planners. There are many work that addresses complete coverage. This segments aims to show the computational improvement made by the proposed method. BSA, PPCPP and GBNN$+$H is executed for complete coverage on the same 2D grid as shown in Fig. 9. PPCPP algorithm is executed on a 3D grid by considering the 3D neighbours in the waypoint selection process. For BSA and PPCCPP (2D), the algorithm is unable to achieve complete coverage (not more than 60%). One way to overcome this is to execute the path planner in a 3D grid space. PPCPP is executed once again in 3D. The PPCPP (3D) algorithm achieves full coverage with 417 steps in 1227.1 ms as shown in Fig. 9. The average performance of GBNN$+$H and PPCPP are 8.54 ms and 36.01 s and a standard deviation of 0.22 ms and 5.66 s, respectively.

Simulation discussion

Evaluating performance of different algorithms for area coverage tasks, the path length or the mumbler of way points generated as steps is a more reliable and consistent metric than execution time, as time measurements can be significantly influenced by CPU conditions. Our test results demonstrate the superior efficiency of GBNN+H: GBNN+H required only 83 steps to complete the task, compared to 563 steps for the standard GBNN-a remarkable 6.7-fold reduction, GBNN(2 L) performed better than GBNN but still required 218 steps, 2.6 times more than GBNN+H. GBNN+H exhibited notable speed improvements, completing the task approximately 5.4 times faster than GBNN and 3.6 times faster than GBNN(2 L).

The challenge becomes apparent with GBNN during its initial simulation, where it exhibits flight times ranging from 11 to 351 steps. The flight duration for $EE_l$ and $EE_r$ accounts for $90.6\%$ and $87.7\%$ of the total steps, respectively. However, this issue is mitigated with the adoption of the 2-layer approach, reducing the flight range to 0–21 steps. The proposed algorithm simulated shows that a 2D problem can be implemented in a 3D path planning requirement. This is resolved by having the path planner plan in 2D with an assumption approach when the EE is in a prohibited area. The EE (or move base in other applications) will move to a free zone away from the cleaning surface plane. Heuristics were known to improve the efficiency of path planners, and it is shown in this study for coverage planners. In terms of flight ratio, i.e., (number of steps not touching/total number steps) both EE performed as $0\%$ meaning steps are touching the surface always and $9.5\%$ (minimum and maximum flight range is from 1–2 steps). Moreover, the reduction of dimension from the 2-layered version of a 3D space cannot be completely reduced to 2D without the use of the heuristics function.

In comparison to other Complete Coverage Path Planning (CCPP) algorithms, the simulation reveals common challenges faced by most path planners that are confined to operate solely on a 2D plane. Unlike typical 2D path planners, GBNN$+$H transcends limitations imposed by the discontinuous surface plane. While implementing Predator-Prey-Based Approach to Adaptive Coverage Path Planner (PPCPP) on a 3D grid enables complete coverage on a discontinuous surface plane, akin to GBNN implementation, it often encounters cyclic behavior and deadlock scenarios. These challenges are effectively addressed by employing a shortest path algorithm and introducing randomness to disrupt repetitive lock-ups, as recommended in [27]. The incorporation of randomness results in a highly varied performance range over 500 iterations, as observed through the standard deviation in Sect. “Comparison with other State-of-the-art coverage algorithm”. Notably, GBNN$+$H surpasses the Backtracking Spiral Algorithm (BSA) and PPCPP in the comparative study.

During this simulation study, the $EE_l$ finishes its task earlier than the $EE_r$ due to the absence of complete obstruction. When the prohibited area encloses part of the workspace, it results in breaking up any of the EE designated workspace and will result in the EE taking longer to complete its intended task at the bottom layer. Comparing the simulations, GBNN+H has shown superiority by having a minimal flight ratio. The EE does not lift into suspension when there is no demand to cross obstruction. It is minimally suspended when there is a demand to cross an obstructed area. Simulation has shown the proposed solution is capable of reducing the computation time by 4 times in terms of steps taken when discontinuous space is present. It has also shown its suitability as a path planning algorithm by significantly reducing flight time and path coverage to have more proportion of time spent on the cleaning surface.

Experiments

This section will demonstrate the corresponding graphs in actual spaces in hospitals and evaluate the proposed algorithm with GBNN and its improved model, GBNN(2 L). The experiment will include testing on vertical, horizontal, and discontinuous surfaces. Additionally, the algorithm is tested to verify its compatibility in an alternative environment with higher environmental complexity. Unlike simulation studies, the PPCPP algorithm and BSA will not be repeated in this section and will focus more on the direct comparison of similar algorithms.

This section will begin by describing the experimental set-up in this study, then followed by the applications on a hospital bed’s vertical surface in Sect. “Vertical test surface: section of hospital bed” and horizontal in Sect. “Horizontal test surface: section of hospital bed” surface. Additionally, the study also included tests with a discontinuity in Sect. “Section of hospital bed: discontinuous test surface” and Sect. “Alternate test surface: disjoint table” as proof of intended purpose and robustness.

Experiment setup

The test environment is shown in Fig. 10. The experiments are conducted in an indoor room with constant lighting to reduce image processing-based uncertainties. The ZED 2 camera as shown in Fig. 2 placed between the two arms was configured to depth stabilization mode to reduce uncertainties in the depth values. Depth stabilization is a feature that temporally fuses and filters the depth map over several frames. This is suitable for the proposed algorithm in real-time applications, as the surfaces of interest in this study are cleaning surfaces in a hospital setting, which are mostly fixed in their respective positions. During experiments, the environment and its corresponding grid maps are shown in Fig. 10. In practice, occupancy grid maps are inflated for reasons such as safety, robot size and robot geometry. The approximated grid maps are presented with 50 mm inflation for obstacles detected, which corresponds to 1 grid width; each grid node is approximately $5 \times 5$. Negative obstacles (seen as gaps) are not inflated like in simulation.

Figure 10a shows a vertical surface, while Fig. 10b–d show horizontal planes. Figure 10c is used to evaluate the CCPP performance in a simple horizontal plane, while Fig. 10b is used to evaluate the CCPP performance in a discontinuous surface plane when a complete split or gap of the workspace exists for one of the end-effectors (EEs). An experiment was also conducted over an alternate workspace, shown in Fig. 10d, which is a specific application where the robot operates between two tables that are disjoint, representing a discontinuous surface and an occupied area. Note that the workspace in the experiments are divided equally throughout for the dual-arm. This is based on the assumption that the mobile base is place optimally to have approximately equal reachability of both the arms. The optimal placement of the MDA near its workspace is not in the scope of current study. The results are discussed in coming sections with interpretations and additional observations will be discussed in Sect. “Discussion on experimental outcomes”.

Vertical test surface: section of hospital bed

This section highlights the experiment carried on a vertical surface of the hospital bed as shown in Fig. 10a and e. This test shows a plain workspace almost similar to an empty map. The surface is considerably simple, as only one elongated gap is present near the targeted surface, as seen in Fig. 11. GBNN will be used as a reference for the state-of-the-art along with GBNN(2 L), where only its input representation is modified instead of the algorithm.

Fig. 10

a Vertical test surface, b Horizontal test surface, c Discontinuous test surface, d Alternate test surface as disjoint table, and e–h corresponding occupancy grid map, respectively

Fig. 11

Algorithms performance in area coverage of vertical surface S1 as shown in Fig. 10a and e. Note that the workspace shown above are divided equally for the left and right arm of MDA robot

Table 2 Summary of performance metrics corresponding to accessing surface as shown in Fig. 11. Flight refers to the number of steps taken when the EE is not on the surface

The computational time for the path planning using the three algorithms, namely, GBNN, GBNN(2 L) and GBNN+H, for the vertical test surface shown in Fig. 11 is summarised in Table 2. The average time for the execution for GBNN, GBNN(2 L) and GBNN+H are 346.556 ms, 70.456 ms and 51.384 ms, respectively. In terms of steps, GBNN, GBNN(2 L) and GBNN+H took 962, 190 and 111 steps. GBNN+H took lesser time and number of steps (1.7 times lesser) compared to the state-of-the-art algorithms taken as GBNN and GBNN(2 L). The discussion is presented in Sub-section “Discussion on experimental outcomes”.

Horizontal test surface: section of hospital bed

This section highlights the experiments conducted on the horizontal test surface of the hospital bed as shown in Fig. 10b and f. Similarly, this is a considerably simple workspace as well. However, there is some inflation present at the bottom and right side of the graph, as seen in Fig. 12. The inflation is due to other parts of the bed present. Additionally, the graph is half from the top, which represents the further end of the hospital bed. The workspace is reduced due to the limited reach of the dual-arm robot on this plane. This is the first demonstration of horizontal surface cleaning for the robot.

Fig. 12

Algorithms performance in area coverage of vertical surface S3 as shown in Fig. 10c and g

Table 3 Summary of performance metrics corresponding to accessing surface as shown in Fig. 12. Flight refers to the number of steps taken when the EE is not on the surface

The computational time for the path planning using the three algorithms, namely, GBNN, GBNN(2 L) and GBNN+H, for the horizontal test surface shown in Fig. 12 is summarised in Table 3. The average time for the execution for GBNN, GBNN(2 L) and GBNN+H are 116.446 ms, 21.633 ms and 17.144 ms, respectively. In terms of the number of steps taken, GBNN, GBNN(2 L) and GBNN+H took 474, 76 and 56 steps for the surface area coverage.

Section of hospital bed: discontinuous test surface

This section highlights the test on a horizontal surface as shown in Fig. 10c and g. This is also a horizontal surface with a reduced grid due to robot arm reach, as seen in Fig. 13. The bottom of the graph shows obstacle inflation from the side of the bed nearer to the robot. The targeted feature that motivated the study of this paper is the complete blockage of not accessible areas which leads to the discontinuation of surfaces. This is described as the discontinuous surfaces graph in this paper. The complete blockage is present for the $EE_l$ while $EE_r$ has to deal with multiple protruded features on the right side of the graph.

Fig. 13

Algorithms performance in area coverage of vertical surface S2 as shown in Fig. 10b and f

Table 4 Summary of performance metrics corresponding to accessing surface as shown in Fig. 13. Term flight refers to the number of steps taken when the EE is not on the surface

Fig. 14

Algorithms performance in area coverage of horizontal with gap S4 as shown in Fig. 10d and h

The computational time for the path planning using the three algorithms, namely, GBNN, GBNN(2 L) and GBNN+H, for the discontinuous test surface shown in Fig. 13 is summarised in Table 4. The average time for the execution for GBNN, GBNN(2 L) and GBNN+H are 108.835 ms, 26.709 ms and 15.262 ms, respectively. In terms of steps, GBNN, GBNN(2 L) and GBNN+H took 446, 103 and 51 steps.

Alternate test surface: disjoint table

As a typical cleaning surface, table tops are commonly needed to be wiped. As the function of the proposed algorithm is to provide complete coverage path planning, the robot will not be performing other modes of cleaning other than wiping surfaces. This section shows an alternate workspace tested that shows that the algorithm is applicable beyond the scope of hospital settings.

In this environment, as seen in Fig. 14, there are significantly more obstacles present seen in the workspaces presented in Sect. “Section of hospital bed: discontinuous test surface” and also the presence of a gap between tables. Inflation is applied in the same fashion as described in the previous subsection. This experiment shows that the algorithm is considered generic and portable to be applied in any form of cleaning surface that requires 3D maneuvering.

The computational time for the path planning using the three algorithms, namely, GBNN, GBNN(2 L) and GBNN+H, for the disjoint table shown in Fig. 11 is summarised in Table 2. The average time for the execution for GBNN, GBNN(2 L) and GBNN+H are 1779.28 ms, 1592.25 ms and 118.78 ms, respectively. In terms of steps, GBNN, GBNN(2 L) and GBNN+H took 2222, 1957 and 120 steps respectively.

Table 5 Summary of performance metrics corresponding to accessing surface as shown in Fig. 14

Discussion on experimental outcomes

The proposed algorithm is generic and versatile in application. It can be applied regardless of the workspace plane orientation, as shown in the experimental tests in Sects.  “Vertical test surface: section of hospital bed” and “Horizontal test surface: section of hospital bed”. In Sects. “Section of hospital bed: discontinuous test surface” and “Alternate test surface: disjoint table”, surfaces with significantly higher complexity are presented. The proposed algorithm is seen to largely outperform the state-of-the-art algorithms by at least 14 times and 16.3 times in the experiment Sect. “Alternate test surface: disjoint table” in terms of the number of steps taken, compared to GBNN and GBNN(2 L), respectively. This is because the heuristics function in the proposed algorithm can optimize the path plan to avoid local maximums, which is a common problem with GBNN and GBNN(2 L).

In a typical workspace, as seen in Figs. 11 and 12, GBNN is naturally inefficient because it is frequently drawn away from the cleaning surface by local maximums. GBNN(2 L) is seen to be less affected by this problem, but it still takes longer to reach all of the uncovered areas.

When a discontinuous workspace is present, as seen in Figs. 12 and 14, the difference in complete coverage efficiency is even more pronounced. Both GBNN and GBNN(2 L) are significantly affected by local maximums, which causes them to take a long time to reach the remaining uncovered areas. However, GBNN+H is able to quickly find the nearest uncovered area and navigate to it, thanks to the proposed heuristics function.

The resolution of the grid size on the graph also plays a role in the efficiency of the path plan. Complex geometrical features on a graphical grid can only be represented given sufficiently small grids. Large grids tend to lose the features of objects. While fine resolution helps CCPP to detail the environment vividly, it also leads to having more nodes on the grid to compute. As discussed earlier, local nodes can be distracting for performing surface cleaning. In this sense, GBNN+H excels in a complex environment by having the ability to navigate efficiently in fine-resolution workspaces.

GBNN+H is also more efficient regarding the time taken for the coverage. In Tables 2 and 3, GBNN+H has shown that it can cover large maps without lifting from the surface. It is worth noting that in Table 3, GBNN+H has a record of being suspended in the air in position control mode. This is not due to the end-effector (EE) lifting from the surface, but rather an initialization requirement due to occupancy on the surface layer. GBNN(2 L) is seen to be performing relatively similar in experiments for vertical surfaces and horizontal surfaces in Sects. “Vertical Test Surface: Section of Hospital Bed” and “Horizontal Test Surface: Section of Hospital Bed”, respectively.

The difference between GBNN+H and GBNN(2 L) becomes more pronounced in Tables 4 and 5. When the algorithms are applied for the use case of the table, there are several isolating cleaning areas. This requires the EE to lift from and press on the cleaning surface multiple times. Despite this scenario, GBNN+H performs better in confined cleaning tasks, which is reflected in the flight time, which is approimately 17 times shorter than GBNN(2 L). In summary, the proposed algorithm has been found to be significantly more efficient than the state-of-the-art when dealing with a discontinuous workspace, with a considerable reduction in the required steps for complete coverage.

Conclusion

This paper introduces a novel framework for achieving comprehensive surface coverage in cleaning tasks using a mobile dual-arm robotic system (MDA). The proposed algorithm, GBNN+H (Glasius Bioinspired Neural Network with Heuristics), addresses the intricate challenges of path planning in a workspace with horizontal and vertical surfaces, exemplified by hospital beds and discontinuous tables. The study highlights the superior performance of GBNN+H in a hospital setting, especially on discontinuous surfaces where existing 2D and 3D complete area-coverage path planners face limitations. The proposed path planner demonstrates efficiency gains over the state-of-the-art algorithms such as GBNN, PPCPP, and BSA, offering a significant reduction in computing time.

The successful experimental demonstration of the proposed framework on various surfaces commonly found in hospitals underscores its real-world applicability. The solution presented here addresses the unique challenge posed by discontinuous surfaces, enabling autonomous waypoint planning for complete coverage. The proposed solution paves the way for automation in hospitals, contributing to improved efficiency and disinfection tasks.

Looking ahead, our future research endeavors aim to optimize GBNN and its heuristics function parameters. Additionally, we plan to extend the application of the algorithm to scenarios involving multi-robot Collaboration, human-robot collaboration, or other robots with similar constraints. In summary, this work provides a comprehensive solution to the complexities of path planning using MDA robotic systems, offering promising results and laying the foundation for advancements in the field of robotic cleaning and automation.