Online planning for multi-robot active perception with self-organising maps

Abstract

We propose a self-organising map (SOM) algorithm as a solution to a new multi-goal path planning problem for active perception and data collection tasks. We optimise paths for a multi-robot team that aims to maximally observe a set of nodes in the environment. The selected nodes are observed by visiting associated viewpoint regions defined by a sensor model. The key problem characteristics are that the viewpoint regions are overlapping polygonal continuous regions, each node has an observation reward, and the robots are constrained by travel budgets. The SOM algorithm jointly selects and allocates nodes to the robots and finds favourable sequences of sensing locations. The algorithm has a runtime complexity that is polynomial in the number of nodes to be observed and the magnitude of the relative weighting of rewards. We show empirically the runtime is sublinear in the number of robots. We demonstrate feasibility for the active perception task of observing a set of 3D objects. The viewpoint regions consider sensing ranges and self-occlusions, and the rewards are measured as discriminability in the ensemble of shape functions feature space. Exploration objectives for online tasks where the environment is only partially known in advance are modelled by introducing goal regions in unexplored space. Online replanning is performed efficiently by adapting previous solutions as new information becomes available. Simulations were performed using a 3D point-cloud dataset from a real robot in a large outdoor environment. Our results show the proposed methods enable multi-robot planning for online active perception tasks with continuous sets of candidate viewpoints and long planning horizons.

Appendix: Convergence of SOM

In this appendix, we elaborate on the convergence properties discussed in Remark 2 of Sect. 5.1.1 to provide insight into the behaviour of the algorithm.

There are two key phases of the algorithm to consider when analysing the convergence properties. After epoch $i_{max}=1/\delta$, the neighbourhood function definition (1) ensures that no further adaptations occur, and thus the algorithm is guaranteed to have converged (Lemma 2). However, the algorithm will typically converge prior to $i_{max}$, but after the neighbourhood function (1) reaches 0 for $l>0$. Because such a convergence depends on the spatial configuration of the viewpoint regions, it is not possible to prove convergence prior to $i_{max}$ for all possible cases. In fact, it is possible to show that for certain configurations, a particular waypoint of the network may oscillate between viewpoint locations during the learning epochs. However, these are specific cases and do not occur frequently; therefore, faster convergence occurs with a high probability. Furthermore, these oscillations do not occur to the best found solution, which is what is actually returned by the algorithm. In the remainder of this appendix, we discuss the intuition behind these claims of convergence. For simplicity and without loss of generality, we suppose a single robot problem ($R=1$) and the viewpoint regions are defined as discrete points.

The crucial property for determining convergence is that, after some epoch, only the winner is moved towards the viewpoint location while the neighbours of the winner are not adapted. This occurs when $f(\sigma ,l)=0$ for $l>0$, and thus only the waypoints with the cardinal distance $l=0$ to $x^\star$ can be adapted, i.e., only the winner waypoint $x^\star$ is moved. In the case of $\sigma _0=4$ and $\delta =0.002$, this occurs at learning epoch $i = 68$; this agrees with our empirical analysis of the convergence of solutions in Fig. 5.

The main intuition behind the convergence is related to the limited travel cost budget $b^1$. The adaptation is performed only if the sequence of waypoints satisfies the budget constraint after the adaptation. Let the value of the neighbourhood function be non-zero only for the winner waypoint, as described above. In cases where the winner is a new waypoint $x_e$ on an edge (Algorithm 2 line 13), the length of the path (sequence of waypoints $X^i$) must increase, as illustrated in Fig. 17. Thus, if edge nodes are selected as winners, the network will stop adapting once the budget limit is reached.

Fig. 17

Illustration of a scenario where the winner is on an edge of the path. All possible adaptations result in an increased path length

Fig. 18

Illustration of a scenario where the winner is an existing waypoint. The path after the adaptation to $z^\star$ is shown as the dashed lines connecting $x^i_j$ with $z^\star$ and $z^\star$ with $x^i_{j'}$. This adaptation results in an increased path length

The other case to consider is when the winner is an existing waypoint $x^\star$. Let the neighbouring waypoints to $x^\star$ be $x^i_j$ and $x^i_{j'}$. This scenario is illustrated in Fig. 18. The shaded area represents possible locations for $z^\star$ that would result in the existing waypoint $x^\star$ being selected as the winner. This area is defined as the intersection of the half-planes with boundaries perpendicular to $(x^i_j,x^\star )$ and $(x^\star ,x^i_{j'})$. In the Fig. 18 adaptation scenario, the length of the path increases in a similar way to the edge waypoint $x_e$ winner case. Thus, this type of adaptation can only occur up until the cost budget is met.

However, when an existing $x^\star$ waypoint is selected as the winner, it is possible for the length of the path to decrease as a result of the adaptation. A simple example of such a situation is visualised in Fig. 19 with two fixed waypoints at $z_1$ and $z_2$. Assume the travel cost budget is such that the path can visit either $z_3$ or $z_4$, but it is impossible to visit both $z_3$ and $z_4$ without exceeding the budget. If the network is in the configuration shown in Fig. 19a, and the random permutation of the viewpoints be such that the node $z_3$ is presented as the first node, then the winner waypoint at $z_4$ will be adapted to $z_3$. When $z_4$ is presented, the network adapts back to $z_4$ (Fig. 19b). In this configuration, the waypoint may continue to oscillate between $z_4$ and $z_3$ until $f(\sigma ,0)$ finally reaches zero. However, it is rare for such a configuration of viewpoint locations to occur. Also, the network may also adapt towards other viewpoints, which will increase the path length until the cost budget is reached, causing the oscillations to eventually cease.

Fig. 19

An example scenario that may cause the network to oscillate between two viewpoint locations $z_3$ and $z_4$

We also note that the algorithm maintains the best found solution (Algorithm 1 line 15). When the network is oscillating as described above, the best found solution is likely to remain constant. This is the solution that is returned by the algorithm. An empirical verification of the presented intuition behind the solution convergence is reported in Fig. 6, where the solution does not change after 50 learning epochs. The network itself does not change after 70 epochs, which further supports the presented idea that the network typically converges much sooner than $i_{max}=1/\delta$ learning epochs.