A neural integrator model for planning and value-based decision making of a robotics assistant

Abstract

Modern manufacturing and assembly environments are characterized by a high variability in the built process which challenges human–robot cooperation. To reduce the cognitive workload of the operator, the robot should not only be able to learn from experience but also to plan and decide autonomously. Here, we present an approach based on Dynamic Neural Fields that apply brain-like computations to endow a robot with these cognitive functions. A neural integrator is used to model the gradual accumulation of sensory and other evidence as time-varying persistent activity of neural populations. The decision to act is modeled by a competitive dynamics between neural populations linked to different motor behaviors. They receive the persistent activation pattern of the integrators as input. In the first experiment, a robot learns rapidly by observation the sequential order of object transfers between an assistant and an operator to subsequently substitute the assistant in the joint task. The results show that the robot is able to proactively plan the series of handovers in the correct order. In the second experiment, a mobile robot searches at two different workbenches for a specific object to deliver it to an operator. The object may appear at the two locations in a certain time period with independent probabilities unknown to the robot. The trial-by-trial decision under uncertainty is biased by the accumulated evidence of past successes and choices. The choice behavior over a longer period reveals that the robot achieves a high search efficiency in stationary as well as dynamic environments.

Initial conditions and parameters

1.1 Initial conditions

1.1.1 Assembly task

For the model simulations, the initial conditions of the fields governed by the Amari dynamics, $u_{per}$, $u_{on}$, $u_{wm}$ and $u_d$ are defined by the inhibition parameter h. For the coupled two-field model the initial conditions are given by:

$$\begin{aligned}&u_m(x,y,0) = -1, \end{aligned}$$

(21a)

$$\begin{aligned}&v_m(x,y,0) = -0.25 - u_m(x,y,0). \end{aligned}$$

(21b)

1.1.2 Value-based decision-making task

The initial condition of the decision field at the start of simulation trial n is given by:

$$\begin{aligned} u_{d_{n}}(x,0) = {\left\{ \begin{array}{ll} I_{prob}(x) - h_{d_{0}} \quad \mathrm{if} &{} n=1, \\ \begin{aligned} &{} \left( u_{r_{n-1}}(x) + v_{r_{n-1}}(x) \right) \\ &{} - c_d \left( u_{c_{n-1}}(x) + v_{c_{n-1}}(x) \right) - h_{d_{0}}, \end{aligned} &{} \text {otherwise.} \end{array}\right. } \end{aligned}$$

(22)

The initial condition for the choice integration layer $(u_c,v_c)$ in the first trial and after each reset is given by

$$\begin{aligned}&u_c(x,0) = -0.5, \end{aligned}$$

(23a)

$$\begin{aligned}&v_c(x,0) = - u_c(x,0). \end{aligned}$$

(23b)

The initial condition for the success integration layer $(u_r,v_r)$ in the first trial and after each reset is given by

$$\begin{aligned}&u_r(x,0) = -0.5, \end{aligned}$$

(24a)

$$\begin{aligned}&v_r(x,0) = I_{prob}(x)- u_r(x,0). \end{aligned}$$

(24b)

1.2 Model parameters

See Tables 4 and 5.

Table 4 Parameter values of the field equations used for sequence learning and planning

Table 5 Parameter values of the field equations used for value-based decision making

1.3 Numerical model simulations

Numerical simulations of the model were done in MATLAB using a forward Euler method with parameters given in Table 6. To compute the spatial convolution of w and f we employ a fast Fourier transform (FFT), using MATLAB’s in-built functions fft and ifft to perform the Fourier transform and the inverse Fourier transform, respectively.

Table 6 Spatial and temporal discretization of the neural field models