Exploiting the Robot Kinematic Redundancy for Emotion Conveyance to Humans as a Lower Priority Task

Abstract

Current approaches do not allow robots to execute a task and simultaneously convey emotions to users using their body motions. This paper explores the capabilities of the Jacobian null space of a humanoid robot to convey emotions. A task priority formulation has been implemented in a Pepper robot which allows the specification of a primary task (waving gesture, transportation of an object, etc.) and exploits the kinematic redundancy of the robot to convey emotions to humans as a lower priority task. The emotions, defined by Mehrabian as points in the pleasure–arousal–dominance space, generate intermediate motion features (jerkiness, activity and gaze) that carry the emotional information. A map from this features to the joints of the robot is presented. A user study has been conducted in which emotional motions have been shown to 30 participants. The results show that happiness and sadness are very well conveyed to the user, calm is moderately well conveyed, and fear is not well conveyed. An analysis on the dependencies between the motion features and the emotions perceived by the participants shows that activity correlates positively with arousal, jerkiness is not perceived by the user, and gaze conveys dominance when activity is low. The results indicate a strong influence of the most energetic motions of the emotional task and point out new directions for further research. Overall, the results show that the null space approach can be regarded as a promising mean to convey emotions as a lower priority task.

Appendices

Appendix 1

The emotional conveyance algorithm: task priority proof

Following the work in [5, 14], a proof of the task prioritization of the proposed solution is shown below, that is, that the execution of the lower priority tasks does not affect the execution of the higher priority tasks.

The next identities will be used:

$$\begin{aligned} A \, A^+ \, A = A \end{aligned}$$

(9)

where $A^+$ is the pseudoinverse of A.

Given an idempotent matrix B, that is, $B = B^2$; and Hermitian, $B = B^*$ in general, with $B^*$ the conjugate of B, and $B = B^T$ in particular for this work, then

$$\begin{aligned} B \, (A B)^+ = (A B)^+ \end{aligned}$$

(10)

Given a matrix C, then $D = I - C^+C$ is the orthogonal projector onto the kernel of C, thus idempotent and Hermitian. So, in light of (10):

$$\begin{aligned} (A D)^+ = D (A D)^+ \end{aligned}$$

(11)

Given a task $x_i$ of the robot defined as a function of its configuration q:

$$\begin{aligned} x_i = f_i(q) \end{aligned}$$

differentiating and applying the chain rule,

$$\begin{aligned} \dot{x}_i = \frac{\partial x_i}{\partial t} = \frac{\partial f_i(q)}{\partial t} = \frac{\partial f_i(q)}{\partial q} \frac{\partial q}{\partial t} = J_i \, \dot{q} \end{aligned}$$

a mapping between the velocity of task i and the joint velocities is obtained.

Similarly, the differential mapping of the main task t, as defined in Sect. 3.4, becomes $\dot{x}_t = J_t \, \dot{q}$. The joint velocities $\dot{q}$ as defined in this work in (8) are:

$$\begin{aligned} \dot{q} = J_t^+ e_t + P_t \; J_h^+ e_h + \left[ P_t - (J_h P_t)^+ (J_h P_t) \right] e_m \end{aligned}$$

(12)

thus substituting (12) them in $x_t = J_t \, \dot{q}$ it is obtained:

$$\begin{aligned} \dot{x}_t = J_t \, \left\{ J_t^+ e_t + P_t \; J_h^+ e_h + \left[ P_t - (J_h P_t)^+ (J_h P_t) \right] e_m \right\} \ \end{aligned}$$

Using (11) in $(J_h P_t)^+$ and rearranging terms the expression can be transformed to

$$\begin{aligned} \dot{x}_t = J_t \, J_t^+ e_t + J_t \, P_t \left\{ J_h^+ e_h + \left[ I - (J_h P_t)^+ (J_h P_t) \right] e_m \right\} \end{aligned}$$

and now using (9) in $J_t \, P_t$ as:

$$\begin{aligned} J_t \, P_t = J_t \left( I - J_t^+ J_t \right) = J_t - J_t J_t^+ J_t = J_t - J_t = 0 \end{aligned}$$

becomes $\dot{x}_t = J_t \, J_t^+ e_t$. This expression shows that the emotional tasks h and m do not affect the execution of the main task t.

Similarly for the task h corresponding to the gaze:

$$\begin{aligned} \dot{x}_h= & {} J_h \, \left\{ J_t^+ e_t + P_t \; J_h^+ e_h + \left[ P_t - (J_h P_t)^+ (J_h P_t) \right] e_m \right\} \\= & {} J_h \, J_t^+ e_t + J_h \, P_t \; J_h^+ e_h + J_h \, \left[ P_t - (J_h P_t)^+ (J_h P_t) \right] e_m \end{aligned}$$

which using (9) and (10)

$$\begin{aligned}&J_h \left[ P_t - (J_h P_t)^+ (J_h P_t) \right] = J_h \left[ P_t - P_t (J_h P_t)^+ (J_h P_t) \right] \\&\quad = J_h P_t - (J_h P_t) (J_h P_t)^+ (J_h P_t) = J_h P_t - J_h P_t = 0 \end{aligned}$$

becomes $\dot{x}_h = J_h \, J_t^+ e_t + J_h \, P_t \; J_h^+ e_h$. In this expression it can be seen that the second priority task h is only affected by the higher priority task t.

Appendix 2

Implementation values

The angular velocity $\omega$ in (3) has been set to $\omega = 2.79$ rad/s. One term has been used in (5) with $n_J = 1$, $a_1 = b_1 = 0.25$ rad and $\omega _1 = 12.57$ rad/s.

Following the convention of Sect. 2 the values $\Theta _{0_i}$, $\Theta _{E_{0i}}$ and $h_i$ implemented in (3) can be seen in Table 4.

Table 4 Implementedvalues in radians