1 Introduction

The use of robots within pedestrian spaces is becoming increasingly common. Mobile robots with various shapes, sizes and functions have been applied in different essential areas. For example, humanoid robots such as Pepper have been used in home and public environment [25], autonomous vehicles have been increasingly observed on the road and smart wheelchairs have been developed and tested in clinical trials [3, 7].

In many of these environments, the robot must interact with pedestrians in a safe and potentially social way, requiring an understanding of pedestrian dynamics in response to different robots. However, state-of-the-art approaches normally model pedestrian dynamics using simulation or with data collected in human-only experiments [11, 13]. The few works that have explored pedestrian dynamics in a robot-populated environment either studied this with a specific type of robot or limited the number of pedestrians [6, 21, 38]. Although human perception and interaction with different types of robots have been studied in many areas, it remains to be explored in a crowd-robot navigation scenario.

In this paper, we aim to understand whether and how pedestrian crowd-dynamics will be changed in the presence of a robot, and how this change would be affected by the robot type. To approach this problem, we identified two specific robots (a Pepper humanoid robot and a smart wheelchair) which have been widely used and researched, and represent two distinct types of robots that pedestrians would be likely to encounter in their daily life in the future. These two types of smart machines, while overlapping in the areas they would operate, differ in appearance, physics and level of autonomy, thus posing interesting cases to be studied. Specifically, we conducted a crowd-robot gate-crossing experiment with these two robots and measured the effect each robot exerted on the crowds macroscopically (i.e. as one moving body of people) and on groups of individuals microscopically (i.e. pedestrians in close proximity and far away from the robot). In addition, the pedestrians’ perception of the robot and the associated actions were measured qualitatively using surveys.

Our study makes the following contributions:

  1. 1.

    The first controlled crowd-robot experiment with recorded pedestrian trajectory dataset in the presence of a robot, which presents novel results.

  2. 2.

    An understanding of how pedestrian and crowd dynamics is affected by a robot. More specifically, we use both local and global metrics to explore further the effect of the robot motion on participants at the closest proximity of the robot. It will inform the design of a more natural crowd prediction method and a more realistic crowd simulation scenario.

  3. 3.

    An understanding of how the type of robot affects pedestrian behaviour, which highlights considerations for designing robotic motion planning algorithms that also take into account the effect of the robot on the crowd.

2 Related Work

2.1 Pedestrian Robot Interaction

The study of human–robot interaction has been an emerging area over the past few years. People’s perception and reaction towards a robot is different from that to another human, and is greatly affected by factors such as demographics [22], appearance and size of the robot [5], perceived likeability and aggressiveness of the robot [23], and personal experience with pets or robots [34]. Despite this wide range of study, human–robot interaction in navigation is still relatively unexplored.

Recently, a small number of works have been exploring pedestrian-robot interaction and its effect on pedestrian behaviours. In general, the studies can be grouped into two categories, uncontrolled data collection with natural pedestrians or controlled experiment with recruited participants. Rothenbucher et al. (2016) studied the interaction between autonomous vehicles and road users such as pedestrians and cyclists in daily road crossing scenarios. The vehicle was operated by a human driver but was disguised as an autonomous one by hiding the driver from other road user’s view [32]. Results indicated that most people managed to interact smoothly with the vehicle while a small minority were hesitant about crossing due to the lack of a driver. Kidokoro et al. [16] investigated the influence of a humanoid robot on pedestrian comfort in a shopping mall, and classified the pedestrian behaviour towards the robot into four distinct types: stop to interact, stop to observe, slow down to look, uninterested. Results showed that 31% (out of 1115) of the pedestrians changed their behaviour by either slowing down or stopping as they encountered the robot.

Although these studies with natural pedestrians demonstrated valuable qualitative results in understanding high-level pedestrian-robot interaction, they are subject to various uncontrolled factors such as scenario context and thus having limited flexibility. On the other hand, some research explored the robot influence on pedestrian dynamics in a controlled environment quantitatively. Chen et al. [6] studied pedestrian-robot interaction in a corridor exiting experiment with 11 participants and 1 robot. Results indicated that pedestrians’ overall speed was affected by the presence of the robot. Vassallo et al. (2018) conducted a study which investigated a gate crossing scenario between one pedestrian and one robot where the robot was programmed to replicate the interaction rules of human walkers [37]. Experimental results indicated that there was no difference in terms of the crossing order between human–human and the human–robot case. Interestingly, their previous study indicated that pedestrians tend to give way to a robot which was programmed to move at a constant speed passively [38]. Similarly, Marvrogiannis et al. [21] conducted a within-subjects study to investigate the effect of distinct robot navigation algorithms on pedestrians’ behaviour. In each trial, 3 participants interacted with a lab-built robot which is either been teleoperated or equipped with one of two navigation algorithms. Results showed that the robot navigation algorithm has an effect on human acceleration.

These studies revealed the influence of robot on pedestrian dynamics in a local level. However, most of the experiments were conducted with a single or a small number of pedestrians, which limits their applicability to the crowd-robot navigation scenario. In addition, these works were concerned with one specific type of lab-made robot, which restricts the result from being generalized to other robots. This leads to a question: Does the occurrence of a robot and its type affect the crowd dynamics, and if so, how? In our work, we consider this question by performing a Crowd-Robot-Interaction (CRI) experiment with two different types of robot, and we highlight different effects on crowd dynamics from these two robots.

2.2 Crowd Prediction and Robot Navigation

A human is capable of navigating through crowds by predicting the motion trajectories of surrounding pedestrians and taking them into account when planning his or her own movement. To achieve safe and human-like navigation for robots, it is crucial to mimic this decision-making process by taking pedestrians’ trajectories into consideration. Early efforts in this area include ‘social force model’ [11] which used ‘attractive’ and ‘repulsive’ force to model pedestrian-obstacles and pedestrian-destination interactions.Several extensions are proposed to this model [14, 40]. Yamaguchi et al. [39] proposed to take the grouping behavior, smoothness of movements and preferred speed of the pedestrian into account. The main concern with these models is that the hand-crafted rules may not perfectly reflect the realistic behaviours of humans.

Data-driven approaches are then proposed to resolve this problem. They allow the natural human–human interaction to be captured and learned directly using real-world data. Machine learning and deep learning methods such as Long Short-Term Memory (LSTM) [1] have been applied to predict individual trajectories with the pairwise pedestrian interactions being learned via a social-pooling layer. Generative Adversarial Networks [9, 18], Transformer model [8], etc are also proposed for this task. However, the majority of these models are trained and validated on human trajectory datasets that either only contain pedestrian trajectories (e.g. ETH dataset [27], UCY dataset [20], Grand Central dataset [42]), or contain other non-robot road-users (e.g. Stanford drone dataset [31] that also contains trajectories of bikers, skateboarders, cars, buses, and golf carts, recorded in a University campus). Moreover, as reported in [2], many of these datasets only cover low-to-medium-density crowd activities. A model trained on such data might fail to make correct predictions in new situations.

State-of-the-art work also presents approaches that incorporate pedestrian trajectory prediction into robot navigation. Kerfs [15] adopted an improved version of social-LSTM to predict trajectories distributions and then used a dynamic A* for robot route planning. Similarly, Pradhan et al. [29] predicted pedestrians’ position and used a potential function based path planner for robot navigation in crowds. These approaches achieved robot navigation in human populated environments considering the human–human interaction and how human behaviour would affect the robot’s decision, but ignored the potential influence that the robot would exert on the pedestrians.

In order to consider this mutual effect, some works combined the prediction and planning process together. Trautman et al. [36] addressed this mutual interaction by reasoning the robot and pedestrians’ future trajectories jointly. Their solution was evaluated on the ETH pedestrian dataset [27]. Similarly, Kuderer et al. [36] also treated the navigation problem as jointly planning for robot and pedestrians. Differently from, they learned natural pedestrian behaviours features from their own lab-collected pedestrian data using the idea of maximum entropy [19]. However, the data was recorded without the presence of a robot, and it has been pointed out by the authors that pedestrians may react differently to robots than to other humans.

While the advances in trajectory prediction and robot navigation are often of great significance, they inevitably leave a gap in the validation of the suitability of pedestrian trajectory dataset in robot navigation which requires consideration of the potential effect created by the robot. This gap boils down to understanding the interaction between the crowd and the robot, which we aim to address in this study.

3 Method

3.1 Research Questions

Motivated by the results presented in the literature, we believe that before solving the social navigation problem, it is essential to understand whether the pedestrian and crowd behaviour would change when a robot is present. In this paper, we aim to address two main experimental questions and test our hypothesis:

Q1. How does the presence of a robot influence pedestrian and crowd dynamics?

H1. We hypothesize the robot will affect crowd dynamics both at a global and local level. Globally, we expect to see longer evacuation time, lower averaged speed and acceleration when a robot is added to the crowd. Locally, we expect to see the robot has a more significant effect on pedestrians who are in close proximity to it in terms of trajectory regularity and interaction complexity.

Q2. If the robot does influence crowd behaviour, how does the response vary across the robot type?

H2. We hypothesize pedestrians will behave differently towards different types of robot. In this study, we expect to see the crowds adopt a more conservative behaviour by giving more space and giving way to the smart wheelchair compared to a humanoid robot.

3.2 Crowd-Robot Gate-Crossing Experiment

In order to answer these questions, we conducted a preliminary study on crowd-robot interaction through a controlled gate-crossing experiment. Ethics approval for the study was granted by UCL Interaction Centre (UCLIC) Ethics Committee (ref. UCLIC_1819_011), following a thorough review of the experiment protocol, participant recruitment plan, data protection plan and risk assessments.

3.2.1 Choice of Robot

While a number of related works investigated pedestrian-robot interaction using a lab-made fully autonomous robot platform [6, 21], we decided to use one commercial humanoid robot (Pepper) and one smart wheelchair as shown in Fig. 1. A smart wheelchair is normally built on a standard powered wheelchair and has a collection of sensors for perception and navigation purpose. It can be either operated autonomously or controlled collaboratively between a user and a motion planner. In this study, we are interested in the latter. For a shared-controlled wheelchair, a user can express his or her driving intention through an interface (eg. Joystick), and the wheelchair’s movement will be the result of a negotiation between the user input and the motion planner. It provides people who have mobility impairments and are considered unsafe to drive a traditional wheelchair with a safe mobility solution and allow them to express their driving intentions.

Fig. 1
figure 1

a The humanoid robot pepper, b our smart wheelchair

Full size image

The reason for using these two robots are twofold. First, these two robots are already prevalent in their areas which would allow us to generalize the findings for a wider range of robots. As the worlds’ first social humanoid robot, over 10,000 Pepper have been sold and adopted by companies in the area of education and hospitality [26]. While not being commercialized, smart wheelchairs have been widely used in research and tested in clinical trials. It can be expected to see these two types of robots being used in crowds in the near future. Secondly, the similarity and difference between these robots pose an interesting case to be studied. Pepper has a height of 120 cm which is comparable to the height of a person sitting on a power wheelchair. However, the fact that a person comes together with the smart wheelchair may greatly alter a pedestrian’s perception and thus generate interesting effects that need to be explored. State-of-the-art research that explores human-aware navigation is mainly designed for fully autonomous robots. However the navigation strategies remain to be fully explored for semi-autonomous robots such as the shared-control wheelchair, where a human driver can be seen by the pedestrians. It was suggested by Bingqing et al. [41] that in contrast to a standalone fully-autonomous robot, an additional interaction channel between the wheelchair user and the surrounding pedestrians should be considered for a shared-control wheelchair due to the assumption that a human driver would affect pedestrians’ perceptions and thus walking behaviours compared to the one with a standalone humanoid robot. Consequently, we included the smart (shared-control) wheelchair and Pepper in our study, and took the first step to collect such interaction data.

It should be noted that in this study, we do not consider the potential influence of different navigation algorithms on the pedestrians’ dynamics. Instead, we assume the robots are equipped with human-level navigation strategies. As a result, a Wizard-of-Oz [30] method was adopted in our experiment—the wheelchair was driven by an expert driver and the humanoid robot was tele-operated by an experienced operator. Although the wheelchair is not actually being shared-controlled by both the user and the planner in this experiment, we believe the intended end result of shared control would be similar as the one demonstrated by the expert driver. As a result, although no path planner was used in this experiment, the whole expert driver plus the wheelchair was considered the same as a special of type of robot—a shared-controlled wheelchair.

3.2.2 Participants

28 participants (15 females, 13 males) from different age groups (\(\hbox {M}=33\), \(\hbox {SD}=8.8\) years old) were recruited from our university and a participant pool. None had a mobility, sight or hearing impairment. The participants were given a copy of the information sheet and time to sign the consent form prior to the experiment. To prevent any potential bias, the actual purpose of the experiment was not revealed to them during the introduction.

3.2.3 Apparatus

The experiment took place in an indoor pedestrian accessibility lab. The lab environment allowed us to have significant control over multiple variables. The place consists of a platform which is constructed from \(6*10\) movable modules, with the size of each module being \(1.2~\hbox {m}*1.2~\hbox {m}\). We used \(6~\hbox {m}*12~\hbox {m}\) of the platform and constructed our gate using movable panels.

3.2.4 Task

In contrast to other work in the literature which studied interaction between one robot and one pedestrian [38] or a small groups of people [21], we designed a robot-in-crowd gate-crossing task. The gate-crossing task has been widely used in analysing crowds dynamics [35], in situations such as entering train stations and evacuation. During the experiment, each participant was asked to wear a coloured hat for detection and tracking purposes (see Fig. 2). In each run, 28 people and the robot were randomly given an initial starting position number which was represented as one circle in Fig. 3). In addition, we made sure that the starting positions assigned to the people in proximity to the “robot” were not the same in each run, so as to reduce the learning effect and potential bias caused by individual behaviour. Furthermore, for more valid comparison, we let participants keep their starting position across different scenarios. For example, the starting position in (S2, run1) is different from (S2, run2) but is the same as (S3, run1). All pedestrians were instructed to walk together from one side of the platform to the other side by crossing through a 2.2 m wide gate (See Fig. 3).

A vocal command was used to inform pedestrians of the start of each trial and the completion was achieved when all the pedestrians crossed a destination line at the end of the platform. After each scenario, all participants were asked to fill in a short survey which was designed for them to reflect on their behaviour.

Fig. 2
figure 2

Overview of the experiment from side and top camera

Full size image
Fig. 3
figure 3

Overview of the experimental plan. Each blue square with the red number represents one module on the platform. Black segments represent the movable panels which were used to form a gate. Gray circles represent the pedestrians and the blue circle stands for the robot. Positions were randomly assigned. (Color figure online)

Full size image

3.2.5 Experimental Scenarios

We designed 4 testing scenarios with 3 independent variables: robot occurrence, robot type and robot speed (see Table 1). Due to the inherent difference in speed-capabilities of the robots, we set the low speed to approximately 0.5 m/s for the wheelchair and Pepper, while the high speed for the wheelchair is about 1 m/s which is comparable to normal pedestrian walking speed. Each scenario was repeated 5 times.

Table 1 Four experimental scenarios
Full size table

3.2.6 Recorded Data

The experiment was recorded by an overhead fish-eye IP video camera (Axis M3037) at 12.5 fps. An additional IP video camera was set up from the side with the aim to observe pedestrian behaviours qualitatively. In order to calibrate the fisheye camera, we used the well-known chessboard method, collecting 22 various shots while changing the location and orientation of the chessboard. We applied the Scaramuzza fisheye model [33] which introduces mapping coefficients between distorted and corrected images. We used the Matlab camera calibrator toolbox for calculating the coefficients and then undistorting the videos (see Fig. 4a) In the next step we computed the homography matrix between the undistorted image plane and the ground plane, by mapping pixel coordinates of 4 of the corners of the platform to their world coordinates. This process compensated the rotation of the camera with respect to the ground plane and gave top-view images which are suitable for object tracking (see Fig. 4b).

We then used a special-purpose software called PeTrack to extract the pedestrian trajectories. The issues of ID switches and tracker drift were solved using semi-automatic processes. Finally, we used separate linear functions to map the head locations to the ground. For the humans we assumed an average height of 170 cm. For Pepper and the wheelchair we used heights of 120 cm and 140 cm, respectively. In order to filter out high frequency jerk from the trajectories we applied a Kalman Smoother with a const-acceleration model. We configured the model with \(\hbox {dt}=1/12.5\), \(\hbox {q}=10\) (process noise constant) and \(\hbox {r}=1\) (observation noise constant). In total, 20(4 scenarios * 5 runs each scenario) trials were performed with valid interactions between the robot and the crowds. 546 pedestrian trajectories and robot trajectories were detected and extracted using the open-source software PeTrack [4].

Fig. 4
figure 4

Camera calibration: a original image from the fisheye camera, b undistorted image after applying the homographic matrix

Full size image

4 Analysis

To better evaluate the effect of robot in crowd dynamics in both global and local level, we measured macroscopic and microscopic features, and reported the result quantitatively. Macroscopic features describe high-level crowd characteristics while microscopic features take individuals’ properties into account. In order to quantify the effect of the presence of a robot and its effect on pedestrian dynamics, we analyzed the extracted trajectories both macroscopically and microscopically based on some common metrics that have been used in previous works [21, 28, 38]. In general, we categorized the applied metrics to measure ‘trajectory regularity’ and ‘interaction complexity’.

4.1 Preprocessing

Before applying metrics to the trajectories, we defined the notion of region-of-interests (ROI) as 2 m before the gate and 0.5 m after the gate, we observed this was the region where most interaction happens. In addition, for more valid comparison, we splited all the trajectories extracted from the ROI into sub-trajectories with each having a fixed time length of 10 frames \((=0.8~\hbox {s})\).

4.2 Trajectory Regularity

We evaluated the geometrical and physical properties of the sub-trajectories in order to reflect their irregularities and deviations from simple linear trajectory models. For this purpose, we used three metrics, average speed, average acceleration and path efficiency. Path efficiency is normally calculated as the ratio of distance between two terminals (\(\vec {x}_{end}\) and \(\vec {x}_{start}\)) of the trajectory segment over the actual length of the segment [21]. However, in our experiment, the existence of the gate inherently affects pedestrians’ path efficiency. This issue is addressed by dividing the sub-trajectories which cross the gate into ’before gate’ and ’after gate’, with the sub-goal (\(\vec {x}_{sub}\)) being introduced as the point on the sub-trajectory at the gate. As a result, path efficiency \(\eta \) for a sub-trajectory \(\mathbf{X}^k\) is defined as:

$$\begin{aligned} \eta (\mathbf{X}^k) = \frac{\Vert \vec {x}^k_{sub}-\vec {x}^k_{start}\Vert + \Vert \vec {x}^k_{end}-\vec {x}^k_{sub}\Vert }{\sum _t \Vert \vec {x}^k_{t+1}-\vec {x}^k_{t}\Vert } \end{aligned}$$
(1)

where t ranges from start to end

4.3 Interaction Complexity

While the above metrics indicate the sub-trajectory regularity and motion complexity of each pedestrian, they do not imply the interaction between pedestrian–pedestrian and pedestrian-robot. Consequently, we applied another three metrics to evaluate the interaction complexity in each scene. They are evacuation time, local density, and pass order inversion.

Evacuation time has been used as to assess crowd dynamics in emergency situations. Here we defined it as the time elapsed from when the first pedestrian passes the gate to the time when the last pedestrian passes the gate. This quantity is further normalized by the number of pedestrian’s.

In terms of local density, Helbing et al. [12] proposed a formula based on the idea that each person occupies a fixed radius of area. In this paper, we adopted the notion proposed by Plaue et al. [28] where a nearest neighbor Gaussian kernel estimator is used, which allows the difference of each pedestrians occupied area to be taken into account. For a point \(x_t\), the local density \(p(x_t)\) is defined as,

$$\begin{aligned} p(\vec {x}_{t}) = \frac{1}{2\pi }\sum _{i=1}^{K_t}\frac{1}{(\lambda {d_t^i)}^2}\exp {\left( -\frac{\Vert {\vec {x}^i_{t}-\vec {x}_{t}}\Vert ^2}{2({\lambda {d_t^i)}^2}}\right) } \end{aligned}$$
(2)

where t is the time, \(K_t\) represents the total number of pedestrians at time t and \(d_t^i = min_{j\ne i}\Vert \vec {x}^j_{t}-\vec {x}^i_t\Vert \) is the Euclidean distance from agent i to its nearest neighbour and \(\lambda >0\) is a smoothing parameter. In this paper, we use the averaged local density for each sub-trajectory as an indicator. In daily life, a human always adapts to others when there is a risk of collision. To analyse whether such adaption exists in the human–robot navigation scenario and how it differs across different types of robot, we used a signed definition of minimum predicted distance (SMPD) to analyze the adaption behaviour [38]. As detailed in [24], minimal predicted distance (MPD) estimated the risk of future collision by calculating the distance to the closest approach (DCA) between the robot and the pedestrian at each time step, assuming they keep a constant velocity.

$$\begin{aligned}&X_{{\textit{pred}}}(t,u)= X(t)+(u-t)V(t)\end{aligned}$$
(3)
$$\begin{aligned}&{\textit{MPD}}(t)=\arg \min _{u}\Vert X_{{\textit{pred}},h}(t,u)-X_{{\textit{pred}},r}(t,u)\Vert \end{aligned}$$
(4)

where u is a future time parameter, \(X_{{\textit{pred}},h}(t,u)\) and \(X_{{\textit{pred}},r}(t,u)\) are future positions of the human and the robot. By adding a sign to this metric, we can estimate whether the robot or the pedestrian is predicted to be ahead. In our study, we define \(t_{{\textit{enter}}}\) as when the robot entered the ROI and \(t_{{\textit{pass}}}\) as when either the pedestrian or the robot passed the gate. Consequently, we computed \(\hbox {SMPD}(t_{{\textit{enter}}})\) and \(\hbox {SMPD}(t_{{\textit{pass}}})\) for each pedestrian-robot pair.

Considering the human perception capability, we only considered pedestrians who are behind the robot at \(t_{{\textit{enter}}}\). We define positive SMPD if the robot should pass first and negative SMPD otherwise. As a result, a change of sign of SMPD means that the future crossing order between the robot and the participant is switched, and thus implies the adaption in the pass order. In general, we define four pass order groups based on the sign of SMPD at \(t_{{\textit{enter}}}\) and \(t_{{\textit{pass}}}\), namely: PosPos, NegNeg, PosNeg and NegPos. We classify ‘PosPos’ and ‘NegNeg’ as pedestrians who keep their pass order, ‘PosNeg’ represents pedestrians who overtake the robot while ‘NegPos’ implies pedestrians give the way to the robot.

4.4 Statistics

In order to assess the effect generated by the robot and whether it varies with robot type, detailed comparisons were made within and across scenarios. To guarantee valid data comparison, normality was assessed with the Kolmogorov–Smirnov test. It was indicated by the test result that statistics have a non-Gaussian distribution for some metrics. As a result, we used Wilcoxon ranked sum tests to determine differences and significant level. All effects were reported at \(p<0.05\). All the figures indicate the significant level with ‘*’, where ‘*’ stands for \(p<0.05\).

5 Results

5.1 Quantitative Result

5.1.1 Average Speed and Average Acceleration

Overall, our results indicate a significant difference between the human-only case (S1) and human–robot case (S2, S3, S4) in terms of average pedestrian speed and acceleration.

In order to further investigate the local effects, we grouped all pedestrian trajectories into two categories based on their spatial relationship to the robot. According to Hall’s personal space theorem [10], we evaluated each pedestrian’s distance to the robot at each time stamp. For a pedestrian sub-trajectory, only those with the median of Euclidean distance less than the robot’s close social space (\(<2.1~\hbox {m}\)) were considered as in proximity with the robot. This gives us 819 and 1067 pedestrian sub-trajectory segments near and far away from the robot.

Figure 5 depicts the average pedestrian speed categorized by its proximity to the robot. The average value obtained in S1 is used as a baseline for comparison. It can be observed that in most scenarios (except S2_farSpeed), the average speed for pedestrians near and far away from the robot are significantly different from the baseline (S1, no robot case). Additionally, within the same scenario, pedestrians’ average speed differs greatly based on their proximity to the wheelchair (S2, S3) or Pepper (S4). In terms of average pedestrian acceleration, similar difference exist between no robot and robot cases, while the local effect is less obvious.

5.1.2 Path Efficiency

In general, high path efficiency (\(>85\%\)) in all scenarios was observed. While comparing the path efficiency of sub-trajectories categorized by its proximity to the robot within the same scenario, pedestrians who are in close proximity with the robot in S3 and S4 have slightly lower path efficiency compared with those who are distinct from the robot.

Fig. 5
figure 5

Average pedestrian speed categorized by its proximity to the robot. ‘N’ and ‘F’ stand for the speed for pedestrians near the robot and far away from the robot. Pedestrian speeds in robot scenarios are significantly different from those in the no-robot scenario (S1). Within the same scenario, pedestrians that are in close proximity with the robot have lower speed compared to pedestrians far away from the robot

Full size image

5.1.3 Evacuation Time

Figure 6 shows the evacuation time per person for all four scenes. By comparing S1 (no robot case) with S2-S4 (robot case), we can observed that crowd evacuation time significantly increased (\(p<0.05\)) when a wheelchair or Pepper is involved. In addition, a significant difference is observed between wheelchair case and Pepper case regardless of the robot speed.

Fig. 6
figure 6

Evacuation time per pedestrian. A significant increase can be observed when a robot is added to the crowd. Pedestrians need longer evacuation time walking with Pepper compared to walking with a smart wheelchair

Full size image

5.1.4 Local Density

Average local density for each pedestrian or the robot sub-trajectory is illustrated in Fig. 7. Significant difference can be observed between no robot (S1) and robot (S2–S4) cases, as well as between the wheelchair (S2, S3) and Pepper (S4). When the wheelchair was involved, the wheelchair speed also showed influence on average pedestrian local density. In terms of the robot, local density around Pepper is significantly higher than that around the wheelchair.

5.1.5 Pass Order

Figure 8 provides a summary plot for the human–robot pass order in all scenarios by analysing SMPD. We can observe that about 50% of pedestrians kept their pass order when they walk with a wheelchair or Pepper. Among the pedestrians who adapted their behaviour, less than 20% of them overtook the wheelchair while this number is over 80% in the case of Pepper. On the contrary, less than 20% of pedestrians gave way to the Pepper while over 80% of them let the wheelchair pass first regardless of the wheelchair speed setting. This behaviour difference is visualized in Fig. 9.

Fig. 7
figure 7

Local density around the pedestrians (pH) and the robot (pR). When a robot is added to the crowd, the local density around pedestrians decreases. Higher local density can be seen around the Pepper robot compared with the smart wheelchair

Full size image
Fig. 8
figure 8

Pass order inversion. In all scenarios, over 50% pedestrians kept their passing order. Among those who adapted their passing order for Pepper, over 80% overtook while less than 20% gave the way to the robot. The situation is the opposite in the wheelchair case

Full size image
Fig. 9
figure 9

Sample robot and pedestrian trajectories. a The pedestrian (blue) gave way to the wheelchair (red). Right: The pedestrian (blue) overtake Pepper (red) (color figure online)

Full size image

5.2 Qualitative Result

After each scenario, participants were asked to fill in a short survey. To begin with, we asked participants if they noticed the wheelchair or the humanoid robot during the last scenario. Among those participants who acknowledged the appearance of the ’robot’, most responded they looked at the robot before passing the gate(44%-wheelchair, 49%-Pepper) and while passing the gate (24%-wheelchair, 17%-Pepper). This result complies with our definition of ’region of the interest (gate area: 2m before the gate and 0.5m after the gate)’, with the assumption that most human–robot interaction happens in that region. We further asked pedestrians to reflect on their behaviour.

In Fig. 10, a noticeable difference in pedestrian walking behaviour can be observed between the wheelchair case and the humanoid robot case. In detail, 68% of the pedestrians admitted that they have adapted their behaviour by giving more space (50%) or deviated from their planned route (18%) when they noticed the wheelchair. In terms of the humanoid robot case, although more than a half of the pedestrians (52%) respond a change in behaviour, it was less obvious than the wheelchair case.

Figure 11 depicts the result of pedestrians’ perception on their collision avoidance behaviour. 73% pedestrians think they avoided the wheelchair in the same way as they avoided a person while more than half (54%) of pedestrians admit they avoided the humanoid robot differently as they would do to a person. This difference can be explained by the fact that pedestrians perception of the robotic wheelchair is affected by the presence of a human driver and their prior mental model towards a wheelchair, which in turn affected their behaviour.

Fig. 10
figure 10

Q1: If you have noticed the humanoid robot/wheelchair and the person on it, did that change your behaviour comparing to that in a scenario without the humanoid robot/wheelchair? Change in avoidance behaviour. In both cases, more than 50% of participants recognized they adapted their walking behavior to avoid the robot. People gave more space to the wheelchair compared to Pepper

Full size image

6 Discussion

We studied how crowd dynamics in terms of trajectory regularity and interaction complexity are affected by the presence of a wheelchair or a humanoid robot Pepper. Overall, we find both H1 and H2 are confirmed.

In terms of H1, our results indicated that the occurrence of a robot indeed affect the crowd dynamics. By looking at the quantitative result, the global influence on trajectory regularity is mainly reflected on lower average walking speed and average acceleration, while the average path efficiency was less affected. This results is consistent with the one from previous studies [6] where the author showed a moving robot slows down a uni-directional flow. In addition, significant disparity has been observed across the scenarios in term of the interaction complexity. Temporally, the evacuation time per person has increased significantly when a wheelchair or Pepper is involved. Spatially, pedestrians’ averaged local density in S1 is higher than that in S2, S3 and S4. Furthermore, pedestrian’s local density decreases when the robot local density increases. These findings suggest pedestrians adapted their trajectory due to the occurrence of a robot, which was further supported by the qualitative results where pedestrians recognized their change in perception as well as walking behaviour. In a local level, when we categorized the sub-trajectories based on the spatial relationship to the robot, we have observed significant differences between the ‘near the robot’ and ‘far away from the robot’ group in terms of average speed and path efficiency. Pedestrians who are near the robot tend to move slower with lower path efficiency compared to those far away from the robot. No significant evidence has been observed on how the result is affected by the robot speed though.

In terms of H2, we observed the effect of robot type on crowd dynamics. The crowds also tend to spend more time in evacuation with the Pepper (S4) than with the wheelchair (S2, S3). When looking at how individuals adapt their behaviour, qualitative results indicated that most pedestrians adapt by overtaking Pepper, while giving the way to the wheelchair. Compared to previous studies where pedestrians gave way to a passively moving robot [38], our results imply an interesting change of such behaviour due to the robot type. Although further investigation on pedestrian factors would help us better understand the behaviour, this difference can be explained by the idea that pedestrians’ perception of the robotic wheelchair is affected by the presence of a human driver and their prior mental model towards a wheelchair, especially when a person appears to be mobility impaired. It makes them tend to follow ‘social rules’ and result in the change of their behaviour, while such a rule is not observed when the overtaking target is replaced by a humanoid robot. Spatially, robot type also has effect on pedestrians’ local density. While looking at the robots, the wheelchair was surrounded by fewer people compared to Pepper. This finding is further supported by our survey result as shown in Fig. 11, where more pedestrians were aware they ’gave more space’ to the wheelchair than the humanoid robot. These results suggest pedestrians’ local adaptive behaviour and the overall crowd interaction are affected by the occurrence of a robot, and this varies across the two tested robot platforms.

Our experiment was conducted with two specific robots, but we believe these results could be generalized to other robots. In this study, our robots are good representatives of mobile robots, which covers a great range of differences in terms of appearance, size, and dynamic constraints. In addition, by introducing the distinct factor of “human driver”, our result could also inform the design of navigation strategy for shared-controlled robots.

Fig. 11
figure 11

Q2: Did you treat avoiding the humanoid robot or wheelchair any differently to how you would have avoided a person? Difference in avoidance behaviour in human only and human–robot environment. Over 70% of participants think they avoided the wheelchair in the same way they would do to a person, while this number is only about 40% in the Pepper case

Full size image

Therefore, we draw the following recommendations from our study:

  1. 1.

    It would be important to consider the effect a robot exerts on the surrounding pedestrians while planning for its next motion—which means prediction and planning should be considered together to capture the natural interaction in complex environments.

  2. 2.

    In order to achieve social robot navigation using data driven methods, pure pedestrian data recorded from an environment without robots may be insufficient, and it is better to obtain the pedestrian data where the specific robot is involved, as both the presence and the type of the robot affects pedestrian’s trajectory regularity as well as crowd interaction complexity. By doing so, more natural social navigation could potentially be achieved, though of course the generalization would need to be further explored. For example, Kim and Pineau (2016) approached the social navigation problem from inverse reinforcement learning, and learnt a cost function from data that was collected when a human demonstrator drove a smart wheelchair in crowds [17]. It allows the wheelchair to navigate in a socially adaptive way, while such a cost function may not generalize well to other types of robot, as the crowd dynamics may be affected by the robot itself (and the driver that comes along with the robot). On the other hand, if the difference in pedestrian dynamics can be modelled in a crowd simulator, more realistic interactions between pedestrians-crowds and the specific type of the robot could be simulated, thus providing us a powerful tool to validate the developed navigation algorithm.

  3. 3.

    The social navigation strategy should potentially be developed differently for a shared-controlled robot and a fully autonomous humanoid robot. In this study, we did not explore the influence of the existence of a human driver on the pedestrian’s perception and behaviour by separating “the driver” and “the robotic wheelchair”, but rather considered the natural use case where the “wheelchair + driver” system is seen as a whole. In this experiment, the user did not actively interact with surrounding pedestrians but merely focused on the navigation task. In the future, it would be interesting to explore the user-pedestrian interaction, such as a wheelchair user approaching his/her friend while navigating through a crowd.

Although the results obtained from this experiment may be limited to certain design factors, we believe this work has set a very important first step to help us better understand crowd-robot interaction in navigation.

7 Conclusion

In this paper, we presented the first crowd-robot crossing experiment with collected trajectory dataset in the presence of two robot representatives: a smart wheelchair and a Pepper humanoid robot. Quantitative analysis implies the presence of the wheelchair and the Pepper affect crowd dynamics both locally and globally. Besides, the influence varies across the robot type. In general, the effect is reflected in the individual trajectory regularity and the interaction complexity. Qualitative results further supported the idea that pedestrians tend to behave more conservatively around the wheelchair compared to the Pepper, potentially due to the perception of a human driver. These results suggest the influence of the robot on crowds should be taken into consideration when designing the pedestrian model in simulation and navigation strategy for different kinds of robot.

In the future, this work could be extended to explore the effect of different types of robot on pedestrian dynamics in bi-directional or even more complex scenarios. In addition, the human factors such as age, gender, familiarity with the robot which would potentially affect crowd dynamics in social navigation could be further investigated.