Quantifying the ambient population using hourly population footfall data and an agent-based model of daily mobility

2.1 Otley and the census data

The study area is centred on the market town Otley in the Leeds Metropolitan District, West Yorkshire, UK (Fig. 2). Otley was chosen for two reasons: firstly it has a relatively self-contained commuting pattern (the majority of the people who work in Otley do not travel far); and secondly because there is an excellent source of high-resolution spatio-temporal data from a number of Wi-Fi sensors that can be used to quantify the real footfall patterns in the town. The aim of the model is to simulate the behaviours of commuters and retired people in Otley. Therefore a study area must be drawn that includes people who live and work (if they work) either in Otley or in the surrounding area. Commuting data were obtained from the 2011 UK Census [28] and contain flows of people from residential to employment locations at the level of output areas (OAs), the smallest statistical unit in the UK. The data were used to identify all of the people who commuted into or within Otley, and the study area was then constructed to include the residential locations of the majority of this group. A very small number of people commuted from outside of this area (some from hundreds of miles away) but these were not included. The study area includes Otley and a few surrounding towns (Horsforth, Yeadon, Guiseley, Menston, Burley-in-Wharfedale, Pool in Wharfedale and Bramhope). The census data contains the number of people who commute between all combinations of OAs in the study area. As such, there are 11,807 commuting agents in 773 OAs. The number of retired people per OA was also extracted from the 2011 census, leading to 32,535 retired agents.

Open image in new window

To accompany the census data, footfall data from eight Wi-Fi sensors in the centre of Otley were provided for by the commercial organisation ‘Noggin’ and the Otley Business Improvement District (a public organisation whose role is to further the economic potential of the town). The sensors capture the signals of Wi-Fi enabled mobile devices of pedestrians. Sensors next to main roads and especially crossroads can also detect car passengers during periods of congestion, when their mobile devices stay near the sensor for a sufficient amount of time. The dataset contains hourly totals for each location which were collected between August 2015 and July 2017.

2.2 The UK time use survey

In order to model the population to the greatest degree of accuracy, the most recent UK Time Use Survey (UKTUS) was used to better understand the activities of commuters and retired people. The UKTUS 2014–2015 [27, 29] contains daily activities of 8278 respondents in 4238 households, after eliminating incomplete records. It is a follow-up of an earlier large-scale survey conducted in 2000–2001, and it is the British part of the Multinational Time Use Survey (MTUS) [30]. The respondents fill in diaries per 10 min interval during two days. For every interval, they specify which main activity they were doing at which type of location (at home, at work, in a shop, on the road, etc.). Additionally, they can mention secondary activities, other people who were with them, a level of enjoyment of the activity, and whether they were using a computer, tablet or smartphone. The survey contains the ceremonial county in which the households live. Due to the limited number of respondents in West Yorkshire (only 268), data for the entire UK were used. There appears to be a bias with new activities starting on the (half) hour rather than on other 10 min intervals. This is probably because the survey is retrospective; respondents do not fill in their diaries on the spot. This issue is not problematic here since as such relatively small biases will not influence the overall model results.

An understanding of the activities that drive peoples’ daily travel behaviour, as well as the approximate start times and durations, is essential for identifying the most important activities to include in the agent-based model. The analysis focused on finding the proportion of respondents (either commuters or retired people) doing away-from-home activities on weekdays, and specifically workdays for commuters (i.e. excluding weekends and all days on which the commuters do not work). Subsequently, the start and end time frequency and average duration of these away-from-home activities were extracted. Many commuters start their day at the office between 07:00 and 10:00, and remain there on average for 8 h (Fig. 3). Part-time working usually takes place in the morning as only very few people start working after lunch. With regard to all other activities of both groups, shopping, sports and social activities are the most popular. At-home activities (household and family care, mass media, hobbies at home) were excluded since no travelling is involved.

Open image in new window

Around 27% of all commuters go shopping for food on an average workday (43% on Friday). Shopping is more popular during lunchtime and the afternoon rush hour (Fig. 4), and the duration is typically short with a mean time of 41 min but a median of only 30 min. More retired people go shopping for food on weekdays (49%), and spend on average 69 min in the supermarket (with a median of 50 min). Mornings are slightly preferred over afternoons (Fig. 4). Leisure shopping is done by 16.6% of retired people per day, and can best be handled as a separate activity.

Open image in new window

The next most popular activity is sports, with 16.2% of commuters and 17.8% of retired people participating. A smaller, but not negligible, proportion of commuters go for lunch (4.5%) or do a social food or drink evening activity (7.9%). Similar percentages can be found for retired people for lunch not at home (7.9%) and social evening activities (7.9% as well). The analysis of this survey indicates that to model both groups, the most important activities, other than being at home and working, are shopping, sports, having lunch, and going out for food and drinks. The following section discusses the model itself and outlines how the results of the time use survey were used to parameterise the agents’ behaviour such that their activities corresponded with the appropriate times as identified in the survey (e.g. as in Fig. 4 for shopping).

3.1 Agent behaviour

The Simulating Urban Flows (surf) model represents the daily routines and travel patterns of individual 9–5 commuter agents and retired agents on weekdays. Analysis of the time use survey highlighted seven different activities that need to be modelled in the first instance (later iterations of the model can continue to add activities as required): being at home, working in an office (commuters only), shopping for food in supermarkets, leisure shopping (retired only), having lunch (if not at home or at work), going out in the evening for food and/or drinks, and doing sports. The behaviour of the agents has been structured to respond to these activities.

Each activity has an intensity – a motivation to do this activity – and the activity with the highest intensity at any given time is the one that drives an agent’s behaviour (although there is also a minimal duration for the present activity that prevents agents rapidly changing activities). The total intensity is the sum of a background intensity and a time intensity, which differ by each activity.

The background intensity reflects the recurrence pattern. It gradually increases when the agent is not performing the activity, and decreases during the activity. The rate of increments and decrements are different for each activity and can be configured. After calibration, the rates should be as such that the typical recurrence and duration of activities are correct, as extracted from the UKTUS (i.e. the agents spend an appropriate amount of time, as per the time use survey, on each activity). For example, the survey suggests that, on average, a commuter goes to the supermarket about every 3 days and spends about 30 min shopping. The rate that the background intensity increases can be configured so that the right proportion of agents go shopping on each day, and the rate that it decreases while an agent is shopping can be configured so that agents spend, on average, 30 min on that activity. Of course this will vary on a daily basis as other activities might be more important on a particular day, such that the agent might not shop at all for 3 days, but then spend more time shopping on the next day. The rate of decrease in background intensity is always more rapid than the increases, such that agents can always perform an activity in order to reduce its intensity back to 0. This is especially true for activities that typically have short time periods (e.g. less than an hour of shopping can reduce a shopping intensity that has been building up over several days).

The time intensity is also activity-specific, reflecting common times when agents do that activity. For example, the time intensity for being at home becomes stronger over night. These times are also based on the UKTUS. Stochasticity is included in both intensity types to create heterogeneous agents. The result of the manual calibration of intensities should be that average activity statistics per hour of the day should be very similar to those observed in the UKTUS. As can be seen in Fig. 4 for shopping for food, the pattern of the model results is similar to reality, though with less variability or outliers. The main reason for the more limited variability is that time intensity was set to zero outside the most important time interval, which vastly improved the pattern of that time interval.

A diagram of a simplified example of an agent only being at home, working and shopping with the changing activities and total intensities during 5 days is given in Fig. 5. The agent begins at home and vertical lines indicate when new activities start. The total intensity can still increase after the start of an activity when the time intensity is going up (e.g. working in the early morning), but the background intensity will decrease nevertheless. This agent goes to work in the morning and returns home in the afternoon at varying times. On days 1 and 4, the intensity for shopping is never sufficiently high, meaning that the agent does not do this activity on those days. On days 2 and 3, the agent goes shopping after work, while they go during their lunch break on day 5. This shows that the intensities indeed make it possible to have flexible agent behaviour.

Open image in new window

For the spatial allocation of flexible activities, it is assumed that it is more likely that agents will do these nearby the location of their previous activity. Data about the functions of buildings are extracted from OpenStreetMap, and activities are allocated to random buildings with the correct function within a search radius of the previous activity. This search radius is iteratively doubled if necessary.

Details on agent behaviour, scheduling, initialisation, and the model environment can be found in the ODD+D description in the Online appendix.

3.2 Implementation

The model is coded in Scala, but makes use of the Java-based multi-agent simulation library MASON [31] and its geographical toolkit GeoMASON [32]. The agents travel through the study area at a constant speed, approximately equal to walking speed, so the model currently does not account for different road types or congestion. The shortest path between activities is computed with the A* algorithm of GeoMASON. The time step is five minutes and the model is typically run for 14 days. Although the model reaches equilibrium after approximately the first two days, only the results of the final day of simulation are used. The models is also executed 50 times to account for its probabilistic nature and the average results across all model runs are reported – i.e. the graphs that report the number of agents who pass each sensor in a particular hour actually show the mean number of agents across all model runs. As the following section will evidence, there is little variability in the separate model runs.

5.1 Results of the commuting model

Figure 6 illustrates the total footfall in the model and compares it to that from the real sensor data. For the simulated data (top), the chart shows the mean footfall across all sensors at each hour of the simulation in the last 24-h period of the final day (to ensure equilibrium has been reached), averaged across all of the 50 separate model runs. The results are depicted with a smooth line generated using a generalised additive model (GAM) [37] as implemented in the mgcy R package. Although 99% boot strap confidence intervals have been included in the figure, these are so narrow as to be nearly impossible to discern, which is encouraging as it demonstrates that each model run produces similar aggregate results. Note that the apparent increase in simulated footfall (top) after 20:00 is an artefact of the GAM line smoothing algorithm. For the real sensor data (bottom), the graph shows the mean footfall across all sensors on Tuesdays, Wednesdays and Thursdays (to capture ‘normal weekday’ behaviour). Note that for both charts Z-scores are used to make the scales visually comparable.

With respect to the simulated activity patterns, there are two activity peaks of particular note. The first occurs in the morning; it is caused by agents travelling from their homes to their workplaces (i.e. the morning rush hour). There follows a reduction in footfall as the agents are at work. Later, there is a gradual increase in activity as some agents go to lunch or shopping, leading to the most substantial peak in the afternoon when they commute home or start other activities after work. These patterns are entirely expected given the way that the model has been parameterised based on the evidence available about daytime activities in the time use survey. In effect, these patterns are a representation of the expected behaviour patterns of commuters who travel in to work in Otley. The differences between the model outputs and real footfall data (the error) can be partially explained by the contributions that other groups make to the overall footfall. Obviously, there are other possible sources of error, e.g.: (1) the spatial and temporal uncertainty of the data used, (2) changing behaviour of the population between the census (2011) and the validation period (2015–2017), or (3) the spatial allocation and the routing without congestion in the model. After discussing the results with local stakeholders, the model was extended to include a second group of agents, who could be expected to constitute a large part of the error: those representing retired people.

5.2 Results of the commuting & retired model

Figure 7 illustrates the results of the commuting model, the commuting & retired model, and the real sensor data. Again Z-scores are used to scale the results to comparable ranges. The results of the updated model are much more similar to those of the real footfall data. The substantial drop in footfall that was being simulated at around 10:00 disappears as retired agents begin their activities. Whilst this is not validation, the success of this incremental addition to the model gives some more confidence that the results can reveal some insight into the dynamics of the case study area. This is information that, before now, was not available other than in entirely aggregate form. Interestingly, the most substantial difference between the model and real data (error) now occurs around 15:00. It is very likely that this error is partially caused by the substantial body of children who leave the various schools at 15:00; they (and their carers for younger children) are therefore a prime target for inclusion in further iterations of the work.

Open image in new window

5.3 Results by individual sensor

Finally, Fig. 8 illustrates the footfall broken down by the individual sensor (see Fig. 9 for a map of the locations). The sensors were installed in the first place to give an average picture of footfall across the whole town, as has been done so until this point. But it is nevertheless interesting to review some of the possible spatial differences across sensors. Sensors 1 and 14 are both placed near to the arterial routes into the town, whereas the others are all within the main town centre. The arterial sensors are more likely to be on routes used by commuters, and less likely to catch the leisure activities of people in the town during the day. It is encouraging to find rush hour peaks in both the sensor data and the model results for locations 1 and 14, even though the exact times are slightly different, and the model has lunch-time peaks too.

Open image in new window

Open image in new window

6.1 Preliminary findings

The activities of some groups of people are easier to estimate than for others. For example, from the 2011 UK census there is an abundance of data about where people live and, for employed people, where they go to work. By coupling these data with estimates of the timings of their activities – in this case from a large time use survey – it is possible to estimate the spatio-temporal activities of a reasonably large group of people. Here we call them commuters. The surf model, in its initial iteration, is a model of the typical routines and displacements of those 9–5 commuters on typical workdays. Having used all available evidence to estimate the activities of commuters, it becomes possible to create simulated estimates of the overall amounts of footfall that this group contributes to. There were, of course, clear discrepancies between the simulation of a single demographic group, and the real data that included a much broader range of individuals. Therefore, following discussions with local stakeholders another substantial group who are present in the case study area was added to the model; that of retired people. The inclusion of this new agent type substantially reduced the simulation error and paved the way to continuing this iterative processes, gradually increasing the number and diversity of groups who are modelled.

These results provide a potential avenue to better understand how urban spaces are being used in the absence of information about the behaviour of individuals in the study area. Although footfall data are available, these do not reveal information about the individual people who contribute to the aggregate footfall. Indeed, we would argue that it would be unethical to attempt to identify, and potentially begin to track, those individuals in the first place. In effect, therefore, the model provides a means to disaggregate the available footfall data by the demographics of the individuals who contribute to the aggregate counts. Agent-based modelling is a methodology that is ideally suited to this task as it offers a means of combining high resolution spatial data (in this case the census) with high resolution behavioural data (the time use survey). The model is used to marry these otherwise disparate datasets to create a more robust picture of daily urban dynamics.

There are, of course, some differences between the simulated and real data that remain. Fig. 7 illustrated these. The most notable difference is the reduction in footfall at approximately 15:00. This begs the question: who are the people who make up this extra footfall? This time corresponds closely with the time that children, both young – who will be collected by carers – and older – who are mostly unsupervised – will leave their schools. It is therefore extremely likely that this group (children and their carers) are the cause of this particular discrepancy. Interestingly, the inclusion of these groups in future iterations will not only require the addition of a new group (schoolchildren), but also the diversification of the existing groups as many commuters and retired people will be responsible for delivering children to and from school. There is evidence for this both in the time use survey and anecdotally following the discussions with stakeholders.

Regarding the location of the activities, it is also worth noting that some sensors, notably 1 and 14, suggest morning and afternoon peaks that are more indicative of commuting behaviour than others. This points to the possibility of identifying the most likely locations, as well as times, at which the different groups might be present. Fig. 8 focussed specifically on the locations of each of the sensors. This, again, has the potential to provide useful information about these non-commuting groups. If, for example, policy makers are concerned about the impacts of pollution on the elderly or young children, a method such as this could be used to provide evidence about the times and locations that the group will be most active. This information is otherwise extremely hard to gather using traditional sources such as surveys and censuses, and even more so using big data sources that are often biased towards certain age groups, potentially excluding the very young or very old. We see this as the main contribution of the paper; a means of simulating the daily spatio-temporal behaviour of different population groups, particularly for those whose activities are otherwise very difficult to interrogate.

6.2 Caveats

There are a number of caveats that are important to note. Firstly, there are questions regarding the real-world footfall data that have yet to be resolved. For example, it is reasonable to assume that most smartphones will be counted by the sensors, so to estimate age or gender bias we can look for data on smartphone saturation within the population at large, at least if there is no specific bias in Wi-Fi usage between different groups. It is harder to estimate factors such as whether small numbers of people who, as an artefact of their activities, will trigger a sensor multiple times in an hour. Although there are some mechanisms that can be used to guard against this, e.g. by temporarily recording the unique identifier (MAC address) of the phone and ignoring it if it is counted more than once per hour, on modern smartphones the identifier changes regularly so this is not possible. Nevertheless, the footfall data are likely to represent a sufficiently accurate proxy for day time activity levels in the town. A few recent studies have shown that a bias in phone usage between demographic groups does not have a major effect on general travel patterns [38, 39].

Another potential problem is that although the footfall data are relatively recent, covering the period October 2015 to June 2017, the census data are from 2011. It is likely that some people will have moved or changed jobs in this time, but estimating the current commuting patterns in the area is beyond the scope of this work.

The intensity-based decision framework seems to work better when agents have at least one not-at-home regular activity with a fixed location that anchors their behaviour. If not, which is the case for retired people, too many agents do all their activities immediately in the morning. This does not reflect the patterns observed in the time use survey. The model seems to be more suited to dealing with commuters who have less flexibility and time (because they spend a large part of their day at work) to start shopping and doing leisure activities.

This model is somewhat unusual in comparison to other agent-based models in that it has not been empirically validated by comparing its outputs to data that it has not been calibrated on. Section 4 discussed this problem at length. Empirical (i.e. “fit to data” [34]) validation has not been undertaken for two reasons. Firstly, there are no data to validate the model against. The usual process of dividing up the real-world data into training and tests sets would not work here because the average daily weekday patterns would be identical in both the test and training sets (as evidenced by the extremely narrow confidence intervals produced in Figs. 6 and 8). Larger cities might have other big data sources that can be used to estimate the daytime population, such as public transport smart cards, but this is not the case in Otley. Secondly, empirical validation is of limited value here anyway. The aim of this paper is to explore whether the model might be able to provide new information about groups for whom activity data not currently exist, not (yet) to produce a robust empirical analysis. That said, attempts at face validation were made by presenting early results to the people who are the most familiar with the case study area. Although their observations are clearly anecdotal and the group itself was self-selected, their insights are still useful.

6.3 Future work

Immediate future developments will improve the behaviour and diversity of the agents, the activity framework, and the routing algorithm. To begin with, the agents should be members of households. This is particularly important for modelling school children and their carers (this group was noted to be the most likely contributor to the difference between the model and real data). Data for households could be estimated from the census through spatial microsimulation [40, 41]. Doing activities with other household members would have an impact on the activity intensities of the agents. An extended model would also include a greater variety of leisure activities. The agents could also remember preferred locations for their flexible activities, which would then lead to a higher probability of that location hosting the activity in the future (i.e. the agents could build up preferences for certain locations). They could also learn to improve the logical order of their activities, so that their travel distance reduces. Furthermore, by accounting for different transportation modes, congestion could begin to emerge.

Although the model was calibrated, the process was performed manually by comparing the observed agent behaviour to the time use survey and footfall data and adjusting parameters accordingly (as discussed in Section 3.1). In the future, the calibration of the model parameters should be more automated. With the current mix of big data and traditional data, optimisation algorithms such as genetic algorithms, neural networks, or Bayesian approaches (e.g. [42]) could be considered. However, the most interesting approach to future calibration will include dynamic data assimilation techniques to make real-time data streams to forecast the ambient population [43] and update a running model accordingly.