In the following section, we describe our results in terms of the different location typologies covered in our deployment. We start by presenting an overall view of the deployment and then comparing the five different typologies, followed by a detailed analysis of the different locations where ground truth data was provided (e.g., pavilion, port and airports and football stadium).
During the study, we analyzed the activity over the different regions of the island. Results relating to the overall data, from all locations, are reported below. The activity depicted in Fig. 2 shows that most device detections occur in the main islands’ city (south) and airport (east) of the island.
One of the parameters analyzed was the number of random device IDs of the users’ devices. The latest device operating systems use randomized MAC addresses when the devices are just probing and not connected to any network. For 4 years (200 weeks), the system tracked more than 3.2 million unique devices (excluding randomly generated MAC addresses) which corresponds to an average of 20.5k new devices detected per week. The overall percentage of randomly generated MAC addresses detected by the system started from 40% in the first weeks of deployment and reached 94% on the last year. In Fig. 3, we depict the percentage of detected random MAC addresses over time for the 4-year period of deployment against the major releases of smartphone and desktop operating systems. This shows us how manufacturers have increased their security measures to prevent tracking of their users by MAC address, and how users have adopted for recent operating systems over the years.
With these percentages, manufacturers partially achieve their goal of anonymizing the requests, hindering the possibility of these tracking platforms, such as the one presented here, to perform large-scale trajectory analysis, by tracking the movements of the same MAC address over different POIs across time, since the percentage of random MAC addresses has become so high, that every random MAC address only appears once in the system. This makes that trajectory analysis only represents a small fraction of the total of trajectories performed.
The distribution of the devices based on their MAC address was obtained from the IEEE vendor database, and is shown in Fig. 4, represented by 35.5% from Samsung, 23.2% from Apple, 7% from Motorola, and 5% from Murata and Huawei, with all the other vendors not exceeding a percentage of 5% of the total 3.2 million non-random devices detected.
To empirically illustrate how our system (a community-based passive Wi-Fi network) can be used to estimate the number of people (RQ1) present and flowing across different location typologies (corresponding to different presence and movement patterns) and (RQ2) we describe the comparison between our data and the ground truth (when provided), of the average count of detected devices for the time interval of 1 month, across the five distinct locations (see Fig. 5). From the data, we detected that the football stadium is mostly empty across the day, occasionally detecting some passing devices (e.g., maintenance staff); the pavilion has two peaks, one in the morning and one in the afternoon, where the event begins and ends with a dip at lunchtime.
The airport has fairly distributed peaks across the day, decreasing during the nighttime because, due to local policies, no flights occur during those times; the plaza has a fairly consistent detection load across the day; the nightclub has an inverted peak compared with the remaining ones, where the most affluence occurs at night remaining the rest of the day with only detection of passersby users. From our data, it is possible distinguish different types of usage, from sporadic peaks, to constant load, and day vs night usage of each location (see Fig. 5).
Analyzing the hourly data for each location typology, as seen in Fig. 5, we can also estimate the number of parasitic devices, such as local computers, smart TVs, or embedded systems. We can assess this by looking at the device counts at nighttime when we know that the people count is close to none. This technique also enables the gathering of labeled data in these environments.
Using the location typology classification methods described in Section 4.1, our data shows that the highest accuracy achieved was used with the Random Forest classifier with 89.6%, noting that the methods RFC, ETC, and GBC all scored above 86% from the train/test ratio of 20%, while the methods GNB and LDA scored the lowest with 68% and 50% respectively (see Fig. 6)
In regard to the data collected in the Fair Pavilion, despite the usual daily activity monitoring, we covered a car exhibition fair. In total, we monitored the space and the flux of people for 8 days. We divided the time span into two categories: (i) 4 days during the car fair event (between Friday and Sunday); (ii) 4 days with no special events occurring in the building. The data was averaged over 24 h for the two sets of 4 days and shows the discrepancies in the occupancy (see Fig. 7).
During the weekdays, the occupancy is higher during the lunch hour, due to the bar activity located in the same open space as the event pavilion, regaining calm during the rest of the hours. However, during the car fair event, the opposite happens: there is significant activity during the hours between 10:00 until 20:00, except for the lunchtime. This can be explained by the fact that during the weekend, the visitors only visit the car fair location but do not remain there after the visit.
We monitored a second exhibition fair, which occurred indoors, in the outer ring of the football stadium, from which we have no ground truth. The focus of this activity was to collect data and then analyze the flow of people between 7 sensor-equipped locations around the outer ring of the stadium for 7 days (as shown in Fig. 8).
The flow was detected by tracing the movement of the device IDs across the different Wi-Fi sensors consecutively during the event with an origin/destination sensor for the same MAC address for each leap. Data shows that the majority of leaps occur in the entrance area (sensors 108 and 109) and in the food area (sensor 102 and 103). In the entrance, the visitors had to follow a predetermined path between sensors 108 and 109 before being able to roam freely. And the dining area had two sensors close together where the leaps were also frequent between the food kiosks. Although the remaining locations had leaps between all the other locations (visitors passing by in a rapid manner may not be captured by all the sensors due to the probe request intervals), there is a major flow between all the sensors to the dining area and between the sensors 104 and 105 located in an indoor area of the stadium outer ring. Results are visualized in Fig. 9 with a total of 130,847 leaps.
Although this experiment returned these results, due to the high percentage of random MAC addresses (94% that only appear once in the system), the remaining 6% of real MAC addresses are not enough of a sample to be representative of the real movements of people. Also since most location typologies only have 1 or 2 sensors in each location, it is not feasible to detect the flows across the same typologies.
In relation to RQ1, data shows that over the period analyzed of 17 weeks, results show a Pearson’s correlation of 0.64 for the arrivals and 0.62 for the departures, between the number of official passengers on the airplanes and the devices detected by our system (Fig. 10).
The captured data represents a ratio between the unique devices detected and the official passenger counts with ratios of (μ= 0.47, σ= 0.23, n= 116) for the arrivals and (μ= 0.37, σ= 0.22, n= 116) for the departures. This correlation was done during 17 weeks, with a total of 181,184 and 158,438 device IDs detected for arrivals and departures respectively (difference may be due to people turning off phones before takeoff, not having Wi-Fi turned on, or being missed by the system).
Regarding RQ1, these results mean that we are able to create a linear regression intersecting the zero (see Fig. 11) with the parameters y = 2.53x for arrivals and y = 1.99x for departures. This successfully estimates around half of the passengers in the arrivals, with the difference for the departures being possibly due to the placement of the router.
In the port, the router was installed at the arrivals, and we compared the number of daily detected unique devices, with the number of ships that arrived in that day. Data shows corresponding spikes of people counting when there is a ship arrival in the port (with the exceptions being for small ships), being visualized as events.
Moreover, in terms of RQ1, we show a reliable detection of people passing by the port station with the event of a ship arrival, with the results being shown in Fig. 12, and a Pearson’s correlation between the number of daily ships and daily device count yields a result of 0.74, n= 49. The total number of device IDs detected during these 49 days was 37,566.
The data gathered in the football stadium shows a clear detection of when a game occurs, acting as an event in the data with a distinct peak from the remaining days. The data reveals the days in which games occur distinguished from the remaining, at 09 March 2019 and 31 March 2019 as shown in Fig. 13 (left).
When analyzed by hour, we can also infer the time at which people started arriving at the stadium and at what time they left, returning back to normality, when compared against an average of the days when the game does not occur.
For the particular day of 31 March 2019 (Sunday), Fig. 13 (right) reveals where the game started at 15:00 with the duration of 90 min + 15 min interval. This information can be easily used to train event detection algorithms to automatically identify the game days, or days of affluence in the area.
During the monitored eight games, we compared our data against the ground truth, provided by the official ticketing information and compared it against the router’s count (see Fig. 14) resulting in a Pearson’s correlation of 0.81, n= 8 and a ratio of devices detected vs official ticketing counts of (μ= 0.81, σ= 0.12, n= 8).
Lastly, although each location typology has significantly different ratios of detected devices and ground truth counts, they all provide low standard deviations for each, meaning that a regression model can be accurately applied to each typology to obtain the real counts.
To finalize the “Results” section, we present a summary of the different results obtained from the different analyses and data sets. These results presented above show the versatility of this data set and how we analyzed different typologies separately, and providing custom ground truth for specific controlled locations. In summary, we achieved 12 main results grouped in Table 2.