Simultaneous dimension reduction and clustering via the NMF-EM algorithm

Abstract

Mixture models are among the most popular tools for clustering. However, when the dimension and the number of clusters is large, the estimation of the clusters become challenging, as well as their interpretation. Restriction on the parameters can be used to reduce the dimension. An example is given by mixture of factor analyzers for Gaussian mixtures. The extension of MFA to non-Gaussian mixtures is not straightforward. We propose a new constraint for parameters in non-Gaussian mixture model: the K components parameters are combinations of elements from a small dictionary, say H elements, with $H \ll K$. Including a nonnegative matrix factorization (NMF) in the EM algorithm allows us to simultaneously estimate the dictionary and the parameters of the mixture. We propose the acronym NMF-EM for this algorithm, implemented in the R package nmfem. This original approach is motivated by passengers clustering from ticketing data: we apply NMF-EM to data from two Transdev public transport networks. In this case, the words are easily interpreted as typical slots in a timetable.

Appendix

1.1 Analysis of the clusters of users

As written above, we have no personal information in our data. Therefore, we are not able to describe individually the users in each cluster. However, for each transaction made, we have the encrypted card number and the transport ticket used. So we can recover for each card the most used transport ticket during the period. This provides interesting information as some schemes are associated to age ranges (Young, Senior...) and to time periods (Unit, Annual or Monthly Subscription). Let us now provide the description of each cluster in terms of age ranges (Fig. 3a–c in Fig. 8).

Fig. 8

Age range analysis of the clusters

Adults are more present in clusters 7 and 9, that are clusters with check-ins mostly in the morning. People benefiting from half-price are present in every cluster but with highest rates in clusters 2, 3, 4 and 5. Children (4–6) are not very present on the network, but they are more represented in clusters 1, 5 and 9. Young travelers (6–25) are more present in clusters 1 and 4. These clusters correspond to scholar time slot. In clusters 8 and 10 there are large rate of seniors and free travelers. As these clusters have profiles of diffuse travels during the week and as free travelers are unemployed or low salaries people, these regroupments make sense.

Figure 9 shows the repartition of transport ticket type through clusters. Unit products are more used in clusters 8 and 10 that are clusters with a lots of seniors and free travelers. As they don’t have obligations, they likely use unit products for occasional trips. Clusters 1, 3, 4 and 9, that have mostly scholar profiles althought have a large majority of annual subscripters. A possible interpretation is that schoolchildren and students are public transportation captives, and have to use the network in order to go to class every day. Thus, buying an annual pass is more advantageous than buying any other product type.

Fig. 9

Transportation ticket type analysis of the clusters

As described in Sect. 4.1, we kept only users whose first trip of the day is made at the same station at least $50\%$ of the study time. That main “morning station” is thus called the “home station” as it gives us an estimation of the residence place of users. In Figs. 10 and 11, we can observe the shares of clusters by home stations. It shows the share of travelers identified as belonging to every cluster leaving near each station.

Fig. 10

Share of clusters per home station—clusters 1–6

Fig. 11

Share of clusters per home station—clusters 7–10

We note that:

• 1. Cluster 1: travelers are over represented at peripheral stations.

• 2. Cluster 2: no particular pattern observed.

• 3. Cluster 3: no particular pattern observed.

• 4. Cluster 4: few stations show over representation of cluster 4.

• 5. Cluster 5: over representation of the cluster at two stations in the north.

• 6. Cluster 6: no particular pattern observed.

• 7. Cluster 7: One station is $100\%$ represented by cluster 7. As only one user is assigned to this station, no particular pattern is observed.

• 8. Cluster 8: the cluster is over represented at one station in the city center and at another further.

• 9. Cluster 9: cluster 9 is over represented in few stations in the center.

• 10. Cluster 10: cluster is over represented in poorest neighborhoods of the city.

1.2 Stations profile clustering

Clustering the different stations of the network would allow us to better know the different type of stations, and to group them by temporal similarity. As we have very few number of stations (475), it is not safe to process as described above for the users clustering. Indeed, a K larger than 6 or 7 leads to very small clusters. In place we fixed H and K a priori to 3 and 5 respectively.

Fig. 12

Words obtained by NMF-EM on stations data with $K=5$ and $H=3$

The 3 words obtained are the ones in Fig. 12. The first time component is described by check-ins at 7 and 8 a.m. We will call it the “morning component”. The second time component shows check-ins at 4 and 5 p.m on Mondays, Tuesdays, Thursdays and Fridays and check-ins at 12 p.m on Wednesdays. We will name it the “end of school component”. The third component shows check-ins at 6 p.m, during Wednesdays afternoons, during Saturdays and off-peaks periods. This component will be called the “off-peak component”.

Fig. 13

Clusters obtained by NMF-EM on stations data with $K=5$ and $H=3$

Figure 13 shows the 5 clusters. Stations in cluster 1 are stations where there are check-ins only in the morning at 7 or 8 a.m. These stations are likely in residential areas. In cluster 2, the stations have check-ins all day long, but with highest probabilities during peaks. Stations in cluster 3 have check-ins in the morning and at the end of school. They are likely to be near schools in residential areas. Stations in cluster 4 have check-ins only at end of school times. Thus, these stations are probably near schools. Finally, stations in cluster 5 are pretty similar than the ones in cluster 1: a large majority of check-ins are made in the morning (7 or 8 p.m). The only difference is that it is more likely to have check-ins during the rest of the day in cluster 5 than in cluster 1.

Thanks to the French National Institute of Statistics and Economics Studies (INSEE), there are open data permitting us to introduce contextual information. Firstly, a database containing socioeconomic data on a grid of 200 m $\times$ 200 m is available. We used two indicator of it: the number of inhabitants and the percentage of households living in collective housing per tiles. Secondly, we used a database referencing and geolocating every french company or administration. In this way, we were able to know the number of employees per tile. By clustering the tiles in the study area, we obtained different group of areas that will allow us to lead the study on stations more finely. Table 3 contains the description of the mean tile by cluster.

Table 3 Description of tiles clusters

Fig. 14

Map of the stations—opacity of the points are proportional to the adequacy between the stations and the clusters

Fig. 15

Words obtained by NMF-EM on users data with $K=10$ and $H=7$

Fig. 16

Clusters obtained by NMF-EM on users data with $K=10$ and $H=7$

As tiles contained in cluster 1 and 2, are those with the least number of employees, they can be described as residential areas. Moreover, the percentage of collective housing allows to distinguish them. Indeed, cluster 1 have more households living in collective housing than cluster 3. That is why we will refer as tiles from cluster 1 as residential areas in collective housing and as residential areas in individual housing for tiles from cluster 2. Since the number of inhabitants and of employees are high, tiles from cluster 3 will be refered as mixed areas. Finally, as the number of employees in cluster 4 is very large, we will refer these tiles as business areas.

The figures in Fig. 14 show the geographical repartition of the five clusters. In Fig. 6a, we oserve the stations contained in cluster 1. This cluster groups stations that have check-ins only in the morning. On the figure, we observe that these stations are distant from the city center and are mainly located in residential areas. Figure 6b shows stations of cluster 2, that have check-ins all day long with stronger attendance during peak-periods. These stations are mainly located in the city center. Figure 6c, d look alike. Indeed, clusters 3 and 4 have the “end of school” component and the points on the map are close to educational establishment. Figure 6e shows stations from cluster 5. These stations have check-ins all day long but most are made in the morning. By looking at the map, we cannot notice any significant pattern.

1.3 Passengers profile clustering on another network

To ensure efficiency of the algorithm, we applied it on another network located in the Netherlands. By applying the same model selection method as in Sect. 4.2, we obtained the optimal values of $K=10$ and $H=7$. Figures 15 and 16 contain respectively the profiles of the words and clusters obtained.

The interpretation of the words is:

• 1. Word 1: travels at 6 or 7 a.m and slightly around 4 p.m during the week.

• 2. Word 2: travels during the week-end.

• 3. Word 3: diffuse travel habits from 8 a.m to 4 p.m Mondays to Fridays.

• 4. Word 4: travels at 7a.m on weekdays.

• 5. Word 5: diffuse habits with highest probabilities from 5 p.m to 12 a.m during the week.

• 6. Word 6: diffuse habits from 9 a.m to 5 p.m with highest probability at 1 p.m Mondays to Saturdays.

• 7. Word 7: travels at 8 a.m and 5 p.m.

We can interpret the cluster as follows:

• 1. Cluster 1: diffuse habits from 9 a.m to 5 p.m with highest probability at 1 p.m Mondays to Saturdays.

• 2. Cluster 2: travels at 6 or 7 a.m and at 4 or 5 p.m during the week.

• 3. Cluster 3: diffuse habits from 7 a.m to 6 p.m on weekdays.

• 4. Cluster 4: diffuse travel habits from 9 a.m to 11 p.m.

• 5. Cluster 5: travels at 7 or 8 a.m diffuse habits during the afternoon.

• 6. Cluster 6: travels at 8 a.m and 5 p.m.

• 7. Cluster 7: diffuse travel habits from 7 a.m to 5 p.m Mondays to Fridays.

• 8. Cluster 8: diffuse habits from 8 a.m to 4 p.m during the week.

• 9. Cluster 9: travels during the week-end.

• 10. Cluster 10: travels at 7 or 8 a.m and around 4 p.m.