Multi-modal Recurrent Graph Neural Networks for Spatiotemporal Forecasting

Abstract

The spatial and temporal dynamics of many real-world systems present a significant challenge to multi-variate forecasting where features of both forms, as well as their inter-dependencies, must be modeled correctly. State-of-the-art approaches utilize a limited set of exogenous features (outside the forecast variable) to model temporal dynamics and Graph Neural Networks, with pre-defined or learned networks, to model spatial dynamics. While much work has been done to model dependencies, existing approaches do not adequately capture the explicit and implicit modalities present in real-world systems. To address these limitations we propose MMR-GNN, a spatiotemporal model capable of (a) augmenting pre-defined (or absent) networks into optimal dependency structures (b) fusing multiple explicit modalities and (c) learning multiple implicit modalities. We show improvement over existing methods using several hydrology and traffic datasets. Our code is publicly available at https://github.com/HipGraph/MMR-GNN.

7 Appendix

Road Network Inference. E-METR-LA and E-PEMS-BAY are extensions of the quintessential METR-LA and PEMS-BAY datasets adding new spatial and spatiotemporal features. Following [11], we wanted to offer pre-defined graphs for each dataset and chose to infer them using new spatial features. We apply GraphAugr to construct a graph heuristically using exponentiated Minkowski ($p=2$) distance to define node similarity and $\boldsymbol{W}, \boldsymbol{B}$ to define rules for edge construction. Each traffic sensor of E-METR-LA and E-PEMS-BAY includes latitude, longitude, freeway name, and freeway bearing. We use latitude and longitude to define node similarity $\boldsymbol{S}$ and freeway/bearing to define a mask $\boldsymbol{W}$ that restricts to intra-highway edges of the same bearing.

Using OpenStreetMap [15], we found many sensor pairs placed closely together but recording traffic in opposite bearings (e.g., U.S. Route 101 in CA with North/South bearing). Using distance-based similarity alone leads to connections between these and many other unrelated sensors. With the previous similarity and restrictions, the modified similarity becomes $\boldsymbol{S}^{*} = \boldsymbol{W} \odot \boldsymbol{S}$ which we prune using K-NN with $k=2$. The original graph of METR-LA and our new graph for E-METR-LA are shown in Fig. 4.

Fig. 4.

The pre-defined road network from METR-LA (a) and our new road network for E-METR-LA (b)