Skip to main content
SpringerLink
Log in

Unified route representation learning for multi-modal transportation recommendation with spatiotemporal pre-training

  • Regular Paper
  • Published:
The VLDB Journal Aims and scope Submit manuscript

Abstract

Multi-modal transportation recommendation aims to provide the most appropriate travel route with various transportation modes according to certain criteria. After analyzing large-scale navigation data, we find that route representations exhibit two patterns: spatio-temporal autocorrelations within transportation networks and the semantic coherence of route sequences. However, there are few studies that consider both patterns when developing multi-modal transportation systems. To this end, in this paper, we study multi-modal transportation recommendation with unified route representation learning by exploiting both spatio-temporal dependencies in transportation networks and the semantic coherence of historical routes. Specifically, we first transform the multi-modal transportation network into time-dependent multi-view transportation graphs and devise a graph-based contextual encoder to impute the missing traffic condition in transportation networks by leveraging various contextual factors. Then, we propose a hierarchical multi-task route representation learning (HMTRL) framework for recommendations, including (1) a spatiotemporal graph neural network module to capture the spatial and temporal autocorrelation, (2) a coherent-aware attentive route representation learning module to explicitly model route coherence from historical routes, and (3) a hierarchical multi-task learning module to differentiate route representations for different transport modes by incorporating multiple auxiliary tasks equipped in different network layers. Moreover, to improve the model generalization capability, we further propose spatiotemporal pre-training strategies to exploit rich self-supervision signals hidden in transportation networks and historical trajectories. Finally, extensive experimental results on two large-scale real-world datasets demonstrate the effectiveness of the proposed system against eight baselines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. https://github.com/hanjindong/HMTRL-Pytorch

References

  1. Baidu maps: https://en.wikipedia.org/wiki/Baidu_Maps, 2021. Accessed: 2021-07-01

  2. wego Here: https://en.wikipedia.org/wiki/Here_WeGo, (2021). Accessed: 2021-07-01

  3. Didi. https://en.wikipedia.org/wiki/DiDi, 2021. Accessed: 2021-07-01

  4. Dibbelt, J. et al.: Engineering algorithms for route planning in multimodal transportation networks. Transportation (2016)

  5. Borole, N., Rout, D., Goel, N., Vedagiri, P., Mathew, T.V.: Multimodal public transit trip planner with real-time transit data. Proc. Soc. Behav. Sci. 104, 775–784 (2013)

    Article  Google Scholar 

  6. Geisberger, R., Sanders, P., Schultes, D., Vetter, C.: Exact routing in large road networks using contraction hierarchies. Transp. Sci. 46(3), 388–404 (2012)

    Article  Google Scholar 

  7. Liu, L.: Data model and algorithms for multimodal route planning with transportation networks. PhD Thesis, Technische Universität München (2011)

  8. Liu, H., Tong, Y., Zhang, P., Lu, X., Duan, J., Xiong, H.: Hydra: A personalized and context-aware multi-modal transportation recommendation system. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2314–2324 (2019)

  9. Yuan, J., Zheng, Y., Zhang, C., Xie, W., Xie, X., Sun, G., Huang, Y.: T-drive: driving directions based on taxi trajectories. In: Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems, pp. 99–108 (2010)

  10. Abedalla, A., Fadel, A., Tuffaha, I., Al-Omari, H., Omari, M., Abdullah, M., Al-Ayyoub, M.: Mtrecs-dlt: Multi-modal transport recommender system using deep learning and tree models. In: 2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS), pp. 274–278. IEEE (2019)

  11. Zhou, H., Zhao, Y., Fang, J., Chen, X., Zeng, K.: Hybrid route recommendation with taxi and shared bicycles. Distrib. Parall. Databases, 38(3), 563 –583 (2019)

  12. Liu, H., Li, T., Hu, R., Fu, Y., Gu, J., Xiong, H.: Joint representation learning for multi-modal transportation recommendation. In: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, pp. 1036–1043 (2019)

  13. Liu, H., Han, J., Yanjie, F., Zhou, J., Xinjiang, L., Xiong, H.: Multi-modal transportation recommendation with unified route representation learning. Proc. VLDB Endowment 14(3), 342–350 (2020)

    Article  Google Scholar 

  14. Liu, H., Li, Y., Fu, Y., Mei, H., Zhou, J., Ma, X., Xiong, H.: Polestar: an intelligent, efficient and national-wide public transportation routing engine. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2321–2329 (2020)

  15. Li, Y., Yu, R., Shahabi, C., Liu, Y.: Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In: 6th International Conference on Learning Representations (2018)

  16. Wang, P., Fu, Y., Zhang, J., Wang, P., Zheng, Y., Aggarwal, C.: You are how you drive: Peer and temporal-aware representation learning for driving behavior analysis. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2457–2466 (2018)

  17. Cvetkovic, D., Cvetković, D.M., Rowlinson, P., Simic, S., Simić, S.: Spectral generalizations of line graphs: On graphs with least eigenvalue-2, vol. 314. Cambridge University Press (2004)

  18. Hu, J., Guo, C., Yang, B., Jensen, C.S.: Stochastic weight completion for road networks using graph convolutional networks. In: 2019 IEEE 35th International Conference on Data Engineering (ICDE), pp. 1274–1285. IEEE (2019)

  19. Hu, J., Yang, B., Guo, C., Jensen, C.S., Xiong, H.: Stochastic origin-destination matrix forecasting using dual-stage graph convolutional, recurrent neural networks. In: 2020 IEEE 36th International Conference on Data Engineering (ICDE), pp. 1417–1428. IEEE (2020)

  20. Guo, C., Yang, B., Hu, J., Jensen, C.: Learning to route with sparse trajectory sets. In: 2018 IEEE 34th International Conference on Data Engineering (ICDE), pp. 1073–1084. IEEE (2018)

  21. Guo, C., Yang, B., Hu, J., Jensen, C.S., Chen, L.: Context-aware, preference-based vehicle routing. VLDB J. 29(5), 1149–1170 (2020)

    Article  Google Scholar 

  22. Yang, B., Kaul, M., Jensen, C.S.: Using incomplete information for complete weight annotation of road networks. IEEE Trans. Knowl. Data Eng. 26(5), 1267–1279 (2013)

    Article  Google Scholar 

  23. Liu, H., Wu, Q., Zhuang, F., Lu, X., Dou, D., Xiong, H.: Community-aware multi-task transportation demand prediction. In: Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (2021)

  24. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: 5th International Conference on Learning Representations (2017)

  25. Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: Advances in Neural Information Processing Systems, pp. 1024–1034 (2017)

  26. Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, (2014)

  27. Li, S., Li, W., Cook, C., Zhu, C., Gao, Y.: Independently recurrent neural network (indrnn): Building a longer and deeper rnn. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5457–5466 (2018)

  28. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)

  29. Zhang, Y., Yang, Q.: A survey on multi-task learning. arXiv preprint arXiv:1707.08114 (2017)

  30. Hashimoto, K., Xiong, C., Tsuruoka, Y., Socher, R.: A joint many-task model: Growing a neural network for multiple nlp tasks. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1923–1933 (2017)

  31. Fei, H., Tan, S., Li, P.: Hierarchical multi-task word embedding learning for synonym prediction. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 834-842 (2019)

  32. Caruana, R.: Multitask learning: A knowledge-based source of inductive bias. Machine Learning. In: Proceedings of the Tenth International Conference on Machine Learning, pp. 41–48 (1993)

  33. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  34. Hendrycks, D., Mazeika, M., Kadavath, S., Song, D.: Using self-supervised learning can improve model robustness and uncertainty. Adv. Neural Inf. Process. Syst., 32 (2019)

  35. You, Y., Chen, T., Wang, Z., Shen, Y.: When does self-supervision help graph convolutional networks? In: International Conference on Machine Learning, pp. 10871–10880. PMLR (2020)

  36. Hu, W., Liu, B., Gomes, J., Zitnik, M., Liang, P., Pande, V., Leskovec, J.: Strategies for pre-training graph neural networks, Percy Liang (2020)

  37. Yuan, J., Zheng, Y., Xie, X.: Discovering regions of different functions in a city using human mobility and pois. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 186–194 (2012)

  38. Yuan, N.J., Zheng, Y., Xie, X., Wang, Y., Zheng, K., Xiong, H.: Discovering urban functional zones using latent activity trajectories. IEEE Trans. Knowl. Data Eng. 27(3), 712–725 (2014)

    Article  Google Scholar 

  39. Hjelm, R.D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Bachman, P., Trischler, A., Bengio, Y.: Learning deep representations by mutual information estimation and maximization (2019)

  40. Lou, Y., Zhang, C., Zheng, Y., Xie, X., Wang, W., Huang, Y.: Map-matching for low-sampling-rate gps trajectories. In: Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pp. 352–361 (2009)

  41. He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738 (2020)

  42. Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of ir techniques. ACM Trans. Inform. Syst. (TOIS), pp. 422–446 (2002)

  43. Kleinbaum, D.G., Dietz, K., Gail, M., Klein, M., Klein, M.: Logistic Regression. Springer, Berlin (2002)

    Google Scholar 

  44. Chen, T., Guestrin, C.: Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794 (2016)

  45. Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: Online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710 (2014)

  46. Guo, H., TANG, R., Ye, Y., Li, Z., He, X.: Deepfm: A factorization-machine based neural network for ctr prediction. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, pp. 1725–1731 (2017)

  47. Li, Y., Fu, K., Wang, Z., Shahabi, C., Ye, J., Liu, Y.: Multi-task representation learning for travel time estimation. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1695–1704 (2018)

  48. Shang, J., Zheng, Y., Tong, W., Chang, E., Yu, Y.: Inferring gas consumption and pollution emission of vehicles throughout a city. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1027–1036 (2014)

  49. Kanoulas, E., Du, Y., Xia, T., Zhang, D.: Finding fastest paths on a road network with speed patterns. In: 22nd International Conference on Data Engineering, pp. 10–10 (2006)

  50. Wei, L.-Y., Zheng, Y., Peng, W.-C.: Constructing popular routes from uncertain trajectories. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 195–203 (2012)

  51. Chen, D., Ong, C.S., Xie, L.: Learning points and routes to recommend trajectories. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp. 2227–2232 (2016)

  52. Shafique, S., Ali, M.E.: Recommending most popular travel path within a region of interest from historical trajectory data. In: Proceedings of the 5th ACM SIGSPATIAL International Workshop on Mobile Geographic Information Systems, pp. 2–11. ACM (2016)

  53. Chen, L., Shang, S., Jensen, C.S., Yao, B., Zhang, Z., Shao, L.: Effective and efficient reuse of past travel behavior for route recommendation. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 488–498 (2019)

  54. Wang, J., Wu, N., Zhao, W.X., Peng, F., Lin, X.: Empowering a* search algorithms with neural networks for personalized route recommendation. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 539–547 (2019)

  55. Yang, S.B., Guo, C., Yang, B.: Context-aware path ranking in road networks. IEEE Trans. Knowl. Data Eng., pp. 1–1 (2020)

  56. Campigotto, P., Rudloff, C., Leodolter, M., Bauer, D.: Personalized and situation-aware multimodal route recommendations: the favour algorithm. IEEE Trans. Intell. Transp. Syst. 18(1), 92–102 (2016)

    Article  Google Scholar 

  57. Liu, H., Tong, Y., Han, J., Zhang, P., Lu, X., Xiong, H.: Incorporating multi-source urban data for personalized and context-aware multi-modal transportation recommendation. IEEE Trans. Knowl. Data Eng. 34(2), 723–735 (2020)

  58. Wang, Y., Yin, H., Chen, H., Wo, T., Xu, J., Zheng, K.: Origin-destination matrix prediction via graph convolution: a new perspective of passenger demand modeling. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1227–1235 (2019)

  59. Zhang, Y., Fu, Y., Wang, P., Li, X., Zheng, Y.: Unifying inter-region autocorrelation and intra-region structures for spatial embedding via collective adversarial learning. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1700–1708 (2019)

  60. Zhang, W., Liu, H., Liu, Y., Zhou, J., Xiong, H.: Semi-supervised hierarchical recurrent graph neural network for city-wide parking availability prediction. In: Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, pp. 1186–1193 (2020)

  61. Li, J., Han, Z., Cheng, H., Su, J., Wang, P., Zhang, J., Pan, L.: Predicting path failure in time-evolving graphs. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1279–1289 (2019)

  62. Søgaard, A., Goldberg, Y.: Deep multi-task learning with low level tasks supervised at lower layers. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp. 231–235 (2016)

  63. Trinh, T.H., Luong, M.-T., Le, Q.V.: Selfie: Self-supervised pretraining for image embedding. arXiv preprint arXiv:1906.02940, (2019)

  64. Qiu, J., Chen, Q., Dong, Y., Zhang, J., Yang, H., Ding, M., Wang, K., Tang, J.: Gcc: Graph contrastive coding for graph neural network pre-training. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1150–1160 (2020)

  65. Hu, Z., Dong, Y., Wang, K., Chang, K.-W., Sun, Y.: Gpt-gnn: Generative pre-training of graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1857–1867 (2020)

  66. Ren, Z., Lee, Y.J.: Cross-domain self-supervised multi-task feature learning using synthetic imagery. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 762–771 (2018)

  67. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Pre-training of deep bidirectional transformers for language understanding, Bert (2019)

  68. Yang, S.B., Guo, C., Hu, J., Tang, J., Yang, B.: Unsupervised path representation learning with curriculum negative sampling. arXiv preprint arXiv:2106.09373, (2021)

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under Grant No. 62102110, Foshan HKUST Projects (FSUST21-FYTRI01A, FSUST21-FYTRI02A), Hong Kong RGC TRS T41-603/20-R, and Hetao Shenzhen-Hong Kong Science and Technology Innovation Cooperation Zone (HZQB-KCZYB-2020083).

Author information

Authors and Affiliations

  1. The Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China

    Hao Liu, Jindong Han & Hui Xiong

  2. The Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong, SAR, China

    Hao Liu, Kai Chen & Hui Xiong

  3. HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute, Shenzhen, China

    Hao Liu

  4. University of Central Florida, Orlando, USA

    Yanjie Fu

  5. Baidu Research, Beijing, China

    Yanyan Li

Authors
  1. Hao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jindong Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanjie Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yanyan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kai Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hui Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hao Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 540 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, H., Han, J., Fu, Y. et al. Unified route representation learning for multi-modal transportation recommendation with spatiotemporal pre-training. The VLDB Journal 32, 325–342 (2023). https://doi.org/10.1007/s00778-022-00748-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00778-022-00748-y

Keywords

Search

Navigation