Enabling Vehicular Data with Distributed Machine Learning

  • Cristian Chilipirea
  • Andreea Petre
  • Ciprian Dobre
  • Florin Pop
  • Fatos Xhafa
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9380)


Vehicular Data includes different facts and measurements made over a set of moving vehicles. Most of us use cars or public transportation for our work commute, daily routines and leisure. But, except of our destination, possible time of arrival and what is directly around us, we know very little about the traffic conditions in the city as a whole. Because all roads are connected in a vast network, events in other parts of town can and will directly affect us. The more we know about the traffic inside a city, the better decisions we can make. Vehicular measurements may contain a vast amount of information about the way our cities function. Information that can be used for more than improving our commute, it is indicative of other features of the city like the amount of pollution in different regions. All the information and knowledge we can extract, can be used to directly improve our life.

We live in a world where data is constantly generated and we store it and process it at an ever growing rate. Vehicular Data does not stray from this fact and is rapidly growing in size and complexity, with more and more ways to monitoring traffic, either from inside cars or from sensors placed on the road. Smartphones and in-car-computers are now common and they can produce a vast amount of data: it can identify a cars location, destination, current speed and even driving habits.

Machine learning is the perfect complement for Big Data, as large data sets can be rendered useless without methods to extract knowledge and information from them. Machine learning, currently a popular research topic, has a large number of algorithms design to achieve this task, of knowledge extraction. Most of these techniques and algorithms can be directly applied to Vehicular Data.

In this article we demonstrate how the use of a simple algorithm, k-Nearest Neighbors, can be used to extract valuable information from even a relatively small vehicular data set. Because of the vast size of most of our cities and the number of cars that are on their roads at any time of the day, standard machine learning systems do not manage to process data in a manner that would permit real time use of the extracted information. A solution to this problem is brought by distributed systems and cloud processing. By parallelizing and distributing machine learning algorithms we can use data at its highest potential and with little delay. Here, we show how this can be achieved by distributing the k-Nearest Neighbors machine learning algorithm over MPI. We hope this would motivate the research into other combinations of merging machine learning algorithms with Vehicular Data sets.


Big data Machine learning MPI Cloud systems Distributed processing 



This work was supported by the Romanian national project MobiWay, Project PN-II-PT-PCCA-2013-4-0321. The authors would like to thank reviewers for their constructive comments and valuable insights.


  1. 1.
    Abdulhai, B., Pringle, R., Karakoulas, G.J.: Reinforcement learning for true adaptive traffic signal control. J. Transp. Eng. 129(3), 278–285 (2003)CrossRefGoogle Scholar
  2. 2.
    Amici, R., Bonola, M., Bracciale, L., Rabuffi, A., Loreti, P., Bianchi, G.: Performance assessment of an epidemic protocol in vanet using real traces. Procedia Comput. Sci. 40, 92–99 (2014)CrossRefGoogle Scholar
  3. 3.
    Apache: Mahout (2015).
  4. 4.
    Apache: Spark mllib (2015).
  5. 5.
    Barrientos, R., Gómez, J., Tenllado, C., Prieto, M.: Heap based k-nearest neighbor search on gpus. In: Congreso Espanol de Informática (CEDI), pp. 559–566 (2010)Google Scholar
  6. 6.
    Biem, A., Bouillet, E., Feng, H., Ranganathan, A., Riabov, A., Verscheure, O., Koutsopoulos, H., Moran, C.: Ibm infosphere streams for scalable, real-time, intelligent transportation services. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, pp. 1093–1104. ACM (2010)Google Scholar
  7. 7.
    Bracciale, L., Bonola, M., Loreti, P., Bianchi, G., Amici, R., Rabuffi, A.: CRAWDAD data set roma/taxi (v. 2014–07-17), Jul 2014. Downloaded from
  8. 8.
    Brakatsoulas, S., Pfoser, D., Salas, R., Wenk, C.: On map-matching vehicle tracking data. In: Proceedings of the 31st International Conference on Very Large Data Bases, pp. 853–864. VLDB Endowment (2005)Google Scholar
  9. 9.
    Chadil, N., Russameesawang, A., Keeratiwintakorn, P.: Real-time tracking management system using gps, gprs and google earth. In: 5th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology, ECTI-CON 2008, vol. 1, pp. 393–396. IEEE (2008)Google Scholar
  10. 10.
    Chen, W.Y., Song, Y., Bai, H., Lin, C.J., Chang, E.Y.: Parallel spectral clustering in distributed systems. IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 568–586 (2011)CrossRefGoogle Scholar
  11. 11.
    Coifman, B., Beymer, D., McLauchlan, P., Malik, J.: A real-time computer vision system for vehicle tracking and traffic surveillance. Transp. Res. Part C: Emerg. Technol. 6(4), 271–288 (1998)CrossRefGoogle Scholar
  12. 12.
    Dartmouth: Crowdad (2015).
  13. 13.
    exploratorium: Cabspotting (2015).
  14. 14.
    Feldman, D., Sugaya, A., Sung, C., Rus, D.: idiary: From gps signals to a text-searchable diary. In: Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems, p. 6. ACM (2013)Google Scholar
  15. 15.
    Fu, L., Hao, J., He, D., He, K., Li, P.: Assessment of vehicular pollution in china. J. Air Waste Manage. Assoc. 51(5), 658–668 (2001)CrossRefGoogle Scholar
  16. 16.
    Gillick, D., Faria, A., DeNero, J.: Mapreduce: Distributed computing for machine learning. Berkley, 18 Dec 2006Google Scholar
  17. 17.
    Hsieh, J.W., Yu, S.H., Chen, Y.S., Hu, W.F.: Automatic traffic surveillance system for vehicle tracking and classification. IEEE Trans. Intell. Transp. Syst. 7(2), 175–187 (2006)CrossRefzbMATHGoogle Scholar
  18. 18.
    IEEE, TomTom: Ieee icdm contest: Tomtom traffic prediction for intelligent gps navigation (2010).
  19. 19.
    Low, Y., Bickson, D., Gonzalez, J., Guestrin, C., Kyrola, A., Hellerstein, J.M.: Distributed graphlab: a framework for machine learning and data mining in the cloud. Proc. VLDB Endowment 5(8), 716–727 (2012)CrossRefGoogle Scholar
  20. 20.
    Lu, W., Shen, Y., Chen, S., Ooi, B.C.: Efficient processing of k nearest neighbor joins using mapreduce. Proc. VLDB Endowment 5(10), 1016–1027 (2012)CrossRefGoogle Scholar
  21. 21.
    Mavromoustakis, C.X., Kormentzas, G., Mastorakis, G., Bourdena, A., Pallis, E., Rodrigues, J.: Context-oriented opportunistic cloud offload processing for energy conservation in wireless devices. In: Globecom Workshops (GC Wkshps), pp. 24–30. IEEE (2014)Google Scholar
  22. 22.
    Microsoft, R.: T-drive: Driving directions based on taxi traces (2015).
  23. 23.
    Mousicou, P., Mavromoustakis, C.X., Bourdena, A., Mastorakis, G., Pallis, E.: Performance evaluation of dynamic cloud resource migration based on temporal and capacity-aware policy for efficient resource sharing. In: Proceedings of the 2nd ACM Workshop on High Performance Mobile Opportunistic Systems, pp. 59–66. ACM (2013)Google Scholar
  24. 24.
    OpenStreetMap: Openstreetmap (2015).
  25. 25.
    Papadakis, S.E., Stykas, V., Mastorakis, G., Mavromoustakis, C.X., et al.: A hyper-box approach using relational databases for large scale machine learning. In: 2014 International Conference on Telecommunications and Multimedia (TEMU), pp. 69–73. IEEE (2014)Google Scholar
  26. 26.
    Pau, G., Tse, R.: Challenges and opportunities in immersive vehicular sensing: lessons from urban deployments. Sig. Process. Image Commun. 27(8), 900–908 (2012)CrossRefGoogle Scholar
  27. 27.
    Piórkowski, M., Sarafijanovic-Djukic, N., Grossglauser, M.: A parsimonious model of mobile partitioned networks with clustering. In: First International Communication Systems and Networks and Workshops, COMSNETS 2009, pp. 1–10. IEEE (2009)Google Scholar
  28. 28.
    Safar, M.: K nearest neighbor search in navigation systems. Mob. Inf. Syst. 1(3), 207–224 (2005)Google Scholar
  29. 29.
    Schubert, R., Richter, E., Wanielik, G.: Comparison and evaluation of advanced motion models for vehicle tracking. In: 2008 11th International Conference on Information Fusion, pp. 1–6. IEEE (2008)Google Scholar
  30. 30.
    Sun, S., Zhang, C., Yu, G.: A bayesian network approach to traffic flow forecasting. IEEE Trans. Intell. Transp. Syst. 7(1), 124–132 (2006)CrossRefGoogle Scholar
  31. 31.
    Williams, B.M., Hoel, L.A.: Modeling and forecasting vehicular traffic flow as a seasonal arima process: Theoretical basis and empirical results. J. Transp. Eng. 129(6), 664–672 (2003)CrossRefGoogle Scholar
  32. 32.
    Zhang, C., Li, F., Jestes, J.: Efficient parallel knn joins for large data in mapreduce. In: Proceedings of the 15th International Conference on Extending Database Technology, pp. 38–49. ACM (2012)Google Scholar
  33. 33.
    Zhu, H., Zhu, Y., Li, M., Ni, L.M.: Hero: Online real-time vehicle tracking in shanghai. In: IEEE The 27th Conference on Computer Communications, INFOCOM 2008. IEEE (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Cristian Chilipirea
    • 1
  • Andreea Petre
    • 1
  • Ciprian Dobre
    • 1
  • Florin Pop
    • 1
  • Fatos Xhafa
    • 2
  1. 1.University Politehnica of BucharestBucharestRomania
  2. 2.Universitat Politecnica de CatalunyaBarcelonaSpain

Personalised recommendations