Enabling Vehicular Data with Distributed Machine Learning
Vehicular Data includes different facts and measurements made over a set of moving vehicles. Most of us use cars or public transportation for our work commute, daily routines and leisure. But, except of our destination, possible time of arrival and what is directly around us, we know very little about the traffic conditions in the city as a whole. Because all roads are connected in a vast network, events in other parts of town can and will directly affect us. The more we know about the traffic inside a city, the better decisions we can make. Vehicular measurements may contain a vast amount of information about the way our cities function. Information that can be used for more than improving our commute, it is indicative of other features of the city like the amount of pollution in different regions. All the information and knowledge we can extract, can be used to directly improve our life.
We live in a world where data is constantly generated and we store it and process it at an ever growing rate. Vehicular Data does not stray from this fact and is rapidly growing in size and complexity, with more and more ways to monitoring traffic, either from inside cars or from sensors placed on the road. Smartphones and in-car-computers are now common and they can produce a vast amount of data: it can identify a cars location, destination, current speed and even driving habits.
Machine learning is the perfect complement for Big Data, as large data sets can be rendered useless without methods to extract knowledge and information from them. Machine learning, currently a popular research topic, has a large number of algorithms design to achieve this task, of knowledge extraction. Most of these techniques and algorithms can be directly applied to Vehicular Data.
In this article we demonstrate how the use of a simple algorithm, k-Nearest Neighbors, can be used to extract valuable information from even a relatively small vehicular data set. Because of the vast size of most of our cities and the number of cars that are on their roads at any time of the day, standard machine learning systems do not manage to process data in a manner that would permit real time use of the extracted information. A solution to this problem is brought by distributed systems and cloud processing. By parallelizing and distributing machine learning algorithms we can use data at its highest potential and with little delay. Here, we show how this can be achieved by distributing the k-Nearest Neighbors machine learning algorithm over MPI. We hope this would motivate the research into other combinations of merging machine learning algorithms with Vehicular Data sets.
KeywordsBig data Machine learning MPI Cloud systems Distributed processing
This work was supported by the Romanian national project MobiWay, Project PN-II-PT-PCCA-2013-4-0321. The authors would like to thank reviewers for their constructive comments and valuable insights.
- 3.Apache: Mahout (2015). https://mahout.apache.org/
- 4.Apache: Spark mllib (2015). https://spark.apache.org/mllib/
- 5.Barrientos, R., Gómez, J., Tenllado, C., Prieto, M.: Heap based k-nearest neighbor search on gpus. In: Congreso Espanol de Informática (CEDI), pp. 559–566 (2010)Google Scholar
- 6.Biem, A., Bouillet, E., Feng, H., Ranganathan, A., Riabov, A., Verscheure, O., Koutsopoulos, H., Moran, C.: Ibm infosphere streams for scalable, real-time, intelligent transportation services. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, pp. 1093–1104. ACM (2010)Google Scholar
- 7.Bracciale, L., Bonola, M., Loreti, P., Bianchi, G., Amici, R., Rabuffi, A.: CRAWDAD data set roma/taxi (v. 2014–07-17), Jul 2014. Downloaded from http://crawdad.org/roma/taxi/
- 8.Brakatsoulas, S., Pfoser, D., Salas, R., Wenk, C.: On map-matching vehicle tracking data. In: Proceedings of the 31st International Conference on Very Large Data Bases, pp. 853–864. VLDB Endowment (2005)Google Scholar
- 9.Chadil, N., Russameesawang, A., Keeratiwintakorn, P.: Real-time tracking management system using gps, gprs and google earth. In: 5th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology, ECTI-CON 2008, vol. 1, pp. 393–396. IEEE (2008)Google Scholar
- 12.Dartmouth: Crowdad (2015). http://crawdad.org/
- 13.exploratorium: Cabspotting (2015). http://cabspotting.org/index.html
- 14.Feldman, D., Sugaya, A., Sung, C., Rus, D.: idiary: From gps signals to a text-searchable diary. In: Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems, p. 6. ACM (2013)Google Scholar
- 16.Gillick, D., Faria, A., DeNero, J.: Mapreduce: Distributed computing for machine learning. Berkley, 18 Dec 2006Google Scholar
- 18.IEEE, TomTom: Ieee icdm contest: Tomtom traffic prediction for intelligent gps navigation (2010). http://tunedit.org/challenge/IEEE-ICDM-2010
- 21.Mavromoustakis, C.X., Kormentzas, G., Mastorakis, G., Bourdena, A., Pallis, E., Rodrigues, J.: Context-oriented opportunistic cloud offload processing for energy conservation in wireless devices. In: Globecom Workshops (GC Wkshps), pp. 24–30. IEEE (2014)Google Scholar
- 22.Microsoft, R.: T-drive: Driving directions based on taxi traces (2015). http://research.microsoft.com/en-us/projects/tdrive/
- 23.Mousicou, P., Mavromoustakis, C.X., Bourdena, A., Mastorakis, G., Pallis, E.: Performance evaluation of dynamic cloud resource migration based on temporal and capacity-aware policy for efficient resource sharing. In: Proceedings of the 2nd ACM Workshop on High Performance Mobile Opportunistic Systems, pp. 59–66. ACM (2013)Google Scholar
- 24.OpenStreetMap: Openstreetmap (2015). https://www.openstreetmap.org
- 25.Papadakis, S.E., Stykas, V., Mastorakis, G., Mavromoustakis, C.X., et al.: A hyper-box approach using relational databases for large scale machine learning. In: 2014 International Conference on Telecommunications and Multimedia (TEMU), pp. 69–73. IEEE (2014)Google Scholar
- 27.Piórkowski, M., Sarafijanovic-Djukic, N., Grossglauser, M.: A parsimonious model of mobile partitioned networks with clustering. In: First International Communication Systems and Networks and Workshops, COMSNETS 2009, pp. 1–10. IEEE (2009)Google Scholar
- 28.Safar, M.: K nearest neighbor search in navigation systems. Mob. Inf. Syst. 1(3), 207–224 (2005)Google Scholar
- 29.Schubert, R., Richter, E., Wanielik, G.: Comparison and evaluation of advanced motion models for vehicle tracking. In: 2008 11th International Conference on Information Fusion, pp. 1–6. IEEE (2008)Google Scholar
- 32.Zhang, C., Li, F., Jestes, J.: Efficient parallel knn joins for large data in mapreduce. In: Proceedings of the 15th International Conference on Extending Database Technology, pp. 38–49. ACM (2012)Google Scholar
- 33.Zhu, H., Zhu, Y., Li, M., Ni, L.M.: Hero: Online real-time vehicle tracking in shanghai. In: IEEE The 27th Conference on Computer Communications, INFOCOM 2008. IEEE (2008)Google Scholar