Abstract
With the advent of the internet of things and industry 4.0 an enormous amount of data is produced at the edge of the network. Due to a lack of computing power, this data is currently send to the cloud where centralized machine learning models are trained to derive higher level knowledge. With the recent development of specialized machine learning hardware for mobile devices, a new era of distributed learning is about to begin that raises a new research question: How can we search in distributed machine learning models? Machine learning at the edge of the network has many benefits, such as low-latency inference and increased privacy. Such distributed machine learning models can also learn personalized for a human user, a specific context, or application scenario. As training data stays on the devices, control over possibly sensitive data is preserved as it is not shared with a third party. This new form of distributed learning leads to the partitioning of knowledge between many devices which makes access difficult. In this paper we tackle the problem of finding specific knowledge by forwarding a search request (query) to a device that can answer it best. To that end, we use a entropy based quality metric that takes the context of a query and the learning quality of a device into account. We show that our forwarding strategy can achieve over 95% accuracy in a urban mobility scenario where we use data from 30 000 people commuting in the city of Trento, Italy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Mark Gurman; BloombergTechnology, Apple Is Working on a Dedicated Chip to Power AI on Devices: https://www.bloomberg.com/news/articles/2017-05-26/apple-said-to-plan-dedicated-chip-to-power-ai-on-devices.
- 2.
For better readability we do not state all fields of the query here (i.e. \(H\), \(R\), \(Q\), \(\overrightarrow{N}\)).
References
Cisco global cloud index : Forecast and methodology, 2013–2018. Online (2014)
Albert, R., Barabási, A.-L.: Statistical mechanics of complex networks. Rev. Mod. Phys. 74(1), 47 (2002)
Amaral, L.A.N., Scala, A., Barthelemy, M., Stanley, H.E.: Classes of small-world networks. Proc. Natl. Acad. Sci. 97, 11149–11152 (2000)
Ashbrook, D., Starner, T.: Using GPS to learn significant locations and predict movement across multiple users. Pers. Ubiquit. Comput. 7(5), 275–286 (2003)
Bach, T., Tariq, M.A., Mayer, C., Rothermel, K.: Utilizing the hive mind – how to manage knowledge in fully distributed environments. In: Debruyne, C., Panetto, H., Meersman, R., Dillon, T., Weichhart, G., An, Y., Ardagna, C.A. (eds.) OTM 2015 Conferences. LNCS, vol. 9415, pp. 219–236. Springer, Cham (2015). Christophe Debruyne
Batko, M., Gennaro, C., Zezula, P.: A scalable nearest neighbor search in P2P systems. In: Ng, W.S., Ooi, B.-C., Ouksel, A.M., Sartori, C. (eds.) DBISP2P 2004. LNCS, vol. 3367, pp. 79–92. Springer, Heidelberg (2005). doi:10.1007/978-3-540-31838-5_6
Bharambe, A.R., Agrawal, M., Seshan, S.: Mercury: supporting scalable multi-attribute range queries. ACM SIGCOMM Comput. Comm. Rev. (2004)
Chen, D., Zhou, J., Le, J.: Reverse nearest neighbor search in peer-to-peer systems. In: Larsen, H.L., Pasi, G., Ortiz-Arroyo, D., Andreasen, T., Christiansen, H. (eds.) FQAS 2006. LNCS, vol. 4027, pp. 87–96. Springer, Heidelberg (2006). doi:10.1007/11766254_8
Domingos, P.: A few useful things to know about machine learning. Commun. ACM 55(10), 78–87 (2012)
Ganesan, P., Yang, B., Garcia-Molina, H.: One torus to rule them all: multi-dimensional queries in P2P systems. In: Proceedings of the 7th International Workshop on the Web and Databases: Colocated with ACM SIGMOD/PODS 2004. ACM (2004)
Garcia Lopez, P., Montresor, A., Epema, D., Datta, A., Higashino, T., Iamnitchi, A., Barcellos, M., Felber, P., Riviere, E.: Edge-centric computing: vision and challenges. ACM SIGCOMM Comput. Commun. Rev. 45(5), 37–42 (2015)
Heigold, G., Vanhoucke, V., Senior, A., Nguyen, P., Ranzato, M., Devin, M., Dean, J.: Multilingual acoustic models using distributed deep neural networks. In: IEEE ICASSP (2013)
Hong, K., Lillethun, D., Ramachandran, U., Ottenwälder, B., Koldehofe, B.: Mobile fog: a programming model for large-scale applications on the internet of things. In: Proceedings of the Second ACM SIGCOMM Workshop on Mobile Cloud Computing, pp. 15–20. ACM (2013)
World’s biggest data breaches (2015). informationisbeautiful.net
Khan, R., Khan, S.U., Zaheer, R., Khan, S.: Future internet: the internet of things architecture, possible applications and key challenges. In: Proceedings of the FIT (2012)
Kienzle, M.G.: Cognitive technologies for smarter cities. In: Proceedings of the ICDCS (2016)
Li, M., Lee, W.-C., Sivasubramaniam, A.: Semantic small world: an overlay network for peer-to-peer search. In: Proceedings of the ICNP (2004)
Li, M., Lee, W.-C., Sivasubramaniam, A., Zhao, J.: Supporting k nearest neighbors query on high-dimensional data in P2P systems. FCS 2(3), 234–247 (2008)
Malkov, Y., Ponomarenko, A., Logvinov, A., Krylov, V.: Approximate nearest neighbor algorithm based on navigable small world graphs. Inf. Syst. 45, 61–68 (2014)
Manning, C.D., Schütze, H., et al.: Foundations of Statistical Natural Language Processing, vol. 999. MIT Press, Cambridge (1999)
Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., Byers, A.H.: Big data: the next frontier for innovation, competition, and productivity (2011)
Mayer, C., Tariq, M.A., Li, C., Rothermel, K.: GrapH: heterogeneity-aware graph computation with adaptive partitioning. In: Proceedings of the ICDCS (2016)
Mayer, R., Gupta, H., Saurez, E., Ramachandran, U.: The fog makes sense: enabling social sensing services with limited internet connectivity. In: Proceedings of the 2nd International Workshop on Social Sensing. ACM (2017)
Mayer, R., Koldehofe, B., Rothermel, K.: Predictable low-latency event detection with parallel complex event processing. IEEE Internet of Things Journal 2(4), 274–286 (2015)
McMahan, B., Ramage, D.: Federated learning: Collaborative machine learning without centralized training data. Technical report, Google (2017)
Montresor, A.: Reflecting on the past, preparing for the future: from peer-to-peer to edge-centric computing. In: Proceedings of the ICDCS (2016)
Montresor, A., Jelasity, M.: Peersim: a scalable P2P simulator. In: IEEE Ninth International Conference on Peer-to-Peer Computing, P2P 2009, pp. 99–100. IEEE (2009)
Müller, W., Henrich, A.: Fast retrieval of high-dimensional feature vectors in P2P networks using compact peer data summaries. In: Proceedings of the ACM SIGMM International Workshop on Multimedia Information Retrieval, pp. 79–86. ACM (2003)
Poxrucker, A., Bahle, G., Lukowicz, P.: Towards a real-world simulator for collaborative distributed learning in the scenario of urban mobility. In: Proceedings of the SASOW 2014 (2014)
Ratnasamy, S., Francis, P., Handley, M., Karp, R., Shenker, S.: A scalable content-addressable network. ACM SIGCOMM Comput. Commun. Rev. 31(4), 161–172 (2001)
Rowstron, A., Druschel, P.: Pastry: scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In: Guerraoui, R. (ed.) Middleware 2001. LNCS, vol. 2218, pp. 329–350. Springer, Heidelberg (2001). doi:10.1007/3-540-45518-3_18
Schalkwyk, J., Beeferman, D., Beaufays, F., Byrne, B., Chelba, C., Cohen, M., Kamvar, M., Strope, B.: Your word is my command: Google search by voice: a case study. In: Neustein, A. (ed.) Advances in Speech Recognition, pp. 61–90. Springer, Boston (2010). doi:10.1007/978-1-4419-5951-5_4
Schmidt, C., Parashar, M.: Squid: enabling search in DHT-based systems. J. Parallel Distrib. Comput. 68(7), 962–975 (2008)
Shu, Y., Ooi, B.C., Tan, K.-L., Zhou, A.: Supporting multi-dimensional range queries in peer-to-peer systems. In: Proceedings of the P2P. IEEE (2005)
Stoica, I., Morris, R., Karger, D., Kaashoek, M.F., Balakrishnan, H.: Chord: a scalable peer-to-peer lookup service for internet applications. ACM SIGCOMM Comput. Commun. Rev. 31(4), 149–160 (2001)
Sutton, C., McCallum, A.: An Introduction to Conditional Random Fields for Relational Learning. Introduction to Statistical Relational Learning, vol. 2. MIT Press, Cambridge (2006)
Tang, C., Xu, Z., Mahalingam, M.: pSearch: information retrieval in structured overlays. ACM SIGCOMM Comput. Commun. Rev. 33(1), 89–94 (2003)
Witschel, H.F.: Content-oriented topology restructuring for search in P2P networks. Technical report, University of Leipzig, Germany (2005)
Ziegeldorf, J.H., Morchon, O.G., Wehrle, K.: Privacy in the internet of things: threats and challenges. Secur. Commun. Netw. 7(12), 2728–2742 (2014)
Acknowledgment
The authors would like to thank the European Union’s Seventh Framework Programme for partially funding this research through the ALLOW Ensembles project (project 600792).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Bach, T., Tariq, M.A., Mayer, R., Rothermel, K. (2017). Knowledge Is at the Edge! How to Search in Distributed Machine Learning Models. In: Panetto, H., et al. On the Move to Meaningful Internet Systems. OTM 2017 Conferences. OTM 2017. Lecture Notes in Computer Science(), vol 10573. Springer, Cham. https://doi.org/10.1007/978-3-319-69462-7_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-69462-7_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-69461-0
Online ISBN: 978-3-319-69462-7
eBook Packages: Computer ScienceComputer Science (R0)