Abstract
In this paper, we propose a large-scale object-based storage platform, named Gem, for data analytics in the Internet of Things (IoT). In Gem, a region covered by an IoT network is partitioned into sub-regions, each of which can be identified by a unique ID and known to all participants, which is automatic and economical. Gem can preserve object locality using type and location sensitive hashing, as well as dynamically distribute objects among a server cluster to keep load balancing. All data from the IoT can be stored as objects with attributes, methods and policies in Object Store Devices (OSDs). For some applications such as data analytics, application-specific operations are executed by OSDs, and only the results are returned to clients, rather than data files are read by the clients. Thus, the platform Gem is able to greatly reduce the overhead of data analytics applications in the IoT.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Atzori, L., Iera, A., Morabito, G.: The Internet of things: a survey. Comput. Netw. 54, 2787–2805 (2010)
Mesnier, M., Ganger, G.R., Riedel, E.: Object-based storage. IEEE Commun. Mag. 41(8), 84–90 (2003)
Xu, Q., Shen, H.T., Chen, Z., Cui, B., Zhou, X., Dai, Y.: Hybrid retrieval mechanisms in vehicle-based P2P networks. In: Proceedings of the International Conference on Computational Science (ICCS’09). Lecture Notes in Computer Science, vol. 5544, pp. 303–314. Springer, Berlin (2009)
Shvachko, K., Huang, H., Radia, S., et al.: The hadoop distributed filesystem. In: MSST 2010 (2010)
Stoica, I., Morris, R., Karger, D.R., Kaashoek, M.F., Balakrishnan, H.: Chord: a scalable peer-to-peer lookup service for internet applications. In: SIGCOMM, pp. 149–160 (2001)
Xu, Q., Shen, H.T., Chen, Z., Cui, B., Zhou, X., Dai, Y.: Hybrid information retrieval policies based on cooperative cache in mobile P2P networks. Front. Comput. Sci. China 3(3), 381–395 (2009)
Xu, Q., Arumugam, R.V., Yong, K.L., Mahadevan, S.: Efficient and scalable metadata management in EB-scale file systems. IEEE Trans. Parallel Distrib. Syst. 25(11), 2840–2850 (2014)
Chekuri, C., Khanna, S.: A polynomial time approximation scheme for the multiple knapsack problem. SIAM J. Comput. 35(3), 713–728 (2005)
Xu, Q., Arumugam, R.V., Yong, K.L., Mahadevan, S.: DROP: facilitating distributed metadata management in EB-scale storage systems. In: MSST, pp. 1–10 (2013)
Xu, Q., Arumugam R.V., Yong K.L., Wen, Y., Ong, Y.S.: C2: adaptive load balancing for metadata server cluster in cloud-scale storage systems. In: IES, pp. 195–209 (2015)
Acknowledgments
This work is supported by A*STAR under Grant No. DSI/14-300009.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Xu, Q., Aung, K.M.M., Zhu, Y., Yong, K.L. (2016). A Large-Scale Object-Based Active Storage Platform for Data Analytics in the Internet of Things. In: Park, J., Chao, HC., Arabnia, H., Yen, N. (eds) Advanced Multimedia and Ubiquitous Engineering. Lecture Notes in Electrical Engineering, vol 354. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-47895-0_49
Download citation
DOI: https://doi.org/10.1007/978-3-662-47895-0_49
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-47894-3
Online ISBN: 978-3-662-47895-0
eBook Packages: EngineeringEngineering (R0)