Abstract
In this paper, we consider load balancing and maintenance of distributed similarity search systems using locality sensitive hashing (LSH) in a structured peer-to-peer (P2P) network based on Distributed Hashing Table (DHT). LSH has been proven efficient in K-Nearest Neighbor (KNN) search in high dimensions. Recently, a number of schemes have been proposed to implement LSH over DHT-based P2P systems to process distributed similarity searches. We provide an efficient structure using virtual nodes to manage the multi-dimensional LSH bucket space in DHT peers and the maintenance algorithm, which improves load balancing in comparison with state-of-the-art techniques such as the virtual node algorithm. Here, we demonstrate effectiveness of the proposed method by experiments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Datar, M., Immorlica, N., Indyk, P., Mirrokni, V.S.: Locality-sensitive hashing scheme based on p-stable distributions. In: The Twentieth Annual Symposium on Computational Geometry, pp. 253–262. ACM, New York (2004)
Andoni, A., Indyk, P.: Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. In: The 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS 06), pp. 459–468. IEEE Press, New York (2006)
Stoica, I., Morris R., Karger, D., Kaashoek, M.F., Balakrishnan, H.: Chord: a scalable peer-to-peer lookup service for internet applications. In: 2001 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM 01), pp. 149–160. ACM, New York (2001)
Ugo, M., Matteo, S.: Network-conscious π-calculus—a model of pastry. Electron. Notes Theor. Comput. Sci. 312, 3–17 (2015)
Li, J., Li, Q., Liu, C.: Community-based collaborative information system for emergency management. Comput. Oper. Res. 42(2), 116–124 (2014)
Haghani, P., Michel, S., Aberer, K.: Distributed similarity search in high dimensions using locality sensitive hashing. In: The 12th International Conference on Extending Database Technology: Advances in Database Technology (EDBT 09), pp. 744–755. ACM, New York (2009)
Lv, Q., Josephson, W., Wang, Z., Charikar, M., Li, K.: Multi-probe LSH: efficient indexing for high-dimensional similarity search. In: The 33rd International Conference on Very Large Data Bases (VLDB 07), pp. 950–961. ACM, New York (2007)
Alexandr, A., Piotr, I., Huy, L., Ilya, R.: Beyond locality sensitive hashing. In: The 25th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA 14), pp. 1018–1028. ACM, New York (2014)
Bawa, M., Condie, T., Ganesan, P.: LSH forest: self-tuning indexes for similarity search. In: The 14th International Conference on World Wide Web (WWW 05), pp. 651–660. ACM, New York (2005)
Tang, Y., Zhou, S., Xu, J., Lee, W.: A lightweight multi-dimensional index for range queries over DHTs. IEEE Trans. Parallel Distrib. Syst. 22(12), 2046–2054 (2011)
Fotiou, N., Katsaros, K.V., Xylomenos, G., Polyzos, G.C.: An inter-domain topology aware overlay for the support of name-resolution services in the future Internet. Comput. Commun. 62, 13–22 (2015)
Sidhu, A., Kinger, S.: Analysis of load balancing techniques in cloud computing. Int. J. Comput. Technol. 4(2), 60–66 (2013)
Dabek, F., Kaashoek, M.F., Karger, D., Morris, R., Stoica, I.: Wide-area cooperative storage with CFS. In: The Eighteenth ACM Symposium on Operating Systems Principles (SOSP 01), pp. 202–215. ACM, New York (2001)
Acknowledgements
This work is supported by National High Technology Research and Development Program of China (No. 2014AA015204), Technology Innovation Program of Shandong Province of China (No. 2014XGA06014) and Yantai Technology Research and Development Program (No. 2014LGS005).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer Science+Business Media Singapore
About this paper
Cite this paper
Liu, DW., Yu, ZH. (2016). Towards Load Balance and Maintenance in a Structured P2P Network for Locality Sensitive Hashing. In: Hussain, A. (eds) Electronics, Communications and Networks V. Lecture Notes in Electrical Engineering, vol 382. Springer, Singapore. https://doi.org/10.1007/978-981-10-0740-8_46
Download citation
DOI: https://doi.org/10.1007/978-981-10-0740-8_46
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-0738-5
Online ISBN: 978-981-10-0740-8
eBook Packages: EngineeringEngineering (R0)