Skip to main content
SpringerLink
Log in

DKNNS: Scalable and accurate distributed K nearest neighbor search for latency-sensitive applications

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

To reduce the access latencies of end hosts, latency-sensitive applications need to choose suitably close service machines to answer the access requests from end hosts. Distributed K nearest neighbor search locates K service machines closest to end hosts, which can efficiently optimize the access latencies for end hosts. Existing work has weakness in terms of the accuracy and scalability. According to the scalable and accurate K nearest neighbor search problem, we propose a distributed K nearest neighbor search method called DKNNS in this paper. Service machines are organized into a locality-aware multilevel ring. DKNNS first locates a service machine that starts the search process based on a farthest neighbor search scheme, then discovers K nearest service machines based on a backtracking approach within the proximity region containing the target in the latency space. Theoretical analysis, simulation results and deployment experiments on the PlanetLab show that, DKNNS can determine K approximately optimal service machines, with modest completion time and query loads. Finally, DKNNS is also quite stable that can be used for reducing frequent searches by caching found nearest neighbors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Agarwal S, Lorch J. Matchmaking for online games and other latency-sensitive P2P systems. In: SIGCOMM’ 09, Barcelona, Spain, ACM, 2009. 315–326

  2. Greenberg A, Hamilton J, Maltz D, et al. The cost of a cloud: research problems in data center networks. Comput Commun Rev, 2009, 39: 68–73

    Article  Google Scholar 

  3. Beigbeder T, Coughlan R, Lusher C, et al. The effects of loss and latency on user performance in unreal tournament 2003. In: NetGames’ 04, Portland, Oregon, USA, ACM, 2004

  4. Szigeti T, Hattingh C. Quality of service design overview. http://www.ciscopress.com/articles/article.asp?p=357102&seqNum=2.2004

  5. Wang Y J, Li X Y. Network distance prediction technology research. J Software, 2009, 20: 1574–1590

    Article  Google Scholar 

  6. Xing C Y, Chen M. Techniques of network distance prediction. J Software, 2009, 20: 2470–2482

    Article  Google Scholar 

  7. Zhang R, Tang C, Hu Y, et al. Impact of the inaccuracy of distance prediction algorithms on Internet applications—an analytical and comparative study. In: INFOCOM’06, Barcelona, Catalunya, Spain, IEEE, 2006. 1–12

  8. Cox R, Dabek F, Kaashoek F, et al. Practical, distributed network coordinates. ACM SIGCOMM Comput Commun Rev, 2004, 34: 113–118

    Article  Google Scholar 

  9. Choffnes D, Sanchez M, Bustamante F. Network positioning from the edge—An empirical study of the effectiveness of network positioning in P2P systems. In: INFOCOM’10, San Diego, CA, USA. IEEE, 2010

  10. Wang G, Zhang B, Ng E. Towards network triangle inequality violation aware distributed systems. In: Internet Measurement Comference, San Diego, California, USA, ACM, 2007. 175–188

  11. Chen Y, Wang X, Song X, et al. Phoenix: Towards an accurate, practical and decentralized network coordinate system. In: Proc of IFIP/TC6 Networking 2009 (Networking’09), Aachen, Germany, 2009

  12. Xing C Y, Chen M. A hierarchical network distance prediction mechanism. Chin J Comput, 2010, 33: 356–364

    Article  Google Scholar 

  13. Madhyastha H, Isdal T, Piatek M, et al. iPlane: An information plane for distributed services. In: OSDI, Seattle, WA, USA, USENIX Association, 2006. 367–380

  14. Madhyastha H, Katz-Bassett E, Anderson T, et al. iPlane Nano: Path prediction for peer-to-peer applications. In: Proceedings of the Usenix Conference on Networked Systems Design and Implementation (NSDI), Boston, Massachusetts, USA, USENIX Association, 2009. 137–152

  15. Xu Z, Sharma P, Lee S. Netvigator: Scalable network proximity estimation. HP Laboratories Technical Report, HPL-2004-28. 2004

  16. Sharma P, Xu Z, Banerjee S, et al. Estimating network proximity and latency. ACM SIGCOMM Comput Commun Rev, 2006, 36: 39–50

    Article  Google Scholar 

  17. Ratnasamy S, Handley M, Karp R, et al. Topologically-aware overlay construction and server selection. In: INFOCOM. New York, USA, IEEE, 2002

  18. Waldvogel M, Rinaldi R. Efficient topology-aware overlay network. SIGCOMM Comput Commun Rev, 2003, 33: 101–106

    Article  Google Scholar 

  19. Costa M, Castro M, Rowstron A, et al. PIC: Practical Internet coordinates for distance estimation. In: Proceedings of the International Conference on Distributed Computing Systems, Hachioji, Tokyo, Japan, IEEE, 2004. 178–187

  20. Wong B, Slivkins A, Sirer E. Meridian: A lightweight network location service without virtual coordinates. In: SIGCOMM, Philadelphia, Pennsylvania, USA, ACM, 2005. 85–96

  21. Freedman M, Lakshminarayanan K, Maziéres D. OASIS: Anycast for any service. In: NSDI, San Jose, California, USA, USENIX Association, 2006

  22. Wendell P, Jiang W, Freedman M, et al. DONAR: decentralized server selection for cloud services. In: SIGCOMM, New Delhi, India, ACM, 2010. 231–242

  23. Vishnumurthy V, Francis P. On the difficulty of finding the nearest peer in P2P systems. In: Internet Measurement Conference, Vouliagmeni, Greece, ACM, 2008. 9–14

  24. Chávez E, Navarro G, Baeza-Yates R, et al. Searching in metric spaces. ACM Comput Surv, 2001, 33: 273–321

    Article  Google Scholar 

  25. Hjaltason G, Samet H. Index-driven similarity search in metric spaces (survey article). ACM Trans Database Syst, 2003, 28: 517–580

    Article  Google Scholar 

  26. Indyk P. Nearest neighbors in high-dimensional spaces. In: Handbook of Discrete and Computational Geometry. 2nd ed. Atlanta, GA: CRC Press LLC, 2004

    Google Scholar 

  27. Clarkson K. Nearest-neighbor searching and metric space dimensions. In: Shakhnarovich G, Darrell T, Indyk P, eds. Nearest-Neighbor Methods for Learning and Vision: Theory and Practice. Cambridge, Massachusetts: MIT Press, 2006. 15–59

    Google Scholar 

  28. Fraigniaud P, Lebhar E, Viennot L. The inframetric model for the internet. In: INFOCOM, Phoenix, AZ, USA, IEEE, 2008. 1085–1093

  29. Zhang B, Ng E, Nandi A, et al. Measurement-based analysis, modelling, and synthesis of the internet delay space. IEEE/ACM Trans Netw, 2010, 18: 229–242

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Key Laboratory for Parallel and Distributed Processing, School of Computer Science, National University of Defense Technology, Changsha, 410073, China

    YongQuan Fu & YiJie Wang

Authors
  1. YongQuan Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. YiJie Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to YongQuan Fu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fu, Y., Wang, Y. DKNNS: Scalable and accurate distributed K nearest neighbor search for latency-sensitive applications. Sci. China Inf. Sci. 56, 1–17 (2013). https://doi.org/10.1007/s11432-011-4449-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11432-011-4449-7

Keywords

Search

Navigation