Skip to main content
SpringerLink
Log in

Large scale arbitrary search with rendezvous search systems

  • Published:
Peer-to-Peer Networking and Applications Aims and scope Submit manuscript

Abstract

As distributed systems are getting larger and more complex, search facilities for finding data and services within the system become crucial. Users expect search engines to deal with complex query languages like keyword search, SQL, or XPath. At the same time, application developers cannot be expected to come up with distributed versions of those query languages from scratch. Rendezvous search systems are a scalable solution to this problem. By separating the query processing from the network communication, existing libraries for query processing can be easily reused. Rendezvous search systems have been successfully deployed in data center environments like the Google web search engine and are a popular research area for arbitrary search in peer-to-peer environments. Due to the decentralized nature of peer-to-peer systems, a high degree of self-organization and self-optimization can be found in such solutions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. An expander graph is a sparse graph with strong connectivity properties. Intuitively, every sub-graph that is not too large has many outgoing edges. Random graphs are typically expander graphs.

  2. A multigraph is a graph that is allowed to have multiple edges between the same two vertices.

References

  1. Barroso LA, Dean J, Hölzle U (2003) Web search for a planet: the google cluster architecture. IEEE Micro 23(2):22–28

    Article  Google Scholar 

  2. Baumgart I, Heep B, Krause S (2009) Oversim: a scalable and flexible overlay framework for simulation and real network applications. In: IEEE ninth international conference on peer-to-peer computing, 2009. P2P’09. IEEE, pp 87–88

  3. Bonifati A, Matrangolo U, Cuzzocrea A, Jain M (2004) Xpath lookup queries in p2p networks. In: WIDM ’04: proceedings of the 6th annual ACM international workshop on Web information and data management. ACM Press, New York, pp 48– 55

    Chapter  Google Scholar 

  4. Bradler D, Krumov L, Mühlhäuser M, Kangasharju J (2011) Pathfinder: efficient lookups and efficient search in peer-to-peer networks. In: Proceedings of the 12th international conference on distributed computing and networking, ICDCN 2011, vol 6522, pp 77–82

  5. Brewer EA (2000) Towards robust distributed systems (invited talk). In: Proceedings of the nineteenth annual ACM symposium on principles of distributed computing, PODC ’00. ACM, New York, p 7

  6. Ceri S, Negri M, Pelagatti G (1982) Horizontal data partitioning in database design. In: Proceedings of the 1982 ACM SIGMOD international conference on management of data, SIGMOD ’82. ACM, New York, pp 128–136

  7. Cheung SY, Ammar MH, Ahamad M (1992) The grid protocol: a high performance scheme for maintaining replicated data. IEEE Trans Knowl Data Engeering 4:582–592

    Article  Google Scholar 

  8. Choi TW, Boykin PO (2010) Deetoo: scalable unstructured search built on a structured overlay. In: HOTP2P - international workshop on hot topics in peer-to-peer systems. IEEE Computer Society, Los Alamitos

  9. Dabek F, Zhao B, Druschel P, Kubiatowicz J, Stoica I (2003) Towards a common api for structured peer-to-peer overlays. In: Kaashoek M, Stoica I (eds) Peer-to-peer systems II. Lecture notes in computer science, vol 2735. Springer Berlin Heidelberg, pp 33–44

  10. Ferreira RA, Ramanathan MK, Awan A, Grama A, Jagannathan S (2005) Search with probabilistic guarantees in unstructured peer-to-peer networks. In: Proceedings of P2P’05. IEEE Computer Society, Washington, DC, pp 165–172

    Google Scholar 

  11. Gao J (2004) A distributed and scalable peer-to-peer content discovery system supporting complex queries. Ph.D. thesis, Carnegie Mellon University, Pittsburgh

  12. Gao J, Steenkiste P (2002) Rendezvous points-based scalable content discovery with load balancing. In: Proceedings of the fourth international COST264 workshop on networked group communication, NGC 2002. ACM, pp 71–78

  13. Gao J, Steenkiste P (2004) Design and evaluation of a distributed scalable content discovery system. IEEE J Sel Areas Commun 22(1):54–66

    Article  Google Scholar 

  14. Gilbert S, Lynch N (2002) Brewer’s conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News 33:51–59

    Article  Google Scholar 

  15. Girdzijauskas S, Galuba W, Darlagiannis V, Datta A, Aberer K (2011) Fuzzynet: ringless routing in a ring-like structured overlay. Peer-to-Peer Netw Appl 4(3):259–273

    Article  Google Scholar 

  16. Gkantsidis C, Mihail M, Saberi A (2004) Random walks in peer-to-peer networks. In: INFOCOM 2004. Twenty-third annual joint conference of the IEEE computer and communications societies, vol 1, p 4

  17. Hautakorpi J, Schultz G (2010) A feasibility study of an arbitrary search in structured peer-to-peer networks. In: Proceedings of 19th International conference on computer communications and networks (ICCCN). IEEE, pp 1–8

  18. Helal AA, Heddaya AA, Bhargava BB (1996) Replication techniques in distributed systems. Kluwer Academic Publishers, Norwell

    MATH  Google Scholar 

  19. Kan G (2001) Gnutella. In: Oram A (ed) Peer-to-peer - harnessing the power of disruptive technologies, 1st edn. OŔeilly & Associates, Inc., Sebastopol, pp 62–79

    Google Scholar 

  20. Kang C (2011) Survey of search and optimization of p2p networks. Peer-to-Peer Netw Appl 4:211–218

    Article  Google Scholar 

  21. Knuth DE (1998) 6.5 retrieval on secondary keys. In: The art of computer programming, vol 3: sorting and searching, 2nd edn. Addison-Wesley, Reading

  22. Krumov L (2010) Local structures determine performance within complex networks. Ph.D. thesis, Technische Universität

    Google Scholar 

  23. Leng C (2012) BubbleStorm: replication, updates, and consistency in rendezvous information systems. Ph.D. thesis, Technische Universität Darmstadt

  24. Leng C, Lehn M, Rehner R, Buchmann A (2011) Designing a testbed for large-scale distributed systems. In: Proceedings of ACM SIGCOMM1́1. ACM, New York. Poster

  25. Leng C, Terpstra WW, Kemme B, Stannat W, Buchmann AP (2008) Maintaining replicas in unstructured P2P systems. In: Proceedings of the 4th ACM international conference on emerging networking experiments and technologies (ACM CoNEXT 2008)

  26. Li J, Loo B, Hellerstein J, Kaashoek F, Karger D, Morris R (2003) On the feasibility of peer-to-peer web indexing and search. In: 2nd international workshop on peer-to-peer systems, IPTPS ’03

  27. Lopes P, Ferreira RA (2010) Splitquest: controlled and exhaustive search in peer-to-peer networks. In: Proceedings of the 9th international conference on peer-to-peer systems, IPTPS’10. USENIX Association, Berkeley, pp 6–6

    Google Scholar 

  28. Massoulié L, Le Merrer E, Kermarrec AM, Ganesh A (2006) Peer counting and sampling in overlay networks: random walk methods. In: Proceedings of the twenty-fifth annual ACM symposium on principles of distributed computing, PODC ’06. ACM, New York, pp 123–132

    Chapter  Google Scholar 

  29. Morton GM (1966) A computer oriented geodetic database and a new technique in file sequencing. IBM, Ottawa

  30. Mosk-Aoyama D, Shah D (2006) Computing separable functions via gossip. In: Proceedings of the twenty-fifth annual ACM symposium on principles of distributed computing, PODC ’06. ACM, New York, pp 113–122

    Chapter  Google Scholar 

  31. Raiciu C (2011) Roar: increasing the flexibility and performance of distributed search. Ph.D. thesis, University College London, London

  32. Raiciu C, Huici F, Handley M, Rosenblum DS (2009) Roar: increasing the flexibility and performance of distributed search. In: Proceedings of SIGCOMM’09. ACM, New York, pp 291–302

    Google Scholar 

  33. Raiciu C, Rosenblum DS, Handley M (2007) Distributed online filtering. In: Proceedings of the 2007 conference on applications, technologies, architectures, and protocols for computer communications, SIGCOMM ’07. ACM, New York, pp 15–16

    Google Scholar 

  34. Rhea S, Geels D, Roscoe T, Kubiatowicz J (2004) Handling churn in a dht. In: Proceedings of the annual conference on USENIX annual technical conference, ATC ’04. USENIX Association, Berkeley, pp 10–10

    Google Scholar 

  35. Risson J, Moors T (2006) Survey of research towards robust peer-to-peer networks: search methods. Comput Netw 50(17): 3485–3521

    Article  MATH  Google Scholar 

  36. Sarshar N, Boykin PO, Roychowdhury VP (2004) Percolation search in power law networks: making unstructured peer-to-peer networks scalable. In: Proceedings of P2P’04. IEEE Computer Society, Washington, pp 2–9

    Google Scholar 

  37. Stoica I, Morris R, Karger D, Kaashoek MF, Balakrishnan H (2001) Chord: a scalable peer-to-peer lookup service for internet applications. In: Proceedings of the 2001 conference on applications, technologies, architectures, and protocols for computer communications, SIGCOMM ’01. ACM, New York, pp 149–160

    Chapter  Google Scholar 

  38. Stutzbach D, Rejaie R, Sen S (2008) Characterizing unstructured overlay topologies in modern p2p file-sharing systems. IEEE/ACM Trans Netw 16: 267–280

    Article  Google Scholar 

  39. Terpstra WW (2006) Distributed cartesian product. Technische Universitt Darmstadt, Diploma thesis

  40. Terpstra WW, Behnel S, Fiege L, Kangasharju J, Buchmann A (2004) Bit zipper rendezvous—optimal data placement for general p2p queries. In: EDBT 04 workshop on peer-to-peer computing & dataBases

  41. Terpstra WW, Kangasharju J, Leng C, Buchmann AP (2007) BubbleStorm: resilient, probabilistic, and exhaustive peer-to-peer search. In: Proceedings of the 2007 conference on applications, technologies, architectures, and protocols for computer communications, SIGCOMM ’07. ACM, New York, pp 49–60

    Chapter  Google Scholar 

  42. Terpstra WW, Leng C, Buchmann AP (2007) Brief announcement: practical summation via gossip. In: Proceedings of the twenty-sixth annual ACM symposium on principles of distributed computing, PODC ’07. ACM, New York, pp 390–391

    Chapter  Google Scholar 

  43. Terpstra WW, Leng C, Buchmann AP (2007) BubbleStorm: analysis of probabilistic exhaustive search in a heterogeneous peer-to-peer system. Tech Rep TUD-CS-2007-2, Technische Universität Darmstadt, Fachbereich Informatik. Darmstadt, Germany

    Google Scholar 

  44. Wendl MC (2003) Collision probability between sets of random variables. Stat & Probab Lett 64(3):249–254

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. International Computer Science Institute, Berkeley, CA, 94704, United States

    Christof Leng

Authors
  1. Christof Leng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christof Leng.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Leng, C. Large scale arbitrary search with rendezvous search systems. Peer-to-Peer Netw. Appl. 8, 229–240 (2015). https://doi.org/10.1007/s12083-014-0247-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12083-014-0247-5

Keywords

Search

Navigation