Skip to main content
Download PDF

Parallel Tree Search in Volunteer Computing: a Case Study

Journal of Grid Computing volume 16, pages 647–662 (2018)Cite this article

Abstract

While volunteer computing, as a restricted model of parallel computing, has proved itself to be a successful paradigm of scientific computing with excellent benefit on cost efficiency and public outreach, many problems it solves are intrinsically highly parallel. However, many efficient algorithms, including backtracking search, take the form of a tree search on an extremely uneven tree that cannot be easily parallelized efficiently in the volunteer computing paradigm. We explore in this article how to perform such searches efficiently on volunteer computing projects. We propose a parallel tree search scheme, and we describe two examples of its real-world implementation, Harmonious Tree and Odd Weird Search, both carried out at the volunteer computing project yoyo@home. To confirm the observed efficiency of our scheme, we perform a mathematical analysis, which proves that, under reasonable assumption that agrees with experimental observation, our scheme is only a constant multiplicative factor away from perfect parallelism. Details on improving the overall performance are also discussed.

References

  1. Abbott, B., Abbott, R., Adhikari, R., Ajith, P., Allen, B., Allen, G., Amin, R.S., Anderson, S.B., Anderson, W.G., Arain, M.A., et al.: Einstein@Home search for periodic gravitational waves in early S5 LIGO data. Phys. Rev. D 80(4), 042003 (2009)

    Article  Google Scholar 

  2. Aichholzer, O., Krasser, H.: Abstract order type extension and new results on the rectilinear crossing number. In: Proceedings of the Twenty-First Annual Symposium on Computational Geometry, pp. 91–98. ACM (2005)

  3. Anderson, D.P.: BOINC: a system for public-resource computing and storage. In: Proceedings. Fifth IEEE/ACM International Workshop on Grid Computing, pp. 4–10. IEEE (2004)

  4. Anderson, D.P., Cobb, J., Korpela, E., Lebofsky, M., Werthimer, D.: SETI@home: an experiment in public-resource computing. Commun. ACM 45(11), 56–61 (2002)

    Article  Google Scholar 

  5. Apt, K.: Principles of Constraint Programming. Cambridge University Press, Cambridge (2003)

    Book  Google Scholar 

  6. Bazinet, A.L., Cummings, M.P.: Subdividing long-running, variable-length analyses into short, fixed-length BOINC workunits. J. Grid Comput. 14(3), 429–441 (2016)

    Article  Google Scholar 

  7. Blelloch, G.E., Gibbons, P.B., Matias, Y.: Provably efficient scheduling for languages with fine-grained parallelism. J. ACM 46(2), 281–321 (1999)

    MathSciNet  Article  Google Scholar 

  8. Blumofe, R.D., Joerg, C.F., Kuszmaul, B.C., Leiserson, C.E., Randall, K.H., Zhou, Y.: Cilk: an efficient multithreaded runtime system. J. Parallel Distrib. Comput. 37(1), 55–69 (1996)

    Article  Google Scholar 

  9. Blumofe, R.D., Leiserson, C.E.: Scheduling multithreaded computations by work stealing. J. ACM 46(5), 720–748 (1999)

    MathSciNet  Article  Google Scholar 

  10. Bouguerra, M.S., Kondo, D., Trystram, D.: On the scheduling of checkpoints in desktop grids. In: 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), 2011, pp. 305–313. IEEE (2011)

  11. Clauset, A., Shalizi, C.R., Newman, M.E.J.: Power-law distributions in empirical data. SIAM Rev. 51(4), 661–703 (2009)

    MathSciNet  Article  Google Scholar 

  12. Dagum, L., Menon, R.: OpenMP: an industry standard API for shared-memory programming. IEEE Comput. Sci. Eng. 5(1), 46–55 (1998)

    Article  Google Scholar 

  13. Distler, A., Jefferson, C., Kelsey, T., Kotthoff, L.: The semigroups of order 10. In: Principles and Practice of Constraint Programming, pp. 883–899. Springer, Berlin (2012)

    Google Scholar 

  14. Distributed.net: Distributed.net Faq-O-Matic: why is progress of OGR-28 stubspaces 1 and 2 so slow? http://faq.distributed.net/cache/299.html, Cited: 21 Jan 2015

  15. Distributed.net: Distributed.net: Project OGR. http://www.distributed.net/OGR, Cited: 21 Jan 2015

  16. Drakakis, K., Iorio, F., Rickard, S., Walsh, J.: Results of the enumeration of Costas arrays of order 29. Adv. Math. Commun. 5(3), 547–553 (2011)

    MathSciNet  Article  Google Scholar 

  17. Estrada, T., Flores, D.A., Taufer, M., Teller, P.J., Kerstens, A., Anderson, D.P.: The effectiveness of threshold-based scheduling policies in BOINC projects. In: Second IEEE International Conference on e-Science and Grid Computing, 2006 (e-Science’06), p. 88. IEEE (2006)

  18. Fischetti, M., Monaci, M., Salvagnin, D.: Self-splitting of workload in parallel computation. In: International Conference on AI and OR Techniques in Constriant Programming for Combinatorial Optimization Problems, pp. 394–404. Springer, Berlin (2014)

    Chapter  Google Scholar 

  19. Gallian, J.A.: A dynamic survey of graph labeling. Electron. J. Comb. Dynamic Surveys, DS6 (2014)

    MathSciNet  MATH  Google Scholar 

  20. Golle, P., Mironov, I.: Uncheatable distributed computations. In: Proceedings of the 2001 Conference on Topics in Cryptology: The Cryptographer’s Track at RSA, pp. 425–440. Springer, Berlin (2001)

    Chapter  Google Scholar 

  21. Gomes, C.P., Selman, B., Crato, N., Kautz, H.: Heavy-tailed phenomena in satisfiability and constraint satisfaction problems. J. Autom. Reason. 24(1), 67–100 (2000)

    MathSciNet  Article  Google Scholar 

  22. Grama, A., Kumar, V.: State of the art in parallel search techniques for discrete optimization problems. IEEE Trans. Knowl. Data Eng. 11(1), 28–35 (1999)

    Article  Google Scholar 

  23. Heien, E.M., Anderson, D.P., Hagihara, K.: Computing low latency batches with unreliable workers in volunteer computing environments. J Grid Comput. 7(4), 501 (2009)

    Article  Google Scholar 

  24. Jaffar, J., Santosa, A.E., Yap, R.H.C., Zhu, K.Q.: Scalable distributed depth-first search with greedy work stealing. In: 16th IEEE International Conference on Tools with Artificial Intelligence, 2004, pp. 98–103. IEEE (2004)

  25. Javadi, B., Kondo, D., Vincent, J.M., Anderson, D.P.: Discovering statistical models of availability in large distributed systems: An empirical study of SETI@home. IEEE Trans. Parallel Distrib. Syst. 22(11), 1896–1903 (2011)

    Article  Google Scholar 

  26. Jia, H., Moore, C.: How much backtracking does it take to color random graphs? Rigorous results on heavy tails. In: Proceedings of Principles and Practice of Constraint Programming - CP 2004, pp. 472–476 (2004)

    Chapter  Google Scholar 

  27. Karp, R.M., Zhang, Y.: Randomized parallel algorithms for backtrack search and branch-and-bound computation. J. ACM 40(3), 765–789 (1993)

    MathSciNet  Article  Google Scholar 

  28. Mersenne Research Inc.: GIMPS history. http://www.mersenne.org/various/history.php, Cited: 21 Jan 2015

  29. Nurmi, D., Brevik, J., Wolski, R.: Modeling machine availability in enterprise and wide-area distributed computing environments. In: Euro-Par 2005 Parallel Processing, pp. 612–612. Springer, Berlin (2005)

    Chapter  Google Scholar 

  30. Pataki, M., Marosi, A.C.: Searching for translated plagiarism with the help of desktop grids. J Grid Comput. 11(1), 149–166 (2013)

    Google Scholar 

  31. Rechenkraft.net e.V.: yoyo@home. http://www.rechenkraft.net/wiki/index.php?title=Yoyo%40home_en, Cited: 21 Jan 2015

  32. Régin, J.C., Rezgui, M., Malapert, A.: Embarrassingly parallel search. In: Principles and Practice of Constraint Programming, pp. 596–610. Springer, Berlin (2013)

    Google Scholar 

  33. Reinefeld, A., Schnecke, V.: Work-load balancing in highly parallel depth-first search. In: Proceedings of the Scalable High-Performance Computing Conference 1994, pp. 773–780. IEEE (1994)

  34. Sonnek, J., Chandra, A., Weissman, J.: Adaptive reputation-based scheduling on unreliable distributed infrastructures. IEEE Trans. Parallel Distrib. Syst. 18(11) (2007)

    Article  Google Scholar 

  35. Stolee, D.: TreeSearch user guide. http://www.math.uiuc.edu/stolee/SearchLib/TreeSearch.pdf (2011)

  36. Thain, D., Tannenbaum, T., Livny, M.: Distributed computing in practice: the Condor experience. Concurr. Comput. - Pract. E 17(2–4), 323–356 (2005)

    Article  Google Scholar 

  37. Wright, R.A., Richmond, B., Odlyzko, A., McKay, B.D.: Constant time generation of free trees. SIAM J. Comput. 15(2), 540–548 (1986)

    MathSciNet  Article  Google Scholar 

Download references

Acknowledgements

Open access funding provided by Graz University of Technology. The first author was partially supported by IRIF (previously LIAFA) of Université Paris Diderot. Both authors thank Prof. David Anderson, Prof. Anne Benoit, Linxiao Chen and the anonymous referees for their useful advises. We also thank the volunteers in yoyo@home that contributed to Harmonious Trees and Odd Weird Search.

Author information

Authors and Affiliations

  1. Institute of Discrete Mathematics, Technical University of Graz, Steyrergasse 30, Graz, 8010, Austria

    Wenjie Fang

  2. Rechenkraft.net e.V., Marburg, Germany

    Uwe Beckert

Authors
  1. Wenjie Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Uwe Beckert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenjie Fang.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Fang, W., Beckert, U. Parallel Tree Search in Volunteer Computing: a Case Study. J Grid Computing 16, 647–662 (2018). https://doi.org/10.1007/s10723-017-9411-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-017-9411-5

Keywords

  • Volunteer computing
  • Parallel tree search
  • Power law
  • Performance
  • Implementation