Advertisement

Journal of Grid Computing

, Volume 16, Issue 4, pp 647–662 | Cite as

Parallel Tree Search in Volunteer Computing: a Case Study

  • Wenjie Fang
  • Uwe Beckert
Open Access
Article
  • 299 Downloads

Abstract

While volunteer computing, as a restricted model of parallel computing, has proved itself to be a successful paradigm of scientific computing with excellent benefit on cost efficiency and public outreach, many problems it solves are intrinsically highly parallel. However, many efficient algorithms, including backtracking search, take the form of a tree search on an extremely uneven tree that cannot be easily parallelized efficiently in the volunteer computing paradigm. We explore in this article how to perform such searches efficiently on volunteer computing projects. We propose a parallel tree search scheme, and we describe two examples of its real-world implementation, Harmonious Tree and Odd Weird Search, both carried out at the volunteer computing project yoyo@home. To confirm the observed efficiency of our scheme, we perform a mathematical analysis, which proves that, under reasonable assumption that agrees with experimental observation, our scheme is only a constant multiplicative factor away from perfect parallelism. Details on improving the overall performance are also discussed.

Keywords

Volunteer computing Parallel tree search Power law Performance Implementation 

Notes

Acknowledgements

Open access funding provided by Graz University of Technology. The first author was partially supported by IRIF (previously LIAFA) of Université Paris Diderot. Both authors thank Prof. David Anderson, Prof. Anne Benoit, Linxiao Chen and the anonymous referees for their useful advises. We also thank the volunteers in yoyo@home that contributed to Harmonious Trees and Odd Weird Search.

References

  1. 1.
    Abbott, B., Abbott, R., Adhikari, R., Ajith, P., Allen, B., Allen, G., Amin, R.S., Anderson, S.B., Anderson, W.G., Arain, M.A., et al.: Einstein@Home search for periodic gravitational waves in early S5 LIGO data. Phys. Rev. D 80(4), 042003 (2009)CrossRefGoogle Scholar
  2. 2.
    Aichholzer, O., Krasser, H.: Abstract order type extension and new results on the rectilinear crossing number. In: Proceedings of the Twenty-First Annual Symposium on Computational Geometry, pp. 91–98. ACM (2005)Google Scholar
  3. 3.
    Anderson, D.P.: BOINC: a system for public-resource computing and storage. In: Proceedings. Fifth IEEE/ACM International Workshop on Grid Computing, pp. 4–10. IEEE (2004)Google Scholar
  4. 4.
    Anderson, D.P., Cobb, J., Korpela, E., Lebofsky, M., Werthimer, D.: SETI@home: an experiment in public-resource computing. Commun. ACM 45(11), 56–61 (2002)CrossRefGoogle Scholar
  5. 5.
    Apt, K.: Principles of Constraint Programming. Cambridge University Press, Cambridge (2003)CrossRefGoogle Scholar
  6. 6.
    Bazinet, A.L., Cummings, M.P.: Subdividing long-running, variable-length analyses into short, fixed-length BOINC workunits. J. Grid Comput. 14(3), 429–441 (2016)CrossRefGoogle Scholar
  7. 7.
    Blelloch, G.E., Gibbons, P.B., Matias, Y.: Provably efficient scheduling for languages with fine-grained parallelism. J. ACM 46(2), 281–321 (1999)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Blumofe, R.D., Joerg, C.F., Kuszmaul, B.C., Leiserson, C.E., Randall, K.H., Zhou, Y.: Cilk: an efficient multithreaded runtime system. J. Parallel Distrib. Comput. 37(1), 55–69 (1996)CrossRefGoogle Scholar
  9. 9.
    Blumofe, R.D., Leiserson, C.E.: Scheduling multithreaded computations by work stealing. J. ACM 46(5), 720–748 (1999)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Bouguerra, M.S., Kondo, D., Trystram, D.: On the scheduling of checkpoints in desktop grids. In: 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), 2011, pp. 305–313. IEEE (2011)Google Scholar
  11. 11.
    Clauset, A., Shalizi, C.R., Newman, M.E.J.: Power-law distributions in empirical data. SIAM Rev. 51(4), 661–703 (2009)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Dagum, L., Menon, R.: OpenMP: an industry standard API for shared-memory programming. IEEE Comput. Sci. Eng. 5(1), 46–55 (1998)CrossRefGoogle Scholar
  13. 13.
    Distler, A., Jefferson, C., Kelsey, T., Kotthoff, L.: The semigroups of order 10. In: Principles and Practice of Constraint Programming, pp. 883–899. Springer, Berlin (2012)Google Scholar
  14. 14.
    Distributed.net: Distributed.net Faq-O-Matic: why is progress of OGR-28 stubspaces 1 and 2 so slow? http://faq.distributed.net/cache/299.html, Cited: 21 Jan 2015
  15. 15.
    Distributed.net: Distributed.net: Project OGR. http://www.distributed.net/OGR, Cited: 21 Jan 2015
  16. 16.
    Drakakis, K., Iorio, F., Rickard, S., Walsh, J.: Results of the enumeration of Costas arrays of order 29. Adv. Math. Commun. 5(3), 547–553 (2011)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Estrada, T., Flores, D.A., Taufer, M., Teller, P.J., Kerstens, A., Anderson, D.P.: The effectiveness of threshold-based scheduling policies in BOINC projects. In: Second IEEE International Conference on e-Science and Grid Computing, 2006 (e-Science’06), p. 88. IEEE (2006)Google Scholar
  18. 18.
    Fischetti, M., Monaci, M., Salvagnin, D.: Self-splitting of workload in parallel computation. In: International Conference on AI and OR Techniques in Constriant Programming for Combinatorial Optimization Problems, pp. 394–404. Springer, Berlin (2014)CrossRefGoogle Scholar
  19. 19.
    Gallian, J.A.: A dynamic survey of graph labeling. Electron. J. Comb. Dynamic Surveys, DS6 (2014)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Golle, P., Mironov, I.: Uncheatable distributed computations. In: Proceedings of the 2001 Conference on Topics in Cryptology: The Cryptographer’s Track at RSA, pp. 425–440. Springer, Berlin (2001)CrossRefGoogle Scholar
  21. 21.
    Gomes, C.P., Selman, B., Crato, N., Kautz, H.: Heavy-tailed phenomena in satisfiability and constraint satisfaction problems. J. Autom. Reason. 24(1), 67–100 (2000)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Grama, A., Kumar, V.: State of the art in parallel search techniques for discrete optimization problems. IEEE Trans. Knowl. Data Eng. 11(1), 28–35 (1999)CrossRefGoogle Scholar
  23. 23.
    Heien, E.M., Anderson, D.P., Hagihara, K.: Computing low latency batches with unreliable workers in volunteer computing environments. J Grid Comput. 7(4), 501 (2009)CrossRefGoogle Scholar
  24. 24.
    Jaffar, J., Santosa, A.E., Yap, R.H.C., Zhu, K.Q.: Scalable distributed depth-first search with greedy work stealing. In: 16th IEEE International Conference on Tools with Artificial Intelligence, 2004, pp. 98–103. IEEE (2004)Google Scholar
  25. 25.
    Javadi, B., Kondo, D., Vincent, J.M., Anderson, D.P.: Discovering statistical models of availability in large distributed systems: An empirical study of SETI@home. IEEE Trans. Parallel Distrib. Syst. 22(11), 1896–1903 (2011)CrossRefGoogle Scholar
  26. 26.
    Jia, H., Moore, C.: How much backtracking does it take to color random graphs? Rigorous results on heavy tails. In: Proceedings of Principles and Practice of Constraint Programming - CP 2004, pp. 472–476 (2004)CrossRefGoogle Scholar
  27. 27.
    Karp, R.M., Zhang, Y.: Randomized parallel algorithms for backtrack search and branch-and-bound computation. J. ACM 40(3), 765–789 (1993)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Mersenne Research Inc.: GIMPS history. http://www.mersenne.org/various/history.php, Cited: 21 Jan 2015
  29. 29.
    Nurmi, D., Brevik, J., Wolski, R.: Modeling machine availability in enterprise and wide-area distributed computing environments. In: Euro-Par 2005 Parallel Processing, pp. 612–612. Springer, Berlin (2005)CrossRefGoogle Scholar
  30. 30.
    Pataki, M., Marosi, A.C.: Searching for translated plagiarism with the help of desktop grids. J Grid Comput. 11(1), 149–166 (2013)Google Scholar
  31. 31.
    Rechenkraft.net e.V.: yoyo@home. http://www.rechenkraft.net/wiki/index.php?title=Yoyo%40home_en, Cited: 21 Jan 2015
  32. 32.
    Régin, J.C., Rezgui, M., Malapert, A.: Embarrassingly parallel search. In: Principles and Practice of Constraint Programming, pp. 596–610. Springer, Berlin (2013)Google Scholar
  33. 33.
    Reinefeld, A., Schnecke, V.: Work-load balancing in highly parallel depth-first search. In: Proceedings of the Scalable High-Performance Computing Conference 1994, pp. 773–780. IEEE (1994)Google Scholar
  34. 34.
    Sonnek, J., Chandra, A., Weissman, J.: Adaptive reputation-based scheduling on unreliable distributed infrastructures. IEEE Trans. Parallel Distrib. Syst. 18(11) (2007)CrossRefGoogle Scholar
  35. 35.
    Stolee, D.: TreeSearch user guide. http://www.math.uiuc.edu/stolee/SearchLib/TreeSearch.pdf (2011)
  36. 36.
    Thain, D., Tannenbaum, T., Livny, M.: Distributed computing in practice: the Condor experience. Concurr. Comput. - Pract. E 17(2–4), 323–356 (2005)CrossRefGoogle Scholar
  37. 37.
    Wright, R.A., Richmond, B., Odlyzko, A., McKay, B.D.: Constant time generation of free trees. SIAM J. Comput. 15(2), 540–548 (1986)MathSciNetCrossRefGoogle Scholar

Copyright information

© The Author(s) 2017

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.Institute of Discrete MathematicsTechnical University of GrazGrazAustria
  2. 2.Rechenkraft.net e.V.MarburgGermany

Personalised recommendations