Abstract
While volunteer computing, as a restricted model of parallel computing, has proved itself to be a successful paradigm of scientific computing with excellent benefit on cost efficiency and public outreach, many problems it solves are intrinsically highly parallel. However, many efficient algorithms, including backtracking search, take the form of a tree search on an extremely uneven tree that cannot be easily parallelized efficiently in the volunteer computing paradigm. We explore in this article how to perform such searches efficiently on volunteer computing projects. We propose a parallel tree search scheme, and we describe two examples of its real-world implementation, Harmonious Tree and Odd Weird Search, both carried out at the volunteer computing project yoyo@home. To confirm the observed efficiency of our scheme, we perform a mathematical analysis, which proves that, under reasonable assumption that agrees with experimental observation, our scheme is only a constant multiplicative factor away from perfect parallelism. Details on improving the overall performance are also discussed.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Abbott, B., Abbott, R., Adhikari, R., Ajith, P., Allen, B., Allen, G., Amin, R.S., Anderson, S.B., Anderson, W.G., Arain, M.A., et al.: Einstein@Home search for periodic gravitational waves in early S5 LIGO data. Phys. Rev. D 80(4), 042003 (2009)
Aichholzer, O., Krasser, H.: Abstract order type extension and new results on the rectilinear crossing number. In: Proceedings of the Twenty-First Annual Symposium on Computational Geometry, pp. 91–98. ACM (2005)
Anderson, D.P.: BOINC: a system for public-resource computing and storage. In: Proceedings. Fifth IEEE/ACM International Workshop on Grid Computing, pp. 4–10. IEEE (2004)
Anderson, D.P., Cobb, J., Korpela, E., Lebofsky, M., Werthimer, D.: SETI@home: an experiment in public-resource computing. Commun. ACM 45(11), 56–61 (2002)
Apt, K.: Principles of Constraint Programming. Cambridge University Press, Cambridge (2003)
Bazinet, A.L., Cummings, M.P.: Subdividing long-running, variable-length analyses into short, fixed-length BOINC workunits. J. Grid Comput. 14(3), 429–441 (2016)
Blelloch, G.E., Gibbons, P.B., Matias, Y.: Provably efficient scheduling for languages with fine-grained parallelism. J. ACM 46(2), 281–321 (1999)
Blumofe, R.D., Joerg, C.F., Kuszmaul, B.C., Leiserson, C.E., Randall, K.H., Zhou, Y.: Cilk: an efficient multithreaded runtime system. J. Parallel Distrib. Comput. 37(1), 55–69 (1996)
Blumofe, R.D., Leiserson, C.E.: Scheduling multithreaded computations by work stealing. J. ACM 46(5), 720–748 (1999)
Bouguerra, M.S., Kondo, D., Trystram, D.: On the scheduling of checkpoints in desktop grids. In: 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), 2011, pp. 305–313. IEEE (2011)
Clauset, A., Shalizi, C.R., Newman, M.E.J.: Power-law distributions in empirical data. SIAM Rev. 51(4), 661–703 (2009)
Dagum, L., Menon, R.: OpenMP: an industry standard API for shared-memory programming. IEEE Comput. Sci. Eng. 5(1), 46–55 (1998)
Distler, A., Jefferson, C., Kelsey, T., Kotthoff, L.: The semigroups of order 10. In: Principles and Practice of Constraint Programming, pp. 883–899. Springer, Berlin (2012)
Distributed.net: Distributed.net Faq-O-Matic: why is progress of OGR-28 stubspaces 1 and 2 so slow? http://faq.distributed.net/cache/299.html, Cited: 21 Jan 2015
Distributed.net: Distributed.net: Project OGR. http://www.distributed.net/OGR, Cited: 21 Jan 2015
Drakakis, K., Iorio, F., Rickard, S., Walsh, J.: Results of the enumeration of Costas arrays of order 29. Adv. Math. Commun. 5(3), 547–553 (2011)
Estrada, T., Flores, D.A., Taufer, M., Teller, P.J., Kerstens, A., Anderson, D.P.: The effectiveness of threshold-based scheduling policies in BOINC projects. In: Second IEEE International Conference on e-Science and Grid Computing, 2006 (e-Science’06), p. 88. IEEE (2006)
Fischetti, M., Monaci, M., Salvagnin, D.: Self-splitting of workload in parallel computation. In: International Conference on AI and OR Techniques in Constriant Programming for Combinatorial Optimization Problems, pp. 394–404. Springer, Berlin (2014)
Gallian, J.A.: A dynamic survey of graph labeling. Electron. J. Comb. Dynamic Surveys, DS6 (2014)
Golle, P., Mironov, I.: Uncheatable distributed computations. In: Proceedings of the 2001 Conference on Topics in Cryptology: The Cryptographer’s Track at RSA, pp. 425–440. Springer, Berlin (2001)
Gomes, C.P., Selman, B., Crato, N., Kautz, H.: Heavy-tailed phenomena in satisfiability and constraint satisfaction problems. J. Autom. Reason. 24(1), 67–100 (2000)
Grama, A., Kumar, V.: State of the art in parallel search techniques for discrete optimization problems. IEEE Trans. Knowl. Data Eng. 11(1), 28–35 (1999)
Heien, E.M., Anderson, D.P., Hagihara, K.: Computing low latency batches with unreliable workers in volunteer computing environments. J Grid Comput. 7(4), 501 (2009)
Jaffar, J., Santosa, A.E., Yap, R.H.C., Zhu, K.Q.: Scalable distributed depth-first search with greedy work stealing. In: 16th IEEE International Conference on Tools with Artificial Intelligence, 2004, pp. 98–103. IEEE (2004)
Javadi, B., Kondo, D., Vincent, J.M., Anderson, D.P.: Discovering statistical models of availability in large distributed systems: An empirical study of SETI@home. IEEE Trans. Parallel Distrib. Syst. 22(11), 1896–1903 (2011)
Jia, H., Moore, C.: How much backtracking does it take to color random graphs? Rigorous results on heavy tails. In: Proceedings of Principles and Practice of Constraint Programming - CP 2004, pp. 472–476 (2004)
Karp, R.M., Zhang, Y.: Randomized parallel algorithms for backtrack search and branch-and-bound computation. J. ACM 40(3), 765–789 (1993)
Mersenne Research Inc.: GIMPS history. http://www.mersenne.org/various/history.php, Cited: 21 Jan 2015
Nurmi, D., Brevik, J., Wolski, R.: Modeling machine availability in enterprise and wide-area distributed computing environments. In: Euro-Par 2005 Parallel Processing, pp. 612–612. Springer, Berlin (2005)
Pataki, M., Marosi, A.C.: Searching for translated plagiarism with the help of desktop grids. J Grid Comput. 11(1), 149–166 (2013)
Rechenkraft.net e.V.: yoyo@home. http://www.rechenkraft.net/wiki/index.php?title=Yoyo%40home_en, Cited: 21 Jan 2015
Régin, J.C., Rezgui, M., Malapert, A.: Embarrassingly parallel search. In: Principles and Practice of Constraint Programming, pp. 596–610. Springer, Berlin (2013)
Reinefeld, A., Schnecke, V.: Work-load balancing in highly parallel depth-first search. In: Proceedings of the Scalable High-Performance Computing Conference 1994, pp. 773–780. IEEE (1994)
Sonnek, J., Chandra, A., Weissman, J.: Adaptive reputation-based scheduling on unreliable distributed infrastructures. IEEE Trans. Parallel Distrib. Syst. 18(11) (2007)
Stolee, D.: TreeSearch user guide. http://www.math.uiuc.edu/stolee/SearchLib/TreeSearch.pdf (2011)
Thain, D., Tannenbaum, T., Livny, M.: Distributed computing in practice: the Condor experience. Concurr. Comput. - Pract. E 17(2–4), 323–356 (2005)
Wright, R.A., Richmond, B., Odlyzko, A., McKay, B.D.: Constant time generation of free trees. SIAM J. Comput. 15(2), 540–548 (1986)
Acknowledgements
Open access funding provided by Graz University of Technology. The first author was partially supported by IRIF (previously LIAFA) of Université Paris Diderot. Both authors thank Prof. David Anderson, Prof. Anne Benoit, Linxiao Chen and the anonymous referees for their useful advises. We also thank the volunteers in yoyo@home that contributed to Harmonious Trees and Odd Weird Search.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Fang, W., Beckert, U. Parallel Tree Search in Volunteer Computing: a Case Study. J Grid Computing 16, 647–662 (2018). https://doi.org/10.1007/s10723-017-9411-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10723-017-9411-5