Abstract
In the age of Grid, Cloud, volunteer computing, massively parallel applications are deployed over tens or hundreds of thousands of resources over short periods of times to complete immense computations. In this work, we consider the problem of deploying such applications with stringent real-time requirements. One major challenge is the server-side management of these tasks, which often number in tens or hundreds of thousands on a centralized server. In this work, we design and implement a real-time task management system for many-task computing, called RT-BOINC. The system gives low O(1) worst-case execution time for task management operations, such as task scheduling, state transitioning, and validation. We implement this system on top of BOINC, a common middleware for volunteer computing. Using micro and macro-benchmarks executed in emulation experiments, we show that RT-BOINC provides significantly lower worst-case execution time, and lessens the gap between the average and the worst-case performance compared with the original BOINC implementation.
This paper was submitted to Euro-Par 2010.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Silberstein, M., Sharov, A., Geiger, D., Schuster, A.: Gridbot: Execution of bags of tasks in multiple grids. In: SC 2009: Proceedings of the 2009 ACM/IEEE Conference on Supercomputing. ACM, New York (2009)
Lee, C.S., Wang, M.H., Chaslot, G., Hoock, J.B., Rimmel, A., Teytaud, O., Tsai, S.R., Hsu, S.C., Hong, T.P.: The Computational Intelligence of MoGo Revealed in Taiwan’s Computer Go Tournaments. IEEE Transactions on Computational Intelligence and AI in Games (2009)
Deep Blue (chess computer), http://en.wikipedia.org/wiki/Deep_Blue_chess_computer
Lopez, J., Aeschlimann, M., Dinda, P., Kallivokas, L., Lowekamp, B., O’Hallaron, D.: Preliminary report on the design of a framework for distributed visualization. In: Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA 1999), pp. 1833–1839 (1999)
Capsicum Group: Digital Forensics, http://www.capsicumgroup.com/content-pages/services/digital-forensics.html
Dinda, P.: A Prediction-Based Real-Time Scheduling Advisor. In: Proceedings of the International Parallel and Distributed Processing Symposium, IPDPS 2002 (2002)
Sonnek, J.D., Nathan, M., Chandra, A., Weissman, J.B.: Reputation-based scheduling on unreliable distributed infrastructures. In: ICDCS, p. 30 (2006)
Andrzejak, A., Kondo, D., Anderson, D.P.: Ensuring collective availability in volatile resource pools via forecasting. In: De Turck, F., Kellerer, W., Kormentzas, G. (eds.) DSOM 2008. LNCS, vol. 5273, pp. 149–161. Springer, Heidelberg (2008)
Ratnasamy, S., Handley, M., Karp, R.M., Shenker, S.: Topologically-aware overlay construction and server selection. In: INFOCOM (2002)
Kacsuk, P., Marosi, A.C., Kovacs, J., Balaton, Z., Gombs, G., Vida, G., Kornafeld, A.: Sztaki desktop grid: A hierarchical desktop grid system. In: Proceedings of the Cracow Grid Workshop 2006, Cracow, Poland (2006)
Gorlatch, S., Glinka, F., Ploss, A., Müller-Iden, J., Prodan, R., Nae, V., Fahringer, T.: Enhancing grids for massively multiplayer online computer games. In: Luque, E., Margalef, T., BenÃtez, D. (eds.) Euro-Par 2008. LNCS, vol. 5168, pp. 466–477. Springer, Heidelberg (2008)
Catalog of BOINC Powered Projects - Unofficial BOINC Wiki, http://www.boinc-wiki.info/Catalog_of_BOINC_Powered_Projects
Anderson, D.P.: Talk at Condor Week, Madison, WI, http://boinc.berkeley.ed-utalkscondorboinc06.ppt (2006)
MySQL: Developer Zone, http://dev.mysql.com/
BOINC Statistics, http://boincstats.com/stats/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yi, S., Kondo, D., Anderson, D.P. (2010). Toward Real-Time, Many-Task Applications on Large Distributed Systems. In: D’Ambra, P., Guarracino, M., Talia, D. (eds) Euro-Par 2010 - Parallel Processing. Euro-Par 2010. Lecture Notes in Computer Science, vol 6271. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15277-1_35
Download citation
DOI: https://doi.org/10.1007/978-3-642-15277-1_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15276-4
Online ISBN: 978-3-642-15277-1
eBook Packages: Computer ScienceComputer Science (R0)