Advertisement

Cluster Computing

, Volume 12, Issue 2, pp 221–235 | Cite as

The Circulate architecture: avoiding workflow bottlenecks caused by centralised orchestration

  • Adam BarkerEmail author
  • Jon B. Weissman
  • Jano I. van Hemert
Article

Abstract

As the number of services and the size of data involved in workflows increases, centralised orchestration techniques are reaching the limits of scalability. In the classic orchestration model, all data passes through a centralised engine, which results in unnecessary data transfer, wasted bandwidth and the engine to become a bottleneck to the execution of a workflow.

This paper presents and evaluates the Circulate architecture which maintains the robustness and simplicity of centralised orchestration, but facilitates choreography by allowing services to exchange data directly with one another. Circulate could be realised within any existing workflow framework, in this paper, we focus on WS-Circulate, a Web services based implementation.

Taking inspiration from the Montage workflow, a number of common workflow patterns (sequence, fan-in and fan-out), input to output data size relationships and network configurations are identified and evaluated. The performance analysis concludes that a substantial reduction in communication overhead results in a 2–4 fold performance benefit across all patterns. An end-to-end pattern through the Montage workflow results in an 8 fold performance benefit and demonstrates how the advantage of using the Circulate architecture increases as the complexity of a workflow grows.

Keywords

Workflow Workflow optimisation Web services Decentralised orchestration 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Abu-Ghazaleh, N., Lewis, M.J., Govindaraju, M.: Differential serialization for optimized SOAP performance. In: Proceedings of HPDC, June 2004 Google Scholar
  2. 2.
    Apache Axis: http://ws.apache.org/axis (2008). Accessed 16 December 2008
  3. 3.
    Barker, A., van Hemert, J.: Scientific workflow: a survey and research directions. In: Wyrzykowski, R., et al. (eds.) Seventh International Conference on Parallel Processing and Applied Mathematics, Revised Selected Papers. LNCS, vol. 4967, pp. 746–753. Springer, Berlin (2008) Google Scholar
  4. 4.
    Barros, A., Dumas, M., Oaks, P.: A critical overview of the Web services choreography description language (WS-CDL). BPTrends Newsletter 3 (2005) Google Scholar
  5. 5.
    Binder, W., Constantinescu, I., Faltings, B.: Decentralized orchestration of composite Web services. In: Proceedings of the International Conference on Web Services, ICWS’06, pp. 869–876. IEEE Comput. Soc., Los Alamitos (2006) Google Scholar
  6. 6.
    Chiu, K., Govindaraju, M., Bramley, R.: Investigating the limits of SOAP performance for scientic computing. In: Proccesings of the 11th International Symposium on High Performance Distributing Computing (HPDC), July 2002 Google Scholar
  7. 7.
    Decker, G., Kopp, O., Leymann, F., Weske, M.: BPEL4Chor: extending BPEL for modeling choreographies. In: Proceedings of the IEEE 2007 International Conference on Web Services (ICWS 2007), pp. 296–303 (2007) Google Scholar
  8. 8.
    Deelman, E., Singh, G., Su, M.-H., Blythe, J., Gil, A., Kesselman, C., Mehta, G., Vahi, K., Berriman, G.B., Good, J., Laity, A., Jacob, J.C., Katz, D.S.: Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Sci. Programm. J. 13(3), 219–237 (2005) Google Scholar
  9. 9.
    Devaram, K., Andresen, D.: Differential serialization for optimized SOAP performance. In: Proceedings of PDCS, November 2003 Google Scholar
  10. 10.
    Fredlund, L.: Implementing WS-CDL. In: Proceedings of the Second Spanish Workshop on Web Technologies (JSWEB 2006) (2006) Google Scholar
  11. 11.
    Grimshaw, A.S., Weissman, J.B., Strayer, W.T.: Portable run-time support for dynamic object-oriented parallel processing. ACM Trans. Comput. Syst. 14(2), 137–170 (1996) CrossRefGoogle Scholar
  12. 12.
    Jacob, J.C., Katz, D.S., et. al.: The Montage architecture for grid-enabled science processing of large, distributed datasets. In: Proceedings of the Earth Science Technology Conference, June 2004 Google Scholar
  13. 13.
    Karasavvas, K., Antonioletti, M., Atkinson, M., Hong, N.C., Sugden, T., Hume, A., Jackson, M., Krause, A., Palansuriya, C.: Introduction to OGSA-DAI Services. LNCS, vol. 3458, pp. 1–12. Springer, Berlin (2005) Google Scholar
  14. 14.
    Kavantzas, N., Burdett, D., Ritzinger, G., Lafon, Y.: Web services choreography description language (WS-CDL) Version 1.0. W3C Candidate Recommendation (2005) Google Scholar
  15. 15.
    Krishnan, S., Wagstrom, P., von Laszewski, G.: GSFL: a workflow framework for grid services. Technical Report, Argonne National Laboratory (2002) Google Scholar
  16. 16.
    Liu, D., Law, K.H., Wiederhold, G.: Analysis of integration models of service composition. In: Proceedings of Third International Workshop on Software and Performance, pp. 158–165. ACM, New York (2002) CrossRefGoogle Scholar
  17. 17.
    Liu, D., Law, K.H., Wiederhold, G.: Data-flow distribution in FICAS service composition infrastructure. In: Proceedings of the 15th International Conference on Parallel and Distributed Computing Systems (2002) Google Scholar
  18. 18.
    Ludascher, B., et al.: Scientific workflow management and the Kepler system. Concurr. Comput., Pract. Exper. 18(10), 1039–1065 (2005) CrossRefGoogle Scholar
  19. 19.
    Martin, D., Wutke, D., Leymann, F.: A novel approach to decentralized workflow enactment. In: EDOC ’08. 12th International IEEE Conference on Enterprise Distributed Object Computing, pp. 127–136 (2008) Google Scholar
  20. 20.
    Oinn, T., Addis, M., Ferris, J., Marvin, D., Senger, M., Greenwood, M., Carver, T., Glover, K., Pocock, M.R., Wipat, A., Li, P.: Taverna: a tool for the composition and enactment of bioinformatics workflows. Bioinformatics 20, 3045–3054 (2004) CrossRefGoogle Scholar
  21. 21.
    Planet Lab: http://www.planet-lab.org (2008). Accessed 16 December 2008
  22. 22.
    Taylor, I., Shields, M., Wang, I., Philp, R.: Distributed P2P computing within Triana: a galaxy visualization test case. In: 17th International Parallel and Distributed Processing Symposium (IPDPS 2003), pp. 16–27. IEEE Comput. Soc., Los Alamitos (2003) Google Scholar
  23. 23.
    The OASIS Committee: Web services business process execution language (WS-BPEL) Version 2.0 (2007) Google Scholar
  24. 24.
    Wassermann, B., et al.: Sedna: A BPEL-based environment for visual scientific workflow modelling. In: Workflows for eScience–Scientific Workflows for Grids, December 2006 Google Scholar
  25. 25.
    Zaha, J.M., Barros, A., Dumas, M., ter Hofstede, A.: Let’s dance: a language for service behavior modelling. In: Meersman, R., Tari, Z. (eds.) OTM Conferences (1). LNCS, vol. 4275, pp. 145–162. Springer, Berlin (2006) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Adam Barker
    • 1
    Email author
  • Jon B. Weissman
    • 2
  • Jano I. van Hemert
    • 3
  1. 1.Department of Engineering ScienceUniversity of OxfordOxfordUK
  2. 2.University of MinnesotaMinneapolisUSA
  3. 3.NeSC, School of InformaticsUniversity of EdinburghEdinburghUK

Personalised recommendations