Synchronization Modeling in Stream Processing

Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 186)

Abstract

Currently used latency models in stream databases are based on the average values analysis that results from Little’s law. The other models apply theory of M/G/1 queuing system. Theses solutions are fast and easy to implement but they omit the impact of streams synchronization. In this paper, we introduce a heuristic method which measures the synchronization impact. Then we have used this solution to extend the popular model based on average values analysis. This modification allows us to achieve better accuracy of latency estimation. Because schedulers and stream operator optimization require a fast and accurate model, we find our model a good starting point to create better optimizers.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
  2. 2.
    Babcock, B., Babu, S., Datar, M., Motwani, R., Thomas, D.: Operator scheduling in data stream systems. The VLDB Journal 13(4), 333–353 (2004)CrossRefGoogle Scholar
  3. 3.
    Babcock, B., Datar, M., Motwani, R.: Load shedding for aggregation queries over data streams. In: ICDE 2004: Proceedings of the 20th International Conference on Data Engineering, p. 350. IEEE Computer Society, Washington, DC (2004)CrossRefGoogle Scholar
  4. 4.
    Bai, Y., Thakkar, H., Wang, H., Zaniolo, C.: Optimizing timestamp management in data stream management systems. IEEE 23rd International Conference on Data Engineering, ICDE 2007, pp. 1334–1338 (2007)Google Scholar
  5. 5.
    Bai, Y., Zaniolo, C.: Minimizing latency and memory in dsms: a unified approach to quasi-optimal scheduling. In: SSPS 2008: Proceedings of the 2nd International Workshop on Scalable Stream Processing System, pp. 58–67. ACM Press, New York (2008)CrossRefGoogle Scholar
  6. 6.
    Balazinska, M.: Fault-tolerance and load management in a distributed stream processing system. Ph.D. thesis, Cambridge, MA, USA (2006); Adviser-Hari BalakrishnanGoogle Scholar
  7. 7.
    Carney, D., Çetintemel, U., Rasin, A., Zdonik, S., Cherniack, M., Stonebraker, M.: Operator scheduling in a data stream manager. In: VLDB 2003: Proceedings of the 29th International Conference on Very Large Data Bases, pp. 838–849. VLDB Endowment (2003)Google Scholar
  8. 8.
    Hammad, M.A., Franklin, M.J., Aref, W.G., Elmagarmid, A.K.: Scheduling for shared window joins over data streams. In: VLDB, pp. 297–308 (2003)Google Scholar
  9. 9.
    Hwang, J.H., Xing, Y., Çetintemel, U., Zdonik, S.B.: A cooperative, self-configuring high-availability solution for stream processing. In: ICDE, pp. 176–185 (2007)Google Scholar
  10. 10.
    Jiang, Q., Chakravarthy, S.: Scheduling Strategies for Processing Continuous Queries over Streams. In: Williams, H., MacKinnon, L.M. (eds.) BNCOD 2004. LNCS, vol. 3112, pp. 16–30. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  11. 11.
    Lamport, L.: Time, clocks, and the ordering of events in a distributed system. Commun. ACM 21(7), 558–565 (1978)MATHCrossRefGoogle Scholar
  12. 12.
    Sharaf, M.A.: Metrics and algorithms for processing multiple continuous queries. Ph.D. thesis, Pittsburgh, PA, USA (2007)Google Scholar
  13. 13.
    Sharaf, M.A., Chrysanthis, P.K., Labrinidis, A.: Preemptive rate-based operator scheduling in a data stream management system. In: AICCSA 2005: Proceedings of the ACS/IEEE 2005 International Conference on Computer Systems and Applications, pp. 46–I. IEEE Computer Society, Washington, DC (2005)Google Scholar
  14. 14.
    Sharaf, M.A., Chrysanthis, P.K., Labrinidis, A., Pruhs, K.: Efficient scheduling of heterogeneous continuous queries. In: VLDB 2006: Proceedings of the 32nd International Conference on Very Large Data Bases, pp. 511–522. VLDB Endowment (2006)Google Scholar
  15. 15.
    Tatbul, E.N.: Load shedding techniques for data stream management systems. Brown University, Providence (2007); Adviser-Zdonik, StanGoogle Scholar
  16. 16.
    Tatbul, N., Çetintemel, U., Zdonik, S., Cherniack, M., Stonebraker, M.: Load shedding in a data stream manager. In: VLDB 2003: Proceedings of the 29th International Conference on Very Large Data Bases, pp. 309–320. VLDB Endowment (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.Institute of Computer ScienceSilesian University of TechnologyGliwicePoland
  2. 2.Institute of Computer ScienceWroclaw University of TechnologyWrocławPoland

Personalised recommendations