Buffer-Preposed QoS Adaptation Framework and Load Shedding Techniques over Streams

  • Rui Zhou
  • Guoren Wang
  • Donghong Han
  • Pizhen Gong
  • Chuan Xiao
  • Hongru Li
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4255)


Maintaining the quality of queries over streaming data is often thought to be of tremendous challenge since data arrival rate and average per-tuple CPU processing cost are highly unpredictable. In this paper, we address a novel buffer-preposed QoS adaptation framework on the basis of control theory and present several load shedding techniques and scheduling strategies in order to guarantee the QoS of processing streaming data. As the most significant part of our framework, buffer manager consisting of scheduler, adaptor and cleaner, is deliberately introduced and analyzed. The experiments on both synthetic data and real life data show that our system, which is built by adding several concrete strategies on the framework, outperforms existing works on both resource utilization and QoS assurance.


Data Stream Queue Length Upstream Part Downstream Part Random Early Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Tu, Y.-C., Hefeeda, M., Xia, Y., Prabhakar, S.: Control-based Quality Adaptation in Data Stream Management Systems. In: Andersen, K.V., Debenham, J., Wagner, R. (eds.) DEXA 2005. LNCS, vol. 3588, pp. 746–755. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  2. 2.
    Kang, J., Naughton, J.F., Viglas, S.D.: Evaluating window joins over unbounded streams. In: Proc. of ICDE, Bangalore, India (March 2003)Google Scholar
  3. 3.
    Xie, J., Yang, J., Chen, Y.: On joining and caching stochastic streams. In: Proc. 2005 ACM SIGMOD Conf., Baltimore, Maryland, USA (June 2005)Google Scholar
  4. 4.
    The STREAM Group. STREAM: The Stanford Stream Data Manager. IEEE Data Engineering Bulletin 26(1), 19–26 (March 2003)Google Scholar
  5. 5.
    Das, A., Gehrke, J., Riedewald, M.: Approximate Join Processing Over Data Streams. In: Proc. 2003 ACM SIGMOD Conf. (June 2003)Google Scholar
  6. 6.
    Babcock, B., Datar, M., Motwani, R.: Load Shedding for Aggregation Queries over Data Streams. In: Proc. 2004 Int. Conf. on Data Engineering (February 2004)Google Scholar
  7. 7.
    Abadi, D., Carney, D., et al.: Aurora: a new model and architecture for data stream management. VLDB Journal 12(2), 120–139 (2003)CrossRefGoogle Scholar
  8. 8.
    Abadi, D., Ahmad, Y., Balazinska, M., Cetintemel, U., Cherniack, M., Hwang, J., Lindner, W., Maskey, A., Rasin, A., Ryvkina, E., Tatbul, N., Zdonik, S.: The Design of the Borealis Stream Processing Engine. In: Procs. of CIDR (January 2005)Google Scholar
  9. 9.
    Tatbul, N., Cetintemel, U., Zdonik, S., Cherniack, M., Stonebraker, M.: Load Shedding in a Data Stream Manager. In: Proc. 29th int. Conf. on VLDB (September 2003)Google Scholar
  10. 10.
    Chen, J., DeWitt, D.J., Tian, F., Wang, Y.: NiagaraCQ: A scalable continous query system for internet databasses. In: Proc. ACM SIGMOD Int. Conf. on Management of Data, pp. 379–390 (2000)Google Scholar
  11. 11.
    Chandrasekaran, S., Deshpande, A., Franklin, M., et al.: TelegraphCQ: Continuous Dataflow Processing for an Uncertain World. In: Proc. of CIDR (January 2003)Google Scholar
  12. 12.
    Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and issues in data stream systems. In: Proc. Principles of Database Systems (PODS) (June 2002)Google Scholar
  13. 13.
    Han, D., Zhou, R., Xiao, C., Wang, G., et al.: Load shedding for Window Joins over Data Streams. In: Yu, J.X., Kitsuregawa, M., Leong, H.-V. (eds.) WAIM 2006. LNCS, vol. 4016, pp. 472–483. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  14. 14.
    Chi, Y., Wang, H., Yu, P.S.: LoadStar: Load Shedding in Data Stream Mining. In: Proc. Of the 31st VLDB Conf., pp. 1302–1305 (August 2005)Google Scholar
  15. 15.
    Reiss, F., Hellerstein, J.M.: Data Triage: An Adaptive Architecture for Load Shedding in TelegraphCQ. In: Proc. of ICDE, pp. 155–156 (April 2005)Google Scholar
  16. 16.
    Chen, L., Ösu, M.T., Oria, V.: Robust and Fast Similarity Search for Moving Object Trajectories. In: Proceedings of 24th ACM International Conference on Management of Data (SIGMOD 2005), pp. 491–502 (June 2005)Google Scholar
  17. 17.
    Chen, L., Ng, R.: On the Marriage of Lp-Norm and Edit Distance. In: Proc. of VLDB, Toronto, Canada, pp. 792–803 (August 2004)Google Scholar
  18. 18.
    Liu, Y., Liu, X., Xiao, L., Ni, L., Zhang, X.: Location-Aware Topology Matching in P2P Systems. In: IEEE INFOCOM (2004)Google Scholar
  19. 19.
    Abdelzaher, T., Sharma, V., Lu, C.: A Utilization Bound for Aperiodic Tasks and Priority Driven Scheduling. IEEE Trans. on Computers 53, 334–350 (2004)CrossRefGoogle Scholar
  20. 20.
    Stewart, D.B., Khosla, P.K.: Real-Time Scheduling of Sensor-Based Control Systems. In: Halang, W., Ramamritham, K. (eds.) Real-Time Programming. Pergamon Press Inc., Tarrytown (1992)Google Scholar
  21. 21.
    Floyd, S., Jacobson, V.: Traffic phase effects in packet-switched gateways. ACM Comp. Commun. Rev. 21(2), 26–42 (1991)CrossRefGoogle Scholar
  22. 22.
    Hashem, E.S.: Analysis of random drop for gateway congestion control. MIT Lab for Computer Science: Technical Report, MIT (1989)Google Scholar
  23. 23.
    Lakshman, T., Neidhardt, A., Ott, T.: The drop from front strategy in TCP and in TCP over ATM. In: IEEE INFOCOM, San Francisco, CA, pp. 1242–1250 (1996)Google Scholar
  24. 24.
    Floyd, S., Jacobson, V.: Random Early Detection gateways for congestion avoidance. IEEE/ACM Transactions on Networking 1(4) (August 1997)Google Scholar
  25. 25.
    Franklin, G.F., Powell, J.D., Emami-Naeini, A.: Feedback Control of Dynamic Systems. Prentice Hall, Massachusetts (2002)Google Scholar
  26. 26.
    Tu, Y.-C., Song, L., Prabhakar, S.: Load Shedding in Stream Databases: A Control-Based Approach. Technical report, Purdue University (March 2006)Google Scholar
  27. 27.
    Paxson, V., Floyd, S.: Wide-Area Traffic: The Failure of Poisson Modeling. IEEE/ACM Transactions on Networking 3(3), 226–244 (1995), CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Rui Zhou
    • 1
  • Guoren Wang
    • 1
  • Donghong Han
    • 1
  • Pizhen Gong
    • 1
  • Chuan Xiao
    • 1
  • Hongru Li
    • 1
  1. 1.Institute of Computer SystemNortheastern UniversityShenyangChina

Personalised recommendations