Efficient and Adaptive Processing of Multiple Continuous Queries
Continuous queries are queries executed on data streams within a potentially open-ended time interval specified by the user and are usually long running. The data streams are likely to exhibit fluctuating characteristics such as varying inter-arrival times, as well as varying data characteristics during the query execution. In the presence of such unpredictable factors, continuous query systems must still be able to efficiently handle large number of queries, as well as to offer acceptable individual query performance.
In this paper, we propose and discuss a novel framework, called AdaptiveCQ, for the efficient processing of multiple continuous queries. In our framework, multiple queries share intermediate results at a fine level of granularity. Unlike previous approaches to sharing or reusing that relied on materialization to disk, AdaptiveCQ allows on-the-fly sharing of results. We show that this feature improves both the initial query response time, and the overall response time. Finally, AdaptiveCQ, which extrapolates the idea proposed by the eddy query-processing model, adapts well to fluctuations of the data streams characteristics by this combination of fine grain and on-the-fly sharing. We implemented AdaptiveCQ from scratch in Java and made use of it to conduct the experiments. We present experimental results that substantiate our claim that AdaptiveCQ can provide substantial performance improvements over existing methods of reusing intermediate results that relied on materialization to disk. In addition, we also show that AdaptiveCQ can adapt well to fluctuations in the query environment.
KeywordsData Stream Garbage Collection Query Plan Continuous Query Multiple Query
Unable to display preview. Download preview PDF.
- 1.Mehmet Altinel and Michael J. Franklin. Efficient filtering of XML documents for selective dissemination of information. In The VLDB Journal, pages 53–64, 2000.Google Scholar
- 3.Laurent Amsaleg, Michael J. Franklin, Anthony Tomasic, and Tolga Urhan. Scrambling query plans to cope with unexpected delays. In PDIS, pages 208–219, 1996.Google Scholar
- 4.Ron Avnur and Joseph M. Hellerstein. Eddies: Continuously adaptive query processing. In Proceedings of the ACM SIGMOD International Conference on Management of Data, pages 261–272, May 2000.Google Scholar
- 5.Philippe Bonnet, Johannes Gehrke, and Praveen Seshadri. Towards sensor database systems. In Proceedings of Mobile Data Management, Second International Conference, MDM 2001, Hong Kong, China, January 2001.Google Scholar
- 6.Surajit Chaudhuri, Ravi Krishnamurthy, Spyros Potamianos, and Kyuseak Shim. Optimizing queries with materialized views. In 11th Int. Conference on Data Engineering, pages 190–200, Los Alamitos, CA, 1995. IEEE Computer Soc. Press.Google Scholar
- 7.C. Chen and N. Roussopoulos. The implementation and performance evaluation of the adms query optimizer: Integrating query result caching and matching, 1994.Google Scholar
- 8.Jianjun Chen, David J. DeWitt, Feng Tian, and Yuan Wang. NiagaraCQ: a scalable continuous query system for Internet databases. In Proceedings of the ACM SIGMOD International Conference on Management of Data, pages 379–390, 2000.Google Scholar
- 9.E. N. Hanson, C. Carnes, L. Huang, M. Konyala, L. Noronha, S. Parthasarathy, J. B. Park, and A. Vernon. Scalable Trigger Processing. In Proceedings of the 15th International Conference on Data Engineering, pages 266–275. IEEE Computer Society Press, 1999.Google Scholar
- 10.Donald Kossmann, Michael J. Franklin, Gerhard Drasch, and Wig Ag. Cache investment: Integrating query optimization and distributed data placement. ACM Transactions on Database Systems (TODS), 25(4), December 2000.Google Scholar
- 12.Prasan Roy, S. Seshadri, S. Sudarshan, and Siddhesh Bhobe. Efficient and extensible algorithms for multi query optimization. In Proceedings of the ACM SIGMOD International Conference on Management of Data, pages 249–260, 2000.Google Scholar
- 14.J. Shim, P. Scheuermann, and R. Vingralek. Dynamic caching of query results for decision support systems, 1999.Google Scholar
- 15.Kian Lee Tan, Shen Tat Goh, and Beng Chin Ooi. Cache-on-demand: Recycling with certainty. In International Conference on Data Engineering, April 2001.Google Scholar
- 16.Douglas Terry, David Goldberg, David Nichols, and Brian Oki. Continuous queries over append-only databases. In Proceedings of the ACM SIGMOD International Conference on Management of Data, pages 321–330, June 1992.Google Scholar
- 17.Tolga Urhan and Michael J. Franklin. XJoin: Getting fast answers from slow and bursty networks. Technical Report CS-TR-3994, University of Maryland, February 1999, 1999.Google Scholar
- 18.Tolga Urhan, Michael J. Franklin, and Laurent Amsaleg. Cost-based query scrambling for initial delays. In Proceedings of the ACM SIGMOD International Conference on Management of Data, pages 130–141, 1998.Google Scholar