Continuous Access to Cloud Event Services with Event Pipe Queries
When cloud services become popular, how to consume a cloud service efficiently by an enterprise application, as the client of the cloud service either on a device or on the application tier of the enterprise software stack, is an important issue. Focusing on the consumption of the real-time events service, in this work we extend the Data Access Object (DAO) pattern of enterprise applications for on-demand access and analysis of real-time events.
We introduce the notion of Operational Event Pipe for caching the most recent events delivered by an event service, and the on-demand data analysis pattern based on this notion. We implemented the operational event pipe as a special kind of continuous query referred to as Event Pipe Query (EPQ). An EPQ is a long-standing SQL query with User Defined Functions (UDFs) that provides a pipe for the stream data to be buffered and to flow continuously in the boundary of a sliding window; when not requested, the EPQ just maintains and updates the buffer but returns noting, once requested, it returns the query processing results on the selected part of the sliding window buffer, under the request-and-rewind mechanism. Integrating event buffering and analysis in a single continuous query leverages the SQL expressive power and the query engine’s data processing capability, and reduces the data movement overhead.
By extending the PostgreSQL query engine, we implement this operation pattern as the Continuous Data Access Object (CDAO) – an extension to the J2EE DAO. While DAO provides static data access interfaces, CDAO adds dynamic event processing interfaces with one or more EPQs.
KeywordsCloud Service Data Warehouse Query Result Table Function Query Execution
Unable to display preview. Download preview PDF.
- 1.Arasu, A., Babu, S., Widom, J.: The CQL Continuous Query Language: Semantic Foundations and Query Execution. VLDB Journal 2(15) (June 2006)Google Scholar
- 2.Abadi, D.J., et al.: The Design of the Borealis Stream Processing Engine. In: CIDR (2005)Google Scholar
- 3.Bryant, R.E.: Data-Intensive Supercomputing: The case for DISC, CMU-CS-07-128 (2007)Google Scholar
- 4.Chaiken, Jenkins, B., Larson, P-Å., Ramsey, B., Shakib, D., Weaver, S., Zhou, J.: SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets. In: VLDB 2008 (2008) Google Scholar
- 5.Chen, Q., Hsu, M., Zeller, H.: Experience in Continuous analytics as a Service. In: EDBT 2011 (2011)Google Scholar
- 6.Chen, Q., Hsu, M.: Cooperating SQL Dataflow Processes for In-DB Analytics. In: Proc. 17th International Conference on Cooperative Information Systems, CoopIS (2009)Google Scholar
- 7.DeWitt, D.J., Paulson, E., Robinson, E., Naughton, J., Royalty, J., Shankar, S., Krioukov, A.: Clustera: An Integrated Computation And Data Management System. In: VLDB 2008 (2008)Google Scholar
- 8.Franklin, M.J., et al.: Continuous Analytics: Rethinking Query Processing in a NetworkEffect World. In: CIDR 2009 (2009)Google Scholar
- 9.Gedik, B., Andrade, H., Wu, K.-L., Yu, P.S., Doo, M.C.: SPADE: The System S Declarative Stream Processing Engine. In: ACM SIGMOD 2008 (2008)Google Scholar
- 10.Liarou E., et al.: Exploiting the Power of Relational Databases for Efficient Stream Processing. In: EDBT 2009 (2009)Google Scholar
- 11.Core J2EE Patterns - Data Access Objects. Sun Microsystems Inc. (2007), http://java.sun.com/blueprints/corej2eepatterns/Patterns/DataAccessObject.html
- 12.IBM pureQuery: A high-performance data access platform, http://www-01.ibm.com/software/data/optim/purequery-platform/
- 13.JingDAO, http://jingdao.sourceforge.net/