Chapter

Databases in Networked Information Systems

Volume 4777 of the series Lecture Notes in Computer Science pp 154-177

Towards Scalable Architectures for Clickstream Data Warehousing

  • Peter AlvaroAffiliated withASK.com, 555 12th Street, Suite 500, Oakland, CA 94607
  • , Dmitriy V. RyaboyAffiliated withASK.com, 555 12th Street, Suite 500, Oakland, CA 94607
  • , Divyakant AgrawalAffiliated withASK.com, 555 12th Street, Suite 500, Oakland, CA 94607

* Final gross prices may vary according to local VAT.

Get Access

Abstract

Click-stream data warehousing has emerged as a monumental information management and processing challenge for commercial enterprises. Traditional solutions based on commercial DBMS technology often suffer from poor scalability and large processing latencies. One of the main problems is that click-stream data is inherently collected in a distributed manner, but in general these distributed click-stream logs are collated and pushed upstream in a centralized database storage repository, creating storage bottlenecks. In this paper, we propose a design of an ad-hoc retrieval system suitable for click-stream data warehouses, in which the data remains distributed and database queries are rewritten to be executed against the distributed data. The query rewrite does not involve any centralized control and is therefore highly scalable. The elimination of centralized control is achieved by supporting a restricted subset of SQL, which is sufficient for most click-stream data analysis. Evaluations conducted using both synthetic and real data establish the viability of this approach.