MemcacheSQL A Scale-Out SQL Cache Engine

  • Qiming Chen
  • Meichun Hsu
  • Ren Wu
Conference paper
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 126)


Real-time enterprise applications and data-intensive analytics often require very low latency access to large volumes of data. In-memory data cache is a promising approach to enable real-time analytics. In this paper we examine the issue of scaling out memory cache over multiple machines while providing a common data query language with rich expressive power, and allowing the data cached in memory to persist with the ACID properties. Designing and building a product quality scaled-out data cache engine from scratch is an option, but it requires large investment in time and engineering efforts.

We propose to build such a system by extending existing SQL systems. In our approach, we extend database buffer pool with a Distributed Caching Platform (DCP). We have developed a prototype of the system, which we call MemcacheSQL, engine by integrating PostgreSQL’s buffer management with Memcached, a widely used DCP platform. Our contributions include optimized data coherence management along the memory hierarchy, and flexibly configurable page buffering policies for accommodating various workloads. Our preliminary experiments show that the proposed system has the potential to achieve scalable performance gains by dynamically leveraging scale-out memory nodes, while retaining the full SQL’s expressive power and DBMS’s ACID support.


Overflow Model Buffer Management Inclusion Model Query Engine Buffer Pool 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Nori, A.: Distributed Caching Platforms. In: VLDB 2010 (2010)Google Scholar
  2. 2.
    Arasu, A., Babu, S., Widom, J.: The CQL Continuous Query Language: Semantic Foundations and Query Execution. VLDB Journal 15(2) (June 2006) Google Scholar
  3. 3.
    Abadi, D.J., et al.: The Design of the Borealis Stream Processing Engine. In: CIDR 2005 (2005)Google Scholar
  4. 4.
    Chen, Q., Hsu, M., Zeller, H.: Experience in Continuous analytics as a Service (CaaaS). In: EDBT 2011 (2011)Google Scholar
  5. 5.
    Franklin, M.J., et al.: Continuous Analytics: Rethinking Query Processing in a Network-Effect World. In: CIDR 2009 (2009)Google Scholar
  6. 6.
    Memcached (2010),
  7. 7.
  8. 8.
    Vmware vFabric GemFire (2010),
  9. 9.
    Gigaspaces Extreme Application Platform (2010),
  10. 10.
  11. 11.
    AppFabric Cache (2010),
  12. 12.
    Liarou, E., et al.: Exploiting the Power of Relational Databases for Efficient Stream Processing. In: EDBT 2009 (2009)Google Scholar
  13. 13.
    The Wafflegrid Project,
  14. 14.
  15. 15.
  16. 16.
    Baer, J., Wang, W.: On the inclusion properties for multi-level cache hierarchies. In: Proceedings of the 15th Annual International Symposium on Computer Architecture, ISCA 1988 (1988)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Qiming Chen
    • 1
  • Meichun Hsu
    • 1
  • Ren Wu
    • 1
  1. 1.HP LabsHewlett Packard Co.Palo AltoUSA

Personalised recommendations