Encyclopedia of Big Data Technologies

Living Edition
| Editors: Sherif Sakr, Albert Zomaya

Wildfire: HTAP for Big Data

Living reference work entry
DOI: https://doi.org/10.1007/978-3-319-63962-8_257-1
  • 88 Downloads

Abstract

Emerging large-scale real-time analytic applications (real-time inventory/pricing/recommendations, fraud detection, risk analysis, IoT, etc.) require data management systems that can handle fast transactions (OLTP) and analytics (OLAP) simultaneously. Some of them even require analytical queries as part of a transaction. Efficient processing of transactional and analytical requests, however, leads to different design decisions in a system. This article presents the Wildfire system, which targets hybrid transactional and analytical processing (HTAP) for big data. Wildfire leverages Apache Spark to enable large-scale data processing with different types of complex analytical requests and columnar data processing to enable fast transactions and analytics concurrently.

Keywords

Analysis Request Analytical Queries Shard Replica Snapshot Isolation SQLContext 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

References

  1. Barber R, Garcia-Arellano C, Grosman R, Mueller R, Raman V, Sidle R, Spilchen M, Storm AJ, Tian Y, Tözün P et al (2017) Evolving databases for new-gen big data applications. In: CIDRGoogle Scholar
  2. Boncz P, Zukowski M, Nes N (2005) MonetDB/X100: hyper-pipelining query execution. In: CIDRGoogle Scholar
  3. Diaconu C, Freedman C, Ismert E, Larson P-Å, Mittal P, Stonecipher R, Verma N, Zwilling M (2013) Hekaton: SQL server’s memory-optimized OLTP engine. In: SIGMOD, pp 1243–1254Google Scholar
  4. Lamb A, Fuller M, Varadarajan R, Tran N, Vandiver B, Doshi L, Bear C (2012) The vertica analytic database: C-store 7 years later. PVLDB 5(12):1790–1801Google Scholar
  5. O’Neil P, Cheng E, Gawlick D, O’Neil E (1996) The log-structured merge-tree (LSM-tree). Acta Inf 33(4): 351–385CrossRefGoogle Scholar
  6. Raman V, Attaluri G, Barber R, Chainani N, Kalmuk D, KulandaiSamy V, Leenstra J, Lightstone S, Liu S, Lohman GM, Malkemus T, Mueller R, Pandis I, Schiefer B, Sharpe D, Sidle R, Storm A, Zhang L (2013) DB2 with BLU acceleration: so much more than just a column store. PVLDB 6:1080–1091Google Scholar
  7. Stonebraker M, Weisberg A (2013) The VoltDB main memory DBMS. IEEE Data Eng Bull 36(2):21–27Google Scholar

Authors and Affiliations

  1. 1.IBM Research – AlmadenSan JoseUSA
  2. 2.IT University of CopenhagenCopenhagenDenmark

Section editors and affiliations

  • Yuanyuan Tian
    • 1
  • Fatma Özcan
    • 2
  1. 1.IBM Almaden Research CenterSAN JOSEUSA
  2. 2.IBM Research – AlmadenSan JoseUSA