, Volume 18, Issue 3, pp 145–156 | Cite as

Integration of FPGAs in Database Management Systems: Challenges and Opportunities

  • Andreas BecherEmail author
  • Lekshmi B.G.
  • David Broneske
  • Tobias Drewes
  • Bala Gurumurthy
  • Klaus Meyer-Wegener
  • Thilo Pionteck
  • Gunter Saake
  • Jürgen Teich
  • Stefan Wildermann


In the presence of exponential growth of the data produced every day in volume, velocity, and variety, online analytical processing (OLAP) is becoming increasingly challenging. FPGAs offer hardware reconfiguration to enable query-specific pipelined and parallel data processing with the potential of maximizing throughput, speedup as well as energy and resource efficiency. However, dynamically configuring hardware accelerators to match a given OLAP query is a complex task. Furthermore, resource limitations restrict the coverage of OLAP operators. As a consequence, query optimization through partitioning the processing onto components of heterogeneous hardware/software systems seems a promising direction. While there exists work on operator placement for heterogeneous systems, it mainly targets systems combining multi-core CPUs with GPUs. However, an inclusion of FPGAs, which uniquely offer efficient and high-throughput pipelined processing at the expense of potential reconfiguration overheads, is still an open problem. We postulate that this challenge can only be met in a scalable fashion when providing a cooperative optimization between global and FPGA-specific optimizers. We demonstrate how this is addressed in two current research projects on FPGA-based query processing.


Database query processing Hardware acceleration Query optimization FPGA OLAP 


  1. 1.
    Barthels C, Alonso G, Hoefler T, Schneider T, Müller I (2017) Distributed join algorithms on thousands of cores. Proceedings VLDB Endowment 10(5):517–528CrossRefGoogle Scholar
  2. 2.
    Becher A, Ziener D, Meyer-Wegener K, Teich J (2015) A co-design approach for accelerated SQL query processing via FPGA-based data filtering. In: FPT, pp 192–195Google Scholar
  3. 3.
    Becher A, Pirkl J, Herrmann A, Teich J, Wildermann S (2016a) Hybrid energy-aware reconfiguration management on Xilinx Zynq SoCs. In: ReConFig, pp 1–7Google Scholar
  4. 4.
    Becher A, Wildermann S, Mühlenthaler M, Teich J (2016b) Reorder: Runtime datapath generation for high-throughput multi-stream processing. In: ReConFig, pp 1–8Google Scholar
  5. 5.
    Becher A, Wildermann S, Teich J (2018) Optimistic regular expression matching on FPGAs for near-data processing. Proc. 14th Int. Workshop on Data Management on New Hardware, Houston, 11.6.2018, pp 1–4Google Scholar
  6. 6.
    Breß S (2013) Why it is time for a HyPE: a hybrid query processing engine for efficient GPU coprocessing in DBMS. Proceedings VLDB Endowment 6(12):1398–1403CrossRefGoogle Scholar
  7. 7.
    Breß S (2014) The design and implementation of CoGaDB: a column-oriented GPU-accelerated DBMS. Datenbank Spektrum 14(3):199–209. CrossRefGoogle Scholar
  8. 8.
    Breß S, Beier F, Rauhe H, Sattler KU, Schallehn E, Saake G (2013) Efficient co-processor utilization in database query processing. Inf Syst 38(8):1084–1096. CrossRefGoogle Scholar
  9. 9.
    Breß S, Funke H, Teubner J (2016) Robust query processing in co-processor-accelerated databases. In: SIGMOD, pp 1891–1906 CrossRefGoogle Scholar
  10. 10.
    Breß S, Köcher B, Funke H, Rabl T, Markl V (2018) Generating custom code for efficient query execution on heterogeneous processors. VLDB J. CrossRefGoogle Scholar
  11. 11.
    Casper J, Olukotun K (2014) Hardware acceleration of database operations. In: FPGA, pp 151–160Google Scholar
  12. 12.
    Funke H, Breß S, Noll S, Markl V, Teubner J (2018) Pipelined query processing in coprocessor environments. In: SIGMOD, pp 1603–1618 CrossRefGoogle Scholar
  13. 13.
    Halstead RJ, Sukhwani B, Min H, Thoennes M, Dube P, Asaad SW, Iyer B (2013) Accelerating join operation for relational databases with FPGAs. In: FCCM, pp 17–20Google Scholar
  14. 14.
    Hampel V, Pionteck T, Maehle E (2012) An approach for performance estimation of hybrid systems with FPGAs and GPUs as coprocessors. In: ARCS, pp 160–171Google Scholar
  15. 15.
    He B, Lu M, Yang K, Fang R, Govindaraju NK, Luo Q, Sander PV (2009) Relational query coprocessing on graphics processors. ACM Trans Database Syst 34(4):Art. No 21. CrossRefGoogle Scholar
  16. 16.
    Heimel M, Saecker M, Pirk H, Manegold S, Markl V (2013) Hardware-oblivious parallelism for in-memory column-stores. Proceedings VLDB Endowment 6(9):709–720CrossRefGoogle Scholar
  17. 17.
    Hemsoth N (2016) Baidu takes FPGA approach to accelerating SQL at scale. The next platform. Accessed 24 Aug 2018Google Scholar
  18. 18.
    István Z, Sidler D, Alonso G (2016) Runtime parameterizable regular expression operators for databases. In: FCCM, pp 204–211Google Scholar
  19. 19.
    Kaldewey T, Lohman GM, Müller R, Volk PB (2012) GPU join processing revisited. In: DaMoN, pp 55–62CrossRefGoogle Scholar
  20. 20.
    Kara K, Giceva J, Alonso G (2017) FPGA-based data partitioning. In: SIGMOD, pp 433–445Google Scholar
  21. 21.
    Karnagel T, Habich D, Schlegel B, Lehner W (2014) Heterogeneity-aware operator placement in column-store DBMS. Datenbank Spektrum 14(3):211–221. Scholar
  22. 22.
    Karnagel T, Habich D, Lehner W (2015) Local vs. global optimization: operator placement strategies in heterogeneous environments. In: Proceedings of the Workshops of the EDBT/ICDT 2015 Joint Conference, Brussels, Belgium, 27 March 2015, pp 48–55Google Scholar
  23. 23.
    Karnagel T, Habich D, Lehner W (2017) Adaptive work placement for query processing on heterogeneous computing resources. Proceedings VLDB Endowment 10(7):733–744CrossRefGoogle Scholar
  24. 24.
    Kim C, Sedlar E, Chhugani J, Kaldewey T, Nguyen AD, Blas AD, Lee VW, Satish N, Dubey P (2009) Sort vs. hash revisited: fast join implementation on modern multi-core CPUs. Proceedings VLDB Endowment 2(2):1378–1389CrossRefGoogle Scholar
  25. 25.
    Kim C, Chhugani J, Satish N, Sedlar E, Nguyen AD, Kaldewey T, Lee VW, Brandt SA, Dubey P (2010) FAST: fast architecture sensitive tree search on modern CPUs and GPUs. In: SIGMOD, pp 339–350Google Scholar
  26. 26.
    Koch D, Tørresen J (2011) FPGASort: a high performance sorting architecture exploiting run-time reconfiguration on FPGAs for large problem sorting. In: FPGA, pp 45–54Google Scholar
  27. 27.
    Lang W, Kandhan R, Patel JM (2011) Rethinking query processing for energy efficiency: slowing down to win the race. IEEE Data Eng Bull 34(1):12–23Google Scholar
  28. 28.
    Manegold S, Boncz P, Kersten ML (2002) Generic database cost models for hierarchical memory systems. In: Proceedings VLDB Endowment  pp 191–202Google Scholar
  29. 29.
    Müller R, Teubner J (2009) FPGA: what’s in it for a database? In: SIGMOD, pp 999–1004Google Scholar
  30. 30.
    Müller R, Teubner J, Alonso G (2009) Streams on wires – A query compiler for FPGAs. Proceedings VLDB Endowment 2(1):229–240CrossRefGoogle Scholar
  31. 31.
    Müller R, Teubner J, Alonso G (2010) Glacier: a query-to-hardware compiler. In: SIGMOD, pp 1159–1162Google Scholar
  32. 32.
    Müller R, Teubner J, Alonso G (2012) Sorting networks on FPGAs. VLDB J 21(1):1–23. Scholar
  33. 33.
    Neumann T, Leis V (2014) Compiling database queries into machine code. IEEE Data Eng Bull 37(1):3–11Google Scholar
  34. 34.
    Owaida M, Sidler D, Kara K, Alonso G (2017) Centaur: a framework for hybrid CPU-FPGA databases. In: FCCM, pp 211–218Google Scholar
  35. 35.
    Pirk H, Moll O, Zaharia M, Madden S (2016) Voodoo – a vector algebra for portable database performance on modern hardware. Proceedings VLDB Endowment 9(14):1707–1718CrossRefGoogle Scholar
  36. 36.
    Pohl C (2017) Exploiting manycore architectures for parallel data stream processing. In: GvDB, pp 66–71 ( Scholar
  37. 37.
    Polychroniou O, Raghavan A, Ross KA (2015) Rethinking SIMD vectorization for in-memory databases. In: SIGMOD, pp 1493–1508Google Scholar
  38. 38.
    Putnam A, Caulfield A, Chung E, Chiou D, Constantinides K, Demme J, Esmaeilzadeh H, Fowers J, Gray J, Haselman M, Hauck S, Heil S, Hormati A, Kim JY, Lanka S, Peterson E, Smith A, Thong J, Xiao PY, Burger D, Larus J, Gopal GP, Pope S (2014) A reconfigurable fabric for accelerating large-scale datacenter services. In: ISCA, pp 13–24Google Scholar
  39. 39.
    Raducanu B, Boncz PA, Zukowski M (2013) Micro adaptivity in vectorwise. In: SIGMOD, pp 1231–1242 CrossRefGoogle Scholar
  40. 40.
    Rosenfeld V, Heimel M, Viebig C, Markl V (2015) The operator variant selection problem on heterogeneous hardware. In: ADMS, pp 1–12Google Scholar
  41. 41.
    Sidhu RPS, Prasanna VK (2001) Fast regular expression matching using FPGAs. In: FCCM, pp 227–238Google Scholar
  42. 42.
    Sidler D, István Z, Owaida M, Alonso G (2017) Accelerating pattern matching queries in hybrid CPU-FPGA architectures. In: SIGMOD, pp 403–415CrossRefGoogle Scholar
  43. 43.
    Sitaridi E, Ross K (2013) Optimizing select conditions on GPUs. In: DaMoN, ACM. pp, vol 4, pp 1–4Google Scholar
  44. 44.
    Sukhwani B, Thoennes M, Min H, Dube P, Brezzo B, Asaad SW, Dillenberger D (2013) Large payload streaming database sort and projection on FPGAs. In: SBAC-PAD, pp 25–32Google Scholar
  45. 45.
    Sukhwani B, Thoennes M, Min H, Dube P, Brezzo B, Asaad SW, Dillenberger D (2015) A hardware/software approach for database query acceleration with FPGAs. Int J Parallel Program 43(6):1129–1159CrossRefGoogle Scholar
  46. 46.
    Ueda T, Ito M, Ohara M (2015) A dynamically reconfigurable equi-joiner on FPGA. IBM Technical Report RT0969Google Scholar
  47. 47.
    Ungethüm A, Habich D, Karnagel T, Lehner W, Asmussen N, Völp M, Noethen B, Fettweis GP (2015) Query processing on low-energy many-core processors. In: ICDE, pp 155–160CrossRefGoogle Scholar
  48. 48.
    Wang Z, Paul J, Cheah HY, He B, Zhang W (2016) Relational query processing on OpenCL-based FPGAs. In: FPL, pp 1–10Google Scholar
  49. 49.
    Xu Z, Tu Y, Wang X (2012) PET: reducing database energy cost via query optimization. Proceedings VLDB Endowment 5(12):1954–1957CrossRefGoogle Scholar
  50. 50.
    Yuan Y, Lee R, Zhang X (2013) The yin and yang of processing data warehousing queries on GPU devices. Proceedings VLDB Endowment 6(10):817–828CrossRefGoogle Scholar
  51. 51.
    Zhang S, He J, He B, Lu M (2013) OmniDB: Towards portable and efficient query processing on parallel CPU/GPU architectures. Proceedings VLDB Endowment 6(12):1374–1377CrossRefGoogle Scholar
  52. 52.
    Ziener D, Bauer F, Becher A, Dennl C, Meyer-Wegener K, Schürfeld U, Teich J, Vogt J, Weber H (2016) FPGA-based dynamically reconfigurable SQL query processing. ACM Trans Reconfigurable Technol Syst 9(4):Article No. 25CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Deutschland, ein Teil von Springer Nature 2018

Authors and Affiliations

  • Andreas Becher
    • 1
    Email author
  • Lekshmi B.G.
    • 1
  • David Broneske
    • 2
  • Tobias Drewes
    • 2
  • Bala Gurumurthy
    • 2
  • Klaus Meyer-Wegener
    • 1
  • Thilo Pionteck
    • 2
  • Gunter Saake
    • 2
  • Jürgen Teich
    • 1
  • Stefan Wildermann
    • 1
  1. 1.Friedrich-Alexander-Universität Erlangen NürnbergNurembergGermany
  2. 2.Otto-von-Guericke-UniversityMagdeburgGermany

Personalised recommendations