Heaven: A Framework for Systematic Comparative Research Approach for RSP Engines

  • Riccardo Tommasini
  • Emanuele Della Valle
  • Marco Balduini
  • Daniele Dell’Aglio
Conference paper

DOI: 10.1007/978-3-319-34129-3_16

Part of the Lecture Notes in Computer Science book series (LNCS, volume 9678)
Cite this paper as:
Tommasini R., Della Valle E., Balduini M., Dell’Aglio D. (2016) Heaven: A Framework for Systematic Comparative Research Approach for RSP Engines. In: Sack H., Blomqvist E., d'Aquin M., Ghidini C., Ponzetto S., Lange C. (eds) The Semantic Web. Latest Advances and New Domains. ESWC 2016. Lecture Notes in Computer Science, vol 9678. Springer, Cham

Abstract

Benchmarks like LSBench, SRBench, CSRBench and, more recently, CityBench satisfy the growing need of shared datasets, ontologies and queries to evaluate window-based RDF Stream Processing (RSP) engines. However, no clear winner emerges out of the evaluation. In this paper, we claim that the RSP community needs to adopt a Systematic Comparative Research Approach (SCRA) if it wants to move a step forward. To this end, we propose a framework that enables SCRA for window based RSP engines. The contributions of this paper are: (i) the requirements to satisfy for tools that aim at enabling SCRA; (ii) the architecture of a facility to design and execute experiment guaranteeing repeatability, reproducibility and comparability; (iii) \(\mathcal {H}\)eaven – a proof of concept implementation of such architecture that we released as open source –; (iv) two RSP engine implementations, also open source, that we propose as baselines for the comparative research (i.e., they can serve as terms of comparison in future works). We prove \(\mathcal {H}\)eaven effectiveness using the baselines by: (i) showing that top-down hypothesis verification is not straight forward even in controlled conditions and (ii) providing examples of bottom-up comparative analysis.

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Riccardo Tommasini
    • 1
  • Emanuele Della Valle
    • 1
  • Marco Balduini
    • 1
  • Daniele Dell’Aglio
    • 1
  1. 1.DEIBPolitecnico of MilanoMilanoItaly

Personalised recommendations