RSPLab: RDF Stream Processing Benchmarking Made Easy

  • Riccardo Tommasini
  • Emanuele Della Valle
  • Andrea Mauri
  • Marco Brambilla
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10588)


In Stream Reasoning (SR), empirical research on RDF Stream Processing (RSP) is attracting a growing attention. The SR community proposed methodologies and benchmarks to investigate the RSP solution space and improve existing approaches. In this paper, we present RSPLab, an infrastructure that reduces the effort required to design and execute reproducible experiments as well as share their results. RSPLab integrates two existing RSP benchmarks (LSBench and CityBench) and two RSP engines (C-SPARQL engine and CQELS). It provides a programmatic environment to: deploy in the cloud RDF Streams and RSP engines, interact with them using TripleWave and RSP Services, and continuously monitor their performances and collect statistics. RSPLab is released as open-source under an Apache 2.0 license.


Semantic Web Stream Reasoning RDF Stream Processing Benchmarking 


  1. 1.
    Ali, M.I., Gao, F., Mileo, A.: CityBench: a configurable benchmark to evaluate RSP engines using smart city datasets. In: Arenas, M., et al. (eds.) ISWC 2015. LNCS, vol. 9367, pp. 374–389. Springer, Cham (2015). doi: 10.1007/978-3-319-25010-6_25 CrossRefGoogle Scholar
  2. 2.
    Balduini, M., Della Valle, E.: A restful interface for RDF stream processors. In: Proceedings of the ISWC 2013 Posters and Demonstrations Track, Sydney, pp. 209–212, 23 October 2013Google Scholar
  3. 3.
    Barbieri, D.F., Braga, D., Ceri, S., Della Valle, E., Grossniklaus, M.: C-SPARQL: a continuous query language for RDF data streams. Int. J. Semant. Comput. 4(1), 3–25 (2010)CrossRefMATHGoogle Scholar
  4. 4.
    Boettiger, C.: An introduction to Docker for reproducible research. Oper. Syst. Rev. 49(1), pp. 71–79 (2015).
  5. 5.
    Dell’Aglio, D., Calbimonte, J.-P., Balduini, M., Corcho, O., Della Valle, E.: On correctness in RDF stream processor benchmarking. In: Alani, H., Kagal, L., Fokoue, A., Groth, P., Biemann, C., Parreira, J.X., Aroyo, L., Noy, N., Welty, C., Janowicz, K. (eds.) ISWC 2013. LNCS, vol. 8219, pp. 326–342. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-41338-4_21 CrossRefGoogle Scholar
  6. 6.
    Dell’Aglio, D., Della Valle, E., Calbimonte, J., Corcho, Ó.: RSP-QL semantics: a unifying query model to explain heterogeneity of RDF stream processing systems. Int. J. Semant. Web Inf. Syst. 10(4), 17–44 (2014)CrossRefGoogle Scholar
  7. 7.
    Difallah, D.E., Pavlo, A., Curino, C., Cudré-Mauroux, P.: Oltp-bench: an extensible testbed for benchmarking relational databases. PVLDB 7(4), 277–288 (2013)Google Scholar
  8. 8.
    Kolchin, M., Wetz, P., Kiesling, E., Tjoa, A.M.: YABench: a comprehensive framework for RDF stream processor correctness and performance assessment. In: Bozzon, A., Cudre-Maroux, P., Pautasso, C. (eds.) ICWE 2016. LNCS, vol. 9671, pp. 280–298. Springer, Cham (2016). doi: 10.1007/978-3-319-38791-8_16 Google Scholar
  9. 9.
    Mauri, A., Calbimonte, J.-P., Dell’Aglio, D., Balduini, M., Brambilla, M., Della Valle, E., Aberer, K.: TripleWave: spreading RDF streams on the web. In: Groth, P., Simperl, E., Gray, A., Sabou, M., Krötzsch, M., Lecue, F., Flöck, F., Gil, Y. (eds.) ISWC 2016. LNCS, vol. 9982, pp. 140–149. Springer, Cham (2016). doi: 10.1007/978-3-319-46547-0_15 CrossRefGoogle Scholar
  10. 10.
    Le-Phuoc, D., Dao-Tran, M., Xavier Parreira, J., Hauswirth, M.: A native and adaptive approach for unified processing of linked streams and linked data. In: Aroyo, L., Welty, C., Alani, H., Taylor, J., Bernstein, A., Kagal, L., Noy, N., Blomqvist, E. (eds.) ISWC 2011. LNCS, vol. 7031, pp. 370–388. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-25073-6_24 CrossRefGoogle Scholar
  11. 11.
    Le-Phuoc, D., Dao-Tran, M., Pham, M.-D., Boncz, P., Eiter, T., Fink, M.: Linked stream data processing engines: facts and figures. In: Cudré-Mauroux, P., Heflin, J., Sirin, E., Tudorache, T., Euzenat, J., Hauswirth, M., Parreira, J.X., Hendler, J., Schreiber, G., Bernstein, A., Blomqvist, E. (eds.) ISWC 2012. LNCS, vol. 7650, pp. 300–312. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-35173-0_20 CrossRefGoogle Scholar
  12. 12.
    Ren, X., Khrouf, H., Kazi-Aoul, Z., Chabchoub, Y., Curé, O.: On measuring performances of C-SPARQL and CQELS. CoRR abs/1611.08269Google Scholar
  13. 13.
    Rietveld, L., Beek, W., Schlobach, S.: LOD lab: experiments at LOD scale. In: Arenas, M., Corcho, O., Simperl, E., Strohmaier, M., d’Aquin, M., Srinivas, K., Groth, P., Dumontier, M., Heflin, J., Thirunarayan, K., Staab, S. (eds.) ISWC 2015. LNCS, vol. 9367, pp. 339–355. Springer, Cham (2015). doi: 10.1007/978-3-319-25010-6_23 CrossRefGoogle Scholar
  14. 14.
    Scharrenbach, T., Urbani, J., Margara, A., Della Valle, E., Bernstein, A.: Seven commandments for benchmarking semantic flow processing systems. In: Cimiano, P., Corcho, O., Presutti, V., Hollink, L., Rudolph, S. (eds.) ESWC 2013. LNCS, vol. 7882, pp. 305–319. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-38288-8_21 CrossRefGoogle Scholar
  15. 15.
    Stupar, A., Michel, S.: Srbench-a benchmark for soundtrack recommendation systems. In: 22nd ACM International Conference on Information and Knowledge Management, CIKM 2013, San Francisco, pp. 2285–2290 October 27 - November 1, (2013)Google Scholar
  16. 16.
    Tommasini, R.: streamreasoning/rsplib: Rsplib beta v0.2.4.
  17. 17.
    Tommasini, R., Della Valle, E., Balduini, M., Dell’Aglio, D.: Heaven: a framework for systematic comparative research approach for RSP engines. In: Sack, H., Blomqvist, E., d’Aquin, M., Ghidini, C., Ponzetto, S.P., Lange, C. (eds.) ESWC 2016. LNCS, vol. 9678, pp. 250–265. Springer, Cham (2016). doi: 10.1007/978-3-319-34129-3_16 CrossRefGoogle Scholar
  18. 18.
    Tommasini, R., Mauri, A.: streamreasoning/rsplab: Rsplab v0.9.

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Riccardo Tommasini
    • 1
  • Emanuele Della Valle
    • 1
  • Andrea Mauri
    • 1
  • Marco Brambilla
    • 1
  1. 1.DEIBPolitecnico di MilanoMilanItaly

Personalised recommendations