P-Bench: Benchmarking in Data-Centric Pervasive Application Development

  • Sabina Surdu
  • Yann Gripay
  • Vasile-Marian Scuturici
  • Jean-Marc Petit
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8290)


Developing complex data-centric applications, which manage intricate interactions between distributed and heterogeneous entities from pervasive environments, is a tedious task. In this paper we pursue the difficult objective of assessing the ”easiness” of data-centric development in pervasive environments, which turns out to be much more challenging than simply measuring execution times in performance analyses and requires highly qualified programmers. We introduce P-Bench, a benchmark that comparatively evaluates the easiness of development using three types of systems: (1) the Microsoft StreamInsight unmodified Data Stream Management System, LINQ and C#, (2) the StreamInsight++ ad hoc framework, an enriched version of StreamInsight, that meets pervasive application requirements, and (3) our SoCQ system, designed for managing data, streams and services in a unified manner. We define five tasks that we implement in the analyzed systems, based on core needs for pervasive application development. To evaluate the tasks’ implementations, we introduce a set of metrics and provide the experimental results. Our study allows differentiating between the proposed types of systems based on their strengths and weaknesses when building pervasive applications.


pervasive environments data-centric pervasive applications heterogeneous data continuous queries benchmarking 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abiteboul, S., Manolescu, I., Taropa, E.: A Framework for Distributed XML Data Management. In: Ioannidis, Y., Scholl, M.H., Schmidt, J.W., Matthes, F., Hatzopoulos, M., Böhm, K., Kemper, A., Grust, T., Böhm, C. (eds.) EDBT 2006. LNCS, vol. 3896, pp. 1049–1058. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  2. 2.
    Arasu, A., Babcock, B., Babu, S., Datar, M., Ito, K., Motwani, R., Nishizawa, I., Srivastava, U., Thomas, D., Varma, R., Widom, J.: STREAM: The Stanford Stream Data Manager. IEEE Data Eng. Bull. 26(1), 19–26 (2003)Google Scholar
  3. 3.
    Arasu, A., Cherniack, M., Galvez, E.F., Maier, D., Maskey, A., Ryvkina, E., Stonebraker, M., Tibbetts, R.: Linear Road: A Stream Data Management Benchmark. In: VLDB 2004, pp. 480–491 (2004)Google Scholar
  4. 4.
    Creus Tomàs, J., Amann, B., Travers, N., Vodislav, D.: RoSeS: A Continuous Query Processor for Large-Scale RSS Filtering and Aggregation. In: CIKM 2011, pp. 2549–2552 (2011)Google Scholar
  5. 5.
    Cuevas-Vicenttín, V., Vargas-Solar, G., Collet, C.: Evaluating Hybrid Queries through Service Coordination in HYPATIA (demo). In: EDBT 2012, pp. 602–605 (2012)Google Scholar
  6. 6.
    Fenton, N.E., Pfleeger, S.L.: Software Metrics: A Rigorous and Practical Approach. PWS Publishing Co., Boston (1998)Google Scholar
  7. 7.
    Gray, J.: Benchmark Handbook: For Database and Transaction Processing Systems. Morgan Kaufmann Publishers Inc., San Francisco (1992)Google Scholar
  8. 8.
    Gripay, Y., Laforest, F., Lesueur, F., Lumineau, N., Petit, J.-M., Scuturici, V.-M., Sebahi, S., Surdu, S.: ColisTrack: Testbed for a Pervasive Environment Management System (demo). In: EDBT 2012, pp. 574–577 (2012)Google Scholar
  9. 9.
    Gripay, Y., Laforest, F., Petit, J.-M.: A Simple (yet Powerful) Algebra for Pervasive Environments. In: EDBT 2010, pp. 359–370 (2010)Google Scholar
  10. 10.
    Kazemitabar, S.J., Demiryurek, U., Ali, M.H., Akdogan, A., Shahabi, C.: Geospatial Stream Query Processing using Microsoft SQL Server StreamInsight. PVLDB 3(2), 1537–1540 (2010)Google Scholar
  11. 11.
    Meijer, E.: The World According to LINQ. Commun. ACM 54(10), 45–51 (2011)CrossRefGoogle Scholar
  12. 12.
    Pugh, W.: Technical Perspective: A Methodology for Evaluating Computer System Performance. Commun. ACM 51(8), 82–82 (2008)CrossRefGoogle Scholar
  13. 13.
    Rys, M.: Scalable SQL. Commun. ACM 54(6), 48–53 (2011)CrossRefGoogle Scholar
  14. 14.
    Scuturici, V.-M., Surdu, S., Gripay, Y., Petit, J.-M.: UbiWare: Web-Based Dynamic Data & Service Management Platform for AmI. In: Middleware 2012, pp. 11:1–11:2 (2012)Google Scholar
  15. 15.
    International Telecommunication Union: ITU Internet Reports. The Internet of Things. International Telecommunication Union (2005)Google Scholar
  16. 16.
    Weiser, M.: The Computer for the 21st Century. Scientific American 265(3), 94–104 (1991)CrossRefGoogle Scholar
  17. 17.
    Xue, W., Luo, Q.: Action-Oriented Query Processing for Pervasive Computing. In: CIDR 2005, pp. 305–316 (2005)Google Scholar
  18. 18.
  19. 19.
  20. 20.
    Transaction Processing Performance Council,
  21. 21.
    Wheeler, D.: Counting Source Lines of Code (SLOC),

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Sabina Surdu
    • 1
    • 2
    • 3
  • Yann Gripay
    • 1
    • 2
  • Vasile-Marian Scuturici
    • 1
    • 2
  • Jean-Marc Petit
    • 1
    • 2
  1. 1.CNRSUniversité de LyonFrance
  2. 2.LIRIS, UMR5205INSA-LyonFrance
  3. 3.Faculty of Mathematics and Computer ScienceUBB Cluj-NapocaRomania

Personalised recommendations