Stream Processing with a Spreadsheet

  • Mandana Vaziri
  • Olivier Tardieu
  • Rodric Rabbah
  • Philippe Suter
  • Martin Hirzel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8586)

Abstract

Continuous data streams are ubiquitous and represent such a high volume of data that they cannot be stored to disk, yet it is often crucial for them to be analyzed in real-time. Stream processing is a programming paradigm that processes these immediately, and enables continuous analytics. Our objective is to make it easier for analysts, with little programming experience, to develop continuous analytics applications directly. We propose enhancing a spreadsheet, a pervasive tool, to obtain a programming platform for stream processing. We present the design and implementation of an enhanced spreadsheet that enables visualizing live streams, live programming to compute new streams, and exporting computations to be run on a server where they can be shared with other users, and persisted beyond the life of the spreadsheet. We formalize our core language, and present case studies that cover a range of stream processing applications.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    The Akka project, http://akka.io (retrieved November 2013)
  2. 2.
    Ali, M., Chandramouli, B., Goldstein, J., Schindlauer, R.: The extensibility framework in Microsoft StreamInsight. In: International Conference on Data Engineering (ICDE), pp. 1242–1253 (2011)Google Scholar
  3. 3.
    Amadio, R.M., Boudol, G., Castellani, I., Boussinot, F.: Reactive concurrent programming revisited. CoRR abs/cs/0512058 (2005)Google Scholar
  4. 4.
    Arasu, A., Babu, S., Widom, J.: The CQL continuous query language: semantic foundations and query execution. Journal on Very Large Data Bases (VLDB J.) 15(2), 121–142 (2006)CrossRefGoogle Scholar
  5. 5.
    Auerbach, J., Bacon, D.F., Cheng, P., Rabbah, R.: Lime: a Java-compatible and synthesizable language for heterogeneous architectures. In: Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), pp. 89–108 (2010)Google Scholar
  6. 6.
    Benveniste, A., Caspi, P., Edwards, S.A., Halbwachs, N., Guernic, P.L., de Simone, R.: The synchronous languages 12 years later. Proceedings of the IEEE 91(1), 64–83 (2003)CrossRefGoogle Scholar
  7. 7.
    Bouillet, E., Feblowitz, M., Liu, Z., Ranganathan, A., Riabov, A.: A tag-based approach for the design and composition of information processing applications. In: Onward! Track of Object-Oriented Programming, Systems, Languages, and Applications (Onward!), pp. 585–602 (2008)Google Scholar
  8. 8.
    Caspi, P., Pilaud, D., Halbwachs, N., Raymond, P.: Lustre: a declarative language for real-time programming. In: Symposium on Principles of Programming Languages (POPL), pp. 178–188 (1987)Google Scholar
  9. 9.
    Caspi, P., Pouzet, M.: Synchronous Kahn networks. In: International Conference on Functional Programming (ICFP), pp. 226–238 (1996)Google Scholar
  10. 10.
    Cloudscale big data analytics, http://www.hashdoc.com/document/8626/big-data-analytics (retrieved November 2013)
  11. 11.
    Demers, A., Gehrke, J., Panda, B., Riedewald, M., Sharma, V., White, W.: Cayuga: A general purpose event monitoring system. In: Conference on Innovative Data Systems Research (CIDR), pp. 412–422 (2007)Google Scholar
  12. 12.
    Fielding, R.T., Taylor, R.N.: Principled design of the modern web architecture. ACM Trans. Internet Technol. 2(2), 115–150 (2002)CrossRefGoogle Scholar
  13. 13.
    Gordon, M.I., Thies, W., Amarasinghe, S.: Exploiting coarse-grained task, data, and pipeline parallelism in stream programs. In: Architectural Support for Programming Languages and Operating Systems (ASPLOS), pp. 151–162 (2006)Google Scholar
  14. 14.
    Hirzel, M.: Partition and compose: Parallel complex event processing. In: Conference on Distributed Event-Based Systems (DEBS), pp. 191–200 (2012)Google Scholar
  15. 15.
    Hirzel, M., Andrade, H., Gedik, B., Jacques-Silva, G., Khandekar, R., Kumar, V., Mendell, M., Nasgaard, H., Schneider, S., Soulé, R., Wu, K.L.: IBM Streams Processing Language: Analyzing big data in motion. IBM Journal of Research & Development 57(3/4), 7:1–7:11 (2013)Google Scholar
  16. 16.
    Lisper, B., Malström, J.: Haxcel: A spreadsheet interface to Haskell. In: Workshop on the Implementation of Functional Languages (IFL), pp. 206–222 (2002)Google Scholar
  17. 17.
    McGarry, J.: Processing continuous data streams in electronic spreadsheets. Patent No. US 6,490,600 B1 (2002)Google Scholar
  18. 18.
    Potop-Butucaru, D., Edwards, S.A., Berry, G.: Compiling Esterel, 1st edn. Springer Publishing Company, Incorporated (2007)Google Scholar
  19. 19.
    Serafimova, I.: Spreadsheet-based template language prototype for tree data structure description and interpretation. In: International Conference on Computer Systems and Technologies (CompSysTech), pp. 148–154 (2012)Google Scholar
  20. 20.
    Sestoft, P.: Implementing function spreadsheets. In: Workshop on End-User Software Engineering (WEUSE), pp. 91–94 (2008)Google Scholar
  21. 21.
  22. 22.
    Wakeling, D.: Spreadsheet functional programming. Journal of Functional Programming (JFP) 17(1), 131–143 (2007)CrossRefMATHGoogle Scholar
  23. 23.
    Woo, A., Seth, S., Olson, T., Liu, J., Zhao, F.: A spreadsheet approach to programming and managing sensor networks. In: Conference on Information Processing in Sensor Networks (IPSN), pp. 424–431 (2006)Google Scholar
  24. 24.
    Wu, E., Diao, Y., Rizvi, S.: High-performance complex event processing over streams. In: International Conference on Management of Data (SIGMOD), pp. 407–418 (2006)Google Scholar
  25. 25.
    Zaharia, M., Das, T., Li, H., Hunter, T., Shenker, S., Stoica, I.: Discretized streams: Fault-tolerant streaming computation at scale. In: Symposium on Operating Systems Principles (SOSP), pp. 423–438 (2013)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Mandana Vaziri
    • 1
  • Olivier Tardieu
    • 1
  • Rodric Rabbah
    • 1
  • Philippe Suter
    • 1
  • Martin Hirzel
    • 1
  1. 1.IBM T.J. Watson Research CenterYorktown HeightUSA

Personalised recommendations