Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Benchmarking, Measuring and Optimization

Bench 2019: Benchmarking, Measuring, and Optimizing pp 335–347Cite as

Apache Spark Streaming, Kafka and HarmonicIO: A Performance Benchmark and Architecture Comparison for Enterprise and Scientific Computing

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12093))

Abstract

Many scientific computing applications generate streams where message sizes exceed one megabyte, in contrast with smaller message sizes in enterprise contexts (order kilobytes, often XML or JSON). Furthermore, the processing cost of messages in scientific computing applications are usually an order of magnitude higher than in typical enterprise applications. Frameworks such as Apache Spark offer high throughput processing of streams with such ‘enterprise’ characteristics, as well as scalability, with high resilience and many other desirable features. Motivated by the development of near real-time image processing pipelines for roboticized microscopy, we evaluate the suitability of Apache Spark for streams more typical of scientific computing applications, those with large message sizes (up to 10 MB), and heavy per-message CPU load, under typical stream integrations. For comparison, we benchmark a P2P stream processing framework, HarmonicIO, developed in-house. Our study reveals a complex interplay of performance trade-offs, revealing the boundaries of good performance for each framework and integration over a wide domain of application loads. Based on these results, we suggest which are likely to offer good performance for a given load. Broadly, the advantages of Spark’s rich features makes its performance sensitive to message size in particular, whereas the simplicity of HarmonicIO offers more robust performance, and better CPU utilization.

The HASTE Project (http://haste.research.it.uu.se/) is funded by the Swedish Foundation for Strategic Research (SSF) under award no. BD15-0008, and the eSSENCE strategic collaboration for eScience. Computational resources were provided by the Swedish National Infrastructure for Computing via the SNIC Science Cloud (SSC) [13], an OpenStack-based community cloud for Swedish academia.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://haste.reserach.it.uu.se.

  2. 2.

    Available at: https://github.com/HASTE-project/benchmarking-tools.

  3. 3.

    https://docs.oracle.com/javase/7/docs/api/java/lang/management/ThreadMXBean.html.

  4. 4.

    Now implemented, see: https://issues.apache.org/jira/browse/SPARK-20568.

References

  1. Apache Software Foundation: Apache Hadoop. http://hadoop.apache.org/ (2011)

  2. Apache Software Foundation: Cassandra Limitations. https://cwiki.apache.org/confluence/display/CASSANDRA2/CassandraLimitations (2019)

  3. Ausmees, K., John, A., Toor, S.Z., Hellander, A., Nettelblad, C.: BAMSI: a multi-cloud service for scalable distributed filtering of massive genome data. BMC Bioinform. 19(1), 240 (2018). https://doi.org/10.1186/s12859-018-2241-z

    Article  Google Scholar 

  4. Blamey, B., Wrede, F., Karlsson, J., Hellander, A., Toor, S.: Adapting the secretary hiring problem for optimal hot-cold tier placement under Top-K workloads. In: 2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID). pp. 576–583, May 2019. https://doi.org/10.1109/CCGRID.2019.00074

  5. Carbone, P., Katsifodimos, A., Ewen, S., Markl, V., Haridi, S., Tzoumas, K.: Apache Flink™: stream and batch processing in a single engine. Bull. IEEE Comput. Soc. Techn. Committee Data Eng. 36(4), 28–38 (2015)

    Google Scholar 

  6. Chintapalli, S., et al.: Benchmarking streaming computation engines: storm, flink and spark streaming. In: 2016 IEEE International Parallel and Distributed Processing Symposium Workshops, pp. 1789–1792. IEEE (2016)

    Google Scholar 

  7. Kreps, J., Narkhede, N., Rao, J., et al.: Kafka: a distributed messaging system for log processing. In: Proceedings of the NetDB, pp. 1–7 (2011)

    Google Scholar 

  8. Lugnegård, L.: Building a high throughput microscope simulator using the Apache Kafka streaming framework (M.sc. Thesis) (2018)

    Google Scholar 

  9. Marcu, O.C., Costan, A., Antoniu, G., Pérez-Hernández, M.S.: Spark versus flink: understanding performance in big data analytics frameworks. In: 2016 IEEE International Conference on Cluster Computing (CLUSTER), pp. 433–442, September 2016. https://doi.org/10.1109/CLUSTER.2016.22

  10. National Energy Research Scientific Computing Center: Characterization of the DOE Mini-apps. https://portal.nersc.gov/project/CAL/designforward.htm, June 2014

  11. Qian, S., Wu, G., Huang, J., Das, T.: Benchmarking modern distributed streaming platforms. In: 2016 IEEE International Conference on Industrial Technology (ICIT), pp. 592–598, March 2016. https://doi.org/10.1109/ICIT.2016.7474816

  12. Salvatore Sanfilippo: Redis (2009). https://redis.io/

  13. Toor, S., et al.: SNIC Science Cloud (SSC): a national-scale cloud infrastructure for Swedish academia. In: 2017 IEEE 13th International Conference On E-Science (e-Science), pp. 219–227. IEEE (2017)

    Google Scholar 

  14. Torruangwatthana, P., Wieslander, H., Blamey, B., Hellander, A., Toor, S.: HarmonicIO: scalable data stream processing for scientific datasets. In: 2018 IEEE 11th International Conference on Cloud Computing (CLOUD), pp. 879–882, July 2018. https://doi.org/10.1109/CLOUD.2018.00126

  15. Wollman, R., Stuurman, N.: High throughput microscopy: from raw images to discoveries. J. Cell Sci. 120(21), 3715–3722 (2007). https://doi.org/10.1242/jcs.013623

    Article  Google Scholar 

  16. Xin, R.: Apache Spark the fastest open source engine for sorting a petabyte, October 2014. https://databricks.com/blog/2014/10/10/spark-petabyte-sort.html

  17. Zaharia, M., et al.: Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, pp. 2–2. NSDI 2012, USENIX Association, San Jose (2012)

    Google Scholar 

  18. Zaharia, M., et al.: Apache spark: a unified engine for big data processing. Commun. ACM 59(11), 56–65 (2016). https://doi.org/10.1145/2934664

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, Division of Scientific Computing, Uppsala University, Uppsala, Sweden

    Ben Blamey, Andreas Hellander & Salman Toor

Authors
  1. Ben Blamey
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andreas Hellander
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Salman Toor
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ben Blamey .

Editor information

Editors and Affiliations

  1. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

    Wanling Gao

  2. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

    Jianfeng Zhan

  3. School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, USA

    Geoffrey Fox

  4. Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, USA

    Xiaoyi Lu

  5. Texas Advanced Computing Center, The University of Texas at Austin, Austin, TX, USA

    Dan Stanzione

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Blamey, B., Hellander, A., Toor, S. (2020). Apache Spark Streaming, Kafka and HarmonicIO: A Performance Benchmark and Architecture Comparison for Enterprise and Scientific Computing. In: Gao, W., Zhan, J., Fox, G., Lu, X., Stanzione, D. (eds) Benchmarking, Measuring, and Optimizing. Bench 2019. Lecture Notes in Computer Science(), vol 12093. Springer, Cham. https://doi.org/10.1007/978-3-030-49556-5_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-49556-5_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-49555-8

  • Online ISBN: 978-3-030-49556-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation