Advertisement

\(\mathbb {ECHO}\): An Adaptive Orchestration Platform for Hybrid Dataflows across Cloud and Edge

  • Pushkara Ravindra
  • Aakash Khochare
  • Siva Prakash Reddy
  • Sarthak Sharma
  • Prateeksha Varshney
  • Yogesh Simmhan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10601)

Abstract

The Internet of Things (IoT) is offering unprecedented observational data that are used for managing Smart City utilities. Edge and Fog gateway devices are an integral part of IoT deployments to acquire real-time data and enact controls. Recently, Edge-computing is emerging as first-class paradigm to complement Cloud-centric analytics. But a key limitation is the lack of a platform-as-a-service for applications spanning Edge and Cloud. Here, we propose \(\mathbb {ECHO}\), an orchestration platform for dataflows across distributed resources. \(\mathbb {ECHO}\) ’s hybrid dataflow composition can operate on diverse data models – streams, micro-batches and files, and interface with native runtime engines like TensorFlow and Storm to execute them. It manages the application’s lifecycle, including container-based deployment and a registry for state management. \(\mathbb {ECHO}\) can schedule the dataflow on different Edge, Fog and Cloud resources, and also perform dynamic task migration between resources. We validate the \(\mathbb {ECHO}\) platform for executing video analytics and sensor streams for Smart Traffic and Smart Utility applications on Raspberry Pi, NVidia TX1, ARM64 and Azure Cloud VM resources, and present our results.

Notes

Acknowledgments

The authors would like to thank Microsoft Azure and NVIDIA for resource access, and VMWare for their technical feedback. We would also like to thank Venkatesh Babu and Avishek from the VAL lab at IISc for their inputs on YOLO.

References

  1. 1.
    Simmhan, Y., Aman, S., Kumbhare, A., Liu, R., Stevens, S., Zhou, Q., Prasanna, V.: Cloud-based software platform for big data analytics in smart grids. IEEE/AIP Comput. Sci. Eng. (2013)Google Scholar
  2. 2.
    Amrutur, B., Rajaraman, V., Acharya, S., Ramesh, R., Joglekar, A., Sharma, A., Simmhan, Y., Lele, A., Mahesh, A., Sankaran, S.: An open smart city IoT test bed: street light poles as smart city spines. In: ACM/IEEE International Conference on Internet of Things Design and Implementation (2017)Google Scholar
  3. 3.
    Simmhan, Y.: IoT analytics across edge and cloud platforms. IEEE IoT Newsl., May 2017Google Scholar
  4. 4.
    Garcia Lopez, P., Montresor, A., Epema, D., Datta, A., Higashino, T., Iamnitchi, A., Barcellos, M., Felber, P., Riviere, E.: Edge-centric computing: vision and challenges. ACM Comput. Comm. Rev. (2015)Google Scholar
  5. 5.
    Ghosh, R., Simmhan, Y.: Distributed scheduling of event analytics across edge and cloud, CoRR, no. 1608.01537 (2016)Google Scholar
  6. 6.
    Varshney, P., Simmhan, Y.: Demystifying fog computing: Characterizing architectures, applications and abstractions. In: IEEE International Conference on Fog and Edge Computing (2017)Google Scholar
  7. 7.
    Mineraud, J., Mazhelis, O., Su, X., Tarkoma, S.: A gap analysis of internet-of-things platforms. Comput. Commun. 89, 5–16 (2016)CrossRefGoogle Scholar
  8. 8.
    Eclipse Kura, http://www.eclipse.org/kura/. Accessed 21 June 2017
  9. 9.
    VMware Liota, https://github.com/vmware/liota. Accessed 21 June 2017
  10. 10.
    Apache Edgent, v1.1.0, http://edgent.apache.org/. Accessed 21 June 2017
  11. 11.
    Abadi, M., et al.: Tensorflow: large-scale machine learning on heterogeneous distributed systems. In: USENIX Symposium on Operating Systems Design and Implementation (2016)Google Scholar
  12. 12.
    Beart, P.: Automatic resource discovery for the internet of things - specification, The British Standards Institution. Tech. Rep. PAS 212:2016 (2016)Google Scholar
  13. 13.
    Ess, A., Leibe, B., Schindler, K., van Gool, L.: A mobile vision system for robust multi-person tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (2008)Google Scholar
  14. 14.
    Shukla, A., Chaturvedi, S., Simmhan, Y.: RIoTBench: a real-time IoT benchmark for distributed stream processing platforms, CoRR, no. 1701.08530 (2017)Google Scholar
  15. 15.
    Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger, CoRR, no. 1612.08242 (2016)Google Scholar
  16. 16.
    Georgantas, N., Billet, B.: Revisiting service-oriented architecture for the IoT: a middleware perspective. In: International Conference on Service Oriented Computing (2016)Google Scholar
  17. 17.
    Billet, B., Issarny, V.: From task graphs to concrete actions: a new task mapping algorithm for the future internet of things. In: IEEE International Conference on Mobile Ad Hoc Sensor Systems (2014)Google Scholar
  18. 18.
    Hong, K., Lillethun, D., Ramachandran, U., Ottenwälder, B., Koldehofe, B.: Mobile fog: a programming model for large-scale applications on the internet of things. In: ACM SIGCOMM Workshop on Mobile Cloud Computing (2013)Google Scholar
  19. 19.
    Yang, L., Cao, J., Yuan, Y., Li, T., Han, A., Chan, A.: A framework for partitioning and execution of data stream applications in mobile cloud computing. ACM SIGMETRICS Performance Eval. Rev. 40(4) (2013)Google Scholar
  20. 20.
    Reiter, A., Prünster, B., Zefferer, T.: Hybrid mobile edge computing: Unleashing the full potential of edge computing in mobile device use cases. In: IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) (2017)Google Scholar
  21. 21.
    Chun, B.-G., Ihm, S., Maniatis, P., Naik, M., Patti, A.: Clonecloud: elastic execution between mobile device and cloud. In: Conference on Computer Systems (2011)Google Scholar
  22. 22.
    Anderson, D.P., Cobb, J., Korpela, E., Lebofsky, M., Werthimer, D.: Seti@ home: an experiment in public-resource computing. CACM 45(11) (2002)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Pushkara Ravindra
    • 1
  • Aakash Khochare
    • 1
  • Siva Prakash Reddy
    • 1
  • Sarthak Sharma
    • 1
  • Prateeksha Varshney
    • 1
  • Yogesh Simmhan
    • 1
  1. 1.Indian Institute of ScienceBangaloreIndia

Personalised recommendations