Towards Intelligent Closed-Loop Workflows for Ecological Research

  • JD Knapp
  • Matias Elo
  • James Shaeffer
  • Paul G. Flikkema
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8964)


Spurred by needs related to research on the effects of climate change on ecological systems, distributed facilities for ecological research are of growing importance. While software infrastructure for low-level networking services are well-established, experiments using these facilities will demand real time data-driven workflows for monitoring, model inference, and control of environmental processes. In this paper, we motivate and present a middleware-based approach that enables construction and deployment of workflows that assimilate real-time streaming data and, if necessary, command and control streams. We demonstrate the approach by developing and deploying a workflow for characterizing the round-trip delays incurred by increasing levels of software infrastructure, and using the workflow to assess time delay performance in laboratory, campus, and remote scenarios.


Closed loop Real-time Workflows Ecology Middleware Experiments Design Delay 


  1. 1.
    Fountain, T., et al.: The open source dataturbine initiative: empowering the scientific community with streaming data middleware. Bull. Ecol. Soc. Am. 93(3), 242–252 (2012)CrossRefGoogle Scholar
  2. 2.
    Aberer, K., Hauswirth, M., Salehi, A.: A middleware for fast and flexible sensor network deployment, In: Proceedings of the 32nd International Conference on Very Large Databases, pp. 1199–1202 (2006)Google Scholar
  3. 3.
    Deelman, E., Singh, G., Su, M.H., Blythe, J., Gil, Y., Kesselman, C., Katz, D.S.: Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Sci. Program. 13(3), 219–237 (2005)Google Scholar
  4. 4.
    Deelman, E., et al.: Workflows and e-Science: an overview of workflow system features and capabilities. future Gener. Comput. Syst. 25(5), 528–540 (2009)CrossRefGoogle Scholar
  5. 5.
    Freire, J.: Making computations and publications reproducible with Vistrails. Comput. Sci. Eng. 14(4), 18–25 (2012)Google Scholar
  6. 6.
    Barseghian, D., Altintas, I., Jones, M.B., Crawl, D., Potter, N., Gallagher, J., Hosseini, P.R.: Workflows and extensions to the Kepler scientific workflow system to support environmental sensor data access and analysis. Ecol. Inf. 5(1), 42–50 (2010)CrossRefGoogle Scholar
  7. 7.
    Wolstencroft, K., Haines, R., Fellows, D., Williams, A., Withers, D., Owen, S., Goble, C.: The Taverna workflow suite: designing and executing workflows of Web Services on the desktop, web or in the cloud. Nucleic Acids Res. 41(W1), W557–W561 (2013)CrossRefGoogle Scholar
  8. 8.
    Zhang, J., Kuc, D., Lu, S.: Confucius: a tool supporting collaborative scientific workflow composition. IEEE Trans. Serv. Comput. 7(1), 2–17 (2014)Google Scholar
  9. 9.
    Ekanayake, J., Gunarathne, T., Fox, G., Balkir, A.S., Poulain, C., Araujo, N., Barga, R.: Dryadlinq for scientific analyses. In: Fifth IEEE International Conference on e-Science (e-Science 2009), pp. 329–336, December 2009Google Scholar
  10. 10.
    Andriescu, E., Speicys Cardoso, R., Issarny, V.: AmbiStream: a middleware for multimedia streaming on heterogeneous mobile devices. In: Kon, F., Kermarrec, A.-M. (eds.) Middleware 2011. LNCS, vol. 7049, pp. 249–268. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  11. 11.
    Eker, J., et al.: Taming heterogeneity-the Ptolemy approach. Proc. IEEE 91(1), 127–144 (2003)CrossRefGoogle Scholar
  12. 12.
    Lee, E., Messerschmitt, D.G.: Static scheduling of synchronous data flow programs for digital signal processing. IEEE Trans. Comput. C-36(1), 24–35 (1987)Google Scholar
  13. 13.
    Kavi, K.M., Buckles, B.P., Bhat, Narayan, U.: A formal definition of data flow graph models. IEEE Trans. Comput. C-35(11), 940–948 (1986)Google Scholar
  14. 14.
    Boose, E., Ellison, A., Osterweil, L., Clarke, L., Podorozhny, R., Hadley, J., Wise, A., Foster, D.: Ensuring reliable datasets for environmental models and forecasts. Ecol. Inform. 2(3, Sp. Iss. SI), 237–247 (1986)CrossRefGoogle Scholar
  15. 15.
    Java Platform, Standard Edition 7 API Specification.
  16. 16.
    Shaeffer, J., Knapp, J.D., Miller, M., Flikkema, P.G.: A middleware-based approach to the design of interconnected sensor/actuator networks. In: 2014 IEEE International Workshop on Real-Time Cyber-Physical Systems, June 2014Google Scholar
  17. 17.
    Ghosh, S., Bell, D.M., Clark, J.S., Gelfand, A.E., Flikkema, P.: Process modeling for soil moisture using sensor network data. Stat. Methodol. (Special Issue on Modern Statistical Methods in Ecology) 17, 99–112 (2014)MathSciNetGoogle Scholar
  18. 18.
    Kampstra, P.: Beanplot: a boxplot alternative for visual comparison of distributions. J. Stat. Softw. 28(Code Snippet 1), 1–9 (2008)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • JD Knapp
    • 1
  • Matias Elo
    • 2
  • James Shaeffer
    • 1
  • Paul G. Flikkema
    • 1
  1. 1.Northern Arizona UniversityFlagstaffUSA
  2. 2.Nokia NetworksEspooFinland

Personalised recommendations