Abstract
Spurred by needs related to research on the effects of climate change on ecological systems, distributed facilities for ecological research are of growing importance. While software infrastructure for low-level networking services are well-established, experiments using these facilities will demand real time data-driven workflows for monitoring, model inference, and control of environmental processes. In this paper, we motivate and present a middleware-based approach that enables construction and deployment of workflows that assimilate real-time streaming data and, if necessary, command and control streams. We demonstrate the approach by developing and deploying a workflow for characterizing the round-trip delays incurred by increasing levels of software infrastructure, and using the workflow to assess time delay performance in laboratory, campus, and remote scenarios.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The authors thank J. Eberle, J.-P. Calbimonte, and A. Marjovi for helpful discussions around this concept and for this label.
References
Fountain, T., et al.: The open source dataturbine initiative: empowering the scientific community with streaming data middleware. Bull. Ecol. Soc. Am. 93(3), 242–252 (2012)
Aberer, K., Hauswirth, M., Salehi, A.: A middleware for fast and flexible sensor network deployment, In: Proceedings of the 32nd International Conference on Very Large Databases, pp. 1199–1202 (2006)
Deelman, E., Singh, G., Su, M.H., Blythe, J., Gil, Y., Kesselman, C., Katz, D.S.: Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Sci. Program. 13(3), 219–237 (2005)
Deelman, E., et al.: Workflows and e-Science: an overview of workflow system features and capabilities. future Gener. Comput. Syst. 25(5), 528–540 (2009)
Freire, J.: Making computations and publications reproducible with Vistrails. Comput. Sci. Eng. 14(4), 18–25 (2012)
Barseghian, D., Altintas, I., Jones, M.B., Crawl, D., Potter, N., Gallagher, J., Hosseini, P.R.: Workflows and extensions to the Kepler scientific workflow system to support environmental sensor data access and analysis. Ecol. Inf. 5(1), 42–50 (2010)
Wolstencroft, K., Haines, R., Fellows, D., Williams, A., Withers, D., Owen, S., Goble, C.: The Taverna workflow suite: designing and executing workflows of Web Services on the desktop, web or in the cloud. Nucleic Acids Res. 41(W1), W557–W561 (2013)
Zhang, J., Kuc, D., Lu, S.: Confucius: a tool supporting collaborative scientific workflow composition. IEEE Trans. Serv. Comput. 7(1), 2–17 (2014)
Ekanayake, J., Gunarathne, T., Fox, G., Balkir, A.S., Poulain, C., Araujo, N., Barga, R.: Dryadlinq for scientific analyses. In: Fifth IEEE International Conference on e-Science (e-Science 2009), pp. 329–336, December 2009
Andriescu, E., Speicys Cardoso, R., Issarny, V.: AmbiStream: a middleware for multimedia streaming on heterogeneous mobile devices. In: Kon, F., Kermarrec, A.-M. (eds.) Middleware 2011. LNCS, vol. 7049, pp. 249–268. Springer, Heidelberg (2011)
Eker, J., et al.: Taming heterogeneity-the Ptolemy approach. Proc. IEEE 91(1), 127–144 (2003)
Lee, E., Messerschmitt, D.G.: Static scheduling of synchronous data flow programs for digital signal processing. IEEE Trans. Comput. C-36(1), 24–35 (1987)
Kavi, K.M., Buckles, B.P., Bhat, Narayan, U.: A formal definition of data flow graph models. IEEE Trans. Comput. C-35(11), 940–948 (1986)
Boose, E., Ellison, A., Osterweil, L., Clarke, L., Podorozhny, R., Hadley, J., Wise, A., Foster, D.: Ensuring reliable datasets for environmental models and forecasts. Ecol. Inform. 2(3, Sp. Iss. SI), 237–247 (1986)
Java Platform, Standard Edition 7 API Specification. http://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime()
Shaeffer, J., Knapp, J.D., Miller, M., Flikkema, P.G.: A middleware-based approach to the design of interconnected sensor/actuator networks. In: 2014 IEEE International Workshop on Real-Time Cyber-Physical Systems, June 2014
Ghosh, S., Bell, D.M., Clark, J.S., Gelfand, A.E., Flikkema, P.: Process modeling for soil moisture using sensor network data. Stat. Methodol. (Special Issue on Modern Statistical Methods in Ecology) 17, 99–112 (2014)
Kampstra, P.: Beanplot: a boxplot alternative for visual comparison of distributions. J. Stat. Softw. 28(Code Snippet 1), 1–9 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Knapp, J., Elo, M., Shaeffer, J., Flikkema, P.G. (2015). Towards Intelligent Closed-Loop Workflows for Ecological Research. In: Ravela, S., Sandu, A. (eds) Dynamic Data-Driven Environmental Systems Science. DyDESS 2014. Lecture Notes in Computer Science(), vol 8964. Springer, Cham. https://doi.org/10.1007/978-3-319-25138-7_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-25138-7_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25137-0
Online ISBN: 978-3-319-25138-7
eBook Packages: Computer ScienceComputer Science (R0)