Abstract
Large amounts of data (“big data”) are readily available and collected daily by global networks worldwide. However, much of the real-time utility of this data is not realized, as data analysis tools for very large datasets, particularly time series data are cumbersome. A methodology for data cleaning and preparation needed to support big data analysis is presented, along with a comparative examination of three widely available data mining tools. This methodology and offered tools are used for analysis of a large-scale time series dataset of environmental data. The case study of environmental data analysis is presented as visualization, providing future direction for data mining on massive data sets gathered from global networks, and an illustration of the use of big data technology for predictive data modeling and assessment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Holtz, S., Valle, G., Howard, J., Morreale, P.: Visualization and Pattern Identification in Large Scale Time Series Data. In: IEEE Symposium on Large Scale Data Analysis and Visualization (LDAV 2011), Providence, RI, pp. 17–18 (2011)
Morreale, P., Qi, F., Croft, P.: A Green Wireless Sensor Network for Environmental Monitoring and Risk Identification. International Journal on Sensor Networks 10(1/2), 73–82 (2011)
Shyu, C., Klaric, M., Scott, G., Mahamaneerat, W.: Knowledge Discovery by Mining Association Rules and Temporal-Spatial Information from Large-Scale Geospatial Image Databases. In: Proceedings of the IEEE International Symposium on Geoscience and Remote Sensing (IGARSS 2006), pp. 17–20 (2006)
Zhu, C., Zhang, X., Sun, J., Huang, B.: Algorithm for Mining Sequential Pattern in Time Series Data. In: Proceedings of the IEEE 2009 WRI International Conference on Communications and Mobile Computing, pp. 258–262 (2009)
NOAA Integrated Surface Database (GSOD), http://www.ncdc.noaa.gov/oa/climate/isd/index.php (retrieved June 12, 2013)
NOAA Global Historical Climatology Network (GHCN) Database, http://www.ncdc.noaa.gov/oa/climate/ghcn-daily/ (retrieved June 12, 2013)
Han, J., Rodriguez, J.C., Beheshti, M.: Diabetes Data Analysis and Prediction Model Discovery Using RapidMiner. In: IEEE Proceedings of the 2nd International Conference on Future Generation Communication and Networking (FGCN 2008), pp. 96–99 (2008)
Weka’s website, http://www.cs.waikato.ac.nz/ml/weka/ (retrieved June 12, 2013)
RapidMiner’s website, http://rapid-i.com/content/view/181/190/ (retrieved June 12, 2013)
Orange’s website, http://orange.biolab.si/ (retrieved June 12, 2013)
Shafait, F., Reif, M., Kofler, C., Breuel, T.R.: Pattern Recognition Engineering. In: RapidMiner Community Meeting and Conference (RMiner 2010), Dortmund, Germany (2010)
DPlot’s website, http://www.dplot.com/ (retrieved June 12, 2013)
Thuraisingham, B., Khan, L., Clifton, C., Maurer, J., Ceruti, M.: Dependable Real-time Data Mining. In: Proceedings of the 8th IEEE International Symposium on Object-Oriented Real-Time Distributed Computing (ISORC 2005), pp. 158–165 (2005)
Martinez, K., Hart, J.K., Ong, R.: Environmental Sensor Networks. IEEE Computer, 50–56 (August 2004)
Lewis, F.L.: Wireless Sensor Networks. In: Cooke, D.J., Das, S.K. (eds.) Smart Environments: Technologies, Protocols, and Applications. John Wiley, New York (2004)
Zimmerman, A.T., Lynch, J.P.: Data Driven Model Updating using Wireless Sensor Networks. In: Proceedings of the 3rd Annual ANCRiSST Workshop (2006)
Chang, N., Guo, D.: Urban Flash Flood Monitoring, Mapping, and Forecasting via a Tailored Sensor Network System. In: Proceedings of the 2006 IEEE International Conference on Networking, Sensing and Control, pp. 757–761 (2006)
Cordova-Lopez, L.E., Mason, A., Cullen, J.D., Shaw, A., Al-Shamma’a, A.I.: Online vehicle and atmospheric pollution monitoring using GIA and wireless sensor networks. Journal of Physics: Conference Series 76(1) (2007)
Gahegan, M., Wachowicz, M., Harrower, M., Rhyne, T.-M.: The Integration of geographic visualization with knowledge discovery in databases and geocomputation. Cartography and Geographic Information Science 28(1), 29–44 (2001)
Arici, T., Akgu, T., Altunbasak, Y.: A Prediction Error-Based Hypothesis Testing Method for Sensor Data Acquisition. ACM Transactions on Sensor Networks 2(4), 529–556 (2006)
Monmonier, M.: Geographic brushing: Enhancing exploratory analysis of the scatter plot matrix. Geographical Analysis 21(1), 81–84 (1989)
MacEachren, A.M., Polsky, C., Haug, D., Brown, D., Boscoe, F., Beedasy, J., Pickle, L., Marrara, M.: Visualizing spatial relationships among health, environmental, and demographic statistics: interface design issues. In: 18th International Cartographic Conference Stockholm, pp. 880–887 (1997)
Monmonier, M.: Strategies for the visualization of geographic time-series data. Cartographica 27(1), 30–45 (1990)
Harrower, M.: Visual Benchmarks: Representing Geographic Change with Map Animation. Ph.D. dissertation, Pennsylvania State University (2002)
Mueen, A., Keogh, E.: Online Discovery and Maintenance of Time Series Motifs. In: Proceedings of 16th ACM Conference on Knowledge Discovery and Data Mining (KDD 2010), pp. 1089–1098 (2010)
Morreale, P., Qi, F., Croft, P., Suleski, R., Sinnicke, B., Kendall, F.: Real-Time Environmental Monitoring and Notification for Public Safety. IEEE Multimedia 17(2), 4–11 (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Morreale, P., Goncalves, A., Silva, C. (2015). Analysis and Visualization of Large-Scale Time Series Network Data. In: Xhafa, F., Barolli, L., Barolli, A., Papajorgji, P. (eds) Modeling and Processing for Next-Generation Big-Data Technologies. Modeling and Optimization in Science and Technologies, vol 4. Springer, Cham. https://doi.org/10.1007/978-3-319-09177-8_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-09177-8_9
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-09176-1
Online ISBN: 978-3-319-09177-8
eBook Packages: EngineeringEngineering (R0)