Skip to main content
SpringerLink
Log in

Efficient Aggregation Methods for Probabilistic Data Streams

  • Conference paper
  • First Online:
Business Modeling and Software Design (BMSD 2018)

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 319))

Included in the following conference series:

Abstract

In this paper, we consider aggregation algorithms for SUM operator for uncertain stream processing. Deterministic algorithms can not be used here because of uncertain data and high rates of data change, time and memory constraints. We compare the most promising available methods. Instead of full distribution functions of query result, we use a set of six parameters based on key moments and quantiles to describe the distributions. It enables us to perform fast recomputations of the aggregation with O(1) complexity. Experimental results demonstrate good performance of uncertain aggregation in comparison to deterministic case. We also found that usage of central limit theorem may be restricted to problems where data satisfy certain conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ge, T., Zdonik, S.: Handling uncertain data in array database systems. In: IEEE 24th International Conference on Data Engineering (ICDE 2008), pp. 1140–1149. IEEE, Cancun (2008). https://doi.org/10.1109/ICDE.2008.4497523

  2. Wang, Y., Li, X., Li, X., Wang, Y.: A survey of queries over uncertain data. Knowl. Inf. Syst. 37(3), 485–530 (2013). https://doi.org/10.1007/s10115-013-0638-6

    Article  Google Scholar 

  3. Tran, T., McGregor, A., Diao, Y., Peng, L., Liu, A.: Conditioning and aggregating uncertain data streams: going beyond expectations. In: The 36th International Conference on Very Large Data Bases (VLDB 2010), vol. 3, no. 1, pp. 1302–1313. VLBD, Singapore (2010)

    Article  Google Scholar 

  4. Tran, T., Peng, P., Diao, Y., McGregor, A., Liu, A.: CLARO: modeling and processing uncertain data streams. VLDB J. 21(5), 651–676 (2012). https://doi.org/10.1007/s00778-011-0261-7

    Article  Google Scholar 

  5. Tran, T., Peng, L., Li, B., Diao, Y., Liu, A.: PODS: a new model and processing algorithms for uncertain data streams. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, pp. 159–170. ACM, New York (2010). https://doi.org/10.1145/1807167.1807187

  6. Murthy, R., Ikeda, R., Widom, J.: Making aggregation work in uncertain and probabilistic databases. IEEE Trans. Knowl. Data Eng. 23(8), 1261–1273 (2011). https://doi.org/10.1109/TKDE.2010.166

    Article  Google Scholar 

  7. Aggarwal, C. (ed.): Managing and Mining Uncertain Data. Springer, Heidelberg (2009). https://doi.org/10.1007/978-0-387-09690-2

    Book  MATH  Google Scholar 

  8. Dallachiesa, M., Jacques-Silva, G., Gedik, B., Wu, K.-L., Palpanas, T.: Sliding windows over uncertain data streams. Knowl. Inf. Syst. 45(1), 159–190 (2015). https://doi.org/10.1007/s10115-014-0804-5

    Article  Google Scholar 

  9. Dezfuli, M.G., Haghjoo, M.S.: Xtream: a system for continuous querying over uncertain data streams. In: Hüllermeier, E., Link, S., Fober, T., Seeger, B. (eds.) SUM 2012. LNCS (LNAI), vol. 7520, pp. 1–15. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33362-0_1

    Chapter  Google Scholar 

  10. Cheng, R., Kalashnikov, D., Prabhakar, S.: Evaluating probabilistic queries over imprecise data. In: Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data, pp. 551–562. ACM, New York (2003). https://doi.org/10.1145/872757.872823

  11. Jayram, T.S., Kale, S., Vee, E.: Efficient aggregation algorithms for probabilistic data. In: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pp. 346–355. Society for Industrial and Applied Mathematics Philadelphia (2007)

    Google Scholar 

  12. Jin, C., Yi, K., Chen, L., Yu, J., Lin, X.: Sliding-window top-k queries on uncertain streams. VLDB J. 19(3), 411–435 (2010). https://doi.org/10.1007/s00778-009-0171-0

    Article  Google Scholar 

  13. Hua, M., Pei, J., Zhang, W., Lin, X.: Ranking queries on uncertain data: a probabilistic threshold approach. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 673–686. ACM, New York (2008). https://doi.org/10.1145/1376616.1376685

  14. Killmann, F., von Collani, E.: A note on the convolution of the uniform and related distributions and their use in quality control. Econ. Qual. Control 16(1), 17–41 (2001)

    Article  MathSciNet  Google Scholar 

  15. Bradley, D., Gupta, R.: On the distribution of the sum of n non-identically distributed uniform random variables. Ann. Inst. Stat. Math. 54(3), 689–700 (2002). https://doi.org/10.1023/A:102248371

    Article  MathSciNet  MATH  Google Scholar 

  16. Mitzenmacher, M., Upfal, E.: Probability and Computing - Randomized Algorithms and Probabilistic Analysis. Cambridge University Press, Cambridge (2005)

    Book  Google Scholar 

  17. Spanos, A.: Probability Theory and Statistical Inference. Econometric Modeling with Observational Data. Cambridge University Press, Cambridge (1999)

    Book  Google Scholar 

  18. Climate Change: Earth Surface Temperature Data. https://www.kaggle.com/berkeleyearth/climate-change-earth-surface-temperature-data

  19. Pfabrik, C++ framework. https://github.com/dbis-ilm/pipefabric

  20. Saleh, O., Sattler, K.-U.: The PipeFlow approach. In: 9th ACM International Conference on Distributed Event-Based Systems (DEBS 2015), pp. 326–327. ACM, New York (2015). https://doi.org/10.1145/2675743.2776763

Download references

Author information

Authors and Affiliations

  1. Department of Business Informatics – Information Engineering, Johannes Kepler University, Linz, Austria

    Maksim Goman

Authors
  1. Maksim Goman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maksim Goman .

Editor information

Editors and Affiliations

  1. Bulgarian Academy of Sciences, Institute of Mathematics and Informatics (IMI)/Interdisciplinary Institute for Collaboration and Research on Enterprise Systems and Technology (IICREST), Sofia, Bulgaria

    Boris Shishkov

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Goman, M. (2018). Efficient Aggregation Methods for Probabilistic Data Streams. In: Shishkov, B. (eds) Business Modeling and Software Design. BMSD 2018. Lecture Notes in Business Information Processing, vol 319. Springer, Cham. https://doi.org/10.1007/978-3-319-94214-8_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-94214-8_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-94213-1

  • Online ISBN: 978-3-319-94214-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation