Skip to main content
SpringerLink
Log in

Data integration with uncertainty

  • Regular Paper
  • Published:
The VLDB Journal Aims and scope Submit manuscript

Abstract

This paper reports our first set of results on managing uncertainty in data integration. We posit that data-integration systems need to handle uncertainty at three levels and do so in a principled fashion. First, the semantic mappings between the data sources and the mediated schema may be approximate because there may be too many of them to be created and maintained or because in some domains (e.g., bioinformatics) it is not clear what the mappings should be. Second, the data from the sources may be extracted using information extraction techniques and so may yield erroneous data. Third, queries to the system may be posed with keywords rather than in a structured form. As a first step to building such a system, we introduce the concept of probabilistic schema mappings and analyze their formal foundations. We show that there are two possible semantics for such mappings: by-table semantics assumes that there exists a correct mapping but we do not know what it is; by-tuple semantics assumes that the correct mapping may depend on the particular tuple in the source data. We present the query complexity and algorithms for answering queries in the presence of probabilistic schema mappings, and we describe an algorithm for efficiently computing the top-k answers to queries in such a setting. Finally, we consider using probabilistic mappings in the scenario of data exchange.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abiteboul, S., Duschka, O.: Complexity of answering queries using materialized views. In: PODS (1998)

  2. Agrawal, S., Chaudhuri, S., Das, G.: DBXplorer: A system for keyword-based search over relational databases. In: ICDE (2002)

  3. Antova, L., Koch, C., Olteanu, D.: World-set decompositions: Expressiveness and efficient algorithms. In: ICDT (2007)

  4. Benjelloun, O., Sarma, A.D., Halevy, A.Y., Widom, J.: ULDBs: Databases with uncertainty and lineage. In: VLDB (2006)

  5. Bernstein, P.A., Green, T.J., Melnik, S., Nash, A.: Implementing mapping composition. In: Proceedings of VLDB, pp. 55–66 (2006)

  6. Cheng, R., Prabhakar, S., Kalashnikov, D.V.: Querying imprecise data in moving object environments. In: ICDE (2003)

  7. de Rougemont, M., Vieilleribiere, A.: Approximate data exchange. In: ICDT (2007)

  8. Dong, X., Halevy, A.: A platform for personal information management and integration. In: CIDR (2005)

  9. Dong, X.L., Halevy, A.Y., Yu, C.: Data integration with uncertainty. In: Proceedings of VLDB (2007)

  10. Fagin, R.: Inverting schema mappings. In: Proceedings of PODS (2006)

  11. Fagin R., Kolaitis P.G., Popa L.: Data exchange: getting to the core. ACM Trans. Database Syst. 30(1), 174–201 (2005)

    Article  MathSciNet  Google Scholar 

  12. Fagin R., Kolaitis P.G., Popa L., Tan W.C.: Composing schema mappings: second-order dependencies to the rescue. ACM Trans. Database Syst. 30(4), 994–1055 (2005)

    Article  Google Scholar 

  13. Fagin, R., Lotem, A., Naor, M.: Optimal aggregation algorithms for middleware. In: PODS (2001)

  14. Florescu, D., Koller, D., Levy, A.: Using probabilistic information in data integration. In: Proceedings of VLDB (1997)

  15. Gal A.: Why is schema matching tough and what can we do about it. SIGMOD Rec. 35(4), 2–5 (2007)

    Article  Google Scholar 

  16. GoogleBase. http://base.google.com/ (2005)

  17. Halevy, A.Y.: Answering queries using views: a survey. VLDB J. 10(4) (2001)

  18. Halevy, A.Y., Ashish, N., Bitton, D., Carey, M.J., Draper, D., Pollock, J., Rosenthal, A., Sikka, V.: Enterprise information integration: successes, challenges and controversies. In: SIGMOD (2005)

  19. Halevy, A.Y., Franklin, M.J., Maier, D.: Principles of dataspace systems. In: PODS (2006)

  20. Halevy, A.Y., Rajaraman, A., Ordille, J.J.: Data integration: the teenage years. In: VLDB (2006)

  21. Hristidis, V., Papakonstantinou, Y.: DISCOVER: keyword search in relational databases. In: VLDB (2002)

  22. Lenzerini, M.: Data integration: a theoretical perspective. In: Proceedings of PODS (2002)

  23. Levy A.Y.: Special issue on adaptive query processing. IEEE Data Eng. Bull. 23(2), 7–18 (2000)

    Google Scholar 

  24. Li, C., Chang, K.C.-C., LLyas, I.F.: Supporting ad-hoc ranking aggregates. In: SIGMOD (2006)

  25. Madhavan, J., Cohen, S., Dong, X., Halevy, A., Jeffery, S., Ko, D., Yu, C.: Navigating the seas of structured web data. In: CIDR (2007)

  26. Madhavan, J., Halevy, A.: Composing mappings among data sources. In: Proceedings of VLDB (2003)

  27. Magnani, M., Montesi, D.: Uncertainty in data integration: current approaches and open problems. In: VLDB workshop on management of uncertain data, pp. 18–32 (2007)

  28. Nottelmann H., Straccia U.: Information retrieval and machine learning for probabilistic schema matching. Inf. Process. Manage. 43(3), 552–576 (2007)

    Article  Google Scholar 

  29. Pearl J.: Probabilistic Reasoning in Intelligent Systems: Networks of  Plausible Inference. Morgan Kaufmann, San Francisco (1988)

    Google Scholar 

  30. Rahm E., Bernstein P.A.: A survey of approaches to automatic schema matching. VLDB J. 10(4), 334–350 (2001)

    Article  MATH  Google Scholar 

  31. Re C., Suciu D., Dalvi N.N.: Query evaluation on probabilistic databases. IEEE Data Eng. Bull. 29(1), 25–31 (2006)

    Google Scholar 

  32. Sarma, A.D., Dong, X.L., Halevy, A.Y.: Bootstrapping pay- as-you-go data integration systems. In: Proceedings of SIGMOD (2008)

  33. Suciu, D., Dalvi, N.N.: Foundations of probabilistic answers to queries. In: SIGMOD (2005)

Download references

Author information

Authors and Affiliations

  1. AT & T Labs-Research, Florham Park, NJ, 07932, USA

    Xin Luna Dong

  2. Google Inc., Mountain View, CA, 94043, USA

    Alon Halevy

  3. Yahoo! Research, New York, NY, 10018, USA

    Cong Yu

Authors
  1. Xin Luna Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alon Halevy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cong Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xin Luna Dong.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dong, X.L., Halevy, A. & Yu, C. Data integration with uncertainty. The VLDB Journal 18, 469–500 (2009). https://doi.org/10.1007/s00778-008-0119-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00778-008-0119-9

Keywords

Search

Navigation