The VLDB Journal

, Volume 18, Issue 2, pp 469–500

Data integration with uncertainty

Regular Paper

Abstract

This paper reports our first set of results on managing uncertainty in data integration. We posit that data-integration systems need to handle uncertainty at three levels and do so in a principled fashion. First, the semantic mappings between the data sources and the mediated schema may be approximate because there may be too many of them to be created and maintained or because in some domains (e.g., bioinformatics) it is not clear what the mappings should be. Second, the data from the sources may be extracted using information extraction techniques and so may yield erroneous data. Third, queries to the system may be posed with keywords rather than in a structured form. As a first step to building such a system, we introduce the concept of probabilistic schema mappings and analyze their formal foundations. We show that there are two possible semantics for such mappings: by-table semantics assumes that there exists a correct mapping but we do not know what it is; by-tuple semantics assumes that the correct mapping may depend on the particular tuple in the source data. We present the query complexity and algorithms for answering queries in the presence of probabilistic schema mappings, and we describe an algorithm for efficiently computing the top-k answers to queries in such a setting. Finally, we consider using probabilistic mappings in the scenario of data exchange.

Keywords

Data integration Probabilistic schema mapping Data exchange 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Abiteboul, S., Duschka, O.: Complexity of answering queries using materialized views. In: PODS (1998)Google Scholar
  2. 2.
    Agrawal, S., Chaudhuri, S., Das, G.: DBXplorer: A system for keyword-based search over relational databases. In: ICDE (2002)Google Scholar
  3. 3.
    Antova, L., Koch, C., Olteanu, D.: World-set decompositions: Expressiveness and efficient algorithms. In: ICDT (2007)Google Scholar
  4. 4.
    Benjelloun, O., Sarma, A.D., Halevy, A.Y., Widom, J.: ULDBs: Databases with uncertainty and lineage. In: VLDB (2006)Google Scholar
  5. 5.
    Bernstein, P.A., Green, T.J., Melnik, S., Nash, A.: Implementing mapping composition. In: Proceedings of VLDB, pp. 55–66 (2006)Google Scholar
  6. 6.
    Cheng, R., Prabhakar, S., Kalashnikov, D.V.: Querying imprecise data in moving object environments. In: ICDE (2003)Google Scholar
  7. 7.
    de Rougemont, M., Vieilleribiere, A.: Approximate data exchange. In: ICDT (2007)Google Scholar
  8. 8.
    Dong, X., Halevy, A.: A platform for personal information management and integration. In: CIDR (2005)Google Scholar
  9. 9.
    Dong, X.L., Halevy, A.Y., Yu, C.: Data integration with uncertainty. In: Proceedings of VLDB (2007)Google Scholar
  10. 10.
    Fagin, R.: Inverting schema mappings. In: Proceedings of PODS (2006)Google Scholar
  11. 11.
    Fagin R., Kolaitis P.G., Popa L.: Data exchange: getting to the core. ACM Trans. Database Syst. 30(1), 174–201 (2005)CrossRefMathSciNetGoogle Scholar
  12. 12.
    Fagin R., Kolaitis P.G., Popa L., Tan W.C.: Composing schema mappings: second-order dependencies to the rescue. ACM Trans. Database Syst. 30(4), 994–1055 (2005)CrossRefGoogle Scholar
  13. 13.
    Fagin, R., Lotem, A., Naor, M.: Optimal aggregation algorithms for middleware. In: PODS (2001)Google Scholar
  14. 14.
    Florescu, D., Koller, D., Levy, A.: Using probabilistic information in data integration. In: Proceedings of VLDB (1997)Google Scholar
  15. 15.
    Gal A.: Why is schema matching tough and what can we do about it. SIGMOD Rec. 35(4), 2–5 (2007)CrossRefGoogle Scholar
  16. 16.
    GoogleBase. http://base.google.com/ (2005)
  17. 17.
    Halevy, A.Y.: Answering queries using views: a survey. VLDB J. 10(4) (2001)Google Scholar
  18. 18.
    Halevy, A.Y., Ashish, N., Bitton, D., Carey, M.J., Draper, D., Pollock, J., Rosenthal, A., Sikka, V.: Enterprise information integration: successes, challenges and controversies. In: SIGMOD (2005)Google Scholar
  19. 19.
    Halevy, A.Y., Franklin, M.J., Maier, D.: Principles of dataspace systems. In: PODS (2006)Google Scholar
  20. 20.
    Halevy, A.Y., Rajaraman, A., Ordille, J.J.: Data integration: the teenage years. In: VLDB (2006)Google Scholar
  21. 21.
    Hristidis, V., Papakonstantinou, Y.: DISCOVER: keyword search in relational databases. In: VLDB (2002)Google Scholar
  22. 22.
    Lenzerini, M.: Data integration: a theoretical perspective. In: Proceedings of PODS (2002)Google Scholar
  23. 23.
    Levy A.Y.: Special issue on adaptive query processing. IEEE Data Eng. Bull. 23(2), 7–18 (2000)Google Scholar
  24. 24.
    Li, C., Chang, K.C.-C., LLyas, I.F.: Supporting ad-hoc ranking aggregates. In: SIGMOD (2006)Google Scholar
  25. 25.
    Madhavan, J., Cohen, S., Dong, X., Halevy, A., Jeffery, S., Ko, D., Yu, C.: Navigating the seas of structured web data. In: CIDR (2007)Google Scholar
  26. 26.
    Madhavan, J., Halevy, A.: Composing mappings among data sources. In: Proceedings of VLDB (2003)Google Scholar
  27. 27.
    Magnani, M., Montesi, D.: Uncertainty in data integration: current approaches and open problems. In: VLDB workshop on management of uncertain data, pp. 18–32 (2007)Google Scholar
  28. 28.
    Nottelmann H., Straccia U.: Information retrieval and machine learning for probabilistic schema matching. Inf. Process. Manage. 43(3), 552–576 (2007)CrossRefGoogle Scholar
  29. 29.
    Pearl J.: Probabilistic Reasoning in Intelligent Systems: Networks of  Plausible Inference. Morgan Kaufmann, San Francisco (1988)Google Scholar
  30. 30.
    Rahm E., Bernstein P.A.: A survey of approaches to automatic schema matching. VLDB J. 10(4), 334–350 (2001)MATHCrossRefGoogle Scholar
  31. 31.
    Re C., Suciu D., Dalvi N.N.: Query evaluation on probabilistic databases. IEEE Data Eng. Bull. 29(1), 25–31 (2006)Google Scholar
  32. 32.
    Sarma, A.D., Dong, X.L., Halevy, A.Y.: Bootstrapping pay- as-you-go data integration systems. In: Proceedings of SIGMOD (2008)Google Scholar
  33. 33.
    Suciu, D., Dalvi, N.N.: Foundations of probabilistic answers to queries. In: SIGMOD (2005)Google Scholar

Copyright information

© Springer-Verlag 2008

Authors and Affiliations

  1. 1.AT & T Labs-ResearchFlorham ParkUSA
  2. 2.Google Inc.Mountain ViewUSA
  3. 3.Yahoo! ResearchNew YorkUSA

Personalised recommendations