OPEN—Enabling Non-expert Users to Extract, Integrate, and Analyze Open Data
Government initiatives for more transparency and participation have lead to an increasing amount of structured data on the web in recent years. Many of these datasets have great potential. For example, a situational analysis and meaningful visualization of the data can assist in pointing out social or economic issues and raising people’s awareness. Unfortunately, the ad-hoc analysis of this so-called Open Data can prove very complex and time-consuming, partly due to a lack of efficient system support.
On the one hand, search functionality is required to identify relevant datasets. Common document retrieval techniques used in web search, however, are not optimized for Open Data and do not address the semantic ambiguity inherent in it. On the other hand, semantic integration is necessary to perform analysis tasks across multiple datasets. To do so in an ad-hoc fashion, however, requires more flexibility and easier integration than most data integration systems provide. It is apparent that an optimal management system for Open Data must combine aspects from both classic approaches.
In this article, we propose OPEN, a novel concept for the management and situational analysis of Open Data within a single system. In our approach, we extend a classic database management system, adding support for the identification and dynamic integration of public datasets. As most web users lack the experience and training required to formulate structured queries in a DBMS, we add support for non-expert users to our system, for example though keyword queries. Furthermore, we address the challenge of indexing Open Data.
KeywordsOpen Data Global Schema Keyword Query Structure Query Public Dataset
- 1.Bergamaschi S, Domnori E, Guerra F, Trillo Lado R, Velegrakis Y (2011) Keyword search over relational databases: a metadata approach. In: Proceedings of the 2011 international conference on management of data (SIGMOD ’11), pp 565–576 Google Scholar
- 2.Bizer C, Heath T, Berners-Lee T (2009) Linked data—the story so far. Int J Semantic Web Inf Syst 5(3) Google Scholar
- 3.Blunschi L, Dittrich PJ, Girard OR, Kirakos S, Marcos K, Salles AV (2007) A dataspace odyssey: the iMeMex personal dataspace management system. In: CIDR, pp 114–119 Google Scholar
- 4.Calvanese D, De Giacomo G, Lembo D, Lenzerini M, Poggi A, Rodriguez-Muro M, Rosati R, Ruzzi M, Savo DF (2011) The MASTRO system for ontology-based data access. J Web Semant 2:43–53 Google Scholar
- 5.Chiticariu L, Hernández MA, Kolaitis PG, Popa L (2007) Semi-automatic schema integration in Clio. In: Proceedings of the 33rd international conference on very large data bases (VLDB ’07), pp 1326–1329 Google Scholar
- 6.Cunningham H, Maynard D, Bontcheva K, Tablan V (2002) Gate: a framework and graphical development environment for robust NLP tools and applications. In: Proceedings of the 40th anniversary meeting of the association for computational linguistics (ACL’02) Google Scholar
- 7.Demartini G, Difallah DE, Cudré-Mauroux P (2012) Zencrowd: leveraging probabilistic reasoning and crowdsourcing techniques for large-scale entity linking. In: Proceedings of the 21st international conference on world wide web (WWW ’12). ACM, New York, pp 469–478. http://doi.acm.org/10.1145/2187836.2187900. doi: 10.1145/2187836.2187900 CrossRefGoogle Scholar
- 8.Finin T, Murnane W, Karandikar A, Keller N, Martineau J, Dredze M (2010) Annotating named entities in twitter data with crowdsourcing. In: Proceedings of the NAACL HLT 2010 workshop on creating speech and language data with amazon’s mechanical turk (CSLDAMT ’10). Association for Computational Linguistics, Stroudsburg, pp 80–88. http://dl.acm.org/citation.cfm?id=1866696.1866709 Google Scholar
- 10.Franklin MJ, Kossmann D, Kraska T, Ramesh S, Xin R (2011) Crowddb: answering queries with crowdsourcing. In: Proceedings of the 2011 international conference on management of data (SIGMOD ’11). ACM, New York, pp 61–72. http://doi.acm.org/10.1145/1989323.1989331. doi: 10.1145/1989323.1989331 Google Scholar
- 12.Lawson N, Eustice K, Perkowitz M, Yetisgen-Yildiz M (2010) Annotating large email datasets for named entity recognition with mechanical turk. In: Proceedings of the NAACL HLT 2010 workshop on creating speech and language data with amazon’s mechanical turk (CSLDAMT ’10). Association for Computational Linguistics, Stroudsburg, pp 71–79. http://dl.acm.org/citation.cfm?id=1866696.1866708 Google Scholar
- 13.Madhavan J, Cohen S, Dong XL, Halevy AY, Jeffery SR, Ko D, Yu C (2007) Web-scale data integration: you can afford to pay as you go. In: CIDR, pp 342–350 Google Scholar
- 14.Oleson D, Sorokin A, Laughlin GP, Hester V, Le J, Biewald L (2011) Programmatic gold: targeted and scalable quality assurance in crowdsourcing. In: Human computation Google Scholar
- 17.Vaz Salles MA, Dittrich JP, Karakashian SK, Girard OR, Blunschi L (2007) iTrails: pay-as-you-go information integration in dataspaces. In: Proceedings of the 33rd international conference on very large data bases (VLDB ’07), pp 663–674 Google Scholar