Observing Linked Data Dynamics

  • Tobias Käfer
  • Ahmed Abdelrahman
  • Jürgen Umbrich
  • Patrick O’Byrne
  • Aidan Hogan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7882)


In this paper, we present the design and first results of the Dynamic Linked Data Observatory: a long-term experiment to monitor the two-hop neighbourhood of a core set of eighty thousand diverse Linked Data documents on a weekly basis. We present the methodology used for sampling the URIs to monitor, retrieving the documents, and further crawling part of the two-hop neighbourhood. Having now run this experiment for six months, we analyse the dynamics of the monitored documents over the data collected thus far. We look at the estimated lifespan of the core documents, how often they go on-line or off-line, how often they change; we further investigate domain-level trends. Next we look at changes within the RDF content of the core documents across the weekly snapshots, examining the elements (i.e., triples, subjects, predicates, objects, classes) that are most frequently added or removed. Thereafter, we look at how the links between dereferenceable documents evolves over time in the two-hop neighbourhood.


Link Data Hybrid Architecture Core Document Blank Node Dynamic Predicate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Brewington, B., Cybenko, G.: Keeping up with the changing web. Computer 33(5), 52–58 (2000)CrossRefGoogle Scholar
  2. 2.
    Cho, J., Garcia-Molina, H.: Estimating frequency of change. ACM Transactions on Internet Technology 3(3), 256–290 (2003)CrossRefGoogle Scholar
  3. 3.
    Coffman Jr., E.G., Liu, Z., Weber, R.R.: Optimal robot scheduling for web search engines. Journal of Scheduling 1, 0–21 (1997)Google Scholar
  4. 4.
    Fetterly, D., Manasse, M., Najork, M., Wiener, J.L.: A large-scale study of the evolution of Web pages. In: WWW, pp. 669–678. ACM (2003)Google Scholar
  5. 5.
    Käfer, T., Umbrich, J., Hogan, A., Polleres, A.: DyLDO: Towards a Dynamic Linked Data Observatory. In: LDOW at WWW. CEUR-WS, vol. 937 (2012)Google Scholar
  6. 6.
    Ke, Y., Deng, L., Ng, W., Lee, D.L.: Web dynamics and their ramifications for the development of Web search engines. Computer Networks 50(10), 1430–1447 (2006)zbMATHCrossRefGoogle Scholar
  7. 7.
    Koehler, W.: An analysis of Web page and web site constancy and permanence. Journal of the American Society for Information Science 50(2), 162–180 (1999)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Lim, L., Wang, M., Padmanabhan, S., Vitter, J.S., Agarwal, R.: Characterizing Web document change. In: Wang, X.S., Yu, G., Lu, H. (eds.) WAIM 2001. LNCS, vol. 2118, pp. 133–144. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  9. 9.
    Ntoulas, A., Cho, J., Olston, C.: What’s new on the Web? The evolution of the Web from a search engine perspective. In: WWW, pp. 1–12. ACM (2004)Google Scholar
  10. 10.
    Popitsch, N., Haslhofer, B.: DSNotify – a solution for event detection and link maintenance in dynamic datasets. J. Web Sem. 9(3), 266–283 (2011)CrossRefGoogle Scholar
  11. 11.
    Stuckenschmidt, H., Vdovjak, R., Houben, G.-J., Broekstra, J.: Index structures and algorithms for querying distributed RDF repositories. In: WWW, pp. 631–639. ACM (2004)Google Scholar
  12. 12.
    Umbrich, J., Hausenblas, M., Hogan, A., Polleres, A., Decker, S.: Towards Dataset Dynamics: Change Frequency of Linked Open Data Sources. In: Proc. of LDOW at WWW. CEUR-WS, vol. 628 (2010)Google Scholar
  13. 13.
    Umbrich, J., Karnstedt, M., Hogan, A., Parreira, J.X.: Hybrid SPARQL Queries: Fresh vs. Fast Results. In: Cudré-Mauroux, P., et al. (eds.) ISWC 2012, Part I. LNCS, vol. 7649, pp. 608–624. Springer, Heidelberg (2012)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Tobias Käfer
    • 1
  • Ahmed Abdelrahman
    • 2
  • Jürgen Umbrich
    • 2
  • Patrick O’Byrne
    • 2
  • Aidan Hogan
    • 2
  1. 1.Institute AIFBKarlsruhe Institute of TechnologyGermany
  2. 2.Digital Enterprise Research InstituteNational University of Ireland GalwayIreland

Personalised recommendations