Abstract
The World Wide Web is a new advertising medium that corporations use to increase their exposure to consumers. Very large websites whose content is derived from a source database need to maintain a freshness that reflects changes that are made to the base data. This issue is particularly significant for websites that present fast-changing information such as stock-exchange information and product information. In this article, we formally define and study the freshness of a website that is refreshed by a scheduled set of queries that fetch fresh data from the databases. We propose several online-scheduling algorithms and compare the performance of the algorithms on the freshness metric. We show that maximizing the freshness of a website is a NP-hard problem and that the scheduling algorithm MiEF performs better than the other proposed algorithms. Our conclusion is verified by empirical results.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
J. Cho and H. Garcia-Molina, “Synchronizing a database to improve freshness,” in Proceedings of 2000 ACM SIGMOD International Conference on Management of Data (SIGMOD 2000), Dallas, TX, May 2000.
D. Florescu, A. Levy, and A. Mendelzon, “Database techniques for the World-Wide Web: A survey,” SIG-MOD Record (ACM Special Interest Group on Management of Data) 27(3), 1998, 59–74.
R. Garey and S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness, Freeman, New York, 1979.
J. V. Harrison and S. W. Dietrich, “Maintenance of materialized views in a deductive database: An update propagation approach,” in Proceedings of the 1992 JICLSP Workshop on Deductive Databases, Washington, DC, 1992, pp. 56–65.
K. Hong and J. Leung, “On-line scheduling of real-time tasks,” IEEE Transactions on Computer 41, 1992, 1326–1331.
R. Howell and K. Venkatrao, “On non-preemptive scheduling of recurring tasks using inserted idle times,” Information and Computation 117, 1995, 50–62.
http://www.akamai.com
N. Huyn, “Multiple-view self-maintenance in data warehousing environments,” in VLDB’97, Proceedings of the 23rd International Conference on Very Large Data Bases, Athens, Greece, August 1997, pp. 26–35.
K. Jeffay, D. Stanat, and C. Martel, “On non-preemptive scheduling of periodic and sporadic tasks,” in Proceedings of the 12th IEEE Real-Time Systems Symposium, San Antonio, TX, 1991, pp. 129–139.
A. Labrinidis and N. Roussopoulos, “WebView materialization,” in Proceedings of the ACM SIGMOD International Conference on Management of Data, Dallas, TX, 2000.
J. Leung and M. Merrill, “A note on preemptive scheduling of periodic, real-time tasks,” Information Processing Letters 11, 1980, 115–118.
J. Leung and J. Whitehead, “On the complexity of fixed-priority scheduling of periodic, real-time tasks,” Performance Evaluation 2, 1982, 237–250.
C. L. Liu and J. Layland, “Scheduling algorithms for multiprogramming in a hard-real-time environment,” Journal of the Association for Computing Machinery 20, 1973, 46–61.
H. Liu, W.-K. Ng, and E.-P. Lim, “Model and research issues for refreshing a very large website,” in Proceedings of the 1st International Conference on Web-Based Information Systems Engineering (WISE2000), Hong Kong, June 2000.
H. Liu, W.-K. Ng, and E.-P. Lim, “Keeping a very large website up-to-date: Some feasibility results,” in Proceedings of the 1st International Conference on Electronic Commerce and Web Technologies (EC-Web2000), Greenwich, UK, September 2000.
H. Liu, W.-K. Ng, and E.-P. Lim, “Query integration for refreshing Web views,” in Proceedings of the 12th International Conference on Database and Expert System Applications (DEXA 2001), Munich, Germany, September 2001.
H. Loeser, “Keeping Web pages up-to-date with SQL:1999,” in Proceedings of the International Database Engineering and Applications Symposium (IDEAS 2000), Yokohama, Japan, September 2000.
A. Mok, “Fundamental design problems of distributed systems for the hard real-time environment,” Ph.D. Thesis, MIT Laboratory for Computer Science, 1983.
X. Qian and G. Wiederhold, “Incremental recomputation of active relational expressions,” IEEE Transac-tions on Knowledge and Data Engineering 3, September 1991, 337–341.
O. Shmueli and A. Itai, “Maintenance of views,” in Proceedings of the ACM SIGMOD International Conference on Management of Data, Boston, MA, May 1984, pp. 240–255.
G. Sindoni, “Incremental maintenance of hypertext views,” in Proceedings of the ACM SIGMOD Workshop on the Web and Databases (WebDB’98), Valencia, Spain, 1998.
J. A. Stankovic, “Strategic directions in real-time and embedded systems,” ACM Computing Surveys 28(4), December 1996.
H. M. Taylor and S. Karlin, An Introduction to Stochastic Modeling, 3rd ed., Academic Press, New York, 1998.
J. Wang, “A survey of Web caching schemes for the Internet,” Technical Report, Cornell University, 1999.
Y. Zhuge, H. Garcia-Molina, J. Hammer, and J. Widom, “View maintenance in a warehousing environment,” in Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data, San Jose, CA, 22–25 May 1995, pp. 316–327.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Liu, H., Ng, WK. & Lim, EP. Scheduling Queries to Improve the Freshness of a Website. World Wide Web 8, 61–90 (2005). https://doi.org/10.1023/B:WWWJ.0000047378.69751.72
Issue Date:
DOI: https://doi.org/10.1023/B:WWWJ.0000047378.69751.72