Abstract
Retrieving large amounts of information over wide area networks, including the Internet, is problematic due to issues arising from latency of response, lack of direct memory access to data serving resources, and fault tolerance. This paper describes a design pattern for solving the issues of handling results from queries that return large amounts of data. Typically these queries would be made by a client process across a wide area network (or Internet), with one or more middle-tiers, to a relational database residing on a remote server. The solution involves implementing a combination of data retrieval strategies, including the use of iterators for traversing data sets and providing an appropriate level of abstraction to the client, double-buffering of data subsets, multi-threaded data retrieval, and query slicing. This design has recently been implemented and incorporated into the framework of a commercial software product developed at Oracle Corporation.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
D. Alur, J. Crupi, and D. Malks. Core J2EE Patterns: Best Practices and Design Strategies, pages 390–407. Prentice Hall, 2001.
D. Alur, J. Crupi, and D. Malks. Core J2EE Patterns: Best Practices and Design Strategies, pages 261–290. Prentice Hall, 2001.
E. Chan and K. Ueda. Efficient query result retrieval over the web. In Proceedings International Conference on Parallel and Distributed Systems. IEEE Computer Society, 2000.
J.D. Foley, A. van Dam, S.K. Feiner, and J.F. Hughes. Computer Graphics: Principles and Practice. AddisonWesley, 1990.
E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design Patterns: Elements of Reusable Object-Oriented Software. AddisonWesley, 1999.
Sun Microsystems Inc. Page-by-page iterator. Available online at http://java.sun.com/blueprints/patterns/j2ee_patterns/page_by_page_iterator/.
B. Long and P. Strooper. A case study in testing distributed systems. In Proceedings of the 3rd International Symposium on Distributed Objects and Applications, pages 20–29. IEEE Computer Society, 2001.
M.R. Lyu. Guest editor’s introduction to the special issue on web technologies. IEEE Transactions on Knowledge and Data Engineering, 11(4):505–508, 1991.
J. Waldo, G. Wyant, A. Wollrath, and S. Kendall. A note on distributed computing. Technical Report 94-29, Sun Microsystems Laboratories, Inc., California, November 1994.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Long, B. (2002). A Design Pattern for Efficient Retrieval of Large Data Sets from Remote Data Sources. In: Meersman, R., Tari, Z. (eds) On the Move to Meaningful Internet Systems 2002: CoopIS, DOA, and ODBASE. OTM 2002. Lecture Notes in Computer Science, vol 2519. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36124-3_44
Download citation
DOI: https://doi.org/10.1007/3-540-36124-3_44
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-00106-5
Online ISBN: 978-3-540-36124-4
eBook Packages: Springer Book Archive