International Journal on Digital Libraries

, Volume 5, Issue 2, pp 120–132 | Cite as

An agent infrastructure for on-demand processing of remote-sensing archives

  • Yanyan Yang
  • Omer F. Rana
  • David W. Walker
  • Roy Williams
  • Christos Georgousopoulos
  • Massimo Caffaro
  • Giovanni Aloisio
Regular contribution

Abstract

Advances in data collection techniques and database technologies, such as remote sensing and satellite telemetry, have led to the collection of huge amounts of data distributed among large databases and heterogeneous remote sites. Intelligent and automatic processing of the distributed data and efficiently supporting scientific collaboration between both professional and casual users is a highly demanding task. It is also particularly challenging when the system must cope with active data that is processed on-demand. These requirements have generated an urgent need for more advanced software infrastructure to create, maintain, evolve, and federate these active digital libraries of scientific data. Traditional models of distributed computing are inadequate to support such complex applications. As part of the ongoing Synthetic Aperture Radar Atlas (SARA) Digital Library project, the research presented here proposes a collaborating mobile agent approach to on-demand processing of remote sensing data. The approach, which is based on autonomous data processing and enables different image analysis algorithms to be wrapped as mobile agents, is expected to be an improvement over the static CGI-based interface and inefficient information discovery that are currently used by SARA. We discuss the agent-based infrastructure we have developed. The SARA system allows users to dispatch their compute-intensive jobs as mobile agents. Since the agents can be programmed to satisfy their specific goals, even if they move and lose contact with their creators they can survive intermittent or unreliable network connections. During their lifetime, the agents can also move themselves autonomously from one server to another for load balancing, and to enhance data locality and fault tolerance. The SARA system relies on XML to support agent communications on clusters of servers. Although the examples presented are based mainly on the SARA system, the proposed techniques are applicable to other active archives. In particular, we believe the proposed agent design can be used to dynamically configure distributed parallel computing resources and automatically integrate data analysis in remote sensing systems.

Keywords

Digital libraries Mobile agents Parallel and distributed computing XML Remote sensing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Andresen D, Carver L, Dolin R, Fischer C, Frew J, Goodchild M, Ibarra O, Kothuri R, Larsgaard M, Manjunath BS, Nebert D, Simpson J, Smith TR, Yang T, Zheng Q (1995) The WWW prototype of the Alexandria Digital Library. Proceedings of ISDL’95: International Symposium on Digital Libraries, Japan, 22–25 August 1995, pp 17–27Google Scholar
  2. 2.
    Aloisio G, Cafaro M, Kremenek G, Williams RD, Messina P (1997) A Distributed Web-Based Metacomputing Environment. In: Proc. HPCN Europe 1997, Vienna, Austria, LNCS, vol 1225. Springer-Verlag, pp 480–486Google Scholar
  3. 3.
    Aloisio G, Milillo G, Williams RD (1999) An XML architecture for high-performance web-based analysis of remote-sensing archives. Future Gener Comput Syst 16:91–100CrossRefGoogle Scholar
  4. 4.
    Bishop AP (1998) Digital libraries and knowledge disaggregation: the use of journal article components. In: Proceedings of the ACM Digital Libraries 1998 Conference. ACM, New YorkGoogle Scholar
  5. 5.
    Birmingham WP, Durfee EH, Mullen T, Wellman MP (1995) The distributed agent architecture of the University of Michigan digital library (extended abstract). In: AAAI Spring Symposium on Information Gathering. http://citeseer.ist.psu.edu/birmingham95distributed.htmlGoogle Scholar
  6. 6.
    Aloisio G, Caffaro M (2003) A Dynamic Earth Observation System. Parallel Computing 29(10):1357–1362, Special Issue on High performance computing with geographical dataGoogle Scholar
  7. 7.
    Christel M, Kanade T, Mauldin M, Reddy R, Stevens S, Wactlar H (1996) Techniques for the Creation and Exploration of Digital Video Libraries. In: Furht B (ed) Multimedia Tools and Applications (Volume 2). Kluwer Academic Publishers, Boston, MA. http://www-2.cs.cmu.edu/∼wactlar/mmtools.pdfGoogle Scholar
  8. 8.
    Christel M, Martin D (1998) Information visualization within a digital video library. J Intell Inf Syst 11(3):235–257 Supplement 26CrossRefGoogle Scholar
  9. 9.
    Coddington PD, Hawick KA, James HA (1999) Web-based access to distributed high-performance geographic information systems for decision support. Proceedings HICSS-32, MauiGoogle Scholar
  10. 10.
    Crum L (1995) University of Michigan Digital Library Project. Commun ACM 38(4):63–65 http://ei.cs.vt.edu/papers/SMART95/CACMcov/9504.htmlGoogle Scholar
  11. 11.
    Document content description for XML (Viewed 2001) http://www.w3.org/TR/NOTE-dcdGoogle Scholar
  12. 12.
    Digital Library Initiative, FY (1993) A joint initiative of the National Science Foundation, the Advanced Research Projects Agency, and the National Aeronautics and Space Administration. U.S. Government document NSF 93-141Google Scholar
  13. 13.
    Fox E (1993) Digital libraries. IEEE Comput 26(11):79–81MathSciNetCrossRefGoogle Scholar
  14. 14.
    Frew J, Freeston M, Freitas N, Hill L, Janee G, Lovette K, Nideffer R, Smith T, Zheng Q (1998) The Alexandria Digital Library architecture. In: Nikolaou C, Stephanidis C (eds) Proceedings of the Second European Conference on Research and Advanced Technology for Digital Libraries (ECDL’98), Heraklion, Crete, Greece, pp 61–73Google Scholar
  15. 15.
    Hamard K, Nie JY, Bochmann G, Godin R, Kerhervé B, Radhakrishnan T, Shinghal R, Turner JM, Berouti F, Ferrie FP (1999) A Multi-Level Agent System for Digital Libraries. Agent Oriented Information System (AOIS) at ACM Autonomous Agents Conference, Seattle, pp 77–91Google Scholar
  16. 16.
    Sun Microsystems (Viewed 2001) Java Wireless Toolkit. http://java.sun.com/products/j2mewtoolkit/Google Scholar
  17. 17.
    Lange DB, Oshima M (1998) Programming and Deploying Java Mobile Agents with Aglets. Addison-Wesley, ISBN 0-201-32582-9Google Scholar
  18. 18.
    Lightweight extensible agent platform (Viewed 2001) http://leap.crm-paris.com/Google Scholar
  19. 19.
    Marchionini G (1998) Digital library research and development. In: Kent A (ed) Encyclopedia of library and information science. Supplement 26. http://www.glue.umd.edu/∼march/digital_library_R_and_D.htmlGoogle Scholar
  20. 20.
    The NASA/JPL Imaging Radar (Viewed 2001) http://soughport.jpl.nasa.govGoogle Scholar
  21. 21.
    Nabil RA, Atluri V, Adiwijaya I (2000) SI in digital libraries. Commun ACM 43(6):64–72CrossRefGoogle Scholar
  22. 22.
    Paepcke A, Baldonado MQW, Chang KC-C, Cousins SB, Garcia-Molina H (1999) Using Distributed Objects to Build the Stanford Digital Library Infobus. IEEE Computer 32(2):80–87CrossRefGoogle Scholar
  23. 23.
    Papastavrou S, Pitoura E, Samaras G (1998) Mobile agents for WWW distributed database access (extended version). Technical Report TR 98-12, Univ. Of Cyprus, Computer Science DepartmentGoogle Scholar
  24. 24.
    Rana OF, Yang Y, Georgousopoulos C, Walker DW, Williams R (2000) Agent based data analysis for the SARA digital library. Workshop on Advanced Data Storage /Management Techniques for HPC, Warrington, UK, 23–25 February 2000Google Scholar
  25. 25.
    Rasmusson A, Olsson T, Hansen P (1998) A virtual community library: SICS digital library infrastructure project. In: Research and advanced technology for digital libraries. Second European Conference ECDL’98, Heraklion, Crete, 19–23 September 1998Google Scholar
  26. 26.
    Resource description framework (RDF) (Viewed 2001) Schema specification. http://www.w3.org/TR/WD-rdf-schema/Google Scholar
  27. 27.
    Richards J (1994) Remote sensing digital image analysis: an introduction, 2nd edn. Springer, Berlin Heidelberg New YorkGoogle Scholar
  28. 28.
    SARA (The Synthetic Aperture Radar Atlas) (Viewed 2001) http://www.cacr.caltech.edu/saraGoogle Scholar
  29. 29.
    SAR Alaska Facility (Viewed 2001) Frequently asked questions. http://www.asf.alaska.edu/user_serv/sar_faq.htmlGoogle Scholar
  30. 30.
    Special Issue on Digital Libraries (1995) Commun ACM 38(4):22–97CrossRefGoogle Scholar
  31. 31.
    Special issue on Building Large Scale Digital Libraries (1996) IEEE Computer 29(5):22–76 http://csdl.computer.org/comp/mags/co/1996/05/r5toc.htmCrossRefGoogle Scholar
  32. 32.
    Sycara K, Lu J, Klusch M (1998) Interoperability among heterogeneous software agents on the Internet. Technical Report CMU-RI-TR-98-22, The Robotics Institute, Carnegie Mellon University, Pittsburgh, PAGoogle Scholar
  33. 33.
    The imaging radar homepage (Viewed 2001) http://southport.jpl.nasa.gov/Google Scholar
  34. 34.
    Voyager 3.1, ObjectSpace Inc (Viewed 2001) http://www.objectspace.comGoogle Scholar
  35. 35.
    Wilensky R (1995) UC Berkeley’s digital library project. Commun ACM 38(4)Google Scholar
  36. 36.
    Williams R, Bunn J, Moore R (1998) Interfaces to scientific data archives. Report of a Workshop sponsored by the National Science Foundation. http://www.cacr.caltech.edu/isdaGoogle Scholar
  37. 37.
    Williams R, Sears B (1998) A High-Performance ActiveDigital Library. Journal of Parallel Computing, Special issue on Metacomputing. Elsevier Science, pp 1791–1806 http://elsevier.lib.sjtu.edu.cn/cgi-bin/sciserv.pl?collection=journals&journal=01678191&issue=v24i12-13Google Scholar
  38. 38.
    XML-data (Viewed 2001) http://www.w3.org/TR/1998/NOTE-XML-dataGoogle Scholar
  39. 39.
    Blackburn K, Lazzarini A, Prince T, Williams R (1999) XSIL: extensible scientific interchange language. HPCN’99, Amsterdam, pp 513–524Google Scholar
  40. 40.
    Yang Y, Rana OF, Georgousopoulos C, Walker DW, Williams RD (2000) Mobile Agents and the SARA Digital Library. In: Proceedings of IEEE Advances in Digital Libraries 2000, Washington, DC, May 22–24 2000. IEEE Computer Society Press, pp 71–77Google Scholar

Copyright information

© Springer-Verlag 2005

Authors and Affiliations

  • Yanyan Yang
    • 1
  • Omer F. Rana
    • 1
  • David W. Walker
    • 1
  • Roy Williams
    • 2
  • Christos Georgousopoulos
    • 1
  • Massimo Caffaro
    • 3
  • Giovanni Aloisio
    • 3
  1. 1.Department of Computer ScienceCardiff UniversityCardiffUK
  2. 2.Center for Advanced Computing ResearchCalifornia Institute of TechnologyPasadenaUSA
  3. 3.Department of Innovative Engineering, Faculty of EngineeringUniversity of LecceLecceItaly

Personalised recommendations