Multi-modal Services for Web Information Collection Based on Multi-agent Techniques
With the rapid information growth on the Internet, web information collection is becoming increasingly important in many web applications, especially in search engines. The performance of web information collectors has a great influence on the quality of search engines, so when it comes to web spiders, we usually focus on their speed and accuracy. In this paper, we point out that customizability is also an important feature of a well-designed spider, which means spiders should be able to provide multi-modal services to satisfy different users with different requirements and preferences. And we have developed a parallel web spider system based on multi-agent techniques. It runs with high speed and high accuracy, and what’s the most important, it can provide its services in multiple perspectives and has good extensibility and personalized customizability.
KeywordsSearch Engine Index Agent ISDN System 10th International World Wide Good Extensibility
Unable to display preview. Download preview PDF.
- 4.Diligenti, M., Coetzee, F.M., Lawrence, S., et al.: Focused Crawling Using Context Graphs. In: Proceedings of the 26th VLDB Conference, Cairo, Egypt (2000)Google Scholar
- 5.Najork, M., Wiener, J.L.: Breadth-first search crawling yields high-quality pages. In: Proceeding of 10th International World Wide Web Conference (2001)Google Scholar
- 6.Dong, M., Liu, S., Zhang, H., Shi, Z.: Parallel Web Spider Based on Intelligent Agent. In: Proceedings of The 5th Pacific Rim International Workshop on Multi-Agents, Tokyo (2002)Google Scholar
- 9.Peng, H., Lin, Z.: Search Engines and Meta Search Engines on Internet. Computer Science 29, 1–12 (2002)Google Scholar