Skip to main content
SpringerLink
Log in

Improvement of HITS for Topic-Specific Web Crawler

  • Conference paper
Advances in Intelligent Computing (ICIC 2005)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 3644))

Included in the following conference series:

Abstract

The rapid growth of the World-Wide Web poses unprecedented scaling challenges for general-purpose crawlers. Topic-specific web crawler is developed to collect relevant web pages of interested topics form the Internet. Based on the analyses of HITS algorithm, a new P-HITS algorithm is proposed for topic-specific web crawler in this paper. Probability is introduced to select the URLs to get more global optimality, and the metadata of hyperlinks is appended in this algorithm to predict the relevance of web pages better. Experimental results indicate that our algorithm has better performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Murry, B.H., Moore, A.: Sizing the Internet. A White Paper. Cyveillance, Inc. (2000)

    Google Scholar 

  2. Bharet, K., Broder, A.: A Technique for Measuring the Relative Size and Overlap of Public Web Search Engines. In: Computer Networks and ISDN Systems, Special Issue on the 7th Int. World Wide Web Conf., Brisbane, vol. 30, pp. 1–7 (1998)

    Google Scholar 

  3. Lawrence, S., Giles, C.L.: Searching the World Wide Web. Science 280, 98–100 (1998)

    Article  Google Scholar 

  4. Debra, P., Post, R.: Information Retrieval in the World Wide Web: Making Client – Based Searching Feasible. In: Proc. 1st International World Wide Web Conference (1994)

    Google Scholar 

  5. Hersovici, M., Jacov, M., et al.: The Shark-search Algorithm-An Application: Tailored Web Site Mapping. In: Proc. 7th International World Wide Web Conference, Brisbane, Australia, pp. 317–326 (1998)

    Google Scholar 

  6. Menczer, F., Monge, A.E.: Scalable Web Search by Adaptive Online Agents: An Infospider Case Study. In: Intelligent Information Agents: Agent-based Information Discovery and Management on the Internet, Berlin, pp. 323–347 (1999)

    Google Scholar 

  7. Mukherjea, S.: WTMS: A System for Collecting and Analyzing Topic-specific Web Information. Computer Networks 33, 457–471 (2000)

    Article  Google Scholar 

  8. Salton, G., McGill, M.: Introduction to Modern Information Retrieval. McGraw-Hill, New York (1983)

    MATH  Google Scholar 

  9. Kleinberg, J.: Authoritative Sources In a Hyperlinked Environment. In: Proc. 9th Ann. ACM-SIAM Symp. Discrete Algorithms, pp. 668–677. ACM Press, New York (1998)

    Google Scholar 

  10. Rungsawang, A., Angkawattanawit, N.: Learnable Topic-specific Web Crawler. Network and Computer Applications 28, 97–114 (2005)

    Article  Google Scholar 

  11. Menczer, F.: Complementing Search Engines With Online Web Mining Agents. Decision Support Systems 35, 195–212 (2003)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Control Science and Engineering, Huazhong University of Science and Technology, Wuhan, Hubei, 430074, China

    Xiaojun Zong, Yi Shen & Xiaoxin Liao

Authors
  1. Xiaojun Zong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoxin Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Intelligent Computing Lab, Institute of Intelligent Machines, Chinese Academy of Sciences,, China

    De-Shuang Huang

  2. School of Computer & Information Technology, Beijing Jiaotong University, 100044, Beijing, P.R. China

    Xiao-Ping Zhang

  3. School of Electrical and Electronic Engineering, Nanyang Technological University, P.O. Box, Singapore

    Guang-Bin Huang

Rights and permissions

Reprints and permissions

Copyright information

© 2005 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Zong, X., Shen, Y., Liao, X. (2005). Improvement of HITS for Topic-Specific Web Crawler. In: Huang, DS., Zhang, XP., Huang, GB. (eds) Advances in Intelligent Computing. ICIC 2005. Lecture Notes in Computer Science, vol 3644. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11538059_55

Download citation

  • DOI: https://doi.org/10.1007/11538059_55

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-28226-6

  • Online ISBN: 978-3-540-31902-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation