Automated Extraction of Hit Numbers from Search Result Pages

  • Yanyan Ling
  • Xiaofeng Meng
  • Weiyi Meng
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4016)


When a query is submitted to a search engine, the search engine returns a dynamically generated result page that contains the number of hits (i.e., the number of matching results) for the query. Hit number is a very useful piece of information in many important applications such as obtaining document frequencies of terms, estimating the sizes of search engines and generating search engine summaries. In this paper, we propose a novel technique for automatically identifying the hit number for any search engine and any query. This technique consists of three steps: first segment each result page into a set of blocks, then identify the block(s) that contain the hit number using a machine learning approach, and finally extract the hit number from the identified block(s) by comparing the patterns in multiple blocks from the same search engine. Experimental results indicate that this technique is highly accurate.


Search Engine Frequent Word Query Keyword Result Page Induce Decision Tree 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lin Can, Zhang Qian, Xiaofeng Meng, and Wenyin Lin. Postal address detection from web documents. In WIRI, pages 40–45. IEEE Computer Society, 2005. Google Scholar
  2. 2.
    Cope, J., Craswell, N., Hawking, D.: Automated discovery of search interfaces on the web. In: Schewe, K.-D., Zhou, X. (eds.) ADC. CRPIT, vol. 17, pp. 181–189. Australian Computer Society (2003)Google Scholar
  3. 3.
    Doorenbos, R.B., Etzioni, O., Weld, D.S.: A scalable comparison-shopping agent for the world-wide web. Agents, 39–48 (1997)Google Scholar
  4. 4.
    Ipeirotis, P.G., Gravano, L., Sahami, M.: Probe, count, and classify: Categorizing hidden web databases. In: SIGMOD Conference (2001)Google Scholar
  5. 5.
    Liu, B., Grossman, R.L., Zhai, Y.: Mining web pages for data records. IEEE Intelligent Systems 19(6), 49–55 (2004)CrossRefGoogle Scholar
  6. 6.
    Witten, I.H., Frank, E.: Data Mining. In: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)Google Scholar
  7. 7.
    Zhao, H., Meng, W., Wu, Z., Raghavan, V., Yu, C.T.: Fully automatic wrapper generation for search engines. In: Ellis, A., Hagino, T. (eds.) WWW, pp. 66–75. ACM, New York (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Yanyan Ling
    • 1
  • Xiaofeng Meng
    • 1
  • Weiyi Meng
    • 2
  1. 1.School of InformationRenmin University of ChinaChina
  2. 2.Dept. of Computer ScienceSUNY at BinghamtonBinghamtonUSA

Personalised recommendations