Web Search pp 309-340 | Cite as

Web Searching: A Quality Measurement Perspective

  • D. Lewandowski
  • N. Höchstötter
Part of the Information Science and Knowledge Management book series (ISKM, volume 14)


The purpose of this paper is to describe various quality measures for search engines and to ask whether these are suitable. We especially focus on user needs and their use of Web search engines. The paper presents an extensive literature review and a first quality measurement model, as well. Findings include that Web search engine quality can not be measured by just retrieval effectiveness (the quality of the results), but should also consider index quality, the quality of the search features and Web search engine usability. For each of these sections, empirical results from studies conducted in the past, as well as from our own research are presented. These results have implications for the evaluation of Web search engines and for the development of better search systems that give the user the best possible search experience.


Search Engine Search Query Search Feature Result List Retrieval Effectiveness 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Acharya A, Cutts M, Dean J, Haahr P, Henzinger M, Hoelzle U, et al. (2005) Information retrieval based on historical data. USAGoogle Scholar
  2. Beitzel S, Jensen C, Chowdhury A, Grossman D, Frieder O (2004) Hourly analysis of a very large topically categorized web query log (pp. 321–328). Paper presented at the ACM SIGIR Conference on Research and Development in Information Retrieval, Sheffield, UK. ACM PressGoogle Scholar
  3. Bergman MK (2001) The deep web: surfacing hidden value. Journal of Electronic Publishing 7Google Scholar
  4. Bharat K, Broder A (1998) A technique for measuring the relative size and overlap of public web search engines. Computer Networks and ISDN Systems 30: 379–388CrossRefGoogle Scholar
  5. Brin S, Page L (1998) The anatomy of a large-scale hypertextual web search engine. Computer Networks and ISDN Systems 30: 107–117CrossRefGoogle Scholar
  6. Broder A (2002) A taxonomy of web search. SIGIR Forum 36: 3–10CrossRefGoogle Scholar
  7. Broder A, Kumar R, Maghoul F, Raghavan P, Rajagopalan S, Stata R, et al. (2000) Graph structure in the web. Retrieved 15.4.2006, from
  8. Cacheda F, Viña Á (2001) Understanding how people use search engines: a statistical analysis for e-business. 1: 319–325. Paper presented at the e-2001 E-Business and E-Work Conference and ExhibitionGoogle Scholar
  9. Chignell MH, Gwizdka J, & Bodner RC (1999) Discriminating meta-search: a framework for evaluation. Information Processing and Management, 35(3), 337–362CrossRefGoogle Scholar
  10. Ding W, Marchionini G (1996) A comparative study of web search service performance. Proceedings of the 59th American Society for Information Science Annual Meeting (pp. 136–142): Learned InformationGoogle Scholar
  11. Fauldrath J, & Kunisch A (2005) Kooperative Evaluation der Usability von Suchmaschineninter-faces. Information: Wissenschaft und Praxis, 56(1), 21–28Google Scholar
  12. Ford N, Miller D, Moss N (2002) Web search strategies and retrieval effectiveness: an empirical study. Journal of Documentation 58: 30–48CrossRefGoogle Scholar
  13. Geoghegan T (2004) Search wars: which is best? from
  14. Gordon M, & Pathak P (1999) Finding information on the World Wide Web: the retrieval effectiveness of search engines. Information Processing & Management, 35(2), 141–180CrossRefGoogle Scholar
  15. Greisdorf H, Spink A (2001) Median measure: an approach to IR systems evaluation. Information Processing and Management 37: 843–857zbMATHCrossRefGoogle Scholar
  16. Griesbaum J (2004) Evaluation of three German search engines:, and Information Research 9Google Scholar
  17. Griesbaum J, Rittberger M, Bekavac B (2002) In: R. Hammwöhner, C. Wolff and C. Womser-Hacker (Eds.), Deutsche Suchmaschinen im Vergleich:,, und (pp. 201–223). Paper presented at the Information und Mobilität. Optimierung und Vermeidung von Mobilität durch Information. 8. Internationales Symposium für Informationswissenschaft. UVKGoogle Scholar
  18. Gulli A, Signorini A (2005) The indexable web is more than 11.5 billion pages (pp. 902–903). Paper presented at the Special Interest Tracks and Posters of the 14th International Conference on World Wide Web, Chiba, JapanGoogle Scholar
  19. Hock R (2004) The latest field trip: An update on field searching in web search engines. Online (Wilton, Connecticut), 28(5), 15–21Google Scholar
  20. Hoelscher C, Strube G (2000) Web search behavior of Internet experts and newbies (pp. 337–346). Paper presented at the 9th International World Wide Web ConferenceGoogle Scholar
  21. Hotchkiss G, Garrison M, & Jensen S (2004) Search Engine Usage in North America, A Research Initiative by Enquiro. Retrieved 16.3.2006, from
  22. Ingwersen P, Järvelin K (2005). The turn: integration of information seeking and retrieval in context. Springer, DordrechtzbMATHGoogle Scholar
  23. Jansen BJ (2000) An investigation into the use of simple queries on web IR systems. Information Research 6Google Scholar
  24. Jansen BJ, Spink A (2003) An analysis of web documents retrieved and viewed (pp. 64–69). Paper presented at the 4th International Conference on Internet ComputingGoogle Scholar
  25. Jansen BJ, Spink A (2006) How we are searching the world wide web? a comparison of nine search engine transaction logs. Information Processing and Management 42: 258–263Google Scholar
  26. Ke Y, Deng L, Ng W, Lee DL (2006) Web dynamics and their ramifications for the development of web search engines. Computer Networks 50: 1430–1447zbMATHCrossRefGoogle Scholar
  27. Kleinberg JM (1999) Authoritative sources in a hyperlinked environment. Journal of the ACM 46: 604–632zbMATHCrossRefMathSciNetGoogle Scholar
  28. Korfhage RR (1997) Information storage and retrieval. Wiley, New YorkGoogle Scholar
  29. Lawrence S, Giles CL (1998) Searching the world wide web. Science 280: 98–100CrossRefGoogle Scholar
  30. Lawrence S, Giles CL (1999) Accessibility of information on the web. Nature 400: 107–109CrossRefGoogle Scholar
  31. Leighton HV, Srivastava J (1999) First 20 precision among world wide web search services (search engines). Journal of the American Society for Information Science 50: 870–881CrossRefGoogle Scholar
  32. Lewandowski D (2004a) Abfragesprachen und erweiterte auchfunktionen von WWW-auchmaschinen. Information Wissenschaft und Praxis 55: 97–102Google Scholar
  33. Lewandowski D (2004b) Bewertung von linktopologischen Verfahren als bestimmender Ranking-Faktor bei WWW-Suchmaschinen, Wissensorganisation und gesellschaftliche Verantwortung. 9. Tagung der Deutschen ISKO (Wissensorganisation 2004). Duisburg, GermanyGoogle Scholar
  34. Lewandowski D (2004c) Date-restricted queries in web search engines. Online Information Review 28: 420–427CrossRefGoogle Scholar
  35. Lewandowski D (2005a) Web searching, search engines and information retrieval. Information Services and Use 18: 137–147Google Scholar
  36. Lewandowski D (2005b) Yahoo–Zweifel an den Angaben zur Indexgröße, Suche in mehreren Sprachen. Password 20: 21–22Google Scholar
  37. Lewandowski D (2006a) Aktualität als erfolgskritischer Faktor bei Suchmaschinen. Information Wissenschaft und Praxis 57: 141–148Google Scholar
  38. Lewandowski D (2006b) Suchmaschinen als Konkurrenten der Bibliothekskataloge: Wie Bibliotheken ihre Angebote durch Suchmaschinentechnologie attraktiver und durch Öffnung für die allgemeinen Suchmaschinen populärer machen können. Zeitschrift für Bibliothekswesen und Bibliographie 53: 71–78Google Scholar
  39. Lewandowski D (2006c) Zur Bewertung der Qualität von Suchmaschinen. In: J. Eberspächer and S. Holtel (Eds.), Suchen und Finden im Internet (pp. 195–199). Heidelberg: SpringerGoogle Scholar
  40. Lewandowski D, Mayr P (2006) Exploring the academic invisible web. Library Hi Tech 24: 529–539CrossRefGoogle Scholar
  41. Lewandowski D, Wahlig H, Meyer-Bautor G (2006) The freshness of web search engine databases. Journal of Information Science 32: 133–150CrossRefGoogle Scholar
  42. MacCall SL, Cleveland AD (1999) A relevance-based quantitative measure for Internet information retrieval evaluation (pp. 763–768). Paper presented at the Proceedings of the American Society for Information Science Annual MeetingGoogle Scholar
  43. Machill M, Neuberger C, Schweiger W, Wirth W (2003) Wegweiser im netz: qualität und nutzung von suchmaschinen. In: M. Machill and C. Welp (Eds.), Wegweiser im Netz. Gütersloh: Bertelsmann StiftungGoogle Scholar
  44. Machill M, Neuberger C, Schweiger W, Wirth W (2004) Navigating the Internet: a study of German-language search engines. European Journal of Communication 19: 321–347CrossRefGoogle Scholar
  45. Nicholson S (2000) Raising reliability of Web search tool research through replication and chaos theory. Journal of the American Society for Information Science, 51(8), 724–729CrossRefGoogle Scholar
  46. Notess GR (2006, 22.5.2006) Search Engine Features Chart. Retrieved 16.3.2006, from
  47. Notess GR (2003) Search engine statistics: freshness showdown. Retrieved 4.1.2005, from
  48. Ntoulas A, Cho J, Olston C (2004) What’s new on the web? the evolution of the web from a search engine perspective. Paper presented at the Thirteenth WWW Conference, New York, USAGoogle Scholar
  49. Ojala M (2002) Web search engines: Search syntax and features. Online (Wilton, Connecticut), 26(5), 28Google Scholar
  50. Ozmutlu H, Spink A, Ozmutlu S (2003) A study of multitasking web search (pp. 145–148). Paper presented at the International Conference on Information Technology: Computers and CommunicationsGoogle Scholar
  51. Page L, Brin S, Motwani R, Winograd T (1998) The pagerank citation ranking : bringing order to the web. Retrieved 24.7.2006, from–66
  52. Parasuraman A, Zeithaml VA, Berry LL (1988). SERVQUAL: a multiple-item scale for measuring consumer perceptions of service quality. Journal of Retailing 64: 12–40Google Scholar
  53. Risvik KM, Michelsen R (2002) Search engines and web dynamics. Computer Networks 39: 289–302CrossRefGoogle Scholar
  54. Saracevic T (1995) Evaluation of evaluation in information retrieval (pp. 138–146). Paper presented at the SIGIRS95, Seattle, CA. ACM PressGoogle Scholar
  55. Schmidt-Maenz (2007) Untersuchung des suchverhaltens im web–Interaktion von Internetnutzern mit Suchmaschinen, Dr. Kovac Verlag, HamburgGoogle Scholar
  56. Schmidt-Maenz N, Bomhardt C (2005) Wie suchen onliner im Internet? Science Factory/Absatzwirtschaftb 2: 5–8Google Scholar
  57. Schmidt-Maenz N, Gaul W (2005). Web mining and online visibility. In: C. Weihs and W. Gaul (Eds.), Classification–the ubiquitous challenge (pp. 418–425): SpringerGoogle Scholar
  58. Schmidt-Maenz N, & Koch M (2005) Patterns in Search Queries. In D. Baier, R. Decker & L. Schmidt-Thieme (Eds.), Data Analysis and Decision Support (pp. 122–129). Heidelberg: SpringerCrossRefGoogle Scholar
  59. Schmidt-Maenz N, Koch M (2006) A general classification of (search) queries and terms (pp. 375–381). Paper presented at the 3rd International Conference on Information Technologies: Next Generations, Las Vegas, Nevada, USAGoogle Scholar
  60. Schwartz C (1998) Web search engines. Journal of the American Society for Information Science and Technology, 49(11), 973–982CrossRefGoogle Scholar
  61. Sherman C, Price G (2001) The invisible web: uncovering information sources search engines can’t see. Information Today, Medford, NJGoogle Scholar
  62. Silverstein C, Henzinger M, Marais H, Moricz M (1999) Analysis of a very large web search engine query log. ACM SIGIR Forum 33: 6–12CrossRefGoogle Scholar
  63. Singhal A, Kaszkiel M (2001) A case study in web search using TREC algorithms (pp. 708–716). Paper presented at the 10th International Conference on World Wide Web, Hong KongGoogle Scholar
  64. Spink A, Jansen BJ (2004) Web search: public searching of the web (Vol. 6). Dordrecht, Boston, London: Kluwer AcademicGoogle Scholar
  65. Spink A, Jansen B, Ozmutlu H (2000) Use of query reformulation and relevance feedback by excite users. Internet Research: Electronic Networking Applications and Policy 19: 317–328Google Scholar
  66. Spink A, Ozmutlu S, Ozmutlu H, Jansen B (2002) U.S. versus european web searching processes. Journal of the American Society for Information Science and Technology 53: 639–652CrossRefGoogle Scholar
  67. Spink A, Wolfram D, Jansen B, Saracevic T (2001) Searching the web: the public and their queries. Journal of the American Society for Information Science and Technology 52: 226–234CrossRefGoogle Scholar
  68. Spink A, Jansen BJ, Blakely C, & Koshman S. (2006). A study of results overlap and uniqueness among major Web search engines. Information Processing & Management, 42(5), 1379–1391CrossRefGoogle Scholar
  69. Su LT (1998) Value of search results as a whole as the best single measure of information retrieval performance. Information Processing and Management 34: 557–579CrossRefGoogle Scholar
  70. Sullivan D (2005) Search engine sizes. Retrieved 24.7.2006, from
  71. Vaughan L (2004) New measurements for search engine evaluation: proposed and tested. Information Processing and Management 40: 677–691zbMATHCrossRefGoogle Scholar
  72. Vaughan L, Thelwall M (2004) Search engine coverage bias: evidence and possible causes. Information Processing and Management 40: 693–707CrossRefGoogle Scholar
  73. Véronis J (2006) A comparative study of six search engines. Retrieved 15.3.2006, from
  74. Wang H, Xie M, Goh TN (1999) Service quality of Internet search engines. Journal of Information Science 25: 499–507CrossRefGoogle Scholar
  75. Williams ME (2005) The state of databases today: 2005. In: Gale Directory of Databases (Vol. 2, pp. XV-XXV). Detroit, Mich.: Gale GroupGoogle Scholar
  76. Wolff, C. (2000) Vergleichende evaluierung von such- und metasuchmaschinen, 7. Internationales Symposium für Informationswissenschaft (ISI 2000) (pp. 31–38). Universitätsverlag Konstanz, Darmstadt, GermanyGoogle Scholar
  77. Xie M, Wang H, Goh TN (1998) Quality dimensions of Internet search engines. Journal of Information Science 24: 365–372CrossRefGoogle Scholar
  78. Zien J, Meyer J, Tomlin J, Liu J (2000) Web query characteristics and their implications on search engines: Almaden Research CenterGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • D. Lewandowski
    • 1
  • N. Höchstötter
    • 2
  1. 1.Department InformationHamburg University of Applied SciencesHamburgGermany
  2. 2.Institute for Decision Theory and Management ScienceUniversitaet Karlsruhe (TH)KarlsruheGermany

Personalised recommendations