Skip to main content
SpringerLink
Log in

A New Measure of the Cluster Hypothesis

  • Conference paper
Advances in Information Retrieval Theory (ICTIR 2009)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 5766))

Included in the following conference series:

Abstract

We have found that the nearest neighbor (NN) test is an insufficient measure of the cluster hypothesis. The NN test is a local measure of the cluster hypothesis. Designers of new document-to-document similarity measures may incorrectly report effective clustering of relevant documents if they use the NN test alone. Utilizing a measure from network analysis, we present a new, global measure of the cluster hypothesis: normalized mean reciprocal distance. When used together with a local measure, such as the NN test, this new global measure allows researchers to better measure the cluster hypothesis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. van Rijsbergen, C.J.: Information Retrieval, 2nd edn., Butterworths (1979)

    Google Scholar 

  2. Jardine, N., van Rijsbergen, C.J.: The use of hierarchic clustering in information retrieval. Information Storage and Retrieval 7(5), 217–240 (1971)

    Article  Google Scholar 

  3. van Rijsbergen, C.J., Sparck Jones, K.: A test for the separation of relevant and non-relevant documents in experimental retrieval collections. Journal of Documentation 29, 251–257 (1973)

    Article  Google Scholar 

  4. Tombros, A., van Rijsbergen, C.J.: Query-sensitive similarity measures for the calculation of interdocument relationships. In: CIKM 2001, pp. 17–24 (2001)

    Google Scholar 

  5. Voorhees, E.M.: The cluster hypothesis revisited. In: SIGIR 1985, pp. 188–196 (1985)

    Google Scholar 

  6. Wilbur, W.J., Coffee, L.: The effectiveness of document neighboring in search enhancement. IPM 30(2), 253–266 (1994)

    Google Scholar 

  7. Smucker, M.D., Allan, J.: Measuring the navigability of document networks. In: SIGIR 2007 Web Information-Seeking and Interaction Workshop (2007)

    Google Scholar 

  8. Latora, V., Marchiori, M.: Efficient behavior of small-world networks. Physical Review Letters 87(19) (October 2001)

    Google Scholar 

  9. Smucker, M.D.: Evaluation of Find-Similar with Simulation and Network Analysis. PhD thesis, University of Massachusetts Amherst (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Management Sciences, University of Waterloo, Canada

    Mark D. Smucker

  2. Center for Intelligent Information Retrieval Department of Computer Science, University of Massachusetts, Amherst, USA

    James Allan

Authors
  1. Mark D. Smucker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James Allan
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computing Science, Sir Alwyn Williams Building, Lilybank Gardens, University of Glasgow, G12 8QQ, Glasgow, Scotland, UK

    Leif Azzopardi

  2. Microsoft Research Ltd, 7 JJ Thomson Avenue, CB3 0FB, Cambridge, UK

    Gabriella Kazai  & Stephen Robertson  & 

  3. Knowledge Media Institute,, The Open University, MK7 6AA, Milton Keynes, UK

    Stefan Rüger

  4. Microsoft Research Ltd, 7 JJ Thomson Avenue, CB3 0FB, Cambridge, United Kingdom

    Milad Shokouhi  & Emine Yilmaz  & 

  5. School of Computing, The Robert Gordon University, St Andrew Street, AB25 1HG, Aberdeen, UK

    Dawei Song

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Smucker, M.D., Allan, J. (2009). A New Measure of the Cluster Hypothesis. In: Azzopardi, L., et al. Advances in Information Retrieval Theory. ICTIR 2009. Lecture Notes in Computer Science, vol 5766. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04417-5_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-04417-5_27

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-04416-8

  • Online ISBN: 978-3-642-04417-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation