Skip to main content
SpringerLink
Log in
Book cover

New Directions in Cognitive Information Retrieval pp 169–186Cite as

Implicit Feedback: Using Behavior to Infer Relevance

  • Chapter

Part of the book series: The Information Retrieval Series ((INRE,volume 19))

5. Conclusions

The chapter presented a framework for understanding and studying how behavior can be used as implicit relevance feedback. This included a classification and discussion of behaviors that have been used as implicit relevance feedback, a general discussion and characterization of implicit feedback research, and a presentation of example studies to illustrate how such studies have been conducted and how feedback has typically been measured and used. Finally, this chapter presented a discussion of key issues and problems associated with implicit feedback research and identified challenges for future research. The use of behavior as implicit relevance feedback is an exciting and promising approach to personalizing IR interactions. Although more effort needs to be made to fully understand how behaviors can be used as implicit relevance feedback, current research efforts offer a promising start.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Anick, P. (2003). Using Terminological Feedback for Web Search Refinement: A Log Based Study. In Proceedings of the 26th Annual ACM SIGIR International Conference on Research and Development in Information Retrieval (SIGIR’ 03) (pp. 88–95), Canada.

    Google Scholar 

  • Beaulieu, M. (1997). Experiments on Interfaces to Support Query Expansion. Journal of Documentation, 53(1), 8–19.

    Article  MathSciNet  Google Scholar 

  • Belkin, N. J., Cool, C., Kelly, D., Lin, S. J., Park, S. Y., Perez-Carballo, J., and Sikora, C. (2001). Iterative Exploration, Design and Evaluation of Support for Query Reformulation in Interactive Information Retrieval. Information Processing and Management, 37(3), 404–434.

    Article  Google Scholar 

  • Billsus, D. and Pazzani, M. J. (1999). A Personal News Agent That Talks, Learns and Explains. In Proceedings of the 3rd International Conference on Autonomous Agents (AGENTS’ 99), (pp. 268–275), USA.

    Google Scholar 

  • Bollacker, K. D., Lawrence, S., and Giles, C. L. (2000). Discovering Relevant Scientific Literature on the Web. IEEE Intelligent Systems, 15(2), 42–47.

    Article  Google Scholar 

  • Borlund, P. (2003). The Concept of Relevance in IR. Journal of the American Society for Information Science, 54(10), 913–925.

    Article  Google Scholar 

  • Budzik, J. and Hammond, K. (1999). Watson: Anticipating and Contextualizing Information Needs. In Proceedings of the 62nd Meeting of the American Society for Information Science (ASIS’ 99), (pp. 727–740), USA.

    Google Scholar 

  • Budzik, J., Hammond, K., and Birnbaum, L. (2001). Information Access in Context. Knowledge Based Systems, 14(1–2), 37–53.

    Article  Google Scholar 

  • Chen, M. LaPaugh, A. S. and Singh, J. P. (2002). Predicting Category Access for a User in Structured Information Space. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’ 02), (pp. 65–72), Finland.

    Google Scholar 

  • Claypool, M. Le, P. Waseda, M. and Brown, D. (2001). Implicit Interest Indicators. In Proceedings of the 6th International Conference on Intelligent User Interfaces (IUI’ 01), (pp. 33–40), USA.

    Google Scholar 

  • Cooper, M. D. and Chen, H.-M. (2001). Predicting the Relevance of a Library Catalog Search. Journal of the American Society for Information Science, 52(10), 813–827.

    Article  Google Scholar 

  • Dumais, S., Cutrell, E., Sarin, R., and Horvitz, E. (2004). Implicit Queries (IQ) for Contextualized Search. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’04), Sheffield, UK, 594.

    Google Scholar 

  • Fendlay, J. M., Walker, R., and Kengridge, R. W. (1995) (Eds.). Eye Movement Research: Mechanisms, Processes and Applications. New York: Elsevier Science Publishing.

    Google Scholar 

  • Fischer, G. and Stevens, C. (1991). Information Access in Complex, Poorly Structured Information Spaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’ 91), (pp. 63–70), USA.

    Google Scholar 

  • Goldberg, D., Nichols, D., Oki, B. M., and Terry, D. (1992). Using Collaborative Filtering to Weave an Information Tapestry. Communications of the ACM, 35(12), 61–70.

    Article  Google Scholar 

  • Golovchinsky, G., Price, M. N., and Schilit, B. N. (1999). From Reading to Retrieval: Freeform Ink Annotations as Queries. In Proceedings of the 24th Annual ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’ 99), (pp. 19–25), USA.

    Google Scholar 

  • Huang, C.-K., Chien, L.-F., and Oyang, Y.-J. (2003). Relevant Term Suggestion in Interactive Web Search Based on Contextual Information in Query Session Logs. Journal of the American Society for Information Science and Technology, 54(7), 638–649.

    Article  Google Scholar 

  • Ikehara, C. S., Chin, D. N., and Crosby, M. E. (2003). A Model for Integrating an Adaptive Information Filter Utilizing Biosensor Data to Assess Cognitive Load. In Proceedings of the Ninth International Conference on User Modeling (UM’ 03), (pp. 208–212), USA.

    Google Scholar 

  • Joachims, T. (2002). Optimizing Search Engines Using Clickthrough Data. In Proceedings of the 8th ACM C SIGKDD International Conference on Knowledge Discovery and Data Mining (SIGKDD’ 02), (pp. 133–142), Canada.

    Google Scholar 

  • Joachims, T. Freitag, D., and Mitchell, T. (1997). WebWatcher: A Tour Guide for the World Wide Web. In Proceedings of the International Joint Conference on Artificial Intelligence (ICML 97), (pp. 770–777), USA.

    Google Scholar 

  • Kamba, T., Sakagami, H., and Koseki, Y. (1997). ANATAGONOMY: A Personalized Newspaper on the World Wide Web. International Journal of Human-Computer Studies, 46, 789–803.

    Article  Google Scholar 

  • Kellar, M., Watters, C., Duffy, J., and Shepherd, M. (2004). Effect of Task on Time Spent Reading as an Implicit Measure of Interest. In Proceedings of the 67th American Society for Information Science and Technology (ASIS and T) Annual Meeting, 41, pp. 168–175.

    Article  Google Scholar 

  • Kelly, D. and Belkin, N. J. (2001). Reading Time, Scrolling and Interaction: Exploring Implicit Sources of User Preferences for Relevance Feedback During Interactive Information Retrieval. In Proceedings of the 24th Annual ACM SIGIR International Conference on Research and Development in Information Retrieval (SIGIR’ 01), (pp. 408–409), USA.

    Google Scholar 

  • Kelly, D. and Belkin, N. J. (2002). AUser Modeling System for Personalized Interaction and Tailored Retrieval in Interactive IR. In Proceedings of the Annual Conference of the American Society for Information Science and Technology (ASIST’ 02), (pp. 316–325), USA.

    Google Scholar 

  • Kelly, D. and Belkin, N. J. (2004). Display Time as Implicit Feedback: Understanding Task Effects. In Proceedingsof the 27th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR’ 04), (pp. 377–384), Sheffield, UK.

    Google Scholar 

  • Kelly, D. and Cool, C. (2002). The Effects of Topic Familiarity on Information Search Behavior. In Proceedings of the Second ACM/IEEE Joint Conference on Digital Libraries (JCDL’ 02), (pp. 74–75), USA.

    Google Scholar 

  • Kelly, D. and Teevan, J. (2003). Implicit Feedback for Inferring User Preference: A Bibliography. SIGIR Forum 37(2), 18–28.

    Google Scholar 

  • Kim. J. Oard, D. W. and Romanik, K. (2000). Using Implicit Feedback for User Modeling in Internet and Intranet Searching. University of Maryland CLIS Technical Report 00-01.

    Google Scholar 

  • Kleinberg, J. M. (1999). Authoritative Sources in a Hyperlinked Environment. Journal of the ACM, 46(5), 604–632.

    Article  MATH  MathSciNet  Google Scholar 

  • Koenemann, J. and Belkin, N. J. (1996). A Case for Interaction: A Study of Interactive Information Retrieval Behavior and Effectiveness. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI’ 96), (pp. 205–212), Canada.

    Google Scholar 

  • Kuhlthau, C. C. (1993). Seeking Meaning: A Process Approach to Library and Information Services. Norwood, NJ: Ablex.

    Google Scholar 

  • Lieberman, H. (1995). Letizia: An Agent That assists Web Browsing. In Proceedings of the International Joint Conference on Artificial Intelligence (pp. 924–929), USA.

    Google Scholar 

  • Liu, F., Yu, C., and Meng, W. (2002). Personalized Web Search By Mapping User Queries to Categories. In Proceedings of the Eleventh International Conference on Information and Knowledge Management (CIKM’ 02), (pp. 558–565), USA.

    Google Scholar 

  • Miller, B. N., Riedl, J. T., and Konstan, J. A. (2003). GroupLens for Usenet: Experiences in Applying Collaborative Filtering to a Social Information System. In: C. Lueg and D. Fisher (Eds.), From Usenet to CoWebs: Interacting With Social Information Spaces (pp. 206–231) London: Springer Press.-

    Google Scholar 

  • Mladenic, D. (1999). Machine Learning Used By Personal WebWatcher. Proceedings of the ACAI-99 Workshop on Machine Learning and Intelligent Agents. Crete.

    Google Scholar 

  • Morita, M. and Shinoda, Y. (1994). Information Filtering Based on User Behavior Analysis and Best Match Text Retrieval. In Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’ 94), (pp. 272–281), Ireland.

    Google Scholar 

  • Nichols, D. M. (1997). Implicit Ratings and Filtering. In Proceedings of the 5th DELOS Workshop on Filtering and Collaborative Filtering (pp. 31–36), Hungary.

    Google Scholar 

  • Oard, D. W. and Kim, J. (2001). Modeling Information Content Using Observable Behavior. In Proceedings of the 64th Annual Meeting of the American Society for Information Science and Technology (ASIST’ 01), (pp. 38–45), USA.

    Google Scholar 

  • Page, L. Brin, S. Motwani, R. and Winograd, T. (1998). The PageRank Citation Ranking: Bringing Order to the Web. Stanford University Technical Report.

    Google Scholar 

  • Picard, R. W. and Klein, J. (2002). Computers That Recognize and Respond to User Emotion: Theoretical and Practical Implications. Interacting with Computers, 14(2), 141–169.

    Article  Google Scholar 

  • Price, M. N., Golovchinsky, G., and Schilit, B. N. (1998). Linking By Inking: Trailblazing in a Paper-Like Hypertext. In Proceedings of the 9th ACM Conference on Hypertext and Hypermedia’ 98 (Hypertext’ 98), (pp. 30–39), USA.

    Google Scholar 

  • Quiroga, L. M. and Mostafa, J. (2002). An Experiment in Building Profiles in Information Filtering: The Role of Context of User Relevance Feedback. Information Processing and Management, 38, 671–694.

    Article  MATH  Google Scholar 

  • Rhodes, B. J. and Starner, T. (1996). Remembrance Agent: A Continuously Running Automated Information Retrieval System. In Proceedings of the 1st International Conference on the Practical Application of Intelligent Agents and Multi-Agent Technology (PAAM’ 96), (pp. 487–495).

    Google Scholar 

  • Rhodes, B. J. (2000). Margin Notes: Building a Contextually Aware Associative Memory. In Proceedings of the 5th International Conference on Intelligent User Interfaces (IUI’ 00), (pp. 219–224), USA.

    Google Scholar 

  • Rucker, J. and Polanco, J. (1997). Citeseer: Personalized Navigation for the Web. Communications of the ACM, 40(3), 73–75.

    Article  Google Scholar 

  • Salton, G. (Ed.) (1971). The SMART Retrieval System: Experiments in Automatic Document Processing. NJ: Prentice-Hall.

    Google Scholar 

  • Seo, Y.W. and Zhang, B. T. (2000). A Reinforcement Learning Agent for Personalized Information Filtering. In Proceedings of the 5th International Conference on Intelligent User Interfaces (IUI’ 00), (pp. 248–251), USA.

    Google Scholar 

  • Shen, X. and Zhai, C.-X. (2003). Exploiting Query History for Document Ranking in Interactive Information Retrieval. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’ 03), (pp. 377–378), Canada.

    Google Scholar 

  • Shipman, F. M., Price, M. N., Marshall, C. C., Golovchinsky, G., and Schilit, B. N. (2003). Identifying Useful Passages in Documents Based on Annotation Patterns. In Proceedings of the 2003 European Conference on Digital Libraries (ECDL’ 03), (pp. 101–112).

    Google Scholar 

  • Spink, A. (1996). Multiple Search Session Model of End-User Behavior: An Exploratory Study. Journal of the American Society for Information Science, 47(8), 603–609.

    Article  Google Scholar 

  • Spink, A. and Losee, R. M. (1996). Feedback in Information Retrieval. Annual Review of Information Science and Technology, 31, 33–78.

    Google Scholar 

  • Terveen, L. Hill, W. Amento, B. McDonald, D. and Crete, J. (1997). PHOAKS: A System for Sharing Recommendations. Communications of the ACM, 40(3), 59–62.

    Article  Google Scholar 

  • Vakkari, P. (2000). Cognition and Changes of Search Terms and Tactics During Task Performance: A Longitudinal Case Study. In Proceedings of the RIAO 2000 Conference, 894–907.

    Google Scholar 

  • Vakkari, P. (2003). Task-Based Information Searching. Annual Review of Information Science and Technology, 37, 413–464.

    Article  Google Scholar 

  • White, R.W. Ruthven, I. and Jose, J. M. (2002a). Finding Relevant Documents Using Top Ranking Sentences: An Evaluation of Two Alternative Schemes. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’ 02), (pp. 57–64), Finland.

    Google Scholar 

  • White, R. W. Jose, J. M. and Ruthven, I. (2003). An Approach for Implicitly Detecting Information Needs. In Proceedings of the 12th Annual ACM Conference on Information and Knowledge Management (CIKM’03), (pp. 504–507), New Orleans, LA.

    Google Scholar 

  • White, R.W. and Jose, J. M. (2004). An Implicit System for Predicting Interests. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’04), (p. 595), Sheffield, UK.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of North Carolina at Chapel Hill, USA

    Diane Kelly

Authors
  1. Diane Kelly
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculty of Information Technology, Queensland University of Technology, Brisbane, Australia

    Amanda Spink

  2. Graduate School of Library and Information Studies, McGill University, Montreal, Quebec, Canada

    Charles Cole

Rights and permissions

Reprints and permissions

Copyright information

© 2005 Springer

About this chapter

Cite this chapter

Kelly, D. (2005). Implicit Feedback: Using Behavior to Infer Relevance. In: Spink, A., Cole, C. (eds) New Directions in Cognitive Information Retrieval. The Information Retrieval Series, vol 19. Springer, Dordrecht . https://doi.org/10.1007/1-4020-4014-8_9

Download citation

  • DOI: https://doi.org/10.1007/1-4020-4014-8_9

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-1-4020-4013-9

  • Online ISBN: 978-1-4020-4014-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation