Chapter

Advances in Information Retrieval

Volume 6611 of the series Lecture Notes in Computer Science pp 350-361

Exploiting Thread Structures to Improve Smoothing of Language Models for Forum Post Retrieval

  • Huizhong DuanAffiliated withUniversity of Illinois
  • , Chengxiang ZhaiAffiliated withUniversity of Illinois

* Final gross prices may vary according to local VAT.

Get Access

Abstract

Due to many unique characteristics of forum data, forum post retrieval is different from traditional document retrieval and web search, raising interesting research questions about how to optimize the accuracy of forum post retrieval. In this paper, we study how to exploit the naturally available raw thread structures of forums to improve retrieval accuracy in the language modeling framework. Specifically, we propose and study two different schemes for smoothing the language model of a forum post based on the thread containing the post. We explore several different variants of the two schemes to exploit thread structures in different ways. We also create a human annotated test data set for forum post retrieval and evaluate the proposed smoothing methods using this data set. The experiment results show that the proposed methods for leveraging forum threads to improve estimation of document language models are effective, and they outperform the existing smoothing methods for the forum post retrieval task.

Keywords

Forum post retrieval language modeling smoothing