Text Content Reliability Estimation in Web Documents: A New Proposal

  • Luis Sanz
  • Héctor Allende
  • Marcelo Mendoza
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7182)


This paper illustrates how a combination of information retrieval, machine learning, and NLP corpus annotation techniques was applied to a problem of text content reliability estimation in Web documents. Our proposal for text content reliability estimation is based on a model in which reliability is a similarity measure between the content of the documents and a knowledge corpus. The proposal includes a new representation of text which uses entailment-based graphs. Then we use the graph-based representations as training instances for a machine learning algorithm allowing to build a reliability model. Experimental results illustrate the feasibility of our proposal by performing a comparison with a state-of-the-art method.


Text reliability content-based trust textual entailment 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Al-Eidan, R.M.B., Al-Khalifa, H.S., Al-Salman, A.S.: Towards the measurement of arabic weblogs credibility automatically. In: Proceedings of the 11th iiWAS Conference, pp. 618–622. ACM, New York (2009)Google Scholar
  2. 2.
    Cusinato, A., Della Mea, V., Di Salvatore, F., Mizzaro, S.: Quwi: quality control in wikipedia. In: Proceedings of the 3rd Workshop on Information Credibility on the Web, WICOW 2009, pp. 27–34. ACM, New York (2009)CrossRefGoogle Scholar
  3. 3.
    Dang, H.T.: Overview of DUC 2005. In: Proceedings of the 2005 Document Understanding Workshop, Vancouver, B.C., Canada (2005)Google Scholar
  4. 4.
    Dang, H.T., Owczarzak, K.: Overview of the tac 2008 update summarization task. In: Proceedings of the First Text Analysis Conference (TAC 2008), Gaithersburg, Maryland, USA, pp. 1–16 (2008)Google Scholar
  5. 5.
    Hickl, A., Bensley, J.: A discourse commitment-based framework for recognizing textual entailment. In: Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pp. 171–176. ACL, Prague (2007)Google Scholar
  6. 6.
    Hovy, E., Lin, C.Y., Zhou, L.: Evaluating duc 2005 using basic elements. In: Proceedings of DUC 2005, Vancouver, B.C., Canada, pp. 1–6 (2005)Google Scholar
  7. 7.
    Juffinger, A., Granitzer, M., Lex, E.: Blog credibility ranking by exploiting verified content. In: Proceedings of the 3rd Workshop on Information Credibility on the Web, WICOW 2009, pp. 51–58. ACM, New York (2009)CrossRefGoogle Scholar
  8. 8.
    Kolluru, B., Gotoh, Y.: On the subjectivity of human authored short summaries. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation, pp. 12–18. ACL, Michigan (2005)Google Scholar
  9. 9.
    Lin, C.Y.: Rouge: A package for automatic evaluation of summaries. In: Proc. ACL workshop on Text Summarization Branches Out, Barcelona, Spain, pp. 9–17 (2004)Google Scholar
  10. 10.
    Metzger, M.J.: Making sense of credibility on the web: Models for evaluating online information and recommendations for future research. Journal of the American Society for Information Science and Technology 58(13), 2078–2091 (2007)CrossRefGoogle Scholar
  11. 11.
    Metzger, M.J., Flanagin, A.J., Medders, R.B.: Social and heuristic approaches to credibility evaluation online. Journal of Communication 60, 413–439 (2010)CrossRefGoogle Scholar
  12. 12.
    Nenkova, A., Passonneau, R.: Evaluating content selection in summarization: The pyramid method. In: Proceedings of the HLT-NAACL Conference, p. 8. Association for Computational Linguistics, New York (2004)Google Scholar
  13. 13.
    Nichols, E., Murakami, K., Inui, K., Matsumoto, Y.: Constructing a scientific blog corpus for information credibility analysis. In: Proceedings of the PACLING Conference, pp. 1–6. ACM, Sapporo (2009)Google Scholar
  14. 14.
    Rieh, S.Y.: Judgment of information quality and cognitive authority in the Web. J. Am. Soc. Inf. Sci. 53(2), 145–161 (2002)CrossRefGoogle Scholar
  15. 15.
    Negri, M., Kouylekov, M., Magnini, B., Mehdad, Y., Cabrio, E.: Towards Extensible Textual Entailment Engines: The EDITS Package. In: Proceedings of AIIA, Emergent Perspectives in Artificial Intelligence, Reggio Emilia, Italy (2009)Google Scholar
  16. 16.
    Vega, L.C., Sun, Y.T., McCrickard, D.S., Harrison, S.: Time: a method of detecting the dynamic variances of trust. In: Proceedings of the 4th Workshop on Information Credibility, WICOW 2010, pp. 43–50. ACM, New York (2010)CrossRefGoogle Scholar
  17. 17.
    Weerkamp, W., Rijke, M.D.: Credibility improves topical blog post retrieval. In: ACL 2008: HLT, pp. 923–931. ACL, Columbus (2008)Google Scholar
  18. 18.
    Xu, Y.C., Chen, Z.: Relevance judgment: What do information users consider beyond topicality? J. Am. Soc. Inf. Sci. Technol. 57(7), 961–973 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Luis Sanz
    • 1
  • Héctor Allende
    • 1
  • Marcelo Mendoza
    • 1
  1. 1.Department of InformaticsUniversidad Técnica Federico Santa MaríaChile

Personalised recommendations