Abstract
Effectively summarizing Web page collections becomes more and more critical as the amount of information continues to grow on the World Wide Web. A concise and meaningful summary of a Web page collection, which is generated automatically, can help Web users understand the essential topics and main contents covered in the collection quickly without spending much browsing time. However, automatically generating coherent summaries as good as human-authored summaries is a challenging task since Web page collections often contain diverse topics and contents. This research aims towards clustering of Web page collections using automatically extracted topical terms, and automatic summarization of the resulting clusters. We experiment with word- and term-based representations of Web documents and demonstrate that term-based clustering significantly outperforms word-based clustering with much lower dimensionality. The summaries of computed clusters are informative and meaningful, which indicates that clustering and summarization of large Web page collections is promising for alleviating the information overload problem.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bilmes, J.: A Gentle Tutorial on the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models. Technical Report ICSI-TR-97-021, University of California, Berkeley (1997)
Boley, D.: Principal Direction Divisive Partitioning. Data Mining and Knowledge Discovery 2(4), 325–344 (1998)
Brill, E.: A Simple Rule-based Part of Speech Tagger. In: Proceedings of the 3rd Conference on Applied Natural Language Processing (ANLP 1992), pp. 152–155 (1992)
Broder, A., Glassman, S., Manasse, M., Zweig, G.: Syntactic Clustering of the Web. In: Proceedings of the 6th International World Wide Web Conference, pp. 391–404 (1997)
Cutting, D., Karger, D., Pedersen, J., Tukey, J.: Scatter/Gather: A Clusterbased Approach to Browsing Large Document Collections. In: Proceedings of the 15th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 318–329 (1992)
Dean, J., Henzinger, M.: Finding Related Pages in the World Wide Web. In: Proceedings of the 8th International World Wide Web Conference, pp. 389–401 (1990)
Dempster, A., Laird, N., Rubin, D.: Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, Series B 39(1), 1–38 (1977)
Frakes, W., Baeza-Yates, R.: Information Retrieval: Data Structures and Algorithms. Prentice Hall, Englewood Cliffs (1992)
Frantzi, K., Ananiadou, S., Mima, H.: Automatic Recognition of Multiword Terms. International Journal of Digital Libraries 3(2), 117–132 (2000)
Goldstein, J., Mittal, V., Carbonell, J., Callan, J.: Creating and Evaluating Multi-document Sentence Extract Summaries. In: Proceedings of the 9th ACM International Conference on Information and Knowledge Management (CIKM 2000), pp. 165–172 (2000)
Han, E., Karypis, G., Kumar, V., Mobasher, B.: Clustering in a Highdimensional Space Using Hypergraph Models. Technical Report TR-97-063, Department of Computer Science and Engineering/Army HPC Research Center, University of Minnesota (1997)
Hotho, A., Maedche, A., Staab, S.: Ontology-based Text Clustering. In: Proceedings of the IJCAI Workshop on “Text Learning: Beyond Supervision” (2001)
Milios, E., Zhang, Y., He, B., Dong, L.: Automatic Term Extraction and Document Similarity in Special Text Corpora. In: Proceedings of the 6th Conference of the Pacific Association for Computational Linguistics (PACLing 2003), pp. 275– 284 (2003)
Modha, D., Spangler, W.: Clustering Hypertext with Applications to Web Searching. In: Proceedings of the 11th ACM Conference on Hypertext and Hypermedia, pp. 143–152 (2001)
Schutze, H., Silverstein, H.: Projections for Efficient Document Clustering. In: Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1997), pp. 74–81 (1997)
Sebastiani, F.: Machine Learning in Automated Text Categorization. ACM Computing Surveys 34(1), 1–47 (2002)
Sinka, M., Corne, D.: Towards Modernized and Web-Specific Stoplists for Web Document Analysis. In: Proceedings of IEEE/WIC International Conference on Web Intelligence (WI 2003), pp. 396–402 (2003)
Steinbach, M., Karypis, G., Kumar, V.: A Comparison of Document Clustering Techniques. In: Proceedings of KDD Workshop on Text Mining (2000)
Turney, P.: Coherent Keyphrase Extraction via Web Mining. In: Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI 2003), pp. 434–439 (2003)
Wang, Y., Kitsuregawa, M.: Use Link-based Clustering to Improve Web Search Results. In: Proceedings of the 2nd International Conference on Web Information Systems Engineering (WISE 2001), pp. 115–124 (2001)
Wang, Y., Kitsuregawa, M.: Evaluating Contents-link Coupled Web Page Clustering forWeb Search Results. In: Proceedings of the 11th ACM International Conference on Information and Knowledge Management (CIKM 2002), pp. 499–506 (2002)
Zhang, Y., Zincir-Heywood, N., Milios, E.: Summarizing Web Sites Automatically. In: Proceedings of the 16th Conference of the Canadian Society for Computational Studies of Intelligence (AI 2003), pp. 283–296 (2003)
Zhao, Y., Karypis, G.: Evaluation of Hierarchical Clustering Algorithms for Document Datasets. In: Proceedings of the 11th ACM International Conference on Information and Knowledge Management (CIKM 2002), pp. 515–524 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, Y., Zincir-Heywood, N., Milios, E. (2004). Term-Based Clustering and Summarization of Web Page Collections. In: Tawfik, A.Y., Goodwin, S.D. (eds) Advances in Artificial Intelligence. Canadian AI 2004. Lecture Notes in Computer Science(), vol 3060. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24840-8_5
Download citation
DOI: https://doi.org/10.1007/978-3-540-24840-8_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22004-6
Online ISBN: 978-3-540-24840-8
eBook Packages: Springer Book Archive