A Novel Distributed Collaborative Filtering Algorithm and Its Implementation on P2P Overlay Network
Collaborative filtering (CF) has proved to be one of the most effective information filtering techniques. However, as their calculation complexity increased quickly both in time and space when the record in user database increases, traditional centralized CF algorithms has suffered from their shortage in scalability. In this paper, we first propose a novel distributed CF algorithm called PipeCF through which we can do both the user database management and prediction task in a decentralized way. We then propose two novel approaches: significance refinement (SR) and unanimous amplification (UA), to further improve the scalability and prediction accuracy of PipeCF. Finally we give the algorithm framework and system architecture of the implementation of PipeCF on Peer-to-Peer (P2P) overlay network through distributed hash table (DHT) method, which is one of the most popular and effective routing algorithm in P2P. The experimental data show that our distributed CF algorithm has much better scalability than traditional centralized ones with comparable prediction efficiency and accuracy.
Unable to display preview. Download preview PDF.
- 2.Herlocker, J.L., Konstan, J.A., Borchers, A., Riedl, J.: An algorithmic framework for performing collaborative filtering. In: Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, pp. 230–237 (1999)Google Scholar
- 3.Breese, J., Heckerman, D., Kadie, C.: Empirical Analysis of Predictive Algorithms for Collaborative Filtering. In: Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence, pp. 43–52 (1998)Google Scholar
- 4.Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., Riedl, J.: GroupLens: an open architecture for collaborative filtering of netnews. In: Proceedings of the 1994 ACM conference on Computer supported cooperative work, October 22-26, pp. 175–186. Chapel Hill, North Carolina (1994)CrossRefGoogle Scholar
- 5.Shardanand, U., Maes, P.: Social information filtering: algorithms for automating “word of mouth”. In: Proceedings of the SIGCHI conference on Human factors in computing systems, Denver, Colorado, United State, May 7-11, pp. 210–217 (1995)Google Scholar
- 6.Eachmovie collaborative filtering data set, http://research.compaq.com/SRC/eachmovie
- 8.Olsson, T.: Bootstrapping and Decentralizing Recommender Systems, Licentiate Thesis 2003-006, Department of Information Technology, Uppsala University and SICS (2003)Google Scholar
- 9.Canny, J.: Collaborative filtering with privacy. In: Proceedings of the IEEE Symposium on Research in Security and Privacy, Oakland, CA, May 2002, pp. 45–57 (2002); IEEE Computer Society, Technical Committee on Security and Privacy, IEEE Computer Society PressGoogle Scholar
- 10.Ratnasamy, S., Francis, P., Handley, M., Karp, R., Shenker, S.: A scalable contentaddressable network. In: SIGCOMM (August 2001)Google Scholar
- 11.Stocal, I., et al.: Chord: A scalable peer-to-peer lookup service for Internet applications. In: ACM SIGCOMM, San Diego, CA, USA, pp. 149–160 (2001)Google Scholar
- 12.Rowstron, A., Druschel, P.: Pastry: Scalable, distributed object location and routing for large scale peer-to-peer systems. In: IFIP/ACM Middleware, Hedelberg, Germany (2001)Google Scholar
- 13.Zhao, B.Y., et al.: Tapestry: An infrastructure for fault-tolerant wide-area location and routing. Tech. Rep. UCB/CSB-0-114, UC Berkeley, EECS 2001 (2001)Google Scholar