Towards Evaluation of Peer-to-Peer-Based Distributed Knowledge Management Systems

  • Marc Ehrig
  • Christoph Schmitz
  • Steffen Staab
  • Julien Tane
  • Christoph Tempich
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2926)


Distributed knowledge management systems (DKMS) have been suggested to meet the requirements of today’s knowledge management. Peer-to-peer systems offer technical foundations for such distributed systems. To estimate the value of P2P-based knowledge management evaluation criteria that measure the performance of such DKMS are required. We suggest a concise framework for evaluation of such systems within different usage scenarios. Our approach is based on standard measures from the information retrieval and the databases community. These measures serve as input to a general evaluation function which is used to measure the efficiency of P2P-based KM systems. We describe test scenarios as well as the simulation software and data sets that can be used for that purpose.


  1. 1.
    Bonifacio, M., Bouquet, P., Traverso, P.: Enabling distributed knowledge management: Managerial and technological implications. Novatica and Informatik/Informatique III (2002)Google Scholar
  2. 2.
    OLeary, D.: Using ai in knowledge management: Knowledge bases and ontologies. IEEE Intelligent Systems 13, 34–39 (1998)Google Scholar
  3. 3.
    Gruber, T.R.: Towards Principles for the Design of Ontologies Used for Knowledge Sharing. In: Guarino, N., Poli, R. (eds.) Formal Ontology in Conceptual Analysis and Knowledge Representation, Deventer, The Netherlands, Kluwer Academic Publishers, Dordrecht (1993)Google Scholar
  4. 4.
    Maedche, A.: Emergent semantics for ontologies. IEEE Intelligent Systems, Los Alamitos (2002)Google Scholar
  5. 5.
    Moreale, E., Watt, S.: An agent-based approach to mailing list knowledge management. In: [36] (2003) (to appear)Google Scholar
  6. 6.
    Voorhees, E.M.: The philosophy of information retrieval evaluation. In: Peters, C., Braschler, M., Gonzalo, J., Kluck, M. (eds.) CLEF 2001. LNCS, vol. 2406, pp. 9–26. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  7. 7.
    Yang, B., Garcia-Molina, H.: Efficient search in peer-to-peer networks. In: Proceedings of the International Conference on Distributed Computing Systems, ICDCS (2002)Google Scholar
  8. 8.
    Kan, G.: Gnutella. In: Peer-to-Peer: Harnessing the Power of Disruptive Technologies, pp. 94–122. O’Reilly, Sebastopol (1999)Google Scholar
  9. 9.
    Bonifacio, M., Bouquet, P., Mameli, G., Nori, M.: Peer-mediated distributed knowldege management.In: [36] (2003) (to appear) Google Scholar
  10. 10.
    Wache, H., Voegele, T., Visser, T., Stuckenschmidt, H., Schuster, H., Neumann, G., Huebner, S.: Ontology-based integration of information - a survey of existing approaches. In: Stuckenschmidt, H. (ed.) IJCAI 2001 Workshop: Ontologies and Information Sharing, pp. 108–117 (2001)Google Scholar
  11. 11.
    Schlosser, M., Sintek, M., Decker, S., Nejdl, W.: HyperCuP - hypercubes, ontologies and efficient search on P2P networks. In: Proceedings to the Eleventh International World Wide Web Conference, Honolulu, Hawaii, USA (2002)Google Scholar
  12. 12.
    Carzaniga, A., Wolf, A.L.: Content-based networking: A new communication infrastructure. In: König-Ries, B., Makki, K., Makki, S.A.M., Pissinou, N., Scheuermann, P. (eds.) IMWS 2001. LNCS, vol. 2538, pp. 59–68. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  13. 13.
    Van Rijsbergen, C.J.: Information Retrieval, 2nd edn. Dept. of Computer Science, University of Glasgow (1979)Google Scholar
  14. 14.
    Mena, E., Kashyap, V., Illarramendi, A., Sheth, A.P.: Imprecise answers in distributed environments: Estimation of information loss for multi-ontology based query processing. International Journal of Cooperative Information Systems 9, 403–425 (2000)CrossRefGoogle Scholar
  15. 15.
    Nodine, M., Bohrer, W., Ngu, A.H.H.: Semantic brokering over dynamic heterogeneous data sources in infosleuth. Technical report, MCC (1998) Google Scholar
  16. 16.
    Yang, B., Garcia-Molina, H.: Designing a super-peer network. Technical report, Stanford (2002) Google Scholar
  17. 17.
    Yang, B., Garcia-Molina, H.: Improvig search in peer-to-peer networks. Technical report, Stanford University (2002) Google Scholar
  18. 18.
    Clarke, I., Sandberg, O., Wiley, B., Hong, T.W.: Freenet: A distributed anonymous information storage and retrieval system. In: Federrath, H. (ed.) Designing Privacy Enhancing Technologies. LNCS, vol. 2009, p. 46. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  19. 19.
    Montresor, A., Meling, H., Babaoglu, O.: Towards adaptive, resilient and self-organizing peer-to-peer systems. In: Proceedings of the International Workshop on Peer-to-Peer Computing, Pisa, Italy (2002)Google Scholar
  20. 20.
    Ball, P.: Introduction to discrete event simulation. In: Proceedings of the 2nd DYCOMANS workshop on Management and Control: Tools in Action (1996)Google Scholar
  21. 21.
    Pease, A., Li, J.: Agent-mediated knowledge engineering collaboration. In: [36] (2003) (to appear)Google Scholar
  22. 22.
    Magnini, B., Serafini, L., Speranza, M.: Linguistic based matching of local ontologies. In: Workshop on Meaning Negotiation (MeaN 2002), Edmonton, Alberta, Canada (2002)Google Scholar
  23. 23.
    Modica, G., Gal, A., Jamil, H.M.: The use of machine-generated ontologies in dynamic information seeking. In: Batini, C., Giunchiglia, F., Giorgini, P., Mecella, M. (eds.) CoopIS 2001. LNCS, vol. 2172, p. 433. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  24. 24.
    Melnik, S., Garcia-Molina, H., Rahm, E.: Similarity flooding: A versatile graph matching algorithm. In: Proc. 18th ICDE Conf. (2001)Google Scholar
  25. 25.
    Steels, L.: The origins of ontologies and communication conventions in multi-agent systems. Autonomous Agents and Multi-Agent Systems 1, 169–194 (1998)CrossRefGoogle Scholar
  26. 26.
    Novak, J., Wurst, M., Fleischmann, M., Strauss, W.: Discovering, visualizing and sharing knowledge throught personalized learning knowledge maps. In: [36] (2003) (to appear)Google Scholar
  27. 27.
    Ng, C.H., Sia, K.C.: Peer clustering and firework query model. Technical report, The Chinese University of Hong Kong (2002) Google Scholar
  28. 28.
    Callan, J.P., Connell, M.E.: Query-based sampling of text databases. Information Systems 19, 97–130 (2001)Google Scholar
  29. 29.
    Zipf, G.K.: Human Behavior and the Principle of Least Effort: An Introduction to Human Ecology. Addison-Wesley, Reading (1949)Google Scholar
  30. 30.
    Sebastiani, F.: Machine learning in automated text categorization. ACM Computing Surveys 34, 1–47 (2002)CrossRefMathSciNetGoogle Scholar
  31. 31.
    Faloutsos, M., Faloutsos, P., Faloutsos, C.: On power-law relationships of the internet topology. In: SIGCOMM, pp. 251–262 (1999)Google Scholar
  32. 32.
    Gravano, L., García-Molina, H.: Generalizing GlOSS to vector-space databases and broker hierarchies. In: International Conference on Very Large Databases, VLDB, pp. 78–89 (1995)Google Scholar
  33. 33.
    Iosif, V., Sure, Y.: Exploring potential benefits of the semantic web for virtual organizations. In: Karagiannis, D., Reimer, U. (eds.) PAKM 2002. LNCS (LNAI), vol. 2569, Springer, Heidelberg (2002) (to appear)Google Scholar
  34. 34.
    Stoica, I., Morris, R., Karger, D., Kaashoek, F., Balakrishnan, H.: A scalable Peer- To-Peer lookup service for internet applications. In: Proceedings of the 2001 ACM SIGCOMM Conference, pp. 149–160 (2001)Google Scholar
  35. 35.
    Dieng, R., Corby, O., Giboin, A., Ribiere, M.: Methods and tools for corporate knowledge management. Int. Journal of Human-Computer Studies 51, 567–598 (1999)CrossRefGoogle Scholar
  36. 36.
    van Elst, L., Dignum, V., Abecker, A. (eds.): AMKM 2003. LNCS (LNAI), vol. 2926. Springer, Heidelberg (2003) (to appear)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Marc Ehrig
    • 1
  • Christoph Schmitz
    • 1
  • Steffen Staab
    • 1
  • Julien Tane
    • 1
  • Christoph Tempich
    • 1
  1. 1.Institute AIFBUniversity of KarlsruheKarlsruheGermany

Personalised recommendations