Abstract
To date, trust and reputation systems have often been evaluated using methods of their designers’ own devising. Recently, we demonstrated that a number of noteworthy trust and reputation systems could be readily defeated, revealing limitations in their original evaluations. Efforts in the trust and reputation community to develop a testbed have yielded a successful competition platform, ART. This testbed, however, is less suited to general experimentation and evaluation of individual trust and reputation technologies. In this paper, we propose an experimentation and evaluation testbed based directly on that used in our investigations into security vulnerabilities in trust and reputation systems for marketplaces. We demonstrate the advantages of this design, towards the development of more thorough, objective evaluations of trust and reputation systems.
Chapter PDF
References
Fullam, K.K., Klos, T.B., Muller, G., Sabater, J., Schlosser, A., Topol, Z., Barber, K.S., Rosenschein, J.S., Vercouter, L., Voss, M.: A specification of the Agent Reputation and Trust (ART) testbed: experimentation and competition for trust in agent societies. In: AAMAS 2005: Proceedings of the fourth international joint conference on Autonomous Agents and Multiagent Systems, pp. 512–518. ACM, New York (2005)
Teacy, W.T.L., Huynh, T.D., Dash, R.K., Jennings, N.R., Luck, M., Patel, J.: The ART of IAM: The winning strategy for the 2006 competition. In: Proceedings of the Sixth International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2007) Workshop on Trust in Agent Societies, Honolulu, Hawaii, USA (2007)
Teacy, W.T., Patel, J., Jennings, N.R., Luck, M.: Travos: Trust and reputation in the context of inaccurate information sources. Autonomous Agents and Multi-Agent Systems 12(2), 183–198 (2006)
Yu, B., Singh, M.P.: Distributed reputation management for electronic commerce. Computational Intelligence 18(4), 535–549 (2002)
Kerr, R., Cohen, R.: Modeling trust using transactional, numerical units. In: PST 2006: Proceedings of the Conference on Privacy, Security and Trust, Markham, Canada (2006)
Kerr, R., Cohen, R.: Smart cheaters do prosper: Defeating trust and reputation systems. In: Proceedings of the Eighth International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2009), Budapest, Hungary (2009)
Jøsang, A., Ismail, R.: The beta reputation system. In: Proceedings of the 15th Bled Electronic Commerce Conference e-Reality: Constructing the e-Economy (June 2002)
Tran, T., Cohen, R.: Improving user satisfaction in agent-based electronic marketplaces by reputation modelling and adjustable product quality. In: AAMAS 2004: Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, Washington, DC, USA, pp. 828–835. IEEE Computer Society, Los Alamitos (2005)
Hang, C.W., Wang, Y., Singh, M.P.: An adaptive probabilistic trust model and its evaluation. In: AAMAS 2008: Proceedings of the 7th international joint conference on Autonomous Agents and Multiagent Systems, Richland, SC, International Foundation for Autonomous Agents and Multiagent Systems, pp. 1485–1488 (2008)
Harbers, M., Verbrugge, R., Sierra, C., Debenham, J.: The examination of an information-based approach to trust. In: Noriega, P., Padget, J. (eds.) International Workshop on Coordination, Organization, Institutions and Norms (COIN), pp. 101–112. Durham University, Durham (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 IFIP International Federation for Information Processing
About this paper
Cite this paper
Kerr, R., Cohen, R. (2009). An Experimental Testbed for Evaluation of Trust and Reputation Systems. In: Ferrari, E., Li, N., Bertino, E., Karabulut, Y. (eds) Trust Management III. IFIPTM 2009. IFIP Advances in Information and Communication Technology, vol 300. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02056-8_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-02056-8_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02055-1
Online ISBN: 978-3-642-02056-8
eBook Packages: Computer ScienceComputer Science (R0)