TREET: the Trust and Reputation Experimentation and Evaluation Testbed
To date, trust and reputation systems have often been evaluated using methods of their designers’ own devising. Recently, we demonstrated that a number of noteworthy trust and reputation systems could be readily defeated, revealing limitations in their original evaluations. Efforts in the trust and reputation community to develop a testbed have yielded a successful competition platform, ART. This testbed, however, is less suited to general experimentation and evaluation of individual trust and reputation technologies. In this paper, we present TREET, an experimentation and evaluation testbed based directly on that used in our investigations into security vulnerabilities in trust and reputation systems for marketplaces. We demonstrate the advantages of TREET, towards the development of more thorough, objective evaluations of trust and reputation systems.
KeywordsTrust and reputation Multiagent systems Electronic marketplaces Simulation Evaluation
Unable to display preview. Download preview PDF.
- 1.Fullam, K. K., Klos, T. B., Muller, G., Sabater, J., Schlosser, A., Topol, Z., Barber, K. S., Rosenschein, J. S., Vercouter, L., & Voss, M. (2005). A specification of the Agent Reputation and Trust (ART) testbed: experimentation and competition for trust in agent societies. In AAMAS ’05: Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems (pp. 512–518). New York: ACM. CrossRefGoogle Scholar
- 2.Teacy, W. T. L., Huynh, T. D., Dash, R. K., Jennings, N. R., Luck, M., & Patel, J. (2007). The ART of IAM: The winning strategy for the 2006 competition. In: Proceedings of the sixth international joint conference on autonomous agents and multiagent systems (AAMAS’07) workshop on trust in agent societies, Honolulu, Hawaii, USA. Google Scholar
- 5.Kerr, R., & Cohen, R. (2006). Modeling trust using transactional, numerical units. In: PST ’06: proceedings of the conference on privacy, security and trust, Markham, Canada. Google Scholar
- 6.Kerr, R., & Cohen, R. (2009). Smart cheaters do prosper: defeating trust and reputation systems. In: Proceedings of the eighth international joint conference on autonomous agents and multiagent systems (AAMAS’09), Budapest, Hungary. Google Scholar
- 7.Kerr, R., & Cohen, R. (2009). An experimental testbed for evaluation of trust and reputation systems. In: Ferrari, E., Li, N., Bertino, E., Karabulut, Y. (Eds.), Trust management III. Advances in information and communication technology, vol. 300 (pp. 252–266). Boston: Springer. CrossRefGoogle Scholar
- 8.Jøsang, A., & Ismail, R. (2002). The beta reputation system. In: Proceedings of the 15th bled electronic commerce conference e-reality: constructing the e-economy, June 2002. Google Scholar
- 9.Tran, T., & Cohen, R. (2004). Improving user satisfaction in agent-based electronic marketplaces by reputation modelling and adjustable product quality. In AAMAS ’04: proceedings of the third international joint conference on autonomous agents and multiagent systems (pp. 828–835). Washington: IEEE Computer Society. Google Scholar
- 10.Hang, C. W., Wang, Y., & Singh, M. P. (2008). An adaptive probabilistic trust model and its evaluation. In AAMAS ’08: Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems (pp. 1485–1488). Richland: International Foundation for Autonomous Agents and Multiagent Systems. Google Scholar
- 11.Harbers, M., Verbrugge, R., Sierra, C., & Debenham, J. (2008). The examination of an information-based approach to trust. In Noriega, P., Padget, J. (Eds.), International workshop on coordination, organization, institutions and norms (COIN) (pp. 101–112). Durham: Durham University. Google Scholar