Electronic Commerce Research

, Volume 10, Issue 3–4, pp 271–290 | Cite as

TREET: the Trust and Reputation Experimentation and Evaluation Testbed



To date, trust and reputation systems have often been evaluated using methods of their designers’ own devising. Recently, we demonstrated that a number of noteworthy trust and reputation systems could be readily defeated, revealing limitations in their original evaluations. Efforts in the trust and reputation community to develop a testbed have yielded a successful competition platform, ART. This testbed, however, is less suited to general experimentation and evaluation of individual trust and reputation technologies. In this paper, we present TREET, an experimentation and evaluation testbed based directly on that used in our investigations into security vulnerabilities in trust and reputation systems for marketplaces. We demonstrate the advantages of TREET, towards the development of more thorough, objective evaluations of trust and reputation systems.


Trust and reputation Multiagent systems Electronic marketplaces Simulation Evaluation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fullam, K. K., Klos, T. B., Muller, G., Sabater, J., Schlosser, A., Topol, Z., Barber, K. S., Rosenschein, J. S., Vercouter, L., & Voss, M. (2005). A specification of the Agent Reputation and Trust (ART) testbed: experimentation and competition for trust in agent societies. In AAMAS ’05: Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems (pp. 512–518). New York: ACM. CrossRefGoogle Scholar
  2. 2.
    Teacy, W. T. L., Huynh, T. D., Dash, R. K., Jennings, N. R., Luck, M., & Patel, J. (2007). The ART of IAM: The winning strategy for the 2006 competition. In: Proceedings of the sixth international joint conference on autonomous agents and multiagent systems (AAMAS’07) workshop on trust in agent societies, Honolulu, Hawaii, USA. Google Scholar
  3. 3.
    Teacy, W. T., Patel, J., Jennings, N. R., & Luck, M. (2006). Travos: trust and reputation in the context of inaccurate information sources. Autonomous Agents and Multi-Agent Systems, 12(2), 183–198. CrossRefGoogle Scholar
  4. 4.
    Yu, B., & Singh, M. P. (2002). Distributed reputation management for electronic commerce. Computational Intelligence, 18(4), 535–549. CrossRefGoogle Scholar
  5. 5.
    Kerr, R., & Cohen, R. (2006). Modeling trust using transactional, numerical units. In: PST ’06: proceedings of the conference on privacy, security and trust, Markham, Canada. Google Scholar
  6. 6.
    Kerr, R., & Cohen, R. (2009). Smart cheaters do prosper: defeating trust and reputation systems. In: Proceedings of the eighth international joint conference on autonomous agents and multiagent systems (AAMAS’09), Budapest, Hungary. Google Scholar
  7. 7.
    Kerr, R., & Cohen, R. (2009). An experimental testbed for evaluation of trust and reputation systems. In: Ferrari, E., Li, N., Bertino, E., Karabulut, Y. (Eds.), Trust management III. Advances in information and communication technology, vol. 300 (pp. 252–266). Boston: Springer. CrossRefGoogle Scholar
  8. 8.
    Jøsang, A., & Ismail, R. (2002). The beta reputation system. In: Proceedings of the 15th bled electronic commerce conference e-reality: constructing the e-economy, June 2002. Google Scholar
  9. 9.
    Tran, T., & Cohen, R. (2004). Improving user satisfaction in agent-based electronic marketplaces by reputation modelling and adjustable product quality. In AAMAS ’04: proceedings of the third international joint conference on autonomous agents and multiagent systems (pp. 828–835). Washington: IEEE Computer Society. Google Scholar
  10. 10.
    Hang, C. W., Wang, Y., & Singh, M. P. (2008). An adaptive probabilistic trust model and its evaluation. In AAMAS ’08: Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems (pp. 1485–1488). Richland: International Foundation for Autonomous Agents and Multiagent Systems. Google Scholar
  11. 11.
    Harbers, M., Verbrugge, R., Sierra, C., & Debenham, J. (2008). The examination of an information-based approach to trust. In Noriega, P., Padget, J. (Eds.), International workshop on coordination, organization, institutions and norms (COIN) (pp. 101–112). Durham: Durham University. Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.David R. Cheriton School of Computer ScienceUniversity of WaterlooWaterlooCanada

Personalised recommendations