Skip to main content
SpringerLink
Log in

How to trust a few among many

Autonomous Agents and Multi-Agent Systems volume 31, pages 531–560 (2017)Cite this article

Abstract

The presence of numerous and disparate information sources available to support decision-making calls for efficient methods of harnessing their potential. Information sources may be unreliable, and misleading reports can affect decisions. Existing trust and reputation mechanisms typically rely on reports from as many sources as possible to mitigate the influence of misleading reports on decisions. In the real world, however, it is often the case that querying information sources can be costly in terms of energy, bandwidth, delay overheads, and other constraints. We present a model of source selection and fusion in resource-constrained environments, where there is uncertainty regarding the trustworthiness of sources. We exploit diversity among sources to stratify them into homogeneous subgroups to both minimise redundant sampling and mitigate the effect of certain biases. Through controlled experiments, we demonstrate that a diversity-based approach is robust to biases introduced due to dependencies among source reports, performs significantly better than existing approaches when sampling budget is limited and equally as good with an unlimited budget.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Notes

  1. See: http://www.bbc.co.uk/news/uk-14490693 for how Twitter was used to spread false rumours during the England riots of 2011.

  2. This is consistent with most real-world economic settings [11].

  3. We use the M5 implementation of Weka [14], a popular open-source machine learning toolkit written in Java

  4. The M5 algorithm can accommodate other feature types including qualitative. As well as this, different metrics exist for computing the distance between features of other kinds [17].

References

  1. Antos, A., Grover, V., & Szepesvári, C. (2008). Active learning in multi-armed bandits. In Proceedings of the 19th International Conference on Algorithmic Learning Theory (pp. 287–302).

  2. Antos, A., Grover, V., & Szepesvári, C. (2010). Active learning in heteroscedastic noise. Theoretical Computer Science, 411(29), 2712–2728.

    Article  MathSciNet  MATH  Google Scholar 

  3. Breiman, L., Friedman, J., Stone, C., & Olshen, R. (1984). Classification and Regression Trees. Belmont, CA: Wadsworth.

    MATH  Google Scholar 

  4. Burke, J., Estrin, D., Hansen, M., Parker, A., Ramanathan, N., Reddy, S., & Srivastava, M. B. (2006). Participatory sensing. In ACM Sensys World Sensor Web Workshop.

  5. Burnett, C., Norman, T. J., & Sycara, K. (2010). Bootstrapping trust evaluations through stereotypes. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems (pp. 241–248).

  6. Campbell, A. T., Lane, N. D., Miluzzo, E., Peterson, R. A., Lu, H., Zheng, X., et al. (2008). The rise of people-centric sensing. Internet Computing, 12(4), 12–21.

    Article  Google Scholar 

  7. Cochran, W. G. (1977). Sampling Techniques. New York: Wiley.

    MATH  Google Scholar 

  8. Dong, X., Berti-Equille, L., & Srivastava D. (2009) Integrating conflicting data: the role of source dependence. In Proceedings of the VLDB Endowment (PVLDB) (vol. 2).

  9. Etoré, P., & Jourdain, B. (2010). Adaptive optimal allocation in stratified sampling methods. Methodology and Computing in Applied Probability, 12, 335–360.

    Article  MathSciNet  MATH  Google Scholar 

  10. Fang, H., Zhang, J., & Magnenat Thalmann, N. (2014) Subjectivity grouping: Learning from users’ rating behavior. In Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems (pp. 1241–1248).

  11. Feldstein, M. (1997). The costs and benefits of going from low inflation to price stability. In Reducing Inflation: Motivation and Strategy (pp. 123–166). Chicago: University of Chicago Press.

  12. Frank, E., Wang, Y., Inglis, S., Holmes, G., & Witten, I. H. (1998). Using model trees for classification. Machine Learning, 32(1), 63–76.

    Article  MATH  Google Scholar 

  13. Ganeriwal, S., Balzano, L., & Srivastava, M. (2008). Reputation-based framework for high integrity sensor networks. ACM Transactions on Sensor Networks, 4(3), 15.

    Article  Google Scholar 

  14. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009). The weka data mining software: An update. ACM SIGKDD Explorations Newsletter, 11, 10–18.

    Article  Google Scholar 

  15. Hang, C.-W., Singh, M.P. (2009). Selecting trustworthy service in service-oriented environments. In The 12th AAMAS Workshop on Trust in Agent Societies (pp. 1–12).

  16. Hang, C.-W., & Singh, M. P. (2011). Trustworthy service selection and composition. ACM Transactions on Autonomous and Adaptive Systems (TAAS), 6(1), 5.

    Google Scholar 

  17. Ichino, M., & Yaguchi, H. (1994). Generalized minkowski metrics for mixed feature-type data analysis. IEEE Transactions Systems on Man and Cybernetics, 24, 698–708.

    Article  MathSciNet  Google Scholar 

  18. Irissappane, A., Oliehoek, F., Zhang, J. (2014). A pomdp based approach to optimally select sellers in electronic marketplaces. In Proceedings of the 2014 International Conference on Autonomous Agents and Multi-Agent Systems (pp. 1329–1336). International Foundation for Autonomous Agents and Multiagent Systems.

  19. Jain, A., Murty, M., & Flynn, P. (1999). Data clustering: A review. ACM Computing Surveys, 31, 264–323.

    Article  Google Scholar 

  20. Jøsang, A. (2013). Subjective Logic. Book Draft

  21. Jøsang, A., & Ismail, R. (2002). The beta reputation system. In Proceedings of the 15th Bled Electronic Commerce Conference (pp. 41–55), New York, USA

  22. Jøsang, A., Ismail, R., & Boyd, C. (2007). A survey of trust and reputation systems for online service provision. Decision Support Systems, 43(2), 618–644.

    Article  Google Scholar 

  23. Jøsang, A., & Presti, S. (2004). Analysing the relationship between risk and trust. In Trust Management (pp. 135–145).

  24. Kamar, E., Hacker, S., & Horvitz, E. (2012) Combining human and machine intelligence in large-scale crowdsourcing. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems (pp. 467–474).

  25. Kittur, A., Chi, E., & Suh, B. (2008). Crowdsourcing user studies with mechanical turk. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 453–456). ACM.

  26. Kwak, H., Lee, C., Park, H., Moon, S. (2010). What is twitter, a social network or a news media? In Proceedings of the 19th International Conference on World Wide Web (pp. 591–600). ACM

  27. Liu, X., Datta, A., Rzadca, K., & Lim, E.-P. (2009). StereoTrust: A group based personalized trust model. In Proceedings of the 18th ACM Conference on Information and Knowledge Management (pp. 7–16).

  28. O’Connor, B., Balasubramanyan, R., Routledge, B., & Smith, N. (2010). From tweets to polls: Linking text sentiment to public opinion time series. In Proceedings of the 4th International AAAI Conference on Weblogs and Social Media (pp. 122–129).

  29. Osborne, M. A., Roberts, S. J., Rogers, A., Ramchurn, S. D., & Jennings, N. R. (2008). Towards real-time information processing of sensor network data using computationally efficient multi-output gaussian processes. In Proceedings of the 7th International Conference on Information Processing in Sensor Networks (pp. 109–120).

  30. Ouyang, R., Srivastava, M., Toniolo, A., & Norman, T. J. (2014). Truth discovery in crowdsourced detection of spatial events. Technical report, International Technology Alliance (ITA), https://www.usukitacs.com/node/2637.

  31. Quinlan, J. (1992). Learning with continuous classes. In Proceedings of the 5th Australian Joint Conference on Artificial Intelligence (pp. 343–348).

  32. Raafat, R., Chater, N., & Frith, C. (2009). Herding in humans. Trends in Cognitive Sciences, 13, 420–428.

    Article  Google Scholar 

  33. Reece, S., Rogers, A., Roberts, S., Jennings, N.R. (2007). Rumours and reputation: Evaluating multi-dimensional trust within a decentralised reputation system. In Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems (pp. 1063–1070).

  34. Regan, K., Poupart, P., & Cohen, R. (2006). Bayesian reputation modeling in e-marketplaces sensitive to subjectivity, deception and change. In Proceedings of the 21st National Conference on Artificial Intelligence (pp. 206–212).

  35. Şensoy, M., Fokoue, A., Pan, J., Norman, T. J., Tang, Y., Oren, N., & Sycara, K. (2013). Reasoning about uncertain information and conflict resolution through trust revision. In Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems (pp. 837–844).

  36. Şensoy, M., Yilmaz, B., & Norman, T. J. (2014). STAGE: Stereotypical trust assessment through graph extraction. Computational Intelligence. doi:10.1111/coin.12046.

  37. Shiller, J. R. (1995). Conversation, information, and herd behaviour. American Economic Review, 85, 181–185.

    Google Scholar 

  38. Surowiecki, J. (2005). The wisdom of crowds. Anchor.

  39. Teacy, W. L., Luck, M., Rogers, A., & Jennings, N. R. (2012). An efficient and versatile approach to trust and reputation using hierarchical Bayesian modelling. Artificial Intelligence, 193, 149–185.

    Article  MathSciNet  MATH  Google Scholar 

  40. Teacy, W.T.L., Chalkiadakis, G., Rogers, A., & Jennings, N. (2008). Sequential decision making with untrustworthy service providers. In Proceedings of the 7th International Conference on Autonomous Agents and Multiagent Systems (pp. 755–762).

  41. Teacy, W. T. L., Patel, J., Jennings, N. R., & Luck, M. (2006). Travos: Trust and reputation in the context of inaccurate information sources. Autonomous Agents and Multi-Agent Systems, 12, 183–198.

    Article  Google Scholar 

  42. Tran-Thanh, L., Huynh, T.D., Rosenfeld, A., Ramchurn, S. D., & Jennings, N.R. (2014). Budgetfix: Budget limited crowdsourcing for interdependent task allocation with quality guarantees. In Proceedings of the 13th International Conference on Autonomous Agents and Multiagent Systems (pp. 477–484).

  43. Uddin, M., Amin, M., Le, H., Abdelzaher, T., Szymanski, B., & Nguyen, T. (2012). On diversifying source selection in social sensing. In Proceedings of the 9th International Conference on Networked Sensing Systems (INSS) (pp. 1–8).

  44. Waksberg, J. (1978). Sampling methods for random digit dialing. Journal of the American Statistical Association, 73(361), 40–46.

    Article  Google Scholar 

  45. Wang, D., Kaplan, L., & Abdelzaher, T. F. (2014). Maximum likelihood analysis of conflicting observations in social sensing. ACM Transactions on Sensor Networks, 10(2), 30:1–30:27.

    Article  Google Scholar 

  46. Whitby, A., Jøsang, A., Indulska, J. (2004). Filtering out unfair ratings in bayesian reputation systems. In Proceedings of the 7th International Workshop on Trust in Agent Societies.

  47. Witten, I. H., & Frank, E. (2005). Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann.

  48. Wolfson, O., Xu, B., & Tanner, R. (2007). Mobile peer-to-peer data dissemination with resource constraints. In Proceedings of the 8th International Conference on Mobile Data Management (pp. 16–23).

  49. Yin, X., Han, J., & Yu, P. (2008). Truth discovery with multiple conflicting information providers on the web. IEEE Transactions on Knowledge and Data Engineering, 20(6), 796–808.

    Article  Google Scholar 

  50. Zhang, W., Das, S., & Liu, Y. (2006). A trust based framework for secure data aggregation in wireless sensor networks. Proceedings of the 3rd Annual IEEE Communications Society Conference on Sensor (Vol. 1, pp. 60–69)., Mesh and Ad Hoc Communications and Networks IEEE: IEEE.

  51. Zheng, Z., & Padmanabhan, B. (2002). On active learning for data acquisition. In Proceedings of IEEE Conference on Data Mining (pp. 562–569).

Download references

Acknowledgments

Dr. Şensoy thanks to the U.S. Army Research Laboratory for its support under grant W911NF-14-1-0199 and The Scientific and Technological Research Council of Turkey (TUBITAK) for its support under grant 113E238.

Author information

Authors and Affiliations

  1. University of Aberdeen, Aberdeen, UK

    Anthony Etuk & Timothy J. Norman

  2. Department of Computer Science, Ozyegin University, Istanbul, Turkey

    Murat Şensoy

  3. IBM T. J. Watson Research Center, Hawthorne, NY, USA

    Mudhakar Srivatsa

Authors
  1. Anthony Etuk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Timothy J. Norman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Murat Şensoy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mudhakar Srivatsa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Murat Şensoy.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Etuk, A., Norman, T.J., Şensoy, M. et al. How to trust a few among many. Auton Agent Multi-Agent Syst 31, 531–560 (2017). https://doi.org/10.1007/s10458-016-9337-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10458-016-9337-5

Keywords

search

Navigation