Skip to main content
SpringerLink
Log in

Service selection in stochastic environments: a learning-automaton based solution

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In this paper, we propose a novel solution to the problem of identifying services of high quality. The reported solutions to this problem have, in one way or the other, resorted to using so-called “Reputation Systems” (RSs). Although these systems can offer generic recommendations by aggregating user-provided opinions about the quality of the services under consideration, they are, understandably, prone to “ballot stuffing” and “badmouthing” in a competitive marketplace. In general, unfair ratings may degrade the trustworthiness of RSs, and additionally, changes in the quality of service, over time, can render previous ratings unreliable. As opposed to the reported solutions, in this paper, we propose to solve the problem using tools provided by Learning Automata (LA), which have proven properties capable of learning the optimal action when operating in unknown stochastic environments. Furthermore, they combine rapid and accurate convergence with low computational complexity. In addition to its computational simplicity, unlike most reported approaches, our scheme does not require prior knowledge of the degree of any of the above mentioned problems associated with RSs. Instead, it gradually learns the identity and characteristics of the users which provide fair ratings, and of those who provide unfair ratings, even when these are a consequence of them making unintentional mistakes.

Comprehensive empirical results show that our LA-based scheme efficiently handles any degree of unfair ratings (as long as these ratings are binary—the extension to non-binary ratings is “trivial”, if we use the S-model of LA computations instead of the P-model). Furthermore, if the quality of services and/or the trustworthiness of the users change, our scheme is able to robustly track such changes over time. Finally, the scheme is ideal for decentralized processing. Accordingly, we believe that our LA-based scheme forms a promising basis for improving the performance of RSs in general.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Agache M, Oommen BJ (2002) Generalized pursuit learning schemes: new families of continuous and discretized learning automata. IEEE Trans Syst Man Cybern, Part B, Cybern 32(6):738–749

    Article  Google Scholar 

  2. Altincay H (2006) On the independence requirement in Dempster-Shafer theory for combining classifiers providing statistical evidence. Appl Intell 25:73–90. doi:10.1007/s10489-006-8867-y

    Article  Google Scholar 

  3. Buchegger S, Le Boudec J (2004) A robust reputation system for P2P and mobile ad-hoc networks. In: Proceedings of the second workshop on the economics of peer-to-peer systems

    Google Scholar 

  4. Chen M, Singh JP (2001) Computing and using reputations for internet ratings. In: Proceedings of the 3rd ACM conference on electronic commerce, Tampa, FL, USA. ACM, New York, pp 154–162

    Chapter  Google Scholar 

  5. Dellarocas C (2000) Immunizing online reputation reporting systems against unfair ratings and discriminatory behavior. In: Proceedings of the 2nd ACM conference on electronic commerce, Minneapolis, Minnesota, USA. ACM, New York, pp 150–157

    Chapter  Google Scholar 

  6. Despotovic Z, Aberer K (2004) A probabilistic approach to predict peers performance in P2P networks. In: Cooperative information agents VIII, pp 62–76

    Chapter  Google Scholar 

  7. Faceli K, de Carvalho A, Rezende S (2004) Combining intelligent techniques for sensor fusion. Appl Intell 20:199–213. doi:10.1023/B:APIN.0000021413.05467.20

    Article  MATH  Google Scholar 

  8. Gale W, Das S, Yu C (1990) Improvements to an algorithm for equipartitioning. IEEE Trans Comput 39(5):706–710

    Article  Google Scholar 

  9. Jøsang A, Ismail R, Boyd C (2007) A survey of trust and reputation systems for online service provision. Decis Support Syst 43(2):618–644

    Article  Google Scholar 

  10. Li X, Dai X, Dezert J, Smarandache F (2010) Fusion of imprecise qualitative information. Appl Intell 33:340–351. doi:10.1007/s10489-009-0170-2

    Article  Google Scholar 

  11. Littlestone N, Warmuth MK (1994) The weighted majority algorithm. Inf Comput 108(2):212–261

    Article  MathSciNet  MATH  Google Scholar 

  12. Mayur Datar M, Gionis A, Indyk P, Motwani R (2002) Maintaining stream statistics over sliding windows. SIAM J Comput 31(6):1794–1813

    Article  MathSciNet  MATH  Google Scholar 

  13. Mundinger J, Le Boudec J-Y (2008) Analysis of a reputation system for mobile ad-hoc networks with liars. Perform Eval 65(3–4):212–226

    Article  Google Scholar 

  14. Narendra KS, Thathachar MAL (1989) Learning automata: an introduction. Prentice-Hall, New Jersey

    Google Scholar 

  15. Obied A, Alhajj R (2009) Fraudulent and malicious sites on the web. Appl Intell 30:112–120. doi:10.1007/s10489-007-0102-y

    Article  Google Scholar 

  16. Oommen B, Ma D (1988) Deterministic learning automata solutions to the equipartitioning problem. IEEE Trans Comput 37(1):2–13

    Article  MathSciNet  MATH  Google Scholar 

  17. Oommen BJ, de St Croix EV (1996) Graph partitioning using learning automata. IEEE Trans Comput 45(2):195–208

    Article  MathSciNet  MATH  Google Scholar 

  18. Oommen BJ, Fothergill C (1993) Fast learning automaton-based image examination and retrieval. Comput J 36(6):542–553

    Article  Google Scholar 

  19. Oommen BJ, Ma DCY (1988) Deterministic learning automata solutions to the equipartitioning problem. IEEE Trans Comput 37(1):2–13

    Article  MathSciNet  MATH  Google Scholar 

  20. Oommen BJ, Ma DCY (1992) Stochastic automata solutions to the object partitioning problem. Comput J 34(6):A105–A120

    Google Scholar 

  21. Poznyak AS, Najim K (1997) Learning automata and stochastic optimization. Springer, Berlin

    MATH  Google Scholar 

  22. Sen S, Sajja N (2002) Robustness of reputation-based trust: boolean case. In: Proceedings of the first international joint conference on autonomous agents and multiagent systems: part 1, Bologna, Italy. ACM, New York, pp 288–293

    Chapter  Google Scholar 

  23. Shapiro C (1982) Consumer information, product quality, and seller reputation. Bell J Econ 13(1):20–35

    Article  Google Scholar 

  24. Thathachar MAL, Sastry PS (2002) Varieties of learning automata: an overview. IEEE Trans Syst Man Cybern, Part B, Cybern 32(6):711–722

    Article  Google Scholar 

  25. Thathachar MAL, Sastry PS (2003) Networks of learning automata: techniques for online stochastic optimization. Kluwer Academic, Boston

    Google Scholar 

  26. Tsetlin ML (1973) Automaton theory and the modeling of biological systems. Academic Press, New York

    Google Scholar 

  27. Whitby A, Jøsang A, Indulska J (2005) Filtering out unfair ratings in bayesian reputation systems. J Manag Res 4(2):48–64

    Google Scholar 

  28. Yu B, Singh MP (2003) Detecting deception in reputation management. In: Proceedings of the second international joint conference on autonomous agents and multiagent systems, Melbourne, Australia. ACM, New York, pp 73–80

    Chapter  Google Scholar 

  29. Yuan W, Guan D, Lee Y-K, Lee S (2010) The small-world trust network. Appl Intell 1–12. doi:10.1007/s10489-010-0230-7

  30. Zacharia G, Maes P (2000) Trust management through reputation mechanisms. Appl Artif Intell 14(9):881–907

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of ICT, University of Agder, Grimstad, Norway

    Anis Yazidi, Ole-Christoffer Granmo & B. John Oommen

  2. School of Computer Science, Carleton University, Ottawa, Canada, K1S 5B6

    B. John Oommen

Authors
  1. Anis Yazidi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ole-Christoffer Granmo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. B. John Oommen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to B. John Oommen.

Additional information

The first author gratefully acknowledges the financial support of the Ericsson Research, Aachen, Germany, and the third author is grateful for the partial support provided by NSERC, the Natural Sciences and Engineering Research Council of Canada. A preliminary version of this paper was presented at IEA/AIE’10, the 2010 International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems, Cordoba, Spain, in June 2010.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yazidi, A., Granmo, OC. & Oommen, B.J. Service selection in stochastic environments: a learning-automaton based solution. Appl Intell 36, 617–637 (2012). https://doi.org/10.1007/s10489-011-0280-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-011-0280-5

Keywords

Search

Navigation