Applied Intelligence

, Volume 36, Issue 3, pp 617–637 | Cite as

Service selection in stochastic environments: a learning-automaton based solution

  • Anis Yazidi
  • Ole-Christoffer Granmo
  • B. John OommenEmail author


In this paper, we propose a novel solution to the problem of identifying services of high quality. The reported solutions to this problem have, in one way or the other, resorted to using so-called “Reputation Systems” (RSs). Although these systems can offer generic recommendations by aggregating user-provided opinions about the quality of the services under consideration, they are, understandably, prone to “ballot stuffing” and “badmouthing” in a competitive marketplace. In general, unfair ratings may degrade the trustworthiness of RSs, and additionally, changes in the quality of service, over time, can render previous ratings unreliable. As opposed to the reported solutions, in this paper, we propose to solve the problem using tools provided by Learning Automata (LA), which have proven properties capable of learning the optimal action when operating in unknown stochastic environments. Furthermore, they combine rapid and accurate convergence with low computational complexity. In addition to its computational simplicity, unlike most reported approaches, our scheme does not require prior knowledge of the degree of any of the above mentioned problems associated with RSs. Instead, it gradually learns the identity and characteristics of the users which provide fair ratings, and of those who provide unfair ratings, even when these are a consequence of them making unintentional mistakes.

Comprehensive empirical results show that our LA-based scheme efficiently handles any degree of unfair ratings (as long as these ratings are binary—the extension to non-binary ratings is “trivial”, if we use the S-model of LA computations instead of the P-model). Furthermore, if the quality of services and/or the trustworthiness of the users change, our scheme is able to robustly track such changes over time. Finally, the scheme is ideal for decentralized processing. Accordingly, we believe that our LA-based scheme forms a promising basis for improving the performance of RSs in general.


Reputation systems Learning automata Stochastic optimization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agache M, Oommen BJ (2002) Generalized pursuit learning schemes: new families of continuous and discretized learning automata. IEEE Trans Syst Man Cybern, Part B, Cybern 32(6):738–749 CrossRefGoogle Scholar
  2. 2.
    Altincay H (2006) On the independence requirement in Dempster-Shafer theory for combining classifiers providing statistical evidence. Appl Intell 25:73–90. doi: 10.1007/s10489-006-8867-y CrossRefGoogle Scholar
  3. 3.
    Buchegger S, Le Boudec J (2004) A robust reputation system for P2P and mobile ad-hoc networks. In: Proceedings of the second workshop on the economics of peer-to-peer systems Google Scholar
  4. 4.
    Chen M, Singh JP (2001) Computing and using reputations for internet ratings. In: Proceedings of the 3rd ACM conference on electronic commerce, Tampa, FL, USA. ACM, New York, pp 154–162 CrossRefGoogle Scholar
  5. 5.
    Dellarocas C (2000) Immunizing online reputation reporting systems against unfair ratings and discriminatory behavior. In: Proceedings of the 2nd ACM conference on electronic commerce, Minneapolis, Minnesota, USA. ACM, New York, pp 150–157 CrossRefGoogle Scholar
  6. 6.
    Despotovic Z, Aberer K (2004) A probabilistic approach to predict peers performance in P2P networks. In: Cooperative information agents VIII, pp 62–76 CrossRefGoogle Scholar
  7. 7.
    Faceli K, de Carvalho A, Rezende S (2004) Combining intelligent techniques for sensor fusion. Appl Intell 20:199–213. doi: 10.1023/B:APIN.0000021413.05467.20 zbMATHCrossRefGoogle Scholar
  8. 8.
    Gale W, Das S, Yu C (1990) Improvements to an algorithm for equipartitioning. IEEE Trans Comput 39(5):706–710 CrossRefGoogle Scholar
  9. 9.
    Jøsang A, Ismail R, Boyd C (2007) A survey of trust and reputation systems for online service provision. Decis Support Syst 43(2):618–644 CrossRefGoogle Scholar
  10. 10.
    Li X, Dai X, Dezert J, Smarandache F (2010) Fusion of imprecise qualitative information. Appl Intell 33:340–351. doi: 10.1007/s10489-009-0170-2 CrossRefGoogle Scholar
  11. 11.
    Littlestone N, Warmuth MK (1994) The weighted majority algorithm. Inf Comput 108(2):212–261 MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Mayur Datar M, Gionis A, Indyk P, Motwani R (2002) Maintaining stream statistics over sliding windows. SIAM J Comput 31(6):1794–1813 MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Mundinger J, Le Boudec J-Y (2008) Analysis of a reputation system for mobile ad-hoc networks with liars. Perform Eval 65(3–4):212–226 CrossRefGoogle Scholar
  14. 14.
    Narendra KS, Thathachar MAL (1989) Learning automata: an introduction. Prentice-Hall, New Jersey Google Scholar
  15. 15.
    Obied A, Alhajj R (2009) Fraudulent and malicious sites on the web. Appl Intell 30:112–120. doi: 10.1007/s10489-007-0102-y CrossRefGoogle Scholar
  16. 16.
    Oommen B, Ma D (1988) Deterministic learning automata solutions to the equipartitioning problem. IEEE Trans Comput 37(1):2–13 MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    Oommen BJ, de St Croix EV (1996) Graph partitioning using learning automata. IEEE Trans Comput 45(2):195–208 MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Oommen BJ, Fothergill C (1993) Fast learning automaton-based image examination and retrieval. Comput J 36(6):542–553 CrossRefGoogle Scholar
  19. 19.
    Oommen BJ, Ma DCY (1988) Deterministic learning automata solutions to the equipartitioning problem. IEEE Trans Comput 37(1):2–13 MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Oommen BJ, Ma DCY (1992) Stochastic automata solutions to the object partitioning problem. Comput J 34(6):A105–A120 Google Scholar
  21. 21.
    Poznyak AS, Najim K (1997) Learning automata and stochastic optimization. Springer, Berlin zbMATHGoogle Scholar
  22. 22.
    Sen S, Sajja N (2002) Robustness of reputation-based trust: boolean case. In: Proceedings of the first international joint conference on autonomous agents and multiagent systems: part 1, Bologna, Italy. ACM, New York, pp 288–293 CrossRefGoogle Scholar
  23. 23.
    Shapiro C (1982) Consumer information, product quality, and seller reputation. Bell J Econ 13(1):20–35 CrossRefGoogle Scholar
  24. 24.
    Thathachar MAL, Sastry PS (2002) Varieties of learning automata: an overview. IEEE Trans Syst Man Cybern, Part B, Cybern 32(6):711–722 CrossRefGoogle Scholar
  25. 25.
    Thathachar MAL, Sastry PS (2003) Networks of learning automata: techniques for online stochastic optimization. Kluwer Academic, Boston Google Scholar
  26. 26.
    Tsetlin ML (1973) Automaton theory and the modeling of biological systems. Academic Press, New York Google Scholar
  27. 27.
    Whitby A, Jøsang A, Indulska J (2005) Filtering out unfair ratings in bayesian reputation systems. J Manag Res 4(2):48–64 Google Scholar
  28. 28.
    Yu B, Singh MP (2003) Detecting deception in reputation management. In: Proceedings of the second international joint conference on autonomous agents and multiagent systems, Melbourne, Australia. ACM, New York, pp 73–80 CrossRefGoogle Scholar
  29. 29.
    Yuan W, Guan D, Lee Y-K, Lee S (2010) The small-world trust network. Appl Intell 1–12. doi: 10.1007/s10489-010-0230-7
  30. 30.
    Zacharia G, Maes P (2000) Trust management through reputation mechanisms. Appl Artif Intell 14(9):881–907 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Anis Yazidi
    • 1
  • Ole-Christoffer Granmo
    • 1
  • B. John Oommen
    • 1
    • 2
    Email author
  1. 1.Department of ICTUniversity of AgderGrimstadNorway
  2. 2.School of Computer ScienceCarleton UniversityOttawaCanada

Personalised recommendations