Mechanism Design for Eliciting Probabilistic Estimates from Multiple Suppliers with Unknown Costs and Limited Precision

  • Athanasios Papakonstantinou
  • Alex Rogers
  • Enrico H. Gerding
  • Nicholas R. Jennings
Conference paper
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 59)


This paper reports on the design of a novel two-stage mechanism, based on strictly proper scoring rules, that allows a centre to acquire a costly probabilistic estimate of some unknown parameter, by eliciting and fusing estimates from multiple suppliers. Each of these suppliers is capable of producing a probabilistic estimate of any precision, up to a privately known maximum, and by fusing several low precision estimates together the centre is able to obtain a single estimate with a specified minimum precision. Specifically, in the mechanism’s first stage M from N agents are pre-selected by eliciting their privately known costs. In the second stage, these M agents are sequentially approached in a random order and their private maximum precision is elicited. A payment rule, based on a strictly proper scoring rule, then incentivises them to make and truthfully report an estimate of this maximum precision, which the centre fuses with others until it achieves its specified precision. We formally prove that the mechanism is incentive compatible regarding the costs, maximum precisions and estimates, and that it is individually rational. We present empirical results showing that our mechanism describes a family of possible ways to perform the pre-selection in the first stage, and formally prove that there is one that dominates all others.


Multi-agent systems mechanism design scoring rules 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hendrickson, A.D., Buehler, R.J.: Proper scores for probability forecasters. The Annals of Mathematical Statistics 42(6), 1916–1921 (1971)zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Savage, L.J.: Elicitation of personal probabilities and expectations. Journal of the American Statistical Association 66(336), 783–801 (1977)CrossRefMathSciNetGoogle Scholar
  3. 3.
    Selten, R.: Axiomatic characterization of the quadratic scoring rule. Experimental Economics 1(1), 43–61 (1998)zbMATHMathSciNetGoogle Scholar
  4. 4.
    Miller, N., Resnick, P., Zeckhauser, R.: Eliciting honest feedback: The peer prediction method. Management Science 51(9), 1359–1373 (2005)CrossRefGoogle Scholar
  5. 5.
    Zohar, A., Rosenschein, J.S.: Mechanisms for information elicitation. Artificial Intelligence 172(16-17), 1917–1939 (2008)zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Jurca, R., Faltings, B.: Reputation-based service level agreements for web services. In: Benatallah, B., Casati, F., Traverso, P. (eds.) ICSOC 2005. LNCS, vol. 3826, pp. 396–409. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  7. 7.
    Papakonstantinou, A., Rogers, A., Gerding, E.H., Jennings, N.R.: A truthful two-stage mechanism for eliciting probabilistic estimates with unknown costs. In: Proceedings of the 18th European Conference on Artificial Intelligence (ECAI 2008), Patras, Greece, pp. 448–452 (2008)Google Scholar
  8. 8.
    DeGroot, M.H., Schervish, M.J.: Probability and Statistics. Addison Wesley, Reading (2002)Google Scholar
  9. 9.
    Jehiel, P., Moldovanu, B.: Efficient design with interdependent valuations. Econometrica 69(5), 1237–1259 (2001)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Arnold, B.C., Balakrishnan, N., Nagaraja, H.N.: A First Course in Order Statistics. SIAM, Philadelphia (2008)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Athanasios Papakonstantinou
    • 1
  • Alex Rogers
    • 1
  • Enrico H. Gerding
    • 1
  • Nicholas R. Jennings
    • 1
  1. 1.School of Electronics and Computer ScienceUniversity of SouthamptonUnited Kingdom

Personalised recommendations