Semantic Communication for Simple Goals Is Equivalent to On-line Learning

  • Brendan Juba
  • Santosh Vempala
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6925)


Previous works [11,6] introduced a model of semantic communication between a “user” and a “server,” in which the user attempts to achieve a given goal for communication. They show that whenever the user can sense progress, there exist universal user strategies that can achieve the goal whenever it is possible for any other user to reliably do so. A drawback of the actual constructions is that the users are inefficient: they enumerate protocols until they discover one that is successful, leading to the potential for exponential overhead in the length of the desired protocol. Goldreich et al. [6] conjectured that this overhead could be reduced to a polynomial dependence if we restricted our attention to classes of sufficiently simple user strategies and goals. In this work, we are able to obtain such universal strategies for some reasonably general special cases by establishing an equivalence between these special cases and the usual model of mistake-bounded on-line learning [3,15]. This equivalence also allows us to see the limits of constructing universal users based on sensing and motivates the study of sensing with richer kinds of feedback. Along the way, we also establish a new lower bound for the “beliefs model” [12], which demonstrates that constructions of efficient users in that framework rely on the existence of a common “belief” under which all of the servers in a class are designed to be efficient.


semantic communication on-line learning feedback models 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Angluin, D.: Queries and concept learning. Mach. Learn. 2(4), 319–342 (1988)MathSciNetGoogle Scholar
  2. 2.
    Auer, P., Long, P.M.: Structural results about on-line learning models with and without queries. Mach. Learn. 36(3), 147–181 (1999)CrossRefMATHGoogle Scholar
  3. 3.
    Bārzdiņš, J., Freivalds, R.: On the prediction of general recursive functions. Soviet Math. Dokl. 13, 1224–1228 (1972)Google Scholar
  4. 4.
    Bertsimas, D., Vempala, S.: Solving convex programs by random walks. J. ACM 51(4), 540–556 (2004)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Chandra, T.D., Toueg, S.: Unreliable failure detectors for reliable distributed systems. J. ACM 43(2), 225–267 (1996)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Goldreich, O., Juba, B., Sudan, M.: A theory of goal-oriented communication. Tech. Rep. TR09-075, ECCC (2009)Google Scholar
  7. 7.
    Grötschel, M., Lovász, L., Schrijver, A.: Geometric methods in combinatorial optimization. In: Pulleybank, W.R. (ed.) Proc. Silver Jubilee Conf. on Combinatorics. Progress in Combinatorial Optimization, pp. 167–183. Academic Press, New York (1984)Google Scholar
  8. 8.
    Grötschel, M., Lovász, L., Schrijver, A.: Geometric algorithms and combinatorial optimization, 2nd edn. Springer, New York (1993)CrossRefMATHGoogle Scholar
  9. 9.
    Jayanti, P., Toueg, S.: Every problem has a weakest failure detector. In: 27th PODC (2008)Google Scholar
  10. 10.
    Juba, B.: Universal Semantic Communication. Ph.D. thesis, MIT (2010)Google Scholar
  11. 11.
    Juba, B., Sudan, M.: Universal semantic communication I. In: 40th STOC (2008)Google Scholar
  12. 12.
    Juba, B., Sudan, M.: Efficient semantic communication via compatible beliefs. In: 2nd Innovations in Computer Science (2011)Google Scholar
  13. 13.
    Kearns, M., Valiant, L.: Cryptographic limitations on learning Boolean formulae and finite automata. J. ACM 41, 67–95 (1994)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Kharitonov, M.: Cryptographic hardness of distribution-specific learning. In: 25th STOC, pp. 372–381 (1993)Google Scholar
  15. 15.
    Littlestone, N.: Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Mach. Learn. 2(4), 285–318 (1988)Google Scholar
  16. 16.
    Maass, W., Turán, G.: How fast can a threshold gate learn? In: Hanson, S.J., Drastal, G.A., Rivest, R.L. (eds.) Computational Learning Theory and Natural Learning Systems: Constraints and Prospects, vol. 1, pp. 381–414. MIT Press, Cambridge (1994)Google Scholar
  17. 17.
    Patt-Shamir, B.: Personal communication (2010)Google Scholar
  18. 18.
    Vaidya, P.M.: A new algorithm for minimizing convex functions over convex sets. Mathematical Programming 73(3), 291–341 (1996)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Brendan Juba
    • 1
  • Santosh Vempala
    • 2
  1. 1.MIT CSAIL and Harvard UniversityCambridge
  2. 2.Georgia Tech Klaus Advanced Computing Building 2224Atlanta

Personalised recommendations