New Algorithms for Learning in Presence of Errors

  • Sanjeev Arora
  • Rong Ge
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6755)


We give new algorithms for a variety of randomly-generated instances of computational problems using a linearization technique that reduces to solving a system of linear equations.

These algorithms are derived in the context of learning with structured noise, a notion introduced in this paper. This notion is best illustrated with the learning parities with noise (LPN) problem —well-studied in learning theory and cryptography. In the standard version, we have access to an oracle that, each time we press a button, returns a random vector \( {a} \in \mbox{GF}(2)^n\) together with a bit \(b \in \mbox{GF}(2)\) that was computed as a·u + η, where \({u}\in \mbox{GF}(2)^n\) is a secret vector, and \(\eta \in \mbox{GF}(2)\) is a noise bit that is 1 with some probability p. Say p = 1/3. The goal is to recover u. This task is conjectured to be intractable.

In the structured noise setting we introduce a slight (?) variation of the model: upon pressing a button, we receive (say) 10 random vectors \({a_1}, {a_2}, \ldots, {a_{10}} \in \mbox{GF}(2)^n\), and corresponding bits b 1, b 2, …, b 10, of which at most 3 are noisy. The oracle may arbitrarily decide which of the 10 bits to make noisy. We exhibit a polynomial-time algorithm to recover the secret vector u given such an oracle. We think this structured noise model may be of independent interest in machine learning.

We discuss generalizations of our result, including learning with more general noise patterns. We also give the first nontrivial algorithms for two problems, which we show fit in our structured noise framework.

We give a slightly subexponential algorithm for the well-known learning with errors (LWE) problem over \(\mbox{GF}(q)\) introduced by Regev for cryptographic uses. Our algorithm works for the case when the gaussian noise is small; which was an open problem. Our result also clarifies why existing hardness results fail at this particular noise rate.

We also give polynomial-time algorithms for learning the MAJORITY OF PARITIES function of Applebaum et al. for certain parameter values. This function is a special case of Goldreich’s pseudorandom generator.

The full version is available at .


Linearization Technique Oblivious Transfer Structure Noise Noise Pattern Statistical Query 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ajtai, M.: Generating hard instances of lattice problems (extended abstract). In: Proceedings of the 28th Annual ACM Symposium on Theory of Computing (1996)Google Scholar
  2. 2.
    Ajtai, M., Dwork, C.: A public-key cryptosystem with worst-case/average-case equivalence. In: Proceedings of the 29th Annual ACM Symposium on Theory of Computing (1997)Google Scholar
  3. 3.
    Akavia, A., Goldwasser, S., Vaikuntanathan, V.: Simultaneous hardcore bits and cryptography against memory attacks. In: Reingold, O. (ed.) TCC 2009. LNCS, vol. 5444, pp. 474–495. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  4. 4.
    Alekhnovich, M.: More on average case vs approximation complexity. In: Proceedings of the 44th Annual IEEE Symposium on Foundations of Computer Science (2003)Google Scholar
  5. 5.
    Applebaum, B., Barak, B., Wigderson, A.: Public key cryptography from different assumptions. In: Proceedings of the 42nd Annual ACM Symposium on Theory of Computing (2010)Google Scholar
  6. 6.
    Bard, G.V.: Algebraic Cryptanalysis. Springer, Heidelberg (2009)CrossRefzbMATHGoogle Scholar
  7. 7.
    Blum, A., Furst, M.L., Kearns, M.J., Lipton, R.J.: Cryptographic primitives based on hard learning problems. In: Proceedings of the 13th Annual International Cryptology Conference on Advances in Cryptology (1994)Google Scholar
  8. 8.
    Blum, A., Kalai, A., Wasserman, H.: Noise-tolerant learning, the parity problem, and the statistical query model. Journal of ACM (2003)Google Scholar
  9. 9.
    Bogdanov, A., Qiao, Y.: On the security of goldreich’s one-way function. In: Dinur, I., Jansen, K., Naor, J., Rolim, J. (eds.) APPROX 2009. LNCS, vol. 5687, pp. 392–405. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  10. 10.
    Feldman, V., Gopalan, P., Khot, S., Ponnuswami, A.K.: New results for learning noisy parities and halfspaces. In: Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science (2006)Google Scholar
  11. 11.
    Friedl, K., Ivanyos, G., Magniez, F., Santha, M., Sen, P.: Hidden translation and orbit coset in quantum computing. In: Proceedings of the 35th Annual ACM Symposium on Theory of Computing (2003)Google Scholar
  12. 12.
    Goldreich, O.: Candidate one-way functions based on expander graphs. technical report. TR00-090, Electronic Colloquium on Computational Complexity, ECCC (2000)Google Scholar
  13. 13.
    Hopper, N.J., Blum, M.: Secure human identification protocols. In: Boyd, C. (ed.) ASIACRYPT 2001. LNCS, vol. 2248, p. 52. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  14. 14.
    Kearns, M.: Efficient noise-tolerant learning from statistical queries. Journal of ACM (1998)Google Scholar
  15. 15.
    Micciancio, D., Regev, O.: Lattice-based cryptography. In: Post Quantum Cryptography (2009)Google Scholar
  16. 16.
    Peikert, C.: Public-key cryptosystems from the worst-case shortest vector problem. In: Proceedings of 41st ACM Symposium on Theory of Computing (2009)Google Scholar
  17. 17.
    Peikert, C., Vaikuntanathan, V., Waters, B.: A framework for efficient and composable oblivious transfer. In: Wagner, D. (ed.) CRYPTO 2008. LNCS, vol. 5157, pp. 554–571. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  18. 18.
    Regev, O.: On lattices, learning with errors, random linear codes, and cryptography. Journal of ACM (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Sanjeev Arora
    • 1
  • Rong Ge
    • 1
  1. 1.Department of Computer SciencePrinceton University, and Center for Computational IntractabilityUSA

Personalised recommendations