A Spin Glass Model for Reconstructing Nonlinearly Encrypted Signals Corrupted by Noise

We define a (symmetric key) encryption of a signal s∈RN\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathbf{s}\in {\mathbb {R^N}}$$\end{document} as a random mapping s↦y=(y1,…,yM)T∈RM\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathbf{s}\mapsto \mathbf y =(y_1,\ldots ,y_M)^T\in \mathbb {R}^M$$\end{document} known both to the sender and a recipient. In general the recipients may have access only to images y\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathbf{y}$$\end{document} corrupted by an additive noise of unknown strength. Given the encryption redundancy parameter (ERP) μ=M/N≥1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mu =M/N\ge 1$$\end{document} and the signal strength parameter R=∑isi2/N\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$R=\sqrt{\sum _i {s_i^2/N}}$$\end{document}, we consider the problem of reconstructing s\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathbf{s}$$\end{document} from its corrupted image by a Least Square Scheme for a certain class of random Gaussian mappings. The problem is equivalent to finding the configuration of minimal energy in a certain version of spherical spin glass model, with squared Gaussian random interaction potential. We use the Parisi replica symmetry breaking scheme to evaluate the mean overlap p∞∈[0,1]\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p_{\infty }\in [0,1]$$\end{document} between the original signal and its recovered image (known as ’estimate’) as N→∞\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N\rightarrow \infty $$\end{document}, for a given (’bare’) noise-to-signal ratio (NSR) γ≥0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma \ge 0$$\end{document}. Such an overlap is a measure of the quality of the signal reconstruction. We explicitly analyze the general case of linear-quadratic family of random mappings and discuss the full p∞(γ)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p_{\infty } (\gamma )$$\end{document} curve. When nonlinearity exceeds a certain threshold but redundancy is not yet too big, the replica symmetric solution is necessarily broken in some interval of NSR. We show that encryptions with a nonvanishing linear component permit reconstructions with p∞>0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p_{\infty }>0$$\end{document} for any μ>1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mu >1$$\end{document} and any γ<∞\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma <\infty $$\end{document}, with p∞∼γ-1/2\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p_{\infty }\sim \gamma ^{-1/2}$$\end{document} as γ→∞\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma \rightarrow \infty $$\end{document}. In contrast, for the case of purely quadratic nonlinearity, for any ERP μ>1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mu >1$$\end{document} there exists a threshold NSR value γc(μ)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma _c(\mu )$$\end{document} such that p∞=0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p_{\infty }=0$$\end{document} for γ>γc(μ)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\gamma >\gamma _c(\mu )$$\end{document} making the reconstruction impossible. The behaviour close to the threshold is given by p∞∼(γc-γ)3/4\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p_{\infty }\sim (\gamma _c-\gamma )^{3/4}$$\end{document} and is controlled by the replica symmetry breaking mechanism.


Introduction
In this paper we consider a schematic model of a (symmetric key) reconstruction of a source signal from its encrypted form corrupted by an additive noise when passed from a sender to a recipient.Signals are represented by N −dimensional source (column) with the collection of random functions V 1 (s), . . ., V M (s) representing an encryption algorithm shared between the parties participating in the signal exchange.Due to imperfect communication channels the recipients however get access to the encrypted signals only in a corrupted form z. We consider only the simplest corruption mechanism when the encrypted images y are modified by an additive random noise, i.e. z = y + b.
The noise vectors b are further assumed to be normally distributed: b ∼ N (0, σ 2 1 M ), i.e. components b k , k = 1, . . ., M are i.i.d.mean zero real Gaussian variables with the covariance b k b l = δ kl σ 2 , where the notation . . .here and henceforth stands for the expected value E[. ..] with respect to all types of random variables.A natural parameter is then the 'bare' noise-to-signal ratio (NSR) γ = σ 2 /R 2 , which will be eventually converted to true NSR dependent on the parameters of encryption algorithm (we will later on refer to such conversion in the text as an appropriate 'scaling') characterizing the level of signal corruption in the chosen type of encryption.The recipient's aim is to reconstruct the source signal s from the knowledge of z.In the presence of noise such reconstruction can be only approximate, and reconstructed signals are known in the signal processing literature as 'estimators' of the source signals.Their properties depend on the reconstruction scheme used.In the Bayesian inference approach philosophy one exploits reconstruction schemes optimized, among other parameters, over the probabilities of the input signal s by postulating its prior distribution over the set of feasible input signals.In that way one of the most popular estimators is the minimum mean square error (MMSE) estimator.We do not follow the Bayesian approach here, and rather consider the input signal through the reconstruction procedure as a fixed vector, and then employ the Least-Square reconstruction scheme, which returns an estimator as where W is a set of feasible signals.Since for a given input s and the Gaussian noise b the probability p(z|s) to observe z is given by p(z|s) ∼ exp − 1 , the estimator Eq.( 2) is also known as the maximum likelihood estimator.Note however that this approach can be given a formal Bayesian meaning as a Maximum-A-Posteriori (MAP) estimator with a uniform prior distribution over the feasibility set, see below.Quality of the signal reconstruction under this scheme is then characterized by the value of a distortion parameter measuring the difference between the fixed source signal s and the reconstructed estimator x.For this one can use any suitable distance function d(x; s), e.g the Euclidean distance normalized to the signal strength: One is interested in getting the expression for the distortion in the asymptotic limit of large signal dimensions (M, N ) → ∞.As long as M remains smaller than N , any solution x ∈ W ⊆ R N of the set of M equations b k + V k (s) − V k (x) = 0, k = 1, . . ., M will be corresponding to the eactly zero value of the cost function, and could be a legitimate estimator.Those estimators then form continuously parametrized manifolds in R N .It is therefore clear that even in the absence of any noise full reconstruction of the signal for M < N under this scheme is impossible.Although such a case is not at all devoid of interest, we do not treat it in the present paper leaving it for a separate study.In contrast, for 'redundantantly' encrypted signals with M ≥ N the set of possible estimators generically consists of isolated points in R N .To this end we introduce the Encryption Redundancy Parameter (ERP) µ = M/N ∈ (1, ∞).We will see that under such conditions signals can be in general faithfully reconstructed in some range of the noise-to-signal ratios γ > 0.
In this paper we are going to apply tools of Statistical Mechanics for calculating the average asymptotic distortion for a certain class of the least square reconstruction of a randomly encrypted noisy signal.As this is essentially a large-scale random optimization problem, methods of statistical mechanics of disordered systems like the replica trick developed in theory of spin glasses are known to be efficient in providing important analytical insights in the statistical properties of the solution, see e.g.[27,30].It is also worth noting that distinctly different aspects of the problem of information reconstruction ( the so-called error-correcting procedures) were discussed in the framework of spin glass ideas already in the seminal work by Sourlas [34].
We consider the reconstruction problem under two technical assumptions.The first assumption is that the recipient is aware of the exact source signal strength R, and therefore can restrict the least square minimization search in Eq.(2) to the feasibility set W given by (N − 1)−dimensional sphere of the radius R √ N .We will refer to such a condition as the 'spherical constraint'.From the point of view of the Bayesian analysis our reconstruction scheme can be considered as a MAP estimator with postulated prior distribution being the uniform measure on the above-mentioned (N − 1)−dimensional sphere.As the lengths of both the input signal s and an estimator x are fixed to R √ N , the distance (3) depends only on the scalar product (x, s).We therefore can conveniently characterize the quality of the reconstruction via the quality parameter defined as where p N = 1 corresponds to a reconstruction without any macroscopic distortion, whereas p N = 0 manifests impossibility to recover any information from the originally encrypted signal.Note that the assumption of the fixed input signal strength R is technically convenient but can be further relaxed; the analysis can be extended, without much difficulty, to the search in a spherical shell N , and hopefully to some other situations.
Our second assumption is that the random functions V k (s) belong to the class of (smooth) isotropic gaussian-distributed random fields on the sphere, independent for different values of k (and independent of the noise b) , with mean-zero and the covariance structure dependent only on the angle between the vectors.Using the scaling appropriate for our problem in the limit of large N we represent such covariances as where the angular brackets . . .denote the expectation with respect to the corresponding probability measures.We will further assume for simplicity that Φ (u) in Eq.( 5) is infinitely differentiable.The simplest case of random fields of this type corresponds to a linear encryption algorithm, with the functions V k (x), k = 1, . . ., M chosen in the form of random linear combinations: where the vectors a k are assumed to be random, mean-zero mutually independent Gaussian, each with N i.i.d.components characterized by the variances a ki a lj = J 2 1 N δ lk δ ij .Such choice implies the covariance (5) with Φ (u) = J 2 1 u.The linear encryption is very special, yet not completely trivial, instance of the reconstruction problem, as in that case one can formally solve the minimization problem by the method of Lagrange multipliers explicitly.To this end we introduce the cost function (cf.(2)) depending on the source signal s as a parameter, and following the standard idea of a constrained minimization consider the stationarity conditions ∇L λ,s (x) = 0 for the Lagrangian L λ,s (x) = H s (x) − λ 2 (x, x), with real λ being the Lagrange multiplier taking care of the spherical constraint.In the general case of a non-linear encryption algorithm this procedure does not seem to help much to our analysis, as the stationarity equations look hard to study.In the linear case one can however introduce a N × M matrix A whose M rows are represented by (transposed) vectors a T k featuring in Eq.( 6).We than can easily see that the stationarity conditions in that case amount to the following matrix equation: which can be then immediately solved and provides the estimator in the form The possible set of Lagrange multipliers is obtained by solving the equation implied by the spherical constraint: (x, x) = N R 2 , which is in general equivalent to a polynomial equation of degree 2N in λ.The number of real solutions of that equation depends on the noise vector b.One of the real solutions corresponds to the minimum of the cost function, others to saddle-points or maxima.In particular, in the (trivial) limiting case of no noise b = 0 the global minimum corresponds to λ = 0 implying reconstruction with no distortion: x = s, hence p N = 1 as is natural to expect.At the same time, for any b = 0 the analysis of Eq.( 9) becomes a non-trivial problem.One possible way is to account for the presence of a weak noise with small variance σ 2 by developing a perturbation theory in the small scaled NSR parameter γ = σ 2 1.Such a theory is outlined in the Appendix A, where we find that for a given value of ERP µ > 1 and in the leading order in γ the asymptotic disorder-averaged quality reconstruction parameter defined in Eq.( 4) is given by: This result is based on the asymptotic mean density of eigenvalues of random Wishart matrices A T A due to Marchenko and Pastur [26].Similarly, one can develop a perturbation theory for very big NSR γ 1, see Appendix A. In this way one finds that the Lagrange multiplier |λ| ∼ (µγ) 1/2 and p ∞ ∼ µ γ .Although the perturbation theories are conceptually straightforward, and can be with due effort extended to higher orders, the calculations quickly become too cumbersome.At the moment we are not aware of any direct approach to our minimization problem in the linear encryption case which may provide non-perturbative results, as N → ∞, for asymptotic distortion p ∞ at values of scaled NSR parameter γ of the order of unity.At the same time we will see that methods of statistical mechanics provide a very explicit expression for any γ.
It is necessary to mention that various instances of not dissimilar linear reconstruction problems in related forms received recently a considerable attention.The emphasis in those studies seems however to be mainly restricted to the case of source signals being subject to a compressed sensing, i.e. represented by a sparse vector with a finite fraction of zero entries, see e.g.[24,37,28] and references therein.To this end especially deserve mentioning the works [6,7,8] which studied the mean value of distortions for MAP estimator for a linear problem (though with prior distribution different from the spherical constraint).Although having a moderate overlap with methods used in this work, the actual calculations and the main message of those papers seem rather different.
In particular, our main emphasis will be on ability to analyse the case of a quite general nonlinear random Gaussian encryptions ‡.The corresponding class of functions V k (x) extends the above-mentioned case of random linear forms to higher-order random forms, the first nontrivial example being the form of degree 2: where entries of N × N real symmetric random matrices J (k) , k = 1, . . ., M are meanzero Gaussian variables (independent of the vectors a k ) with the covariance structure which eventually results in the covariance (5) of the form Φ (u) = J 2 1 u + 1 2 J 2 2 u 2 .We will refer to the above class of random encryptions as the linear-quadratic family.
In fact, the general covariance structure of isotropic Gaussian random fields on a sphere of radius R √ N is also well-known from the theory of spherical spin glasses: these are functions which can be represented by a (possibly, terminating) series with non-negative coefficients: such that Φ (R 2 ) has a finite value, see e.g.[2].Although our theory of encrypted signal reconstruction will be developed for the general case, all the explicit analysis ‡ In the context of compressed sensing some reconstruction aspects of nonlinear models were considered, e.g. in [9,33], but our approach seems distinctly different.
of the ensuing equations will be restricted to the case of the linear-quadratic family, Eq.( 11).

Main Results
Our first main result is the following Proposition 1: Given a value of R > 0 characterizing the source signal strength, and the value of the Encryption Redundancy Parameter µ > 1, consider the functional +µ where the variable v ≥ 0, the variables t and Q take values in intervals [−R, R] and [0, R 2 ], correspondingly, and Then in the framework of the Parisi scheme of the replica trick the mean value of the parameter p N characterising quality of the information recovery in the signal reconstruction scheme, Eqs.( 2)-( 5), with normally distributed noise b ∼ N (0, where the specific value of the parameter t to be substituted to (15) should be found by simultaneously minimizing the functional E[w s (u); Q, v, t] over t and maximizing it over all other parameters and the function w s (u).
Our next result provides an explicit solution to this variational problem in a certain range of parameters.

Proposition 2:
In the range of parameters such that the solution t to the equation satisfies the inequality the variational problem Eq.( 14) is solved by the Replica-Symmetric Ansatz Q = 0.In particular, for a given 'bare' Noise-to-Signal ratio γ = σ 2 /R 2 the quality parameter p ∞ definied in Eq.( 15) is given by the solution of the following equation: In addition, for the range of parameters such that the solution t of Eq.(68) violates the inequality Eq.(69) the variational problem Eq.( 14) is solved by the Full Replica-Symmetry Breaking Ansatz.In this case the value p of the quality parameter p ∞ definied in Eq.( 15) is given by the solution of the following system of two equations in the variables p ∈ [0, 1] and We finally note in passing that an attempt to extremize the functional Eq.( 14) in the space of the so-called 1-Step Replica Symmetry Breaking Ansatz (1-RSB) does not yield any solution respecting the required constraints on the parameters v, t and Q.
1.1.1.Results for the linear-quadratic family of encryptions.Both Propositions providing the solution of our reconstruction problem in full generality, for every specific choice of the covariance structure Φ(u) the equations need to be further analyzed.In this work we performed a detailed analysis of the case of encryptions belonging to the linearquadratic family Eq.( 11) with the covariance structure of the form Φ (u) = J 2 1 u+ 1 2 J 2 2 u 2 .The most essential qualitative features of the analysis are summarized below.
For such a family, apart from our main control parameters, scaled NSR γ = γ/J 2 1 and ERP µ, the reconstruction is very essentially controlled by an important parameter a = R 2 J 2 2 /J 2 1 which reflects the degree of nonlinearity in the encryption mapping.Our first result is that there exists a threshold value of this parameter, a = 1, such that for all encryptions in the family with a < 1 the variational problem is always solved with the Replica-Symmetric Ansatz Eq. (18).In contrast, for linear-quadratic encryptions with higher nonlinearity a > 1 there exists a threshold value of the Encryption Redundancy Parameter µ = (a 2/3 −a 1/3 +1) 3 a := µ AT (a) > 1 such that for any µ ∈ (1, µ AT (a)) the replica symmetric solution is broken in some interval of NSR.This implies that increasing redundancy for a fixed non-linearity one eventually always ends up in the replicasymmetric phase, see the phase diagram in Fig. 1.
In contrast, at a fixed nonlinearity a > 1 and not too big redundancy values µ ∈ (1, µ AT (a)) there exists generically an interval of scaled NSR's γ(1) AT such that the replica-symmetry is broken inside and preserved for γ outside that interval.

The exact values γ(1,2)
AT can be in general found only by numerically solving the 4thorder polynomial equation, see Eq.(57).At the same time, using that for large enough scaled NSR γ > γ(2) AT the replica symmetry is restored, one can employ the RS equation Eq.(57) to determine the behaviour of the quality parameter p ∞ (γ) as γ → ∞.One finds that in all cases but one this quantity vanishes for asymptotically large values of NSR as p ∞ ∼ γ−1/2 , see Eq.(58), i.e. in qualitatively the same way as for purely linear system with a = 0.The only exceptional case, showing qualitatively different behaviour to the above picture, is that of purely quadratic encryption with vanishing linear component §, when J 1 → 0 at a fixed value of J 2 > 0. The appropriately rescaled NSR in this case is γ = γ J 2 2 R 2 .In this limit the second threshold γ (2) AT escapes to infinity and the replica symmetry is broken for all γ > γAT = (µ − 1) 2 /µ.Moreover, most importantly there exists a threshold NSR value γc = µ − 1 2 > γAT such that p ∞ = 0 for γ > γc making the reconstruction impossible.The full curve p ∞ (γ) can be explicitly described in this case analytically.In particular, the behaviour close to the threshold NSR is given by p ∞ ∼ (γ c − γ) 3/4 and the non-trivial exponent 3/4 is fully controlled by the replica symmetry breaking mechanism.
The existence of a sharp NSR threshold γc in the pure quadratic encryption case may have useful consequences for security of transmitting the encrypted signal.Indeed, it is a quite common assumption that an eavesdropper may get access to the transmitted § It is worth noting that in the absence of linear component the encryption mapping V (s) in Eq.( 11) becomes invariant with respect to the reflections s → −s.As a result, the least-square reconstruction may formally return solutions with negative values of the parameter p N in Eq.( 4).To avoid this we consider the pure quadratic case as the limit J 1 → 0 taken after N → ∞, which is enough to break the mentioned invariance.
signal by a channel with inferior quality, characterized by higher level of noise.This may then result in impossibility for eavesdroppers to reconstruct the quadratically encoded signal even if the encoding algorithm is perfectly known to them.
1.1.2.General remarks on the method The task of optimizing various random 'cost functions', not unlike H s (x) in Eq.( 7), is long known to be facilitated by a recourse to the methods of statistical mechanics, see e.g.[30] and [27] for early references and introduction to the method.In that framework one encounters the task of evaluating expectated values over distributions of random variables coming through the cost function in both numerator and denominator of the equations describing the quantities of interest, see the right-hand side of Eq.( 22) below.Performing such averaging is known to be one of the central technical problems in the theory of disordered systems.One of the most powerful, though non-rigorous, methods of dealing with this problem at the level of theoretical physics is the (in)famous replica trick, see [27] and references therein.A considerable progress achieved in the last decades in developing rigorous aspects of that theory [4,29] makes this task, in principle, feasible for the cases when the random energy function H s (x) is Gaussian-distributed.The model where configurations are restricted to the surface of a sphere are known in the spin-glass literature as 'spherical models', but their successful treatment, originally nonrigorous [13,12,23] and in recent years rigorous [2,5,10,11,36,35], seems again be restricted to the normally-distributed case.In the present case however the cost function is per se not Gaussian, but represented as a sum of squared Gaussian-distributed terms.We are not aware of any systematic treatment of spherical spin glass models with such type of spin interaction.Some results obtained by extending replica trick treatment to this type of random functions were given by the present author in [16], but details were never published.To present the corresponding method on a meaningful example is one of the goals of the present paper.Indeed, we shall see that, with due modifications, the method is very efficient, and, when combined with the Parisi replica symmetry breaking Ansatz allows to get a reasonably detailed insight into the reconstruction problem.As squared gaussian-ditributed terms are common to many optimization problems based on the Least Square method, one may hope that the approach proposed in the present paper may prove to be of wider utility.In particular, an interesting direction of future research may be study of the minima, saddles and other structures of this type in the arising 'optimization landscape' following an impressive recent progress in this direction for Gaussian spherical model, see [31] and references therein.This may help to devise better search algorithms for solutions of the optimization problems of this type.
Another technical aspect of our treatment which is worth mentioning is as follows.In problems of this sort replica treatment is much facilitated by noticing that after performing the disorder averaging the replicated partition function possesses a high degree of invariance: an arbitrary simultaneous O(N ) rotation of all n replica vectors x a , a = 1, . . ., n leaves the integrand invariant.To exploit such an invariance in the most efficient way one may use a method suggested in the framework of Random Matrix Theory in the works [15,21] That method allowed one to convert the integrals over N − component vectors x a , a = 1, . . ., n to a single positive-definite n × n matrix Q ab ≥ 0. Such transformation than allows to represent the integrand in a form ideally suited for extracting the large-N asymptotic of the integral.In the context of spin glasses and related statistical mechanics systems this method was first successfully used in [20] and then [18,19], and most recently in [25], and proved to be a very efficient framework for implementing the Parisi scheme of replica symmetry breaking.In the present problem however the integrand has lesser invariance due to presence of a fixed direction exemplified by the original message s.Namely, it is invariant only with respect to rotations forming a subgroup of O(N ) consisting of all In the Appendix C we prove a Theorem which is instrumental in adjusting our approach to the present case of a fixed direction.One may hope that this generalization may have other applications beyond the present problem. Acknowledgements.
The author is grateful to Jean-Philippe Bouchaud, Christian Schmidt, Guilhem Semerjian, Nicolas Sourlas and Francesco Zamponi for enlightening discussions and encouraging interest in this work, and to Dr. Mihail Poplavskyi for his help with analysis of Eq.(71) and preparing figures for this article.The financial support by EPSRC grant EP/N009436/1 "The many faces of random characteristic polynomials" is acknowledged with thanks.

General setting of the problem
To put the least square minimization problem (2) in the context of Statistical Mechanics, one follows the standard route and interprets the cost function in Eq.( 7) as an energy associated with a configuration x T of N spin variables (x 1 , . . ., x N ), constrained to the sphere of radius |x| = N √ R.This allows one to treat our minimization problem as a problem of Statistical Mechanics, by introducing the temperature parameter T > 0, and considering the Boltzmann-Gibbs weights π β (x) = Z −1 β e −βHs(x) associated with any configuration x on the sphere, with Z β being the partition function of the model for the inverse temperature β = T −1 : Equivalent transformations were also suggested earlier in [32], see also [14].
The power of the method is that in the zero-temperature limit β → ∞ the Boltzmann-Gibbs weights concentrate on the set of globally minimal values of the cost function, so that for any well-behaving function g(x) the thermal average value g β (x) := g(x)π β (x)dx should tend to the value of that function evaluated at the argument corresponding to solutions of the minimization problem (2).To this end we introduce the thermal average p (β) N of the distance function defined in eq.( 3) and consider its expected value with respect to both the set of random functions V k (x) and the noise b: Our goal is to evaluate the above quantity for finite β = 1/T in the limit of large N 1, and eventually perform the zero temperature limit T → 0 thus extracting providing us with a measure of the quality of the asymptotic signal reconstruction in the original optimization problem.

Replica trick
In the framework of the replica trick one represents the normalization factor Z −1 β in the Boltzmann-Gibbs weights formally as 1/Z β = lim n→0 Z n−1 β and treats the parameter n before the limit as a positive integer.This allows to rewrite (22) formally as where we defined The disorder average can be now performed in the following steps.First, using the additive form of the cost function in Eq.( 7) and independence of H Using the Gaussian nature of V (x) entering to H (k) s (x) in a squared form, see Eq.( 7), and exploiting the covariance structure (5) one can show that where we introduce the (positive definite) n × n matrix G(x 1 , . . ., x n ; s) with entries For convenience of the reader we provide a derivation of the formula Eq.( 26) in the Appendix B. Note that this result is well-known in the probability literature, see e.g.[22].We see that At this step it is very helpful to notice that the integrand in (28) possesses a high degree of invariance.Namely, consider all possible rotations around the axis whose direction is given by the vector s.Such rotations form a subgroup of O(N ) consisting of all N × N orthogonal transformations O s satisfying O T s O s = 1 N and O s s = s.Then the integrand in (28) remains invariant under a simultaneous change x a → O s x a for all a = 1, . . ., n.In the Appendix C we prove a Theorem which is instrumental for implementing our previous approach to similar problems [20] to the present case of somewhat lesser invariance.Not surprisingly, in such a case the integration needs to go not only over n × n matrix of scalar products Q ab = (x a , x b ) ≥ 0, but also over an n-component vector t = (t 1 , . . ., t n ) ∈ R n of projections t a = (x a , s).Applying the Theorem and rescaling for convenience the integration variables Q ab → N Q ab and t → √ N t we bring Eq.( 29) to the form where N,n is defined in Eq.( 110), the integration goes over the domain and we defined with n × n matrix g(Q, t) characterized by its entries (cf.Eq.( 27)) So far our treatment of p was exact for any positive integer values N and n satisfying N > n + 1 and involved no approximations.Our goal is however to extract the leading behaviour of that object as N 1 and allowing formally n to take noninteger values to be able to perform the replica limit n → 0.

Variational problem in the framework of Parisi Ansatz
Clearly, the form of the integrand in Eq.( 29) being proportional to the factor e − N 2 F (Q,t) is suggestive of using the Laplace (a.k.a saddle-point or steepest descent) method.In following this route we resort to a non-rigorous and heuristic, but computationally efficient scheme of Parisi replica symmetry breaking [27].We implement this scheme in a particular variant most natural for models with rotational invariance, going back to Crisanti and Sommers paper [12], and somewhat better explained in the Appendix A of [20], and in even more detail in the Appendix C of [17].We therefore won't discuss the method itself in the present paper, only giving a brief account of necessary steps.
The scheme starts with a standard assumption that in the replica limit n → 0 the integral is dominated by configurations of matrices Q which for finite integer n have a special hierarchically built structure characterized by the sequence of integers and the values placed in the off-diagonal entries of the Q matrix block-wise, and satisfying: Finally, we complete the procedure by filling in the n diagonal entries Q aa of the matrix Q with one and the same value Q aa = q d := q k+1 ≥ q k .Note that in our particular case the diagonal entries q d must in fact be chosen in the form q d = R 2 − t 2 a , in order to respect the constraints impose by the integration domain Eq.( 30).As to the vector t of variables t a , we are making an additional assumption that with respect to those variables the integral is in fact dominated by equal values: t a = t, ∀a = 1, . . ., n.
Obviously, the matrix g(Q, t) defined in (32) inherits the hierarchical structure from Q, with parameters m l , l = 0, k + 1 shared by both matrices, but parameters q l replaced by parameters g l given by and The next task of the scheme is to express both T r ln g(Q, t) and Tr ln Q in terms of the parameters entering Eqs.( 33) and (34).This is most easily achieved by writing down all distinct eigenvalues λ 1 , . . ., λ k+2 of the involved matrices, and their degeneracies d i+2 = n m −1 i+1 − m −1 i ∀i = 0, . . ., k, and d 1 = 1.For the matrix Q those eigenvalues are listed, e.g., in the appendix C of [17], and for the matrix g(Q, t) the corresponding expressions can be obtained from those for Q by replacing q l by parameters g l from Eq. (35).The subsequent treatment is much facilitated by introducing the following (generalized) function of the variable q: where we use the notation θ(z) for the Heaviside step function: θ(z) = 1 for z > 0 and zero otherwise.In view of the inequalities Eq.(33,34) the function x(q) is piecewiseconstant non-increasing, and changes between x(0 < q < q 0 ) = m 0 ≡ n through x(q i−1 < q < q i ) = m i for i = 1, . . ., k to finally x(q k < q < q d ) = m k+1 ≡ 1.A clever observation by Crisanti and Sommers allows one to express eigenvalues of any function of the hierarchical matrix Q in terms of simple integrals involving x(q).In particular, for eigenvalues λ (g) l of the matrix g(Q, t) we have: x(q) dg t dq dq, and λ where we introduced a piecewise-continuous function g t (q), q ∈ [0, q d ] such that in the interval q ∈ [q 0 , q k ] it is given by whereas outside that interval it has two constant values: In particular, λ 1 can be further rewritten as x(q) dg t dq dq = ng t (q 0 ) + Such a representation, together with the definition Eq.( 36) of the function x(q) facilitates calculating quantities interesting to us in the replica limit as: = lim n→0 1 n ln 1 + βng t (q 0 ) + β q d q 0 x(q) dg t dq dq In the last term it is convenient to integrate by parts, and use x(q k + 0) = 1 and x(q 0 − 0) = n, which after obvious regrouping of terms reduces the right-hand side of Eq.(41) as lim n→0 1 n ln 1 + βng t (q 0 ) + β q d q 0 x(q) g t (q) dq − ln 1 + β q d q 0 x(q)g t (q) dq (42) g t (q)dq 1 + β q d q x(q)g t (q)dq where we denoted g t (q) := dgt dq .The limit n → 0 is now easy to perform following the general prescription of the Parisi method: in such a limit the inequality Eq.( 33) should be reversed: and the function x(q) is now transformed to a non-decreasing function of the variable q in the interval q 0 ≤ q ≤ q k , and satisfying outside that interval the following properties x(q < q 0 ) = 0, and x(q > q k ) = 1. (44) In general, such a function also depends on the increasing sequence of k real parameters m l described in Eq.(43) .Performing the corresponding limit and taking into account that in view of x(q < q 0 )) = 0 we have q 0 0 g t (q)dq 1 + β q d q x(q)g t (q)dq = g t (q 0 ) 1 + β q d q 0 x(q)g t (q) dq we eventually see that and by a similar calculation also find: The two formulas Eq.( 45)-( 46) provide us therefore with a full formal control of the main exponential factor e − N 2 F (Q,t) in Eq.( 29) for N → ∞ in the replica limit n → 0. Note that the fact that the limit in the left hand-side of Eq.( 46) is finite implies also (29).Collecting finally all factors in Eq.( 29) when performing the replica limit n → 0, using explicit forms Eq.( 38)-(39), remembering q d = R 2 − t 2 and finally understanding by x(q) only its non-trivial part in the interval q 0 , q k we arrive to the following Proposition 2.1: Given the values of real parameters R > 0 and µ > 1, consider the functional which depends on the parameters −R ≤ t ≤ R, and 0 ≤ q 0 ≤ q k < q d = R 2 − t 2 and a non-decreasing function x(q) of the variable q in the interval q 0 ≤ q ≤ q k .Then in

Replica symmetric solution
Notice that the last equation Eq.( 52) is identically satisfied with the choice Q = 0, defining the so-called Replica Symmetric (RS) solution.For such a choice the interval [R 2 − Q, R 2 ] of the support of the function w s (u) shrinks to zero, making that function immaterial for the variational procedure.Moreover, the equations ( 50)-( 51) drastically simplify yielding the pair: Remarkably, this pair can be further reduced to a single equation in the variable p = t/R, precisely one given in the list of Main Results, Eq.( 18), thus providing the asymptotic value of the quality reconstruction parameter p ∞ = p.As expected, in the case of no noise γ = 0 the solution of Eq.( 18) is provided by p = 1 corresponding to the perfect reconstruction of the source signal.It is also easy to treat the equation perturbatively in the case of weak noise γ → 0 and obtain to the leading order: In particular, this result agrees with the first-order perturbation theory analysis for the linear case Φ(u) = u, see Eq.( 10) and Appendix A, and generalizes it to a generic nonlinearity.It also emphasizes the natural fact that the signal recovery becomes very sensitive to the noise for the values of Encryption Redundancy Parameter µ → 1.
For the linear-quadratic family Eq.( 11) with the covariance structure of the form Φ (u) = J 2 1 u + 1 2 J 2 2 u 2 the equation Eq.( 18) can be readily studied non-perturbatively for any value of NSR.We start with two limiting cases in the family: that of 'purely linear' and 'purely quadratic' encryptions.In the former case J 2 = 0, but J 1 = 0 and after introducing the scaled NSR γ := γ/J 2  1 we arrive at a cubic equation In particular, p is nonvanishing for any value of the scaled NSR 0 ≤ γ < ∞, and tends to zero as p ∼ µ/γ for γ 1, in full agreement with the direct perturbation theory approach, see appendix A. For intermediate NSR values the solution can be easily plotted numerically, see Fig. 2.
In the opposite case of purely quadratic encryption when J 1 = 0 but J 2 = 0 the equation Eq.( 18) is biquadratic, so that one can find the RS solution explicitly.Introducing the rescaled NSR pertinent to this limit as γ := γ Thus, in this case the replica-symmetric solution predicts the existence of a NSR threshold γ(RS) c = µ − 1 beyond which meaningful reconstruction of the encrypted signal is impossible.We will see later on that although this conclusion is qualitatively correct, the actual value for the threshold and the critical exponent controlling the behaviour close to the threshold is different and is obtained when the phenomenon of the replica symmetry breaking is taken into account.
Finally, in the case of a generic linear-quadratic encryption with both J 1 = 0 and J 2 = 0 the resulting equation Eq.( 18) is a general polynomial of the fourth degree.Introducing again the scaled NSR γ := γ/J 2 1 and a parameter characterizing effective non-linearity of the mapping a := (R J 2 /J 1 ) 2 we can rewrite the equation as: In particular, we see that p tends to zero as NSR γ → ∞ as in the purely linear case: We will see in the next section that generically for linear-quadratic encryptions with big enough, but finite nonlinear component 1 < a < ∞ the replica-symmetric solution of the variational problem is not correct in some interval of the scaled NSR γ(1) AT , and should be replaced with one involving Q = 0. Nevertheless, asymptotic decay of the quality parameter p for γ → ∞ is always given by Eq.(58), apart from the only limiting case of purely quadratic encryption, when a → ∞.

Solution with fully broken replica symmetry
The goal of the present section is to seek for a solution of the variational problem for the functional Eq.( 14) which breaks the replica symmetry, so that Q > 0. Doing this necessarily implies taking the function w s (u) into account, and deriving the equation involving such a function.The corresponding equation is obtained by requiring stationarity of the functional E with respect to a variation of w s (u), assuming that function to be continuous in the interval q ∈ [R 2 − Q, R 2 ] ¶ .For every value of q in that interval it yields the equation which using again Eq.( 51) can be simplified into Our first observation is that setting q = R 2 in Eq.(60) in fact reproduces Eq.( 52), so the fundamental system comprises three rather than four independent stationarity conditions: Eq.(50), Eq.( 51) and either Eq.(59) or Eq.( 60) .Next we observe that Eq.( 50) can be rewritten as which when substituted to Eq.(51) yields the following equation After introducing the variable p = t/R, and the NSR γ = σ 2 /R 2 the above equation is presented in the Main Results section, see (19).
At the next step we differentiate Eq.(59) over the variable q, and find that for any Now, by comparing Eq.( 63) with Eq.( 59) and assuming that Φ (q) = 0 one arrives to the following relation: We further substitute the value q = R 2 − Q in the above getting which we further rearrange into Finally, upon using Eq.( 61) the above relation is transformed into the following equation: which is yet another equation presented in the Main Results section, see Eq. (20).We therefore conclude that the pair of equations Eq.(62) and Eq.( 67) is sufficient for finding the values of the parameters t and Q, and hence for determining the value of p giving the quality of the reconstruction procedure.
Using the above pair, the first task is to determine the range of NSR parameter γ = σ 2 /R 2 where the solution with Q > 0 is at all possible.The boundary of this region which we denote as γ AT (in the general spin-glass context such boundaries are known as the de-Almeida-Thouless lines [1]) can be found by setting Q = 0 in Eq.(62) and Eq.( 67), yielding the system of two equations: and Moreover, it is not difficult to understand that by replacing in Eq.( 69) the equality sign = with the inequality sign ≤ defines the NSR domain γ ≤ γ AT corresponding to solutions with stable unbroken replica symmetry, Q = 0.

Analysis of Replica Symmetry
Breaking for the linear-quadratic family of encryptions.
In this section we use the following scaling variables naturally arising when performing the analysis of the general case of linear-quadratic family: the scaled NSR γ = σ 2 /J 2 1 R 2 , the variables p = t/R and Q = Q/R 2 and the non-linearity parameter a = (RJ 2 /J 1 ) 2 .

3.3.1.
Position of the de-Almeida Thouless boundary.Not surprisingly, the equation Eq.(68) in scaled variables simply coincides with Eq.(57), which we repeat below for convenience of the exposition: whereas Eq.(69) takes after simple rearrangements the form One can further use Eq.(71) to bring Eq.(70) to a more convenient form explicitly defining γAT as: To find γAT for given values of the parameters µ ≥ 1 and a ≥ 0 one has to find a value p ∈ [0, 1] by solving Eq.( 71), and substitute it to Eq.(72).A simple consideration shows that both sides of eq.( 71), f L (p) = p 2 and f R (p) = µ (1+a) 3 (1 + ap 2 )(1 + ap) 2 are monotonically increasing and convex for p ∈ [0, 1], with the value of the righthand side being larger than the left-hand side at both ends of the interval, see Fig. 3.This implies that generically there must be either no solutions if µ > µ AT (a), or two solutions: 0 ≤ p 2 < p 1 < 1 if µ < µ AT (a).The parameter µ AT (a) is precisely one when only a single solution is possible, and corresponds geometrically to the situation when the curves f L (p) and f R (p) touch each other at some p = p AT ∈ [0, 1], see the Fig. 3 below.The latter can be then found as a solution to the system of two equations: In the former case the two curves intersects in two points p 2 < p 1 , in the latter case the two curves touch each other at p = a −2/3 = 0.25.For µ > (3/2) 3 the two curves do not intersect (not shown).
f L (p) = f R (p) and f L (p) = f R (p) for p AT and µ AT for a given a > 1. Surprisingly, the system can be solved explicitly: A detailed mathematical analysis of the discriminant of the 4th-order polynomial equation Eq.(71) + fully confirms the picture outlined above, giving the explicit criterion for existence of solutions in the (µ, a) parameter plane, cf.Fig. 1: (i) For a given µ > 1 no solutions with p ∈ [0, 1] are possible for a < 1, whereas for a fixed a > 1 no solutions exist for µ > µ AT (a).
(ii) For µ = µ AT (a) there exists a single solution: p = p AT .
Correspondingly, in the case (i) the RS solution is valid for all values of the scaled Noiseto-Signal ratio γ, with the parameter p given by solving the RS equation Eq.(57).In contrast, in the last case (iii) the two solutions give rise to two different AT thresholds in the scaled N RS values: γ(2) AT > γ(1) AT .In other words, for fixed values of parameters µ and a there is generically an interval of NSR's γ(1) AT such that the replicasymmetry is broken inside and preserved for γ outside that interval.
+ I am grateful to Dr. Mihail Poplavskyi for his help with the corresponding analysis.
As is easy to see, for the minimal value ERP µ = 1 and any a > 1 one must have only one solution at the edge of the interval: p = 1, with γAT = 0. Let us increase µ slightly so that µ − 1 1.A simple perturbation analysis then shows that a solution to Eq.(71) close to the interval edge exists, and is given by: We conclude that for a fixed a > 1 and small ERP values µ − 1 1 the replica symmetry is broken for NRS satisfying Finally, one may also consider the AT equations in the limiting case of large nonlinearity a 1 when the quadratic term in the covariance is dominant over the linear term.In this limit one easily finds two solutions of Eq.( 71), given to the leading orders by We see that the ratio γ(1) AT /a remains finite in the limit a → 0, whereas γ(2) AT /a → ∞.To interpret this fact we recall that a → ∞ is equivalent to J 2  1 → 0 at a fixed value of J 2 2 > 0. Then the value γ a ≡ γ J 2 2 := γ.We conclude that the value γ(1) AT /a = (µ−1) 2 µ := γAT must give the value of AT boundary in NRS for a given ERP µ > 1 for the purely quadratic encryption, with the second threshold in this limiting case escaping to infinity and leaving the system in the RSB phase for all γ > (µ−1) 2 µ .This conclusion will be fully confirmed by a detailed analysis of the purely quadratic case given in the next section.
3.3.2.Analysis of solutions with the broken replica symmetry for the linear-quadratic family of encryptions.After getting some understanding of the domain of parameters where replica symmetry is expected to be broken, let us analyse the pair of equations Eq.(62) and Eq.( 67), looking for a solution with 0 < Q ≤ 1.As before introduce p = t/R and Q = Q/R 2 as our main variables of interest.
We start with considering the two limiting cases in the family: that of purely linear scheme with Φ(u) = J 2 1 u and the opposite limiting case of purely quadratic encryption scheme with Φ(u) = J 2 2 u 2 2 .In the former case the previous analysis indicates that only RS solution must be possible.Indeed, we immediately notice that for purely linear scheme Eq.(67) takes the form p 2 = µ which can not have any solution as µ > 1 but p ∈ [0, 1].* .We conclude that a solution with broken replica symmetry Q > 0 does not * One can in fact easily demonstrate that the pair Eq.(62) and Eq.( 67) can not have a real solution for any µ > 0.
exist, so in this case the correct value of p is always given by solving the RS equation Eq.(55), as anticipated.
In the opposite limiting case of purely quadratic encryption we first need to introduce a different scaling for NRS as γ = γ J 2 2 R 2 .Then one may notice that unless p = 0 (which is always a solution) the pair Eq.(62) and Eq.(67) reduces to the form Since p = 0 implies Q = 1, we can further simplify this system and bring it to the form where we introduced, in accordance with the Eq.( 76), At this point we need to recall that the broken replica symmetry corresponds to It is easy to show that the cubic equation in Eq.( 78) may have a positive solution in that interval only for δ > 0. We conclude that the replica symmetry is broken for γ > γAT whereas for γ < γAT the RS solution with Q ≡ 0 and p given by Eq.(57) remains valid.For small 0 < δ 1 one easily finds Q = µ 3(µ−1 )δ + O(δ 2 ) .On the other hand one can see that the solution Q(δ) → 1 as δ → δ c = 3  2 − 1 µ .so that a meaningful solution only exists in the interval δ ∈ [0, δ c ].Moreover, it is easy to show that for δ → δ c we have Q = 1 − 2 3 (δ c − δ).We see that the second of Eq.(78) then implies that when approaching the true threshold value γ(RSB) c = µ − 1 2 dictated by broken replica symmetry the quality parameter vanishes as p ∼ (γ (RSB) c − γ) 3/4 rather than as a square root, as in the replica-symmetric solution Eq.(56).
To study the behaviour of the solution Q(δ) for δ of the order of one it is instructive to consider a particular (but generic) case of µ = 2, when δ c = 1.The cubic equation for Q takes then a particular simple form: which represents one of the rare instances when the Cardano formula for solving cubic equations is really helpful for the analysis.Indeed, according to the Cardano formula in this case the solution is given by The two other solutions of the cubic equation can be shown to be out of the interval (0, 1], see the explicit example below. As δ ∈ [0, 1] we can further parametrize δ = sin φ, φ ∈ [0, π/2], and obtain the three different solutions to Eq.(80) in the following form We see then that Q0,1 are outside the interval [0, 1], as Q0 = 2 cos ], whereas Q2 = 2 sin φ 3 is exactly the valid solution.The nature of the solution for purely quadratic scheme for a general µ > 1 is exactly of the same type.After finding Q from the cubic equation for γAT < γ ≤ γ(RSB) c = µ − 1/2 we find the quality parameter p from the second of Eq.( 78), and combining it with RS expression Eq.( 56) obtain the full corresponding curve for p(γ) for a given µ.In particular, for the above special value µ = 2 the full curve can be described by an explicit expression: and is depicted in the left figure below.For µ = 2 analytic solution of the cubic equation is less instructive, and it is easier to solve the equation numerically.After full understanding of the limiting cases we briefly discuss the solution for a generic linear-quadratic encryption algorithm with some finite value 1 < a < ∞ of the nonlinearity parameter.In this case the NRS scaling is still given by the NRS variable γ = γ/J 2 1 .One recalls that for a given a as long as µ < µ AT (a) = (a 2/3 −a 1/3 +1) AT , γ AT ] the curve p(γ) is probability tending to one is invertable, one can safely expand G ≈ W −1 + λ W −2 + ... and substituting this to Eq.(86) find the first and then second order coefficient in the Lagrange multiplier as: where we introduced the following notations: Using this one can get the following expansion for the quality parameter Eq.( 4): 1−p N = 1 N R 2 σ λ 1 p 1 + s, W −1 A T ξ + σ 2 λ 2 1 p 2 + λ 2 p 1 + λ 1 s, W −2 A T ξ + . . .(89) valid at every realization of both the noise and the random matrix A. Substituting here Eq.( 87) and taking the expected value first only over the Gaussian noise ξ gives after straightforward, but somewhat lengthy manipulations, to the leading order: In particular, for λ = 0 we have lim resulting in Eq.( 10) valid in the first-order in small-noise value.
One can also straightforwardly extract the behaviour for asymptotoically large noise variance values (b, b) ∝ σ 2 → ∞ when Eq.( 9) implies that and further averaging over the Gaussian noise b the above relation yields: It is clear that for γ = σ 2 R 2 1 the relevant value of the Lagrange multiplier λ has to be large in modulus, |λ| 1, which immediately implies in the large-N limit: so that |λ| ≈ √ γµ 1.Now, the Eq.( 9) implies for the quality parameter Expanding for large |λ| ∼ √ γ 1 as shows that to the leading order which upon averaging over the Wishart matrices W and the noise b, taking the limit N → ∞, and taking into account that actually λ → −∞, yields Proof: it is convenient to linearize the squared terms in the exponential by exploiting the Gaussian integration of an auxiliary real variable u a for every a = the integration in the Eq.( 109) goes over the manifold of real symmetric non-negative definite n × n matrices Q and the vector t ∈ R n , whereas the diadic product T = t ⊗ t T is used to denote a (rank one) n × n matrix T with entries T ab = t a t b .
Proof: Denote e N = (0, . . ., 0, 1) the last of the standard basis vectors in R N .Then there exists an orthogonal transformation O(s) ∈ O(N ) such that we can represent the vector s as s = |s|O(s)e N .Perform the transformation of variables Note that the last integral with respect to vectors v a , a = 1, . . ., n has the full O(N −1) invariance of the integrand.The statement of the Theorem then immediately follows by applying to this situation the 'dimensional reduction' formula suggested for the first time in [32] and essentially rediscovered in [15]; see the Appendix D of [21] and the appendix B of [17] for alternative proofs.

1 N
and we define the associated signal strength R via the Euclidean norm as R = (s, s), where (•, •) stands for the Euclidean inner product in R N .By a (symmetric key) encryption of the source signal we understand a random mapping s → y = M known both to the sender and a recipient.For further reference we find it useful to write the mapping component-wise explicitly as

Figure 1 . 3 a
Figure 1.Schematic Phase diagram in (a, µ) plane.In the shaded region of parameters 1 < µ < (a 2/3 −a 1/3 +1) 3 a replica symmetry can be broken for some amplitude of the noise.

Figure 2 .
Figure 2. The quality parameter p as a function of the scaled noise-to-signal ratio γ for purely linear encryptions and two different values of the Encryption Redundancy Parameter, µ = 2 and µ = 4.

Figure 4 .
Figure 4.The quality parameter p as a function of the scaled noise-to-signal ratio γ for purely quadratic encryptions and two different values of the Encryption Redundancy Parameter, µ = 2 (left) and µ = 4 (right).The blue broken curve is the continuation of the replica-symmetric solution in the region of Full RSB.

> 0 holds e − β 2 na=1 V 2
In this Appendix we give a proof of the following Lemma Let x ∈ R N , and V (x) be a Gaussian random field with mean zero and the covarianceV (x 1 )V (x 2 ) = φ (x 1 ; x 2 ) (101)where φ(x; y) = φ(y; x) is any suitable covariance structure function.Then for anyβ (xa) = [det G(x 1 , . . ., x n ] −1/2 (102)where the (positive definite) n × n matrix G(x 1 , . . ., x n ) has the entries G ab (x 1 , . . ., x n ) = δ ab + βφ(x a ; x b ) (103) x a → y a = O(s)x a , ∀a = 1, . . ., n in the integrand of Eq.(108).Such transformation leaves invariant the volume element: a dx a = a dy a and the scalar products Qab = (x a , x b ) = (y a , y b ), 1 ≤ a, b ≤ n but transforms the n projections ta = (x a , s) for all a = 1, . . ., n into ta = |s|y aN , where y aN is the N −th component of the vector y a .Now decompose each vector y a as y a = (v a , y aN ), where v a are (N − 1)-dimensional vectors.Such a decomposition implies: Qab = (y a , y b ) = (v a , v b ) + y aN y bN , a dy a = a dv a a dy N a so that using the notations of the Theorem, renaming y N a → t a and introducing Q (v) ab = (v a , v b ) and t = (t 1 , . . ., t n ) we can rewrite Eq.(108) as I (t; s) dt, I (F ) N,n (t; s) := R N −1 dv 1 ...