Signal Detection in Correlated Non-Gaussian Noise Using Higher-Order Statistics

The authors of this paper study the synthesis of new models and methods for signal detection in additive correlated non-Gaussian noise. A new moment quality criterion decision making is proposed based on a random process description using moments and a formation of polynomial decision rules. Taking into account parameters of non-Gaussian distribution of random variables (such as the moments of third and higher orders and joint cumulants), it is shown that nonlinear processing of samples can increase the signal processing efficiency. A synthesis of effective methods and algorithms of data processing in non-Gaussian noise is also presented in this work.


Introduction
High quality signal detection is of great importance for the development of advanced technical systems. The main characteristics of complex signal processing systems are used, for example, in radars, communication, wireless communication, sonar, acoustics and navigation systems. The characteristics of signal processing deteriorate because of the noise effect. Generally, the development of such systems is based on classical methods derived from the theory of statistical hypotheses testing [23]. These methods do not define constraints with respect to the use probability density functions (PDFs) of stochastic processes. In applications, however, the Gaussian PDF is much more widely used. The assumption of the Gaussian PDF is in many cases a convenient mathematical idealization of a real stochastic process, although it does not describe real processes with satisfactory accuracy [8]. The synthesis and analysis of optimal signal detection algorithms in non-Gaussian noise are generally very difficult. Classical methods are characterized by significant limitations associated with the complexity of the algorithmic implementation and the requisite increase in computational resources.
In the scientific literature, two different approaches are typically applied toward the solution of this problem. One approach is based on the use of a PDF for the description of random processes from which the signal detection methods are developed [4][5][6]21]. Despite the fact that PDFs provide a complete description of stochastic processes, these methods have some limitations, and the computational complexity associated with non-Gaussian processes is notable.
Another approach toward describing random processes is based on the use of the moment and cumulant functions. In this case, the properties of decision functions can be described using other characteristics, such as the mean and variance of decision rules (DRs). For example, a deflection criterion was developed for a class of linearquadratic (L-Q) systems [3,18,19]. Further development of this criterion is shown in [2]. It is worth noting that the deflection criterion and its modifications are weakly connected with the classical criteria that are based on the use of PDFs.
This second approach can be represented in the form of higher-order statistics (HOS), see [12], such as moments and cumulant (semivariants) functions [14,20]. Such functions allow the description of the statistical properties of non-Gaussian processes with reasonable accuracy [10,14]. The HOS techniques are used for the development of the signal detection methods [15,22]. However, these methods have some restrictions, for instance the detection of deterministic signals, and the imposition that only thirdorder statistics can be used. A new method was proposed for signal detection based on the use of the moment quality criterion for decision making [13]. This approach has led to improvements in the accuracy of non-Gaussian signal processing relative to traditional methods, along with a reduction in the complexity of signal detection algorithms [16,17]. In those papers, the signal detection methods and algorithms were proposed only for uncorrelated non-Gaussian noise. However, in practical cases, the signal often propagates through turbulent media or along multiple paths. With that in mind, one should model a signal more rigorously in the form of a correlated non-Gaussian process.
The main objective of this paper is the synthesis and analysis of the signal detection methods in correlated non-Gaussian noise based on the moment-cumulant description of random variables and polynomial DRs. This approach seems to optimally satisfy the adapted moment quality criterion of statistical hypotheses testing. Such an approach provides an opportunity to create effective algorithms for the operation of data receiving and processing systems.

Mathematical Models of Correlated Non-Gaussian Processes Using Higher-Order Statistics
A multidimensional (MD) probability density function (PDF) is a complete mathematical description of a statistically dependent stochastic process ξ(t). However, the PDF is not always known, or there may the estimation of its parameters pose a challenge (ϑ 1 , ϑ 2 , . . . , ϑ n ). A different approach is based on the moment and cumulant characteristics [10,12,14,20]. One way to describe statistically dependent random variables involves the use of MD moments and cumulants. The MD moments are defined as the coefficients of the characteristic function f ξ 1 ,ξ 2 ,...,ξ n (ϑ 1 , ϑ 2 , . . . , ϑ n ) and MD cumulants are defined as As noted above, the MD characteristic function, including the MD PDF, is a complete description of the statistically dependent random variables. However, a twodimensional (2D) PDF is often sufficient for describing characteristics of the statistical relationship between random variables [9].
Suppose there are sample values of a stationary random process that can be considered to be different random variables. Then the relationship between two random variables is a simple and widely used example of statistically dependent variables. This is equivalent to using a 2D PDF.
Let's consider the case when two random variables ξ and η are statistically independent of each other with their respective p ξ and p η PDF, that are described as initial moments m (ξ ) i and m (η) i . If the random variables ξ and η are statistically dependent on each other, then they have joint moments of (i, j) dimension. The joint moments of these two random variables are the values that equal the mathematical expectation of the product of the two variables ξ i , η j and are defined as where p(x, y) is a joint PDF of the random variables ξ and η. Joint moments of dimension (i, j) will be used for the description of various statistical relationships. It was shown [10,14] that for non-Gaussian statistically independent random variables with zero mean and variance χ 2 the relationship between the initial moments m i and cumulants χ i up to the fourth order is as follows: Often it is convenient to use dimensionless cumulants called cumulant coefficients γ n = χ n /χ n/2 2 . Using joint moments allows application of the skewness γ 3 and kurtosis γ 4 coefficients to describe characteristics of the non-Gaussian processes. Let us note that for Gaussian random variables the cumulant coefficients of third and higher order (γ 3 , γ 4 , . . .) are equal to zero.
The relationship between the joint moments m i, j and cumulants χ i, j up to the fourth order for the correlated non-Gaussian random variables is defined as The 2D moments can be transformed into 1D moments if random variables are statistically independent.
It is known that the joint second-order cumulant χ 1,1 (covariance) describes statistical relationships of the first-order (or correlation of the random variables) and is defined as where The moment-cumulant description of the correlated non-Gaussian processes requires additional research and development. For this purpose it will be convenient to work with the additional (new) concept of punched random variables developed in [10]. The cumulants of the series expansion of any characteristic function can be separated into various classes in which the characteristic function has similar properties. Non-Gaussian random variables classified as described above are called punched random variables. The mathematical models of random variables were proposed and approved in classes of uncorrelated non-Gaussian random processes using punched random variables (when the moment-cumulant models are represented only by a part of the cumulants from all possible sets that can match the real existing process). Using this punched classification, there are also various skewness, kurtosis, and skewnesskurtosis statistically independent random variables [10]. Definition 1 A variable is said to be a punched random variable if in its cumulant description one part of cumulant coefficients of the 3rd order is distinct from zero and another part is strictly equal to zero. The remaining higher order cumulant coefficients can assume arbitrary values [10].
In this paper we propose development of new moment and cumulant models of the non-Gaussian statistically dependent random variables. On the basis of our approach, it is then possible to create signal detection DRs using the adapted new moment quality criterion for statistical hypothesis testing.
The classification of new mathematical models is obtained from 2D moments and cumulants of non-Gaussian correlated processes. Cumulants and 2D moments depend on the form of the correlation function and samples at times t v and t k : Definition 2 A variable is said to be a Gaussian statistically dependent random variable if its 1D and joint cumulants of the 2nd order are not equal to zero (χ 2 = 0, χ (τ ) 1,1 = 0) while all other cumulants and joint cumulants of third and higher orders are equal to zero.
In this case the initial moments up to the fourth order are as follows: and joint moments and cumulants are defined as: where ρ (τ ) is a correlation function of a given form. For example, the correlation function could be presented in either an exponential ρ (τ ) = e −A|τ | or an alternative form.
Definition 3 A variable is said to be a skewness dependent random variable of the first type and the first kind if the 1D cumulants (χ 2 = 0, χ 3 = 0) and joint cumulants (χ (τ ) 1,2 = 0) are not equal to zero, while the other cumulants and joint cumulants of the higher orders are equal to zero.
In this case the initial moments up to the fourth order are as follows: and joint moments and cumulants are defined as: Definition 4 A variable is called a kurtosis dependent random variable of the first type and the first kind if the 1D cumulants (χ 2 = 0, χ 4 = 0) and joint cumulants (χ (τ ) 2,2 = 0) are not equal to zero. The other cumulants and joint cumulants of higher orders must equal zero.
In this case the initial moments up to the fourth order are as follows: and joint moments and cumulants are defined as: Definition 5 A variable is called a skewness-kurtosis dependent random variable of the second type and the first kind if the 1D cumulants (χ 2 = 0, χ 3 = 0, χ 4 = 0) and joint cumulants (χ (τ ) 2,2 = 0) are not equal to zero. The other cumulants and joint cumulants of the higher orders must equal zero.
In this case, the initial moments up to fourth order are as follows: and joint moments and cumulants are defined as: It should be noted that 1D and MD cumulant coefficients cannot take arbitrary values. This condition is determined by the positive definiteness of the characteristic functions [10]. The proposed models are different from well-known models as they account for the properties of non-Gaussian correlation random processes using higher order cumulant coefficients. These models will be used for the development and adaptation of the new moment quality criterion for statistical hypothesis testing and signal detection methods.

Adaptation of the Moment Quality Criterion Decision Making
Assume that random signals ξ (t) are observed in some time interval (0, T ). We need to define the signal processing algorithms and their characteristics of the input stochastic process ξ (t) based on decision making. A "Yes" signal S(t) (hypothesis H 1 ) or a "No" signal (hypothesis H 0 ), expressed by S(t), in the input stochastic process ξ (t), where ξ(t) = S(t) + η(t); η(t)-is a non-Gaussian stationary random process that describes the sequence of moments and cumulants, and the detection of S(t)-is useful.
Let us use the moment-cumulant description of ξ (t) which is represented as a finite sequence of moments and cumulants. Let us assume that such a sequence has the form (m If the sampling signal is ξ (t), and its discrete values at the time t v are X = {x 1 , x 2 , . . . , x n }, then we have: i, j are non-Gaussian random variables with known parameters in the form of cumulants γ k and joint cumulants χ In the classical approach, the optimal Bayesian algorithm of signal detection is determined as the minimum average risk [23]. The minimal sufficient statistic for simple hypothesis testing is defined as a likelihood ratio, and it can be calculated from The solution of such problems is mainly done under the assumption of a Gaussian PDF of the random variables. In other cases, defining the PDF and finding solutions in the form (19) is difficult. Therefore, it is more convenient to use a different approach when the likelihood ratio is represented as a power polynomial function [16].
Assume that the likelihood ratio is a continuous function that is represented as a stochastic power polynomial of degree s for independent random samples x v : The unknown coefficients k iv and k 0 in (20) can be determined from the minimum of one well-known probabilistic quality criterion (Bayes, Neyman-Pearson etc.), but generally it is not possible to obtain them explicitly. Therefore, a new moment quality criterion for statistical hypothesis testing was proposed in [13].
Let us assume that there is a decision function where γ (X) is a function of sample values X; k 0 is chosen so that According to the Chebyshev inequality, the probability of the first and the second kind errors (21) is defined as: where Then the criterion of the sum of probability errors can be written as the following inequality Let us assume that for M 0 and M 1 the coefficient k 0 is defined as where E i (γ ) = E γ (X) |H i is the mean of the decision function γ (X) with hypotheses H i , i = 0, 1, respectively. Then, the composite function 1 (M, G) for the coefficient k 0 defined above (27), is determined as 1 The functional Ku1(E, G) is the quality criterion of decision making of (21). This criterion is called "The moment quality criterion of upper bounds of error probability" or "Ku criterion" for short.
If we take into account the probability of occurrence of hypotheses H i then (26) can be written as the following inequality where p i = P {H i } is the probability of occurrence of hypotheses H i , Then the composite function 2 (M, G) for such a coefficient k 0 is determined as Let us now take into account the probability of occurrence of hypotheses p i (i = 0, 1) and cost C ij , i, j = 0, 1. Here, the cost C ij is associated with a decision Dec i , given that H j is the true hypothesis [1]. Then, the criterion of probability errors sum given in (26) can be written as the following inequality Using the coefficient k 0 given in (27), the composite function 3 (M, G) is defined as where where b = p 0 (C 01 − C 00 ) , d = p 1 (C 10 − C 11 ). Let us consider the moment quality criterion Ku1(E, G) to demonstrate the efficiency of signal detection. Such a Ku criterion provides the minimum of the sum probability of the first and second kind of errors. The decision rule (DR) will be optimal if the sum of variances for the hypothesis and the alternative is minimal, and the distance between the mean functions is as large as possible.
The new moment quality criterion (28) is different from the well-known probabilistic quality criterion, but it has a definite correlation with them. The Ku criterion is used to create effective methods and algorithms for signal detection in uncorrelated non-Gaussian noise [16,17]. In order to solve the problem of signal processing in correlated non-Gaussian noise, the moment quality criterion (34), defined below, needs to be adapted. This can be done using the new moment and cumulant models that were obtained previously in Sect. 2.
In this case, the adapted criterion should take into account the correlation of sample values. This can be achieved by using 2D cumulant models of the dimension (i, j), where the unknown coefficients k iv and k 0 of the DR are obtained from (20). Then the mean and the variance of the polynomial stochastic DR (20) under our hypotheses and the alternative, are defined as: where the parameters that take into account the correlation relationship are defined as: Taking into account the above equations we conclude that the adapted new moment quality criterion for statistical hypothesis testing is: The optimal coefficients k iv of the polynomial stochastic DR (20) have to minimize the adapted new moment quality criterion Ku1(E, G, ρ) and are defined as Numerical methods and Schur complement of a matrix block are used for solution of algebraic equations (37). This is possible since the 2D joint moments u depend on the correlation function ρ (τ ) which is defined as where, for example, the correlation function has an exponential form ρ (τ v,k The quality criterion in parameter estimation theory is a variance of parameter estimation of random variables. In [11] it was shown that the minimum variance is inversely proportional to the Fisher information function. This information function is defined as a PDF form of the sample values. It is easy to show that the mean and variance of the polynomial stochastic DR (20) can be represented as Kullback-Leibler information number using a PDF for the hypothesis and the alternative. Then the moment quality criterion Ku1(E, G, ρ) decision making can also be defined by using a PDF. Therefore, it is appropriate to introduce the concept of extracted information from samples of volume n related to the difference in hypotheses H 1 and H 0 , just as it would be done in parameter estimation theory.

Definition 6
Inverse value of the moment quality criterion Ku1(E, G, ρ) will be called the extracted information from samples based on discrimination hypotheses H 0 , H 1 , and it will be denoted as Kus .
It can be shown that I Kus can also be defined as Our new method of signal detection in correlated non-Gaussian noise is developed based on new moment-cumulant models and an adapted moment quality criterion for statistical hypotheses testing. This method will be used for synthesis and analysis of the non-linear polynomial stochastic DR.

Synthesis of the Polynomial Algorithms of Signal Detection in Correlated Non-Gaussian Noise
Let us consider the efficiency of the method presented in this paper by using an example of signal detection. Assume that a random signal ξ(t) is observed in the time interval [0, T ] and that it consists of the useful fully known signal a [7] and noise η(t) where η(t) is a non-Gaussian statistically dependent random process with zero mean and variance χ 2 , and that it is described by a sequence of moments and joint cumulants. If the sampling signal is ξ (t) and discrete values at the time t v are X = {x 1 , x 2 , . . . , x n } then we have Let us consider asymmetrical-excess non-Gaussian statistically dependent random variables. Then the initial and joint moments up to the fourth order for the hypothesis H 0 are as follows: and assuming hypothesis H 1 are defined as The polynomial DR (20) of degree s = 1 for signal detection in a correlated non-Gaussian noise is defined as where the unknown coefficient k 0 is defined in (27), and it can be written as Furthermore, using the adapted new moment quality criterion, the unknown coefficients k iv are obtained from the system of equations (37), and can be written as where F Note that then the Eq. (48) is transformed into the form For a linear DR (46), the optimal coefficient is defined as k 1v = q 0.5 A v 1 , where q = a 2 /χ 2 is a signal-to-noise ratio (SNR), A v is a determinant obtained from 1 when the v-th column is replaced by a column consisting of ones only; 1 is as follows: Here, F k) and ρ (v,k) = ρ (τv,k) = e −A|v−k| is the exponential correlation function. Then the linear DR (46) becomes If one takes into account (39) and (40), the value of the information extracted from the samples based on hypotheses H 0 , H 1 , is defined as x v − a 2 It has been shown in [2] that the DR (54) can be obtained from the likelihood ratio under the assumption of non-correlated Gaussian noise. The first and second order moments and joint cumulant χ (τ ) 1,1 are used for the synthesis of the linear DR (51). These parameters are typical characteristics of a Gaussian PDF. Note that the DR (51) does not take into account the non-Gaussian noise distribution. Let us consider the case when the degree of the polynomial is equal to s = 2. The DR will then be non-linear, and in the general case it takes the form where the optimal coefficients, defined from the equation systems (37), are equal to Here B v is a determinant obtained from 2 when v-th column (v = 1, n) is replaced by another one with elements q 0.5 , q 0.5 , . . . , q 0.5 q, q, . . . , q , C v -is defined in a similar way when v = n + 1, 2n and 2 can be written as In general, the threshold k 0 of DR (55) with the coefficients k 1v and k 2v is defined as The DR (55) takes into account the correlated non-Gaussain noise distribution in the form of skewness γ 3 and kurtosis γ 4 coefficients as well as joint cumulants χ (τ ) i, j , i, j = 1, 2.
Thus, for coefficients k 1v and k 2v , the value of the extracted information (discrimination) from samples following from hypotheses H 0 , H 1 , using DR (55), equals Similarly, it is possible to synthesize a non-linear polynomial DR with a higher degree s. The block diagram of the general DR (55) of order s = 2 is shown in Fig. 1. The analysis of the efficiency of the algorithm is obtained from the values of the moment quality criterion Ku1(E, G, ρ) or from the comparison of the extracted information I Kus for the linear and nonlinear DRs of degree s = 1, 2. It will be shown later that the nonlinear processing of samples by the nonlinear DR (s = 2) can increase the signal detection efficiency in correlated non-Gaussian noises.

Results and Discussion
Using a new concept of moment quality criterion and new methods, we have developed non-linear algorithms, computer tools and a new strategy for addressing the problem Rule (DR) Fig. 1 Block diagram of implementation of a power polynomial DR of signal detection in correlated non-Gaussian noise. In addition, we have developed a new generator of correlated non-Gaussian processes to carry out simulation. The generator is based on certain Gaussian Mixture Models (GMM) and an adaptation to correlated processes. The GMM is based on the use of multiple Gaussian generators with given parameters of the PDF and is defined as: where m i , σ 2 i are the mean and the variance of Gaussian components respectively; δ i are the proportionality coefficients of Gaussian components such that they are subject to the condition Consider the two Gaussian components mentioned above. The resulting multidimensional non-Gaussian PDF is then defined as follows where D is the determinant of the n-th order, whose elements are the correlation coefficients, and D μν is an algebraic complement of D.
A block diagram of the Gaussian Mixture generator is presented in Fig. 2. Output random sequence ( x kor ) is generated based on the values of the initial input moment (α 1 , α 2 , . . . , α n ), the number of samples (n), and the correlation function R i j . Blocks of the diagram perform the following functions: (1) is an arithmetic unit for calculation of the distribution of the parameters of Gaussian generators with the number of samples (1 − δ)n going into the unit (2) and the number δn going into block (3); (4) and (5) are multipliers of the random sequences by σ 1 and σ 2 respectively; (6) and (7) are blocks for adding of the random sequences and the means m 1 and m 2 respectively; (8) is a   01 (b, d), γ 3 = 0.8, γ 4 = 0.8 mixer of the two random sequences, (9) is a multiplier of the random sequence and correlation function; (10) is a generator of the correlation relationships of samples, (11) is a cumulative adder and (12) is the "former" of the output random non-Gaussian correlated sequence.
The simulation results of the uncorrelated (a) and correlated (b) non-Gaussian processes are shown in Fig. 3. Correlation fields of these processes are shown in Fig. 3c, d respectively. It can be seen that randomness of the correlated non-Gaussian process is decreased and this is the result of the relationship between sample valu. In this paper we have developed a linear DR of signal detection in correlated non-Gaussian noise. In addition, values of the moment quality criterion Ku1 (E, G, ρ) as inverse of (58) were also obtained. It is shown that the linear adapted DR (57) is transformed into linear classical DR (60) under the assumption of uncorrelated noise (when the parameter A of the exponential correlation function satisfies A > 5). The values of the criteria Ku1 (E, G, ρ) and Ku1 (E, G) are also the same as in uncorrelated noise (Fig. 4).
Using adequate mathematical models of random processes and methods of signal processing allows us to improve the efficiency of signal detection in correlated non-Gaussian noise. The efficiency of the adapted (57) and classic (60) DR is shown in Fig. 5. It is clear that the probability errors of signal detection of the adapted DR are smaller than in the classic DR. The results that are being compared are dependent on the correlation of random process. The efficiency of both DRs is the same for uncorrelated processes, for example when A>5. Figures 6 and 7 show the ratio of the extracted information values I 1 /I 2 in the context of the discrimination of hypotheses H 0 and H 1 from the SNR and skewness coefficientγ 3 . We have used the exponential correlation function for the simulation. It can been seen that the ratio is less than one; therefore, the extracted information value I 2 (65) for a non-linear DR is greater than the extracted information value I 1 (58) for a linear DR. Accordingly, the value of moment quality criterion Ku1 (E, G, ρ) s=2 (DR has degree s = 2) is less than Ku1 (E, G, ρ) s=1 (DR has degree s = 1), and the efficiency of the non-linear DR (61) is better than that of the linear DR (57). Smaller values of the criterion correspond to smaller values for the probability of the errors of the polynomial DR.
As it can be seen for Gaussian noise, when the skewness and kurtosis coefficients are equal to zero (γ 3 = γ 4 = 0), the values of the criterion for s = 1 and s = 2 are the   (Figs. 6, 7). The efficiency of signal processing will improve due to the increase in the degree of the polynomial DR. For convenience we write "gam3" and "gama4" instead of γ 3 and γ 4 in (Figs. 6, 7). It has been shown in this paper that a significant improvement in signal processing efficiency is obtained for small values of parameter q (SNR) and for the boundary values of the asymmetry coefficient γ 3 (γ 3 ≤ √ γ 4 + 2) [10,14]. Research also focused on other types of correlated noise: asymmetrical, excess and asymmetrical-excess non-Gaussian. For these cases, the efficiency of signal processing improved in comparison to the well-known results under the assumption of Gaussian noise.

Conclusions
The complexity associated with description of non-Gaussian processes in the theory of signal processing requires a new approach toward solving the problems of signal detection. The approach described herein is based on the application of the momentcumulant function of random processes and moment criterion quality for the statistical hypotheses testing. New mathematical models of correlated non-Gaussian processes have been developed. An adaptation of the moment quality criterion of upper bounds of error probability was also proposed. Furthermore, power polynomial algorithms were developed for the correlated non-Gaussian processes based on the new method. This approach enables description of the characteristics of correlated non-Gaussian stochastic processes while taking into account the cumulant coefficients of the third and higher orders as well as joint cumulants. An appropriate description of such processes and a non-linear polynomial processing of sample values under DR (s = 2) allow the signal detection efficiency to be increased relative to the well-known results.