Abstract
We present an operator calculus based on Krawtchouk polynomials, including Krawtchouk transforms and corresponding convolution structure which provides an inherently discrete alternative to Fourier analysis. This approach is well suited for applications such as digital image processing. This paper includes the theoretical aspects and some basic examples.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Krawtchouk polynomials are part of the legacy of Mikhail Kravchuk (Krawtchouk), see [19] as a valuable resource about his life and work, including developments up through 2004 based on his work. Krawtchouk polynomials appear in diverse areas of mathematics and science. Important applications such as to image processing may be found in [20].
We cite [1, 16, 18] where Krawtchouk polynomials are used as the foundation for discrete models of quantum physics. And they appear naturally when studying random walks in quantum probability [7, 8].
Multivariate Krawtchouk polynomials play an increasingly important role, in mathematical physics as well as probability and statistics [4, 11]. The approach of [21] on composition Markov chains ties in with the multivariate polynomials as well, cf. [14].
After this Introduction, we continue with the probabilistic construction of Krawtchouk polynomials. They appear as the elementary symmetric functions in the jumps of a random walk, providing a system of martingales based on the random walk. Some fundamental recurrence relations are presented as well. The construction immediately yields their orthogonality relations. Alternative probabilistic approaches to ours of Sect. 2 are to be found in [2, 13,14,15].
Section 3 provides the linearization and convolution formulas that are the core of the paper. They are related to formulas found in [10, 17]. The next section, Sect. 4, specializes to the case of a symmetric random walk, where the formulas simplify considerably.
In Sect.5 we introduce shift operators and use them to develop a computationally effective approach to finding transforms and convolutions. This differs from our principal work with operator calculus [6] and recent approach to Krawtchouk transforms [9] and is suitable for numerical as well as symbolic computations.
The article concludes with Sect. 6 which presents special bases in which the Krawtchouk matrices are anti-diagonal. These basis functions have limited support and look to be useful in implementing filtering methods in the Krawtchouk setting.
2 Combinatorial and probabilistic basis: main features
Consider a collection of N bits \(B=\{0,1\}\) or signs \(S=\{-1,1\}\). Correspondingly, we let j denote the number of 0’s or \(-1\)’s. And we denote the sum in either case by x. So \(x=N-j\) for bits, \(x=N-2j\) for signs. Order the elements of B or S and denote them by \(X_i\), \(1\le i\le N\). We can encode this information in the generating function
Now introduce a binomial probability space with the \(X_i\) a sequence of independent, identically distributed Bernoulli variables. With p the probability of “success”, \(q=1-p\), the centered random variables are distributed as follows:
Bits: \(\displaystyle X_i-p={\left\{ \begin{array}{ll}\ \ q,&{}\text {with probability }p\\ -p,&{}\text {with probability }q \end{array}\right. } \)
Signs: \(\displaystyle X_i-(p-q)={\left\{ \begin{array}{ll}\ \ 2q,&{}\text {with probability }p\\ -2p,&{}\text {with probability }q \end{array}\right. } \)
To get a sequence of orthogonal functionals of the process we redefine
where \(\mu \) is the expected value of \(X_i\).
Remark
Note that this effectively defines the polynomials \(k_n(j,N)\) as the elementary symmetric functions in the variables \(X_i-\mu \). See [12] where this is extended to the multivariate case and [3] where this viewpoint plays an important role in the study of positivity results for bilinear sums of Krawtchouk polynomials.
We see that the two cases differ effectively as a rescaling of v. To see how this comes about, consider general Bernoulli variables \(X_i\) taking values a and b with probabilities p and q respectively. Then the centered variables take values
where \(\lambda =a-b\). We can take as standard model \(b=0\) and \(a=\lambda \). Then
are the mean and variance of \(X_i\). Thus, G has the form
with j counting the number of 0’s and
These are polynomials in the variable j, Krawtchouk polynomials. We define a corresponding matrix
which acts as a transformation on \(\mathbb R^{N+1}\), which we consider as the space of functions defined on the set \(\{0,1,\ldots ,N\}\). The generic form, Eq. (1), is convenient for revealing and proving properties of the Krawtchouk polynomials, and of the transform \(\Phi \).
We review here some principal features of this construction [7, 8].
Remark
Denote expectation with respect to the underlying binomial distribution with angle brackets:
and corresponding inner product \(\langle f,g \rangle =\langle f(X)g(X) \rangle \) .
2.1 Martingale property
Since the \(X_i\) are independent and \(X_i-\mu \) has mean zero, we have the martingale property
where \({\mathcal F}_N\) is the \(\sigma \)-field generated by \(\{X_1,\ldots ,X_N\}\). Thus each coefficient \(k_n(j,N)\) is a martingale, where j denotes the number of 0’s in the random sequence of 0’s and 1’s which is the sample path of the underlying Bernoulli process. This gives immediately
Proposition 2.1
Martingale recurrence
One can derive this purely algebraically by the Pascal recurrences presented in the next paragraph.
2.2 Pascal recurrences and square identity
As is evident from the form of the generating function G, we have recurrences analogous to the Pascal triangle for binomial coefficients.
Proposition 2.2
Pascal recurrences
-
1.
\(k_n(j,N+1)=k_n(j,N)+\lambda q\,k_{n-1}(j,N)\)
-
2.
\(k_n(j+1,N+1)=k_n(j,N)-\lambda p\,k_{n-1}(j,N)\)
These follow directly, first considering \((1+\lambda qv)G_N(v)=G_{N+1}(v)\) and second
Note that the martingale property follows by combining p times the first equation with q times the second.
Given four contiguous entries forming a \(2\times 2\) submatrix of \(\Phi ^{(N)}\), the square identity produces the lower left corner from the other three values. In terms of the k’s:
Proposition 2.3
Square identity
Proof
Combine p times the first equation above with q times that same equation with \(j\rightarrow j+1\). Applying the martingale recurrence on the left-hand side yields
Subtracting off \(p\,k_n( j,N)\) and dividing out a common factor of q yields the result. \(\square \)
2.3 Orthogonality
For orthogonality, we wish to show that \(\langle G(v)G(w) \rangle \) is a function of the product vw only. We have, using independence and centering,
where the variance \(\sigma ^2=\lambda ^2 pq\) in our context. This yields the squared norms
Introducing matrices, we can express the orthogonality relations compactly. Let B, the binomial distribution matrix, be the diagonal matrix
Let \(\Gamma \) denote the diagonal matrix of squared norms,
For fixed N, we write \(\Phi \) for \(\Phi ^{(N)}\) which has ij entry equal to \(k_i(j,N)\). Now \(G(v)=\sum v^i\Phi _{ij}\), and we have
In other words, the orthogonality relation takes the form
which gives for the inverse
In the following sections we will detail linearization formulas for the symmetric and non-symmetric cases, derive the corresponding recurrence formulas and then look at the associated convolution operators on functions.
3 Krawtchouk polynomials: general case
We have the generating function
with j running from 0 to N. The main feature is the relation
where \(X_i\) are independent Bernoulli variables taking values \(\lambda \) and 0 with probabilities p and q respectively.
3.1 Linearization coefficients
We want the expansion of the product \(k_\ell k_m\) in terms of \(k_n\). First, a simple lemma
Lemma 3.1
Let X take values \(\lambda \) and 0. Then the identity
holds.
Proof
It is immediately checked. To derive it, expand \(x(x-\lambda )\) in Taylor series about \(\lambda p\) and equate the result to zero. \(\square \)
In our context, we can write this as
Now multiply
by the Lemma. Factoring out \(1+\sigma ^2vw\) from each term and re-expanding yields
Expanding the coefficient of \(k_n\), we have
Fixing
yields
Theorem 3.2
Linearization formula.
The coefficient of \(k_n\) in the expansion of the product \(k_\ell k_m\) is
Remark
This expansion is interesting as well in the study of bilinear sums of Krawtchouk polynomials, see [3, 5].
3.1.1 Recurrence formula
The three-term recurrence formula characteristic of orthogonal polynomials follows by specializing \(\ell =1\) in the linearization formula. First, compute the constant term and coefficient of v from the generating function G :
From the linearization formula, we pick up three terms, with \(n=m\) and \(n=m\pm 1\). We get
Proposition 3.3
Recurrence formula
The terms \(k_m\) and \(k_{m+1}\) arise with \(\delta =0\), with the term \(k_{m-1}\) the only contribution for \(\delta =1\).
3.2 Krawtchouk transforms: inversion
We identify functions on \(\{0,1,\ldots ,N\}\) with \(\mathbb R^{N+1}\) and the Krawtchouk transforms via the action of the matrix \(\Phi ^{(N)}\) on that space. For given N, \(\Phi \) denotes \(\Phi ^{(N)}\).
For our standard transform, we think of row vectors with multiplication by \(\Phi \) on the right. Thus, the transform F of a function f is given by
where, e.g., \(\mathbf{f}\) is the column vector with entries the corresponding values of f.
The inversion formula is conveniently expressed in terms of matrices.
Proposition 3.4
Let P be the diagonal matrix
Let \(P'\) be the diagonal matrix
Then
The proof is similar to that for orthogonality.
Proof
The matrix equation is the same as the corresponding identity via generating functions. Namely,
First, sum over i, using the generating function G(v), with j replaced by n. Then sum over n, again using the generating function. Finally, summing over j using the binomial theorem yields the desired result, via \(p+q=1\). \(\square \)
Thus,
Corollary 3.5
which is the basis for an efficient inversion algorithm, being a simple modification of the original transform.
3.3 Convolution
Corresponding to the product of two transforms F and G is the convolution of the original functions f and g. We have, following the proof of the linearization formula, Eq. (3),
Thus, we may define the convolution of two functions f and g on \(\{0,1,\ldots ,N\}\) by
and we have the relation
Now, using the inversion formula, Corollary 3.5, we have the relation
for the convolution of functions.
4 Krawtchouk polynomials: symmetric case
For the symmetric case, it is convenient to consider the “signs” process where \(X_i\) takes values \(\pm 1\) with equal probability, \(p=q=1/2\). Thus, \(\lambda =2\) and we have the generating function
While j runs from 0 to N, the sum \(x=N-2j\) runs from \(-N\) to N in steps of 2. Now, \(q-p=0\) and \(\sigma ^2=1\).
In terms of \(x=k_1=N-2j\), write \(k_n(j,N)=K_n(x,N)/n!\). We have the recurrence
with initial conditions \(K_0=1\), \(K_1=x\). For example, we can generate the next few polynomials
The special identities and recurrences hold with \(\lambda =2\), \(p=q=1/2\) and simplify accordingly. Of particular interest is the simplification of the convolution structure.
4.1 Linearization coefficients
We want the expansion of the product \(k_\ell k_m\) in terms of \(k_n\). In Theorem 3.2, since \(q=p\), we have the condition
and the sum over delta disappears. This leads to a particular set of conditions, namely, that the numbers \(\ell \), m, and n satisfy the conditions that they should form the sides of a triangle. So, define the triangle function
where all terms with a factorial must be nonnegative. Note that this is a multinomial coefficient.
Proposition 4.1
In the symmetric case, the expansion of the product \(k_\ell k_m\) is
Alternatively, we have
Remark
If \(\ell +m\ge N\), then the two sides differ by a polynomial vanishing on the spectrum \(\{-N,2-N,\ldots ,N-2,N\}\).
Proof
The “triangular” form follows from the binomial form by rearranging factorials. \(\square \)
4.2 Krawtchouk transforms: inversion
In the symmetric case, the matrices P and \(P'\) in Proposition 3.4 and Corollary 3.5 become identity matrices. Thus, we have
So the inversion is essentially an immediate application of the original transform.
Remark
We note here as well the duality property of suitably scaled Krawtchouk polynomials, invariance under interchange of the index and argument. In the matrix formulation this appears as the property that the matrix \(\Phi B\) is symmetric, where B is the diagonal matrix with the binomial coefficients along the diagonal.
4.3 Convolution
Corresponding to the product of two transforms F and G is the convolution of the original functions f and g. In Eq. (4), the condition \(q-p=0\) entails \(n=\alpha +\beta \). We write a for \(\alpha \), replacing \(\beta =n-a\) and write b for \(\delta \). This gives for the convolution
We have the relation
and the inversion simplifies to
for the convolution of the original functions.
5 Shift operators and matrix formulation of Krawtchouk transform and convolution
We will show how the transform and convolution can be represented by matrices acting on appropriate spaces.
5.1 Transforms
Introduce the shift operator \(T_x\) which acts on a function f(x) by
Similarly, \(T_yf(y)=f(y+1)\) shifts the variable y by 1. For the transform, in the generating function, we replace v by \(T_n\), the matrix shifting the argument n of f:
Representing f by the (column) vector of values
\(T_n\) is represented by the \(N+1\) by \(N+1\) matrix with 1’s on the superdiagonal and zeros elsewhere and the above formula can be computed recursively using matrices of a very simple form. The value F(j) will be the top entry in the resulting vector at each step.
One approach is to form
and compute successively
Form a matrix with these vectors as columns. Then the entries along the top row are the values F(j).
Remark
Even though we are using the column vector \(\mathbf{f}\), we are taking the transform multiplying by \(\Phi \) on the right, that is, computing the entries of \(\mathbf{f}^\dag \Phi \).
Considering vectors \(\mathbf{f}\) with a single nonzero entry equal to one leads to another way to describe the result. Namely, the matrix with successive columns equal to the first row from each of the generated matrices \(U^jT(N)\) produces \(\Phi \). (See “Appendix”.)
Remark
Note that the matrix U has the expansion
This follows from the identity
which may be verified by multiplying both sides by \((I+\lambda qT)\). Expanding in geometric series, noting that T is nilpotent, yields the above formula for U. The coefficients are the entries constant on successive superdiagonals of U.
Example
Let \(N=4\). We have
and
Starting with a column vector \(\mathbf{f}\), first multiplying by T(4), then successively by U produces one-by-one the entries of the transform of \(\mathbf{f}\).
For the symmetric case, \(\lambda =2\), \(p=q=1/2\), T(N) has the binomial coefficients along the superdiagonals while, except for 1’s on the diagonal, the entries of U are \(\pm \,2\) on alternating superdiagonals. Thus,
Similarly, replacing the variables v and w in Eq. (3), by \(T_n\) and \(T_m\) respectively yields the formula
Representing \(T_mT_n\) by the Kronecker/tensor product of the corresponding shift matrices provides an explicit matrix that when applied to the tensor product of the vectors \(\mathbf{f}\) and \(\mathbf{g}\) yields the convolution \(f*g\). (See “Appendix” for an example.)
So the convolution can be computed analogously to the transform. Start with
and compute successively as in Eq. (5).
6 Dual transforms: binomial bases
Of course, one could define transforms dually by multiplying column vectors :
Let’s begin with an example.
Example
For the symmetric case, we observe the result
The matrix on the left is \(\Phi ^{(4)}\). Observe that column, m, say, of binomial coefficients is mapped to its partner column indexed by \(N-m\), scaled by \(2^m\). Note that as functions, functions with zero tails are mapped to functions with zero tails, analogously to Fourier transforms of compactly supported functions or cutoff functions for filtering.
In general we have
Proposition 6.1
Let \(f_m(j)=\displaystyle \left( {\begin{array}{c}m\\ j\end{array}}\right) \,p^{m-j}q^j\). Then the dual transform, \(\mathbf{F}_m =\Phi \mathbf{f}_m\), is given by
Proof
We show the generating function version of the relation. Thus,
\(\square \)
So in this basis, call it the binomial basis, \(\Phi \) is represented by a matrix with entries on the antidiagonal. Continuing our example, write
With D the diagonal matrix \(\mathrm{diag}({1,2,2^2,2^3,2^4})\), we have
with
the matrix representing the transform in the binomial basis. These relations extend to all N.
A related family of transforms is indicated by the similar calculation
which can be expressed in the form
with J, D, and \({\mathcal B}\) as above. Comparing with Eq. (7) indicates a connection between \(\Phi \) and \(\Phi ^T\). At this point it is straightforward to give a direct proof of the properties we want.
Proposition 6.2
Let \(f_i(n)=\displaystyle \left( {\begin{array}{c}N-n\\ N-i\end{array}}\right) \,p^{i-n}\lambda ^{-n}\). Then the transform, \(\mathbf{F}_i^\dag =\mathbf{f}_i^\dag \Phi \), is given by
Proof
As in the previous proposition, we show the generating function version of the relation. Consider
\(\square \)
7 Concluding remarks
We have presented Krawtchouk transforms which have the potential to provide an inherently discrete, efficient alternative to Fourier analysis. By presenting effective algorithms using matrix techniques to compute transforms and convolution products, we have demonstrated useful tools that are not only of theoretical interest but are ready for practical applications. As well, the special binomial transforms we have indicated provide a solid basis for filtering techniques. Thus, the Krawtchouk analogs of the standard Fourier toolkit are now available. Digital image analysis, for example, will provide an important arena for illustrating and developing Krawtchouk methods as presented in this work.
References
Atakishiyev, N.M., Pogosyan, G.S., Wolf, K.B.: Finite models of the oscillator. Phys. Part. Nucl 36(3), 521–555 (2005)
Cooper, R.D., Hoare, M.R.: Stochastic processes and special functions: on the probabilistic origin of some positive kernels associated with classical orthogonal polynomials. J. Math. Anal. Appl. 61(1), 262–291 (1977)
Diaconis, P., Griffiths, R.: Exchangeable pairs of Bernoulli random variables, Krawtchouk polynomials, and Ehrenfest urns. Aust. N. Z. J. Stat. 54(1), 81–101 (2012)
Diaconis, Persi, Griffiths, Robert: An introduction to multivariate Krawtchouk polynomials and their applications. J.Stat. Plan. Inference 154, 39–54 (2014)
Eagleson, G.K.: A characterization theorem for positive definite sequences on the Krawtchouk polynomials. Aust. J. Stat. 11, 29–38 (1969)
Feinsilver, P., Schott, R.: Algebraic Structures and Operator Calculus, vols. I–III: Representations and Probability Theory. Kluwer Academic Publishers, Dordrecht (1993–1995)
Feinsilver, P., Kocik, J.: Krawtchouk matrices from classical and quantum random walks. In: Viana, M.A.G., Richards, D.S.P. (eds.) Algebraic Methods in Statistics and Probability (Notre Dame, IN, 2000). Contemporary Mathematics, vol. 287, pp. 83–96. American Mathematical Society, Providence (2001)
Feinsilver, P., Kocik, J.: Krawtchouk polynomials and Krawtchouk matrices. In: Baeza-Yates, R., Glaz, J., Gzyl, H., Hüsler, J., Palacios, J.L. (eds.) Recent advances in applied probability, pp. 115–141. Springer, New York (2005)
Feinsilver, P., Schott, R.: On Krawtchouk transforms. In: Autexier, S., Calmet, J., et al. (eds) Intelligent Computer Mathematics, 10th International Conference, AISC 2010, 17th Symposium, Calculemus 2010, and 9th International Conference, MKM 2010, Paris, France, July 5–10, 2010. Proceedings, vol. 6167 of Lecture Notes in Artificial Intelligence. Springer, Berlin (2010)
Gasper, George: Nonnegativity of a discrete Poisson kernel for the Hahn polynomials. J. Math. Anal. Appl. 42, 438–451 (1973). Collection of articles dedicated to Salomon Bochner
Genest, V.X., Vinet, L., Zhedanov, A.: The multivariate Krawtchouk polynomials as matrix elements of the rotation group representations on oscillator states. J. Phys. A 46, 50 (2013)
Griffiths, R.C.: Orthogonal polynomials on the multinomial distribution. Aust. J. Stat. 13, 27–35 (1971)
Grünbaum, F.A.: Random walks and orthogonal polynomials: some challenges. In: Probability, geometry and integrable systems, vol. 55 of Math. Sci. Res. Inst. Publ., pp. 241–260. Cambridge University Press, Cambridge (2008)
Hoare, M.R., Rahman, Mizan: Cumulative Bernoulli trials and Krawtchouk processes. Stoch. Process. Appl. 16(2), 113–139 (1984)
Hoare, M.R., Rahman, M.: A probabilistic origin for a new class of bivariate polynomials. SIGMA Symmetry Integra. Geom. Methods Appl., 4:Paper 089, 18 (2008)
Lorente, M.: Quantum Mechanics on discrete space and time. In: Ferrero, M., van der Merwe, A. (eds.) New Developments on Fundamental Problems in Quantum Physics, pp. 213–224. Kluwer, Dordrecht (1997). arXiv:quant-ph/0401004v1
Rahman, Mizan: An elementary proof of Dunkl’s addition theorem for Krawtchouk polynomials. SIAM J. Math. Anal. 10(2), 438–445 (1979)
Santhanam, T.S.: Finite-Space Quantum Mechanics and Krawtchuk Functions. In: Proceedings of the Workshop on Special Functions and Differential Equations, Madras, 1997, 192, Allied Publishers, Delhi (1998)
Virchenko, N., Katchanovski, I., et al., (eds). Development of the Mathematical Ideas of Mykhailo Kravchuk (Krawtchouk), Shevchenko Scientific Society and National Technical University of Ukraine “KPI”. Kyiv-New York (2004)
Yap, P.-T., Paramesran, R.: Image analysis by Krawtchouk moments. IEEE Trans. Image process. 12, 1367–1377 (2003)
Zhou, H., Lange, K.: Composition Markov chains of multinomial type. Adv. Appl. Probab. 41, 270–291 (2009)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Ari Laptev.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Here we show examples of a transform and of a convolution computation using the matrix techniques discussed in the text.
1.1 Krawtchouk transform
For a non-symmetric example, we take \(N=4\), \(\lambda =2\), \(p=1/4\). We have
The matrices \(U^jT(4)\), \(0\le j\le 4\), are successively generated, yielding
and
Think of applying each of these matrices to the column vector consisting of all zeros except for 1 in the \(j\mathrm{th}\) spot. Then the transform would be the \(j\mathrm{th}\) column of \(\Phi \). These come from the successive entries in the top row, column j. In other words, the top row of \(U^jT(N)\) is the \(j\mathrm{th}\) column of \(\Phi \). Concatenating the transposed first rows yields
Forming the diagonal matrices P and \(P'\), as in Proposition 3.4, pre-multiplying by \(\lambda ^{-4}P\) and post-multiplying by \(P'^{-1}\) yields \(\Phi ^{-1}\) immediately.
1.2 Krawtchouk convolution
For an example of convolution, take \(N=2\), \(\lambda =2\), \(p=1/4\), \(\sigma ^2=3/4\). We have the formulas
Let \(T= \left[ {\begin{matrix} 0&{}1&{}0\\ 0&{}0&{}1\\ 0&{}0&{}0\end{matrix}}\right] \). Set
with I the \(3\times 3\) identity.
[Note the tensor sign denotes Kronecker product associated to the left.]
Then form T(2) and U. As for the transform, calculate \(U^jT(N)\) successively. Here we show the top row(s) only, stacked to form a matrix
Multiplying the column \(\mathbf{f} \otimes \mathbf{g}\) on the left by the above matrix produces the convolution \(f\star g\):
As in the previous section, we can compute
We have \(\mathbf{F}^\dag =\mathbf{f}^\dag \,\Phi \) and \(\mathbf{G}^\dag =\mathbf{g}^\dag \,\Phi \), for example,
and similarly for \(\mathbf{G}\). One verifies that the \(j\mathrm{th}\) component of \((\mathbf{{f}}\star \mathbf{{g}}) ^\dag \,\Phi \) is indeed F(j)G(j).
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Feinsilver, P., Schott, R. Krawtchouk transforms and convolutions. Bull. Math. Sci. (2018). https://doi.org/10.1007/s13373-018-0132-2
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s13373-018-0132-2