1 Introduction

The integral transforms are important mathematical tools used in many fields such as vibration analysis, sound engineering, and image processing. They have wide applications in physics, engineering, and in other scientific and mathematical disciplines. The study of orthogonal polynomials and their transformations have been the subject of many papers during the last several years. By the Fourier transform or other integral transforms, some univariate orthogonal polynomials systems which are mapped onto each other exist [1]. As an example, it is known that Hermite functions which are multiplied Hermite polynomials \(H_{n}\left( x\right) \) by \(\exp \left( -x^{2}/2\right) \) are eigenfunctions of Fourier transform [2,3,4,5]; similarly, Koelink [2] has showed that Jacobi polynomials are mapped onto the continuous Hahn polynomials using the Fourier transform. Also, in [5], it is seen that classical Jacobi polynomials can be mapped onto Wilson polynomials by the Fourier–Jacobi transform. By inspired of Koelink’s paper, in [6], Masjed-Jamei et al. have introduced two new families of orthogonal functions using Fourier transforms of the generalized ultraspherical polynomials and the generalized Hermite polynomials. In [7], the Fourier transform of Routh–Romanovski polynomials \(J_{n}^{\left( u,v\right) }\left( x;a,b,c,d\right) \) has been obtained. Furthermore, in [3, 8], four new examples of finite orthogonal functions have been derived by use of the Fourier transforms of the finite classical orthogonal polynomials \(M_{n}^{\left( p,q\right) }\left( x\right) \) and \(N_{n}^{\left( p\right) }\left( x\right) \), and two symmetric sequences of finite orthogonal polynomials and the Parseval’s identity (see [9,10,11] for details of the finite orthogonal polynomials and symmetric sequences of finite orthogonal polynomials). Tratnik [12, 13] presented multivariable generalization both of all continuous and discrete families of the Askey tableau, providing hypergeometric representation, orthogonality weight function which applies with respect to subspaces of lower degree and biorthogonality within a given subspace. In [14], Koelink et al. gave a non-trivial interaction for multivariable continuous Hahn polynomials. Recently, Güldoğan et al. [15] have obtained new families of orthogonal functions by means of Fourier transforms of bivariate orthogonal polynomials introduced by Koornwinder [16] and Fernández et al. [17].

In the present paper, we study the Fourier transformations of the classical polynomials orthogonal by means of the extension of Jacobi weight function to several variables on the simplex \(T^{r}\). When \(r=1\), the simplex \(T^{r}\) becomes the interval [0, 1] and the corresponding orthogonal polynomials are Jacobi polynomials \(P_{n}^{\left( \alpha ,\beta \right) }\left( 2x-1\right) \) on the interval [0, 1]. In this case, Koelink [2] has derived the Fourier transform of certain Jacobi polynomials \(P_{n}^{\left( \alpha ,\beta \right) }\left( x\right) \) on the interval \([-1,1]\) in terms of continuous Hahn polynomials and has discussed some applications. The Fourier transforms of some other bivariate orthogonal polynomials as well as orthogonal polynomials on the triangle have been studied by Güldoğan et al. [15]. By the motivation of these papers, the main aim is to produce a new family of multivariate orthogonal functions.

In [12], Tratnik obtained multivariate orthogonal polynomials on the simplex by taking limit of continuous multivariate Hahn polynomials. As different from the results in Tratnik’s paper, in this paper, we handle multivariate orthogonal polynomials on the simplex, and using Fourier transforms of some special orthogonal functions in terms of these polynomials, we get some special orthogonal functions in terms of continuous Hahn polynomials using the method in [2, 15]. While doing these, first, we define specific special functions, so that they are determined with the motivation to use the orthogonality relation of orthogonal polynomials on the simplex in Parseval’s identity created with the help of Fourier transform. To give the results on \(T^{r}\), we shall proceed by induction and we first discuss the results for \(r=1\) and \(r=2\).

The manuscript is organized as follows: in Sect. 2, we introduce some basic definitions and notations. The main results are stated in the third section, where we will obtain Fourier transforms of the multivariate orthogonal polynomials on the r-simplex, write them in terms of continuous Hahn polynomials, define a new family of multivariate orthogonal functions using the Parseval’s identity, and derive several recurrence relations for the new family of multivariate orthogonal functions. Finally, the proofs of the main results are given in the fourth section.

2 Basic Definitions and Notations

The univariate Jacobi polynomials \(P_{n}^{\left( \alpha ,\beta \right) }\left( x\right) \) are defined by the explicit representation

$$\begin{aligned} P_{n}^{\left( \alpha ,\beta \right) }\left( x\right) =2^{-n}{\displaystyle \sum \limits _{k=0}^{n}} \left( {\begin{array}{c}n+\alpha \\ k\end{array}}\right) \left( {\begin{array}{c}n+\beta \\ n-k\end{array}}\right) \left( x+1\right) ^{k}\left( x-1\right) ^{n-k}. \end{aligned}$$
(2.1)

For \(\alpha ,\beta >-1\) they are orthogonal on the interval \(\left[ -1,1\right] \) with respect to the weight function \(w\left( x\right) =\left( 1-x\right) ^{\alpha }\left( 1+x\right) ^{\beta }\) [19,   p. 68, Eq. (4.3.2)]

$$\begin{aligned}&\int \limits _{-1}^{1}\left( 1-x\right) ^{\alpha }\left( 1+x\right) ^{\beta }P_{n}^{\left( \alpha ,\beta \right) }\left( x\right) P_{m}^{\left( \alpha ,\beta \right) }\left( x\right) \mathrm{d}x \\&\quad =\frac{2^{\alpha +\beta +1}\Gamma \left( \alpha +n+1\right) \Gamma \left( \beta +n+1\right) }{n!\left( \alpha +\beta +2n+1\right) \Gamma \left( \alpha +\beta +n+1\right) } \delta _{m,n}, \end{aligned}$$

where \(m,n\in \mathbb {N}_{0}:=\mathbb {N}\cup \left\{ 0\right\} \), \(\delta _{m,n}\) is the Kronecker delta, and \(\Gamma (\lambda )\) is the Gamma function (cf. [20]) defined by

$$\begin{aligned} \Gamma \left( \lambda \right) ={\displaystyle \int \limits _{0}^{\infty }}x^{\lambda -1}e^{-x}\mathrm{d}x, \quad \Re \left( \lambda \right) >0. \end{aligned}$$
(2.2)

The (p, q) generalized hypergeometric function is defined by (cf. [20])

$$\begin{aligned} \,{}_{p}F_{q}\left( \begin{array}{cc}{\displaystyle {a_{1},a_{2},\dots ,a_{p}}}\\ [-0.1ex] {\displaystyle {b_{1},b_{2},\dots ,b_{q}}} \end{array}\Big | \,{\displaystyle {x}} \right) ={\displaystyle \sum \limits _{k=0}^{\infty }} \frac{\left( a_{1}\right) _{k}\left( a_{2}\right) _{k} \cdots \left( a_{p}\right) _{k}}{\left( b_{1}\right) _{k}\left( b_{2}\right) _{k} \cdots \left( b_{q}\right) _{k}}\frac{x^{k}}{k!}, \end{aligned}$$
(2.3)

where \(\left( a\right) _{k}={ \prod _{r=0}^{k-1}} \left( a+r\right) \), \(\left( a\right) _{0}=1\), \(k\in \mathbb {N}_{0}\), is the Pochhammer symbol, and the beta function is given by (cf. [20])

$$\begin{aligned} B\left( a,b\right) ={\displaystyle \int \limits _{0}^{1}}x^{a-1}\left( 1-x\right) ^{b-1}\mathrm{d}x=\frac{\Gamma \left( a\right) \Gamma \left( b\right) }{\Gamma \left( a+b\right) }, \qquad \Re \left( a\right) ,\Re \left( b\right) >0. \end{aligned}$$
(2.4)

The continuous Hahn polynomials are defined in terms of hypergeometric series as [21]

$$\begin{aligned}&p_{n}\left( x;a,b,c,d\right) \nonumber \\&\quad =i^{n}\frac{\left( a+c\right) _{n}\left( a+d\right) _{n}}{n!} \,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {-n,\ n+a+b+c+d-1,\ a+ix}}\\ [-0.1ex] {\displaystyle {a+c,\ a+d}} \end{array}\Big | \,{\displaystyle {1}} \right) . \end{aligned}$$
(2.5)

These polynomials can be presented as a limiting case of the Wilson polynomials [21]. Koelink [2] gave a nice survey of the history this family of orthogonal polynomials. Recently, in [22], some applications of continuous Hahn polynomials to quantum mechanics have been presented.

Let \(\left| \varvec{x}\right| :=x_{1}+\cdots +x_{r}\) for \(\varvec{x}=\left( x_{1},\dots ,x_{r}\right) \in \mathbb {R}^{r}\). Let \(V_{n}^{r}\) be the linear space of orthogonal polynomials respect to the weight function \(W_{\varvec{\alpha }}\left( \varvec{x}\right) =x_{1}^{\alpha _{1}} \cdots x_{r}^{\alpha _{r}}\left( 1-\left| \varvec{x}\right| \right) ^{\alpha _{r+1}}\) , \(\alpha =(\alpha _{1},\dots ,\alpha _{r+1})\), \(\alpha _{i}>-1\), on the simplex \(T^{r}=\left\{ \varvec{x}\in \mathbb {R} ^{r}:x_{1}\ge 0,\dots ,x_{r}\ge 0,1-\left| \varvec{x}\right| \ge 0\right\} \). The elements of the linear space \(V_{n}^{r}\) satisfy the following partial differential equation (cf. [23]):

$$\begin{aligned}&\sum \limits _{i=1}^{r}x_{i}\left( 1-x_{i}\right) \frac{\partial ^{2} P}{\partial x_{i}^{2}}-2\sum \limits _{1\le i<j\le r}x_{i}x_{j}\frac{\partial ^{2}P}{\partial x_{i}\partial x_{j}} \\&\quad +\sum \limits _{i=1}^{r}\left( \left( \alpha _{i}+1\right) -\left( \left| {\varvec{\alpha }}\right| +r+1\right) x_{i}\right) \frac{\partial P}{\partial x_{i}} =-n\left\{ n+\left| {\varvec{\alpha }}\right| +r+1\right\} P, \end{aligned}$$

which has \(\genfrac(){0.0pt}0{n+r-1}{n}\) linear independent polynomial solutions of total degree n. In here, \(\left| {\varvec{\alpha }}\right| =\alpha _{1}+\cdots +\alpha _{r+1}\). The space \(V_{n}^{r}\) has several different bases. One of these bases can be introduced as follows. Let \(\mathbf {x}_{j}\) and \(\mathbf {n}^{j}\) be defined as

$$\begin{aligned} \varvec{x}_{0}=0,\quad \varvec{x}_{j}=\left( x_{1},\dots ,x_{j}\right) , \quad \left| \varvec{x}_{j}\right| =x_{1}+\cdots +x_{j}, \\ \mathbf {n}^{j}=\left( n_{j},\dots ,n_{r}\right) , \quad \left| \mathbf {n}^{j}\right| =n_{j}+\cdots +n_{r}, \quad 1\le j\le r, \end{aligned}$$

and \(\mathbf {n}^{r+1} :=0\).

Also, let \(\mathbf {n}=\left( n_{1},\dots ,n_{r}\right) \), \(\left| \mathbf {n}\right| =n_{1}+\cdots +n_{r}=n\), \({\varvec{\alpha }}=\left( \alpha _{1},\dots ,\alpha _{r+1}\right) \), and \({\varvec{\alpha }}^{j}=\left( \alpha _{j},\dots ,\alpha _{r+1}\right) \), \(\left| {\varvec{\alpha }}^{j}\right| =\alpha _{j}+\cdots +\alpha _{r+1}\), \(1\le j\le r+1\). In this case, an orthogonal base of the space \(V_{n}^{r}\) is of form (cf. [23])

$$\begin{aligned} P^{\left( \alpha \right) }_{\mathbf {n}}\left( \varvec{x}\right) =\prod \limits _{j=1}^{r}\left( 1-\left| \varvec{x}_{j-1}\right| \right) ^{n_{j}}P_{n_{j}}^{\left( a_{j},b_{j}\right) }\left( \frac{2x_{j}}{1-\left| \varvec{x}_{j-1}\right| }-1\right) , \end{aligned}$$
(2.6)

where \(P_{n_{j}}^{\left( a_{j},b_{j}\right) }\left( x_{j}\right) \) indicates the classical univariate Jacobi polynomial defined in (2.1), \(a_{j}=2\left| \mathbf {n}^{j+1}\right| +\left| {\varvec{\alpha }}^{j+1}\right| +r-j\) and \(b_{j}=\alpha _{j}\). It follows:

$$\begin{aligned} {\displaystyle \int \limits _{T^{r}}}W_{\alpha }\left( \varvec{x}\right) P_{\mathbf {n}}^{\left( {\alpha }\right) }\left( \varvec{x}\right) P_{\mathbf {m}}^{\left( {\alpha }\right) }\left( \varvec{x}\right) \mathbf {\mathrm{d}x=}h_{\mathbf {n}}^{\left( {\alpha }\right) }\delta _{\mathbf {n,m}}, \end{aligned}$$
(2.7)

where \(\mathbf {\mathrm{d}x=}\mathrm{d}x_{1}\cdots \mathrm{d}x_{r}\), and

$$\begin{aligned} h_{\mathbf {n}}^{\left( {\alpha }\right) }&= \prod \limits _{j=1}^{r}\frac{\Gamma \left( 2\left| \mathbf {n}^{j+1}\right| +\left| {\varvec{\alpha }}^{j+1}\right| +r-j+1+n_{j}\right) }{n_{j}!\left( 2\left| \mathbf {n}^{j+1}\right| +\left| {\varvec{\alpha }}^{j+1}\right| +r-j+1+\alpha _{j}+2n_{j}\right) } \nonumber \\&\quad \times \frac{ \Gamma \left( 1+\alpha _{j}+n_{j}\right) }{\Gamma \left( 2\left| \mathbf {n}^{j+1}\right| +\left| {\varvec{\alpha }}^{j+1}\right| +r-j+1+\alpha _{j}+n_{j}\right) } . \end{aligned}$$
(2.8)

When \(r=1\), the simplex \(T^{r}\) becomes the interval [0, 1] and the corresponding orthogonal polynomials in (2.6) are the Jacobi polynomials

$$\begin{aligned} P_{n}^{\left( \alpha \right) }\left( x\right) := P_{n}^{\left( \alpha _{2}, \alpha _{1} \right) }\left( 2x-1\right) . \end{aligned}$$

3 Main Results

Our main results are to obtain the Fourier transformation of the classical orthogonal polynomials on the simplex \(T^{r}\), to define a new family of multivariate orthogonal functions in terms of multivariable Hahn polynomials and, to derive several recurrence relations. Let us consider the function

(3.1)

for \(r\ge 1\), where

$$\begin{aligned}&\Upsilon _{1}(x_{1})=\Upsilon _{1}= \frac{ 1+\tanh x_{1}}{2}, \\&\Upsilon _{r}(x_{1},\dots ,x_{r})\!=\!\Upsilon _{r} \!= \!\frac{\left( 1-\tanh x_{1}\right) \left( 1-\tanh x_{2}\right) \cdots \left( 1-\tanh x_{r-1}\right) \left( 1+\tanh x_{r}\right) }{2^{r}}, \end{aligned}$$

for \(r\ge 2\) and

$$\begin{aligned} \mathbf {n}=\left( n_{1},\dots ,n_{r}\right) ,\ {\varvec{\alpha }}=\left( \alpha _{1},\dots ,\alpha _{r+1}\right) , \quad \text { and }\ \left| \mathbf {a}^{j}\right| =a_{j}+\cdots +a_{r+1}. \end{aligned}$$

For \(r=1\), it gives

$$\begin{aligned} g_{1}\left( x;n,a_{1},a_{2},\alpha _{1},\alpha _{2}\right) =\left( 1+\tanh x\right) ^{a_{1}}\left( 1-\tanh x\right) ^{a_{2}} P_{n}^{\left( \alpha _{1},\alpha _{2}\right) }\left( \Upsilon _{1}\right) , \end{aligned}$$
(3.2)

where \(a_{1},a_{2},\alpha _{1},\) and \(\alpha _{2}\) are real parameters and

$$\begin{aligned} \Upsilon _{1}(x)=\Upsilon _{1}= \frac{1+\tanh x}{2}. \end{aligned}$$

From (3.1), we can write \(g_{r+1}\) in terms of \(g_{r}\)

$$\begin{aligned}&g_{r+1}\left( x_{1},\dots ,x_{r+1};n_{1},\dots ,n_{r+1},a_{1},\dots ,a_{r+2},\alpha _{1},\dots ,\alpha _{r+2}\right) \nonumber \\&\quad =2^{-rn_{r+1}}\left( 1+\tanh x_{r+1}\right) ^{a_{r+1}}\left( 1-\tanh x_{r+1}\right) ^{a_{r+2}}P_{n_{r+1}}^{\left( \alpha _{r+2},\alpha _{r+1}\right) }\left( \tanh x_{r+1}\right) \nonumber \\&\qquad \times g_{r}\left( x_{1},\dots ,x_{r};n_{1},\dots ,n_{r},a_{1},\dots ,a_{r},a_{r+1}+a_{r+2}+n_{r+1}, \right. \nonumber \\&\qquad \left. \alpha _{1} ,\dots ,\alpha _{r},\alpha _{r+1}+\alpha _{r+2}+2n_{r+1}+1\right) , \end{aligned}$$
(3.3)

where \(P_{n}^{\left( \alpha _{1},\alpha _{2}\right) }\left( x\right) \) stands for classical univariate Jacobi polynomials defined in (2.1).

3.1 The Fourier Transform of r-Dimensional Jacobi Polynomials on r-Simplex

The Fourier transform for a function g(x) is defined as ([24,   p. 111, Eq. (7.1)])

$$\begin{aligned} \mathcal {F} \left( g\left( x\right) \right) =\int \limits _{-\infty }^{\infty }e^{-i\xi x}g\left( x\right) \mathrm{d}x, \end{aligned}$$
(3.4)

and the Fourier transform for a function \(g(x_{1},\dots ,x_{r})\) in r variables is defined as ([24, p. 182, Eq. (11.1a)])

$$\begin{aligned} \mathcal {F} \left( g\left( x_{1},\dots ,x_{r}\right) \right) =\int \limits _{-\infty }^{\infty }\dots \int \limits _{-\infty }^{\infty }e^{-i\left( \xi _{1}x_{1}+\cdots +\xi _{r}x_{r}\right) }g\left( x_{1},\dots ,x_{r}\right) \mathrm{d}x_{1} \dots \mathrm{d}x_{r}.\qquad \quad \end{aligned}$$
(3.5)

Theorem 3.1

The following result holds true:

$$\begin{aligned}&\mathcal {F}\left( g_{r+1}\left( x_{1},\dots ,x_{r+1};n_{1},\dots ,n_{r+1},a_{1},\dots ,a_{r+2},\alpha _{1},\dots ,\alpha _{r+2}\right) \right) \nonumber \\&\quad =2^{a_{r+1}+a_{r+2}-rn_{r+1}-1}\frac{\left( \alpha _{r+2}+1\right) _{n_{r+1}} }{n_{r+1}!} B\left( a_{r+1}-\frac{i\xi _{r+1}}{2},a_{r+2}+\frac{i\xi _{r+1}}{2}\right) \nonumber \\&\qquad \times \,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {-n_{r+1},\ n_{r+1}+\alpha _{r+1}+\alpha _{r+2}+1,\ a_{r+2}+\frac{i\xi _{r+1}}{2}}}\\ [-0.1ex] {\displaystyle {\alpha _{r+2}+1,\ a_{r+1}+a_{r+2} }} \end{array}\Big | \,{\displaystyle {1}} \right) \nonumber \\&\qquad \times \mathcal {F} \left( g_{r}\left( x_{1},\dots ,x_{r};n_{1},\dots ,n_{r},a_{1},\dots ,a_{r},a_{r+1}+a_{r+2}+n_{r+1}, \right. \right. \nonumber \\&\qquad \left. \left. \alpha _{1},\dots ,\alpha _{r},\alpha _{r+1}+\alpha _{r+2}+2n_{r+1}+1\right) \right) . \end{aligned}$$
(3.6)

Theorem 3.2

By substituting Fourier transform of the specific function in the right-hand side of the equality (3.6), we have

$$\begin{aligned}&\mathcal {F} \left( g_{r}\left( \varvec{x};\mathbf {n},\mathbf {a},{\varvec{\alpha }} \right) \right) =2^{r\left( {a}_{r}+a_{r+1}-1 \right) +\sum _{j=1}^{r-1}ja_{j}} \nonumber \\&\quad \times \prod \limits _{j=1}^{r}\left\{ \frac{\left( 2\left| \mathbf {n}^{j+1}\right| +\left| {\varvec{\alpha }}^{j+1}\right| +r-j+1\right) _{n_{j}} }{n_{j}!} \Lambda _{j}^{r}(\mathbf {a},{\varvec{\alpha }},\mathbf {n};\xi _{j}) \right\} , \end{aligned}$$
(3.7)

where

$$\begin{aligned}&\Lambda _{j}^{r}(\mathbf {a},{\varvec{\alpha }},\mathbf {n};\xi _{j})=B\left( a_{j}-\frac{i\xi _{j}}{2},\ \left| \mathbf {n}^{j+1}\right| +\left| \mathbf {a}^{j+1}\right| +\frac{i\xi _{j}}{2}\right) \\&\quad \times \,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {-n_{j},\ n_{j}+2\left| \mathbf {n}^{j+1}\right| +\left| {\varvec{\alpha }}^{j}\right| +r-j+1,\ \left| \mathbf {n}^{j+1}\right| +\left| \mathbf {a}^{j+1}\right| +\frac{i\xi _{j}}{2}}}\\ [-0.1ex] {\displaystyle {2\left| \mathbf {n}^{j+1}\right| +\left| {\varvec{\alpha }}^{j+1}\right| +r-j+1,\ \left| \mathbf {n}^{j+1}\right| +\left| \mathbf {a}^{j}\right| }} \end{array}\Big | \,{\displaystyle {1}} \right) , \end{aligned}$$

or, in terms of the continuous Hahn polynomials (2.5)

$$\begin{aligned}&\Lambda _{j}^{r}(\mathbf {a},{\varvec{\alpha }},\mathbf {n};\xi _{j})= \frac{n_{j}!B\left( a_{j}-\frac{i\xi _{j}}{2},\ \left| \mathbf {n}^{j+1}\right| +\left| \mathbf {a}^{j+1}\right| +\frac{i\xi _{j}}{2}\right) }{i^{n_{j}}\left( \left| \mathbf {n}^{j+1}\right| +\left| \mathbf {a}^{j}\right| \right) _{n_{j}}\left( 2\left| \mathbf {n}^{j+1}\right| +\left| \alpha ^{j+1}\right| +r-j+1\right) _{n_{j}}} \\&\quad \times p_{nj}\left( \frac{\xi _{j}}{2};\left| \mathbf {n}^{j+1}\right| +\left| \mathbf {a}^{j+1}\right| ,\alpha _{j}-a_{j}+1,\left| \mathbf {n}^{j+1}\right| +\left| \alpha ^{j+1}\right| -\left| \mathbf {a}^{j+1}\right| +r-j+1,a_{j}\right) , \end{aligned}$$

\( g_{r}\left( \varvec{x};\mathbf {n},\mathbf {a},{\varvec{\alpha }} \right) \) is the function given in (3.1) for \(r\ge 1\), and

$$\begin{aligned} \pmb {x}= & {} \left( x_{1},\dots ,x_{r}\right) , \quad \mathbf {n}=\left( n_{1},\dots ,n_{r}\right) , \quad \mathbf {n}^{j}=\left( n_{j},\dots ,n_{r}\right) , \\ \left| \mathbf {n}^{j}\right|= & {} n_{j}+\cdots +n_{r},\quad {\varvec{\alpha }} =\left( \alpha _{1},\dots ,\alpha _{r+1}\right) ,\quad {\varvec{\alpha }}^{j}=\left( \alpha _{j},\dots ,\alpha _{r+1}\right) , \\ \mathbf {a}= & {} \left( a_{1},\dots ,a_{r+1}\right) , \mathbf {a}^{j}=\left( a_{j},\dots ,a_{r+1}\right) , \quad \left| \mathbf {a}^{j}\right| =a_{j}+\cdots +a_{r+1}. \end{aligned}$$

3.2 The Class of Special Functions Using Fourier Transform of r-Dimensional Polynomials on the r-Simplex

The Parseval’s identity corresponding to (3.4) is given by the statement [24,   p. 118, Eq. (7.17)]

$$\begin{aligned} \int \limits _{-\infty }^{\infty }g\left( x\right) \overline{h\left( x\right) }\mathrm{d}x=\frac{1}{2\pi }\int \limits _{-\infty }^{\infty }\mathcal {F}\left( g\left( x\right) \right) \overline{\mathcal {F}\left( h\left( x\right) \right) }\mathrm{d}\xi , \end{aligned}$$

and the Parseval’s identity corresponding to (3.5) is given by [24,   p. 183, (iv)]

$$\begin{aligned}&\int \limits _{-\infty }^{\infty }\cdots \int \limits _{-\infty }^{\infty }g\left( x_{1},\dots ,x_{r}\right) \overline{h\left( x_{1},\dots ,x_{r}\right) }\mathrm{d}x_{1}\dots \mathrm{d}x_{r}\nonumber \\&\quad =\frac{1}{\left( 2\pi \right) ^{r}}\int \limits _{-\infty }^{\infty }\dots \int \limits _{-\infty }^{\infty }\mathcal {F}\left( g\left( x_{1},\dots ,x_{r}\right) \right) \overline{\mathcal {F}\left( h\left( x_{1},\dots ,x_{r}\right) \right) }\mathrm{d}\xi _{1}\cdots \mathrm{d}\xi _{r}. \end{aligned}$$
(3.8)

Theorem 3.3

Let be \(\varvec{n}:=\left( n_{1},n_{2},\dots ,n_{r}\right) \), \(\varvec{m}:=\left( m_{1},m_{2},\dots ,m_{r}\right) \), \(\left| \mathbf {n}^{j}\right| =n_{j}+n_{j+1}+\dots +n_{r}\) and \(\varvec{x} :=\left( x_{1},x_{2},\dots ,x_{r}\right) \) for \(\varvec{x}\in \mathbb {R}^{r}\). In here, \(\mathbf {a}:=\left( a_{1},a_{2},\dots ,a_{r+1}\right) \) and \(\mathbf {b}:=\left( b_{1},b_{2},\dots ,b_{r+1}\right) . \) Let

$$\begin{aligned} \mathbf {a}^{j}&=\left( a_{j},\dots ,a_{r+1}\right) ,\ \ \ \ 1\le j\le r+1,\\ \mathbf {b}^{j}&=\left( b_{j},\dots ,b_{r+1}\right) ,\ \ \ \ 1\le j\le r+1, \end{aligned}$$

so \(\left| \mathbf {a}^{j}\right| =a_{j}+a_{j+1}+\cdots +a_{r+1}\) and \(\left| \mathbf {b}^{j}\right| =b_{j}+b_{j+1}+\cdots +b_{r+1}\). The following equality is satisfied:

$$\begin{aligned}&\int \limits _{-\infty }^{\infty }\cdots \int \limits _{-\infty }^{\infty }W_{r}\left( \varvec{x},\mathbf {a},\mathbf {b}\right) ~_{r}S_{\mathbf {n}}\left( i\varvec{x};\mathbf {a},\mathbf {b}\right) ~ _{r}S_{\mathbf {m}}\left( -i\varvec{x};\mathbf {b},\mathbf {a}\right) \mathbf {\mathrm{d}x}\\&\quad =2^{2r}\pi ^{r}h_{\mathbf {n}}^{\left( \mathbf {a+b-1}\right) } \prod \limits _{j=1}^{r} \frac{\left( n_{j}!\right) ^{2} \Gamma \left( \left| \mathbf {n} ^{j+1}\right| +\left| \mathbf {a}^{j}\right| \right) \Gamma \left( \left| \mathbf {n}^{j+1}\right| +\left| \mathbf {b}^{j}\right| \right) }{\left( 2\left| \mathbf {n} ^{j+1}\right| +\left| \mathbf {a} ^{j+1}\right| +\left| \mathbf {b} ^{j+1}\right| \right) _{n_{j}}^{2} }\delta _{n_{j},m_{j}}, \end{aligned}$$

where

$$\begin{aligned}&W_{r}\left( \varvec{x},\mathbf {a},\mathbf {b}\right) :=W_{r}\left( x_{1},\dots ,x_{r};a_{1},\dots ,a_{r+1},b_{1},\dots ,b_{r+1}\right) \\&\quad =\prod \limits _{j=1}^{r}\left\{ \Gamma \left( a_{j}-\frac{ix_{j}}{2}\right) \Gamma \left( \left| \mathbf {a}^{j+1}\right| +\frac{ix_{j}}{2}\right) \Gamma \left( b_{j}+\frac{ix_{j}}{2}\right) \Gamma \left( \left| \mathbf {b}^{j+1}\right| -\frac{ix_{j}}{2}\right) \right\} \end{aligned}$$

for \(a_{j}, b_{j}>0\) ; \(j=1,2,\dots ,r+1\) and

$$\begin{aligned}&_{r}S_{\mathbf {n}}\left( \varvec{x};\mathbf {a},\mathbf {b}\right) =\prod \limits _{j=1}^{r}\left\{ \left( \left| \mathbf {a}^{j+1}\right| +\frac{x_{j}}{2}\right) _{\left| \mathbf {n}^{j+1}\right| }\right. \nonumber \\&\quad \left. \times \,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {-n_{j}, n_{j}+2\left| \mathbf {n}^{j+1}\right| +\left| \mathbf {a}^{j}\right| +\left| \mathbf {b}^{j}\right| -1 ,\ \left| \mathbf {n}^{j+1}\right| +\left| \mathbf {a}^{j+1}\right| +\frac{x_{j}}{2}}}\\ [-0.1ex] {\displaystyle {2\left| \mathbf {n}^{j+1}\right| +\left| \mathbf {a}^{j+1}\right| +\left| \mathbf {b}^{j+1}\right| ,\ \left| \mathbf {n}^{j+1}\right| +\left| \mathbf {a}^{j}\right| }} \end{array}\Big | \,{\displaystyle {1}} \right) \right\} \end{aligned}$$
(3.9)

or, in terms of Hahn polynomials (2.5)

$$\begin{aligned}&_{r}S_{\mathbf {n}}\left( \varvec{x};\mathbf {a},\mathbf {b}\right) = \prod \limits _{j=1}^{r}\left\{ \frac{n_{j}!i^{-n_{j}}}{\left( \left| \mathbf {n}^{j+1}\right| +\left| \mathbf {a}^{j}\right| \right) _{n_{j}}\left( 2\left| \mathbf {n}^{j+1}\right| +\left| \mathbf {a}^{j+1}\right| +\left| \mathbf {b}^{j+1}\right| \right) _{n_{j}}} \right. \nonumber \\&\quad \times \left. \left. \left. \left( \left| \mathbf {a}^{j+1}\right| +\frac{x_{j}}{2}\right) _{\left| \mathbf {n}^{j+1}\right| }\right. p_{n_{j}}\left( \frac{-ix_{j}}{2};\left| \mathbf {n}^{j+1}\right| +\left| \mathbf {a}^{j+1}\right| ,b_{j},\left| \mathbf {n}^{j+1}\right| +\left| \mathbf {b}^{j+1}\right| ,a_{j}\right) \right. \right\} \end{aligned}$$

for \(r\ge 1\).

Remark 3.4

The weight function of this orthogonality relation is positive when all equalities hold simultaneously: \(a_{j}=b_{j}\) for \(j=1,2,\dots ,r+1\).

3.3 Recurrence Relations for the Functions \(_{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a},\mathbf {b}\right) \)

We now derive recurrence relations for the special functions \(\ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a},\mathbf {b}\right) \). First, we recall the following relations for hypergeometric function \(\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {m_{1},m_{2},m_{3}}}\\ [-0.1ex] {\displaystyle {s_{1},s_{2}}} \end{array}\Big | \,{\displaystyle {z}} \right) _{}\) that might be obtained by considering the Zeilberger’s algorithm [25] based on the Fasenmyer seminal works [26].

Lemma 3.5

For \(\left| z\right| <1\), the hypergeometric function \(\ _{3}F_{2}\) satisfies the following known recurrence relations:

  1. (i)
    $$\begin{aligned}&\left( z-1\right) {\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {m_{1}+1,m_{2},m_{3}}}\\ [-0.1ex] {\displaystyle {s_{1},s_{2}}} \end{array}\Big | \,{\displaystyle {z}} \right) }=\left( B_{1}+C_{1}z\right) {\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {m_{1},m_{2},m_{3}}}\\ [-0.1ex] {\displaystyle {s_{1},s_{2}}} \end{array}\Big | \,{\displaystyle {z}} \right) } \nonumber \\&\quad +\left( B_{2}+C_{2}z\right) {\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {m_{1}-1,m_{2},m_{3}}}\\ [-0.1ex] {\displaystyle {s_{1},s_{2}}} \end{array}\Big | \,{\displaystyle {z}} \right) } +B_{3}{\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {m_{1}-2,m_{2},m_{3}}}\\ [-0.1ex] {\displaystyle {s_{1},s_{2}}} \end{array}\Big | \,{\displaystyle {z}} \right) }, \end{aligned}$$
    (3.10)

    where

    $$\begin{aligned} B_{1}= & {} \frac{s_{1}+s_{2}+1-3m_{1}}{m_{1}},\ B_{2}=\frac{s_{1}s_{2}+\left( m_{1}-1\right) \left( 3m_{1}-2\left( s_{1}+s_{2}+1\right) \right) }{\left( m_{1}-1\right) _{2}}, \\ B_{3}= & {} -\frac{\left( m_{1}-s_{1}-1\right) \left( m_{1}-s_{2}-1\right) }{\left( m_{1}-1\right) _{2}}, \\ C_{1}= & {} \frac{2m_{1}-m_{2}-m_{3}-1}{m_{1}},\ C_{2}=-\frac{\left( m_{1}-1\right) \left( m_{1}-m_{2}-m_{3}-1\right) +m_{2}m_{3}}{\left( m_{1}-1\right) _{2}}; \end{aligned}$$
  2. (ii)
    $$\begin{aligned}&\left( z-1\right) {\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {m_{1},m_{2},m_{3}}}\\ [-0.1ex] {\displaystyle {s_{1}-1,s_{2}}} \end{array}\Big | \,{\displaystyle {z}} \right) }=\left( B_{1}+C_{1}z\right) {\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {m_{1},m_{2},m_{3}}}\\ [-0.1ex] {\displaystyle {s_{1},s_{2}}} \end{array}\Big | \,{\displaystyle {z}} \right) } \nonumber \\&\quad +\left( B_{2}+C_{2}z\right) {\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {m_{1},m_{2},m_{3}}}\\ [-0.1ex] {\displaystyle {s_{1}+1,s_{2}}} \end{array}\Big | \,{\displaystyle {z}} \right) } +C_{3}z{\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {m_{1},m_{2},m_{3}}}\\ [-0.1ex] {\displaystyle {s_{1}+2,s_{2}}} \end{array}\Big | \,{\displaystyle {z}} \right) }, \end{aligned}$$
    (3.11)

    where

    $$\begin{aligned} B_{1}&=\frac{s_{2}-2s_{1}}{s_{1}-1},\ B_{2}=\frac{s_{1}-s_{2}+1}{s_{1}-1},\ C_{1}=\frac{3s_{1}-m_{1}-m_{2}-m_{3}}{s_{1}-1} , \\ C_{3}&=\frac{\left( s_{1}-m_{1}+1\right) \left( s_{1}-m_{2}+1\right) \left( s_{1}-m_{3}+1\right) }{\left( s_{1}-1\right) _{3}}, \\ C_{2}&=\frac{\left( 2s_{1}+1\right) \left( m_{1}+m_{2}+m_{3}\right) -3\left( s_{1}-1\right) \left( s_{1}+2\right) }{\left( s_{1}-1\right) _{2}} \\&\quad -\frac{7+m_{1}m_{2}+m_{1}m_{3}+m_{2}m_{3}}{\left( s_{1}-1\right) _{2}}; \end{aligned}$$
  3. (iii)
    $$\begin{aligned}&s_{1}{\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {m_{1},m_{2},m_{3}}}\\ [-0.1ex] {\displaystyle {s_{1},s_{2}}} \end{array}\Big | \,{\displaystyle {z}} \right) } +\left( m_{1}-s_{1}\right) {\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {m_{1},m_{2},m_{3}}}\\ [-0.1ex] {\displaystyle {s_{1}+1,s_{2}}} \end{array}\Big | \,{\displaystyle {z}} \right) }\nonumber \\&\quad =m_{1}{\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {m_{1}+1,m_{2},m_{3}}}\\ [-0.1ex] {\displaystyle {s_{1}+1,s_{2}}} \end{array}\Big | \,{\displaystyle {z}} \right) } . \end{aligned}$$
    (3.12)

To derive recurrence relations, by taking into account (3.9), we first give the following relationships between special functions \(\ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a},\mathbf {b}\right) \) as follows:

$$\begin{aligned}&_{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a},\mathbf {b}\right) \nonumber \\&\quad = \,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {-n_{r} ,\ n_{r}+\left| \mathbf {a}^{r}\right| +\left| \mathbf {b}^{r}\right| -1,\ a_{r+1}+ \frac{x_{r}}{2}}}\\ [-0.1ex] {\displaystyle {a_{r+1}+b_{r+1},\left| \mathbf {a}^{r}\right| }} \end{array}\Big | \,{\displaystyle {1}} \right) \prod \limits _{j=1}^{r-1}\left( \left| \mathbf {a} ^{j+1}\right| +\frac{x_{j}}{2}\right) _{n_{r}}\nonumber \\&\qquad \times _{r-1}S_{n_{1},\dots ,n_{r-1}}\left( x_{1},\dots ,x_{r-1};a_{1},\dots ,a_{r-1},a_{r}+a_{r+1}+n_{r}, \right. \nonumber \\&\qquad \left. b_{1},\dots ,b_{r-1},b_{r}+b_{r+1}+n_{r}\right) \end{aligned}$$
(3.13)

or

$$\begin{aligned}&_{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a},\mathbf {b}\right) =\left( \left| \mathbf {a}^{2}\right| +\frac{x_{1}}{2}\right) _{\left| \mathbf {n}^{2}\right| }~_{r-1}S_{n_{2},\dots ,n_{r}}\left( x_{2},\dots ,x_{r}; \mathbf {a}^{2} ,\mathbf {b} ^{2} \right) \nonumber \\&\quad \times \,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {-n_{1} ,\ n_{1}+2\left| \mathbf {n} ^{2}\right| +\left| \mathbf {a}\right| +\left| \mathbf {b} \right| -1,\left| \mathbf {n}^{2}\right| +\left| \mathbf {a} ^{2}\right| +\frac{x_{1}}{2}}}\\ [-0.1ex] {\displaystyle {2\left| \mathbf {n}^{2}\right| +\left| \mathbf {a}^{2}\right| +\left| \mathbf {b}^{2}\right| ,\left| \mathbf {n}^{2}\right| +\left| \mathbf {a}\right| }} \end{array}\Big | \,{\displaystyle {1}} \right) , \end{aligned}$$
(3.14)

for \(r\ge 2\) and

$$\begin{aligned} _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1},b_{2}\right) =\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {-n_{1} ,\ a_{2}+x_{1} /2,\ n_{1} +a_{1}+a_{2}+b_{1}+b_{2}-1}}\\ [-0.1ex] {\displaystyle {a_{2}+b_{2},\ a_{1}+a_{2}}} \end{array}\Big | \,{\displaystyle {1}} \right) \end{aligned}$$

for \(r=1\).

Proposition 3.6

For \(r\ge 1\), the family of the special function\(\ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a},\mathbf {b}\right) \) satisfies the following recurrence relation:

$$\begin{aligned}&\left( B_{1}+C_{1}\right) \ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a}, \mathbf {b}\right) +\left( B_{2}+C_{2}\right) \ _{r}S_{n_{1}+1,n_{2},\dots ,n_{r}}\left( \mathbf {x};\mathbf {a} ,b_{1}-1,b_{2},\dots ,b_{r+1}\right) \\&\quad +B_{3}\ _{r}S_{n_{1}+2,n_{2},\dots ,n_{r}}\left( \mathbf {x};\mathbf {a} ,b_{1}-2,b_{2},\dots ,b_{r+1}\right) =0, \end{aligned}$$

where

$$\begin{aligned} B_{1}+C_{1}&=\frac{b_{1}-1+\frac{x_{1}}{2}}{n_{1}}, \\ B_{3}&=-\frac{\left( n_{1}+2\left| \mathbf {n}^{2}\right| +\left| \mathbf {a}^{2}\right| +\left| \mathbf {b}^{2}\right| +1\right) \left( \left| \mathbf {n}\right| +\left| \mathbf {a}\right| +1\right) }{\left( n_{1}\right) _{2}}, \\ B_{2}+C_{2}&=\frac{\left( 2\left| \mathbf {n}^{2}\right| +\left| \mathbf {a}^{2}\right| +\left| \mathbf {b}^{2}\right| \right) \left( \left| \mathbf {n}^{2}\right| +\left| \mathbf {a} \right| \right) }{\left( n_{1}\right) _{2}} \\&\quad - \frac{\left( n_{1}+2\left| \mathbf {n}^{2}\right| +\left| \mathbf {a}\right| +\left| \mathbf {b}\right| -1\right) \left( \left| \mathbf {n}^{2}\right| +\left| \mathbf {a} ^{2}\right| +\frac{x_{1}}{2}\right) }{\left( n_{1}\right) _{2}} \\&\quad +\frac{n_{1}+3\left| \mathbf {n}^{2}\right| +a_{1}+2\left| \mathbf {a}^{2}\right| -b_{1}+\left| \mathbf {b}^{2}\right| +2- \frac{x_{1}}{2}}{n_{1}}. \end{aligned}$$

Proposition 3.7

For \(r\ge 1\), the family of the special function\(\ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a},\mathbf {b}\right) \) satisfies the following recurrence relation:

$$\begin{aligned}&\left( B_{1}+C_{1}\right) \ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a}, \mathbf {b}\right) +\left( B_{2}+C_{2}\right) \ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a},b_{1}-1,b_{2},\dots ,b_{r+1}\right) \\&\quad +B_{3}\ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a},b_{1}-2,b_{2},\dots ,b_{r+1} \right) =0, \end{aligned}$$

where

$$\begin{aligned} B_{1}+C_{1}&=-\frac{b_{1}-1+\frac{x_{1}}{2}}{n_{1}+2\left| \mathbf {n} ^{2}\right| +\left| \mathbf {a}\right| +\left| \mathbf {b} \right| -1}, \\ B_{3}&=-\frac{\left( n_{1}+a_{1}+b_{1}-2\right) \left( \left| \mathbf {n }\right| +\left| \mathbf {b}\right| -2\right) }{\left( n_{1}+2\left| \mathbf {n}^{2}\right| +\left| \mathbf {a} \right| +\left| \mathbf {b}\right| -2\right) _{2}}, \\ B_{2}+C_{2}&=\frac{\left( 2\left| \mathbf {n}^{2}\right| +\left| \mathbf {a}^{2}\right| +\left| \mathbf {b}^{2}\right| \right) \left( \left| \mathbf {n}^{2}\right| +\left| \mathbf {a} \right| \right) +n_{1}\left( \left| \mathbf {n}^{2}\right| +\left| \mathbf {a}^{2}\right| +\frac{x_{1}}{2}\right) }{\left( n_{1}+2\left| \mathbf {n}^{2}\right| +\left| \mathbf {a} \right| +\left| \mathbf {b}\right| -2\right) _{2}} \\&\quad +\frac{n_{1}-\left| \mathbf {n}^{2}\right| -\left| \mathbf {a} ^{2}\right| +2b_{1}-3+\frac{x_{1}}{2}}{n_{1}+2\left| \mathbf {n} ^{2}\right| +\left| \mathbf {a}\right| +\left| \mathbf {b} \right| -1}. \end{aligned}$$

Proposition 3.8

For the family of the special function\(\ _{r}S_{\mathbf {n}}\left( \mathbf {x}; \mathbf {a},\mathbf {b}\right) \), we get

$$\begin{aligned}&\left( B_{2}+C_{2}\right) \ _{r}S_{\mathbf {n}}\left( \mathbf {x};a_{1},\dots ,a_{r-1},a_{r}+1,a_{r+1}-1,b_{1},\dots ,b_{r-1},b_{r}-1,b_{r+1}+1\right) \\&\quad +B_{3}\ _{r}S_{\mathbf {n}}\left( \mathbf {x};a_{1},\dots ,a_{r-1},a_{r}+2,a_{r+1}-2,b_{1},\dots ,b_{r-1},b_{r}-2,b_{r+1}+2\right) \\&\quad +\left( B_{1}+C_{1}\right) \ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a}, \mathbf {b}\right) =0, \end{aligned}$$

for \(r\ge 1\) where

$$\begin{aligned} B_{1}+C_{1}&=\left( b_{r}-1+\frac{x_{r}}{2}\right) \left( a_{r+1}+\frac{ x_{r}}{2}-1\right) , \\ B_{3}&=\left( a_{r}-\frac{x_{r}}{2}+1\right) \left( b_{r+1}-\frac{x_{r}}{2} +1\right) , \\ B_{2}+C_{2}&=\left( \left| \mathbf {a}^{r}\right| -b_{r}+b_{r+1}+2-x_{r}\right) \left( a_{r+1}+\frac{x_{r}}{2}-1\right) \\&\quad -\left| \mathbf {a}^{r}\right| \left( a_{r+1}+b_{r+1}\right) -n_{r}\left( n_{r}+\left| \mathbf {a}^{r}\right| +\left| \mathbf {b }^{r}\right| -1\right) . \end{aligned}$$

Proposition 3.9

For the family of the special function\(\ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a},\mathbf {b}\right) \)

$$\begin{aligned}&\left( B_{1}+C_{1}\right) \ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a},\mathbf {b}\right) \\&\quad +\left( B_{2}+C_{2}\right) \ _{r}S_{\mathbf {n}}\left( \mathbf {x};a_{1}+1,a_{2},\dots ,a_{r+1},b_{1}-1,b_{2},\dots ,b_{r+1}\right) \\&\quad +C_{3}\ _{r}S_{\mathbf {n}}\left( \mathbf {x};a_{1}+2,a_{2},\dots ,a_{r+1},b_{1}-2,b_{2},\dots ,b_{r+1}\right) =0, \end{aligned}$$

holds for \(r\ge 1\) where

$$\begin{aligned} B_{1}+C_{1}&=-\frac{b_{1}-1+\frac{x_{1}}{2}}{\left| \mathbf {n} ^{2}\right| +\left| \mathbf {a}\right| -1}, \\ C_{3}&=-\frac{\left( \left| \mathbf {n}\right| +\left| \mathbf {a }\right| +1\right) \left( \left| \mathbf {n}\right| +\left| \mathbf {b}\right| -2\right) \left( a_{1}-\frac{x_{1}}{2}+1\right) }{ \left( \left| \mathbf {n}^{2}\right| +\left| \mathbf {a} \right| -1\right) _{3}}, \\ B_{2}+C_{2}&=\frac{\left( 2\left| \mathbf {n}^{2}\right| +2\left| \mathbf {a}\right| +1\right) \left( 2\left| \mathbf {n} ^{2}\right| +\left| \mathbf {a}^{2}\right| +\left| \mathbf {b} \right| -2+\frac{x_{1}}{2}\right) }{{\left( \left| \mathbf {n}^{2}\right| +\left| \mathbf {a}\right| -1\right) _{2}}} \\&\quad +\frac{n_{1}\left( n_{1}+2\left| \mathbf {n}^{2}\right| +\left| \mathbf {a}\right| +\left| \mathbf {b}\right| -1\right) }{{\left( \left| \mathbf {n}^{2}\right| +\left| \mathbf {a}\right| -1\right) _{2}}} \\&\quad -\frac{\left( 2\left| \mathbf {n}^{2}\right| +\left| \mathbf {a}\right| +\left| \mathbf {b}\right| -1\right) \left( \left| \mathbf {n}^{2}\right| +\left| \mathbf {a}^{2}\right| +\frac{x_{1}}{2}\right) }{\left( \left| \mathbf {n}^{2}\right| +\left| \mathbf {a}\right| -1\right) _{2}} \\&\quad -\frac{2\left| \mathbf {n}^{2}\right| +\left| \mathbf {a}^{2}\right| +\left| \mathbf {b}^{2}\right| -1}{\left| \mathbf {n}^{2}\right| +\left| \mathbf {a}\right| -1}. \end{aligned}$$

Proposition 3.10

For the family of the special function\(\ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a},\mathbf {b}\right) \), we have

$$\begin{aligned}&\left( B_{1}+C_{1}\right) \ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a}, \mathbf {b}\right) +\left( B_{2}+C_{2}\right) \ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a},b_{1},\dots ,b_{r-1},b_{r}-1,b_{r+1}+1\right) \\&\quad +C_{3}\ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a} ,b_{1},\dots ,b_{r-1},b_{r}-2,b_{r+1}+2\right) =0, \end{aligned}$$

for \(r\ge 1\) where

$$\begin{aligned} B_{1}+C_{1}&=-\frac{b_{r}-1+\frac{x_{r}}{2}}{a_{r+1}+b_{r+1}-1}, \\ C_{3}&=-\frac{\left( n_{r}+a_{r}+b_{r}-2\right) \left( n_{r}+a_{r+1}+b_{r+1}+1\right) \left( b_{r+1}-\frac{x_{r}}{2}+1\right) }{ \left( a_{r+1}+b_{r+1}-1\right) _{3}}, \\ B_{2}+C_{2}&=\frac{n_{r}\left( n_{r}+\left| \mathbf {a}^{r}\right| +\left| \mathbf {b}^{r}\right| -1\right) -\left( a_{r+1}+\frac{x_{r}}{ 2}\right) \left( \left| \mathbf {a}^{r}\right| +\left| \mathbf {b} ^{r}\right| -1\right) }{\left( a_{r+1}+b_{r+1}-1\right) _{2}} \\&\quad +\frac{\left( 2a_{r+1}+2b_{r+1}+1\right) \left( \left| \mathbf {a} ^{r}\right| +b_{r}-2+\frac{x_{r}}{2}\right) }{\left( a_{r+1}+b_{r+1}-1\right) _{2}}-\frac{\left| \mathbf {a}^{r}\right| -1 }{a_{r+1}+b_{r+1}-1}. \end{aligned}$$

Proposition 3.11

The family of the special function\(\ _{r}S_{\mathbf {n}}\left( \mathbf {x}; \mathbf {a},\mathbf {b}\right) \) satisfies the following recurrence relation:

$$\begin{aligned}&\left( \left| \mathbf {n}^{2}\right| +\left| \mathbf {a} \right| \right) \ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a},\mathbf { b}\right) \\&\quad -\left( n_{1}+2\left| \mathbf {n}^{2}\right| +\left| \mathbf {a}\right| +\left| \mathbf {b}\right| -1\right) \ _{r}S_{ \mathbf {n}}\left( \mathbf {x};a_{1}+1,a_{2},\dots ,a_{r+1},b_{1},\dots ,b_{r+1} \right) \\&\quad +\left( \left| \mathbf {n}\right| +\left| \mathbf {b}\right| -1\right) \ _{r}S_{\mathbf {n}}\left( \mathbf {x} ;a_{1}+1,a_{2},\dots ,a_{r+1},b_{1}-1,b_{2},\dots ,b_{r+1}\right) =0 \end{aligned}$$

for \(r\ge 1\).

4 Proof of Theorems and Propositions

4.1 Proof of Theorem 3.1

Using the equation (3.3) and the equation in [19,   p. 62, Eq. (4.21.2)], the Fourier transform of the function \(g_{r+1}\) is as follows:

$$\begin{aligned}&\mathcal {F}\left( g_{r+1}\left( x_{1},\dots ,x_{r+1};n_{1},\dots ,n_{r+1},a_{1},\dots ,a_{r+2},\alpha _{1},\dots ,\alpha _{r+2}\right) \right) \\&\quad =\mathcal {F}\left( 2^{-rn_{r+1}} \left( 1+\tanh x_{r+1}\right) ^{a_{r+1}}\left( 1-\tanh x_{r+1}\right) ^{a_{r+2}}P_{n_{r+1}}^{\left( \alpha _{r+2},\alpha _{r+1}\right) }\left( \tanh x_{r+1}\right) \right. \\&\qquad \left. \times g_{r}\left( x_{1},\dots ,x_{r};n_{1},\dots ,n_{r},a_{1},\dots ,a_{r},a_{r+1}+a_{r+2}+n_{r+1}, \right. \right. \\&\qquad \left. \left. \alpha _{1},\dots ,\alpha _{r},\alpha _{r+1}+\alpha _{r+2}+2n_{r+1}+1\right) \right) \\&\quad =2^{-rn_{r+1}}\int \limits _{-\infty }^{\infty }\cdots \int \limits _{-\infty }^{\infty }e^{-i\left( \xi _{1}x_{1}+\dots +\xi _{r}x_{r}+\xi _{r+1}x_{r+1}\right) }\left( 1+\tanh x_{r+1}\right) ^{a_{r+1}} \\&\qquad \times \left( 1-\tanh x_{r+1}\right) ^{a_{r+2}} P_{n_{r+1}}^{\left( \alpha _{r+2},\alpha _{r+1}\right) }\left( \tanh x_{r+1}\right) \\&\qquad \times g_{r}\left( x_{1},\dots ,x_{r};n_{1},\dots ,n_{r},a_{1},\dots ,a_{r},a_{r+1}+a_{r+2}+n_{r+1}, \right. \\&\qquad \left. \alpha _{1},\dots ,\alpha _{r},\alpha _{r+1}+\alpha _{r+2}+2n_{r+1}+1\right) \mathrm{d}x_{r+1}\mathrm{d}x_{r}\cdots \mathrm{d}x_{1}\\&\quad =2^{-rn_{r+1}}\mathcal {F}\left( g_{r}\left( x_{1},\dots ,x_{r};n_{1},\dots ,n_{r},a_{1},\dots ,a_{r},a_{r+1}+a_{r+2}+n_{r+1} , \right. \right. \\&\qquad \left. \left. \alpha _{1},\dots ,\alpha _{r},\alpha _{r+1}+\alpha _{r+2}+2n_{r+1}+1\right) \right) \\&\qquad \times \int \limits _{-\infty }^{\infty }e^{-i\xi _{r+1} x_{r+1}}\left( 1+\tanh x_{r+1}\right) ^{a_{r+1}}\left( 1-\tanh x_{r+1}\right) ^{a_{r+2}} \\&\qquad \times P_{n_{r+1}}^{\left( \alpha _{r+2},\alpha _{r+1}\right) }\left( \tanh x_{r+1}\right) \mathrm{d}x_{r+1} =2^{a_{r+1}+a_{r+2}-rn_{r+1}-1} \\&\qquad \times \mathcal {F} \left( g_{r}\left( x_{1},\dots ,x_{r};n_{1},\dots ,n_{r},a_{1},\dots ,a_{r},a_{r+1}+a_{r+2}+n_{r+1} , \right. \right. \\&\qquad \left. \left. \alpha _{1},\dots ,\alpha _{r},\alpha _{r+1}+\alpha _{r+2}+2n_{r+1}+1\right) \right) \\&\qquad \times \int \limits _{0}^{1} t ^{a_{r+1}-\frac{i\xi _{r+1}}{2}-1}\left( 1-t\right) ^{a_{r+2}+\frac{i\xi _{r+1}}{2}-1}P_{n_{r+1}}^{\left( \alpha _{r+2},\alpha _{r+1}\right) }\left( 2t-1\right) \mathrm{d}t \\&\quad =2^{a_{r+1}+a_{r+2}-rn_{r+1}-1} \mathcal {F} \left( g_{r}\left( x_{1},\dots ,x_{r};n_{1},\dots ,n_{r},a_{1},\dots ,a_{r}, \right. \right. \\&\qquad \left. \left. a_{r+1}+a_{r+2}+n_{r+1} ,\alpha _{1},\dots ,\alpha _{r},\alpha _{r+1}+\alpha _{r+2}+2n_{r+1}+1\right) \right) \\&\qquad \times \int \limits _{0}^{1} t ^{a_{r+1}-\frac{i\xi _{r+1}}{2}-1}\left( 1-t\right) ^{a_{r+2}+\frac{i\xi _{r+1}}{2}-1} \frac{\left( \alpha _{r+2}+1\right) _{n_{r+1}} }{ n_{r+1}! } \\&\qquad \times \,{}_{2}F_{1}\left( \begin{array}{cc}{\displaystyle {-n_{r+1}, n_{r+1}+\alpha _{r+1}+\alpha _{r+2}+1}}\\ [-0.1ex] {\displaystyle {\alpha _{r+2}+1 }} \end{array}\Big | \,{\displaystyle {1-t}} \right) \mathrm{d}t \\&\quad = \frac{2^{a_{r+1}+a_{r+2}-rn_{r+1}-1}\left( \alpha _{r+2}+1\right) _{n_{r+1}} }{ n_{r+1}! } \mathcal {F}\left( g_{r}\left( x_{1},\dots ,x_{r};n_{1},\dots ,n_{r} ,\right. \right. \\&\qquad \left. \left. a_{1},\dots ,a_{r},a_{r+1}+a_{r+2}+n_{r+1},\alpha _{1},\dots ,\alpha _{r},\alpha _{r+1}+\alpha _{r+2}+2n_{r+1}+1\right) \right) \\&\qquad \times \sum \limits _{l=0}^{n_{r+1}}\frac{\left( -n_{r+1}\right) _{l}\left( n_{r+1}+\alpha _{r+1}+\alpha _{r+2}+1\right) _{l}}{\left( \alpha _{r+2}+1\right) _{l} l!}\\&\qquad \times \int \limits _{0}^{1} t ^{a_{r+1}-\frac{i\xi _{r+1}}{2}-1}\left( 1-t\right) ^{a_{r+2}+\frac{i\xi _{r+1}}{2}-1+l}\mathrm{d}t \\&\quad = \frac{2^{a_{r+1}+a_{r+2}-rn_{r+1}-1}\left( \alpha _{r+2}+1\right) _{n_{r+1}} }{ n_{r+1}! } B\left( a_{r+1}-\frac{i\xi _{r+1}}{2},\ a_{r+2}+\frac{i\xi _{r+1}}{2}\right) \\&\qquad \times \,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {-n_{r+1},\ n_{r+1}+\alpha _{r+1}+\alpha _{r+2}+1,\ a_{r+2}+\frac{i\xi _{r+1}}{2}}}\\ [-0.1ex] {\displaystyle {\alpha _{r+2}+1,\ a_{r+1}+a_{r+2}}} \end{array}\Big | \,{\displaystyle {1}} \right) \\&\qquad \times \mathcal {F}\left( g_{r}\left( x_{1},\dots ,x_{r};n_{1},\dots ,n_{r},a_{1},\dots ,\right. \right. \\&\qquad \left. \left. a_{r},a_{r+1}+a_{r+2}+n_{r+1} ,\alpha _{1},\dots ,\alpha _{r},\alpha _{r+1}+\alpha _{r+2}+2n_{r+1}+1\right) \right) , \end{aligned}$$

which proves the theorem.

4.2 Proof of Theorem 3.2

For the case \(r=1\), the Fourier transform of the function

$$\begin{aligned} g_{1}\left( x_{1};n_{1},a_{1},a_{2},\alpha _{1},\alpha _{2}\right) =\left( 1+\tanh x_{1}\right) ^{a_{1}}\left( 1-\tanh x_{1}\right) ^{a_{2}} P_{n_{1}}^{\left( \alpha _{1},\alpha _{2}\right) }\left( \Upsilon _{1} \right) , \end{aligned}$$

is

$$\begin{aligned}&\mathcal {F}\left( g_{1}\left( x_{1};n_{1},a_{1},a_{2},\alpha _{1},\alpha _{2}\right) \right) \nonumber \\&\quad =\int \limits _{-\infty }^{\infty }e^{-i\xi x_{1}}\left( 1+\tanh x_{1}\right) ^{a_{1}}\left( 1-\tanh x_{1}\right) ^{a_{2}}P_{n_{1}}^{\left( \alpha _{2},\alpha _{1}\right) }\left( \tanh x_{1}\right) \mathrm{d}x_{1} \nonumber \\&\quad = \frac{2^{a_{1}+a_{2}-1}\left( \alpha _{2}+1\right) _{n_{1}}}{{n_{1}}!}\Lambda _{1}^{1}(\mathbf {a},{\varvec{\alpha }},n_{1};\xi ), \end{aligned}$$
(4.1)

where

$$\begin{aligned}&\Lambda _{1}^{1}(\mathbf {a},{\varvec{\alpha }},n_{1};\xi ) \\&\quad = \,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {-n_{1} ,\ a_{2}+\frac{i\xi }{2},\ n_{1}+\alpha _{1}+\alpha _{2}+1}}\\ [-0.1ex] {\displaystyle {\alpha _{2}+1,\ a_{1}+a_{2}}} \end{array}\Big | \,{\displaystyle {1}} \right) B\left( a_{2}+ \frac{i\xi }{2},\ a_{1}-\frac{i\xi }{2}\right) , \\ \end{aligned}$$

which was proved by Koelink [2] and it was also rewritten in terms of the continuous Hahn polynomials (2.5) as

$$\begin{aligned}&\mathcal {F}\left( g_{1}\left( x_{1};n_{1},a_{1},a_{2},\alpha _{1},\alpha _{2}\right) \right) =\frac{2^{a_{1}+a_{2}-1}}{i^{n_{1}}\left( a_{1}+a_{2}\right) _{n_{1}}}B\left( a_{2}+ \frac{i\xi }{2},\ a_{1}-\frac{i\xi }{2}\right) \\&\quad \times p_{n_{1}}\left( \frac{\xi }{2};\ a_{2},\ \alpha _{1}-a_{1}+1,\ \alpha _{2}-a_{2}+1,\ a_{1}\right) , \end{aligned}$$

where \(\mathbf {a}=\left( a_{1},a_{2}\right) \), \({\varvec{\alpha }}=\left( \alpha _{1},\alpha _{2}\right) \) and \(\ _{3}F_{2}\) is a special case of the hypergeometric function (2.3). Now, let us consider the following specific function in terms of the orthogonal polynomials on 2-simplex:

$$\begin{aligned}&g_{2}\left( x_{1},x_{2};n_{1},n_{2},a_{1},a_{2},a_{3},\alpha _{1},\alpha _{2},\alpha _{3}\right) =\left( 1+\tanh x_{1}\right) ^{a_{1}}\left( 1-\tanh x_{1}\right) ^{a_{2}+a_{3}} \nonumber \\&\quad \times \left( 1+\tanh x_{2}\right) ^{a_{2}} \left( 1-\tanh x_{2}\right) ^{a_{3}} P_{n_{1},n_{2}}^{\left( \alpha _{1},\alpha _{2},\alpha _{3}\right) }\left( \Upsilon _{1},\Upsilon _{2} \right) , \end{aligned}$$
(4.2)

where \(a_{1},a_{2},a_{3},\alpha _{1},\alpha _{2},\alpha _{3}\) are real parameters, and

$$\begin{aligned} \Upsilon _{1}(x_{1})=\Upsilon _{1}= \frac{1+\tanh x_{1}}{2},\qquad \Upsilon _{2}(x_{1},x_{2})=\Upsilon _{2}=\frac{\left( 1-\tanh x_{1}\right) \left( 1+\tanh x_{2}\right) }{4}. \end{aligned}$$

It is easily seen that the function \(g_{2}\) can also be expressed in terms of \(g_{1}\) from (3.3)

$$\begin{aligned}&g_{2}\left( x_{1},x_{2};n_{1},n_{2},a_{1},a_{2},a_{3},\alpha _{1},\alpha _{2},\alpha _{3}\right) \nonumber \\&\quad =2^{-n_{2}}\left( 1+\tanh x_{2}\right) ^{a_{2}}\left( 1-\tanh x_{2}\right) ^{a_{3}} P_{n_{2}}^{\left( \alpha _{3},\alpha _{2}\right) }\left( \tanh x_{2} \right) \nonumber \\&\qquad \times g_{1}\left( x_{1};n_{1},a_{1},a_{2}+a_{3}+n_{2},\alpha _{1},\alpha _{2}+\alpha _{3}+2n_{2}+1\right) . \end{aligned}$$
(4.3)

The corresponding Fourier transform is calculated, under the substitution \( \tanh x_{2}=2v-1\), as follows from:

$$\begin{aligned}&\mathcal {F}\left( g_{2}\left( x_{1},x_{2};n_{1},n_{2},a_{1},a_{2},a_{3},\alpha _{1},\alpha _{2},\alpha _{3}\right) \right) \\&\quad =\int \limits _{-\infty }^{\infty }\int \limits _{-\infty }^{\infty }e^{-i\left( \xi _{1}x_{1}+\xi _{2}x_{2}\right) }g_{2}\left( x_{1},x_{2};n_{1},n_{2},a_{1},a_{2},a_{3},\alpha _{1},\alpha _{2},\alpha _{3}\right) \mathrm{d}x_{1}\mathrm{d}x_{2} \\&\quad =2^{-n_{2}}\int \limits _{-\infty }^{\infty }e^{-i\xi _{1}x_{1}}g_{1}\left( x_{1};n_{1},a_{1},a_{2}+a_{3}+n_{2},\alpha _{1},\alpha _{2}+\alpha _{3}+2n_{2}+1\right) \mathrm{d}x_{1} \\&\qquad \times \int \limits _{-\infty }^{\infty }e^{-i\xi _{2}x_{2}}\left( 1+\tanh x_{2}\right) ^{a_{2}}\left( 1-\tanh x_{2}\right) ^{a_{3}}P_{n_{2}}^{\left( \alpha _{3},\alpha _{2}\right) }\left( \tanh x_{2}\right) \mathrm{d}x_{2} \\&\quad =\mathcal {F}\left( g_{1}\left( x_{1};n_{1},a_{1},a_{2}+a_{3}+n_{2},\alpha _{1},\alpha _{2}+\alpha _{3}+2n_{2}+1\right) \right) \\&\qquad \times 2^{a_{2}+a_{3}-n_{2}-{1}}\int \limits _{0}^{1}v^{a_{2}-\frac{i\xi _{2}}{ 2}-1}\left( 1-v\right) ^{a_{3}+\frac{i\xi _{2}}{2}-1}P_{n_{2}}^{\left( \alpha _{3},\alpha _{2}\right) }\left( 2v-1\right) dv \\&\quad =\mathcal {F}\left( g_{1}\left( x_{1};n_{1},a_{1},a_{2}+a_{3}+n_{2},\alpha _{1},\alpha _{2}+\alpha _{3}+2n_{2}+1\right) \right) \\&\qquad \times 2^{a_{2}+a_{3}-n_{2}-{1}}\int \limits _{0}^{1}v^{a_{2}-\frac{i\xi _{2}}{ 2}-1}\left( 1-v\right) ^{a_{3}+\frac{i\xi _{2}}{2}-1}\frac{\left( \alpha _{3}+1\right) _{n_{2}}}{n_{2}!} \\&\qquad \times \,{}_{2}F_{1}\left( \begin{array}{cc}{\displaystyle {-n_{2}, n_{2}+\alpha _{2}+\alpha _{3}+1}}\\ [-0.1ex] {\displaystyle { \ \alpha _{3}+1 }} \end{array}\Big | \,{\displaystyle {1-v}} \right) dv \\&\quad =\mathcal {F}\left( g_{1}\left( x_{1};n_{1},a_{1},a_{2}+a_{3}+n_{2},\alpha _{1},\alpha _{2}\!+\!\alpha _{3}\!+\!2n_{2}+1\right) \right) \frac{\left( \alpha _{3}+1\right) _{n_{2}}}{n_{2}!} 2^{a_{2}+a_{3}-n_{2}-{1}} \\&\qquad \times \sum \limits _{l_{2}=0}^{n_{2}}\frac{\left( -n_{2}\right) _{l_{2}}\left( n_{2}+\alpha _{2}+\alpha _{3}+1\right) _{l_{2}}}{\left( \alpha _{3}+1\right) _{l_{2}}l_{2}!} \int \limits _{0}^{1}v^{a_{2}-\frac{i\xi _{2}}{2}-1}\left( 1-v\right) ^{a_{3}+\frac{i\xi _{2}}{2}-1+l_{2}}dv, \end{aligned}$$

that is

$$\begin{aligned}&\mathcal {F}\left( g_{2}\left( x_{1},x_{2};n_{1},n_{2},a_{1},a_{2},a_{3},\alpha _{1},\alpha _{2},\alpha _{3}\right) \right) \nonumber \\&\quad =\frac{2^{a_{1}+2\left( a_{2}+a_{3}\right) - {2}} \left( 2n_{2}+\alpha _{2}+\alpha _{3}+2\right) _{n_{1}}\left( \alpha _{3}+1\right) _{n_{2}}}{n_{1}!n_{2}!}\Lambda _{1}^{2}(\mathbf {a},{\varvec{\alpha }},\mathbf {n};\xi _{1}) \Lambda _{2}^{2}(\mathbf {a},{\varvec{\alpha }},\mathbf {n};\xi _{2}),\nonumber \\ \end{aligned}$$
(4.4)

where

$$\begin{aligned}&\Lambda _{1}^{2}(\mathbf {a},{\varvec{\alpha }},\mathbf {n};\xi _{1})= B\left( a_{1}-\frac{i\xi _{1}}{2},n_{2}+a_{2}+a_{3}+\frac{i\xi _{1}}{2}\right) \\&\quad \times \,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {-n_{1}, n_{1}+2n_{2}+\alpha _{1}+\alpha _{2}+\alpha _{3}+2 ,\ n_{2}+a_{2}+a_{3}+\frac{i\xi _{1}}{2}}}\\ [-0.1ex] {\displaystyle {2n_{2}+\alpha _{2}+\alpha _{3}+2,\ n_{2}+a_{1}+a_{2}+a_{3}}} \end{array}\Big | \,{\displaystyle {1}} \right) , \\&\quad \Lambda _{2}^{2}(\mathbf {a},{\varvec{\alpha }},\mathbf {n};\xi _{2})=B\left( a_{2}-\frac{i\xi _{2}}{2},a_{3}+\frac{i\xi _{2}}{2}\right) \,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {-n_{2},n_{2}+\alpha _{2}+\alpha _{3}+1,a_{3}+\frac{i\xi _{2}}{2}}}\\ [-0.1ex] {\displaystyle {\alpha _{3}+1,\ a_{2}+a_{3}}} \end{array}\Big | \,{\displaystyle {1}} \right) , \end{aligned}$$

or, in terms of the continuous Hahn polynomials (2.5)

$$\begin{aligned} \Lambda _{1}^{2}(\mathbf {a},{\varvec{\alpha }},\mathbf {n};\xi _{1})&= \frac{n_{1}!B\left( a_{1}-\frac{i\xi _{1}}{2},n_{2}+a_{2}+a_{3}+\frac{ i\xi _{1}}{2}\right) }{i^{n_{1}}\left( n_{2}+a_{1}+a_{2}+a_{3}\right) _{n_{1}}\left( 2n_{2}+\alpha _{2}+\alpha _{3}+2\right) _{n_{1}}} \\&\quad \times p_{n_{1}}\left( \frac{\xi _{1}}{2};n_{2}+a_{2}+a_{3},\alpha _{1}-a_{1}+1,n_{2}+\alpha _{2}+\alpha _{3}-a_{2}-a_{3}+2,a_{1}\right) ,\\ \Lambda _{2}^{2}(\mathbf {a},{\varvec{\alpha }},\mathbf {n};\xi _{2})&=\frac{n_{2}!B\left( a_{2}-\frac{i\xi _{2}}{2},a_{3}+\frac{i\xi _{2}}{2} \right) }{i^{n_{2}}\left( a_{2}+a_{3}\right) _{n_{2}}\left( \alpha _{3}+1\right) _{n_{2}}} \\&\quad \times p_{n_{2}}\left( \frac{\xi _{2}}{2};a_{3},\alpha _{2}-a_{2}+1,\alpha _{3}-a_{3}+1,a_{2}\right) , \end{aligned}$$

where \(\mathbf {a}=\left( a_{1},a_{2},a_{3}\right) \), \({\varvec{\alpha }}=\left( \alpha _{1},\alpha _{2},\alpha _{3}\right) \), \(\mathbf {n}=\left( n_{1},n_{2}\right) \).

Using induction method, when we apply Theorem (3.1) successively, it is satisfied for each r.

4.3 Proof of Theorem 3.3

For \(r=1\), by use of (3.2) and (4.1) in Parseval’s identity, Koelink [2] proved that the special function

$$\begin{aligned}&_{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1},b_{2}\right) =\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {-n_{1} ,a_{2}+x_{1}/2,\ n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1}}\\ [-0.1ex] {\displaystyle {a_{2}+b_{2},a_{1}+a_{2}}} \end{array}\Big | \,{\displaystyle {1}} \right) \nonumber \\&\quad =\frac{{n_{1}}!i^{-n_{1}}}{\left( a_{1}+a_{2}\right) _{n_{1}}\left( a_{2}+b_{2}\right) _{n_{1}}}p_{n_{1}}\left( \frac{-ix_{1}}{2};a_{2},b_{1},b_{2},a_{1}\right) \end{aligned}$$
(4.5)

has an orthogonality relation of the form

$$\begin{aligned}&\int \limits _{-\infty }^{\infty }\Gamma \left( a_{1}-\frac{ix_{1}}{2}\right) \Gamma \left( a_{2}+\frac{ix_{1}}{2}\right) \Gamma \left( b_{1}+\frac{ix_{1}}{2} \right) \Gamma \left( b_{2}-\frac{ix_{1}}{2}\right) \\&\qquad \times \ _{1}S_{n_{1}}\left( ix_{1};a_{1},a_{2},b_{1},b_{2}\right) \ _{1}S_{m_{1}}\left( -ix_{1};b_{1},b_{2},a_{1},a_{2}\right) \mathrm{d}x_{1} \\&\quad =\frac{2^{2}\pi {n_{1}}!\Gamma \left( n_{1}+a_{1}+b_{1}\right) \Gamma ^{2}\left( a_{2}+b_{2}\right) \Gamma \left( a_{1}+a_{2}\right) }{\left( 2n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1\right) \Gamma \left( n_{1}+a_{2}+b_{2}\right) } \frac{\Gamma \left( b_{1}+b_{2}\right) }{\Gamma \left( n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1\right) }\delta _{n_{1},m_{1}}\\&\quad = \frac{2^{2}\pi \left( {n_{1}}!\right) ^{2}~\Gamma \left( a_{1}+a_{2}\right) \Gamma \left( b_{1}+b_{2}\right) }{\left( \left( a_{2}+b_{2}\right) _{n_{1}}\right) ^{2}} h_{n_{1}}^{\left( a_{1}+b_{1}-1,a_{2}+b_{2}-1\right) } \delta _{n_{1},m_{1}}, \end{aligned}$$

where \(h_{n_{1}}^{\left( a_{1}+b_{1}-1,a_{2}+b_{2}-1\right) }\) is defined as (2.8), from which it follows:

$$\begin{aligned}&\int \limits _{-\infty }^{\infty }\Gamma \left( a_{2}+ix_{1}\right) \Gamma \left( a_{1}-ix_{1}\right) \Gamma \left( b_{1}+ix_{1}\right) \Gamma \left( b_{2}-ix_{1}\right) \\&\qquad \times p_{n_{1}}\left( x_{1};\ a_{2},\ b_{1},\ b_{2},\ a_{1}\right) p_{m_{1}}\left( x_{1};\ a_{2},\ b_{1},\ b_{2},\ a_{1}\right) \mathrm{d}x_{1} \\&\quad =\frac{2\pi \Gamma \left( n_{1}+a_{2}+b_{2}\right) \Gamma \left( n_{1}+a_{1}+b_{1}\right) \Gamma \left( n_{1}+a_{1}+a_{2}\right) \Gamma \left( n_{1}+b_{1}+b_{2}\right) }{{n_{1}}!\left( 2n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1\right) \Gamma \left( n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1\right) }\delta _{n_{1},m_{1}}, \end{aligned}$$

for \(a_{1}\), \(a_{2}\), \(b_{1}\), \(b_{2}>0\), which gives the orthogonality relation for continuous Hahn polynomials denoted by \(p_{n_{1}}\left( x_{1};\ a_{2},\ b_{1},\ b_{2},\ a_{1}\right) \).

For the case \(r=2\), by substituting (4.2) and (4.7) in Parseval’s identity, we get

$$\begin{aligned}&\int \limits _{-\infty }^{\infty }\int \limits _{-\infty }^{\infty }g_{2}\left( x_{1},x_{2};n_{1},n_{2},a_{1},a_{2},a_{3},\alpha _{1},\alpha _{2},\alpha _{3}\right) \\&\qquad \times g_{2}\left( x_{1},x_{2};m_{1},m_{2},b_{1},b_{2},b_{3},\beta _{1},\beta _{2},\beta _{3}\right) \mathrm{d}x_{1}\mathrm{d}x_{2} \\&\quad =\int \limits _{-\infty }^{\infty }\int \limits _{-\infty }^{\infty }\left( 1+\tanh x_{1}\right) ^{a_{1}+b_{1}}\left( 1-\tanh x_{1}\right) ^{a_{2}+a_{3}+b_{2}+b_{3}}\left( 1+\tanh x_{2}\right) ^{a_{2}+b_{2}} \\&\qquad \times \left( 1-\tanh x_{2}\right) ^{a_{3}+b_{3}} P_{n_{1},n_{2}}^{\left( \alpha _{1},\alpha _{2},\alpha _{3}\right) }\left( \Upsilon _{1}(x_{1}),\Upsilon _{2}(x_{1},x_{2}) \right) \\&\qquad \times P_{m_{1},m_{2}}^{\left( \beta _{1},\beta _{2},\beta _{3}\right) }\left( \Upsilon _{1}(x_{1}),\Upsilon _{2}(x_{1},x_{2}) \right) \mathrm{d}x_{1}\mathrm{d}x_{2} \\&\quad =\frac{1}{\left( 2\pi \right) ^{2}}\int \limits _{-\infty }^{\infty }\int \limits _{-\infty }^{\infty }\frac{\left( 2n_{2}+\alpha _{2}+\alpha _{3}+2\right) _{n_{1}}\left( 2m_{2}+\beta _{2}+\beta _{3}+2\right) _{m_{1}} \left( \alpha _{3}+1\right) _{n_{2}}}{ 2^{4-a_{1}-b_{1}-2\left( a_{2}+a_{3}+b_{2}+b_{3}\right) } n_{1}!n_{2}!m_{1}!m_{2}! }\\&\qquad \times \left( \beta _{3}+1\right) _{m_{2}} \Lambda _{1}^{2}(\mathbf {a},{\varvec{\alpha }},\mathbf {n};\xi _{1}) \overline{\Lambda _{1}^{2}(\mathbf {b},\varvec{\beta },\mathbf {m};\xi _{1}) } \Lambda _{2}^{2}(\mathbf {a},{\varvec{\alpha }},\mathbf {n};\xi _{2}) \overline{\Lambda _{2}^{2}(\mathbf {b},\varvec{\beta },\mathbf {m};\xi _{2}) } \mathrm{d}\xi _{1}\mathrm{d}\xi _{2}, \end{aligned}$$

where \(\mathbf {n}=\left( n_{1}, n_{2}\right) \), \(\mathbf {m}=\left( m_{1}, m_{2}\right) \), \({\varvec{\alpha }}=\left( \alpha _{1},\alpha _{2},\alpha _{3}\right) \), \(\varvec{\beta }=\left( \beta _{1},\beta _{2},\beta _{3}\right) \), \(\mathbf {a}=\left( a_{1},a_{2},a_{3}\right) \) and \(\mathbf {b}=\left( b_{1},b_{2},b_{3}\right) \).

Now, using the transforms \(\tanh x_{1}=2u-1\) and \(\tanh x_{2}=\frac{2v}{1-u}-1\) in the left-hand side of the equality, respectively, we have

$$\begin{aligned}&2^{4}\pi ^{2}\int \limits _{0}^{1}\int \limits _{0}^{1-u} u^{a_{1}+b_{1}-1} v^{a_{2}+b_{2}-1}\left( 1-u-v\right) ^{a_{3}+b_{3}-1}\nonumber \\&\quad P_{n_{1},n_{2}}^{\left( \alpha _{1},\alpha _{2},\alpha _{3}\right) }\left( u,v\right) P_{m_{1},m_{2} }^{\left( \beta _{1},\beta _{2},\beta _{3}\right) }\left( u,v\right) dvdu\nonumber \\&\quad =\int \limits _{-\infty }^{\infty }\int \limits _{-\infty }^{\infty }\frac{ \left( 2n_{2}+\alpha _{2}+\alpha _{3}+2\right) _{n_{1}} \left( 2m_{2}+\beta _{2}+\beta _{3}+2\right) _{m_{1}} \left( \alpha _{3}+1\right) _{n_{2}} \left( \beta _{3}+1\right) _{m_{2}}}{n_{1}!n_{2}!m_{1}!m_{2}!}\nonumber \\&\qquad \times \Lambda _{1}^{2}(\mathbf {a},{\varvec{\alpha }},\mathbf {n};\xi _{1})\overline{\Lambda _{1}^{2}(\mathbf {b},\varvec{\beta },\mathbf {m};\xi _{1}) } \Lambda _{2}^{2}(\mathbf {a},{\varvec{\alpha }},\mathbf {n};\xi _{2}) \overline{\Lambda _{2}^{2}(\mathbf {b},\varvec{\beta },\mathbf {m};\xi _{2}) } \mathrm{d}\xi _{1}\mathrm{d}\xi _{2}. \end{aligned}$$
(4.6)

On the other hand, if in the left-hand side of (4.6), we take \(a_{1}+b_{1}-1=\alpha _{1}=\beta _{1}\), \(a_{2}+b_{2}-1=\alpha _{2}=\beta _{2}\), and \(a_{3}+b_{3}-1=\alpha _{3}=\beta _{3}\), then according to the orthogonality relation (2.7), Eq. (4.6) reduces to

$$\begin{aligned}&2^{4}\pi ^{2}\int \limits _{0}^{1}\int \limits _{0}^{1-u} u^{a_{1}+b_{1}-1} v^{a_{2}+b_{2}-1}\left( 1-u-v\right) ^{a_{3}+b_{3}-1}\\&\qquad \times P_{n_{1},n_{2}}^{\left( a_{1}+b_{1}-1,a_{2}+b_{2}-1,a_{3}+b_{3}-1\right) }\left( u,v\right) P_{m_{1},m_{2} }^{\left( a_{1}+b_{1}-1,a_{2}+b_{2}-1,a_{3}+b_{3}-1\right) }\left( u,v\right) dvdu\\&\quad =2^{4}\pi ^{2}h_{n_{1},n_{2}}^{\left( a_{1}+b_{1}-1,a_{2}+b_{2}-1,a_{3}+b_{3}-1\right) }\delta _{n_{1},m_{1}}\delta _{n_{2},m_{2}} \\&\quad =\int \limits _{-\infty }^{\infty }\int \limits _{-\infty }^{\infty }\frac{ \left( 2n_{2}+\alpha _{2}+\alpha _{3}+2\right) _{n_{1}} \left( 2m_{2}+\beta _{2}+\beta _{3}+2\right) _{m_{1}} \left( \alpha _{3}+1\right) _{n_{2}} \left( \beta _{3}+1\right) _{m_{2}}}{n_{1}!n_{2}!m_{1}!m_{2}!}\\&\qquad \times \Lambda _{1}^{2}(\mathbf {a},{\varvec{\alpha }},\mathbf {n};\xi _{1})\overline{\Lambda _{1}^{2}(\mathbf {b},\varvec{\beta },\mathbf {m};\xi _{1}) } \Lambda _{2}^{2}(\mathbf {a},{\varvec{\alpha }},\mathbf {n};\xi _{2}) \overline{\Lambda _{2}^{2}(\mathbf {b},\varvec{\beta },\mathbf {m};\xi _{2}) } \mathrm{d}\xi _{1}\mathrm{d}\xi _{2}, \end{aligned}$$

so

$$\begin{aligned}&\frac{2^{4}\pi ^{2}\left( n_{1}!\right) ^{2}\left( n_{2}!\right) ^{2}\Gamma \left( n_{2}+a_{1}+a_{2}+a_{3}\right) \Gamma \left( n_{2}+b_{1}+b_{2}+b_{3}\right) \Gamma \left( a_{2}+a_{3}\right) \Gamma \left( b_{2}+b_{3}\right) }{\left( 2n_{2}+a_{2}+b_{2}+a_{3}+b_{3}\right) _{n_{1}}^{2} \left( a_{3}+b_{3}\right) _{n_{2}}^{2} }\\&\qquad \times h_{n_{1},n_{2}}^{\left( a_{1}+b_{1}-1,a_{2}+b_{2}-1,a_{3}+b_{3}-1\right) }\delta _{n_{1},m_{1}}\delta _{n_{2},m_{2}} \\&\quad =\int \limits _{-\infty }^{\infty }\int \limits _{-\infty }^{\infty }\Theta _{2}\left( \xi _{1},\xi _{2};a_{1},a_{2},a_{3}\right) \Lambda _{2}\left( \xi _{1},\xi _{2};n_{1},n_{2},a_{1},a_{2},a_{3},b_{1},b_{2},b_{3}\right) \\&\qquad \times \overline{\Theta _{2}\left( \xi _{1},\xi _{2};b_{1},b_{2},b_{3}\right) \Lambda _{2}\left( \xi _{1},\xi _{2};m_{1},m_{2},b_{1},b_{2},b_{3},a_{1},a_{2},a_{3}\right) }\mathrm{d}\xi _{1}\mathrm{d}\xi _{2}, \end{aligned}$$

where

$$\begin{aligned}&\Theta _{2}\left( \xi _{1},\xi _{2};a_{1},a_{2},a_{3}\right) \\&\quad =\Gamma \left( a_{1}-\frac{i\xi _{1}}{2}\right) \Gamma \left( a_{2}+a_{3}+\frac{i\xi _{1}}{2}\right) \Gamma \left( a_{2}-\frac{i\xi _{2}}{2}\right) \Gamma \left( a_{3}+\frac{i\xi _{2}}{2}\right) \left( a_{2}+a_{3}+\frac{i\xi _{1}}{2}\right) _{n_{2}}, \\&\Lambda _{2}\left( \xi _{1},\xi _{2};n_{1},n_{2},a_{1},a_{2},a_{3},b_{1},b_{2},b_{3}\right) \\&\quad =\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {-n_{2},\ n_{2}+a_{2}+b_{2}+a_{3}+b_{3}-1,\ a_{3}+\frac{i\xi _{2}}{2}}}\\ [-0.1ex] {\displaystyle {a_{3}+b_{3},\ a_{2}+a_{3} }} \end{array}\Big | \,{\displaystyle {1}} \right) \\&\qquad \times \,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {-n_{1},\ n_{1}+2n_{2}+a_{1}+b_{1}+a_{2}+b_{2}+a_{3}+b_{3}-1 ,\ n_{2}+a_{2}+a_{3}+\frac{i\xi _{1}}{2}}}\\ [-0.1ex] {\displaystyle {2n_{2}+a_{2}+b_{2}+a_{3}+b_{3},\ n_{2}+a_{1}+a_{2}+a_{3}}} \end{array}\Big | \,{\displaystyle {1}} \right) , \end{aligned}$$

or, in terms of the Hahn polynomials (2.5)

$$\begin{aligned}&\Lambda _{2}\left( \xi _{1},\xi _{2};n_{1},n_{2},a_{1},a_{2},a_{3},b_{1},b_{2},b_{3}\right) \\&\quad =\frac{n_{1}!n_{2}!i^{-n_{1}-n_{2}}}{\left( a_{2}+a_{3}\right) _{n_{2}}\left( a_{3}+b_{3}\right) _{n_{2}}\left( n_{2}+a_{1}+a_{2}+a_{3}\right) _{n_{1}}\left( 2n_{2}+a_{2}+b_{2}+a_{3}+b_{3}\right) _{n_{1}}} \\&\qquad \times p_{n_{1}}\left( \frac{\xi _{1}}{2} ;n_{2}+a_{2}+a_{3},b_{1},n_{2}+b_{2}+b_{3},a_{1}\right) p_{n_{2}}\left( \frac{\xi _{2}}{2};a_{3},b_{2},b_{3},a_{2}\right) , \end{aligned}$$

and \(h_{n_{1},n_{2}}^{\left( \alpha _{1},\alpha _{2},\alpha _{3}\right) }\) is given by (2.8). By applying the method in the cases \(r=1\) and \(r=2\), if we substitute (3.1) and (3.7) in the Parseval’s identity (3.8), after the necessary calculations, we obtain the desired result.

4.4 Proof of Proposition 3.6

We first consider the case \(r=1\). Substituting \(m_{1}\rightarrow -n_{1}\), \(m_{2}\rightarrow n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1\), \(m_{3}\rightarrow a_{2}+\frac{x_{1}}{2}\), \( s_{1}\rightarrow a_{2}+b_{2}\), \(s_{2}\rightarrow a_{1}+a_{2}\) and \( z\rightarrow 1\) in the relation (3.10) and taking into account the relation

$$\begin{aligned} _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1},b_{2}\right) =\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {-n_{1} ,\ a_{2}+x_{1} /2,\ n_{1} +a_{1}+a_{2}+b_{1}+b_{2}-1}}\\ [-0.1ex] {\displaystyle {a_{2}+b_{2},\ a_{1}+a_{2}}} \end{array}\Big | \,{\displaystyle {1}} \right) \end{aligned}$$

given by (4.5), we obtain the recurrence relation for the family of the special functions \(\ _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1},b_{2}\right) \)

$$\begin{aligned}&\left( B_{1}+C_{1}\right) \ _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1},b_{2}\right) +\left( B_{2}+C_{2}\right) \ _{1}S_{n_{1}+1}\left( x_{1};a_{1},a_{2},b_{1}-1,b_{2}\right) \nonumber \\&\quad +B_{3}\ _{1}S_{n_{1}+2}\left( x_{1};a_{1},a_{2},b_{1}-2,b_{2}\right) =0, \end{aligned}$$
(4.7)

where

$$\begin{aligned} B_{1}+C_{1}&=\frac{b_{1}-1+\frac{x_{1}}{2}}{n_{1}}, \quad B_{3}=-\frac{\left( n_{1}+a_{2}+b_{2}+1\right) \left( n_{1}+a_{1}+a_{2}+1\right) }{\left( n_{1}\right) _{2}}, \\ B_{2}+C_{2}&=\frac{\left( a_{2}+b_{2}\right) \left( a_{1}+a_{2}\right) -\left( n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1\right) \left( a_{2}+\frac{x_{1}}{2} \right) }{\left( n_{1}\right) _{2}} \\&\quad +\frac{n_{1}+a_{1}+2a_{2}-b_{1}+b_{2}+2-\frac{x_{1}}{2}}{n_{1}}. \end{aligned}$$

For \(r=2\), the relation (3.13) gives the result

$$\begin{aligned}&\ _{2}S_{n_{1},n_{2}}\left( x_{1},x_{2};a_{1},a_{2},a_{3},b_{1},b_{2},b_{3}\right) \nonumber \\&\quad =\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle {-n_{2},\ n_{2}+a_{2}+a_{3}+b_{2}+b_{3}-1,\ a_{3}+\frac{x_{2}}{2}}}\\ [-0.1ex] {\displaystyle {a_{3}+b_{3},\ a_{2}+a_{3}}} \end{array}\Big | \,{\displaystyle {1}} \right) \nonumber \\&\qquad \times \left( a_{2}+a_{3}+\frac{x_{1}}{ 2}\right) _{n_{2}} \ _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2}+a_{3}+n_{2},b_{1},b_{2}+b_{3}+n_{2}\right) . \end{aligned}$$
(4.8)

If the equation obtained by taking \(b_{2}\rightarrow n_{2}+b_{2}+b_{3}\) ve \( a_{2}\rightarrow n_{2}+a_{2}+a_{3}\) in relation (4.7) is multiplied by

$$\begin{aligned} \left( a_{2}+a_{3}+\frac{x_{1}}{2}\right) _{n_{2}}\,{}_{3}F_{2}\left( \begin{array}{cc}{\displaystyle { -n_{2},\ n_{2}+a_{2}+a_{3}+b_{2}+b_{3}-1,\ a_{3}+\frac{x_{2}}{2}}}\\ [-0.1ex] {\displaystyle { a_{3}+b_{3},\ a_{2}+a_{3}}} \end{array}\Big | \,{\displaystyle {1}} \right) \end{aligned}$$

and the relation (4.8) is used, we get

$$\begin{aligned}&\frac{b_{1}-1+\frac{x_{1}}{2}}{n_{1}}\ _{2}S_{n_{1},n_{2}}\left( x_{1},x_{2};a_{1},a_{2},a_{3},b_{1},b_{2},b_{3}\right) \\&\quad +\left( \frac{\left( n_{2}+a_{1}+a_{2}+a_{3}\right) \left( 2n_{2}+a_{2}+a_{3}+b_{2}+b_{3}\right) }{\left( n_{1}\right) _{2}}\right. \\&\quad -\frac{\left( n_{1}+2n_{2}+a_{1}+a_{2}+a_{3}+b_{1}+b_{2}+b_{3}-1\right) \left( n_{2}+a_{2}+a_{3}+\frac{x_{1}}{2}\right) }{\left( n_{1}\right) _{2}} \\&\quad \left. +\frac{n_{1}+3n_{2}+a_{1}+2\left( a_{2}+a_{3}\right) -b_{1}+b_{2}+b_{3}+2-\frac{x_{1}}{2}}{n_{1}}\right) \\&\quad \times \ _{2}S_{n_{1}+1,n_{2}}\left( x_{1},x_{2};a_{1},a_{2},a_{3},b_{1}-1,b_{2},b_{3}\right) \\&\quad -\frac{\left( n_{1}+2n_{2}+a_{2}+a_{3}+b_{2}+b_{3}+1\right) \left( n_{1}+n_{2}+a_{1}+a_{2}+a_{3}+1\right) }{\left( n_{1}\right) _{2}} \\&\quad \times \ _{2}S_{n_{1}+2,n_{2}}\left( x_{1},x_{2};a_{1},a_{2},a_{3},b_{1}-2,b_{2},b_{3}\right) =0. \end{aligned}$$

Similarly, if relation (4.7) is applied consecutively in view of the relation (3.13), then the statement of Proposition (3.6) is obtained.

4.5 Proof of Proposition 3.7

By taking \(m_{1}\rightarrow n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1\), \( m_{2}\rightarrow -n_{1}\), \(m_{3}\rightarrow a_{2}+\frac{x_{1}}{2}\), \( s_{1}\rightarrow a_{2}+b_{2}\), \(s_{2}\rightarrow a_{1}+a_{2}\) and \( z\rightarrow 1\) in the relation (3.10), we obtain

$$\begin{aligned}&\left( B_{1}+C_{1}\right) \ _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1},b_{2}\right) +\left( B_{2}+C_{2}\right) \ _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1}-1,b_{2}\right) \nonumber \\&\quad +B_{3}\ _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1}-2,b_{2}\right) =0, \end{aligned}$$
(4.9)

where

$$\begin{aligned} B_{1}+C_{1}&=-\frac{b_{1}-1+\frac{x_{1}}{2}}{ n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1}, \\ B_{3}&=-\frac{\left( n_{1}+a_{1}+b_{1}-2\right) \left( n_{1}+b_{1}+b_{2}-2\right) }{\left( n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-2\right) _{2}}, \\ B_{2}+C_{2}&=\frac{\left( a_{2}+b_{2}\right) \left( a_{1}+a_{2}\right) +n_{1}\left( a_{2}+\frac{x_{1}}{2}\right) }{\left( n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-2\right) _{2}}+\frac{n_{1}-a_{2}+2b_{1}-3+ \frac{x_{1}}{2}}{n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1} \end{aligned}$$

for the case \(r=1\). If we apply the recurrence relation (4.9) successively, we obtain the required result in view of the relation (3.14).

4.6 Proof of Proposition 3.8

For the case \(r=1\), if we get \(m_{1}\rightarrow a_{2}+\frac{x_{1}}{2}\), \(m_{2}\rightarrow -n_{1}\), \(m_{3}\rightarrow n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1\), \( s_{1}\rightarrow a_{2}+b_{2}\), \(s_{2}\rightarrow a_{1}+a_{2}\) and \( z\rightarrow 1\) in relation (3.10), the following relation holds for the family of the special function \(\ _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1},b_{2}\right) \)

$$\begin{aligned}&\left( B_{1}+C_{1}\right) \ _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1},b_{2}\right) -\left( B_{2}+C_{2}\right) \ _{1}S_{n_{1}}\left( x_{1};a_{1}+1,a_{2}-1,\right. \nonumber \\&\quad \left. b_{1}-1,b_{2}+1\right) +B_{3}\ _{1}S_{n_{1}}\left( x_{1};a_{1}+2,a_{2}-2,b_{1}-2,b_{2}+2\right) =0, \end{aligned}$$

where

$$\begin{aligned} B_{1}+C_{1}&=\left( b_{1}-1+\frac{x_{1}}{2}\right) \left( a_{2} -1+\frac{x_{1}}{2}\right) , \\ B_{3}&=\left( a_{1}+1-\frac{x_{1}}{2}\right) \left( b_{2}+1-\frac{x_{1}}{2}\right) ,\\ B_{2}+C_{2}&=\left( a_{2}+b_{2}\right) \left( a_{1}+a_{2}\right) +n_{1}\left( n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1\right) \\&\quad -\left( a_{1}+a_{2}-b_{1}+b_{2}+2-x_{1}\right) \left( a_{2}-1+\frac{x_{1}}{2}\right) . \end{aligned}$$

It is enough to consider the relation (3.14) and use the last relation successively to obtain the statement of Proposition (3.8).

4.7 Proof of Proposition 3.9

Taking \(m_{1}\rightarrow -n_{1}\), \(m_{2}\rightarrow n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1\), \(m_{3}\rightarrow a_{2}+\frac{x_{1}}{2}\), \( s_{1}\rightarrow a_{1}+a_{2}\), \(s_{2}\rightarrow a_{2}+b_{2}\) and \( z\rightarrow 1\) in (3.11), we arrive at the following relation for the case \(r=1\):

$$\begin{aligned}&\left( B_{1}+C_{1}\right) \ _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1},b_{2}\right) +\left( B_{2}+C_{2}\right) \ _{1}S_{n_{1}}\left( x_{1};a_{1}+1,a_{2},b_{1}-1,b_{2}\right) \\&\quad +C_{3}\ _{1}S_{n_{1}}\left( x_{1};a_{1}+2,a_{2},b_{1}-2,b_{2}\right) =0, \end{aligned}$$

where

$$\begin{aligned} B_{1}+C_{1}&=-\frac{b_{1}-1+\frac{x_{1}}{2}}{a_{1}+a_{2}-1}, \\ C_{3}&=-\frac{\left( n_{1}+a_{1}+a_{2}+1\right) \left( n_{1}+b_{1}+b_{2}-2\right) \left( a_{1}+1-\frac{x_{1}}{2}\right) }{\left( a_{1}+a_{2}-1\right) _{3}}, \\ B_{2}+C_{2}&=\frac{\left( 2\left( a_{1}+a_{2}\right) +1\right) \left( a_{2}+b_{1}+b_{2}-2+\frac{x_{1}}{2}\right) }{\left( a_{1}+a_{2}-1\right) _{2}} \\&\quad +\frac{n_{1}\left( n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1\right) }{\left( a_{1}+a_{2}-1\right) _{2}} \\&\quad -\frac{\left( a_{1}+a_{2}+b_{1}+b_{2}-1\right) \left( a_{2}+\frac{x_{1}}{2}\right) }{\left( a_{1}+a_{2}-1\right) _{2}}-\frac{a_{2}+b_{2}-1}{a_{1}+a_{2}-1}. \end{aligned}$$

If this relation is applied consecutively in (3.13), the proof is completed.

4.8 Proof of Proposition 3.10

For the case \(r=1\), if we replace \(m_{1}\rightarrow -n_{1}\), \(m_{2}\rightarrow n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1\), \(m_{3}\rightarrow a_{2}+\frac{x_{1}}{2}\), \( s_{1}\rightarrow a_{2}+b_{2}\), \(s_{2}\rightarrow a_{1}+a_{2}\) and \( z\rightarrow 1\) in relation (3.11), the family of the special function\(\ _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1},b_{2}\right) \) satisfies the relation

$$\begin{aligned}&\left( B_{1}+C_{1}\right) \ _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1},b_{2}\right) +\left( B_{2}+C_{2}\right) \ _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1}-1,b_{2}+1\right) \\&\quad +C_{3}\ _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1}-2,b_{2}+2\right) =0, \end{aligned}$$

where

$$\begin{aligned} B_{1}+C_{1}&=-\frac{b_{1}-1+\frac{x_{1}}{2}}{a_{2}+b_{2}-1}, \\ C_{3}&=-\frac{\left( n_{1}+a_{2}+b_{2}+1\right) \left( n_{1}+a_{1}+b_{1}-2\right) \left( b_{2}+1-\frac{x_{1}}{2}\right) }{\left( a_{2}+b_{2}-1\right) _{3}}, \\ B_{2}+C_{2}&=\frac{\left( 2\left( a_{2}+b_{2}\right) +1\right) \left( a_{1}+a_{2}+b_{1}-2+\frac{x_{1}}{2}\right) }{\left( a_{2}+b_{2}-1\right) _{2}} \\&\quad +\frac{n_{1}\left( n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1\right) }{\left( a_{2}+b_{2}-1\right) _{2}} \\&\quad -\frac{\left( a_{1}+a_{2}+b_{1}+b_{2}-1\right) \left( a_{2}+\frac{x_{1}}{2}\right) }{\left( a_{2}+b_{2}-1\right) _{2}}-\frac{a_{1}+a_{2}-1}{a_{2}+b_{2}-1}. \end{aligned}$$

Applying this relation consecutively in (3.14) provides to obtain the statement of Proposition (3.10).

4.9 Proof of Proposition 3.11

By taking \(m_{1}\rightarrow n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1\), \(m_{2}\rightarrow -n_{1}\), \(m_{3}\rightarrow a_{2}+\frac{x_{1}}{2}\), \(s_{1}\rightarrow a_{1}+a_{2}\), \(s_{2}\rightarrow a_{2}+b_{2}\) and \(z\rightarrow 1\) in relation (3.12), we find the recurrence relation for the family of the special function\(\ _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1},b_{2}\right) \) for the case \(r=1\)

$$\begin{aligned}&\left( a_{1}+a_{2}\right) \ _{1}S_{n_{1}}\left( x_{1};a_{1},a_{2},b_{1},b_{2}\right) \\&\quad +\left( n_{1}+b_{1}+b_{2}-1\right) \ _{1}S_{n_{1}}\left( x_{1};a_{1}+1,a_{2},b_{1}-1,b_{2}\right) \\&\quad -\left( n_{1}+a_{1}+a_{2}+b_{1}+b_{2}-1\right) \ _{1}S_{n_{1}}\left( x_{1};a_{1}+1,a_{2},b_{1},b_{2}\right) =0. \end{aligned}$$

If we use the last relation consecutively by taking into account (3.14), it is seen that the family of the special function\(\ _{r}S_{\mathbf {n}}\left( \mathbf {x};\mathbf {a},\mathbf {b}\right) \) satisfies the desired recurrence relation for \(r\ge 1\).