1 Introduction

The objective of this note is to derive some exponential tail bounds for chisquared random variables. The bounds are non-asymptotic, but they can be used very successfully for asymptotic derivations as well. As a corollary, one can get tail bounds for F-statistics as well. Also, I show how some exact moderate deviation [4] inequalities can be obtained as special cases of these tail bounds.

The chisquared random variables are special cases of sub-exponential random variables. We examine when the bounds obtained here are sharper than the ones that use only the sub-exponentiality of chisquares.

The outline of the next two sections is as follows. Exponential tail bounds for central chisquares are given in Sect. 2. Corresponding bounds for non-central chisquares are given in Sect. 3.

2 Central Chisquare

We begin with an upper tail bound for central chisquares. The following theorem is proved.

Theorem 1

Suppose \(X\sim \chi _p^2\). Then for \(a>p\), \(P(X>a)\le \exp [-\frac{p}{2}\{\frac{a}{p}-1-\log (\frac{a}{p})\}]\).

Proof

From Markov’s inequality in its exponential form (see for example, [2] or [3], one gets

$$\begin{aligned} P(X>a)\le & {} \text{ inf}_{0<t<1/2}[\exp (-ta)E\{\exp (tX)\}]\nonumber \\\le & {} \text{ inf}_{0<t<1/2}[\exp (-ta)(1-2t)^{-p/2}]. \end{aligned}$$
(1)

Let \(g(t)=-ta-(p/2)\log (1-2t)\). Then \(g^{\prime }(t)=-a+p(1-2t)^{-1}\) and \(g^{\prime \prime }(t)=2p(1-2t)^{-2}(>0)\). Hence, g(t) is minimized at \(t=t_0=(1/2) \left( 1-\frac{p}{a}\right)\). Substitution in (1) yields

$$\begin{aligned} P(X>a)&\le \exp (-t_0a)(1-2t_0)^{-p/2}\\&= \exp \left[ -\frac{a}{2}\left( 1-\frac{p}{a}+\frac{p}{2}\log \left( \frac{a}{p}\right) \right) \right] \\&= \exp \left[ -\frac{p}{2}\left( \frac{a}{p}-1-\log \left( \frac{a}{p}\right) \right) \right] . \end{aligned}$$

This proves the theorem.

Suppose \(a=p+c\). Then an equivalent way of writing the above result is

$$\begin{aligned} P(X-p>c)\le \exp \left[ -\frac{p}{2}\left\{ \frac{c}{p}-\log \left( 1+\frac{c}{p}\right) \right\} \right] . \end{aligned}$$
(2)

By the inequality

$$\begin{aligned} x-\log (1+x)=\int _{0}^x\left[ 1-\frac{1}{1+y}\right] \mathrm{d}y=\int _{0}^x\frac{y}{1+y}\mathrm{d}y \ge \frac{x^2}{1+x}, \end{aligned}$$

a weaker version of (2) is given by

$$\begin{aligned} P(X>p+c)\le \exp \left[ -\frac{p}{2}(c/p)^{2}(1+c/p)^{-1}\right] \le \exp \left[ -\frac{c^2}{4(p+c)}\right] . \end{aligned}$$
(3)

It may be noted that a chisquare random variable is a special case of a sub-exponential random variable. There are several equivalent definitions of sub-exponential random variables. The one we find convenient is given as follows (see [5], p 26).

Definition

A random variable X with mean \(\mu\) is said to be sub-exponential (\(\nu ,\alpha )\) if \(E[\exp \{t(X-i\mu )\}]\le \exp (t^2\nu ^2/2)\) whenever \(|t|<\alpha ^{-1}\).

If \(X\sim \chi _p^2\), then X is subexponential (2p, 4). To see this, we note that \(E[\exp \{t(X-p)\}]=\exp (-tp)(1-2t)^{-p/2}\le \exp (2pt^2)\) with \(|t|\le 1/4\). Now \(P(X-p>c)\le \text{ inf}_{0<t<1/4} \exp (-tc+2pt^2)=\exp (-c^2/(8p))\). The inequality given in (3) is sharper than the last one when \(0<c<p\). Moreover, as \(p\rightarrow \infty\), it follows from (2) that \(P(X-p>c)\sim \exp (-c^2/(4p))\), while sub-exponentiality continues to yield the same upper bound \(\exp (-c^2/(8p))\).

The next inequality is related to the lower tail of a chisquared random variable. The following theorem is proved.

Theorem 2

Suppose \(X\sim \chi _p^2\). Then for \(0<c<p\),

$$\begin{aligned} P(X-p<-c)\le \exp \left[ (p/2)\left\{ \frac{c}{p}+\log \left( 1-\frac{c}{p}\right) \right\} \right] \le \exp \left( -\frac{c^2}{4p}\right) . \end{aligned}$$

Proof

The second inequality is an easy consequence of expansion of a logarithmic function. To prove the first inequality, we begin with

$$\begin{aligned} P(X-p<-c)\le \text{ inf}_{t<0}\left[ \exp \{-t(p-c)\}(1-2t)^{-p/2}\right] . \end{aligned}$$
(4)

Similar as before, let \(g(t)=-t(p-c)-(p/2)\log (1-2t)\). Then \(g^{\prime }(t)=-(p-c)+p(1-2t)^{-1}\) and \(g^{\prime \prime }(t)=2p(1-2t)^{-2}\). Hence, g(t) is minimized at \(t=t_0\) where \(1-2t_0=p/(p-c)\), i.e., \(t_0=-c/[2(p-c)]\). Substituting \(t_0\) for t in (4), one gets the inequality

$$\begin{aligned} P(X-p<-c)\le \exp (c)\left( \frac{p}{p-c}\right) ^{-p/2} =\exp \left[ \frac{p}{2}\left\{ \frac{c}{p}+\log \left( 1-\frac{c}{p}\right) \right\} \right] . \end{aligned}$$

This proves the theorem.

The exact upper bound given in the rightmost side of Theorem 2 is stronger than the similar sub-exponential bound \(\exp \left( -\frac{c^2}{8p}\right)\). Moreover, since \(p+c>p\), it is possible to combine (3) with Theorem 2 to get the inequality

$$\begin{aligned} P(|\chi _p^2-p|>c)\le 2\exp \left[ -\frac{c^2}{4(p+c)}\right] . \end{aligned}$$
(5)

Since \(\chi _p^2/p\) is the average of p iid \(\chi _1^2\) random variables, each with mean 1 and variance 2, the central limit theorem leads to \((\chi _p^2-p)/(\sqrt{2p}{\mathop {\rightarrow }\limits ^{d}}\text{ N }(0,1)\). For averages of p iid random variables with nonzero and finite variance, [4] provided an asymptotic two sided tail bound for deviations of the order \(\sqrt{\log p}\). Putting \(c=\sqrt{2p\log p}\), one gets the asymptotic upper bound \(\exp \left( -\frac{2p\log p}{4p}\right) =p^{-1/2}\) which is slightly weaker than \(O(p^{-1/2}(\log p)^{-1/2})\), one obtained by Rubin and Sethuraman in conformity with Mill’s ratio.

It is possible to use Theorems 1 and 2 to obtain some crude tail bounds for the F-statistic as well. To see this, suppose rXx and Y are two independent chisquared random variables with respective degrees of freedom \(m_1\) and \(m_2\). I write \(F=\frac{X/m_1}{Y/m_2}\). Then for \(d>1\), and writing \(\delta =(d-1)/(d+1)\),

$$\begin{aligned} P(F>d)&= P[F>(1+\delta )/(1-\delta )] \\&\le P[\{X>m_1(1+\delta )\}\cup \{Y<m_2(1-\delta )\}] \\&\le Pr([X>m_1(1+\delta ))]+P(Y<m_2(1-\delta )). \end{aligned}$$

Putting \(d=m_1(1+\delta )\) in (3) and \(d=m_2(1-\delta )\) in Theorem 2, one gets the bound

$$\begin{aligned} P[F>(1+\delta )/(1-\delta )]\le \exp \left[ -\frac{m_1\delta ^2}{4(m_1+\delta )}\right] +\exp \left[ -\frac{m_2\delta ^2}{4}\right] . \end{aligned}$$
(6)

It is well-known that asymptotically as \(m_2\rightarrow \infty\), the F-statistic reduces to a chisquare statistic divided by its degrees of freedom. This is also reflected in (6). In particular, we get the inequality

$$\begin{aligned} \text{ lim } \text{ sup}_{m_2\rightarrow \infty }P[F>(1+\delta )/(1-\delta )] \le \exp \left[ -\frac{m_1\delta ^2}{4(m_1+\delta )}\right] . \end{aligned}$$

3 Non-Central Chisquare

I find in this section, upper and lower tail bounds for non-central chisquare. These upper bounds are not the sharpest ones that one might get, but they are simple enough for potential use in statistics. I begin with the upper bound.

Theorem 3

Suppose \(X\sim \chi _p^2(\lambda )\). Then for \(c>0\),

$$\begin{aligned} P(X>p+\lambda +c)\le & {} \exp \left[ -\frac{p}{2}\left\{ \frac{c}{p+2\lambda }- \log \left( 1+\frac{c}{p+2\lambda }\right) \right\} \right] \\\le & {} \exp \left[ -\frac{pc^2}{4(p+2\lambda )(p+2\lambda +c)}\right] . \end{aligned}$$

Proof

The second inequality is based on an argument similar to the one used In Theorem 1. For getting the first inequality, I begin with the moment generating function of a non-central chisquare and get

$$\begin{aligned} P(X>p+\lambda +c)\le \text{ inf }f_{0<t<1/2}\exp [-t(p+\lambda +c)] \exp \left( \frac{\lambda t}{1-2t}\right) (1-2t)^{-p/2}. \end{aligned}$$
(7)

Let \(g(t)=-t(p+\lambda +c)+\frac{\lambda t}{1-2t}-\frac{p}{2}\log (1-2t)\). Then \(g^{\prime }(t)=-(p+\lambda +c)+\lambda (1-2t)^{-2}+p(1-2t)^{-1}\) and \(g^{\prime \prime }(t)=4\lambda (1-2t)^{-3}+2p(1-2t)^{-2}>0\). Thus the infimum in (7) is obtained at \(t=t_0\), where \(g^{\prime }(t_0)=0\). Letting \(u=(1-2t)^{-1}\) and noting that u is strictly increasing in t, this amounts to solving the equation \(\lambda u^2+pu-(p+\lambda +c)=0\). The solution is given by \(u_0=\frac{-p+\sqrt{(p+2\lambda )^2+4\lambda c)}}{2\lambda }\). This solution is not too convenient for use in practice. Instead I use the simple inequality \((1+z)^{1/2}<1+\frac{z}{2}\) to get \(u_0<1+\frac{c}{p+2\lambda }=u_1\), say. Correspondingly, \(t_0<t_1=(u_1-1)/(2u_1)=\frac{c}{2(p+2\lambda +c)}\). Substitution of this \(t_1\) for t in (7) yields

$$\begin{aligned} P(X>p+\lambda +c)\le \exp \left[ -\frac{c(p+\lambda +c)}{p+2\lambda +c)}+ \frac{\lambda c}{p+2\lambda }+\frac{p}{2}\log \left( 1+\frac{c}{p+2\lambda }\right) \right] . \end{aligned}$$
(8)

By the inequality, \((p+\lambda +c)/(p+2\lambda +c)>(p+\lambda )/(p+2\lambda )\), it follows on simplification, the right-hand side of (8) is bounded above by \(\exp \left[ -\frac{p}{2}\left( \frac{c}{p+2\lambda }- \log \left( 1+\frac{c}{p+2\lambda }\right) \right) \right]\). This proves the theorem.

The final theorem of this this paper provides a lower tail bound for non-central chisquares.

Theorem 4

Suppose \(X\sim \chi _p^2(\lambda )\). Then for \(0<c<p+\lambda\),

$$\begin{aligned} P(X<p+\lambda -c)\le \exp \left[ \frac{p}{2}\left\{ \frac{c}{p+2\lambda }+ \log \left( 1-\frac{c}{p+2\lambda }\right) \right\} \right] \le \exp \left[ -\frac{pc^2}{4(p+2\lambda )^2}\right] . \end{aligned}$$

Proof

Again, the second inequality is obtained by log expansion. To prove the first inequality, we start with

$$\begin{aligned} P(X<p+\lambda -c)\le \text{ inf}_{t<0}\exp \left[ -t(p+\lambda -c)+\frac{\lambda t}{1-2t}\right] (1-2t)^{-p/2}. \end{aligned}$$
(9)

Let \(g(t)=-t(p+\lambda -c)+\frac{\lambda t}{1-2t}-(p/2)\log (1-2t)\). As before, g(t) is minimized at \(t=t_0\), where \(t_0\) is a solution of \(-(p+\lambda -c)+\lambda (1-2t)^{-2}+p(1-2t)^{-1}=0\). Again, writing \(u=(1-2t)^{-1}\), one needs solving \(\lambda u^2+pu-(p+\lambda -c)=0\). The solution is given by \(u_0=\frac{-p+\sqrt{(p+2\lambda )^2-4\lambda c}}{2\lambda }\). Now by the inequality \((1-z)^{1/2}<1-\frac{z}{2}\), one gets \(u_0<1-\frac{c}{p+2\lambda }=u_1\), say. The corresponding \(t_1=-c/(p+2\lambda -c)(<0)\). Substitution of \(t_1\) for t in (8) leads to the inequality

$$\begin{aligned} P(X<p+\lambda -c)\le \exp \left[ \frac{c(p+\lambda -c)}{p+2\lambda -c}-\frac{\lambda c}{p+2\lambda }+(p/2)\log \left( 1-\frac{c}{p+2\lambda }\right) \right] . \end{aligned}$$

By the inequality, \((p+\lambda -c)/(p+2\lambda -c)\le (p+\lambda )/(p+2\lambda )\), one gets after simplification,

$$\begin{aligned} P(X<p+\lambda -c)\le \exp \left[ \frac{p}{2}\left[ \frac{c}{p+2\lambda }+ \log \left( 1-\frac{c}{p+2\lambda }\right) \right] \right] . \end{aligned}$$

This proves the theorem.

Remark

It is possible to obtain exponential tail-bounds for non-central F-statistic as well. Suppose, for example X and Y are independently distributed with \(X\sim \chi ^2_{m_1}(\lambda _1)\) and \(Y\sim \chi _{m_2}^2(\lambda _2)\). We may recall that \(E(X)=m_1+\lambda _1\) and \(E(Y)=m_2+\lambda _2\). Writing \(F=(X/m_1)/(Y/m_2)\), if \(d=(1+\lambda _1/m_1)/(1+\lambda _2/m_2)(1+\delta )/(1-\delta )\), one can as in (6), get the inequality,

$$\begin{aligned} P(F>d)\le P(X>(m_1+\lambda _1)(1+\delta ))+P(Y<(m_2+\lambda _2)(1-\delta )). \end{aligned}$$

The exponential bounds are now obtained using \(c=(m_1+\lambda _1)\delta\) in Theorem 3 and \(c=(m_2+\lambda _2)\delta\) in Theorem 4.