Implicitly Weighted Methods in Robust Image Analysis

Abstract

This paper is devoted to highly robust statistical methods with applications to image analysis. The methods of the paper exploit the idea of implicit weighting, which is inspired by the highly robust least weighted squares regression estimator. We use a correlation coefficient based on implicit weighting of individual pixels as a highly robust similarity measure between two images. The reweighted least weighted squares estimator is considered as an alternative regression estimator with a clear interpretation. We apply implicit weighting to dimension reduction by means of robust principal component analysis. Highly robust methods are exploited in tasks of face localization and face detection in a database of 2D images. In this context we investigate a method for outlier detection and a filter for image denoising based on implicit weighting.

Appendix: Technical Details

Definition 9

(Weight function)

Let a function ψ:[0,1]→[0,1] be non-increasing and continuous on [0,1], let ψ(0)=1 and ψ(1)=0. Moreover, we assume that both one-sided derivatives of ψ exist in all points of (0,1), that they are bounded by a common constant and we assume the existence of a finite left derivative in 0 and finite right derivative in point 1. Then the function ψ is called a weight function.

Definition 10

(Least weighted squares with adaptive weights)

In the model (1), let b ₀ denote an initial robust estimator of β and let $\hat{\sigma}_{0}^{2}$ denote a corresponding initial robust estimator of σ ². Let F _χ denote the distribution function of $\chi^{2}_{1}$ distribution. The least weighted squares estimator of β with adaptive weights is defined as

$$ \arg\min_{\mathbf{b} \in\mathbb{R}^p} \sum_{i=1}^n \hat{w}_n \biggl[ G_n\bigl(u_i^2( \mathbf{b})\bigr) - \frac{1}{2n} \biggr] u_i^2( \mathbf{b}), $$

(42)

where

$$ \hat{w}_n(t) = \left \{ \begin{array}{l@{\quad}l} \hat{\sigma}_0 \frac{F_\chi^{-1}(\max\{t, c_n\})}{(G_n^0)^{-1} (\max\{t, c_n\})}, & \mbox{if } t < 1-d_n, \\ 0, & \mbox{otherwise}, \end{array} \right . $$

(43)

G _n is the empirical distribution function of $u_{i}^{2}(\mathbf{b})$, $G_{n}^{0}$ is the empirical distribution function of $u_{i}^{2}(\mathbf{b}_{0})$,

$$ c_n = \min \biggl\{ \frac{m}{n}; u^2_{(m)}( \mathbf{b}_0)>0 \biggr\} $$

(44)

is used to avoid dividing by zero,

$$ d_n = \sup_{t \geq c} \max \bigl\{ 0, \hat{\sigma}_0 F_\chi(t) - G_n^0(t) \bigr\}, $$

(45)

$c=F_{\chi}^{-1}(q)$ and q∈[0.9999,1) is a chosen constant.

Assumptions $\mathcal{A}$

We assume a sequence of non-random vectors $\{\mathbf{X}_{n}\}_{n=1}^{\infty}$ with values in ℝ^p and a sequence of independent and identically distributed random variables $\{e_{n}\}_{n=1}^{\infty}$ with values in ℝ, which form the model (1) for each n. The distribution function F(z) of the random error e ₁ is symmetric and absolutely continuous with a bounded density f(z), which is decreasing on ℝ⁺. The density is positive on (−∞,∞) and its second derivative is bounded. Moreover,

$$ \sum_{i=1}^n \Vert\mathbf{X}_i \Vert^3 = \mathcal{O}(n) \quad\mbox{\textit{and}}\quad \mathsf{E}e_1^2 \in(0, \infty). $$

(46)

Let

$$ \lim_{n\to\infty} \hat{\mathbf{Q}}_n = \mathbf{Q} $$

(47)

in probability, where Q is a regular matrix. There exists a distribution function H(x) for x∈ℝ^p such that

(48)

Let $B(\boldsymbol {\beta },\delta) = \{ \tilde{\boldsymbol {\beta }}\in\mathbb{R}^{p}; ||\tilde {\boldsymbol {\beta }}-\boldsymbol {\beta }||<\delta \}$ for an arbitrary fixed δ>0. For any compact set W with W∖B(β,δ)≠0, there exists γ _δ >0 such that

$$ \inf_{\boldsymbol {\omega }\in W\backslash B(\boldsymbol {\beta },\delta)} \frac{1}{n}\sum_{i=1}^n \bigl( \mathbf{X}_i^T(\boldsymbol {\omega }- \boldsymbol {\beta }) \bigr)^2 > \gamma_\delta. $$

(49)

Proof

(Theorem 1) Let us define

$$ s_{LWS} = \frac{1}{n-2} \bigl( \mathbf{u}^{LWS} \bigr)^T \mathbf{u}^{LWS}, $$

(50)

where $\mathbf{u}^{LWS} = (u_{1}^{LWS}, \ldots, u_{n}^{LWS})^{T}$ are residuals of the least weighted squares estimator. We can express

$$ T=\frac{b^{LWS}_1}{s_{LWS}} \sqrt{\sum_{i=1}^n (X_i-\bar{X})^2} $$

(51)

and the statement follows from Theorem 4 for the linear regression context. □

Proof

(Theorem 2) Consequence of a general result of [50], who derived (19) as a consistent estimator of σ ². The constant γ in (20) is independent on n and can be approximated by numerical integration as

$$ \sum_{i=2}^m (t_{i-1} - t_i) \psi \bigl( F(|t_i|) \bigr) t_i^2 f(t_i) $$

(52)

using a partition −∞<t ₁<t ₂<⋯<t _m <∞ of the real line into intervals. Normal distribution N(0,σ ²) of errors is assumed. Without loss of generality, we use the N(0,1) distribution for the numerical integration for the examples of weights in (7) and (8). □

Proof

(Theorem 3) The LWS estimator (5) in the model (27) is defined as the minimum of $\sum_{i=1}^{n} w_{i} u^{2}_{(i)}(\hat{\mu})$ over $\hat{\mu} \in{\mathbb{R}}$ and over all permutations of the weights with magnitudes w ₁,…,w _n . The location model (27) is a special case of linear regression and therefore the solution has a form of a weighted mean $\hat{\mu} = \sum_{i=1}^{n} w^{*}_{i} Y_{i}$, where $w_{1}^{*},\ldots,w_{n}^{*}$ are permuted values of w ₁,…,w _n . Therefore, we can express the LWS estimator as

$$ \arg\min\sum_{i=1}^n w_i \Biggl(\mathbf{Y}-\sum_{j=1}^n w_j Y_j \Biggr)^2_{(i)}, $$

(53)

where the minimum is considered over all permutations of the weights with magnitudes w ₁,…,w _n and the notation

$$ \Biggl(\mathbf{Y}-\sum_{j=1}^n w_j Y_j \Biggr)_{(1)} \leq\cdots\leq \Biggl( \mathbf{Y}-\sum_{j=1}^n w_j Y_j \Biggr)_{(n)} $$

(54)

is used for ordered coordinates of

$$ \Biggl(Y_1 - \sum_{j=1}^n w_j Y_j, \ldots, Y_n - \sum _{j=1}^n w_j Y_j \Biggr)^T. $$

(55)

However, (53) minimizes the weighted variance $S^{2}_{w}(\mathbf{Y})$ (28), which concludes the proof. □

Proof

(Theorem 4) The first part follows immediately from the asymptotic normality of b ^LWS of [37]. The independence between the numerator and the denominator is assured asymptotically in probability, because b ^LWS is asymptotically in probability equivalent with (X ^T X)⁻¹ X ^T Y and (u ^LWS)^T u ^LWS is asymptotically equivalent with e ^T Me in probability [49], where $\mathbf{M} = \mbox{\boldmath$\mathcal{I}$}_{n} - \mathbf{H}$, H=X(X ^T X)⁻¹ X ^T and $\mbox{\boldmath$\mathcal{I}$}_{n}$ denotes an identity matrix of dimension n.

The second part follows from the asymptotic representation for the LWS estimator [49]. The asymptotic representation holds in the form

(56)

for n→∞, where

$$ \hat{\mathbf{Q}}_n^{(1)}= \hat{\mathbf{Q}}_n \biggl(1-\int_0^1 \alpha d\psi(\alpha) - 2\int _0^1 u_\alpha f(u_\alpha)d\psi( \alpha) \biggr), $$

(57)

ψ is the weight function defining the LWS estimator and coordinates of η=(η ₁,…,η _p )^T are of order o _P (1). Let us denote $\boldsymbol {\tau }= - \frac{1}{\sqrt{n}}\mathbf{X}\boldsymbol {\eta }$, κ=u ^LWS−τ and Ψ(e)=(ψ ₁(e),…,ψ _n (e))^T with

(58)

and

$$ \boldsymbol {\phi }= \mathbf{H}\mathbf{e} - \mathbf{X}\bigl(\mathbf{X}^T\mathbf{X} \bigr)^{-1} \mathbf{X}^T\boldsymbol {\varPsi }(\mathbf{e})= \mathbf{H}\bigl[ \mathbf{e}- \boldsymbol {\varPsi }(\mathbf{e})\bigr]. $$

(59)

It holds κ=Me+ϕ and

$$ \boldsymbol {\kappa }^T\boldsymbol {\kappa }= \mathbf{e}^T \mathbf{M} \mathbf{e} + [ \mathbf{e}- \boldsymbol {\varPsi }\mathbf{e} ]^T \mathbf{H}\bigl[\mathbf{e}- \boldsymbol {\varPsi }( \mathbf{e})\bigr]. $$

(60)

To complete the proof of the second part we can repeat the steps of [26], who considered the test statistic of the Durbin-Watson test. The residual sum of squares (u ^LWS)^T u ^LWS is asymptotically equivalent in probability with κ ^T κ and e ^T Me, which is the test statistic computed with least squares residuals. At the same time, e ^T Me/σ ² follows $\chi^{2}_{n-p}$ distribution. The third part of the theorem is proven by analogous reasoning. □

Proof

(Corollary 1) Starting with (34), we exploit the scale-invariance of the denominator. $\sigma^{2}_{\psi}$ is estimated consistently following [37] to obtain (39). □