Outlier-robust parameter estimation for unnormalized statistical models

Abstract

Unnormalized statistical models are ubiquitous in modern statistical data analysis. Recent methods take a classification approach to estimate unnormalized models. However, the classification problem is often solved based on the maximum-likelihood estimation, which can be seriously hampered by the contamination of outliers. In this paper, we propose two outlier-robust methods for estimation of unnormalized statistical models. The proposed methods are developed by combining robust divergences with the classification approach, and their robustness is theoretically investigated based on influence function. Interestingly, our theoretical analysis reveals a counter-intuitive robustness of the proposed methods, and shows the importance of not only employing robust divergences but also taking the classification approach for outlier-robust estimation. Finally, we experimentally demonstrate that the proposed methods are robust against outliers.

Appendices

Appendix A: Limits of $D_{\gamma }$ and $D_{\beta }$

As in Fujisawa and Eguchi (2008), to show the respective limits of $D_{\gamma }$ and $D_{\beta }$, we employ the following Taylor expansions: For $x>0$ and $\gamma >0$

$$\begin{aligned} x^{\gamma }&=1+\gamma \log {x}+O(\gamma ^2) \end{aligned}$$

(A1)

$$\begin{aligned} \log (1+x)&=x+O(x^2). \end{aligned}$$

(A2)

First, in the limit of $\gamma \rightarrow 0$, we show that $D_{\gamma }(\varvec{\alpha })$ for GNCE equals to $D_{\textrm{lr}}(\varvec{\alpha })$ for NCE. Applying (A1) to $h_{\gamma }(\varvec{X};\varvec{\alpha },\nu )^{\frac{\gamma }{\gamma +1}}$ yields

$$\begin{aligned} h_{\gamma }(\varvec{X};\varvec{\alpha },\nu )^{\frac{\gamma }{\gamma +1}}&= 1+\frac{\gamma }{\gamma +1}\log h_{\gamma }(\varvec{X};\varvec{\alpha },\nu ) +O\left( \left( \frac{\gamma }{\gamma +1}\right) ^2\right) . \end{aligned}$$

(A3)

Equation (A3) enables us to express $D_{\gamma }(\varvec{\alpha })$ as

$$\begin{aligned} D_{\gamma }(\varvec{\alpha })&=-\frac{1}{\gamma }\log \left[ 1+\frac{p_0\gamma }{\gamma +1}E_{\textrm{d}}[\log h_{\gamma }(\varvec{X};\varvec{\alpha },\nu )]\right. \\&\quad \left. +\frac{p_1\gamma }{\gamma +1}E_{\textrm{r}}[\log \left( 1-h_{\gamma }(\varvec{Y};\varvec{\alpha },\nu )\right) ] +O\left( \left( \frac{\gamma }{\gamma +1}\right) ^2\right) \right] ,\\&= -\frac{p_0}{\gamma +1}\left( E_{\textrm{d}}[\log h_{\gamma }(\varvec{X};\varvec{\alpha },\nu )] +\nu E_{\textrm{r}}[\log \left( 1-h_{\gamma }(\varvec{Y};\varvec{\alpha },\nu )\right) ]\right) \\&\quad +O\left( \frac{\gamma }{(\gamma +1)^2}\right) , \end{aligned}$$

where (A2) was applied on the last line. Since $\lim _{\gamma \rightarrow {0}}h_{\gamma }(\varvec{u};\varvec{\alpha },\nu )=f_0(\varvec{u};\varvec{\alpha },\nu )$, we have

$$\begin{aligned} \lim _{\gamma \rightarrow {0}}D_{\gamma }(\varvec{\alpha })&=D_{\textrm{lr}}(\varvec{\alpha }). \end{aligned}$$

Next, we investigate the relationship between $D_{\beta }(\varvec{\alpha })$ and $D_{\textrm{lr}}(\varvec{\alpha })$ in the limit of $\beta \rightarrow 0$. Equation (A1) gives the following expression of $D_{\beta }(\varvec{\alpha })$:

$$\begin{aligned} D_{\beta }(\varvec{\alpha })&=-p_0\left( 1+\nu +E_{\textrm{d}}[\log {f}_0(\varvec{X};\varvec{\alpha },\nu )] +\nu E_{\textrm{r}}[\log {f}_1(\varvec{Y};\varvec{\alpha },\nu )]+O(\beta )\right) \nonumber \\&\quad +\frac{p_0}{\beta +1}\left( E_{\textrm{d}}\left[ \sum _{C=0}^1f_C(\varvec{X};\varvec{\alpha },\nu )^{\beta +1}\right] +\nu E_{\textrm{r}}\left[ \sum _{C=0}^1f_C(\varvec{Y};\varvec{\alpha },\nu )^{\beta +1}\right] \right) . \end{aligned}$$

Since $\lim _{\beta \rightarrow 0}\sum _{C=0}^1f_C(\varvec{u};\varvec{\alpha },\nu )^{\beta +1}=1$ for all $\varvec{u}$,

$$\begin{aligned} \lim _{\beta \rightarrow 0}D_{\beta }(\varvec{\alpha })&=D_{\textrm{lr}}(\varvec{\alpha }). \end{aligned}$$

Appendix B: Consistency of GNCE and PNCE

This appendix first proves the consistency of GNCE. To this end, we recall

$$\begin{aligned} \varvec{\alpha }_{\star }:=\mathop {\textrm{argmin}}\limits _{\varvec{\alpha }}D_{\gamma }(\varvec{\alpha })\quad \text {and}\quad \widehat{\varvec{\alpha }}:=\mathop {\textrm{argmin}}\limits _{\varvec{\alpha }}\widehat{D}_{\gamma }(\varvec{\alpha }),\qquad , \end{aligned}$$

and decompose them into $\varvec{\alpha }_{\star }=(\varvec{\theta }_{\star },c_{\star })$ and $\widehat{\varvec{\alpha }}=(\widehat{\varvec{\theta }},\widehat{c})$, respectively. The notion of identifiability (Van der Vaart, 1998) is defined as follows: A statistical model $p_{\textrm{m}}(\cdot ;\varvec{\theta })$ is said to be identifiable if $p_{\textrm{m}}(\cdot ;\varvec{\theta })=p_{\textrm{m}}(\cdot ;\varvec{\theta }')$ implies $\varvec{\theta }=\varvec{\theta }'$.

Intuitively, when $p_{\textrm{d}}(\cdot )=p_{\textrm{m}}(\cdot ;\varvec{\theta }_0)$ with some constant vector $\varvec{\theta }_0$, the definition of $\varvec{\alpha }_{\star }$ ensures $p^0_{\textrm{m}}(\cdot ;\varvec{\alpha }_{\star })=p_{\textrm{m}}(\cdot ;\varvec{\theta }_0)$ and the identifiability of $p_{\textrm{m}}(\cdot ;\varvec{\theta })$ implies $\varvec{\theta }_{\star }=\varvec{\theta }_0$. Thus, $\widehat{\varvec{\theta }}$ should converge to $\varvec{\theta }_0$ in probability as $n\rightarrow \infty$ and $n_{\textrm{y}}\rightarrow \infty$. To formally state the consistency of GNCE, we establish the following theorem:

Theorem 8

Suppose that the following assumptions hold:

• (A0) $\textrm{supp}(p_{\textrm{m}}(\cdot ;\varvec{\theta }))$ is included in $\textrm{supp}(p_{\textrm{r}})$ for all $\varvec{\theta }$.

• (A1) $p_{\textrm{d}}(\cdot )$ belongs to the same parametric family as $p_{\textrm{m}}(\cdot ;\varvec{\theta })$, i.e., there exists a constant vector $\varvec{\theta }_0$, such that $p_{\textrm{d}}(\cdot )=p_{\textrm{m}}(\cdot ;\varvec{\theta }_0)$.

• (A2) $p_{\textrm{m}}(\cdot ;\varvec{\theta })$ is identifiable.

• (A3) $\inf _{\varvec{\alpha }:\Vert \varvec{\alpha }-\varvec{\alpha }_{\star }\Vert \ge \epsilon }D_{\gamma }(\varvec{\alpha })>D_{\gamma }(\varvec{\alpha }_{\star })$ for all $\epsilon >0$.

• (A4) $\sup _{\varvec{\alpha }}|D_{\gamma }(\varvec{\alpha })-\widehat{D}_{\gamma }(\varvec{\alpha })|$ converges to zero in probability.

• (A5) $\widehat{\nu }=n_{\textrm{y}}/n\rightarrow {\nu }$ as $n\rightarrow \infty$.

Then, $\widehat{\varvec{\theta }}$ converges to $\varvec{\theta }_0$ in probability.

The proof of Theorem 8 is given in Appendix B.1. Since $\textrm{supp}(p_{\textrm{m}}(\cdot ;\varvec{\theta }))=\textrm{supp}(p^0_{\textrm{m}}(\cdot ;\varvec{\alpha }))$ with $\varvec{\alpha }=(\varvec{\theta },c)$ by definition, Assumption (A0) is required to properly estimate $p^0_{\textrm{m}}(\cdot ,\varvec{\alpha })$ on its support as discussed in Sect. 3.1 or assumed in the previous works related to NCE (Gutmann & Hyvärinen, 2012; Matsuda et al., 2021). Assumption (A3) means that $\varvec{\alpha }_{\star }$ is a well-separated point of minimum of $D_{\gamma }(\varvec{\alpha })$ (Van der Vaart, 1998). Assumption (A4) is well known as the uniform law of large numbers. Assumption (A5) implies $n_{\textrm{y}}\rightarrow \infty$ as $n\rightarrow \infty$.

For PNCE, consistency can be established by replacing Assumptions (A3–A4) with the following ones, while the other assumptions are still maintained:

• (B3) $\inf _{\varvec{\alpha }:\Vert \varvec{\alpha }-\varvec{\alpha }_{\star }^{\beta }\Vert \ge \epsilon }D_{\beta }(\varvec{\alpha })>D_{\beta }(\varvec{\alpha }_{\star }^{\beta })$ for all $\epsilon >0$.

• (B4) $\sup _{\varvec{\alpha }}|D_{\beta }(\varvec{\alpha })-\widehat{D}_{\beta }(\varvec{\alpha })|$ converges to zero in probability.

Since it is fundamentally the same as GNCE, we omit the proof for consistency of PNCE.

1.1 B.1: Proof of Theorem 8

Proof

Our proof essentially follows existing proofs (Van der Vaart, 1998; Wasserman, 2004). We first state the following lemma whose proof is given in Appendix B.2:

Lemma 9

Suppose that Assumptions (A0–A2) hold. Then, $\varvec{\theta }_{\star }=\varvec{\theta }_0$.

By Lemma 9, it suffices to prove that $\widehat{\varvec{\alpha }}$ converges to $\varvec{\alpha }_{\star }$ in probability. Assumption (A3) implies that for all $\epsilon >0$, there exists some $\eta >0$, such that $D_{\gamma }(\varvec{\alpha })>D_{\gamma }(\varvec{\alpha }_{\star })+\eta$ for every $\varvec{\alpha }$ with $\Vert \varvec{\alpha }-\varvec{\alpha }_{\star }\Vert \ge \epsilon$. Thus, the event $\{\Vert \widehat{\varvec{\alpha }}-\varvec{\alpha }_{\star }\Vert \ge \epsilon \}$ is included in the event $\{D_{\gamma }(\widehat{\varvec{\alpha }})>D_{\gamma }(\varvec{\alpha }_{\star })+\eta \}$, and the probability of these events can be expressed as

$$\begin{aligned} P(\Vert \widehat{\varvec{\alpha }}-\varvec{\alpha }_{\star }\Vert \ge \epsilon )\le {P}(D_{\gamma }(\widehat{\varvec{\alpha }})-D_{\gamma }(\varvec{\alpha }_{\star })>\eta ). \end{aligned}$$

The inequality above indicates that the convergence of $\widehat{\varvec{\alpha }}$ to $\varvec{\alpha }_{\star }$ in probability is confirmed by showing that $P(D_{\gamma }(\widehat{\varvec{\alpha }})-D_{\gamma }(\varvec{\alpha }_{\star })>\eta )$ converges to zero. To this end, we derive the following upper bound:

$$\begin{aligned} D_{\gamma }(\widehat{\varvec{\alpha }})-D_{\gamma }(\varvec{\alpha }_{\star })&=D_{\gamma }(\widehat{\varvec{\alpha }})-\widehat{D}_{\gamma }(\varvec{\alpha }_{\star })+\widehat{D}_{\gamma }(\varvec{\alpha }_{\star })-D_{\gamma }(\varvec{\alpha }_{\star })\\&\le D_{\gamma }(\widehat{\varvec{\alpha }})-\widehat{D}_{\gamma }(\widehat{\varvec{\alpha }})+\widehat{D}_{\gamma }(\varvec{\alpha }_{\star })-D_{\gamma }(\varvec{\alpha }_{\star })\\&\le 2\sup _{\varvec{\alpha }}|D_{\gamma }(\varvec{\alpha })-\widehat{D}_{\gamma }(\varvec{\alpha })|. \end{aligned}$$

Assumption (A4) ensures that $D_{\gamma }(\widehat{\varvec{\alpha }})-D_{\gamma }(\varvec{\alpha }_{\star })$ converges to zero in probability, indicating that $P(D_{\gamma }(\widehat{\varvec{\alpha }})-D_{\gamma }(\varvec{\alpha }_{\star })>\eta )$ also converges to zero. The proof is completed. $\square$

1.2 B.2: Proof of Lemma 9

Proof

We first define the $\gamma$-divergence $d(\varvec{\alpha })$ (not cross entropy) as

$$\begin{aligned} d(\varvec{\alpha })&:= \frac{1}{\gamma }\log \int \left\{ \sum _{C=0}^1 p(C|\varvec{u})^{\gamma +1}\right\} ^{\frac{1}{\gamma +1}} \hspace{-4mm}p(\varvec{u})\textrm{d}\varvec{u}\\&\quad -\frac{1}{\gamma }\log \int \left\{ \frac{\sum _{C=0}^1 p(C|\varvec{u})f_C(\varvec{u};\varvec{\alpha },\nu )^{\gamma }}{\left( \sum _{C=0}^1f_C(\varvec{u};\varvec{\alpha },\nu )^{\gamma +1}\right) ^{\frac{\gamma }{\gamma +1}}} \right\} p(\varvec{u})\textrm{d}\varvec{u}, \end{aligned}$$

where we note that the second term in the right-hand side above is equal to the $\gamma$-cross entropy $D_{\gamma }(\varvec{\alpha })$ used in this paper. Non-negativity of the $\gamma$-divergence (i.e., $d(\varvec{\alpha })\ge 0$) is guaranteed by the following Hölder inequality (Fujisawa & Eguchi, 2008): For all $\varvec{u}\in \textrm{supp}(p_{\textrm{r}})$

$$\begin{aligned} \left\{ \sum _{C=0}^1 p(C|\varvec{u})^{\gamma +1}\right\} ^{\frac{1}{\gamma +1}} \left\{ \sum _{C=0}^1 f_C(\varvec{u};\varvec{\alpha },\nu )^{\gamma +1}\right\} ^{\frac{\gamma }{\gamma +1}} \ge \sum _{C=0}^1 p(C|\varvec{u})f_C(\varvec{u};\varvec{\alpha },\nu )^{\gamma }. \end{aligned}$$

(B4)

Equality in (B4), i.e., $d(\varvec{\alpha })=0$ holds if and only if $p(C|\varvec{u})$ and $f_C(\varvec{u};\varvec{\alpha },\nu )$ are linearly dependent. In addition, by definition of $\varvec{\alpha }_{\star }$, $d(\varvec{\alpha }_{\star })=0$. Thus, with a positive function $\omega (\varvec{u})>0$, we obtain

$$\begin{aligned} p(C|\varvec{u})=\omega (\varvec{u})f_C(\varvec{u};\varvec{\alpha }_{\star },\nu ). \end{aligned}$$

(B5)

By substituting (5) and (7) into (B5) with $p_{\textrm{d}}(\cdot )=p_{\textrm{m}}(\cdot ;\varvec{\theta }_0)$, we have

$$\begin{aligned} \frac{p_{\textrm{m}}(\varvec{u};\varvec{\theta }_0)}{p_{\textrm{m}}(\varvec{u};\varvec{\theta }_0)+\nu p_{\textrm{r}}(\varvec{u})}= \omega (\varvec{u})\frac{p^0_{\textrm{m}}(\varvec{u};\varvec{\alpha }_{\star })}{p^0_{\textrm{m}}(\varvec{u};\varvec{\alpha }_{\star })+\nu p_{\textrm{r}}(\varvec{u})}, \end{aligned}$$

which yields

$$\begin{aligned} (1-\omega (\varvec{u}))p_{\textrm{m}}(\varvec{u};\varvec{\theta }_0)p^0_{\textrm{m}}(\varvec{u};\varvec{\alpha }_{\star }) +\nu \{p_{\textrm{m}}(\varvec{u};\varvec{\theta }_0)-\omega (\varvec{u})p^0_{\textrm{m}}(\varvec{u};\varvec{\alpha }_{\star })\}p_{\textrm{r}}(\varvec{u})=0. \end{aligned}$$

(B6)

It follows from (B6) that the supports of $p_{\textrm{m}}(\varvec{u};\varvec{\theta }_0)$ and $p^0_{\textrm{m}}(\varvec{u};\varvec{\alpha }_{\star })$ coincide; otherwise, there exists a point $\tilde{\varvec{u}}$, such that $p_{\textrm{m}}(\tilde{\varvec{u}};\varvec{\theta }_0)=0$ yet $p^0_{\textrm{m}}(\tilde{\varvec{u}};\varvec{\alpha }_{\star }),p_{\textrm{r}}(\tilde{\varvec{u}}), \omega (\tilde{\varvec{u}})>0$, and (B6) does not hold at $\varvec{u}=\tilde{\varvec{u}}$. Since (B6) holds for all $\varvec{u}\in \textrm{supp}(p_{\textrm{r}})$, we obtain $\omega (\varvec{u})=1$ and

$$\begin{aligned} p_{\textrm{m}}(\varvec{u};\varvec{\theta }_0)=p^0_{\textrm{m}}(\varvec{u};\varvec{\alpha }_{\star }). \end{aligned}$$

(B7)

Integrating both sides on (B7) indicates that $p^0_{\textrm{m}}(\varvec{u};\varvec{\alpha }_{\star })$ is normalized, i.e., $p^0_{\textrm{m}}(\varvec{u};\varvec{\alpha }_{\star })=p_{\textrm{m}}(\varvec{u};\varvec{\theta }_{\star })$. Finally, the identifiability of $p_{\textrm{m}}$ ensures $\varvec{\theta }_0=\varvec{\theta }_{\star }$. $\square$

Appendix C: Proof of Proposition 1

Proof

Let us express

$$\begin{aligned} L_{\gamma }(\varvec{\alpha })&=\exp \left( -\gamma D_{\gamma }(\varvec{\alpha })+\log {p}_0\right) \\&=E_{\textrm{d}}\Big [h_{\gamma }(\varvec{X};\varvec{\alpha },\nu )^{\frac{\gamma }{\gamma +1}}\Big ] +\nu E_{\textrm{r}}\Big [\left( 1-h_{\gamma }(\varvec{Y};\varvec{\alpha },\nu )\right) ^{\frac{\gamma }{\gamma +1}}\Big ]. \end{aligned}$$

Since $\nabla _{\varvec{\alpha }}D_{\gamma }(\varvec{\alpha })\bigr |_{\varvec{\alpha }=\varvec{\alpha }_{\star }}=\varvec{0}$ from the definition of $\varvec{\alpha }_{\star }$, we have

$$\begin{aligned} \nabla _{\varvec{\alpha }}L_{\gamma }(\varvec{\alpha })\Bigr |_{\varvec{\alpha }=\varvec{\alpha }_{\star }}&=\varvec{0}, \end{aligned}$$

(C8)

which implies that

$$\begin{aligned} \left\{ E_{\textrm{d}}\Big [\nabla _{\varvec{\alpha }}h_{\gamma }(\varvec{X};\varvec{\alpha },\nu )^{\frac{\gamma }{\gamma +1}}\Big ] +\nu E_{\textrm{r}}\Big [\nabla _{\varvec{\alpha }}(1-h_{\gamma }(\varvec{Y};\varvec{\alpha },\nu ))^{\frac{\gamma }{\gamma +1}}\Big ] \right\} \Bigr |_{\varvec{\alpha }=\varvec{\alpha }_{\star }}=\varvec{0}. \end{aligned}$$

(C9)

Similarly, we define the contaminated version of $L_{\gamma }(\varvec{\alpha })$ as

$$\begin{aligned} \bar{L}_{\gamma }(\varvec{\alpha })&:=\exp \left( -\gamma \bar{D}_{\gamma }(\varvec{\alpha })+\log {p_0}\right) \nonumber \\ {}&= \bar{E}_{\textrm{d}}\Big [h_{\gamma }(\varvec{X};\varvec{\alpha },\nu )^{\frac{\gamma }{\gamma +1}}\Big ]+\nu \bar{E}_{\textrm{r}}\Big [\left( 1-h_{\gamma }(\varvec{Y};\varvec{\alpha },\nu )\right) ^{\frac{\gamma }{\gamma +1}}\Big ]. \end{aligned}$$

Then, by the definition of $\varvec{\alpha }_{\epsilon }$

$$\begin{aligned} \nabla _{\varvec{\alpha }}\bar{L}_{\gamma }(\varvec{\alpha })\Bigr |_{\varvec{\alpha }=\varvec{\alpha }_{\epsilon }}&=\varvec{0}. \end{aligned}$$

(C10)

Here, we recall that influence function in (20) is the (Gâteaux) derivative of $\varvec{\alpha }_{\epsilon }$ with respect to $\epsilon$ at $\epsilon =0$. Thus, since the gradient of $\bar{L}_{\gamma }(\varvec{\alpha })$ is computed as

$$\begin{aligned} \nabla _{\varvec{\alpha }}\bar{L}_{\gamma }(\varvec{\alpha })&=\nabla _{\varvec{\alpha }}L_{\gamma }(\varvec{\alpha }) +\epsilon \left\{ \nabla _{\varvec{\alpha }}h_{\gamma }(\varvec{x}_{\textrm{o}};\varvec{\alpha },\nu )^{\frac{\gamma }{\gamma +1}} -E_{\textrm{d}}\Big [\nabla _{\varvec{\alpha }}h_{\gamma }(\varvec{X};\varvec{\alpha },\nu )^{\frac{\gamma }{\gamma +1}}\Big ] \right. \\&\quad \left. +\nu \eta \nabla _{\varvec{\alpha }}(1-h_{\gamma }(\varvec{y}_{\textrm{o}};\varvec{\alpha },\nu ))^{\frac{\gamma }{\gamma +1}} -\nu \eta E_{\textrm{r}}\Big [\nabla _{\varvec{\alpha }}(1-h_{\gamma }(\varvec{Y};\varvec{\alpha },\nu ))^{\frac{\gamma }{\gamma +1}}\Big ] \right\} , \end{aligned}$$

we obtain the following equation by differentiating (C10) respect to $\epsilon$ at $\epsilon =0$:

$$\begin{aligned}&\varvec{H}\cdot \textrm{IF}(\varvec{x}_{\textrm{o}},\varvec{y}_{\textrm{o}}) +\left\{ (\eta -1)E_{\textrm{d}}\Big [\nabla _{\varvec{\alpha }}h_{\gamma }(\varvec{X};\varvec{\alpha },\nu )^{\frac{\gamma }{\gamma +1}}\Big ] \right. \nonumber \\&\quad \left. +\nabla _{\varvec{\alpha }}h_{\gamma }(\varvec{x}_{\textrm{o}};\varvec{\alpha },\nu )^{\frac{\gamma }{\gamma +1}} +\nu \eta \nabla _{\varvec{\alpha }}(1-h_{\gamma }(\varvec{y}_{\textrm{o}};\varvec{\alpha },\nu ))^{\frac{\gamma }{\gamma +1}} \right\} \Bigr |_{\varvec{\alpha }=\varvec{\alpha }_{\star }}=\varvec{0}, \end{aligned}$$

(C11)

where $\varvec{\alpha }_{\epsilon }\bigr |_{\epsilon =0}=\varvec{\alpha }_{\star }$, $\varvec{H}:=\nabla ^2_{\varvec{\alpha }}L_{\gamma }(\varvec{\alpha })\Bigr |_{\varvec{\alpha }=\varvec{\alpha }_{\star }}$, and we derived the following equation from (C9) and applied it:

$$\begin{aligned}&\left\{ E_{\textrm{d}}\Big [\nabla _{\varvec{\alpha }}h_{\gamma }(\varvec{X};\varvec{\alpha },\nu )^{\frac{\gamma }{\gamma +1}}\Big ] +\nu \eta E_{\textrm{r}}\Big [\nabla _{\varvec{\alpha }}(1-h_{\gamma }(\varvec{Y};\varvec{\alpha },\nu ))^{\frac{\gamma }{\gamma +1}}\Big ] \right\} \Bigr |_{\varvec{\alpha }=\varvec{\alpha }_{\star }}\\&\quad =(1-\eta )E_{\textrm{d}}\Big [\nabla _{\varvec{\alpha }}h_{\gamma }(\varvec{X};\varvec{\alpha },\nu )^{\frac{\gamma }{\gamma +1}}\Big ]\Bigr |_{\varvec{\alpha }=\varvec{\alpha }_{\star }}. \end{aligned}$$

Finally, by denoting $\nabla _{\varvec{\alpha }}\log {p^0_{\textrm{m}}}(\varvec{x};\varvec{\alpha })$ by $\varvec{g}_{\textrm{m}}(\varvec{x};\varvec{\alpha })$, it completes the proof to substitute

$$\begin{aligned} \nabla _{\varvec{\alpha }}h_{\gamma }(\varvec{u};\varvec{\alpha },\nu )^{\frac{\gamma }{\gamma +1}}&=\gamma h_{\gamma }(\varvec{u};\varvec{\alpha },\nu )^{\frac{\gamma }{\gamma +1}}(1-h_{\gamma }(\varvec{u};\varvec{\alpha },\nu ))\varvec{g}_{\textrm{m}}(\varvec{u};\varvec{\alpha })\\ \nabla _{\varvec{\alpha }}(1-h_{\gamma }(\varvec{u};\varvec{\alpha },\nu ))^{\frac{\gamma }{\gamma +1}}&=-\gamma h_{\gamma }(\varvec{u};\varvec{\alpha },\nu )(1-h_{\gamma }(\varvec{u};\varvec{\alpha },\nu ))^{\frac{\gamma }{\gamma +1}} \varvec{g}_{\textrm{m}}(\varvec{u};\varvec{\alpha }), \end{aligned}$$

into (C11) and to use the relation

$$\begin{aligned} \varvec{H}&=\left\{ \gamma ^2L_{\gamma }(\varvec{\alpha })(\nabla _{\varvec{\alpha }}D_{\gamma }(\varvec{\alpha }))(\nabla _{\varvec{\alpha }}D_{\gamma }(\varvec{\alpha }))^{\top } -\gamma L_{\gamma }(\varvec{\alpha })\nabla ^2_{\varvec{\alpha }}D_{\gamma }(\varvec{\alpha })\right\} \Bigr |_{\varvec{\alpha }=\varvec{\alpha }_{\star }} \nonumber \\&=-\gamma L_{\gamma }(\varvec{\alpha })\nabla ^2_{\varvec{\alpha }}D_{\gamma }(\varvec{\alpha })\Bigr |_{\varvec{\alpha }=\varvec{\alpha }_{\star }} =\gamma \tilde{\varvec{H}}, \end{aligned}$$

(C12)

where $\nabla _{\varvec{\alpha }}D_{\gamma }(\varvec{\alpha })\Bigr |_{\varvec{\alpha }=\varvec{\alpha }_{\star }}=\varvec{0}$. $\square$

Appendix D: Proof of Proposition 4

Here, we give a brief proof of Proposition 4, because it essentially follows the same process as Proposition 1.

Proof

We first compute the gradient of $\bar{D}_{\beta }(\varvec{\alpha })/p_0$ as

$$\begin{aligned} \frac{\nabla _{\varvec{\alpha }}\bar{D}_{\beta }(\varvec{\alpha })}{p_0}&=\frac{\nabla _{\varvec{\alpha }}D_{\beta }(\varvec{\alpha })}{p_0} +\epsilon \left\{ \frac{1}{\beta }\left( E_{\textrm{d}}\big [\nabla _{\varvec{\alpha }}f_0(\varvec{X};\varvec{\alpha },\nu )^{\beta }\big ]-\nabla _{\varvec{\alpha }}f_0(\varvec{x}_{\textrm{o}};\varvec{\alpha },\nu )^{\beta } \right. \right. \\&\quad \left. +\nu \eta E_{\textrm{r}}\big [\nabla _{\varvec{\alpha }}f_1(\varvec{Y};\varvec{\alpha },\nu )^{\beta }\big ] -\nu \eta \nabla _{\varvec{\alpha }}f_1(\varvec{y}_{\textrm{o}};\varvec{\alpha },\nu )^{\beta }\right) \\&\quad +\frac{1}{\beta +1}\sum _{C=0}^1\left( \nabla _{\varvec{\alpha }}f_C(\varvec{x}_{\textrm{o}};\varvec{\alpha },\nu )^{\beta +1}-E_{\textrm{d}}\big [\nabla _{\varvec{\alpha }}f_C(\varvec{X};\varvec{\alpha },\nu )^{\beta +1}\big ] \right. \\&\quad \left. \left. +\nu \eta \nabla _{\varvec{\alpha }}f_C(\varvec{y}_{\textrm{o}};\varvec{\alpha },\nu )^{\beta +1} -\nu \eta E_{\textrm{r}}\big [\nabla _{\varvec{\alpha }}f_C(\varvec{Y};\varvec{\alpha },\nu )^{\beta +1}\big ] \right) \right\} . \end{aligned}$$

Then, we differentiate both sides of $\nabla _{\varvec{\alpha }}\bar{D}_{\beta }(\varvec{\alpha })/p_0|_{\varvec{\alpha }=\varvec{\alpha }_{\epsilon }}=\varvec{0}$ with respect to $\epsilon$ at $\epsilon =0$, and have

$$\begin{aligned}&\varvec{H}_{\beta }\cdot \textrm{IF}(\varvec{x}_{\textrm{o}},\varvec{y}_{\textrm{o}}) +\Biggl \{(1-\eta )\left( \frac{1}{\beta }E_{\textrm{d}}[\nabla _{\varvec{\alpha }}f_0(\varvec{X};\varvec{\alpha },\nu )^{\beta }]\right. \nonumber \\&\quad \left. -\frac{1}{\beta +1}E_{\textrm{d}}\left[ \sum _{C=0}^1\nabla _{\varvec{\alpha }}f_C(\varvec{X};\varvec{\alpha },\nu )^{\beta +1} \right] \right) \nonumber \\&\quad -\frac{1}{\beta }\left( \nabla _{\varvec{\alpha }}f_0(\varvec{x}_{\textrm{o}};\varvec{\alpha },\nu )^{\beta } +\nu \eta \nabla _{\varvec{\alpha }}f_1(\varvec{y}_{\textrm{o}};\varvec{\alpha },\nu )^{\beta }\right) \nonumber \\&\quad +\frac{1}{\beta +1}\sum _{C=0}^1\left( \nabla _{\varvec{\alpha }}f_C(\varvec{x}_{\textrm{o}};\varvec{\alpha },\nu )^{\beta +1} +\nu \eta \nabla _{\varvec{\alpha }}f_C(\varvec{y}_{\textrm{o}};\varvec{\alpha },\nu )^{\beta +1}\right) \Biggr \} \Biggr |_{\varvec{\alpha }=\varvec{\alpha }_{\star }^{\beta }}=\varvec{0}, \end{aligned}$$

(D13)

where we used the following relation derived from $\nabla _{\varvec{\alpha }}D_{\beta }(\varvec{\alpha })|_{\varvec{\alpha }=\varvec{\alpha }_{\star }^{\beta }}=\varvec{0}$:

$$\begin{aligned}&-\frac{1}{\beta }\left( E_{\textrm{d}}[\nabla _{\varvec{\alpha }}f_0(\varvec{X};\varvec{\alpha },\nu )^{\beta }] +\nu E_{\textrm{r}}\big [\nabla _{\varvec{\alpha }}f_1(\varvec{Y};\varvec{\alpha },\nu )^{\beta }\big ]\right) \\&\quad +\frac{1}{\beta +1}\sum _{C=0}^1\left( E_{\textrm{d}}\big [\nabla _{\varvec{\alpha }}f_C(\varvec{X};\varvec{\alpha },\nu )^{\beta +1}\big ] +\nu E_{\textrm{r}}\big [\nabla _{\varvec{\alpha }}f_C(\varvec{Y};\varvec{\alpha },\nu )^{\beta +1}\big ]\right) =\varvec{0}. \end{aligned}$$

We complete the proof by substituting the formulas

$$\begin{aligned} \nabla _{\varvec{\alpha }}f_0(\varvec{X};\varvec{\alpha },\nu )^{\beta }&=\beta {f}_0(\varvec{X};\varvec{\alpha },\nu )^{\beta } f_1(\varvec{X};\varvec{\alpha },\nu )\varvec{g}_{\textrm{m}}(\varvec{X};\varvec{\alpha })\\ \nabla _{\varvec{\alpha }}f_1(\varvec{X};\varvec{\alpha },\nu )^{\beta }&=-\beta {f}_0(\varvec{X};\varvec{\alpha },\nu )f_1(\varvec{X};\varvec{\alpha },\nu )^{\beta }\varvec{g}_{\textrm{m}}(\varvec{X};\varvec{\alpha }), \end{aligned}$$

into (D13). $\square$

Appendix E: Proof of Proposition 7

Since the proof is similar as Proposition 1, we give a short proof of Proposition 7.

Proof

First, the gradient of $\bar{J}_{\gamma }(\varvec{\alpha })$ is computed as

$$\begin{aligned} \nabla _{\varvec{\alpha }}\bar{J}_{\gamma }(\varvec{\alpha })&=\frac{\bar{E}_{\textrm{r}}\left[ \nabla _{\varvec{\alpha }}r(\varvec{Y};\varvec{\alpha })^{\gamma +1}\right] }{(\gamma +1)\bar{E}_{\textrm{r}}\left[ r(\varvec{Y};\varvec{\alpha })^{\gamma +1}\right] } -\frac{\bar{E}_{\textrm{d}}\left[ \nabla _{\varvec{\alpha }}r(\varvec{X};\varvec{\alpha })^{\gamma }\right] }{\gamma \bar{E}_{\textrm{d}}\left[ r(\varvec{X};\varvec{\alpha })^{\gamma }\right] }\\&\simeq \nabla _{\varvec{\alpha }}J_{\gamma }(\varvec{\alpha })\\&\quad +\eta \epsilon \frac{E_{\textrm{r}}\left[ r(\varvec{Y};\varvec{\alpha })^{\gamma +1}\right] \left\{ \nabla _{\varvec{\alpha }}r(\varvec{y}_{\textrm{o}};\varvec{\alpha })^{\gamma +1}\right\} -E_{\textrm{r}}\left[ \nabla _{\varvec{\alpha }}r(\varvec{Y};\varvec{\alpha })^{\gamma +1}\right] r(\varvec{y}_{\textrm{o}};\varvec{\alpha })^{\gamma +1}}{(\gamma +1)E_{\textrm{r}}\left[ r(\varvec{Y};\varvec{\alpha })^{\gamma +1}\right] ^2}\\&\quad -\epsilon \frac{ E_{\textrm{d}}\left[ r(\varvec{X};\varvec{\alpha })^{\gamma }\right] \left\{ \nabla _{\varvec{\alpha }}r(\varvec{x}_{\textrm{o}};\varvec{\alpha })^{\gamma }\right\} -E_{\textrm{d}}\left[ \nabla _{\varvec{\alpha }}r(\varvec{X};\varvec{\alpha })^{\gamma }\right] r(\varvec{x}_{\textrm{o}};\varvec{\alpha })^{\gamma }}{\gamma E_{\textrm{d}}\left[ r(\varvec{X};\varvec{\alpha })^{\gamma }\right] ^2}, \end{aligned}$$

where $\simeq$ denotes the equality up to $O(\epsilon ^2)$. As done in previous proofs, differentiating both sides of $\nabla _{\varvec{\alpha }}\bar{J}_{\gamma }(\varvec{\alpha })|_{\varvec{\alpha }=\varvec{\alpha }_{\epsilon }^{\gamma }}=\varvec{0}$ with respect to $\epsilon$ at $\epsilon =0$ yields

$$\begin{aligned}&\varvec{H}_{\gamma }\cdot \textrm{IF}(\varvec{x}_{\textrm{o}},\varvec{y}_{\textrm{o}})\nonumber \\&\quad +\left[ \eta \frac{E_{\textrm{r}}\left[ r(\varvec{Y};\varvec{\alpha })^{\gamma +1}\right] \left\{ \nabla _{\varvec{\alpha }}r(\varvec{y}_{\textrm{o}};\varvec{\alpha })^{\gamma +1}\right\} -E_{\textrm{r}}\left[ \nabla _{\varvec{\alpha }}r(\varvec{Y};\varvec{\alpha })^{\gamma +1}\right] r(\varvec{y}_{\textrm{o}};\varvec{\alpha })^{\gamma +1}}{(\gamma +1) E_{\textrm{r}}\left[ r(\varvec{Y};\varvec{\alpha })^{\gamma +1}\right] ^2} \right. \nonumber \\&\quad \left. -\frac{E_{\textrm{d}}\left[ r(\varvec{X};\varvec{\alpha })^{\gamma }\right] \left\{ \nabla _{\varvec{\alpha }}r(\varvec{x}_{\textrm{o}};\varvec{\alpha })^{\gamma }\right\} -E_{\textrm{d}}\left[ \nabla _{\varvec{\alpha }}r(\varvec{X};\varvec{\alpha })^{\gamma }\right] r(\varvec{x}_{\textrm{o}};\varvec{\alpha })^{\gamma }}{\gamma E_{\textrm{d}}\left[ r(\varvec{X};\varvec{\alpha })^{\gamma }\right] ^2} \right] \Biggr |_{\varvec{\alpha }=\varvec{\alpha }_{\star }^{\gamma }}=\varvec{0}. \end{aligned}$$

(E14)

The proof is completed by applying

$$\begin{aligned} \nabla _{\varvec{\alpha }}r(\varvec{x};\varvec{\alpha })^{\gamma }&=\gamma \cdot {r}(\varvec{x};\varvec{\alpha })^{\gamma }\varvec{g}_{\textrm{m}}(\varvec{x};\varvec{\alpha })\\ \frac{E_{\textrm{r}}\left[ \nabla _{\varvec{\alpha }}r(\varvec{Y};\varvec{\alpha })^{\gamma +1}\right] }{(\gamma +1)E_{\textrm{r}}\left[ r(\varvec{Y};\varvec{\alpha })^{\gamma +1}\right] } \Biggr |_{\varvec{\alpha }=\varvec{\alpha }_{\star }^{\gamma }}&=\frac{E_{\textrm{d}}\left[ \nabla _{\varvec{\alpha }}r(\varvec{X};\varvec{\alpha })^{\gamma }\right] }{\gamma E_{\textrm{d}}\left[ r(\varvec{X};\varvec{\alpha })^{\gamma }\right] } \Biggr |_{\varvec{\alpha }=\varvec{\alpha }_{\star }^{\gamma }}, \end{aligned}$$

in (E14), where the second equation is derived from $\nabla J_{\gamma }(\varvec{\alpha })|_{\varvec{\alpha }=\varvec{\alpha }_{\star }^{\gamma }}=\varvec{0}$. $\square$