A new patchwork simulation method with control of the local-mean histogram

Abstract

We present a new stochastic simulation method that builds two-dimensional images by assembling together square image pieces called blocks. The blocks are taken from a reference image. Our method, called patchwork simulation method (PSM), enforces pattern continuity in the image. Moreover, PSM allows to control the image local-mean histogram. This histogram bin-frequencies can be set to user-defined target values that may differ from the reference image local-mean histogram. This flexibility enhances the PSM generality by enlarging the set of all possible simulations. The local-mean histogram control is achieved by adjusting suitably the transition probabilities that associate a new block to an existing neighborhood in the partly simulated image. For several types of synthetic images and one polymer blend image, we show that PSM reproduces faithfully the reference image visual appearance (i.e. patterns are correctly shaped) and that simulated images are statistically compatible with the target local-mean histogram. Moreover, we show that our method has the ability to produce simulations that respect conditional hard data as well as a target local-mean histogram.

Appendix 1

1.1 (a) Determination of ${\varvec{\omega}}$

The bin-probabilities ${\user2{p}}$ of images simulated using the corrected transition probabilities (30) depend on ${\varvec{\omega}}.$ The problem is to find a weight vector ${\varvec{\omega}}^{\varvec{*}}$ such that ${\user2{p}}({\varvec{\omega}}^{\varvec{*}})={\user2{p}}_{t}.$ This is a root-finding problem that we tackle with a variant of Newton’s method, i.e via a linearization of the function ${\user2{p}}({\varvec{\omega}})$ around a point ${\varvec{\omega}} = \hat{{\varvec{\omega}}}^{(0)}$. For $\parallel { {\varvec{\omega}} - \hat{{\varvec{\omega}}}^{(0)}\parallel }$ small enough, ${\user2{p}}({\varvec{\omega}})$ can be approximated by its first order Taylor series around the point ${\varvec{\omega}} = \hat{ {\varvec{\omega}}}^{(0)}$, i.e. ${\user2{p}}({\varvec{\omega}}) \approx {\user2{p}}(\hat{ {\varvec{\omega}}}^{(0)}) + B \left({\varvec{\omega}} -\hat{ {\varvec{\omega}}}^{(0)}\right)$ for $\parallel {{\varvec{\omega}} - \hat{{\varvec{\omega}}}^{(0)}\parallel } < \varepsilon,$ where $\varepsilon >0$ and B is the M × M Jacobian matrix of ${\user2{p}}({\varvec{\omega}})$ evaluated at ${\varvec{\omega}} = \hat{{\varvec{\omega}}}^{(0)}.$ Since both ${\user2{p}}(\hat{{\varvec{\omega}}}^{(0)})$ and B are unknown, it can be assumed equivalently that

$$ {\user2{p}}({\varvec{\omega}}) \approx {\user2{a}} + B\,{\varvec{\omega}} \quad \text {for} \ \parallel{{\varvec{\omega}} - \hat{{\varvec{\omega}}}^{(0)} \parallel} < \varepsilon, $$

(31)

where ${{\user2{ a}} \in \mathbb{R}^M}$ and B are unknown parameters. Newton’s method consists in using the linear extrapolation (31) to estimate the root of ${\user2{p}}({\varvec{\omega}}) - {\user2{p}}_{t} = 0$. More precisely, if $\parallel{ {\varvec{\omega}} - \hat{ {\varvec{\omega}}}^{(0)} \parallel} < \varepsilon$ and if (31) holds, then ${\user2{p}}({\varvec{\omega}}) = {\user2{p}}_{t} \Rightarrow {\user2{a}}+ B {\varvec{\omega}} \approx {\user2{p}}_{\user2{t}} \Rightarrow {\varvec{\omega}} \approx B^{-1} \left( {\user2{p}}_{\user2{t}} - {\user2{ a}} \right) := \hat{{\varvec{\omega}}}^{(1)},$ as long as B ⁻¹ exists. If $\hat{{\varvec{\omega}}}^{(0)}$ is close enough to ${\varvec{\omega}}^{\varvec{*}},$ then $\hat{{\varvec{\omega}}}^{(1)}$ is a better estimate of ${\varvec{\omega}}^{\varvec{*}}$ than $\hat{{\varvec{\omega}}}^{(0)}.$ This linearisation and extrapolation procedure can be repeated around the point ${\varvec{\omega}} = \hat{{\varvec{\omega}}}^{(1)}$ to produce an even better estimate $\hat{{\varvec{\omega}}}^{(2)}$ of ${\varvec{\omega}}^{\varvec{*}}.$

In the context of our problem, the difficulty is that the function ${\user2{p}}( {\varvec{\omega}})$ is not known. At each step of the method, we must estimate the parameters ${\user2{a}}$ and B by performing a multidimensional linear regression in the neighborhood of the best estimate of ${\varvec{\omega}}^{\varvec{*}}.$ At the nth step, we choose a neighborhood ${\varvec{\omega}}(\hat{{\varvec{\omega}}}^{(n-1)}, \Updelta {\varvec{\omega}})$ that has the shape of a rectangular parallepiped centered on $\hat{{\varvec{\omega}}}^{(n-1)},$ and for which the widths in each direction are specified by the array $\Updelta {\varvec{\omega}} := (\delta {\varvec{\omega}}, \ldots, \delta {\varvec{\omega}})^T,$ where $\delta {\varvec{\omega}} = \frac{\alpha}{M}$ and $\alpha \in ]0,1[$ is an adjustable parameter. The width $\delta {\varvec{\omega}}$ must small enough for ${\user2{p}}({\varvec{\omega}})$ to be approximately linear within ${\varvec{\omega}}(\hat{{\varvec{\omega}}}^{(n-1)}, \Updelta {\varvec{\omega}}).$

The estimation of $({\user2{a}}, B)$ at the nth step is based on Q(n) simulated images generated using distinct weight vectors ${\varvec{\omega}}_q, q=1,\ldots,Q(n),$ which all belong to the box ${\varvec{\omega}}(\hat{{\varvec{\omega}}}^{(n-1)}, \Updelta {\varvec{\omega}}).$ For each image we compute the bin-frequencies ${\user2{f}}_{\user2{q}}({\varvec{\omega}}_q),$ which are estimates of the bin probabilities ${\user2{p}}({\varvec{\omega}}_q).$ Since ${\user2{f}}_q( {\varvec{\omega}}_q)$ and ${\user2{p}}({\varvec{\omega}}_q)$ differ by a random fluctuation, it follows from (31) that ${\user2{f}}_q({\varvec{\omega}}_q)$ is related to ${\varvec{\omega}}_q$ by the model

$$ {\user2{f}}_q({\varvec{\omega}}_q) = {\user2{a}} + B\,{\varvec{\omega}}_q+ {\user2{W}}_q, \ \ q=1,\ldots,Q(n), $$

(32)

where ${{\user2{W}}_q \in \mathbb{R}^M}$ is a random noise that includes both the random fluctuation and the linearization error. Using model (32), the least-squares estimate $(\hat{ {\user2{a}}},\hat{ B})$ of $({\user2{a}}, B)$ is

$$ (\hat{{\user2{a}}}, \hat{ B}) = \mathop{\text{argmin}}\limits_{{\user2{a}}, B} \ \ \frac{1}{2} \sum_{q=1}^{Q(n)} \left\| {\user2{f}}_q ({\varvec{\omega}}_q) - {\user2{a}} - B {\varvec{\omega}}_q \right\|^2. $$

(33)

The multidimensional linear regression used to obtain $(\hat{{\user2{a}}},\hat{ B})$ from (33) is discussed in the Appendix 1(b). Note that the condition Q(n) ≥ M + 1 must be satisfied for each n for the problem (33) to be well posed. To reduce the variance of $\hat{{\user2{a}}}$ and $\hat B,$ we use Q(n) ≥ Q _min > M + 1, where the value Q _min := M + 20 was used in our simulations. The resulting estimate $\hat{{\varvec{\omega}}}^{(n)}$ of ${\varvec{\omega}}^{\varvec{*}}$ is

$$ \hat{{\varvec{\omega}}}^{(n)} = \hat{B}^{-1} \left({\user2{p}}_{t} - \hat{{\user2{a}}}\right). $$

(34)

At each step, the weight vector ${\varvec{\omega}}_q$ must be suitably chosen. The iteration is initiated with uniform weights, i.e. with $\hat{{\varvec{\omega}}}^{(0)} = (1/M,\ldots,1/M)^T$. Note that the choice ${\varvec{\omega}}= \hat{{\varvec{\omega}}}^{(0)}$ usually gives a fair approximation of ${\varvec{\omega}}^{\varvec{*}}$ in the case where ${\user2{p}}_{{t}} = {\user2{p}}_{{r}} .$ In the first iteration, i.e. for n = 1, we generate randomly Q(1) = Q _min values of ${\varvec{\omega}}$ that lie in the box ${\varvec{\omega}}(\hat{{\varvec{\omega}}}^{(0)}, \Updelta {\varvec{\omega}}).$ More precisely, the ${\varvec{\omega}}_q$s are generated with

$$ {\varvec{\omega}}_q= {\frac{\hat{{\varvec{\omega}} }^{(n-1)} + \delta{\varvec{\omega}} \, {\varvec{\varepsilon}}_q^{(n)}}{\sum_{i=1}^{M} \hat{\varvec{\omega}}^{(n-1)}(i) + \delta {\varvec{\omega}} \ \varepsilon_{q}^{(n)}(i) }} $$

(35)

q = 1, …, Q(1), where the $\varepsilon_{q}^{(n)}(i)$s are mutually independent random variables uniformly distributed in [ − 1/2,1/2]. The denominator in (35) insures that the normalization constraint on the weights is satisfied. The estimation is performed from the values of ${\varvec{\omega}}$ contained in the list ${\mathbb{L}_1 := \{ {\varvec{\omega}}_1, \ldots, {\varvec{\omega}}_{Q(1)} \},}$ which leads to the first estimate $\hat{{\varvec{\omega}}}^{(1)}.$

In the next iterations, i.e. for n ≥ 2, the updated box ${\varvec{\Omega}}(\hat{{\varvec{\omega}}}^{(n-1)}, \Updelta{\varvec{\omega}})$ is centered on the latest estimate $\hat{{\varvec{\upomega}}}^{(n-1)}$ and contains a subset of the ${\varvec{\upomega}}_q$s contained in ${\mathbb{L}_{n-1}.}$ Indeed, the box displacement may cause the exclusion of some ${\varvec{\upomega}}_q$s. If the number k of ${\varvec{\upomega}}_q$s in ${ \mathbb{L}_{n-1} \cap {\varvec{\Omega}}(\hat{{\varvec{\upomega}}}^{(n-1)}, \Updelta {\varvec{\upomega}})}$ is less than Q _min , then we generate randomly a complementary list ${\mathbb{L}_{n}^{*}}$ of at least Q _min  − k new values of ${\varvec{\omega}}$ contained in ${\varvec{\Omega}}(\hat{ {\varvec{\upomega}}}^{(n-1)}, \Updelta {\varvec{\upomega}}).$ If k ≥ Q _min , then we generate only one additional value of ${\varvec{\upomega}}$ contained in ${\varvec{\Omega}}(\hat{{\varvec{\omega}}}^{(n-1)}, \Updelta {\varvec{\omega}}).$ In both cases, the new values of ${\varvec{\omega}}$ are generated via (35). The new estimate $\hat{ {\varvec{\omega}}}^{(n)}$ is computed from the values of ${\varvec{\omega}}$ contained in the updated list ${\mathbb{L}_n}$ defined by

$$ {\mathbb{L}}_n := \{{\varvec{\omega}}_q \in {\mathbb{L}}_{n-1} \cap {\varvec{\Omega}}(\hat{{\\varvec{\omega}}}^{(n-1)}, \Updelta {\varvec{\omega}}) \} \cup {\mathbb{L}}_{n}^{*}, $$

(36)

which always contains at least Q _min elements. Note that we do not include in ${\mathbb{L}_n}$ the estimates $\hat{ {\varvec{\omega}}}^{(1)}, \hat{{\varvec{\omega}}}^{(2)}, \ldots,\hat{{\varvec{\omega}}}^{(n-1)},$ which results in a homogeneous distribution of the ${\varvec{\omega}}$s in ${\varvec{\Omega}}(\hat{{\varvec{\omega}}}^{(n-1)}, \Updelta {\varvec{\omega}}).$

As n increases, the box movement slows down and comes to a near-stop around the optimal solution. For n large enough, the box position is nearly constant and the number of ${\varvec{\Omega}}_q$s in ${\varvec{\omega}}(\hat{{\varvec{\omega}}}^{(n-1)}, \Updelta {\varvec{\omega}})$ increases. We stop iterating when $|\mathbb{L}_{n}|=Q(n)=Q_{{\rm max}},$ which occurs for n = n _max, and we keep $\hat{{\varvec{\omega}}}^{(n_{{\rm max}})}$ as the best estimate of ${\varvec{\omega}}^{\varvec{*}}.$ The value of Q _max depends on the accuracy needed for the estimate ${\varvec{\omega}}^{\varvec{*}}.$

Note that the box size $\delta {\varvec{\omega}} = \frac{\alpha}{M}$ must be small enough for the linear model (32) to be valid within the box. To test this condition, we repeat the estimation process with a series of decreasing values of α, e.g. {1/2, 1/4, 1/8, …, 1/2ⁿ, …}, and we stop when the estimate of ${\varvec{\omega}}^{\varvec{*}}$ no longer changes significantly with α.

1.2 (b) Estimation of ${\user2{a}}$ and B

To find the least-squares estimates $(\hat{{\user2{ a}}},\hat B)$ of the parameters $({\user2{a}}, B),$ we search for the minimum of the function

$$ \begin{array}{l} E({\user2{a}}, B) = \frac{1}{2} \sum\limits_{q=1}^Q \left\| \hat{{\user2{p}}}_q -{\user2{ a}} - B {\varvec{\omega}}_q\right\|^2 \\ =\frac{1}{2}\sum\limits_{q=1}^Q\sum\limits_{i=1}^M \left(\hat{p}_q(i) - a(i) - \sum\limits_{j=1}^M b_{i,j} {\varvec{\omega}}_q(j)\right)^2 \end{array} $$

(37)

with respect to ${\user2{a}}$ and B. In (37), the b _i,j s are the elements of the Jacobian matrix B. The unknown parameters are the M components of ${\user2{a}}$ and the M ² matrix elements of B, for a total of M + M ² = M(M + 1) unknowns. Note that the minimum value $E({\user2{a}}, B) = 0$ can be obtained if

$$ \hat{{\user2{p}}}_q - {\user2{a}} - B\,{\varvec{\omega}}_q = 0, \quad \quad q \in \{ 1,2,\ldots,Q \}. $$

(38)

The vectors in (38) are M-dimensional, hence (38) is a linear system of Q M equations that has a unique solution if the number of equations is equal to the number of unknowns, i.e. if Q M = M (M + 1) ⇒ Q = M + 1. Hence we expect the minimization problem (33) to be uniquely determined if Q = M + 1, in which case $E(\hat{{\user2{a}}}, \hat{B}) = 0,$ and over-determined if Q > M + 1, in which case $E(\hat{{\user2{a}}}, \hat{B}) \ge 0.$

The gradient of E vanishes at the minimum of E. The conditions $\frac{\partial E}{\partial a_i}=0$ and $\frac{\partial E}{\partial b_{i,j}}=0$ lead respectively to the equations

$$ \begin{array}{l} Q a(i) + \sum\limits_{j=1}^M b_{i,j} \sum\limits_{q=1}^Q{\varvec{\omega}}_q(j) = \sum\limits_{q=1}^Q \hat{p}_q(i) \quad \quad \ \\ a_i \sum\limits_{q=1}^Q {\varvec{\omega}}_q(j) + \sum\limits_{k=1}^M b_{i,k} \sum\limits_{q=1}^Q {\varvec{\omega}}_q(k) {\varvec{\omega}}_q(j) = \sum\limits_{q=1}^Q \hat{p}_q(i) {\varvec{\omega}}_q(j) \end{array} $$

(39)

for $i,j\in \{1,\ldots,M\}.$ The equations (39) can be rewritten in the equivalent matrix form

$$ \left( \begin{array}{cc} Q & \sum_{q=1}^Q {\varvec{\omega}}_q^T\\ \sum_{q=1}^Q {\varvec{\omega}}_q & \sum_{q=1}^Q {\varvec{\omega}}_q {\varvec{\omega}}_q^T \end{array} \right) \ \left( \begin{array}{l} {\user2{a}}^T \\ B^T \end{array} \right) = \left( \begin{array}{c} \sum_{q=1}^Q \hat{{\user2{p}}}_q^T\\ \sum_{q=1}^Q {\varvec{\omega}} _q \hat{{\user2{p}} }_{q}^{T} \end{array} \right).$$

(40)

If Q ≥ M + 1, the linear systems of equations (40) can be solved numerically for ${\user2{a}}$ and B.