Estimation, prediction and interpretation of NGG random effects models: an application to Kevlar fibre failure times

Raffaele Argiento; Alessandra Guglielmi; Antonio Pievatolo

doi:10.1007/s00362-013-0528-8

Statistical Papers

August 2014, Volume 55, Issue 3, pp 805–826 | Cite as

Estimation, prediction and interpretation of NGG random effects models: an application to Kevlar fibre failure times

Authors
Authors and affiliations

Raffaele ArgientoEmail author
Alessandra Guglielmi
Antonio Pievatolo

Regular Article

First Online: 21 May 2013

Received: 03 May 2012

Revised: 07 February 2013

189 Downloads
1 Citations

Abstract

We propose a class of Bayesian semiparametric mixed-effects models; its distinctive feature is the randomness of the grouping of observations, which can be inferred from the data. The model can be viewed under a more natural perspective, as a Bayesian semiparametric regression model on the log-scale; hence, in the original scale, the error is a mixture of Weibull densities mixed on both parameters by a normalized generalized gamma random measure, encompassing the Dirichlet process. As an estimate of the posterior distribution of the clustering of the random-effects parameters, we consider the partition minimizing the posterior expectation of a suitable class of loss functions. As a merely illustrative application of our model we consider a Kevlar fibre lifetime dataset (with censoring). We implement an MCMC scheme, obtaining posterior credibility intervals for the predictive distributions and for the quantiles of the failure times under different stress levels. Compared to a previous parametric Bayesian analysis, we obtain narrower credibility intervals and a better fit to the data. We found that there are three main clusters among the random-effects parameters, in accordance with previous frequentist analysis.

Keywords

Bayesian nonparametrics Clustering Hierarchical models Mixture models Nonparametric models Generalized linear mixed models

Mathematics Subject Classification (2000)

62F15 62N01 62N05

This is a preview of subscription content, to check access.

Notes

Acknowledgments

We thank Andrea Cremaschi for computing the partition estimate illustrated in Sect. 4 using a R code.

Appendix: Posterior functionals of interest and MCMC algorithm

Computation of full Bayesian inferences requires the knowledge of the posterior distribution of the random density $f(v;G)$ in (2), or the corresponding distribution function $F(v;G)$, as well as the posterior distribution of the regression parameters $\beta $. Moreover, we will be interested in predicting the lifetime $T_{n+1}$ of a new pressure vessel wrapped with fibre from a new random spool, subject to a $\log $-stress $x$; according to the model considered, this predictive distribution will be computed as:

$$\begin{aligned} \!\!\!\mathbb P (T_{n+1}> t\mid \text{ data }, x) = \int \limits _\mathbb{R ^2\times \mathcal P } \left( \int \limits _{\Theta } \exp \left( {-\left( \frac{t}{\lambda _{x}} \right) ^{\vartheta _{1}}}\right) G(d\theta )\! \right) \mathcal L (d\beta _0,d\beta _1,dG| \text{ data }), \quad \end{aligned}$$

(16)

where $\mathcal P $ is the space of all probability measures on $\Theta $, $\lambda _x:=\beta _0+\beta _1 x +\log \vartheta _2$, and $\mathcal L (d\beta _0, d\beta _1,$ $d G| \text{ data } )$ denotes the joint posterior of $\beta _0$, $\beta _1$ and the random probability $G$. Therefore an MCMC algorithm is needed in order to approximate the posterior distribution of $G$, and also to provide credibility intervals for $F(v;G)$. Indeed, if $\left\{ \beta _0^{(b)},\beta _1^{(b)},G^{(b)}\right\} _{b=1}^{B}$ is an irreducible Markov chain with $\mathcal L (\beta _0,\beta _1,G|\text{ data })$ as the invariant distribution, then an estimate of $\mathbb P (T_{n+1}> t\mid \text{ data }, x)$ is the ergodic mean

$$\begin{aligned} \frac{1}{B}\sum _{b=1}^{B} \int \limits _{\Theta } \exp \left( {-\left( \frac{t}{\lambda _{x}^{(b)}} \right) ^{\vartheta _{1}^{(b)}}}\right) G^{(b)}(d\theta ), \end{aligned}$$

where the integral with respect to any $G^{(b)}(d\theta )$ is a infinite sum, since the NGG process selects discrete probabilities.

As far as the censored data are concerned, we imputed their values as usual, by augmenting the parameter space with the unobserved survival times $T_m^{n}=(T_{m+1},\dots ,T_n)$ (the last $m$ observations in the $n$-dimensional data vector are censored). The main difficulty here is to simulate an approximation of the infinite dimensional parameter $G$. By the posterior characterization in James et al. (2008), we resort to the algorithm illustrated in Argiento et al. (2010a). Indeed, we further extended the state space, by introducing a sample $\underline{\theta }=(\theta _1,\dots ,\theta _n)$ from $G$ and an auxiliary variable $U:=\Gamma _n/T$, where $\Gamma _n$ is distributed according the gamma$(n,1)$, and $T$ is as in (4); see the two references above for more details about $U$. Of course, since

$$\begin{aligned} \mathcal L (\beta _0,\beta _1,G|\text{ data })=\int \mathcal L (\beta _0,\beta _1,G,d\underline{\theta },du,dT_{m}^n|\text{ data }), \end{aligned}$$

we will resort to a Gibbs sampler to sequentially draw from the following full conditionals, omitting the conditioning on the data for the ease of notation:

$$\begin{aligned}&\text{ i. }\, \mathcal L (G|\underline{\theta },U,T_{m}^n,\beta ) \quad \text{ ii. }\, \mathcal L (\underline{\theta }|G,U,T_{m}^n,\beta ) \quad \text{ iii. } \, \mathcal L (\beta |\underline{\theta },G,U,T_{m}^n)\\&\text{ iv. }\, \mathcal L (U|\underline{\theta },G,T_{m}^n,\beta )\quad \text{ v. }\, \mathcal L (T_{m}^n|\underline{\theta },G,U,\beta ). \end{aligned}$$

i. Sampling $G$. By the hierarchical structure of our model, $G$ depends on the data only through the vector $\underline{\theta }$. Therefore, using the equivalent representation $(\underline{\psi }, \pi )$ for $\underline{\theta }$,

$$\begin{aligned} \mathcal L (G\mid \underline{\psi }, \pi ,{\beta },{T}_{m}^{n},U) =\mathcal L (G\mid \psi , \pi , u) \ . \end{aligned}$$

As illustrated in James et al. (2008), $\mathcal L (G\mid \psi , \pi , u)$ is the law of the random distribution function

$$\begin{aligned} \!\!\!\! G^*= \frac{1}{T_{n(\pi )}+\sum _{j=1}^{n(\pi )}L_j}\; \sum _{j=1}^{+\infty } J_j\delta _{\tau _j} +\frac{1}{T_{n(\pi )}+\sum _{j=1}^{n(\pi )}L_j}\; \sum _{j=1}^{n(\pi )}L_j\delta _{\psi _j}, \qquad T_{n(\pi )} = \sum _{j=1}^\infty J_j\ , \quad \end{aligned}$$

(17)

where the $\psi _j$’s are fixed points in the support of $G^*$ and the remaining weights and support points are random. In particular, the $L_j$’s are independently gamma $(e_j-\sigma ,\omega +u )$-distributed, with $e_j = \#C_j$, the $\tau _j$’s are a random sample from $G_0$ and the sequence $(J_j)_j$ is generated according to representation (4) of an $NGG(\sigma , \kappa \,G_0, \omega )$, with $\omega =1$ and $\kappa =\sigma \eta $.

While the sequence $(J_j)_j$ should be infinite, we use a finite sequence $(J_j)_{1\le j\le M}$, for $M$ such that $\mathbb P \left( \sum _{M+1}^{+\infty }J_j\le \tilde{\eta }\mathbb E (T_{n(\pi )} ) \right) \ge 1-\varepsilon $, where $\epsilon $ and $\tilde{\eta }$ are suitably small. For details about the truncation method and the simulation procedure from the Poisson process we refer to Argiento et al. (2010a).

ii. Sampling $\underline{\theta }$. From Bayes’ theorem, we have, for each $i=1,\ldots ,n$,

$$\begin{aligned} \mathcal L (\theta _i\mid \underline{\theta }_{-i}, G^*,U, {\beta },{T}_{m}^{n})\propto \sum _{j=1}^{M}J_j k(v_i;\theta _i)\delta _{\tau _j}(d\theta _i)+ \sum _{j=1}^{n(\pi )}L_j k(v_i;\theta _i)\delta _{\psi _j}(d\theta _i) \end{aligned}$$

(18)

where $v_i = \text{ e }^{- {x}_i^{\prime }{\beta }} t_i$, and $G^*$ is the truncated distribution sampled at the previous step.

This update may change both the partition $\pi $ and the $\underline{\psi }$ vector. However, as it is well known, there are often groups of observations that with high probability are associated with the same $\theta _i$. Since each full conditional (18) changes $\underline{\theta }$ in one single observation, an update of the cluster common value $\theta _i$ can be obtained with very low probability. A more efficient algorithm can be obtained resorting to an “acceleration step”, by which the $\psi _j$’s are updated for all the failure times indexed through $C_j$.

Acceleration step. This step is done by sampling from the full conditional distribution of $\psi _j$, as $j=1,\ldots ,n(\pi )$, which is proportional to

$$\begin{aligned} G_0(d\psi _j) \prod _{i\in C_j} k(v_i;\psi _j) \!. \end{aligned}$$

Recall that our specific kernel density is Weibull, with unknown shape and scale. Since there are no known conjugate prior distributions for these parameters, a Metropolis-within-Gibbs algorithm is required in our update scheme.

iii. Sampling ${\beta }$. If the components of ${\beta }$ are taken a priori independent with Gaussian distribution with zero mean and $\sigma _0^2$ variance, the full conditional of $\beta _j$ is proportional to the posterior distribution of the vector ${\beta }$:

$$\begin{aligned} \pi ({\beta }) \prod _{i=1}^n v_i^{\vartheta _{1i}} \exp \left\{ -\left( \frac{v_i}{\vartheta _{2i}}\right) ^{\vartheta _{1i}} \right\} \propto \exp \left\{ -\beta _j \sum _{i=1}^n \vartheta _{1i} x_{ij} - \sum _{i=1}^n \left( \frac{v_i}{\vartheta _{2i}}\right) ^{\vartheta _{1i}}-\frac{\beta _j^2}{2\sigma _0^2} \right\} \ , \end{aligned}$$

where $v_i = \text{ e }^{- {x}_i^{\prime }{\beta }} t_i$. We do not develop this expression further to avoid tedious calculations. The full conditional of $\beta _j$ is log-concave and is amenable to adaptive rejection sampling. The actual implementation of the adaptive rejection sampling, at least with our specific dataset, proved nontrivial, because for some values of the shape parameters the full conditional is very asymmetric and decreasing quickly on one side of the mode with respect to the other side. In this situation, the points for the approximating envelope on the side of the mode where the decrease is faster must be selected very carefully, so that they do not all lie where the density is negligible, causing numerical instability.

iv. Sampling $U$. It is shown in James et al. (2008) that the full conditional of $U$ is absolutely continuous with density proportional to

$$\begin{aligned} \left( \frac{u}{u+\omega }\right) ^n \frac{(u+\omega )^{n(\pi )\sigma }}{u} \text{ e }^{-\frac{\kappa }{\sigma } (u+\omega )^\sigma } \end{aligned}$$

where again we meet a non-standard distribution and a Metropolis step must be designed.

v. Sampling ${T}_{m}^{n}$. Let us suppose that all the censored failure times have the same censoring point at $t^*$, as it will be in the application considered in Sect. 5. Since the failure times are conditionally independent given $\underline{\theta }$, it is straightforward to see that, for any $i=m+1,\ldots ,n$,

$$\begin{aligned} \mathcal L (T_i|\underline{\theta },G,U,\beta )=\mathcal L (T_i|\theta _i,\beta ); \end{aligned}$$

the last distribution is Weibull with shape $\vartheta _{1i}$ and scale $\vartheta _{2i}$ conditioned to assume values greater than $t^*$, with survival function

$$\begin{aligned} \frac{\text{ e }^{-\left( \frac{t_i}{\vartheta _{2i}} \right) ^{\vartheta _{1i}}}}{\text{ e }^{-\left( \frac{t^*}{\vartheta _{2i}}\right) ^{\vartheta _{1i}}}},\qquad t_i>t^* \ , \end{aligned}$$

which can be inverted exactly.

It is worth mentioning that, since the algorithm just described is a conditional method, i.e. we do not integrate out the infinite dimensional parameter $G$, we are allowed to make full Bayesian inference; therefore posterior credibility bands for the survival function functional (16) can be computed, and nonlinear functional (like the quantiles) can be estimated as well. See Argiento et al. (2010a) for a theoretical justification of the functional estimation through the algorithm described above.

References

Argiento R, Guglielmi A, Pievatolo A (2008) Nonparametric Bayesian mixture modelling for failure time data. Technical report no. 08.3-MI, CNR-IMATI, MilanoGoogle Scholar
Argiento R, Guglielmi A, Pievatolo A (2010a) Bayesian density estimation and model selection using nonparametric hierarchical mixtures. Comput Stat Data Anal 54:816–832CrossRefMATHMathSciNetGoogle Scholar
Argiento R, Guglielmi A, Pievatolo A (2010b) Mixed-effects modelling of Kevlar fibre failure times through Bayesian non-parametrics. In: Mantovan P, Secchi P (eds) Complex data modelling and computationally intensive statistical methods. Springer, Milano, pp 13–26CrossRefGoogle Scholar
Argiento R, Guglielmi A, Soriano J (2012) A semiparametric Bayesian generalized linear mixed model for the reliability of Kevlar fibers. Appl Stoch Models Bus Ind (to appear)Google Scholar
Binder DA (1978) Bayesian cluster analysis. Biometrika 65:31–38CrossRefMATHMathSciNetGoogle Scholar
Binder DA (1981) Approximations to Bayesian clustering rules. Biometrika 68:275–285CrossRefMathSciNetGoogle Scholar
Crowder MJ, Kimber AC, Smith RL, Sweeting TJ (1991) Statistical analysis of reliability data. Chapman & Hall, LondonCrossRefGoogle Scholar
De Iorio M, Johnson WO, Müller P, Rosner GL (2009) Bayesian nonparametric nonproportional hazards survival modeling. Biometrics 65:762–771CrossRefMATHMathSciNetGoogle Scholar
Fritsch A, Ickstadt K (2009) Improved criteria for clustering based on the posterior similarity matrix. Bayesian Anal 4:367–392CrossRefMathSciNetGoogle Scholar
Gelfand AE, Dey DK (1994) Bayesian model choice: asymptotics and exact calculations. J R Stat Soc B 56:501–514MATHMathSciNetGoogle Scholar
Gerstle FP Jr, Kunz SC (1983) Prediction of long-term failure in Kevlar 49 composites. In: O’Brien TK (ed) Long-term behavior of composites. American Society for Testing and Materials, Philadelphia, pp 263–292Google Scholar
Ghosh SK, Ghosal S (2006) Semiparametric accelerated failure time models for censored data. In: Upadhyay SK et al (eds) Bayesian statistics and its applications. Anamaya Publishers, New Delhi, pp 213–219Google Scholar
Hanson TE (2006) Modeling censored lifetime data using a mixture of gammas baseline. Bayesian Anal 1:575–594CrossRefMathSciNetGoogle Scholar
Hanson TE, Jara A (2013) Surviving fully Bayesian nonparametric regression models. In: Damien P, Dellaportas P, Polson N, Stephens D (eds) Bayesian theory and applications. Oxford University Press, Oxford, pp 592–615Google Scholar
Ishwaran H, Zarepour M (2002) Dirichlet prior sieves in finite normal mixtures. Stat Sin 12:941–963MATHMathSciNetGoogle Scholar
James L, Lijoi A, Prünster I (2008) Posterior analysis for normalized random measures with independent increments. Scand J Stat 36:76–97CrossRefGoogle Scholar
Jara A, Lesaffre E, de Iorio M, Quintana F (2010) Bayesian semiparametric inference for multivariate doubly-interval-censored data. Ann Appl Stat 4:2126–2149CrossRefMATHMathSciNetGoogle Scholar
Kleinman KP, Ibrahim JG (1998) A semi-parametric Bayesian approach to generalized linear mixed models. Stat Med 17:2579–2596CrossRefGoogle Scholar
Kyung M, Gill J, Casella G (2010) Estimation in Dirichlet random effects models. Ann Stat 38:979–1009CrossRefMATHMathSciNetGoogle Scholar
Lau JW, Green PJ (2007) Bayesian model-based clustering procedures. J Comput Graph Stat 16:526–558CrossRefMathSciNetGoogle Scholar
León RV, Ramachandran R, Ashby A, Thyagarajan J (2007) Bayesian modeling of accelerated life tests with random effects. J Qual Technol 39:3–13Google Scholar
Lijoi A, Mena RH, Prünster I (2005) Hierarchical mixture modeling with normalized inverse-Gaussian prior. J Am Stat Assoc 100:1278–1291CrossRefMATHGoogle Scholar
Lijoi A, Mena RH, Prünster I (2007) Controlling the reinforcement in Bayesian nonparametric mixture models. J R Stat Soc Series B Stat Methodol 69:715–740CrossRefGoogle Scholar
Medvedovic M, Yeung KY, Bumgarner RE (2004) Bayesian mixture model based clustering of replicated microarray data. Bioformatics 20:1222–1232CrossRefGoogle Scholar
Müller P, Quintana FA, Rosner GL (2011) A product partition model with regression on covariates. J Comput Graph Stat 20:260–278CrossRefGoogle Scholar
Nieto-Barajas LF, Prünster I (2009) A sensitivity analysis for Bayesian nonparametric density estimators. Stat Sin 19:685–705MATHGoogle Scholar
Pitman J (1996) Some developments of the Blackwell-MacQueen urn scheme. In: Ferguson TS et al (eds) Statistic, probability and game theory, papers in honor of David Blackwell. Lectures Notes-Monograph Series, vol 30. Institute of Mathematica Statistics, Hayward, CA, pp 245–267Google Scholar
Pitman J, Yor M (1997) The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. Ann Probab 25:855–900CrossRefMATHMathSciNetGoogle Scholar
Quintana FA, Iglesias PL (2003) Bayesian clustering and product partition models. J R Stat Soc Series B 65:557–574CrossRefMATHMathSciNetGoogle Scholar
Tojeiro C, Louzada F (2012) A general threshold stress hybrid hazard model for lifetime data. Stat Pap 53:833848CrossRefMathSciNetGoogle Scholar

Copyright information

Authors and Affiliations

Raffaele Argiento
- 1
Email author
Alessandra Guglielmi
- 2
Antonio Pievatolo
- 1

1.CNR-IMATIMilanItaly
2.Dipartimento di MatematicaPolitecnico di MilanoMilanItaly

Estimation, prediction and interpretation of NGG random effects models: an application to Kevlar fibre failure times

Abstract

Keywords

Mathematics Subject Classification (2000)

Notes

Acknowledgments

References

Copyright information

Authors and Affiliations

Personalised recommendations

Cite article