Skip to main content

Maximum Likelihood Estimation of Parameters of a Random Variable Using Monte Carlo Methods

Abstract

In a parametric estimation framework, this paper proposes different properties for the maximum likelihood estimators of unknown parameters of a given random variable having a known distribution, where different parameter estimation cases are studied. The Refined Descriptive Sampling (RDS) method is chosen to generate samples used for the estimation purpose. Then, we compare the RDS maximum likelihood estimators to their competitors provided by simple random samples with the same size and issued from the same distribution, through their Fisher information. Furthermore, the Maximum likelihood RDS mean is written as a function of its corresponding empirical estimator where the expression can be used to determine the estimator value when a refined descriptive sample is provided. All these results allow us to conclude that the proposed Maximum Likelihood Estimation (MLE) using refined descriptive samples is more efficient than that already obtained from simple random samples, which means that MLE using RDS has advantage in estimating parameters when the samples are not independent and identically distributed. Some Monte Carlo simulations are provided to validate the obtained theoretical results.

This is a preview of subscription content, access via your institution.

References

  1. Aldrich, J. (1997). R. A. Fisher and the making of Maximum Likelihood 1912–1922. Stat. Sci. 12, 162–176.

    MathSciNet  Article  Google Scholar 

  2. Aloui, A., Zioui, A., Ourbih-Tari, M. and Alioui, A. (2015). A general purpose module using refined descriptive sampling for installation in simulation systems. Comput. Stat. 30, 477–490.

    MathSciNet  Article  Google Scholar 

  3. Baghdali-Ourbih, L., Ourbih-Tari, M. and Dahmani, A. (2017). Implementing refined descriptive sampling into three phase Discrete-Event simulation models, communications in statistics: simulation and computation. Taylor and Francis 5, 4035–4049.

    MATH  Google Scholar 

  4. Baiche, L. and Ourbih-Tari, M. (2017). Large sample variance of simulation using refined descriptive sampling: case of independent variables. Commun. Stat. - Theory Methods 46, 510–519.

    MathSciNet  Article  Google Scholar 

  5. David, H. A. and Nagaraja, H.N. (2003). Order statistics, 3rd edn. Wiley, Canada.

    Book  Google Scholar 

  6. Dell, T. R. and Clutter, J. L. (1972). Ranked-set sampling theory with order statistics background. Biometrika 28, 545–555.

    Article  Google Scholar 

  7. Dimov, I. (2008). Monte Carlo methods for applied scientists. World Scientific Publishing, Singapore.

    MATH  Google Scholar 

  8. Farrell, P. J., Ehsanes Saleh, A. K. M. and Zhang, Z. (2011). Methods of moments estimation in finite mixtures. Sankhya A: Indian J. Stat. 73, 218–230.

    MathSciNet  Article  Google Scholar 

  9. Idjis, K., Ourbih-Tari, M. and Baghdali-Ourbih, L. (2017). Variance reduction in M/M/1 retrial queues using refined descriptive sampling. Commun. Stat. Simul. Comput. 46, 5002–5020.

    MathSciNet  Article  Google Scholar 

  10. Lynne Stokes, S. and Sager, T. W. (1988). Characterization of a ranked-set sample with application to estimating distribution function. J. Am. Stat. Assoc.83, 374–381.

    MathSciNet  Article  Google Scholar 

  11. Ourbih-Tari, M. and Azzal, M. (2017). Survival function estimation with nonparametric adaptive refined descriptive sampling algorithm: A case study. Commun. Stat.: Theory Methods 12, 5840–5850.

    Article  Google Scholar 

  12. Ourbih-Tari, M. and Guebli, S. (2015). A comparison of methods for selecting values of simulation input variables. ESAIM: Probab. Stat. 49, 135–147.

    MathSciNet  Article  Google Scholar 

  13. Robert, C. P. and Casella, G. (2004). Monte Carlo Statistical methods, 2nd edn. Springer Science+BusinessMedia Inc, UK.

    Book  Google Scholar 

  14. Saliby, E. (1990). Descriptive sampling: a better approach to monte carlo simulation. J. Oper. Res. Soc. 41, 1133–1142.

    Article  Google Scholar 

  15. Seung-Chunm, L. (2017). The restricted maximum likelihood estimation of a censored regression model. Commun. Stat. Appl. Methods 24, 291–301.

    Google Scholar 

  16. Stokes, L. (1995). Parametric ranked set sampling. Ann. Inst. Stat. Math. 47, 465–482.

    MathSciNet  MATH  Google Scholar 

  17. Tamiti, K., Ourbih-Tari, M., Aloui, A. and Idjis, K. (2018). The use of variance reduction, relative error and bias in testing the performance of M/G/1retrial queues estimators in Monte Carlo simulation. Monte Carlo Methods Appl.24, 165–178.

    MathSciNet  Article  Google Scholar 

  18. Tari, M. and Dahmani, A. (2006). Refined descriptive sampling: a better approach to monte carlo simulation. Simul. Model. Pract. Theory 14, 143–160.

    Article  Google Scholar 

  19. Tiryakioglu, M. (2008). On estimating Weibull modulus by moments and maximum likelihood methods. J. Mater. Sci. 43, 793–798.

    Article  Google Scholar 

Download references

Funding

This study was funded by The Directorate General of Scientific Research and Technological Development (DGRSDT), Ministry of Higher Education and Scientific Research of Algeria

Author information

Affiliations

Authors

Corresponding author

Correspondence to Oualid Saci.

Ethics declarations

Conflict of Interests

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1

Proof of equation (5.2) given in Theorem 1 of the Section 5.1.1. Equation (5.2) has been subdivided into three parts denoted Ii,i = 1,2,3 such as

$$I_{RDS}(\mu )=I_{1}+I_{2}+I_{3}$$

where

$$\left\{ \begin{array}{l} I_{1}=\frac{1}{\sigma^{2}}{\sum}_{j=1}^{m}{\sum}_{i=1}^{p_{j}} E\left[ \frac{-f^{\prime \prime} (Z_{(i)}^{j})}{ f(Z_{(i)}^{j})}+\left( \frac{f^{\prime} (Z_{(i)}^{j})}{f(Z_{(i)}^{j})}\right)^{2} \right] \\ I_{2}=\frac{1}{\sigma^{2}}{\sum}_{j=1}^{m}{\sum}_{i=1}^{p_{j}} (i-1)E\left[ \frac{-f^{\prime }(Z_{(i)}^{j})}{ F(Z_{(i)}^{j})}+\left( \frac{f(Z_{(i)}^{j})}{F(Z_{(i)}^{j})}\right)^{2}\right] \\ I_{3}=\frac{1}{\sigma^{2}}{\sum}_{j=1}^{m}{\sum}_{i=1}^{p_{j}} (p_{j}-i)E\left[ \frac{f^{\prime} (Z_{(i)}^{j})}{ 1-F(Z_{(i)}^{j})}+\left( \frac{f(Z_{(i)}^{j})}{1-F(Z_{(i)}^{j})}\right)^{2}\right]. \end{array} \right.$$
  • (1) Let us calculate the first term I1.

Let

$$\ h_{1}(z)=\left( \frac{f^{\prime} (z)}{f(z)}\right)^{2}-\frac{f^{\prime \prime} (z)}{f(z)},$$

and according to the definition of expectation

$$ E\left[ h_{1}(Z_{(i)}^{j})\right]={\int}_{-\infty }^{+\infty }h_{1}(z)f_{(i)}^{j}(z)dz, $$

using the expression of \(f_{(i)}^{j}(z)\) given in Section 5, we obtain

$$ E\left[ h_{1}(Z_{(i)}^{j})\right]={\int}_{-\infty }^{+\infty }h_{1}(z)\frac{p_{j}!}{(i-1)!(p_{j}-i)!}\left[ F(x)\right]^{i-1}f(x)\left[ 1-F(x)\right]^{p_{j}-i}dz.$$

It follows

$$ I_{1}=\frac{1}{\sigma^{2}}\sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} {\int}_{-\infty }^{+\infty }h_{1}(z)p_{j}C_{i-1}^{p_{j}-1}f(z)\left( F(z)\right)^{i-1}\left[ 1-F(z) \right]^{p_{j}-i}dz.$$

By making some rearrangements of the latter equation, we write

$$ \begin{array}{@{}rcl@{}} I_{1} &=&\frac{1}{\sigma^{2}}{\int}_{-\infty }^{+\infty }h_{1}(z)f(z)\sum\limits_{j=1}^{m} p_{j}\left[ \sum\limits_{i=0}^{p_{j}-1} C_{i}^{p_{j}-1}\left( F(z)\right)^{i}\left[ 1-F(z)\right]^{p_{j}-1-i} \right] dz \\ &&+\frac{1}{\sigma^{2}}{\int}_{-\infty }^{+\infty }h_{1}(z)f(z)\sum\limits_{j=1}^{m} p_{j}dz. \end{array} $$

By simplifying, we get

$$ \begin{array}{@{}rcl@{}} I_{1} &=&\frac{n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }h_{1}(z)f(z)dz \\ &=&\frac{n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left( \frac{f\prime (z)}{ f(z)}\right)^{2}f(z)dz-\frac{n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }f^{\prime \prime} (z)dz, \end{array} $$

by integrating the second term, it follows

$$ I_{1}=\frac{n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left( \frac{f^{\prime} (z) }{f(z)}\right)^{2}f(z)dz-\lim\limits_{x\rightarrow +\infty}\left[ \frac{n}{\sigma^{2}}f^{\prime} (z)\right]_{-x}^{x}. $$

Using the hypothesis of the Theorem 1, we have\(\lim \limits _{x\rightarrow +\infty } f^{\prime } (x)=\lim \limits _{x\rightarrow -\infty } f^{\prime } (x)\), so,

$$ I_{1}=\frac{n}{\sigma^{2}}E\left[\left( \frac{f^{\prime} (Z_{k})}{f(Z_{k})} \right)^{2}\right]. $$
  • (2) Let us calculate the second term I2.

Let

$$ h_{2}(z)=\left( \frac{f(z)}{F(z)}\right)^{2}-\frac{f^{\prime} (z)}{F(z)}.$$

Using the same method of calculation as when developing I1, we get

$$ \begin{array}{@{}rcl@{}} I_{2} &=&\frac{1}{\sigma^{2}}\sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} {\int}_{-\infty }^{+\infty }(i-1)h_{2}(z)p_{j}C_{i-1}^{p_{j}-1}f(z)\left( F(z)\right)^{i-1}\left[ 1-F(z)\right]^{p_{j}-i}dz \\ &=&\frac{1}{\sigma^{2}}{\int}_{-\infty }^{+\infty }h_{2}(z)f(z)\sum\limits_{j=1}^{m} p_{j}\left[ \sum\limits_{i=1}^{p_{j}} (i-1)C_{i-1}^{p_{j}-1}\left( F(z)\right)^{i-1}\left[ 1-F(z)\right] ^{p_{j}-i}\right] dz. \end{array} $$

As the term of the series is equal to zero for i = 1 so,

$$ I_{2}=\frac{1}{\sigma^{2}}{\int}_{-\infty }^{+\infty }h(z)f(z)\sum\limits_{j=1}^{m} p_{j}\left[ F(z)\sum\limits_{i=2}^{p_{j}} (i-1)C_{i-1}^{p_{j}-1}\left( F(z)\right)^{i-2}\left[ 1-F(z)\right]^{p_{j}-i}\right] dz. $$

By making some rearrangements of the latter equation, we obtain

$$ I_{2}=\frac{1}{\sigma^{2}}{\int}_{-\infty }^{+\infty }h_{2}(z)f(z)F(z) \sum\limits_{j=1}^{m}p_{j}\left[ \sum\limits_{i=1}^{p_{j}-1} \frac{(p_{j}-1)!}{(i-1)!(p_{j}-1-i)!}\left( F(z)\right)^{i-1}\left[ 1-F(z)\right]^{p_{j}-1-i}\right] dz $$

so,

$$ \begin{array}{@{}rcl@{}} I_{2} &=&\frac{1}{\sigma^{2}}{\int}_{-\infty }^{+\infty }h_{2}(z)f(z)F(z) \\ &&\sum\limits_{j=1}^{m}p_{j}\frac{(p_{j}-1)}{f(x)}\left[ \frac{1 }{(p_{j}-1)}\sum\limits_{i=1}^{p_{j}-1}\frac{(p_{j}-1)!}{ (i-1)!(p_{j}-1-i)!}f(x)\left( F(z)\right)^{i-1}\left[ 1-F(z)\right] ^{p_{j}-1-i}\right] dz. \end{array} $$

As

$$ \frac{1}{(p_{j}-1)}\sum\limits_{i=1}^{p_{j}-1}\frac{(p_{j}-1)!}{ (i-1)!(p_{j}-1-i)!}f(x)\left( F(z)\right)^{i-1}\left[ 1-F(z)\right]^{p_{j}-1-i}=f(x), $$

so,

$$ I_{2}=\frac{1}{\sigma^{2}}{\int}_{-\infty }^{+\infty }h_{2}(z)f(z)F(z) \sum\limits_{j=1}^{m}p_{j}(p_{j}-1)dz.$$

Taking \({\sum }_{j=1}^{m}(p_{j})^{2}=q\), it follows

$$ \begin{array}{@{}rcl@{}} I_{2} &=&\frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }f(z)h_{2}(z)F(z)dz \\ &=&\frac{q-n}{\sigma^{2}}E[\frac{\left( f(Z_{k})\right)^{2}}{ F(Z_{k})}-f^{\prime} (Z_{k})]. \end{array} $$
  • (3) Let us calculate the third term I3.

Let

$$ h_{3}(z)=\left( \frac{f(z)}{1-F(z)}\right)^{2}+\frac{f^{\prime }(z)}{ 1-F(z)}. $$

Using the same method of calculation as when developing I1, we get

$$ \begin{array}{@{}rcl@{}} I_{3} &=&\frac{1}{\sigma^{2}}\sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} {\int}_{-\infty }^{+\infty }(p_{j}-i)h_{3}(z)p_{j}C_{i-1}^{p_{j}-1}f(z)\left( F(z)\right)^{i-1}\left[ 1-F(z)\right]^{p_{j}-i}dz \\ &=&\frac{1}{\sigma^{2}}{\int}_{-\infty }^{+\infty }h_{3}(z)f(z)\sum\limits_{j=1}^{m} p_{j}\left[ \sum\limits_{i=1}^{p_{j}} (p_{j}-i)C_{i-1}^{p_{j}-1}\left( F(z)\right)^{i-1}\left[ 1-F(z)\right] ^{p_{j}-i}\right] dz. \end{array} $$

As the term of the series is equal to zero for i = pj so,

$$ I_{3}=\frac{1}{\sigma^{2}}{\int}_{-\infty }^{+\infty }h_{3}(z)f(z)\sum\limits_{j=1}^{m} p_{j}\left[ (1-F(z))\sum\limits_{i=1}^{p_{j}-1} (p_{j}-i)C_{i-1}^{p_{j}-1}\left( F(z)\right)^{i-1}\left[ 1-F(z) \right]^{p_{j}-1-i}\right] dz, $$

it follows

$$ \begin{array}{@{}rcl@{}} I_{3} &=&\frac{1}{\sigma^{2}}{\int}_{-\infty }^{+\infty }h_{3}(z)f(z)(1-F(z)) \\ &&\sum\limits_{j=1}^{m}\frac{p_{j}(p_{j}-1)}{f(z)}\left[ \frac{1 }{p_{j}-1}\sum\limits_{i=1}^{p_{j}-1}\frac{(p_{j}-1)!}{ (i-1)!(p_{j}-1-i)}f(z)\left( F(z)\right)^{i-1}\left[ 1-F(z)\right] ^{p_{j}-1-i}\right] dz. \end{array} $$

As

$$\frac{1}{p_{j}-1}\sum\limits_{i=1}^{p_{j}-1}\frac{(p_{j}-1)!}{ (i-1)!(p_{j}-1-i)}f(z)\left( F(z)\right)^{i-1}\left[ 1-F(z)\right]^{p_{j}-1-i}=f(z),$$

so,

$$ \begin{array}{@{}rcl@{}} I_{3} &=&\frac{1}{\sigma^{2}}{\int}_{-\infty }^{+\infty }h_{3}(z)f(z)(1-F(z)) \sum\limits_{j=1}^{m}p_{j}(p_{j}-1)dz \\ &=&\frac{q-n}{\sigma^{2}}E\left[\frac{\left( f(Z_{k})\right)^{2}}{ 1-F(Z_{k})}+f^{\prime} (Z_{k})\right]. \end{array} $$

By summing the three terms I1,I2 and I3, we obtain

$$ \begin{array}{@{}rcl@{}} I_{RDS}(\mu ) &=&\frac{n}{\sigma^{2}}E\left[\left( \frac{f^{\prime} (Z_{k})}{ f(Z_{k})}\right)^{2}\right] +\frac{q-n}{\sigma^{2}}E\left[\frac{\left( f(Z_{k})\right)^{2}}{F(Z_{k})}-f^{\prime} (Z_{k})\right] \\ &&+\frac{q-n}{\sigma^{2}}E\left[\frac{\left( f(Z_{k})\right)^{2}}{ 1-F(Z_{k})}+f^{\prime} (Z_{k})\right] \end{array} $$

which yields to the result.

Appendix 2

Proof of equation (5.1) given in Theorem 3 of the Section 5.2.1. Equation 5.1 has been subdivided into three parts denoted Ii,i = 4,5,6 such as

$$ I_{RDS}(\sigma)=I_{4}+I_{5}+I_{6} $$

where

$$ \left\{ \begin{array}{l} I_{4}=E\left[ \frac{\partial }{\partial \sigma }\left[ \frac{-1}{\sigma } \left( {\sum}_{j=1}^{m}{\sum}_{i=1}^{p_{j}} (i-1)\frac{Z_{(i)}^{j}f(Z_{(i)}^{j})}{F(Z_{(i)}^{j})}\right) \right] \right] \\ I_{5}=E\left[ \frac{\partial }{\partial \sigma }\left[ \frac{-1}{\sigma } \left( {\sum}_{j=1}^{m}{\sum}_{i=1}^{p_{j}} \frac{Z_{(i)}^{j}f^{\prime} (Z_{(i)}^{j})}{f(Z_{(i)}^{j})}\right) \right] \right] \\ I_{6}=E\left[ \frac{\partial }{\partial \sigma }\left[ \frac{-1}{\sigma } \left( {\sum}_{j=1}^{m}{\sum}_{i=1}^{p_{j}} (p_{j}-i)\frac{Z_{(i)}^{j}f(Z_{(i)}^{j})}{1-F(Z_{(i)}^{j})}\right) \right] \right]. \end{array} \right.$$
  • (1) Let us calculate the first term I4.

We have

$$ I_{4}=E\left[ \frac{\partial }{\partial \sigma }\left[ \frac{-1}{\sigma } \left( \sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} (i-1)\frac{Z_{(i)}^{j}f(Z_{(i)}^{j})}{F(Z_{(i)}^{j})}\right) \right] \right], $$

using the product derivation rule, we write

$$I_{4}=E\left[ \frac{1}{\sigma^{2}}\left( \sum\limits_{j=1}^{m} \sum\limits_{i=1}^{p_{j}}(i-1)\frac{Z_{(i)}^{j}f(Z_{(i)}^{j})}{ F(Z_{(i)}^{j})}\right) -\frac{1}{\sigma }\times \frac{\partial }{\partial \sigma }\left( \sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} (i-1)\frac{Z_{(i)}^{j}f(Z_{(i)}^{j})}{F(Z_{(i)}^{j})}\right) \right]. $$

By the derivation of the second term of I4 according to σ and by simplifying, we get

$$ I_{4}=\frac{1}{\sigma^{2}}E\left[ \sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} (i-1)\left( \frac{2Z_{(i)}^{j}f(Z_{(i)}^{j})}{ F(Z_{(i)}^{j})}+\frac{\left( Z_{(i)}^{j}\right)^{2}f^{\prime} (Z_{(i)}^{j})}{ F(Z_{(i)}^{j})}-\left( \frac{Z_{(i)}^{j}f(Z_{(i)}^{j})}{F(Z_{(i)}^{j})}\right)^{2}\right) \right]. $$

Let

$$ h_{4}(z)=\frac{2zf(z)}{F(z)}+\frac{\left( z\right)^{2}f^{\prime} (z)}{F(z)} -\left( \frac{Z_{(i)}^{j}f(z)}{F(z)}\right)^{2}, $$
(A.1)

the latter equality becomes

$$ \begin{array}{@{}rcl@{}} I_{4}&=&\frac{1}{\sigma^{2}}\sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} (i-1){\int}_{-\infty }^{+\infty }h_{4}(z)f_{(i)}^{j}(z)dz \\ &=&\frac{1}{\sigma^{2}}{\int}_{-\infty }^{+\infty }h_{4}(z)\sum\limits_{j=1}^{m} \sum\limits_{i=1}^{p_{j}}(i-1)\frac{p_{j}! }{(i-1)!(p_{j}-i)!}\left[ F(x)\right]^{i-1}f(x)\left[ 1-F(x)\right] ^{p_{j}-i}dz. \end{array} $$

Using the same method of calculation as when developing I2, we get

$$ \begin{array}{@{}rcl@{}} I_{4} &=&\frac{1}{\sigma^{2}}{\int}_{-\infty }^{+\infty }h_{4}(z)f(z)F(z) \sum\limits_{j=1}^{m}p_{j}(p_{j}-1)dz \\ &=&\sum\limits_{j=1}^{m}p_{j}(p_{j}-1)\frac{1}{\sigma^{2}} {\int}_{-\infty }^{+\infty }h_{4}(z)f(z)F(z)dz, \end{array} $$

substituting the Eq. A.1 in the latter equality, we obtain

$$ I_{4}=\frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left[ 2z\left( f(z)\right)^{2}+\left( z\right)^{2}f^{\prime} (z)f(z)-\frac{\left( zf(z)\right)^{2}f(z)}{F(z)}\right] dz. $$
(A.2)
  • (2) Let us calculate the second term I5.

We have

$$ I_{5}=E\left[ \frac{\partial }{\partial \sigma }\left[ \frac{-1}{\sigma } \left( \sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} \frac{Z_{(i)}^{j}f^{\prime} (Z_{(i)}^{j})}{f(Z_{(i)}^{j})}\right) \right] \right]. $$

By the derivation according to σ, we obtain

$$ \begin{array}{@{}rcl@{}} I_{5}&=&E\left[ \frac{1}{\sigma^{2}}\sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} \frac{Z_{(i)}^{j}f^{\prime} (Z_{(i)}^{j})}{f(Z_{(i)}^{j}) }\right] \\ &&-E\left[ \frac{1}{\sigma }\left[\sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} \frac{\frac{-Z_{(i)}^{j}f^{\prime} (Z_{(i)}^{j})f(Z_{(i)}^{j})}{\sigma }+\frac{-\left( Z_{(i)}^{j}\right)^{2}f^{\prime \prime} (Z_{(i)}^{j})f(Z_{(i)}^{j})}{\sigma }+\frac{\left( Z_{(i)}^{j}\right)^{2}\left( f^{\prime} (Z_{(i)}^{j})\right)^{2}}{\sigma }}{\left( f(Z_{(i)}^{j})\right)^{2}}\right] \right]. \end{array} $$

By simplifying the latter equality, we get

$$ I_{5}=E\left[ \frac{1}{\sigma^{2}}\sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} \frac{Z_{(i)}^{j}f^{\prime} (Z_{(i)}^{j})}{ f(Z_{(i)}^{j})}+\frac{Z_{(i)}^{j}f^{\prime} (Z_{(i)}^{j})}{f(Z_{(i)}^{j})}+\frac{\left( Z_{(i)}^{j}\right)^{2}f^{\prime \prime} (Z_{(i)}^{j})}{f(Z_{(i)}^{j})}-\left( \frac{ Z_{(i)}^{j}f^{\prime} (Z_{(i)}^{j})}{f(Z_{(i)}^{j})}\right)^{2}\right]. $$

Let

$$ h_{5}(z)=\frac{2zf^{\prime} (z)}{f(z)}+\frac{\left( z\right)^{2}f^{\prime \prime} (z)}{f(z)}-\left( \frac{zf^{\prime} (z)}{f(z)}\right)^{2}, $$
(A.3)

and using the same method of calculation as when developing I1, we obtain

$$ I_{5}=\frac{n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left[ 2zf^{\prime} (z)+\left( z\right)^{2}f^{\prime \prime} (z)-\left( \frac{zf^{\prime} (z)}{f(z)} \right)^{2}f(z)\right] dz. $$
(A.4)

(3) Let us calculate the third term I6.

We have

$$ I_{6}=E\left[ \frac{\partial }{\partial \sigma }\left[ \frac{-1}{\sigma } \left( \sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} (p_{j}-i)\frac{Z_{(i)}^{j}f(Z_{(i)}^{j})}{1-F(Z_{(i)}^{j})}\right) \right] \right].$$

By the derivation according to σ, we obtain

$$ \begin{array}{@{}rcl@{}} I_{6}&=&E\left[ \frac{1}{\sigma^{2}}\sum\limits_{j=1}^{m} \sum\limits_{i=1}^{p_{j}}(p_{j}-i)\frac{Z_{(i)}^{j}f(Z_{(i)}^{j}) }{1-F(Z_{(i)}^{j})}\right] \\ &&+E\left[ -\frac{1}{\sigma }\left[ \sum\limits_{j=1}^{m} \sum\limits_{i=1}^{p_{j}}(p_{j}-i)\frac{\frac{ -Z_{(i)}^{j}f(Z_{(i)}^{j})\left[ 1-F(Z_{(i)}^{j})\right] }{\sigma }+\frac{ -\left( Z_{(i)}^{j}\right)^{2}f^{\prime} (Z_{(i)}^{j})\left[ 1-F(Z_{(i)}^{j}) \right] }{\sigma }+\frac{-\left( Z_{(i)}^{j}\right)^{2}\left( f(Z_{(i)}^{j})\right)^{2}}{\sigma }}{\left[ 1-F(Z_{(i)}^{j})\right]^{2}} \right] \right], \end{array} $$

by simplifying the latter equation, we have

$$ \begin{array}{@{}rcl@{}} I_{6} &=&E\left[ \frac{1}{\sigma^{2}}\sum\limits_{j=1}^{m} \sum\limits_{i=1}^{p_{j}}(p_{j}-i)\frac{Z_{(i)}^{j}f(Z_{(i)}^{j}) }{1-F(Z_{(i)}^{j})}\right] \\ &&+E\left[ \frac{1}{\sigma^{2}}\sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} (p_{j}-i)\left[ \frac{Z_{(i)}^{j}f(Z_{(i)}^{j})}{ 1-F(Z_{(i)}^{j})}+\frac{\left( Z_{(i)}^{j}\right)^{2}f^{\prime} (Z_{(i)}^{j})}{ 1-F(Z_{(i)}^{j})}+\left( \frac{Z_{(i)}^{j}f(Z_{(i)}^{j})}{1-F(Z_{(i)}^{j})} \right)^{2}\right] \right], \end{array} $$

and by making some rearrangements of the latter equation’s terms, we obtain

$$ I_{6}=\frac{1}{\sigma^{2}}E\left[ \sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} (p_{j}-i)\left[ \frac{2Z_{(i)}^{j}f(Z_{(i)}^{j}) }{1-F(Z_{(i)}^{j})}+\frac{\left( Z_{(i)}^{j}\right)^{2}f^{\prime} (Z_{(i)}^{j})}{ 1-F(Z_{(i)}^{j})}+\left( \frac{Z_{(i)}^{j}f(Z_{(i)}^{j})}{1-F(Z_{(i)}^{j})} \right)^{2}\right] \right]. $$

Let

$$ h_{6}(z)=\frac{2zf(z)}{1-F(z)}+\frac{\left( z\right)^{2}f^{\prime} (z)}{1-F(z)} +\left( \frac{zf(z)}{1-F(z)}\right)^{2}. $$
(A.5)

Using the same method of calculation as when developing I3, it follows

$$ \begin{array}{@{}rcl@{}} I_{6} &=&\frac{1}{\sigma^{2}}{\int}_{-\infty }^{+\infty }h_{6}(z)f(z)(1-F(z)) \sum\limits_{j=1}^{m}p_{j}(p_{j}-1)dz \\ &=&\frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }h_{6}(z)f(z)(1-F(z))dz, \end{array} $$

so,

$$ I_{6}=\frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left[ 2z\left( f(z)\right)^{2}+\left( z\right)^{2}f^{\prime} (z)f(z)+\frac{\left( zf(z)\right)^{2}}{1-F(z)}f(z)\right] dz. $$
(A.6)

By summing the three terms I4,I5 and I6, we obtain

$$ \begin{array}{@{}rcl@{}} I_{RDS}(\sigma )&=&\frac{-n}{\sigma^{2}}-\left[ \frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left[ 2z\left( f(z)\right)^{2}+\left( z\right)^{2}f^{\prime} (z)f(z)-\frac{\left( zf(z)\right)^{2}f(z)}{F(z)}\right] dz\right] \\ &&-\left[ \frac{n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left[ 2zf^{\prime} (z)+\left( z\right)^{2}f^{\prime \prime} (z)-\left( \frac{zf^{\prime} (z)}{f(z)} \right) f(z)\right] dz\right] \\ &&+\frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left[ 2z\left( f(z)\right)^{2}+\left( z\right)^{2}f^{\prime} (z)f(z)+\frac{\left( zf(z)\right)^{2}}{1-F(z)}f(z)\right] dz. \end{array} $$

By simplifying the latter equality, it follows

$$ \begin{array}{@{}rcl@{}} I_{RDS}(\sigma )&=&\frac{-n}{\sigma^{2}}+\frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty } \frac{\left( zf(z)\right)^{2}f(z)}{F(z)}dz \\ &&-\frac{n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }2zf^{\prime} (z)dz-\frac{n}{ \sigma^{2}}{\int}_{-\infty }^{+\infty }\left( z\right)^{2}f^{\prime \prime} (z)dz \\ &&+\frac{n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left( \frac{zf^{\prime} (z)}{f(z)}\right)^{2}f(z)dz+\frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\frac{\left( zf(z)\right)^{2}}{1-F(z)}f(z)dz, \end{array} $$

using the integration part of the fourth term of the right hand side of the latter equation, we get

$$ \begin{array}{@{}rcl@{}} I_{RDS}(\sigma )&=&\frac{-n}{\sigma^{2}}+\frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty } \frac{\left( zf(z)\right)^{2}f(z)}{F(z)}dz-\frac{n}{\sigma^{2}} {\int}_{-\infty }^{+\infty }2zf^{\prime} (z)dz \\ &&-\frac{n}{\sigma^{2}}\left( \left[ \left( z\right)^{2}f^{\prime} (z)\right]_{-\infty }^{+\infty }-2{\int}_{-\infty }^{+\infty }zf^{\prime} (z)dz\right) \\ &&+\frac{n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left( \frac{zf^{\prime} (z)}{ f(z)}\right)^{2}f(z)dz+\frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\frac{\left( zf(z)\right)^{2}}{1-F(z)}f(z)dz, \end{array} $$

after simplifying, we obtain

$$ \begin{array}{@{}rcl@{}} I_{RDS}(\sigma )&=&\frac{-n}{\sigma^{2}}+\frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty } \frac{\left( zf(z)\right)^{2}f(z)}{F(z)}dz \\ &&-\frac{n}{\sigma^{2}}\left[ \left( z\right)^{2}f^{\prime} (z)\right]_{-\infty }^{+\infty }+\frac{n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left( \frac{zf^{\prime} (z)}{f(z)}\right)^{2}f(z)dz \\ &&+\frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\frac{\left( zf(z)\right)^{2}}{1-F(z)}f(z)dz. \end{array} $$

According to the hypothesis of the Theorem 3, we have \(\left [ \left (z\right )^{2}f^{\prime } (z)\right ]_{-\infty }^{+\infty }=0\), so,

$$ \begin{array}{@{}rcl@{}} I_{RDS}(\sigma )&=&\left[ \frac{n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left( \frac{ zf^{\prime} (z)}{f(z)}\right)^{2}f(z)dz-\frac{n}{\sigma^{2}}\right] \\ &&+\frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left( \frac{\left( zf(z)\right)^{2} }{F(z)}+\frac{\left( zf(z)\right)^{2}}{1-F(z)}\right) f(z)dz \\ &=&\left[ \frac{n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left( \frac{ zf^{\prime} (z)}{f(z)}\right)^{2}f(z)dz-\frac{n}{\sigma^{2}}\right] \\ &&+\frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left( \frac{\left( zf(z)\right)^{2} }{F(z)\left[ 1-F(z)\right] }\right) f(z)dz \end{array} $$

which yields to the result.

Appendix 3

Proof of equation (5.3) given in Theorem 5 of the Section 5.3. By the derivation of each term of the right hand side of the Eq. 5.8, we get

$$ \begin{array}{@{}rcl@{}} I_{RDS}(\mu ,\sigma )&=&-E\left[ \frac{1}{\sigma^{2}}\left( \sum\limits_{j=1}^{m} \sum\limits_{i=1}^{p_{j}}(i-1)\frac{ f(Z_{(i)}^{j})}{F(Z_{(i)}^{j})}+\sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} \frac{f^{\prime} (Z_{(i)}^{j})}{f(Z_{(i)}^{j})}\right) \right] \\ &&-E\left[ \frac{-1}{\sigma }\frac{\partial }{\partial \sigma }\left( \sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}}(i-1) \frac{f(Z_{(i)}^{j})}{F(Z_{(i)}^{j})}+\sum\limits_{j=1}^{m} \sum\limits_{i=1}^{p_{j}}\frac{f^{\prime} (Z_{(i)}^{j})}{f(Z_{(i)}^{j}) }\right) \right] \\ &&+E\left[ \frac{1}{\sigma^{2}}\left( \sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} (p_{j}-i)\frac{f(Z_{(i)}^{j})}{1-F(Z_{(i)}^{j})}\right) \right] \\ &&-E\left[ \frac{1}{\sigma }\frac{\partial }{\partial \sigma }\left( \sum\limits_{j=1}^{m} \sum\limits_{i=1}^{p_{j}} (p_{j}-i)\frac{f(Z_{(i)}^{j})}{1-F(Z_{(i)}^{j})}\right) \right]. \end{array} $$
(A.7)

By developing the latter equality, we write the following expressions

$$ \frac{\partial }{\partial \sigma }\left( \sum\limits_{j=1}^{m} {\sum}_{i=1}^{p_{j}}(i-1)\frac{f(Z_{(i)}^{j})}{ F(Z_{(i)}^{j})}\right) =\sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} (i-1)\left[ \frac{Z_{(i)}^{j}f^{\prime} (Z_{(i)}^{j})}{ F(Z_{(i)}^{j})}-Z_{(i)}^{j}\left( \frac{f(Z_{(i)}^{j})}{F(Z_{(i)}^{j})}\right)^{2}\right], $$
$$ \frac{\partial }{\partial \sigma }\left( \sum\limits_{j=1}^{m} \sum\limits_{i=1}^{p_{j}}\frac{f^{\prime} (Z_{(i)}^{j})}{f(Z_{(i)}^{j}) }\right) =\sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} \left[ \frac{Z_{(i)}^{j}f^{\prime \prime} (Z_{(i)}^{j})}{f(Z_{(i)}^{j})} -Z_{(i)}^{j}\left( \frac{f^{\prime} (Z_{(i)}^{j})}{f(Z_{(i)}^{j})}\right)^{2} \right], $$
$$\frac{\partial }{\partial \sigma }\left( \sum\limits_{j=1}^{m} \sum\limits_{i=1}^{p_{j}}(p_{j}-i)\frac{f(Z_{(i)}^{j})}{ 1-F(Z_{(i)}^{j})}\right) =\sum\limits_{j=1}^{m}\sum\limits_{i=1}^{p_{j}} (p_{j}-i)\left[ \frac{Z_{(i)}^{j}f^{\prime} (Z_{(i)}^{j})}{ 1-F(Z_{(i)}^{j})}+Z_{(i)}^{j}\left( \frac{f(Z_{(i)}^{j})}{1-F(Z_{(i)}^{j})} \right)^{2}\right], $$

so, by replacing these expressions in Eq. A.7, we obtain

$$ \begin{array}{@{}rcl@{}} I_{RDS}(\mu ,\sigma )& = &-E\left[ \frac{1}{\sigma^{2}}\left( \sum\limits_{j=1}^{m} \sum\limits_{i=1}^{p_{j}}(i-1)\left[ \frac{f(Z_{(i)}^{j})}{F(Z_{(i)}^{j})}+\frac{Z_{(i)}^{j}f^{\prime} (Z_{(i)}^{j})}{ F(Z_{(i)}^{j})}-Z_{(i)}^{j}\left( \frac{f(Z_{(i)}^{j})}{F(Z_{(i)}^{j})}\right)^{2}\right] \right) \right] \\ &&-E\left[ \frac{1}{\sigma^{2}}\left( \sum\limits_{j=1}^{m} \sum\limits_{i=1}^{p_{j}}\frac{f^{\prime} (Z_{(i)}^{j})}{f(Z_{(i)}^{j}) }+\frac{Z_{(i)}^{j}f^{\prime \prime} (Z_{(i)}^{j})}{f(Z_{(i)}^{j})} -Z_{(i)}^{j}\left( \frac{f^{\prime} (Z_{(i)}^{j})}{f(Z_{(i)}^{j})}\right)^{2}\right) \right] \\ &&+E\left[ \frac{1}{\sigma^{2}}\left( \sum\limits_{j=1}^{m} \sum\limits_{i=1}^{p_{j}}(p_{j}-i)\left[ \frac{f(Z_{(i)}^{j})}{ 1-F(Z_{(i)}^{j})}+\frac{Z_{(i)}^{j}f^{\prime} (Z_{(i)}^{j})}{1-F(Z_{(i)}^{j})} +Z_{(i)}^{j}\left( \frac{f(Z_{(i)}^{j})}{1-F(Z_{(i)}^{j})}\right)^{2}\right] \right) \right]. \end{array} $$

Using the same method of calculation as in appendices 1 and 2, we get

$$ \begin{array}{@{}rcl@{}} I_{RDS}(\mu ,\sigma ) &=&-\frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty } \left[ f(z)+zf^{\prime} (z)-\frac{z(f(z))}{F(z)}^{2}\right] f(z)fdz \\ &&-\frac{n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left[ f^{(1)}(z)+zf^{\prime \prime} (z)-z\left( \frac{f^{\prime} (z)}{f(z)}\right)^{2}f(z)\right] dz \\ &&+\frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }\left[ f(z)+zf^{\prime} (z)+ \frac{z(f(z))^{2}}{1-F(z)}\right] f(z)dz, \end{array} $$

by simplifying the latter equation, we get

$$ \begin{array}{@{}rcl@{}} I_{RDS}(\mu ,\sigma ) &=&\frac{q-n}{\sigma^{2}}{\int}_{-\infty }^{+\infty } \left[ \frac{z(f(z))^{2}}{F(z)\left( 1-F(z)\right) }\right] f(z)dz \\ &&+\frac{n}{\sigma^{2}}{\int}_{-\infty }^{+\infty }z\left( \frac{f^{\prime} (z)}{ f(z)}\right)^{2}f(z)dz \end{array} $$

which yields to the result.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Saci, O., Ourbih-Tari, M. & Baiche, L. Maximum Likelihood Estimation of Parameters of a Random Variable Using Monte Carlo Methods. Sankhya A (2021). https://doi.org/10.1007/s13171-021-00265-0

Download citation

Keywords

  • Refined Descriptive Sampling
  • Monte Carlo method
  • Order statistics
  • Cumulative distribution function
  • Maximum likelihood

AMS (2000) subject classification

  • Primary: 65C05
  • Secondary: 62F07
  • 62G30
  • 11K45