Skip to main content
SpringerLink
Log in

Reproducibility of Statistical Tests Based on Randomised Response Data

  • Original Article
  • Published:
Journal of Statistical Theory and Practice Aims and scope Submit manuscript

Abstract

Reproducibility of experimental conclusions is an important topic in various fields, including social studies. The lack of reproducibility in research results not only limits scientific progress, but also wastes time, resources, and undermines society’s confidence in scientific findings. This paper focuses on the statistical reproducibility of hypothesis test outcomes based on data collected using randomised response techniques (RRT). Nonparametric predictive inference (NPI) is used to quantify reproducibility, which is well-suited to treat reproducibility as a prediction problem. NPI relies on few model assumptions and provides lower and upper bounds for reproducibility probabilities. This paper concludes that less variability in the reported responses of RRT methods leads to higher reproducibility of statistical hypothesis tests based on RRT data with the same degree of privacy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Goodman SN (1992) A comment on replication, p-values and evidence. Stat Med 11:875–879

    Article  CAS  PubMed  Google Scholar 

  2. Senn S (2002) A comment on replication, p-values and evidence S.N Goodman. Stat Med 21:2437–2444

    Article  PubMed  Google Scholar 

  3. Coolen FPA, BinHimd S (2014) Nonparametric predictive inference for reproducibility of basic nonparametric tests. J Stat Theory Pract 8:591–618

    Article  MathSciNet  Google Scholar 

  4. Augustin T, Coolen FPA (2004) Nonparametric predictive inference and interval probability. J Stat Plan Infer 124:251–272

    Article  MathSciNet  Google Scholar 

  5. Coolen FPA (1998) Low structure imprecise predictive inference for Bayes’ problem. Stat Prob Lett 36:349–357

    Article  MathSciNet  Google Scholar 

  6. BinHimd S (2014) Nonparametric predictive methods for bootstrap and test reproducibility. PhD Thesis. Durham University. Available at: http://npi-statistics.com

  7. Billheimer D (2019) Predictive inference and scientific reproducibility. Am Stat 73:291–295

    Article  MathSciNet  Google Scholar 

  8. De Finetti B (1974) Theory of probability. Wiley, London

    Google Scholar 

  9. Lovig M, Khalil S, Rahman S, Sapra P, Gupta S (2023) A mixture binary RRT model with a unified measure of privacy and efficiency. Commun Stat Simul Comput 52:2727–2737

    Article  MathSciNet  Google Scholar 

  10. Parker M, Gupta S, Khalil S (2024) A mixture quantitative randomized response model that improves trust in RRT methodology. Axioms 13:11

    Article  Google Scholar 

  11. Warner SL (1965) Randomized response: a survey technique for eliminating evasive answer bias. J Am Stat Assoc 62:63–69

    Article  Google Scholar 

  12. Greenberg BG, Abul-Ela ALA, Simmons WR, Horvitz DG (1969) The unrelated question randomized response model: theoretical framework. J Am Stat Assoc 65:520–539

    Article  MathSciNet  Google Scholar 

  13. Chaudhuri A (2016) Randomized response and indirect questioning techniques in surveys. CRC, New York

    Book  Google Scholar 

  14. Abernathy JR, Greenberg BG, Horvitz DG (1970) Estimates of induced abortion in urban North Carolina. Demography 7:19–29

    Article  CAS  PubMed  Google Scholar 

  15. Zhimin H, Zaizai Y (2012) Measure of privacy in randomized response model. Qual Quant 46:1167–1180

    Article  Google Scholar 

  16. Kuk AY (1990) Asking sensitive questions indirectly. Biometrika 77:436–438

    Article  MathSciNet  Google Scholar 

  17. Adebola FB, Adediran AA, Ewemooje OS (2017) Hybrid tripartite randomized response technique. Commun Stat Theory Methods 46:11756–11763

    Article  MathSciNet  Google Scholar 

  18. Fleiss JL, Levin B, Paik MC (2003) Statistical methods for rates and proportions, 3rd edn. Wiley, New York

    Book  Google Scholar 

  19. Eriksson SA (1973) A new model for randomized response. Int Stat Rev 41:101–113

    Article  Google Scholar 

  20. Young A, Gupta S, Parks R (2019) A binary unrelated-question RRT model accounting for untruthful responding. Invol J Math 12:1163–1173

    Article  MathSciNet  Google Scholar 

  21. Anderson H (1977) Efficiency versus protection in a general randomized response model. Scand J Stat 4:11–19

    MathSciNet  Google Scholar 

  22. Ljungqvist L (1993) A unified approach to measures of privacy in randomized response models: a utilitarian perspective. J Am Stat Assoc 88:97–103

    Google Scholar 

  23. Lanke J (1976) On the degree of protection in randomized interviews. Int Stat Rev 44:197–203

    Article  MathSciNet  Google Scholar 

  24. Hill BM (1968) Posterior distribution of percentiles: Bayes’ theorem for sampling from a population. J Am Stat Assoc 63:677–691

    MathSciNet  Google Scholar 

  25. Coolen FPA (2006) On nonparametric predictive inference and objective Bayesiansm. J Logic Lang Inf 15:21–47

    Article  MathSciNet  Google Scholar 

  26. Walley P (1991) Statistical reasoning with imprecise probabilities. Chapman and Hall, London

    Book  Google Scholar 

  27. Weichselberger K (2001) Elementare Grundbegriffe einer Allgemeineren Wahrscheinlichkeitsrechnung I. Intervallwahrscheinlichkeit als Umfassendes Konzept (In German). Physika, Heidelberg

  28. Arts GRJ, Coolen FPA (2008) Two nonparametric predictive control charts. J Stat Theory Pract 2:499–512

    Article  MathSciNet  Google Scholar 

  29. Arts GRJ, Coolen FPA, Van der Laan P (2004) Nonparametric predictive inference in statistical process control. Q Technol Quant Manag 1:201–216

    Article  MathSciNet  Google Scholar 

  30. Chen J, Coolen FPA, Coolen-Maturi T (2019) On nonparametric predictive inference for asset and European option trading in the binomial tree model. J Oper Res Soc 70:1678–1691

    Article  Google Scholar 

  31. Baker RM, Coolen-Maturi T, Coolen FPA (2017) Nonparametric predictive inference for stock returns. J Appl Stat 44:1333–1349

    Article  MathSciNet  Google Scholar 

  32. He T, Coolen FPA, Coolen-Maturi T (2019) Nonparametric predictive inference for European option pricing based on the Binomial Tree Model. J Oper Res Soc 70:1692–1708

    Article  Google Scholar 

  33. Coolen FPA, Coolen-Schrijner P (2006) Nonparametric predictive subset selection for proportions. Stat Prob Lett 76:1675–1684

    Article  MathSciNet  Google Scholar 

  34. Coolen FPA, Coolen-Schrijner P (2005) Nonparametric predictive reliability demonstration for failure-free periods. IMA J Manag Math 16:1–11

    MathSciNet  Google Scholar 

  35. Gibbons JD, Chakraborti S (2011) Nonparametric statistical inference, 5th edn. Chapman and Hall, Boca Raton, Florida

    Google Scholar 

  36. Coolen-Maturi T, Coolen-Schrijner P, Coolen FPA (2009) Nonparametric predictive pairwise comparison for real-valued data with terminated tails. Int J Approx Reason 51:141–150

    Article  MathSciNet  Google Scholar 

  37. Alqifari HN (2017) Nonparametric predictive inference for future order statistics. PhD Thesis. Durham University. Available at: http://npi-statistics.com

  38. Coolen FPA, Marques FJ (2020) Nonparametric predictive inference for test reproducibility by sampling future data orderings. J Stat Theory Pract 14:1–22

    MathSciNet  Google Scholar 

  39. Marques FJ, Coolen FPA, Coolen-Maturi T (2019) Approximations for the likelihood ratio statistic for hypothesis testing between two Beta distributions. J Stat Theory Pract 13:17

    Article  MathSciNet  Google Scholar 

  40. Simkus A, Coolen FPA, Coolen-Maturi T, Karp NA, Bendtsen C (2022) Statistical reproducibility for pairwise t-tests in pharmaceutical research. Stat Methods Med Res 31:673–688

    Article  MathSciNet  PubMed  Google Scholar 

  41. Chow S, Shao J, Wang H (2008) Sample size calculations in clinical research, 2nd edn. CRC, New York

    Google Scholar 

Download references

Acknowledgements

The research described in this article was conducted during Fatimah Alghamdi’s PhD studies at the Department of Mathematical Sciences, Durham University, funded by the Ministry of Education in Saudi Arabia, Princess Nourah bint Abdulrahman University, and the Saudi Arabian Cultural Bureau in London. We express our gratitude to Professor Sat Gupta for his valuable contributions and insightful discussions during this research project.

Author information

Authors and Affiliations

  1. Department of Mathematical Sciences, Princess Nourah bint Abdulrahman University, 11564, Riyadh, Saudi Arabia

    Fatimah M. Alghamdi

  2. Department of Mathematical Sciences, Durham University, Durham, DH1 3LE, UK

    Frank P. A. Coolen & Tahani Coolen-Maturi

Authors
  1. Fatimah M. Alghamdi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frank P. A. Coolen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tahani Coolen-Maturi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tahani Coolen-Maturi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alghamdi, F.M., Coolen, F.P.A. & Coolen-Maturi, T. Reproducibility of Statistical Tests Based on Randomised Response Data. J Stat Theory Pract 18, 13 (2024). https://doi.org/10.1007/s42519-024-00366-7

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42519-024-00366-7

Keywords

Search

Navigation