More Powerful and Reliable Second-Level Statistical Randomness Tests for NIST SP 800-22

  • Shuangyi Zhu
  • Yuan Ma
  • Jingqiang Lin
  • Jia Zhuang
  • Jiwu Jing
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10031)


Random number generators (RNGs) are essential for cryptographic systems, and statistical tests are usually employed to assess the randomness of their outputs. As the most commonly used statistical test suite, the NIST SP 800-22 suite includes 15 test items, each of which contains two-level tests. For the test items based on the binomial distribution, we find that their second-level tests are flawed due to the inconsistency between the assessed distribution and the assumed one. That is, the sequence that passes the test could still have statistical flaws in the assessed aspect. For this reason, we propose Q-value as the metric for these second-level tests to replace the original P-value without any extra modification, and the first-level tests are kept unchanged. We provide the correctness proof of the proposed Q-value based second-level tests. We perform the theoretical analysis to demonstrate that the modification improves not only the detectability, but also the reliability. That is, the tested sequence that dissatisfies the randomness hypothesis has a higher probability to be rejected by the improved test, and the sequence that satisfies the hypothesis has a higher probability to pass it. The experimental results on several deterministic RNGs indicate that, the Q-value based method is able to detect some statistical flaws that the original SP 800-22 suite cannot realize under the same test parameters.


Statistical randomness test NIST SP 800-22 Random number generator P-value 



We thank the anonymous reviewers of CHES 2016 and ASIACRYPT 2016, for their invaluable suggestions and comments to improve the quality and fairness of this paper. This work was partially supported by National Basic Research Program of China (973 Program No. 2013CB338001), National Natural Science Foundation of China (No. 61602476) and Strategy Pilot Project of Chinese Academy of Sciences (No. XDA06010702).


  1. 1.
    The R project for statistical computing.
  2. 2.
    Blum, L., Blum, M., Shub, M.: A simple unpredictable pseudo-random number generator. SIAM J. Comput. 15(2), 364–383 (1986)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Devroye, L.: Introduction. In: Devroye, L. (ed.) Non-Uniform Random Variate Generation, pp. 1–26. Springer, New York (1986)CrossRefGoogle Scholar
  4. 4.
    Hamano, K.: The distribution of the spectrum for the discrete fourier transform test included in SP800-22. IEICE Trans. 88–A(1), 67–73 (2005)CrossRefGoogle Scholar
  5. 5.
    Hamano, K., Kaneko, T.: Correction of overlapping template matching test included in NIST randomness test suite. IEICE Trans. 90–A(9), 1788–1792 (2007)CrossRefGoogle Scholar
  6. 6.
    Kim, S., Umeno, K., Hasegawa, A.: Corrections of the NIST statistical test suite for randomness. IACR Cryptology ePrint Archive 2004, 18 (2004).
  7. 7.
    Marsaglia, G.: Diehard Battery of Tests of Randomness.
  8. 8.
  9. 9.
    Pareschi, F., Rovatti, R., Setti, G.: Second-level NIST randomness tests for improving test reliability. In: International Symposium on Circuits and Systems (ISCAS 2007), pp. 1437–1440 (2007)Google Scholar
  10. 10.
    Pareschi, F., Rovatti, R., Setti, G.: On statistical tests for randomness included in the NIST SP800-22 test suite and based on the binomial distribution. IEEE Trans. Inf. Forensics Secur. 7(2), 491–505 (2012)CrossRefGoogle Scholar
  11. 11.
    Rukhin, A., et al.: A statistical test suite for random and pseudorandom number generators for cryptographic applications. NIST Special Publication 800–22.
  12. 12.
    Storey, J.D.: The positive false discovery rate: A bayesian interpretation and the q-value. Ann. Stat. 31(6), 2013–2035 (2003)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Storey, J.D., Tibshirani, R.: Statistical significance for genomewide studies. Proc. Nat. Acad. Sci. 100(16), 9440–9445 (2003)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Sulak, F., Doğanaksoy, A., Ege, B., Koçak, O.: Evaluation of randomness test results for short sequences. In: Carlet, C., Pott, A. (eds.) SETA 2010. LNCS, vol. 6338, pp. 309–319. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15874-2_27 CrossRefGoogle Scholar
  15. 15.
    Zhuang, J., Ma, Y., Zhu, S., Lin, J., Jing, J.: Q_value test: a new method on randomness statistical test. J. Cryptologic Res. 3(2), 192–201 (2016). (in Chinese)Google Scholar

Copyright information

© International Association for Cryptologic Research 2016

Authors and Affiliations

  • Shuangyi Zhu
    • 1
    • 2
    • 3
  • Yuan Ma
    • 1
    • 2
  • Jingqiang Lin
    • 1
    • 2
  • Jia Zhuang
    • 1
    • 2
  • Jiwu Jing
    • 1
    • 2
  1. 1.Data Assurance and Communication Security Research CenterChinese Academy of SciencesBeijingChina
  2. 2.State Key Laboratory of Information Security, Institute of Information EngineeringChinese Academy of SciencesBeijingChina
  3. 3.University of Chinese Academy of SciencesBeijingChina

Personalised recommendations