More Powerful and Reliable Second-Level Statistical Randomness Tests for NIST SP 800-22
Random number generators (RNGs) are essential for cryptographic systems, and statistical tests are usually employed to assess the randomness of their outputs. As the most commonly used statistical test suite, the NIST SP 800-22 suite includes 15 test items, each of which contains two-level tests. For the test items based on the binomial distribution, we find that their second-level tests are flawed due to the inconsistency between the assessed distribution and the assumed one. That is, the sequence that passes the test could still have statistical flaws in the assessed aspect. For this reason, we propose Q-value as the metric for these second-level tests to replace the original P-value without any extra modification, and the first-level tests are kept unchanged. We provide the correctness proof of the proposed Q-value based second-level tests. We perform the theoretical analysis to demonstrate that the modification improves not only the detectability, but also the reliability. That is, the tested sequence that dissatisfies the randomness hypothesis has a higher probability to be rejected by the improved test, and the sequence that satisfies the hypothesis has a higher probability to pass it. The experimental results on several deterministic RNGs indicate that, the Q-value based method is able to detect some statistical flaws that the original SP 800-22 suite cannot realize under the same test parameters.
KeywordsStatistical randomness test NIST SP 800-22 Random number generator P-value
We thank the anonymous reviewers of CHES 2016 and ASIACRYPT 2016, for their invaluable suggestions and comments to improve the quality and fairness of this paper. This work was partially supported by National Basic Research Program of China (973 Program No. 2013CB338001), National Natural Science Foundation of China (No. 61602476) and Strategy Pilot Project of Chinese Academy of Sciences (No. XDA06010702).
- 1.The R project for statistical computing. http://www.r-project.org
- 6.Kim, S., Umeno, K., Hasegawa, A.: Corrections of the NIST statistical test suite for randomness. IACR Cryptology ePrint Archive 2004, 18 (2004). http://eprint.iacr.org/2004/018
- 7.Marsaglia, G.: Diehard Battery of Tests of Randomness. http://www.stat.fsu.edu/pub/diehard/
- 8.NIST: Statistical test suite (sts 2.1). http://csrc.nist.gov/groups/ST/toolkit/rng/documents/sts-2.1.2.zip
- 9.Pareschi, F., Rovatti, R., Setti, G.: Second-level NIST randomness tests for improving test reliability. In: International Symposium on Circuits and Systems (ISCAS 2007), pp. 1437–1440 (2007)Google Scholar
- 11.Rukhin, A., et al.: A statistical test suite for random and pseudorandom number generators for cryptographic applications. NIST Special Publication 800–22. http://csrc.nist.gov/publications/nistpubs/800-22-rev1a/SP800-22rev1a.pdf
- 15.Zhuang, J., Ma, Y., Zhu, S., Lin, J., Jing, J.: Q_value test: a new method on randomness statistical test. J. Cryptologic Res. 3(2), 192–201 (2016). (in Chinese)Google Scholar