# The entropy of a distributed computation random number generation from memory interleaving

- 175 Downloads

## Abstract

We ask to what extent processes communicating through shared memory can extract *randomness* from their underlying scheduler, e.g., to generate random numbers for cryptographic applications. We introduce the quantitative notions of *entropy rate* and *information capacity* of a distributed algorithm. Whilst the entropy rate measures the Shannon information that may pass from a given scheduler to the processes executing the algorithm, the information capacity measures the optimal entropy rate over all possible schedulers. We present a general method for computing these quantities by classifying distributed algorithms according to their pattern of shared memory accesses. We then address the issue of effectively extracting, online, the information produced by the scheduler into a meaningful format at every process. We present Duez, an algorithm solving this problem with an optimal memory consumption. Putting these principles into practice, we introduce Co-RNG, a random number generator that leverages the unpredictability of modern processors state. The power of Co-RNG comes from its simplicity. No specialized hardware is required: two concurrent threads actively perform successive reads and writes to shared memory locations. Another thread collects the sequences of values read by these two threads and seeks to reconstruct the interleaving of read and write operations. The resulting (Markovian) interaction scheme is then used to produce random bits. This simplicity yields a transparent behavior. If the hardware exhibits enough entropy, then Co-RNG efficiently extracts random numbers from it. We successfully experimented Co-RNG on various idle as well as loaded platforms, from laptops and desktops featuring Intel Core processors, to servers with Intel Xeon and AMD Opteron. Co-RNG passes all state-of-the-art random number generator statistical test suites while being faster than current I/O sampling based methods by 2–3 orders of magnitude.

## Keywords

Distributed systems Shared memory Random number generation Information theory Asynchronous systems## References

- 1.DieHarder: A random number test suite. http://www.phy.duke.edu/~rgb/General/dieharder.php
- 2.ENT—a pseudorandom number sequence test program. http://fourmilab.ch/random
- 3.ID Quantique—quantum-safe crypto-photon counting—randomness. http://www.idquantique.com/
- 4.Tails bug report #7675. https://labs.riseup.net/code/issues/7675
- 5.Tor bug report #10402. https://trac.torproject.org/projects/tor/ticket/10402
- 6.Intel Digital Random Number Generator (DRNG) Software Implementation Guide. https://software.intel.com/sites/default/files/m/d/4/1/d/8/441_Intel_R__DRNG_Software_Implementation_Guide_final_Aug7.pdf (2012)
- 7.Aaronson, S.: Quantum randomness. Am. Sci.
**102**(4), 266 (2014). doi: 10.1511/2014.109.266 CrossRefGoogle Scholar - 8.Aaronson, S.: The quest for randomness. Am. Sci.
**102**(3), 170 (2014). doi: 10.1511/2014.108.170 CrossRefGoogle Scholar - 9.Abbes, S.: The information rate of asynchronous sources. In: Information and Communication Technologies, 2006. ICTTA ’06. 2nd, vol. 2, pp. 3463–3467. IEEE, Damascus, 24–28 Apr (2006). doi: 10.1109/ICTTA.2006.1684974
- 10.Agafin, S., Krasnopevtsev, A.: Memory access time as entropy source for RNG. In: Proceedings of the 7th International Conference on Security of Information and Networks, SIN ’14, pp. 176:176–176:179. ACM, New York, NY, USA (2014). doi: 10.1145/2659651.2659695
- 11.Alistarh, D., Censor-Hillel, K., Shavit, N.: Are lock-free concurrent algorithms practically wait-free? In: Symposium on Theory of Computing, STOC 2014, New York, NY, USA, May 31–June 03, 2014, pp. 714–723 (2014). doi: 10.1145/2591796.2591836
- 12.Alistarh, D., Sauerwald, T., Vojnovic, M.: Lock-free algorithms under stochastic schedulers. In: Proceedings of the 2015 ACM Symposium on Principles of Distributed Computing, PODC 2015, Donostia-San Sebastián, Spain, July 21–23, 2015, pp. 251–260 (2015). doi: 10.1145/2767386.2767430
- 13.Anthes, G.: The quest for randomness. Commun. ACM
**54**(4), 13–15 (2011). doi: 10.1145/1924421.1924427 CrossRefGoogle Scholar - 14.Aspnes, J.: Fast deterministic consensus in a noisy environment. J. Algorithms
**45**(1), 16–39 (2002). doi: 10.1016/S0196-6774(02)00220-1 MathSciNetCrossRefzbMATHGoogle Scholar - 15.Barker, E., Kelsley, J.: Recommendation for random bit generator (rbg) constructions. SP 800-90C (2012)Google Scholar
- 16.Barker, E., Kelsley, J.: Recommendation for random number generation using deterministic random bit generators. SP 800-90A (2012)Google Scholar
- 17.Barker, E., Kelsley, J.: Recommendation for the entropy sources used for random bit generation. SP 800-90B (2012)Google Scholar
- 18.Bhat, B., Mueller, F.: Making DRAM refresh predictable. Real Time Syst.
**47**(5), 430–453 (2011). doi: 10.1007/s11241-011-9129-6 CrossRefGoogle Scholar - 19.Blanchard, P., Guerraoui, R., Stainer, J., Zablotchi, I.: The disclosure power of shared objects. In: Networked Systems—5th International Conference, NETYS 2017, Marrakech, Morocco, May 17–19, 2017, Proceedings, pp. 222–227 (2017). doi: 10.1007/978-3-319-59647-1_17
- 20.Colesa, A., Tudoran, R., Banescu, S.: Software random number generation based on race conditions. In: SYNASC 2008, 10th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, Timisoara, Romania, 26–29 September 2008, pp. 439–444 (2008). doi: 10.1109/SYNASC.2008.36
- 21.Davis, D., Ihaka, R., Fenstermacher, P.: Cryptographic randomness from air turbulence in disk drives. In: Desmedt, Y. (ed.) Advances in Cryptology—CRYPTO ’94, Lecture Notes in Computer, vol. 839, pp. 114–120. Springer, Berlin (1994). doi: 10.1007/3-540-48658-5_13 Google Scholar
- 22.Devietti, J., Lucia, B., Ceze, L., Oskin, M.: DMP: Deterministic shared memory multiprocessing. In: Proceedings of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XIV, pp. 85–96. ACM, New York, NY, USA (2009). doi: 10.1145/1508244.1508255
- 23.Diffie, W., Van Oorschot, P., Wiener, M.: Authentication and authenticated key exchanges. Des. Codes Cryptogr.
**2**(2), 107–125 (1992). doi: 10.1007/BF00124891 MathSciNetCrossRefGoogle Scholar - 24.Dingledine, R., Mathewson, N., Syverson, P.: Tor: the second-generation Onion Router. https://svn.torproject.org/svn/projects/design-paper/tor-design.html
- 25.Fidge, C.J.: Timestamps in message passing systems that preserve the partial ordering. In: Australian Computer Science Conference (1988)Google Scholar
- 26.Fidge, C.J.: A limitation of vector timestamps for reconstructing distributed computations. Inf. Process. Lett.
**68**(2), 87–91 (1998). doi: 10.1016/S0020-0190(98)00143-4 CrossRefzbMATHGoogle Scholar - 27.Fischer, M.J., Michael, A.: Sacrificing serializability to attain high availability of data. In: Proceedings of the ACM Symposium on Principles of Database Systems, March 29–31, 1982, Los Angeles, California, USA, pp. 70–75 (1982). doi: 10.1145/588111.588124
- 28.Goubault, E.: Geometry and concurrency: a user’s guide. Math. Struct. Comput. Sci.
**10**(4), 411–425 (2000). http://journals.cambridge.org/action/displayAbstract?aid=54593 - 29.Gutterman, Z., Pinkas, B., Reinman, T.: Analysis of the Linux Random Number Generator. In: 2006 IEEE Symposium on Security and Privacy (S&P 2006), 21–24 May 2006, Berkeley, California, USA, pp. 371–385 (2006). doi: 10.1109/SP.2006.5
- 30.Heninger, N., Durumeric, Z., Wustrow, E., Halderman, J.A.: Mining your Ps and Qs: detection of widespread weak keys in network devices. In: Presented as part of the 21st USENIX Security Symposium (USENIX Security 12), pp. 205–220. USENIX, Bellevue, WA (2012). https://www.usenix.org/conference/usenixsecurity12/technical-sessions/presentation/heninger
- 31.Herlihy, M., Kozlov, D.N., Rajsbaum, S.: Distributed Computing Through Combinatorial Topology. Morgan Kaufmann (2013). https://store.elsevier.com/product.jsp?isbn=9780124045781
- 32.Herlihy, M., Wing, J.M.: Linearizability: a correctness condition for concurrent objects. ACM Trans. Program. Lang. Syst.
**12**(3), 463–492 (1990). doi: 10.1145/78969.78972 CrossRefGoogle Scholar - 33.Jakobsson, M., Shriver, E., Hillyer, B.K., Juels, A.: A practical secure physical random bit generator. In: Fifth ACM Conference on Computer and Communications Security pp. 103–111 (1998)Google Scholar
- 34.Lamport, L.: Time, clocks, and the ordering of events in a distributed system. Commun. ACM
**21**(7), 558–565 (1978). doi: 10.1145/359545.359563 CrossRefzbMATHGoogle Scholar - 35.Lamport, L.: The mutual exclusion problem—part I: a theory of interprocess communication. J. ACM
**33**, 313–326 (1986)MathSciNetCrossRefzbMATHGoogle Scholar - 36.Lamport, L.: The mutual exclusion problem—part II: statement and solutions. J. ACM
**33**, 327–348 (1986)MathSciNetCrossRefzbMATHGoogle Scholar - 37.Lamport, L.: On interprocess communication. Distrib. Comput.
**1**, 86–101 (1986)CrossRefzbMATHGoogle Scholar - 38.Li, M., Vitányi, P.M.B.: An Introduction to Kolmogorov Complexity and Its Applications. Texts in Computer Science, 3rd edn. Springer, Berlin (2008)CrossRefGoogle Scholar
- 39.Lu, M., Fang, J.Z.: A solution of the cache ping-pong problem in multiprocessor systems. J. Parallel Distrib. Comput.
**16**(2), 158–171 (1992)MathSciNetCrossRefzbMATHGoogle Scholar - 40.Luby, M.G., Michael, L.: Pseudorandomness and Cryptographic Applications. Princeton University Press, Princeton (1994)zbMATHGoogle Scholar
- 41.Marandi, A., Leindecker, N.C., Vodopyanov, K.L., Byer, R.L.: All-optical quantum random bit generation from intrinsically binary phase of parametric oscillators. Opt. Express
**20**(17), 19,322–19,330 (2012). doi: 10.1364/OE.20.019322. http://www.opticsexpress.org/abstract.cfm?URI=oe-20-17-19322 - 42.Mezard, M., Montanari, A.: Information, Physics, and Computation. Oxford University Press Inc., New York (2009)CrossRefzbMATHGoogle Scholar
- 43.Müller, S.: Cpu time jitter based non-physical true random number generator. In: Ottawa Linux Symposium (2014)Google Scholar
- 44.Neumann, J.V.: Various techniques used in connection with random digits. Appl. Math. Ser.
**12**, 36–38 (1951)Google Scholar - 45.Nisan, N., Zuckerman, D.: Randomness is linear in space. J. Comput. Syst. Sci.
**52**(1), 43–52 (1996). doi: 10.1006/jcss.1996.0004 MathSciNetCrossRefzbMATHGoogle Scholar - 46.Potter, B., Wood, S.: Understanding and managing entropy usage. BlackHat, Las Vegas, Navada (2015)Google Scholar
- 47.Pratt, V.R.: Modeling concurrency with geometry. In: Conference Record of the Eighteenth Annual ACM Symposium on Principles of Programming Languages, Orlando, Florida, USA, January 21–23, 1991, pp. 311–322 (1991). doi: 10.1145/99583.99625
- 48.Raynal, M.: Concurrent Programming—Algorithms, Principles, and Foundations. Springer, Berlin (2013). doi: 10.1007/978-3-642-32027-9
- 49.Ruhkin, A., Soto, J., Nechvatal, J., Smid, M., Barker, E., Leigh, S., Levenson, M., Vangel, M., Banks, D., Heckert, A., Dray, J., Vol, S., Bassham III, L.E.: A statistical test suite for random and pseudorandom number generators for cryptographic applications. SP 800–22 Rev. 1a (2010)Google Scholar
- 50.Santha, M., Vazirani, U.V.: Generating quasi-random sequences from semi-random sources. J. Comput. Syst. Sci.
**33**(1), 75–87 (1986). doi: 10.1016/0022-0000(86)90044-9 CrossRefzbMATHGoogle Scholar - 51.Seznec, A., Sendrier, N.: HAVEGE: a user-level software heuristic for generating empirically strong random numbers. ACM Trans. Model. Comput. Simul.
**13**(4), 334–346 (2003). doi: 10.1145/945511.945516 CrossRefGoogle Scholar - 52.Shaltiel, R.: Recent developments in explicit constructions of extractors. Bull. Eur. Assoc. Theor. Comput. Sci. (EATCS)
**77**, 67–95 (2002)MathSciNetzbMATHGoogle Scholar - 53.Shannon, C.E.: A mathematical theory of communication. ACM SIGMOBILE Mob. Comput. Commun. Rev.
**5**(1), 3–55 (2001)MathSciNetCrossRefGoogle Scholar - 54.Zhou, H., Bruck, J.: Generalizing the Blum-Elias method for generating random bits from markov chains. In: Proceedings of IEEE International Symposium on Information Theory (ISIT) (2010)Google Scholar