Distributed Computing

, Volume 31, Issue 5, pp 389–417 | Cite as

The entropy of a distributed computation random number generation from memory interleaving

  • Karolos Antoniadis
  • Peva Blanchard
  • Rachid Guerraoui
  • Julien Stainer


We ask to what extent processes communicating through shared memory can extract randomness from their underlying scheduler, e.g., to generate random numbers for cryptographic applications. We introduce the quantitative notions of entropy rate and information capacity of a distributed algorithm. Whilst the entropy rate measures the Shannon information that may pass from a given scheduler to the processes executing the algorithm, the information capacity measures the optimal entropy rate over all possible schedulers. We present a general method for computing these quantities by classifying distributed algorithms according to their pattern of shared memory accesses. We then address the issue of effectively extracting, online, the information produced by the scheduler into a meaningful format at every process. We present Duez, an algorithm solving this problem with an optimal memory consumption. Putting these principles into practice, we introduce Co-RNG, a random number generator that leverages the unpredictability of modern processors state. The power of Co-RNG comes from its simplicity. No specialized hardware is required: two concurrent threads actively perform successive reads and writes to shared memory locations. Another thread collects the sequences of values read by these two threads and seeks to reconstruct the interleaving of read and write operations. The resulting (Markovian) interaction scheme is then used to produce random bits. This simplicity yields a transparent behavior. If the hardware exhibits enough entropy, then Co-RNG efficiently extracts random numbers from it. We successfully experimented Co-RNG on various idle as well as loaded platforms, from laptops and desktops featuring Intel Core processors, to servers with Intel Xeon and AMD Opteron. Co-RNG passes all state-of-the-art random number generator statistical test suites while being faster than current I/O sampling based methods by 2–3 orders of magnitude.


Distributed systems Shared memory Random number generation Information theory Asynchronous systems 


  1. 1.
    DieHarder: A random number test suite.
  2. 2.
    ENT—a pseudorandom number sequence test program.
  3. 3.
    ID Quantique—quantum-safe crypto-photon counting—randomness.
  4. 4.
  5. 5.
  6. 6.
    Intel Digital Random Number Generator (DRNG) Software Implementation Guide. (2012)
  7. 7.
    Aaronson, S.: Quantum randomness. Am. Sci. 102(4), 266 (2014). doi: 10.1511/2014.109.266 CrossRefGoogle Scholar
  8. 8.
    Aaronson, S.: The quest for randomness. Am. Sci. 102(3), 170 (2014). doi: 10.1511/2014.108.170 CrossRefGoogle Scholar
  9. 9.
    Abbes, S.: The information rate of asynchronous sources. In: Information and Communication Technologies, 2006. ICTTA ’06. 2nd, vol. 2, pp. 3463–3467. IEEE, Damascus, 24–28 Apr (2006). doi: 10.1109/ICTTA.2006.1684974
  10. 10.
    Agafin, S., Krasnopevtsev, A.: Memory access time as entropy source for RNG. In: Proceedings of the 7th International Conference on Security of Information and Networks, SIN ’14, pp. 176:176–176:179. ACM, New York, NY, USA (2014). doi: 10.1145/2659651.2659695
  11. 11.
    Alistarh, D., Censor-Hillel, K., Shavit, N.: Are lock-free concurrent algorithms practically wait-free? In: Symposium on Theory of Computing, STOC 2014, New York, NY, USA, May 31–June 03, 2014, pp. 714–723 (2014). doi: 10.1145/2591796.2591836
  12. 12.
    Alistarh, D., Sauerwald, T., Vojnovic, M.: Lock-free algorithms under stochastic schedulers. In: Proceedings of the 2015 ACM Symposium on Principles of Distributed Computing, PODC 2015, Donostia-San Sebastián, Spain, July 21–23, 2015, pp. 251–260 (2015). doi: 10.1145/2767386.2767430
  13. 13.
    Anthes, G.: The quest for randomness. Commun. ACM 54(4), 13–15 (2011). doi: 10.1145/1924421.1924427 CrossRefGoogle Scholar
  14. 14.
    Aspnes, J.: Fast deterministic consensus in a noisy environment. J. Algorithms 45(1), 16–39 (2002). doi: 10.1016/S0196-6774(02)00220-1 MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Barker, E., Kelsley, J.: Recommendation for random bit generator (rbg) constructions. SP 800-90C (2012)Google Scholar
  16. 16.
    Barker, E., Kelsley, J.: Recommendation for random number generation using deterministic random bit generators. SP 800-90A (2012)Google Scholar
  17. 17.
    Barker, E., Kelsley, J.: Recommendation for the entropy sources used for random bit generation. SP 800-90B (2012)Google Scholar
  18. 18.
    Bhat, B., Mueller, F.: Making DRAM refresh predictable. Real Time Syst. 47(5), 430–453 (2011). doi: 10.1007/s11241-011-9129-6 CrossRefGoogle Scholar
  19. 19.
    Blanchard, P., Guerraoui, R., Stainer, J., Zablotchi, I.: The disclosure power of shared objects. In: Networked Systems—5th International Conference, NETYS 2017, Marrakech, Morocco, May 17–19, 2017, Proceedings, pp. 222–227 (2017). doi: 10.1007/978-3-319-59647-1_17
  20. 20.
    Colesa, A., Tudoran, R., Banescu, S.: Software random number generation based on race conditions. In: SYNASC 2008, 10th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, Timisoara, Romania, 26–29 September 2008, pp. 439–444 (2008). doi: 10.1109/SYNASC.2008.36
  21. 21.
    Davis, D., Ihaka, R., Fenstermacher, P.: Cryptographic randomness from air turbulence in disk drives. In: Desmedt, Y. (ed.) Advances in Cryptology—CRYPTO ’94, Lecture Notes in Computer, vol. 839, pp. 114–120. Springer, Berlin (1994). doi: 10.1007/3-540-48658-5_13 Google Scholar
  22. 22.
    Devietti, J., Lucia, B., Ceze, L., Oskin, M.: DMP: Deterministic shared memory multiprocessing. In: Proceedings of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS XIV, pp. 85–96. ACM, New York, NY, USA (2009). doi: 10.1145/1508244.1508255
  23. 23.
    Diffie, W., Van Oorschot, P., Wiener, M.: Authentication and authenticated key exchanges. Des. Codes Cryptogr. 2(2), 107–125 (1992). doi: 10.1007/BF00124891 MathSciNetCrossRefGoogle Scholar
  24. 24.
    Dingledine, R., Mathewson, N., Syverson, P.: Tor: the second-generation Onion Router.
  25. 25.
    Fidge, C.J.: Timestamps in message passing systems that preserve the partial ordering. In: Australian Computer Science Conference (1988)Google Scholar
  26. 26.
    Fidge, C.J.: A limitation of vector timestamps for reconstructing distributed computations. Inf. Process. Lett. 68(2), 87–91 (1998). doi: 10.1016/S0020-0190(98)00143-4 CrossRefzbMATHGoogle Scholar
  27. 27.
    Fischer, M.J., Michael, A.: Sacrificing serializability to attain high availability of data. In: Proceedings of the ACM Symposium on Principles of Database Systems, March 29–31, 1982, Los Angeles, California, USA, pp. 70–75 (1982). doi: 10.1145/588111.588124
  28. 28.
    Goubault, E.: Geometry and concurrency: a user’s guide. Math. Struct. Comput. Sci. 10(4), 411–425 (2000).
  29. 29.
    Gutterman, Z., Pinkas, B., Reinman, T.: Analysis of the Linux Random Number Generator. In: 2006 IEEE Symposium on Security and Privacy (S&P 2006), 21–24 May 2006, Berkeley, California, USA, pp. 371–385 (2006). doi: 10.1109/SP.2006.5
  30. 30.
    Heninger, N., Durumeric, Z., Wustrow, E., Halderman, J.A.: Mining your Ps and Qs: detection of widespread weak keys in network devices. In: Presented as part of the 21st USENIX Security Symposium (USENIX Security 12), pp. 205–220. USENIX, Bellevue, WA (2012).
  31. 31.
    Herlihy, M., Kozlov, D.N., Rajsbaum, S.: Distributed Computing Through Combinatorial Topology. Morgan Kaufmann (2013).
  32. 32.
    Herlihy, M., Wing, J.M.: Linearizability: a correctness condition for concurrent objects. ACM Trans. Program. Lang. Syst. 12(3), 463–492 (1990). doi: 10.1145/78969.78972 CrossRefGoogle Scholar
  33. 33.
    Jakobsson, M., Shriver, E., Hillyer, B.K., Juels, A.: A practical secure physical random bit generator. In: Fifth ACM Conference on Computer and Communications Security pp. 103–111 (1998)Google Scholar
  34. 34.
    Lamport, L.: Time, clocks, and the ordering of events in a distributed system. Commun. ACM 21(7), 558–565 (1978). doi: 10.1145/359545.359563 CrossRefzbMATHGoogle Scholar
  35. 35.
    Lamport, L.: The mutual exclusion problem—part I: a theory of interprocess communication. J. ACM 33, 313–326 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Lamport, L.: The mutual exclusion problem—part II: statement and solutions. J. ACM 33, 327–348 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Lamport, L.: On interprocess communication. Distrib. Comput. 1, 86–101 (1986)CrossRefzbMATHGoogle Scholar
  38. 38.
    Li, M., Vitányi, P.M.B.: An Introduction to Kolmogorov Complexity and Its Applications. Texts in Computer Science, 3rd edn. Springer, Berlin (2008)CrossRefGoogle Scholar
  39. 39.
    Lu, M., Fang, J.Z.: A solution of the cache ping-pong problem in multiprocessor systems. J. Parallel Distrib. Comput. 16(2), 158–171 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Luby, M.G., Michael, L.: Pseudorandomness and Cryptographic Applications. Princeton University Press, Princeton (1994)zbMATHGoogle Scholar
  41. 41.
    Marandi, A., Leindecker, N.C., Vodopyanov, K.L., Byer, R.L.: All-optical quantum random bit generation from intrinsically binary phase of parametric oscillators. Opt. Express 20(17), 19,322–19,330 (2012). doi: 10.1364/OE.20.019322.
  42. 42.
    Mezard, M., Montanari, A.: Information, Physics, and Computation. Oxford University Press Inc., New York (2009)CrossRefzbMATHGoogle Scholar
  43. 43.
    Müller, S.: Cpu time jitter based non-physical true random number generator. In: Ottawa Linux Symposium (2014)Google Scholar
  44. 44.
    Neumann, J.V.: Various techniques used in connection with random digits. Appl. Math. Ser. 12, 36–38 (1951)Google Scholar
  45. 45.
    Nisan, N., Zuckerman, D.: Randomness is linear in space. J. Comput. Syst. Sci. 52(1), 43–52 (1996). doi: 10.1006/jcss.1996.0004 MathSciNetCrossRefzbMATHGoogle Scholar
  46. 46.
    Potter, B., Wood, S.: Understanding and managing entropy usage. BlackHat, Las Vegas, Navada (2015)Google Scholar
  47. 47.
    Pratt, V.R.: Modeling concurrency with geometry. In: Conference Record of the Eighteenth Annual ACM Symposium on Principles of Programming Languages, Orlando, Florida, USA, January 21–23, 1991, pp. 311–322 (1991). doi: 10.1145/99583.99625
  48. 48.
    Raynal, M.: Concurrent Programming—Algorithms, Principles, and Foundations. Springer, Berlin (2013). doi: 10.1007/978-3-642-32027-9
  49. 49.
    Ruhkin, A., Soto, J., Nechvatal, J., Smid, M., Barker, E., Leigh, S., Levenson, M., Vangel, M., Banks, D., Heckert, A., Dray, J., Vol, S., Bassham III, L.E.: A statistical test suite for random and pseudorandom number generators for cryptographic applications. SP 800–22 Rev. 1a (2010)Google Scholar
  50. 50.
    Santha, M., Vazirani, U.V.: Generating quasi-random sequences from semi-random sources. J. Comput. Syst. Sci. 33(1), 75–87 (1986). doi: 10.1016/0022-0000(86)90044-9 CrossRefzbMATHGoogle Scholar
  51. 51.
    Seznec, A., Sendrier, N.: HAVEGE: a user-level software heuristic for generating empirically strong random numbers. ACM Trans. Model. Comput. Simul. 13(4), 334–346 (2003). doi: 10.1145/945511.945516 CrossRefGoogle Scholar
  52. 52.
    Shaltiel, R.: Recent developments in explicit constructions of extractors. Bull. Eur. Assoc. Theor. Comput. Sci. (EATCS) 77, 67–95 (2002)MathSciNetzbMATHGoogle Scholar
  53. 53.
    Shannon, C.E.: A mathematical theory of communication. ACM SIGMOBILE Mob. Comput. Commun. Rev. 5(1), 3–55 (2001)MathSciNetCrossRefGoogle Scholar
  54. 54.
    Zhou, H., Bruck, J.: Generalizing the Blum-Elias method for generating random bits from markov chains. In: Proceedings of IEEE International Symposium on Information Theory (ISIT) (2010)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.Distributed Programming LaboratoryÉcole Polytechnique Fédérale de LausanneLausanneSwitzerland

Personalised recommendations