Beyond Differential Privacy: Composition Theorems and Relational Logic for f-divergences between Probabilistic Programs

  • Gilles Barthe
  • Federico Olmedo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7966)


f-divergences form a class of measures of distance between probability distributions; they are widely used in areas such as information theory and signal processing. In this paper, we unveil a new connection between f-divergences and differential privacy, a confidentiality policy that provides strong privacy guarantees for private data-mining; specifically, we observe that the notion of α-distance used to characterize approximate differential privacy is an instance of the family of f-divergences. Building on this observation, we generalize to arbitrary f-divergences the sequential composition theorem of differential privacy. Then, we propose a relational program logic to prove upper bounds for the f-divergence between two probabilistic programs. Our results allow us to revisit the foundations of differential privacy under a new light, and to pave the way for applications that use different instances of f-divergences.


Relative Entropy Advance Encryption Standard Relational Logic Statistical Distance Probabilistic Program 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abate, A.: Approximation metrics based on probabilistic bisimulations for general state-space markov processes: a survey. Electronic Notes in Theoretical Computer Sciences (2012) (in print)Google Scholar
  2. 2.
    Ali, S.M., Silvey, S.D.: A general class of coefficients of divergence of one distribution from another. Journal of the Royal Statistical Society. Series B (Methodological) 28(1), 131–142 (1966)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Alvim, M.S., Andrés, M.E., Chatzikokolakis, K., Palamidessi, C.: On the relation between differential privacy and Quantitative Information Flow. In: Aceto, L., Henzinger, M., Sgall, J. (eds.) ICALP 2011, Part II. LNCS, vol. 6756, pp. 60–76. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  4. 4.
    Barthe, G., Grégoire, B., Heraud, S., Béguelin, S.Z.: Computer-aided security proofs for the working cryptographer. In: Rogaway, P. (ed.) CRYPTO 2011. LNCS, vol. 6841, pp. 71–90. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  5. 5.
    Barthe, G., Grégoire, B., Zanella-Béguelin, S.: Formal certification of code-based cryptographic proofs. In: 36th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2009, pp. 90–101. ACM, New York (2009)Google Scholar
  6. 6.
    Barthe, G., Köpf, B., Olmedo, F., Zanella-Béguelin, S.: Probabilistic relational reasoning for differential privacy. In: 39th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2012, pp. 97–110. ACM, New York (2012)Google Scholar
  7. 7.
    Chaudhuri, S., Gulwani, S., Lublinerman, R., Navidpour, S.: Proving programs robust. In: 19th ACM SIGSOFT Symposium on the Foundations of Software Engineering and 13rd European Software Engineering Conference, ESEC/FSE 2011, pp. 102–112. ACM, New York (2011)Google Scholar
  8. 8.
    Cortes, C., Mohri, M., Rastogi, A.: Lp distance and equivalence of probabilistic automata. Int. J. Found. Comput. Sci. 18(4), 761–779 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Cortes, C., Mohri, M., Rastogi, A., Riley, M.: On the computation of the relative entropy of probabilistic automata. Int. J. Found. Comput. Sci. 19(1), 219–242 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Csiszár, I.: Eine informationstheoretische ungleichung und ihre anwendung auf den beweis der ergodizitat von markoffschen ketten. Publications of the Mathematical Institute of the Hungarian Academy of Science 8, 85–108 (1963)zbMATHGoogle Scholar
  11. 11.
    Deng, Y., Du, W.: Logical, metric, and algorithmic characterisations of probabilistic bisimulation. Tech. Rep. CMU-CS-11-110, Carnegie Mellon University (March 2011)Google Scholar
  12. 12.
    Dwork, C.: Differential privacy. In: Bugliesi, M., Preneel, B., Sassone, V., Wegener, I. (eds.) ICALP 2006. LNCS, vol. 4052, pp. 1–12. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  13. 13.
    Ebanks, B., Sahoo, P., Sander, W.: Characterizations of Information Measures. World Scientific (1998)Google Scholar
  14. 14.
    Gaboardi, M., Haeberlen, A., Hsu, J., Narayan, A., Pierce, B.C.: Linear dependent types for differential privacy. In: 40th ACM SIGPLAN–SIGACT Symposium on Principles of Programming Languages, POPL 2013, pp. 357–370. ACM, New York (2013)CrossRefGoogle Scholar
  15. 15.
    Jonsson, B., Yi, W., Larsen, K.G.: Probabilistic extensions of process algebras. In: Bergstra, J., Ponse, A., Smolka, S. (eds.) Handbook of Process Algebra, pp. 685–710. Elsevier, Amsterdam (2001)CrossRefGoogle Scholar
  16. 16.
    McSherry, F.: Privacy integrated queries: an extensible platform for privacy-preserving data analysis. Commun. ACM 53(9), 89–97 (2010)CrossRefGoogle Scholar
  17. 17.
    McSherry, F.D.: Privacy integrated queries: an extensible platform for privacy-preserving data analysis. In: 35th SIGMOD International Conference on Management of Data, SIGMOD 2009, pp. 19–30. ACM, New York (2009)CrossRefGoogle Scholar
  18. 18.
    Pardo, M., Vajda, I.: About distances of discrete distributions satisfying the data processing theorem of information theory. IEEE Transactions on Information Theory 43(4), 1288–1293 (1997)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Pierce, B.C.: Differential privacy in the programming languages community. Invited Tutorial at DIMACS Workshop on Recent Work on Differential Privacy Across Computer Science (2012)Google Scholar
  20. 20.
    Di Pierro, A., Hankin, C., Wiklicky, H.: Measuring the confinement of probabilistic systems. Theor. Comput. Sci. 340(1), 3–56 (2005)zbMATHCrossRefGoogle Scholar
  21. 21.
    Reed, J., Pierce, B.C.: Distance makes the types grow stronger: a calculus for differential privacy. In: 15th ACM SIGPLAN International Conference on Functional programming, ICFP 2010, pp. 157–168. ACM, New York (2010)Google Scholar
  22. 22.
    Roy, I., Setty, S.T.V., Kilzer, A., Shmatikov, V., Witchel, E.: Airavat: security and privacy for MapReduce. In: 7th USENIX Conference on Networked Systems Design and Implementation, NSDI 2010, pp. 297–312. USENIX Association, Berkeley (2010)Google Scholar
  23. 23.
    Segala, R., Turrini, A.: Approximated computationally bounded simulation relations for probabilistic automata. In: 20th IEEE Computer Security Foundations Symposium, CSF 2007, pp. 140–156. IEEE Computer Society (2007)Google Scholar
  24. 24.
    Steinberger, J.: Improved security bounds for key-alternating ciphers via hellinger distance. Cryptology ePrint Archive, Report 2012/481 (2012),
  25. 25.
    Tracol, M., Desharnais, J., Zhioua, A.: Computing distances between probabilistic automata. In: Proceedings of QAPL. EPTCS, vol. 57, pp. 148–162 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Gilles Barthe
    • 1
  • Federico Olmedo
    • 1
  1. 1.IMDEA Software InstituteMadridSpain

Personalised recommendations