Cognitive Computation

, Volume 11, Issue 2, pp 241–261 | Cite as

A Framework for Privacy Quantification: Measuring the Impact of Privacy Techniques Through Mutual Information, Distance Mapping, and Machine Learning

  • Yoan MicheEmail author
  • Wei Ren
  • Ian Oliver
  • Silke Holtmanns
  • Amaury Lendasse


In this paper, we propose to investigate how the effects of privacy techniques can be practically assessed in the specific context of data anonymization, and present some possible tools for measuring the effects of such anonymization. We develop an approach using mutual information for measuring the information content in any dataset, including over non-Euclidean data spaces, by means of mapping non-Euclidean distances to a Euclidean space. We further evaluate the proposed approach over toy datasets composed of timestamped GPS traces, and attempt to quantify the information content loss created by three state-of-the-art anonymization approaches. The results allow for an objective quantification of the effects of the k-anonymity and differential privacy algorithms, and illustrate on the toy data used, that such privacy techniques have very non-linear effects on the information content of the data.


Distance mapping Non-Euclidean data Data privacy Privacy quantification Mutual information 


Funding Information

This work was supported by the research from SCOTT project. SCOTT ( has received funding from the Electronic Component Systems for European Leadership Joint Undertaking under grant agreement no. 737422. This Joint Undertaking receives support from the European Union’s Horizon 2020 research and innovation programme and Austria, Spain, Finland, Ireland, Sweden, Germany, Poland, Portugal, Netherlands, Belgium, and Norway.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflicts of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.


  1. 1.
    Abramowitz M. Handbook of mathematical functions, with formulas, graphs, and mathematical tables. New York: Dover Publications; 1974.Google Scholar
  2. 2.
    Asgarian E, Kahani M, Sharifi S. The impact of sentiment features on the sentiment polarity classification in Persian reviews. Cogn Comput 2018;10(1):117–35. 00001.CrossRefGoogle Scholar
  3. 3.
    Auer P, Burgsteiner H, Maass W. A learning rule for very simple universal approximators consisting of a single layer of perceptrons. Neural Netw 2008;21(5):786–95.CrossRefGoogle Scholar
  4. 4.
    Belghazi M I, Baratin A, Rajeswar S, Ozair S, Bengio Y, Courville A, Hjelm RD. 2018. MINE: mutual information neural estimation. arXiv:1801.04062 [cs, stat]. 00003.
  5. 5.
    Cambria E, Huang G-B, Kasun L L C, Zhou H, Vong C M, Lin J, Yin J, Cai Z, Liu Q, Li K, et al. Extreme learning machines [trends & controversies]. IEEE Intell Syst 2013;28(6):30–59.CrossRefGoogle Scholar
  6. 6.
    European Commission. 2012. European Commission’s press release announcing the proposed comprehensive reform of data protection rules, 25 January.Google Scholar
  7. 7.
    Cover TM, Thomas JA. Elements of information theory (Wiley series in telecommunications and signal processing). New York: Wiley-Interscience; 2006.Google Scholar
  8. 8.
    Cybenko G. Approximation by superpositions of a sigmoidal function. Math Control Signals Syst (MCSS) 1989; 2(4):303–14.CrossRefGoogle Scholar
  9. 9.
    Dashtipour K, Poria S, Hussain A, Cambria E, Hawalah AY A, Gelbukh A, Zhou Q. Multilingual sentiment analysis: state of the art and independent comparison of techniques. Cogn Comput 2016;8(4): 757–71. 00025.CrossRefGoogle Scholar
  10. 10.
    Ding S, Zhao H, Zhang Y, Xu X, Nie R. Extreme learning machine: algorithm, theory and applications. Artif Intell Rev 2015;44(1):103–15.CrossRefGoogle Scholar
  11. 11.
    Dwork C. Differential privacy. Berlin: Springer; 2006, pp. 1–12.Google Scholar
  12. 12.
    Dwork C. Differential privacy: a survey of results. Theory and applications of models of computation, volume 4978 of Lecture Notes in Computer Science. Berlin: Springer; 2008. p. 1–19.Google Scholar
  13. 13.
    Dwork C, McSherry F, Nissim K, Smith A. Calibrating noise to sensitivity in private data analysis. Berlin: Springer; 2006, pp. 265–84.Google Scholar
  14. 14.
    EU. 2000. 2000/520/EC: Commission Decision of 26 July 2000 pursuant to Directive 95/46/EC of the European Parliament and of the Council on the adequacy of the protection provided by the safe harbour privacy principles and related frequently asked questions issued by the US Department of Commerce (notified under document number C(2000) 2441) (Text with EEA relevance.)Google Scholar
  15. 15.
    François D. 2008. High-dimensional data analysis: optimal metrics and feature selection. VDM Verlag, 01.Google Scholar
  16. 16.
    Goss R N. Information theory with applications (silviu guiaşu). SIAM Rev 1979;21(4):579–80.CrossRefGoogle Scholar
  17. 17.
    Hafiz M. A collection of privacy design patterns. Proceedings of the 2006 conference on pattern languages of programs, PLoP ’06. New York: ACM; 2006. p. 7:1–13.Google Scholar
  18. 18.
    Holmes C, Nemenman I. Progress in estimation of mutual information for real-valued data. Bulletin of the American Physical Society; 2018.Google Scholar
  19. 19.
    Huang G-B, Chen L, Siew C K, et al. Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 2006;17(4):879–92.CrossRefGoogle Scholar
  20. 20.
    Huang G-B, Zhu Q-Y, Siew C-K. Extreme learning machine: theory and applications. Neurocomputing 2006;70(1):489–501.CrossRefGoogle Scholar
  21. 21.
    The Information Commissioner’s Office (UK). Direct marketing: data protection act privacy and electronic communications regulations, 24 November 2013. Version 1.1.Google Scholar
  22. 22.
    Kraskov A, Stögbauer H, Grassberger P. Estimating mutual information. Phys Rev E 2004;69(6): 066138.CrossRefGoogle Scholar
  23. 23.
    Lauren P, Qu G, Yang J, Watta P, Huang G-B, Lendasse A. 2018. Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks. Cognit Comput. 1–14. 00000.Google Scholar
  24. 24.
    Li N, Li T. t-closeness: privacy beyond κ-anonymity and -diversity. Proceedings of IEEE 23rd international conference on data engineering (ICDE’07); 2007.Google Scholar
  25. 25.
    Ma Y, Peng H, Khan T, Cambria E, Hussain A. Sentic LSTM: a hybrid network for targeted aspect-based sentiment analysis. Cognit Comput. 2018;10(4):639–650. Scholar
  26. 26.
    Machanavajjhala A, Gehrke J, Kifer D, Venkitasubramaniam M. -diversity: privacy beyond κ-anonymity. 2013 IEEE 29th international conference on data engineering (ICDE); 2006. p. 24.Google Scholar
  27. 27.
    Mahmud M, Kaiser M S, Hussain A, Vassanelli S. 2017. Applications of deep learning and reinforcement learning to biological data. IEEE Trans Neural Netw Learn Syst. PP. 00004.Google Scholar
  28. 28.
    Miche Y, Oliver I, Holtmanns S, Akusok A, Lendasse A, Björk K-M. On mutual information over non-Euclidean Spaces, data mining and data privacy levels. Cham: Springer International Publishing; 2016, pp. 371–83.Google Scholar
  29. 29.
    Miche Y, Oliver I, Holtmanns S, Kalliola A, Akusok A, Lendasse A, Björk K-M. Data anonymization as a vector quantization problem: control over privacy for health data. Availability, reliability, and security in information systems, Lecture Notes in Computer Science. Cham: Springer; 2016. p. 193– 203.Google Scholar
  30. 30.
    Miche Y, Oliver I, Ren W, Holtmanns S, Akusok A, Lendasse A. Practical estimation of mutual information on non-Euclidean spaces. Machine learning and knowledge extraction. Cham: Springer; 2017. p. 123–36.Google Scholar
  31. 31.
    Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A. Op-elm: optimally pruned extreme learning machine. IEEE Trans Neural Netw 2010;21(1):158–62.CrossRefGoogle Scholar
  32. 32.
    Miche Y, Van Heeswijk M, Bas P, Simula O, Lendasse A. Trop-elm: a double-regularized elm using lars and tikhonov regularization. Neurocomputing 2011;74(16):2413–21.CrossRefGoogle Scholar
  33. 33.
    Molina D, LaTorre A, Herrera F. An insight into bio-inspired and evolutionary algorithms for global optimization: review, analysis, and lessons learnt over a decade of competitions. Cognit Comput 2018;10(4):517–544. Scholar
  34. 34.
    Nissenbaum H. A contextual approach to privacy online. Daedalus 2011;140(4):32–48.CrossRefGoogle Scholar
  35. 35.
    Oliver I. Privacy engineering: a data flow and ontological approach. CreateSpace Independent Publishing, July 2014. 978-1497569713.Google Scholar
  36. 36.
    Pál D, Póczos B, Szepesvári C. Estimation of rényi entropy and mutual information based on generalized nearest-neighbor graph. Advances in neural information processing systems; 2010. p. 1849–57.Google Scholar
  37. 37.
    Rao C R, Mitra S K. 1971. Generalized inverse of matrices and its applications.Google Scholar
  38. 38.
    Reed J, Pierce BC. Distance makes the types grow stronger: a calculus for differential privacy. ACM SIGPLAN international conference on functional programming (ICFP), Baltimore; 2010.Google Scholar
  39. 39.
    Savitha R, Suresh S, Kim H J. A meta-cognitive learning algorithm for an extreme learning machine classifier. Cogn Comput 2014;6(2):253–63. 00048.CrossRefGoogle Scholar
  40. 40.
    Schneier B. Architecture of privacy. IEEE Secur Priv 2009;7(1):88.CrossRefGoogle Scholar
  41. 41.
    Singh P K. Similar vague concepts selection using their euclidean distance at different granulation. Cogn Comput 2018;10(2):228–41. 00001.CrossRefGoogle Scholar
  42. 42.
    Sweeney L. κ-anonymity: a model for protecting privacy. Int J Uncertain Fuzziness Knowl-Based Syst 2002;10(5):557–70.CrossRefGoogle Scholar
  43. 43.
    Ustaran E, editor. European Privacy: Law and Practice for Data Protection Professionals. An IAPP Publication, 2012. 978-0-9795901-5-3.Google Scholar
  44. 44.
    Van Heeswijk M, Miche Y, Oja E, Lendasse A. Gpu-accelerated and parallelized elm ensembles for large-scale regression. Neurocomputing 2011;74(16):2430–7.CrossRefGoogle Scholar
  45. 45.
    Wang H, Zhang Y, Waytowich N R, Krusienski D J, Zhou G, Jin J, Wang X, Cichocki A. Discriminative feature extraction via multivariate linear regression for SSVEP-based BCI. IEEE Trans Neural Syst Rehabil Eng 2016;24(5):532–41.CrossRefGoogle Scholar
  46. 46.
    Wang R, Zhang Y, Zhang L. An adaptive neural network approach for operator functional state prediction using psychophysiological data. Integrated Computer Aided Eng 2015;23:81–97. 00006.CrossRefGoogle Scholar
  47. 47.
    Zeng D, Zhao F, Shen W, Ge S. Compressing and accelerating neural network for facial point localization. Cognit Comput 2018;10(2):359–67. 00001.CrossRefGoogle Scholar
  48. 48.
    Zhang Y, Wang Y, Jin J, Wang X. Sparse Bayesian learning for obtaining sparsity of EEG frequency bands based feature vectors in motor imagery classification. Int J Neural Syst. 2017;27(02):1650032. Scholar
  49. 49.
    Zhang Y, Zhou G, Jin J, Zhao Q, Wang X, Cichocki A. Sparse Bayesian classification of EEG for brain-computer interface. IEEE Trans Neural Netw Learn Syst 2015;27:1–1. 00058.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Nokia Bell LabsEspooFinland
  2. 2.Information and Logistics Technology DepartmentCollege of Technology, University of HoustonHoustonUSA

Personalised recommendations