In this paper, we propose to investigate how the effects of privacy techniques can be practically assessed in the specific context of data anonymization, and present some possible tools for measuring the effects of such anonymization. We develop an approach using mutual information for measuring the information content in any dataset, including over non-Euclidean data spaces, by means of mapping non-Euclidean distances to a Euclidean space. We further evaluate the proposed approach over toy datasets composed of timestamped GPS traces, and attempt to quantify the information content loss created by three state-of-the-art anonymization approaches. The results allow for an objective quantification of the effects of the k-anonymity and differential privacy algorithms, and illustrate on the toy data used, that such privacy techniques have very non-linear effects on the information content of the data.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Abramowitz M. Handbook of mathematical functions, with formulas, graphs, and mathematical tables. New York: Dover Publications; 1974.
Asgarian E, Kahani M, Sharifi S. The impact of sentiment features on the sentiment polarity classification in Persian reviews. Cogn Comput 2018;10(1):117–35. 00001.
Auer P, Burgsteiner H, Maass W. A learning rule for very simple universal approximators consisting of a single layer of perceptrons. Neural Netw 2008;21(5):786–95.
Belghazi M I, Baratin A, Rajeswar S, Ozair S, Bengio Y, Courville A, Hjelm RD. 2018. MINE: mutual information neural estimation. arXiv:1801.04062 [cs, stat]. 00003.
Cambria E, Huang G-B, Kasun L L C, Zhou H, Vong C M, Lin J, Yin J, Cai Z, Liu Q, Li K, et al. Extreme learning machines [trends & controversies]. IEEE Intell Syst 2013;28(6):30–59.
European Commission. 2012. European Commission’s press release announcing the proposed comprehensive reform of data protection rules, 25 January.
Cover TM, Thomas JA. Elements of information theory (Wiley series in telecommunications and signal processing). New York: Wiley-Interscience; 2006.
Cybenko G. Approximation by superpositions of a sigmoidal function. Math Control Signals Syst (MCSS) 1989; 2(4):303–14.
Dashtipour K, Poria S, Hussain A, Cambria E, Hawalah AY A, Gelbukh A, Zhou Q. Multilingual sentiment analysis: state of the art and independent comparison of techniques. Cogn Comput 2016;8(4): 757–71. 00025.
Ding S, Zhao H, Zhang Y, Xu X, Nie R. Extreme learning machine: algorithm, theory and applications. Artif Intell Rev 2015;44(1):103–15.
Dwork C. Differential privacy. Berlin: Springer; 2006, pp. 1–12.
Dwork C. Differential privacy: a survey of results. Theory and applications of models of computation, volume 4978 of Lecture Notes in Computer Science. Berlin: Springer; 2008. p. 1–19.
Dwork C, McSherry F, Nissim K, Smith A. Calibrating noise to sensitivity in private data analysis. Berlin: Springer; 2006, pp. 265–84.
EU. 2000. 2000/520/EC: Commission Decision of 26 July 2000 pursuant to Directive 95/46/EC of the European Parliament and of the Council on the adequacy of the protection provided by the safe harbour privacy principles and related frequently asked questions issued by the US Department of Commerce (notified under document number C(2000) 2441) (Text with EEA relevance.)
François D. 2008. High-dimensional data analysis: optimal metrics and feature selection. VDM Verlag, 01.
Goss R N. Information theory with applications (silviu guiaşu). SIAM Rev 1979;21(4):579–80.
Hafiz M. A collection of privacy design patterns. Proceedings of the 2006 conference on pattern languages of programs, PLoP ’06. New York: ACM; 2006. p. 7:1–13.
Holmes C, Nemenman I. Progress in estimation of mutual information for real-valued data. Bulletin of the American Physical Society; 2018.
Huang G-B, Chen L, Siew C K, et al. Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 2006;17(4):879–92.
Huang G-B, Zhu Q-Y, Siew C-K. Extreme learning machine: theory and applications. Neurocomputing 2006;70(1):489–501.
The Information Commissioner’s Office (UK). Direct marketing: data protection act privacy and electronic communications regulations, 24 November 2013. Version 1.1.
Kraskov A, Stögbauer H, Grassberger P. Estimating mutual information. Phys Rev E 2004;69(6): 066138.
Lauren P, Qu G, Yang J, Watta P, Huang G-B, Lendasse A. 2018. Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks. Cognit Comput. 1–14. 00000.
Li N, Li T. t-closeness: privacy beyond κ-anonymity and ℓ-diversity. Proceedings of IEEE 23rd international conference on data engineering (ICDE’07); 2007.
Ma Y, Peng H, Khan T, Cambria E, Hussain A. Sentic LSTM: a hybrid network for targeted aspect-based sentiment analysis. Cognit Comput. 2018;10(4):639–650. https://doi.org/10.1007/s12559-018-9549-x.
Machanavajjhala A, Gehrke J, Kifer D, Venkitasubramaniam M. ℓ-diversity: privacy beyond κ-anonymity. 2013 IEEE 29th international conference on data engineering (ICDE); 2006. p. 24.
Mahmud M, Kaiser M S, Hussain A, Vassanelli S. 2017. Applications of deep learning and reinforcement learning to biological data. IEEE Trans Neural Netw Learn Syst. PP. 00004.
Miche Y, Oliver I, Holtmanns S, Akusok A, Lendasse A, Björk K-M. On mutual information over non-Euclidean Spaces, data mining and data privacy levels. Cham: Springer International Publishing; 2016, pp. 371–83.
Miche Y, Oliver I, Holtmanns S, Kalliola A, Akusok A, Lendasse A, Björk K-M. Data anonymization as a vector quantization problem: control over privacy for health data. Availability, reliability, and security in information systems, Lecture Notes in Computer Science. Cham: Springer; 2016. p. 193– 203.
Miche Y, Oliver I, Ren W, Holtmanns S, Akusok A, Lendasse A. Practical estimation of mutual information on non-Euclidean spaces. Machine learning and knowledge extraction. Cham: Springer; 2017. p. 123–36.
Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A. Op-elm: optimally pruned extreme learning machine. IEEE Trans Neural Netw 2010;21(1):158–62.
Miche Y, Van Heeswijk M, Bas P, Simula O, Lendasse A. Trop-elm: a double-regularized elm using lars and tikhonov regularization. Neurocomputing 2011;74(16):2413–21.
Molina D, LaTorre A, Herrera F. An insight into bio-inspired and evolutionary algorithms for global optimization: review, analysis, and lessons learnt over a decade of competitions. Cognit Comput 2018;10(4):517–544. https://doi.org/10.1007/s12559-018-9554-0.
Nissenbaum H. A contextual approach to privacy online. Daedalus 2011;140(4):32–48.
Oliver I. Privacy engineering: a data flow and ontological approach. CreateSpace Independent Publishing, July 2014. 978-1497569713.
Pál D, Póczos B, Szepesvári C. Estimation of rényi entropy and mutual information based on generalized nearest-neighbor graph. Advances in neural information processing systems; 2010. p. 1849–57.
Rao C R, Mitra S K. 1971. Generalized inverse of matrices and its applications.
Reed J, Pierce BC. Distance makes the types grow stronger: a calculus for differential privacy. ACM SIGPLAN international conference on functional programming (ICFP), Baltimore; 2010.
Savitha R, Suresh S, Kim H J. A meta-cognitive learning algorithm for an extreme learning machine classifier. Cogn Comput 2014;6(2):253–63. 00048.
Schneier B. Architecture of privacy. IEEE Secur Priv 2009;7(1):88.
Singh P K. Similar vague concepts selection using their euclidean distance at different granulation. Cogn Comput 2018;10(2):228–41. 00001.
Sweeney L. κ-anonymity: a model for protecting privacy. Int J Uncertain Fuzziness Knowl-Based Syst 2002;10(5):557–70.
Ustaran E, editor. European Privacy: Law and Practice for Data Protection Professionals. An IAPP Publication, 2012. 978-0-9795901-5-3.
Van Heeswijk M, Miche Y, Oja E, Lendasse A. Gpu-accelerated and parallelized elm ensembles for large-scale regression. Neurocomputing 2011;74(16):2430–7.
Wang H, Zhang Y, Waytowich N R, Krusienski D J, Zhou G, Jin J, Wang X, Cichocki A. Discriminative feature extraction via multivariate linear regression for SSVEP-based BCI. IEEE Trans Neural Syst Rehabil Eng 2016;24(5):532–41.
Wang R, Zhang Y, Zhang L. An adaptive neural network approach for operator functional state prediction using psychophysiological data. Integrated Computer Aided Eng 2015;23:81–97. 00006.
Zeng D, Zhao F, Shen W, Ge S. Compressing and accelerating neural network for facial point localization. Cognit Comput 2018;10(2):359–67. 00001.
Zhang Y, Wang Y, Jin J, Wang X. Sparse Bayesian learning for obtaining sparsity of EEG frequency bands based feature vectors in motor imagery classification. Int J Neural Syst. 2017;27(02):1650032. https://doi.org/10.1142/S0129065716500325.
Zhang Y, Zhou G, Jin J, Zhao Q, Wang X, Cichocki A. Sparse Bayesian classification of EEG for brain-computer interface. IEEE Trans Neural Netw Learn Syst 2015;27:1–1. 00058.
This work was supported by the research from SCOTT project. SCOTT (http://www.scott-project.eu) has received funding from the Electronic Component Systems for European Leadership Joint Undertaking under grant agreement no. 737422. This Joint Undertaking receives support from the European Union’s Horizon 2020 research and innovation programme and Austria, Spain, Finland, Ireland, Sweden, Germany, Poland, Portugal, Netherlands, Belgium, and Norway.
Conflict of Interest
The authors declare that they have no conflicts of interest.
This article does not contain any studies with human participants or animals performed by any of the authors.
About this article
Cite this article
Miche, Y., Ren, W., Oliver, I. et al. A Framework for Privacy Quantification: Measuring the Impact of Privacy Techniques Through Mutual Information, Distance Mapping, and Machine Learning. Cogn Comput 11, 241–261 (2019). https://doi.org/10.1007/s12559-018-9604-7
- Distance mapping
- Non-Euclidean data
- Data privacy
- Privacy quantification
- Mutual information