Abstract
In today’s connected world, privacy decision-making is crucial for people to maintain control over their personal information and effectively manage their privacy. However, people’s decisions on privacy are likely to be subject to bias and can lead to frustration and regret. Privacy strategies in Conversational AI can aim at debiasing peoples’ choices by drawing from dual-process theory and triggering a more rational thinking process. Previous research on evaluation measures for such strategies has focused on minimizing regret or aligning user behaviour with their attitudes. In this paper, we propose a subjective measure of uncertainty to evaluate the effectiveness of debiasing strategies in a Conversational AI privacy scenario. We investigate two different scales of uncertainty - an adapted privacy uncertainty scale consisting of four subscales and the PANAS-X scale on the affective state of fear. We find that only one of the adapted subscales and the scale on fear showed sufficient reliability and validity results. Moreover, we did not find differences in uncertainty between our tested strategies. Finally, we propose alternative measures to investigate uncertainty and evaluate privacy strategies that promote rational thinking in the future.
The International Audio Laboratories Erlangen are a joint institution of the Friedrich-Alexander-Universität Erlangen-Nürnberg and Fraunhofer IIS.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Acquisti, A., et al.: Nudges for privacy and security: understanding and assisting users’ choices online. ACM Comput. Surv. 50(3), 1–41 (2018). https://doi.org/10.1145/3054926
Acquisti, A., Grossklags, J.: Uncertainty, ambiguity and privacy. In: Workshop on the Economics of Information Security (2005)
Al-Natour, S., Cavusoglu, H., Benbasat, I., Aleem, U.: An empirical investigation of the antecedents and consequences of privacy uncertainty in the context of mobile apps. Inf. Syst. Res. 31(4), 1037–1063 (2020). https://doi.org/10.1287/isre.2020.0931
Almuhimedi, H., et al.: Your location has been shared 5,398 times! A field study on mobile app privacy nudging. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 787–796, CHI ’15, Association for Computing Machinery, New York, NY, USA (2015)
Bermejo Fernandez, C., Chatzopoulos, D., Papadopoulos, D., Hui, P.: This website uses nudging: MTurk workers’ behaviour on cookie consent notices. Proc. ACM Hum.-Comput. Interact. 5(CSCW2), 1–22 (2021)
Brüggemeier, B., Lalone, P.: Perceptions and reactions to conversational privacy initiated by a conversational user interface. Comput. Speech Lang. 71, 101269 (2022)
Buçinca, Z., Malaya, M.B., Gajos, K.Z.: To trust or to think: cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proc. ACM Hum.-Comput. Interact. 5(CSCW1), 1–21 (2021)
Büyükkurt, B.K., Büyükkurt, M.D.: An experimental study of the effectiveness of three debiasing techniques*. Decis. Sci. 22(1), 60–73 (1991). https://doi.org/10.1111/j.1540-5915.1991.tb01262.x
Ciuk, D., Troy, A., Jones, M.: Measuring emotion: self-reports vs. physiological indicators. SSRN Electron. J. (2015). https://doi.org/10.2139/ssrn.2595359
Croskerry, P.: A universal model of diagnostic reasoning. Acad. Med. 84(8), 1022–1028 (2009). https://doi.org/10.1097/ACM.0b013e3181ace703
Dash, G., Paul, J.: CB-SEM vs PLS-SEM methods for research in social sciences and technology forecasting. Technol. Forecast. Soc. Chang. 173, 121092 (2021). https://doi.org/10.1016/j.techfore.2021.121092
Díaz Ferreyra, N.E., Meis, R., Heisel, M.: Learning from online regrets: from deleted posts to risk awareness in social network sites. In: Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization, pp. 117–125. ACM, Larnaca Cyprus, June 2019. https://doi.org/10.1145/3314183.3323849
European Commission: Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (Text with EEA relevance) (2016)
Greis, M., Karolus, J., Schuff, H., Woźniak, P.W., Henze, N.: Detecting uncertain input using physiological sensing and behavioral measurements. In: Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia, pp. 299–304. ACM, Stuttgart Germany, November 2017. https://doi.org/10.1145/3152832.3152859
Hair Jr., J.F., Hult, G.T.M., Ringle, C.M., Sarstedt, M., Danks, N.P., Ray, S.: Partial Least Squares Structural Equation Modeling (PLS-SEM) Using R. CCB, Springer, Cham (2021). https://doi.org/10.1007/978-3-030-80519-7
Harkous, H., Fawaz, K., Shin, K.G., Aberer, K.: PriBots: conversational privacy with chatbots. In: Twelfth Symposium on Usable Privacy and Security (SOUPS 2016), USENIX Association, Denver, CO, June 2016
Kahneman, D.: Thinking, fast and slow. Farrar, Straus and Giroux, New York (2011)
Kline, R.B.: Principles and Practice of Structural Equation Modeling. Guilford publications, New York (2015)
Knight, F.H.: Risk, Uncertainty and Profit (1921)
Lambe, K.A., O’Reilly, G., Kelly, B.D., Curristan, S.: Dual-process cognitive interventions to enhance diagnostic reasoning: a systematic review. BMJ Qual. Saf. 25(10), 808–820 (2016)
Lau, J., Zimmerman, B., Schaub, F.: Alexa, are you listening? Privacy perceptions, concerns and privacy-seeking behaviors with smart speakers. Proc. ACM Hum.-Comput. Interact. 2(CSCW), 1–31 (2018)
Leschanowsky, A., Brüggemeier, B., Peters, N.: Design implications for human-machine interactions from a qualitative pilot study on privacy. In: Proceedings of the 2021 ISCA Symposium on Security and Privacy in Speech Communication, pp. 76–79, November 2021. https://doi.org/10.21437/SPSC.2021-16
Leschanowsky, A., Popp, B., Peters, N.: Adapting debiasing strategies for conversational AI. Zagreb, Croatia, p. 74 (2022)
Martin, S.: Measuring cognitive load and cognition: metrics for technology-enhanced learning. Educ. Res. Eval. 20(7–8), 592–621 (2014). https://doi.org/10.1080/13803611.2014.997140
Mirhoseini, M., Early, S., Hassanein, K.: All eyes on misinformation and social media consumption: a pupil dilation study. In: Davis, F.D., Riedl, R., vom Brocke, J., Leger, P.M., Randolph, A.B., Müller-Putz, G.R. (eds.) Information Systems and Neuroscience. NeuroIS 2022. LNISO, vol. 58, pp. 73–80. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-13064-9_7
Mushtaq, F., Bland, A.R., Schaefer, A.: Uncertainty and cognitive control. Front. Psychol. 2, 249 (2011). https://doi.org/10.3389/fpsyg.2011.00249
Neys, W.D.: Advancing theorizing about fast-and-slow thinking. Behav. Brain Sci. 1–68 (2022). https://doi.org/10.1017/S0140525X2200142X, publisher: Cambridge University Press
Pearman, S., Young, E., Cranor, L.F.: User-friendly yet rarely read: a case study on the redesign of an online HIPAA authorization. Proc. Priv. Enhanc. Technol. 2022(3), 558–581 (2022). https://doi.org/10.56553/popets-2022-0086
Popp, B., Lalone, P., Leschanowsky, A.: Chatbot language - crowdsource perceptions and reactions to dialogue systems to inform dialogue design decisions. J. Behav. Res. Methods 55, 1601–1623 (2022)
Schomakers, E.M., Lidynia, C., Müllmann, D., Ziefle, M.: Internet users’ perceptions of information sensitivity - insights from Germany. Int. J. Inf. Manag. 46, 142–150 (2019)
Smith, C., Ellsworth, P.: Patterns of cognitive appraisal in emotion. J. Pers. Soc. Psychol. 48, 813–38 (1985). https://doi.org/10.1037//0022-3514.48.4.813
Tversky, A., Kahneman, D.: Judgment under uncertainty: heuristics and biases. Science 185(4157), 1124–1131 (1974). https://doi.org/10.1126/science.185.4157.1124
Wang, Y., Leon, P.G., Acquisti, A., Cranor, L.F., Forget, A., Sadeh, N.: A field trial of privacy nudges for Facebook. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2367–2376. ACM, Toronto Ontario Canada, April 2014. https://doi.org/10.1145/2556288.2557413
Wang, Y., Leon, P.G., Scott, K., Chen, X., Acquisti, A., Cranor, L.F.: Privacy nudges for social media: an exploratory Facebook study. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 763–770, WWW ’13 Companion, Association for Computing Machinery, New York, NY, USA (2013)
Watson, D.B., Clark, L.A.: The PANAS-X manual for the positive and negative affect schedule (1994)
Acknowledgments
Our work is partially funded by the German Federal Ministry for Economic Affairs and Energy as part of their AI innovation initiative (funding code 01MK20011A).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix
Appendix
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Leschanowsky, A., Popp, B., Peters, N. (2023). Uncertain yet Rational - Uncertainty as an Evaluation Measure of Rational Privacy Decision-Making in Conversational AI. In: Salvendy, G., Wei, J. (eds) Design, Operation and Evaluation of Mobile Communications . HCII 2023. Lecture Notes in Computer Science, vol 14052. Springer, Cham. https://doi.org/10.1007/978-3-031-35921-7_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-35921-7_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-35920-0
Online ISBN: 978-3-031-35921-7
eBook Packages: Computer ScienceComputer Science (R0)