Abstract
Although cybersecurity is a domain where data analysis and training are considered of the highest importance, few virtual environments for cybersecurity are specifically developed, while they are used efficiently in other domains to tackle these issues.
By taking into account cyber analysts’ practices and tasks, we have proposed the 3D Cyber Common Operational Picture model (3D CyberCOP), that aims at mediating analysts’ activities into a Collaborative Virtual Environment (CVE), in which users can perform alert analysis scenarios.
In this article, we present a usability study we have performed with non-expert users. We have proposed three virtual environments (a graph-based, an office-based, and the coupling of the two previous ones) in which users should perform a simplified alert analysis scenario based on the WannaCry ransomware. In these environments, users must switch between three views (alert, cyber and physical ones) which all contain different kinds of data sources. These data have to be used to perform the investigations and to determine if alerts are due to malicious activities or if they are caused by false positives.
We have had 30 users, with no prior knowledge in cybersecurity. They have performed very well at the cybersecurity task and they have managed to interact and navigate easily. SUS usability scores were above 70 for the three environments and users have shown a preference towards the coupled environment, which was considered more practical and useful.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brooke, J., et al.: SUS-a quick and dirty usability scale. Usability Eval. Ind. 189(194), 4–7 (1996)
Casarin, J., Pacqueriaud, N., Bechmann, D.: UMI3D: a Unity3D toolbox to support CSCW systems properties in generic 3D user interfaces. Proc. ACM Hum.-Comput. Interact. 2(CSCW), 29:1–29:20 (2018). https://doi.org/10.1145/3274298. http://doi.acm.org/10.1145/3274298
D’Amico, A., Buchanan, L., Kirkpatrick, D., Walczak, P.: Cyber operator perspectives on security visualization. In: Nicholson, D. (ed.) Advances in Human Factors in Cybersecurity, pp. 69–81. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41932-9_7
Evesti, A., Kanstrén, T., Frantti, T.: Cybersecurity situational awareness taxonomy. In: 2017 International Conference on Cyber Situational Awareness, Data Analytics And Assessment (Cyber SA), pp. 1–8, June 2017. https://doi.org/10.1109/CyberSA.2017.8073386
Gutzwiller, R.: Situation awareness in defensive cyberspace operations: an annotated bibliographic assessment through 2015. Technical report, NIWC Pacific San Diego United States (2019)
Hackathorn, R., Margolis, T.: Immersive analytics: building virtual data worlds for collaborative decision support. In: 2016 Workshop on Immersive Analytics (IA), pp. 44–47, March 2016. https://doi.org/10.1109/IMMERSIVE.2016.7932382
Hámornik, B.P., Krasznay, C.: Prerequisites of virtual teamwork in security operations centers: knowledge, skills, abilities and other characteristics. Acad. Appl. Res. Mil. Public Manag. Sci. 16, 73 (2017)
Kabil, A., Duval, T., Cuppens, N., Le Comte, G., Halgand, Y., Ponchel, C.: 3D CyberCOP: a collaborative platform for cybersecurity data analysis and training. In: Luo, Y. (ed.) CDVE 2018. LNCS, vol. 11151, pp. 176–183. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00560-3_24
Kabil, A., Duval, T., Cuppens, N., Le Comte, G., Halgand, Y., Ponchel, C.: From cyber security activities to collaborative virtual environments practices through the 3D CyberCOP platform. In: Ganapathy, V., Jaeger, T., Shyamasundar, R.K. (eds.) ICISS 2018. LNCS, vol. 11281, pp. 272–287. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-05171-6_14
Kabil, A., Duval, T., Cuppens, N., Le Comte, G., Halgand, Y., Ponchel, C.: Why should we use 3D collaborative virtual environments for cyber security? In: IEEE Fourth VR International Workshop on Collaborative Virtual Environments (IEEEVR 2018), Reutlingen, Germany, March 2018. https://hal.archives-ouvertes.fr/hal-01770064
McKenna, S., Staheli, D., Meyer, M.: Unlocking user-centered design methods for building cyber security visualizations. In: 2015 IEEE Symposium on Visualization for Cyber Security (VizSec), pp. 1–8. IEEE (2015)
Mohurle, S., Patil, M.: A brief study of wannacry threat: ransomware attack 2017. Int. J. Adv. Res. Comput. Sci. 8(5) (2017)
Pahi, T., Leitner, M., Skopik, F.: Data exploitation at large: your way to adequate cyber common operating pictures. In: Proceedings of the 16th European Conference on Cyber Warfare and Security, pp. 307–315 (2017)
Salzman, M.C., Dede, C., Loftin, R.B.: VR’s frames of reference: a visualization technique for mastering abstract multidimensional information. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 1999, pp. 489–495. ACM, New York (1999). https://doi.org/10.1145/302979.303141. http://doi.acm.org/10.1145/302979.303141
Sebok, A., Nystad, E., Droivoldsmo, A.: Improving safety and human performance in maintenance and outage planning through virtual reality-based training systems. In: Proceedings of the IEEE 7th Conference on Human Factors and Power Plants, p. 8, September 2002. https://doi.org/10.1109/HFPP.2002.1042867
Sethi, A., Wills, G.: Expert-interviews led analysis of EEVi - a model for effective visualization in cyber-security. In: 2017 IEEE Symposium on Visualization for Cyber Security (VizSec), pp. 1–8, October 2017. https://doi.org/10.1109/VIZSEC.2017.8062195
Staheli, D., et al.: Collaborative data analysis and discovery for cyber security. In: Twelfth Symposium on Usable Privacy and Security (SOUPS 2016). USENIX Association, Denver (2016). https://www.usenix.org/conference/soups2016/workshop-program/wsiw16/presentation/staheli
Sundaramurthy, S.C., McHugh, J., Ou, X., Wesch, M., Bardas, A.G., Rajagopalan, S.R.: Turning contradictions into innovations or: how we learned to stop whining and improve security operations. In: Twelfth Symposium on Usable Privacy and Security (SOUPS 2016), pp. 237–251. USENIX Association, Denver (2016). https://www.usenix.org/conference/soups2016/technical-sessions/presentation/sundaramurthy
Takahashi, T., Kadobayashi, Y., Nakao, K.: Toward global cybersecurity collaboration: cybersecurity operation activity model. In: Proceedings of ITU Kaleidoscope 2011: The Fully Networked Human? - Innovations for Future Networks and Services (K-2011), pp. 1–8, December 2011
Varga, M., Winkelholz, C., Träber-Burdin, S.: The application of visual analytics to cyber security (2017)
Zhang, S., Shi, R., Zhao, J.: A visualization system for multiple heterogeneous network security data and fusion analysis. KSII Trans. Internet Inf. Syst. 10(6) (2016)
Zhong, C., Yen, J., Liu, P., Erbacher, R.F., Garneau, C., Chen, B.: Studying analysts’ data triage operations in cyber defense situational analysis. In: Liu, P., Jajodia, S., Wang, C. (eds.) Theory and Models for Cyber Situation Awareness. LNCS, vol. 10030, pp. 128–169. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-61152-5_6
Zhong, Z., et al.: A user-centered multi-space collaborative visual analysis for cyber security. Chin. J. Electron. 27(5), 910–919 (2018). https://doi.org/10.1049/cje.2017.09.021
Acknowledgments
This work was supported by the Cyber CNI Chair of Institute Mines Télécom, which is held by IMT Atlantique and supported by Airbus Defence and Space, Amossys, BNP Paribas, EDF, Nokia and the Regional Council of Brittany. It has been acknowledged by the Center of excellence in Cyber Security.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Kabil, A., Duval, T., Cuppens, N. (2020). Alert Characterization by Non-expert Users in a Cybersecurity Virtual Environment: A Usability Study. In: De Paolis, L., Bourdot, P. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2020. Lecture Notes in Computer Science(), vol 12242. Springer, Cham. https://doi.org/10.1007/978-3-030-58465-8_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-58465-8_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58464-1
Online ISBN: 978-3-030-58465-8
eBook Packages: Computer ScienceComputer Science (R0)