Abstract
AI Forensics is a novel research field that aims at providing techniques, mechanisms, processes, and protocols for an AI failure investigation. In this paper, we pave the way towards further exploring a sub-domain of AI forensics, namely AI model forensics, and introduce AI model ballistics as a subfield inspired by forensic ballistics. AI model forensics studies the forensic investigation process, including where available evidence can be collected, as it applies to AI models and systems.
We elaborate on the background and nature of AI model development and deployment, and highlight the fact that these models can be replaced, trojanized, gradually poisoned, or fooled by adversarial input.
The relationships and the dependencies of our newly proposed sub-domain draws from past literature in software, cloud, and network forensics. Additionally, we share a use-case mini-study to explore the peculiarities of AI model forensics in an appropriate context. Blockchain is discussed as a possible solution for maintaining audit trails. Finally, the challenges of AI model forensics are discussed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alsulami, B., Dauber, E., Harang, R., Mancoridis, S., Greenstadt, R.: Source code authorship attribution using long short-term memory based networks. In: Foley, S.N., Gollmann, D., Snekkenes, E. (eds.) ESORICS 2017. LNCS, vol. 10492, pp. 65–82. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66402-6_6
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Mané, D.: Concrete Problems in AI Safety (2016)
Behzadan, V., Baggili, I.M.: Founding the domain of AI forensics. In: SafeAI@ AAAI, pp. 31–35 (2020)
Behzadan, V., Hsu, W.: Sequential triggers for watermarking of deep reinforcement learning policies. arXiv preprint arXiv:1906.01126 (2019)
Binkley, D.: Source code analysis: a road map. In: Future of Software Engineering (FOSE 2007), pp. 104–119 (2007). https://doi.org/10.1109/FOSE.2007.27
Chen, X., Liu, C., Li, B., Lu, K., Song, D.: Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526 (2017)
Datt, S.: Learning Network Forensics. Community Experience Distilled. Packt Publishing, Birmingham (2016)
Digital Forensics Market: Market Research Firm (2018). https://www.marketsandmarkets.com/Market-Reports/digital-forensics-market-230663168.html
Frantzeskou, G., MacDonell, S., Stamatatos, E.: Source code authorship analysis for supporting the cybercrime investigation process. In: Proceedings of the 1st International Conference on E-Business and Telecommunications Networks, pp. 85–92 (2004)
Grispos, G., Storer, T., Glisson, W.B.: Calm before the storm: the challenges of cloud computing in digital forensics. Int. J. Digit. Crime Forensics (IJDCF) 4(2), 28–48 (2012)
Herman, M., et al.: NIST cloud computing forensic science challenges. Technical report, National Institute of Standards and Technology (2020)
Jeong, D.: Artificial intelligence security threat, crime, and forensics: taxonomy and open issues. IEEE Access 8, 184560–184574 (2020)
Jiang, F., et al.: Artificial intelligence in healthcare: past, present and future. Stroke Vasc. Neurol. 2(4), 230–243 (2017). https://doi.org/10.1136/svn-2017-000101
Karpisek, F., Baggili, I., Breitinger, F.: Whatsapp network forensics: decrypting and understanding the whatsapp call signaling messages. Digit. Investig. 15, 110–118 (2015). https://doi.org/10.1016/j.diin.2015.09.002. https://www.sciencedirect.com/science/article/pii/S1742287615000985
Kent, K., Chevalier, S., Grance, T., Dang, H.: Guide to integrating forensic techniques into incident response. NIST Special Publication 800–86 10(14) (2006)
Konečný, J., McMahan, H.B., Yu, F.X., Richtárik, P., Suresh, A.T., Bacon, D.: Federated learning: strategies for improving communication efficiency. CoRR abs/1610.05492 (2016). http://arxiv.org/abs/1610.05492
Kurtukova, A., Romanov, A., Shelupanov, A.: Source code authorship identification using deep neural networks. Symmetry 12(12) (2020). https://doi.org/10.3390/sym12122044. https://www.mdpi.com/2073-8994/12/12/2044
Levinson, J., et al.: Towards fully autonomous driving: systems and algorithms. In: 2011 IEEE Intelligent Vehicles Symposium (IV), pp. 163–168 (2011). https://doi.org/10.1109/IVS.2011.5940562
Li, J.: Cyber security meets artificial intelligence: a survey. Front. Inf. Technol. Electron. Eng. 19(12), 1462–1474 (2018). https://doi.org/10.1631/FITEE.1800573
Li, Z., Hu, C., Zhang, Y., Guo, S.: How to prove your model belongs to you. In: Proceedings of the 35th Annual Computer Security Applications Conference (2019). https://doi.org/10.1145/3359789.3359801
MacDonell, S.G., Buckingham, D., Gray, A.R., Sallis, P.J.: Software forensics: extending authorship analysis techniques to computer programs. JL Inf. Sci. 13, 34–69 (2002)
Mell, P., Grance, T., et al.: The NIST definition of cloud computing. NIST Special Publication 800–145 (2011)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–33 (2015). https://doi.org/10.1038/nature14236
MOBILedit: Camera Ballistics. https://www.mobiledit.com/camera-ballistics
Mukkamala, S., Sung, A.H.: Identifying significant features for network forensic analysis using artificial intelligent techniques. Int. J. Digit. Evid. 1, 1–17 (2003)
Nassar, M., Itani, A., Karout, M., El Baba, M., Kaakaji, O.A.S.: Shoplifting smart stores using adversarial machine learning. In: 2019 IEEE/ACS 16th International Conference on Computer Systems and Applications (AICCSA), pp. 1–6. IEEE (2019)
Nassar, M., Salah, K., ur Rehman, M.H., Svetinovic, D.: Blockchain for explainable and trustworthy artificial intelligence. Wiley Interdiscip. Rev. Data Mining Knowl. Discov. 10(1), e1340 (2020)
NIST: Ballistics (2021). https://www.nist.gov/ballistics
Palmer, G.: A road map for digital forensic research. Technical report. DFRWS (DTRT0010-01) (2001)
PyTorch: PyTorch tutorials: saving and loading models (2017). https://pytorch.org/tutorials/beginner/saving_loading_models.html#saving-loading-model-for-inference
Ruan, K., Carthy, J., Kechadi, M.T., Baggili, I.: Cloud forensics definitions and critical criteria for cloud forensic capability: an overview of survey results. Digit. Investig. 10, 34–43 (2013)
Ruan, K., Carthy, J., Kechadi, T., Crosbie, M.: Cloud forensics. In: Peterson, G., Shenoi, S. (eds.) Advances in Digital Forensics VII (2011). https://doi.org/10.1007/978-3-642-24212-0_3
Sallis, P., Aakjaer, A., MacDonell, S.: Software forensics: old methods for a new science. In: Proceedings 1996 International Conference Software Engineering: Education and Practice, pp. 481–485. IEEE (1996)
Schneider, J., Breitinger, F.: AI forensics: did the artificial intelligence system do it? Why? (2020)
Shah, J.J., Malik, L.G.: Cloud forensics: issues and challenges. In: 6th International Conference on Emerging Trends in Engineering and Technology, pp. 138–139 (2013). https://doi.org/10.1109/ICETET.2013.44
Sikos, L.F.: Packet analysis for network forensics: a comprehensive survey. Forensic Sci. Int. Digit. Investig. 32, 200892 (2020). https://doi.org/10.1016/j.fsidi.2019.200892. https://www.sciencedirect.com/science/article/pii/S1742287619302002
Spafford, E.H., Weeber, S.A.: Software forensics: can we track code to its authors? Comput. Secur. 12(6), 585–595 (1993)
TensorFlow: TensorFlow core: save and load models (2021). https://www.tensorflow.org/tutorials/keras/save_and_load#save_the_entire_model
Tilstone, W., Tilstone, W., Savage, K., Clark, L.: Forensic Science: An Encyclopedia of History, Methods, and Techniques. ABC-CLIO (2006). https://books.google.com/books?id=zIRQOssWbaoC
Wang, Z., Liu, C., Cui, X.: Evilmodel: hiding malware inside of neural network models. arXiv preprint arXiv:2107.08590 (2021)
Zhang, J., et al.: Protecting intellectual property of deep neural networks with watermarking. In: Proceedings of the 2018 on Asia Conference on Computer and Communications Security, ASIACCS 2018, pp. 159–172. Association for Computing Machinery (2018). https://doi.org/10.1145/3196494.3196550
Acknowledgements
This material is based upon work supported by the National Science Foundation under Grant Number 1921813. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Edwards, T., McCullough, S., Nassar, M., Baggili, I. (2022). On Exploring the Sub-domain of Artificial Intelligence (AI) Model Forensics. In: Gladyshev, P., Goel, S., James, J., Markowsky, G., Johnson, D. (eds) Digital Forensics and Cyber Crime. ICDF2C 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 441. Springer, Cham. https://doi.org/10.1007/978-3-031-06365-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-06365-7_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06364-0
Online ISBN: 978-3-031-06365-7
eBook Packages: Computer ScienceComputer Science (R0)