Skip to main content

Neural Trojans

  • Living reference work entry
  • First Online:
Encyclopedia of Cryptography, Security and Privacy

Introduction

While neural networks demonstrate exceptional capabilities in various tasks of machine learning nowadays, they are also becoming larger and deeper. As a result, the requirement of hardware, time, and data to train a network also increases dramatically. Under this scenario, machine learning as a service (MLaaS) becomes an increasingly popular business model. However, the training process in MLaaS is not transparent and may embed neural Trojans, i.e., hidden malicious functionalities, into the neural network. Many research papers have demonstrated the severity of this attack (Liu et al., 2017, 2018b,c; Chen et al., 2017; Liao et al., 2018; Clements and Lao, 2018a,b; Yang et al., 2019; Saha et al., 2019; Tan and Shokri, 2019; Gu et al., 2019; Li et al., 2018, 2019; Yao et al., 2019; Barni et al., 2019; Geigel, 2013; Rakin et al., 2019). The effect of neural Trojans in the neural network’s deployment is illustrated in Fig. 1. If the input is benign (i.e., without the Trojan...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

References

  • Baluta T, Shen S, Shinde S, Meel KS, Saxena P (2019) Quantitative verification of neural networks and its security applications. arXiv preprint arXiv:190610395

    Google Scholar 

  • Baracaldo N, Chen B, Ludwig H, Safavi A, Zhang R (2018) Detecting poisoning attacks on machine learning in iot environments. In: 2018 IEEE International Congress on Internet of Things (ICIOT). IEEE, pp 57–64

    Google Scholar 

  • Barni M, Kallas K, Tondi B (2019) A new backdoor attack in CNNS by training set corruption without label poisoning. arXiv preprint arXiv:190211237

    Google Scholar 

  • Chakarov A, Nori A, Rajamani S, Sen S, Vijaykeerthy D (2016) Debugging machine learning tasks. 1603.07292

    Google Scholar 

  • Chakraborty A, Jayasankaran NG, Liu Y, Rajendran J, Sinanoglu O, Srivastava A, Xie Y, Yasin M, Zuzak M (2019) Keynote: a disquisition on logic locking. IEEE Trans Comput Aided Des Integr Circ Syst 39:1952–1972

    Article  Google Scholar 

  • Chen X, Liu C, Li B, Lu K, Song D (2017) Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:171205526

    Google Scholar 

  • Chen B, Carvalho W, Baracaldo N, Ludwig H, Edwards B, Lee T, Molloy I, Srivastava B (2018) Detecting backdoor attacks on deep neural networks by activation clustering. arXiv preprint arXiv:181103728

    Google Scholar 

  • Chen H, Fu C, Zhao J, Koushanfar F (2019) Deepinspect: a black-box trojan detection and mitigation framework for deep neural networks. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. AAAI Press, pp 4658–4664

    Google Scholar 

  • Clements J, Lao Y (2018a) Backdoor attacks on neural network operations. In: 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP). IEEE, pp 1154–1158

    Google Scholar 

  • Clements J, Lao Y (2018b) Hardware trojan attacks on neural networks. arXiv preprint arXiv:180605768

    Google Scholar 

  • Doan BG, Abbasnejad E, Ranasinghe D (2019) Deepcleanse: a black-box input sanitizationframework against backdoor attacks on deepneural networks. arXiv preprint arXiv:190803369

    Google Scholar 

  • Gao Y, Kim Y, Doan BG, Zhang Z, Zhang G, Nepal S, Ranasinghe DC, Kim H (2019a) Design and evaluation of a multi-domain trojan detection method on deep neural networks. arXiv preprint arXiv:191110312

    Google Scholar 

  • Gao Y, Xu C, Wang D, Chen S, Ranasinghe DC, Nepal S (2019b) Strip: a defence against trojan attacks on deep neural networks. arXiv preprint arXiv:190206531

    Google Scholar 

  • Geigel A (2013) Neural network trojan. J Comput Secur 21(2):191–232

    Article  Google Scholar 

  • Gu T, Liu K, Dolan-Gavitt B, Garg S (2019) Badnets: evaluating backdooring attacks on deep neural networks. IEEE Access 7:47230–47244

    Article  Google Scholar 

  • Guo W, Wang L, Xing X, Du M, Song D (2019) Tabor: a highly accurate approach to inspecting and restoring trojan backdoors in ai systems. arXiv preprint arXiv:190801763

    Google Scholar 

  • He Z, Zhang T, Lee R (2019) Sensitive-sample fingerprinting of deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4729–4737

    Google Scholar 

  • Ji Y, Zhang X, Wang T (2017) Backdoor attacks against learning systems. In: 2017 IEEE Conference on Communications and Network Security (CNS). IEEE, pp 1–9

    Google Scholar 

  • Kolouri S, Saha A, Pirsiavash H, Hoffmann H (2019) Universal litmus patterns: revealing backdoor attacks in cnns. arXiv preprint arXiv:190610842

    Google Scholar 

  • Li W, Yu J, Ning X, Wang P, Wei Q, Wang Y, Yang H (2018) Hu-Fu: hardware and software collaborative attack framework against neural networks. In: 2018 IEEE Computer Society Annual Symposium on VLSI (ISVLSI). IEEE, pp 482–487

    Google Scholar 

  • Li S, Zhao BZH, Yu J, Xue M, Kaafar D, Zhu H (2019) Invisible backdoor attacks against deep neural networks. arXiv preprint arXiv:190902742

    Google Scholar 

  • Liao C, Zhong H, Squicciarini A, Zhu S, Miller D (2018) Backdoor embedding in convolutional neural network models via invisible perturbation. arXiv preprint arXiv:180810307

    Google Scholar 

  • Liu Y, Xie Y, Srivastava A (2017) Neural trojans. In: 2017 IEEE International Conference on Computer Design (ICCD). IEEE, pp 45–48

    Google Scholar 

  • Liu K, Dolan-Gavitt B, Garg S (2018a) Fine-pruning: defending against backdooring attacks on deep neural networks. In: International Symposium on Research in Attacks, Intrusions, and Defenses. Springer, pp 273–294

    Google Scholar 

  • Liu T, Wen W, Jin Y (2018b) Sin 2: stealth infection on neural network – a low-cost agile neural trojan attack methodology. In: 2018 IEEE International Symposium on Hardware Oriented Security and Trust (HOST). IEEE, pp 227–230

    Google Scholar 

  • Liu Y, Ma S, Aafer Y, Lee WC, Zhai J, Wang W, Zhang X (2018c) Trojaning attack on neural networks. In: Network and Distributed Systems Security (NDSS) Symposium 2018

    Google Scholar 

  • Liu Y, Lee WC, Tao G, Ma S, Aafer Y, Zhang X (2019) Abs: scanning neural networks for back-doors by artificial brain stimulation. In: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security. ACM, pp 1265–1282

    Google Scholar 

  • Nelson B, Barreno M, Jack Chi F, Joseph AD, Rubinstein BIP, Saini U, Sutton C, Tygar JD, Xia K (2009) Misleading learners: co-opting your spam filter, pp 17–51

    Google Scholar 

  • Rakin AS, He Z, Fan D (2019) TBT: targeted neural network attack with bit trojan. arXiv preprint arXiv:190905193

    Google Scholar 

  • Saha A, Subramanya A, Pirsiavash H (2019) Hidden trigger backdoor attacks. arXiv preprint arXiv:191000033

    Google Scholar 

  • Tan TJL, Shokri R (2019) Bypassing backdoor detection algorithms in deep learning. arXiv preprint arXiv:190513409

    Google Scholar 

  • Tran B, Li J, Madry A (2018) Spectral signatures in backdoor attacks. In: Advances in neural information processing systems, pp 8000–8010

    Google Scholar 

  • Udeshi S, Peng S, Woo G, Loh L, Rawshan L, Chattopadhyay S (2019) Model agnostic defence against backdoor attacks in machine learning. arXiv preprint arXiv:190802203

    Google Scholar 

  • Wang B, Yao Y, Shan S, Li H, Viswanath B, Zheng H, Zhao BY (2019) Neural cleanse: identifying and mitigating backdoor attacks in neural networks. In: Neural cleanse: identifying and mitigating backdoor attacks in neural networks

    Google Scholar 

  • Xiang Z, Miller DJ, Kesidis G (2019) Revealing backdoors, post-training, in DNN classifiers via novel inference on optimized perturbations inducing group misclassification. arXiv preprint arXiv:190810498

    Google Scholar 

  • Xu X, Wang Q, Li H, Borisov N, Gunter CA, Li B (2019) Detecting ai trojans using meta neural analysis. arXiv preprint arXiv:191003137

    Google Scholar 

  • Yang Z, Iyer N, Reimann J, Virani N (2019) Design of intentional backdoors in sequential models. arXiv preprint arXiv:190209972

    Google Scholar 

  • Yao Y, Li H, Zheng H, Zhao BY (2019) Latent backdoor attacks on deep neural networks. In: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pp 2041–2055

    Google Scholar 

  • Zhao B, Lao Y (2018) Resilience of pruned neural network against poisoning attack. In: 2018 13th International Conference on Malicious and Unwanted Software (MALWARE). IEEE, pp 78–83

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ankur Srivastava .

Editor information

Editors and Affiliations

Section Editor information

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Science+Business Media LLC

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Liu, Y. et al. (2021). Neural Trojans. In: Jajodia, S., Samarati, P., Yung, M. (eds) Encyclopedia of Cryptography, Security and Privacy. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-27739-9_1654-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-27739-9_1654-1

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-27739-9

  • Online ISBN: 978-3-642-27739-9

  • eBook Packages: Springer Reference Computer SciencesReference Module Computer Science and Engineering

Publish with us

Policies and ethics