Skip to main content
View expanded cover

Handbook of Big Data Privacy pp 371–383Cite as

RAT Hunter: Building Robust Models for Detecting Remote Access Trojans Based on Optimum Hybrid Features

Abstract

Nowadays, critical infrastructures are severely exposed to a wide range of malicious attempts. The malicious activities are becoming more sophisticated. They infect victim’s machines and seek to obtain information from users instead of doing a destructiveness to the machine. Remote Access Trojan (RAT) is a type of malware that tries to control the victim’s machine remotely without victim awareness. Accordingly, the number and harmful effect of RAT threats for information thieves has increased dramatically. In this chapter, we propose an optimum feature set for hunting RAT malware based on intelligence feature selection for machine learning classification tasks. For building a robust model, we collected real-world samples from well-known repositories like Virus Total and Virus Share. Afterwards, the behaviour of these types of malware are analyzed through a modified sandbox as a reverse engineering tool to extract features from dynamic and static analysis. With the feature selection process, we applied a two-layer feature selection algorithm like information gain and correlated feature selection for obtaining the optimum set of features to tackles RAT threats. By implementing different models like the generative and deep learning models, we obtained an accuracy rate of 99.75% and a false alarm rate of 0.3%.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-38557-6_18
  • Chapter length: 13 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   169.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-38557-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   219.99
Price excludes VAT (USA)
Hardcover Book
USD   219.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 18.1
Fig. 18.2
Fig. 18.3
Fig. 18.4
Fig. 18.5

References

  1. E.M. Dovom, A. Azmoodeh, A. Dehghantanha, D.E. Newton, R.M. Parizi, H. Karimipour, Fuzzy pattern tree for edge malware detection and categorization in IoT. J. Syst. Archit. 97, 1–7 (2019). https://doi.org/10.1016/j.sysarc.2019.01.017

    CrossRef  Google Scholar 

  2. J. Sakhnini, H. Karimipour, A. Dehghantanha, R.M. Parizi, G. Srivastava, Security aspects of internet of things aided smart grids: a bibliometric survey. Internet of Things 2019, 100111 (2019). https://doi.org/10.1016/j.iot.2019.100111

    CrossRef  Google Scholar 

  3. R.T. Shoniwa, G. George, Scanning tool for the detection of images embedded with malicious programs, in 2015 International Conference on Electrical, Electronics, Signals, Communication and Optimization (EESCO) (2015)

    Google Scholar 

  4. D. Kiwia, A. Dehghantanha, K.-K.R. Choo, J. Slaughter, A cyber kill chain based taxonomy of banking Trojans for evolutionary computational intelligence. J. Comput. Sci. 27, 394–409 (2018)

    CrossRef  Google Scholar 

  5. P.N. Bahrami, A. Dehghantanha, T. Dargahi, R.M. Parizi, K.R. Choo, H.H.S. Javadi, Cyber kill chain-based taxonomy of advanced persistent threat actors: analogy of tactics, techniques, and procedures. J. Inf. Process. Syst. 15, 865–889 (2019). https://doi.org/10.3745/JIPS.03.0126

    Google Scholar 

  6. S.C. Pallaprolu, J.M. Namayanja, V.P. Janeja, C.S. Adithya, Label propagation in big data to detect remote access Trojans, in 2016 IEEE International Conference on Big Data (Big Data) (IEEE, Piscataway, 2016), pp. 3539–3547

    CrossRef  Google Scholar 

  7. R. HosseiniNejad, H. HaddadPajouh, A. Dehghantanha, R. M. Parizi, A cyber kill chain based analysis of remote access Trojans, in Handbook of Big Data and IoT Security, ed. by A. Dehghantanha, K.-K.R. Choo (Springer, Cham, 2019), pp. 273–299. https://doi.org/10.1007/978-3-030-10543-3_12

    CrossRef  Google Scholar 

  8. T. Dargahi, A. Dehghantanha, P.N. Bahrami, M. Conti, G. Bianchi, L. Benedetto, A Cyber-Kill-Chain based taxonomy of crypto-ransomware features. J. Comput. Virol. Hack Tech. 15(4), 277–305 (2019). https://doi.org/10.1007/s11416-019-00338-7

    CrossRef  Google Scholar 

  9. S. Samuel, J. Graham, C. Hinds, Hunting Malware: An example using Gh0st, in 2017 International Conference on Computational Science and Computational Intelligence (CSCI) (IEEE, 2017 Dec), pp. 97–102

    Google Scholar 

  10. H. Mwiki, T. Dargahi, A. Dehghantanha, K.-K.R. Choo, Analysis and triage of advanced hacking groups targeting western countries critical national infrastructure: APT28, RED October, and Regin, in Critical Infrastructure Security and Resilience (Springer, Berlin, 2019), pp. 221–244

    Google Scholar 

  11. M. Rezaeirad, B. Farinholt, H. Dharmdasani, P. Pearce, K. Levchenko, D. McCoy, Schrödinger’s RAT: profiling the stakeholders in the remote access Trojan ecosystem, in 27th USENIX Security Symposium (USENIX Security 18) (2018), pp. 1043–1060

    Google Scholar 

  12. M. Mimura, Y. Otsubo, H. Tanaka, Evaluation of a brute forcing tool that extracts the rat from a malicious document file, in 2016 11th Asia Joint Conference on Information Security (AsiaJCIS) (IEEE, Piscataway, 2016), pp. 147–154

    Google Scholar 

  13. A. Pektaş, T. Acarman, Classification of malware families based on runtime behaviors. J. Inform. Secur. Appl. 37, 91–100 (2017)

    Google Scholar 

  14. S. Wu, S. Liu, W. Lin, X. Zhao, S. Chen, Detecting remote access Trojans through external control at area network borders, in Proceedings of the Symposium on Architectures for Networking and Communications Systems (IEEE Press, New York, 2017), pp. 131–141

    Google Scholar 

  15. H.H. Pajouh, G. Dastghaibyfard, S. Hashemi, Two-tier network anomaly detection model: a machine learning approach. J. Intell. Inf. Syst. 48(1), 61–74 (2017). https://doi.org/10.1007/s10844-015-0388-x

    CrossRef  Google Scholar 

  16. R.M. Parizi, A. Dehghantanha, K.-K.R. Choo, A. Singh, Empirical vulnerability analysis of automated smart contracts security testing on blockchains, in Proceedings of the 28th Annual International Conference on Computer Science and Software Engineering, CASCON ’18 (2018), pp. 103–113

    Google Scholar 

  17. D. Jiang, K. Omote, An approach to detect remote access Trojan in the early stage of communication, in 2015 IEEE 29th International Conference on Advanced Information Networking and Applications (IEEE, Piscataway, 2015), pp. 706–713

    Google Scholar 

  18. M. Yamada, M. Morinaga, Y. Unno, S. Torii, M. Takenaka, RAT-based malicious activities detection on enterprise internal networks, in 2015 10th International Conference for Internet Technology and Secured Transactions (ICITST) (IEEE, Piscataway, 2015), pp. 321–325

    Google Scholar 

  19. A.A. Awad, S.G. Sayed, S.A. Salem, A network-based framework for RAT-bots detection, in 2017 8th IEEE Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON) (IEEE, Piscataway, 2017), pp. 128–133

    Google Scholar 

  20. B. Kolosnjaji, A. Zarras, G. Webster, C. Eckert, Deep learning for classification of malware system call sequences, in Australasian Joint Conference on Artificial Intelligence (Springer, 2016), pp. 137–149

    Google Scholar 

  21. H. HaddadPajouh, A. Dehghantanha, R. Khayami, K.-K.R. Choo, A deep Recurrent Neural Network based approach for Internet of Things malware threat hunting. Futur. Gener. Comput. Syst. 85, 88–96 (2018)

    CrossRef  Google Scholar 

  22. P. Wang, Y.-S. Wang, Malware behavioural detection and vaccine development by using a support vector model classifier. J. Comput. Syst. Sci. 81(6), 1012–1026 (2015)

    CrossRef  Google Scholar 

  23. Z. Xu, S. Ray, P. Subramanyan, S. Malik, Malware detection using machine learning based analysis of virtual memory access patterns, in Proceedings of the Conference on Design, Automation and Test in Europe, European Design and Automation Association (2017), pp. 169–174

    Google Scholar 

  24. M. Sikorski, A. Honig, Practical Malware Analysis: The Hands-on Guide to Dissecting Malicious Software (No Starch Press, San Francisco, 2012)

    Google Scholar 

  25. J.M. Van Campenhout, On the peaking of the Hughes mean recognition accuracy: The resolution of an apparent paradox. IEEE Trans. Syst. Man Cybern. 8(5), 390–395 (1978 May)

    MathSciNet  CrossRef  Google Scholar 

  26. Y. Yang, J.O. Pedersen, A comparative study on feature selection in text categorization, in Proceedings of the International Conference on Machine Learning, vol. 97 (1997), p. 35

    Google Scholar 

  27. M.A. Hall, Correlation-based feature selection for machine learning. Ph.D Thesis, The University of Waikato, Hamilton, 1999

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering Department, Pishtazan Institute of Higher Education, Shiraz, Iran

    Mohammad Mehdi BehradFar

  2. Cyber Science Lab, University of Guelph, Guelph, ON, Canada

    Hamed HaddadPajouh, Ali Dehghantanha & Amin Azmoodeh

  3. School of Engineering, University of Guelph, Guelph, ON, Canada

    Hadis Karimipour

  4. College of Computer and Software Engineering, Kennesaw State University, Marietta, GA, USA

    Reza M. Parizi

  5. Department of Mathematics and Computer Science, Brandon University, Brandon, MB, Canada

    Gautam Srivastava

Authors
  1. Mohammad Mehdi BehradFar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hamed HaddadPajouh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ali Dehghantanha
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Amin Azmoodeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hadis Karimipour
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Reza M. Parizi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Gautam Srivastava
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hamed HaddadPajouh .

Editor information

Editors and Affiliations

  1. University of Texas at San Antonio, San Antonio, TX, USA

    Kim-Kwang Raymond Choo

  2. Cyber Science Lab, School of Computer Science, University of Guelph, Guelph, ON, Canada

    Ali Dehghantanha

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

BehradFar, M.M. et al. (2020). RAT Hunter: Building Robust Models for Detecting Remote Access Trojans Based on Optimum Hybrid Features. In: Choo, KK., Dehghantanha, A. (eds) Handbook of Big Data Privacy. Springer, Cham. https://doi.org/10.1007/978-3-030-38557-6_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-38557-6_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-38556-9

  • Online ISBN: 978-3-030-38557-6

  • eBook Packages: Computer ScienceComputer Science (R0)