Skip to main content

MORTON: Detection of Malicious Routines in Large-Scale DNS Traffic

Part of the Lecture Notes in Computer Science book series (LNSC,volume 12972)

Abstract

We present MORTON, a method that identifies compromised devices in enterprise networks based on the existence of routine DNS communication between devices and disreputable host names. With its compact representation of the input data and use of efficient signal processing and a neural network for classification, MORTON is designed to be accurate, robust, and scalable. We evaluate MORTON using a large dataset of corporate DNS logs and compare it with two recently proposed beaconing detection methods aimed at detecting malware communication. The results demonstrate that while MORTON ’s accuracy in a synthetic experiment is comparable to that of the other methods, it outperforms those methods in terms of its ability to detect sophisticated bot communication techniques, such as multistage channels. Additionally, MORTON was the most efficient method, running at least 13 times faster than the other methods on large-scale datasets, thus reducing the time to detection. In a real-world evaluation, which includes previously unreported threats, MORTON and the two compared methods were deployed to monitor the (unlabeled) DNS traffic of two global enterprises for a week-long period; this evaluation demonstrates the effectiveness of MORTON in real-world scenarios where it achieved the highest F1-score.

Keywords

  • DNS
  • PSD
  • Neural networks
  • Botnet

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-88418-5_35
  • Chapter length: 21 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-88418-5
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.

Notes

  1. 1.

    https://aws.amazon.com/ec2/instance-types/c5/.

References

  1. Stefana Gal -Software Engineer, Bitdefender ATD Team: Who iserik: A resurface of an advanced persistent adware? https://www.bitdefender.com/files/News/CaseStudies/study/284/Bitdefender-WhitePaper-Erik-CREA3910-en-EN-GenericUse.pdf

  2. Agency, N.S.: Adopting Encrypted DNS in Enterprise Environments. https://media.defense.gov/2021/Jan/14/2002564889/-1/-1/0/CSI_ADOPTING_ENCRYPTED_DNS_U_OO_102904_21.PDF (2021)

  3. Alina, O., Li, Z., Norris, R., Bowers, K.: MADE: security analytics for enterprise threat detection. In: Proceedings of the 34th Annual Computer Security Applications Conference, pp. 124–136. ACM (2018)

    Google Scholar 

  4. Meshkov, A.: AdGuard Research: Fake ad blockers 2: Now with cookies and ad fraud. https://adguard.com/en/blog/fake-ad-blockers-part-2.html

  5. Axelsson, S.: The base-rate fallacy and the difficulty of intrusion detection. ACM Trans. Inf. Syst. Secur. (TISSEC) 3(3), 186–205 (2000)

    CrossRef  Google Scholar 

  6. Bilge, L., Balzarotti, D., Robertson, W., Kirda, E., Kruegel, C.: Disclosure: detecting botnet command and control servers through large-scale netflow analysis. In: Proceedings of the 28th Annual Computer Security Applications Conference, pp. 129–138 (2012)

    Google Scholar 

  7. Cobalt Strike.com: Cobalt strike release notes. https://www.cobaltstrike.com/releasenotes.txt

  8. Elfeky, M.G., Aref, W.G., Elmagarmid, A.K.: WARP: time warping for periodicity detection. In: Fifth IEEE International Conference on Data Mining (ICDM 2005), p. 8. IEEE (2005)

    Google Scholar 

  9. FireEye: Highly Evasive Attacker Leverages SolarWinds Supply Chain to Compromise Multiple Global Victims With SUNBURST Backdoor (2020). https://www.fireeye.com/blog/threat-research/2020/12/evasive-attacker-leverages-solarwinds-supply-chain-compromises-with-sunburst-backdoor.html

  10. Gao, H., et al.: An empirical reexamination of global DNs behavior. In: Proceedings of the ACM SIGCOMM 2013 Conference on SIGCOMM, pp. 267–278 (2013)

    Google Scholar 

  11. Haffey, M., Arlitt, M., Williamson, C.: Modeling, analysis, and characterization of periodic traffic on a campus edge network. In: 2018 IEEE 26th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), pp. 170–182. IEEE (2018)

    Google Scholar 

  12. Holz, T., Gorecki, C., Rieck, K., Freiling, F.C.: Measuring and detecting fast-flux service networks. In: NDSS (2008)

    Google Scholar 

  13. Hu, X., et al.: BAYWATCH: robust beaconing detection to identify infected hosts in large-scale enterprise networks. In: 2016 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 479–490. IEEE (2016)

    Google Scholar 

  14. Hubballi, N., Goyal, D.: FlowSummary: summarizing network flows for communication periodicity detection. In: Maji, P., Ghosh, A., Murty, M.N., Ghosh, K., Pal, S.K. (eds.) PReMI 2013. LNCS, vol. 8251, pp. 695–700. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-45062-4_98

    CrossRef  Google Scholar 

  15. Huynh, N.A.: Frequency analysis and online learning in malware detection. Ph.D. thesis, Nanyang Technological University (2019)

    Google Scholar 

  16. Invernizzi, L., et al.: Nazca: detecting malware distribution in large-scale networks. In: NDSS, vol. 14, pp. 23–26. Citeseer (2014)

    Google Scholar 

  17. Johnson, J.: Purple team: About beacons, https://ci.security/resources/news/article/purple-team-about-beacons

  18. Jiang, J., Yin, Q., Shi, Z., Li, M., Lv, B.: A new c&c channel detection framework using heuristic rule and transfer learning. In: 2019 IEEE 38th International Performance Computing and Communications Conference (IPCCC), pp. 1–9. IEEE (2019)

    Google Scholar 

  19. Jin, H., Song, Q., Hu, X.: Auto-Keras: an efficient neural architecture search system. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1946–1956 (2019)

    Google Scholar 

  20. Khan, R.U., Zhang, X., Kumar, R., Sharif, A., Golilarz, N.A., Alazab, M.: An adaptive multi-layer botnet detection technique using machine learning classifiers. Appl. Sci. 9(11), 2375 (2019)

    CrossRef  Google Scholar 

  21. Kolodenker, E., Koch, W., Stringhini, G., Egele, M.: PayBreak: defense against cryptographic ransomware. In: Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, pp. 599–611 (2017)

    Google Scholar 

  22. Kotzias, P., Bilge, L., Vervier, P.A., Caballero, J.: Mind your own business: a longitudinal study of threats and vulnerabilities in enterprises. In: NDSS (2019)

    Google Scholar 

  23. Kührer, M., Rossow, C., Holz, T.: Paint it black: evaluating the effectiveness of malware blacklists. In: Stavrou, A., Bos, H., Portokalidis, G. (eds.) RAID 2014. LNCS, vol. 8688, pp. 1–21. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11379-1_1

    CrossRef  Google Scholar 

  24. Manasrah, A.M., Domi, W.B., Suppiah, N.N.: Botnet detection based on DNs traffic similarity. Int. J. Adv. Intell. Paradigms 15(4), 357–387 (2020)

    CrossRef  Google Scholar 

  25. Massey, F.J., Jr.: The Kolmogorov-Smirnov test for goodness of fit. J. Am. Stat. Assoc. 46(253), 68–78 (1951)

    CrossRef  Google Scholar 

  26. MITRE ATT&CK: MITRE ATT&CK tactics and techniques for enterprise. https://attack.mitre.org/matrices/enterprise/

  27. MITRE ATT&CK: Multi-stage channels technique. https://attack.mitre.org/techniques/T1104/

  28. Nadler, A., Aminov, A., Shabtai, A.: Detection of malicious and low throughput data exfiltration over the DNs protocol. Comput. Secur. 80, 36–53 (2019)

    CrossRef  Google Scholar 

  29. Plohmann, D., Yakdan, K., Klatt, M., Bader, J., Gerhards-Padilla, E.: A comprehensive measurement study of domain generating malware. In: 25th \(\{USENIX\}\) Security Symposium (\(\{USENIX\}\) Security 16), pp. 263–278 (2016)

    Google Scholar 

  30. Rendell, D.: Understanding the evolution of malware. Comput. Fraud Secur. 2019(1), 17–19 (2019)

    CrossRef  Google Scholar 

  31. Caragay, R., Cureg, F., Lagrazon, I., Mendoza, E., Yaneza, J.: (Threats Analysts): Exposing modular adware: How dealply, iserik, and managex persist in systems. https://blog.trendmicro.com/trendlabs-security-intelligence/exposing-modular-adware-how-dealply-iserik-and-managex-persist-in-systems

  32. Schales, D.L., Hu, X., Jang, J., Sailer, R., Stoecklin, M.P., Wang, T.: FCCE: highly scalable distributed feature collection and correlation engine for low latency big data analytics. In: 2015 IEEE 31st International Conference on Data Engineering, pp. 1316–1327. IEEE (2015)

    Google Scholar 

  33. Shalaginov, A., Franke, K., Huang, X.: Malware beaconing detection by mining large-scale DNs logs for targeted attack identification. In: 18th International Conference on Computational Intelligence in Security Information Systems. WASET (2016)

    Google Scholar 

  34. Sharif, M., Urakawa, J., Christin, N., Kubota, A., Yamada, A.: Predicting impending exposure to malicious content from user behavior. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 1487–1501 (2018)

    Google Scholar 

  35. Sidi, L., Mirsky, Y., Nadler, A., Elovici, Y., Shabtai, A.: Helix: DGA domain embeddings for tracking and exploring botnets. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 2741–2748 (2020)

    Google Scholar 

  36. Singh, M., Singh, M., Kaur, S.: Issues and challenges in DNs based botnet detection: a survey. Comput. Secur. 86, 28–52 (2019)

    CrossRef  Google Scholar 

  37. Sivakorn, S., et al.: Countering malicious processes with process-DNs association. In: NDSS (2019)

    Google Scholar 

  38. Tran, M.C., Nakamura, Y.: In-host communication pattern observed for suspicious http-based auto-ware detection. Int. J. Comput. Commun. Eng. 4(6), 379 (2015)

    CrossRef  Google Scholar 

  39. Urban, T., Tatang, D., Holz, T., Pohlmann, N.: Towards understanding privacy implications of adware and potentially unwanted programs. In: Lopez, J., Zhou, J., Soriano, M. (eds.) ESORICS 2018. LNCS, vol. 11098, pp. 449–469. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99073-6_22

    CrossRef  Google Scholar 

  40. Welzel, A., Rossow, C., Bos, H.: On measuring the impact of DDoS botnets. In: Proceedings of the Seventh European Workshop on System Security, pp. 1–6 (2014)

    Google Scholar 

  41. Yeh, Y.R., Tu, T.C., Sun, M.K., Pi, S.M., Huang, C.Y.: A malware beacon of botnet by local periodic communication behavior. In: 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC), vol. 2, pp. 653–657. IEEE (2018)

    Google Scholar 

  42. Zhauniarovich, Y., Khalil, I., Yu, T., Dacier, M.: A survey on malicious domains detection through DNs data analysis. ACM Comput. Surveys (CSUR) 51(4), 1–36 (2018)

    CrossRef  Google Scholar 

  43. Zhu, S., et al.: Measuring and modeling the label dynamics of online anti-malware engines. In: 29th \(\{USENIX\}\) Security Symposium (\(\{USENIX\}\) Security 20), pp. 2361–2378 (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Asaf Nadler .

Editor information

Editors and Affiliations

Appendices

Appendix A Detecting Multiple Host Names

The primary drawback of bot communication techniques that use a single host name (e.g., malware beaconing technique with single host name C&C communication) is their lack of robustness. The single host name is effectively a single point of failure, and if the host name is unavailable for any reason, the attackers cannot control their bots. An additional drawback is that communication with a single host name may be less covert. For instance, in DNS data exfiltration, every exfiltration message is sent to an attacker’s host name. A single host name that receives a large volume of exfiltration messages is more detectable by security systems [28]. Therefore, the activities of bots that split their DNS exfiltration messages and send them to multiple host names are less suspicious.

The most well-known use of multiple host names for botnet communication is through domain generation algorithms [26] (DGAs), which are used by over 40 known botnets [29]. Most bots that use DGAs generate new domain names on a daily basis [29], thus pointing to the importance of detecting bot communication that uses multiple domain names.

Multistage channels (MSC) are another bot communication technique in which multiple host names are used. The initial installation of the bot on a compromised device is referred to as the first stage of the infection. Throughout the first stage, the bot communicates with its C&C through either a single host name or multiple host names. However, the host names will change when the first stage bot requires an upgrade. A bot upgrade typically involves communicating with a new host name to download a module that enhances the bot’s capabilities. The process of upgrading the bot is referred to as the second stage of the infection. The MSC bot communication technique often involves several stages, where multiple host names are gradually upgrading the bot. The use of MSC improves the robustness of a botnet’s infrastructure, because security researchers cannot easily identify the different host names that will be used by a botnet in order to shut down its operation (i.e., prevent bots from upgrading).

Other cases of bot communication techniques in which multiple host names are used include fallback channels and multihop proxies [26]. In fallback channels, a bot that fails to communicate with its C&C host name attempts to communicate to the host name next in line, based on a prioritized list of host names. Multihop proxies is a bot communication technique in which the C&C channel is established through a series of proxy servers that are associated with different host names. The series of proxy servers between bots and their C&C servers prevents security researchers from easily matching a bot communicating with its C&C server based on network logs. MORTON is designed to detect every multiple host communication technique mentioned, as long as it is used in a periodic manner.

Appendix B Neural Network Parameters

The architecture and the learning rate were selected, because they performed best with regard to the area-under-curve metric when compared against more than 25 alternative architectures originating from an ablation study and the use of AutoML for structured data [19], as can be seen in Table 4. (note that all of the settings were trained and evaluated on a smaller subset of the data to reduce training time).

Table 4. Neural network parameters

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Daihes, Y., Tzaban, H., Nadler, A., Shabtai, A. (2021). MORTON: Detection of Malicious Routines in Large-Scale DNS Traffic. In: Bertino, E., Shulman, H., Waidner, M. (eds) Computer Security – ESORICS 2021. ESORICS 2021. Lecture Notes in Computer Science(), vol 12972. Springer, Cham. https://doi.org/10.1007/978-3-030-88418-5_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-88418-5_35

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-88417-8

  • Online ISBN: 978-3-030-88418-5

  • eBook Packages: Computer ScienceComputer Science (R0)