Abstract
Abstract. Botmasters increasingly encrypt command-and-control (C&C) communication to evade existing intrusion detection systems. Our detailed C&C traffic analysis shows that at least ten prevalent malware families avoid well-known C&C carrier protocols, such as IRC and HTTP. Six of these families - e.g., Zeus P2P, Pramro, Virut, and Sality - do not exhibit any characteristic n-gram that could serve as payload-based signature in an IDS.
Given knowledge of the C&C encryption algorithms, we detect these evasive C&C protocols by decrypting any packet captured on the network. In order to test if the decryption results in messages that stem from malware, we propose ProVex, a system that automatically derives probabilistic vectorized signatures. ProVex learns characteristic values for fields in the C&C protocol by evaluating byte probabilities in C&C input traces used for training. This way, we identify the syntax of C&C messages without the need to manually specify C&C protocol semantics, purely based on network traffic. Our evaluation shows that ProVex can detect all studied malware families, most of which are not detectable with traditional means. Despite its naive approach to decrypt all traffic, we show that ProVex scales up to multiple Gbit/s line speed networks.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Caballero, J., Johnson, N.M., McCamant, S., Song, D.: Binary Code Extraction and Interface Identification for Security Applications. In: Proceedings of the 17th Annual Network and Distributed System Security Symposium (NDSS), San Diego, CA (February 2010)
Caballero, J., Yin, H., Liang, Z., Song, D.X.: Polyglot: Automatic Extraction of Protocol Message Format Using Dynamic Binary Analysis. In: Proceedings of the ACM Conference on Computer and Communications Security (CCS) (November 2007)
Comparetti, P.M., Wondracek, G., Kruegel, C., Kirda, E.: Prospex: Protocol Specification Extraction. In: Proceedings of the 30th IEEE Symposium on Security and Privacy (S&P) (May 2009)
Cui, W.: Discoverer: Automatic Protocol Reverse Engineering from Network Traces. In: Proceedings of the 16th USENIX Security Symposium (August 2007)
Dietrich, C.J., Rossow, C., Freiling, F.C., Bos, H., van Steen, M., Pohlmann, N.: On Botnets that Use DNS for Command and Control. In: Proceedings of European Conference on Computer Network Defense (EC2ND) (September 2011)
Dietrich, C.J., Rossow, C., Pohlmann, N.: CoCoSpot: Clustering and Recognizing Botnet Command and Control Channels Using Traffic Analysis. A Special Issue of Computer Networks On Botnet Activity: Analysis, Detection and Shutdown (July 2012)
Gröbert, F., Willems, C., Holz, T.: Automated Identification of Cryptographic Primitives in Binary Programs. In: Sommer, R., Balzarotti, D., Maier, G. (eds.) RAID 2011. LNCS, vol. 6961, pp. 41–60. Springer, Heidelberg (2011)
Hadžiosmanović, D., Simionato, L., Bolzoni, D., Zambon, E., Etalle, S.: N-gram Against the Machine: On the Feasibility of the N-gram Network Analysis for Binary Protocols. In: Balzarotti, D., Stolfo, S.J., Cova, M. (eds.) RAID 2012. LNCS, vol. 7462, pp. 354–373. Springer, Heidelberg (2012)
Kolbitsch, C., Holz, T., Kruegel, C., Kirda, E.: Inspector Gadget: Automated Extraction of Proprietary Gadgets from Malware Binaries. In: Proceedings of the 30th IEEE Symposium on Security & Privacy (S&P) (May 2009)
Krueger, T., Gascon, H., Krämer, N., Rieck, K.: Learning Stateful Models for Network Honeypots. In: Proceedings of the ACM Workshop on Artificial Intelligence and Security (AISec) (October 2012)
Leder, F., Martini, P., Wichmann, A.: Finding and Extracting Crypto Routines from Malware. In: Proceedings of the International Performance Computing and Communications Conference (IPCCC) (December 2009)
Needleman, S.B., Wunsch, C.D.: A General Method Applicable to the Search for Similarities in the Amino Acid Sequence of Two Proteins. Journal of Molecular Biology 48(3), 443–453 (1970)
Newsome, J., Brumley, D., Franklin, J., Song, D.: Replayer: Automatic Protocol Replay by Binary Analysis. In: Proceedings of the 13th ACM Conference on Computer and Communications Security (CCS (November 2006)
Newsome, J., Karp, B., Song, D.: Polygraph: Automatically Generating Signatures for Polymorphic Worms. In: Proceedings of the 26th IEEE Symposium on Security & Privacy (S&P) (May 2005)
Olivain, J., Goubault-Larrecq, J.: Detecting Subverted Cryptographic Protocols by Entropy Checking. Research Report LSV-06-13, Laboratoire Spécification et Vérification, ENS Cachan, France (June 2006)
Perdisci, R., Lee, W., Feamster, N.: Behavioral Clustering of HTTP-Based Malware and Signature Generation Using Malicious Network Traces. In: Proceedings of the USENIX Symposium on Networked Systems Designs and Implementation (NSDI) (April 2010)
Rieck, K., Schwenk, G., Limmer, T., Holz, T., Laskov, P.: Botzilla: Detecting the “Phoning Home” of Malicious Software. In: Proceedings of the 25th ACM Symposium on Applied Computing (SAC) (March 2010)
Rossow, C., Andriesse, D., Werner, T., Stone-Gross, B., Plohmann, D., Dietrich, C.J., Bos, H.: P2PWNED: Modeling and Evaluating the Resilience of Peer-to-Peer Botnets. In: Proceedings of the 34th IEEE Symposium on Security and Privacy (S&P), San Francisco, CA (May 2013)
Rossow, C., Dietrich, C.J., Bos, H., Cavallaro, L., van Steen, M., Freiling, F.C., Pohlmann, N.: Sandnet: Network Traffic Analysis of Malicious Software. In: Proceedings of ACM EuroSys BADGERS (April 2011)
Rossow, C., Dietrich, C.J., Kreibich, C., Grier, C., Paxson, V., Pohlmann, N., Bos, H., van Steen, M.: Prudent Practices for Designing Malware Experiments: Status Quo and Outlook. In: Proceedings of the 33rd IEEE Symposium on Security and Privacy (S&P), San Francisco, CA (May 2012)
Sommer, R., Paxson, V.: Outside the Closed World: On Using Machine Learning for Network Intrusion Detection. In: Proceedings of the 31st IEEE Symposium on Security & Privacy (May 2010)
Wang, Z., Jiang, X., Cui, W., Wang, X., Grace, M.: ReFormat: Automatic Reverse Engineering of Encrypted Messages. In: Backes, M., Ning, P. (eds.) ESORICS 2009. LNCS, vol. 5789, pp. 200–215. Springer, Heidelberg (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rossow, C., Dietrich, C.J. (2013). ProVeX: Detecting Botnets with Encrypted Command and Control Channels. In: Rieck, K., Stewin, P., Seifert, JP. (eds) Detection of Intrusions and Malware, and Vulnerability Assessment. DIMVA 2013. Lecture Notes in Computer Science, vol 7967. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39235-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-39235-1_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39234-4
Online ISBN: 978-3-642-39235-1
eBook Packages: Computer ScienceComputer Science (R0)