Embedded Malware Detection Using Markov n-Grams

  • M. Zubair Shafiq
  • Syed Ali Khayam
  • Muddassar Farooq
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5137)


Embedded malware is a recently discovered security threat that allows malcode to be hidden inside a benign file. It has been shown that embedded malware is not detected by commercial antivirus software even when the malware signature is present in the antivirus database. In this paper, we present a novel anomaly detection scheme to detect embedded malware. We first analyze byte sequences in benign files to show that benign files’ data generally exhibit a 1-st order dependence structure. Consequently, conditional n-grams provide a more meaningful representation of a file’s statistical properties than traditional n-grams. To capture and leverage this correlation structure for embedded malware detection, we model the conditional distributions as Markov n -grams. For embedded malware detection, we use an information-theoretic measure, called entropy rate, to quantify changes in Markov n-gram distributions observed in a file. We show that the entropy rate of Markov n-grams gets significantly perturbed at malcode embedding locations, and therefore can act as a robust feature for embedded malware detection. We evaluate the proposed Markov n-gram detector on a comprehensive malware dataset consisting of more than 37,000 malware samples and 1,800 benign samples of six well-known filetypes. We show that the Markov n-gram detector provides better detection and false positive rates than the only existing embedded malware detection scheme.


Entropy Rate Benign File Anomaly Detection Scheme Victim Machine Statistical Anomaly Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Symantec Internet Security Threat Report XI: Trends for July – December 2007 (September 2007)Google Scholar
  2. 2.
    Stolfo, S.J., Wang, K., Li, W.J.: Towards Stealthy Malware Detection. Advances in Information Security, vol. 27, pp. 231–249. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  3. 3.
    Li, W.J., Stolfo, S.J., Stavrou, A., Androulaki, E., Keromytis, A.D.: A Study of Malcode-Bearing Documents. In: Hämmerli, B.M., Sommer, R. (eds.) DIMVA 2007. LNCS, vol. 4579. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  4. 4.
    Leyden, J.: Trojan exploits unpatchedWord vulnerability. The Register (May 2006)Google Scholar
  5. 5.
    Evers, J.: Zero-day attacks continue to hit Microsoft. (September 2006)Google Scholar
  6. 6.
    Kierznowski, D.: Backdooring PDF Files (September 2006)Google Scholar
  7. 7.
    Damashek, M.: Gauging Similarity with n-Grams: Language-Independent Categorization of Text. Science 267, 842–848 (1995)CrossRefGoogle Scholar
  8. 8.
    Friedl, S.: Steve Friedl’s Tech Tips: Analysis of the new ‘Code Red II’ Variant,
  9. 9.
    Lakhina, A., Crovella, M., Diot, C.: Mining anomalies using traffic feature distributions. ACM SIGCOMM (September 2005)Google Scholar
  10. 10.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley-Interscience, Chichester (1991)zbMATHGoogle Scholar
  11. 11.
    VX Heavens Virus Collection, VX Heavens,
  12. 12.
    McAfee Anti-virus and Internet security, McAfee, CAGoogle Scholar
  13. 13.
    Kaspersky Antivirus, Kaspersky Lab HQ, Moscow, RussiaGoogle Scholar
  14. 14.
    AVG Anti-Virus and Internet Security, GRISOFT Inc., FL, USAGoogle Scholar
  15. 15.
    Huang, Y.: Vulnerabilities in Portable Executable (PE) File Format For Win32 Architecture, TR, Exurity Inc., Canada (2006)Google Scholar
  16. 16.
    Fawcett, T.: ROC Graphs: Notes and Practical Considerations for Researchers, TR, HP Labs, CA, USA (2003-2004)Google Scholar
  17. 17.
    Wagner, D., Soto, P.: Mimicry Attacks on Host-Based Intrusion Detection Systems. ACM CCS (November 2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • M. Zubair Shafiq
    • 1
  • Syed Ali Khayam
    • 2
  • Muddassar Farooq
    • 1
  1. 1.Next Generation Intelligent Networks Research Center (nexGINRC)National University of Computer & Emerging Sciences (NUCES)IslamabadPakistan
  2. 2.School of Electrical Engineering & Computer Science (SEECS)National University of Sciences & Technology (NUST)RawalpindiPakistan

Personalised recommendations