Skip to main content

Measuring virtual machine detection in malware using DSD tracer


Most methods for detecting that a process is running inside a virtual environment such as VMWare or Microsoft Virtual PC are well known and the paper briefly discusses the most common methods measured during the research. The measurements are conducted over a representative set of malicious files, with special regards to packer code. The results are broken down with respect to malware category, families and various commercial and non-commercial packers and presented in a graphical and tabular format. The extent of virtual machine detection problem is estimated based on the results of the research. The main subject of the paper is measurement of actual usage of Virtual machine detection methods in current malware. The research uses DSD Tracer, a dynamic-static tracing system based on an instrumented Bochs virtual machine. The system employs tracing to produce traces of execution that can be scripted or used as a basis for disassembly/emulation in IDA Pro when combined with a customised version of IDAEmul (emulator). The paper gives an overview of design and usage of DSD Tracer.

This is a preview of subscription content, access via your institution.


  1. 1

    Lau, B.: DSD-Tracer: experimentation and implementation. In: Virus Bulletin 2007 Conference proceedings (2007)

  2. 2

    Moser, A., Kruegel, C., Kirda, E.: Exploring Multiple Execution Paths for Malware Analysis (2006)

  3. 3

    Bayer, U.: TTAnalyze: a tool for analyzing Malware. Master’s Thesis, Technical University of Vienna (2005)

  4. 4

    Vasudevan, A., Yerraballi, R.: Cobra: fine-grained Malware analysis using stealth localized-executions. In: IEEE and Signature Generation of Exploits on Commodity Software (2006)

  5. 5

    Willems, A., Holz, C., Freiling, T., Felix A.: Toward Automated Dynamic Malware Analysis Using CWSandbox. (2007)

  6. 6

    Simplified Wrapper and Interface Generator. (2000)

  7. 7

    Natvig, K.: Norman sandbox white paper. (2003)

  8. 8

    Vidstrom, A.: Evading the Norman SandBox Analyzer. BugTraq bulletin (2007)

  9. 9

    Eagle, C.: Attacking Packed Code with IDA Pro., Black-hat Asia (2006)

  10. 10

    Bellard, F.: QEMU Emulator User Documentation # GDB usage. (2005)

  11. 11

    Ormandy, T.: An empirical study into the security exposure to hosts of hostile virtualized environments, CanSecWest (2007)

  12. 12

    Ferrie, P.: Attacks on virtual machine emulators (2007)

  13. 13

    Xu M., et al.: ReTrace: Collecting execution trace with virtual machine deterministic replay (2007)

  14. 14

    Herrod, S.: The amazing VM record/replay feature in VMware Workstation 6. (2007)

  15. 15

    Technology, O.: Themida overview. (2007)

  16. 16

    Malyugin, V.: Application debugging with Record/Replay. (2007)

  17. 17

    Malyugin, V.: VMware forum thread. (2007)

  18. 18

    Callanan, S.: Terminate-on-error patch for GDBcli. (2005)

  19. 19

    Schneider, O.: Redpill getting colorless? (2007)

  20. 20

    Rutkowska, J.: Red Pill. (2004)

  21. 21

    Klein, T.: Jerry. (2005)

  22. 22

    Klein, T.: Scoopy Doo. (2005)

  23. 23

    Kato, K.: VMWare Back. (2003)

  24. 24

    Liston, T., Skoudis, E.: On the cutting edge: thwarting virtual machine detection. (2006)

  25. 25

    O’Dea, H.: Trapping worms in a virtual net. In: Virus Bulletin 2004 Conference Proceedings (2004)

  26. 26

    Intel.: Intel architecture software developer’s manual, vol 2: instruction set reference manual. (2003)

  27. 27

    Quist, D.: Vmdetect. (2006)

Download references

Author information



Corresponding author

Correspondence to Boris Lau.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Lau, B., Svajcer, V. Measuring virtual machine detection in malware using DSD tracer. J Comput Virol 6, 181–195 (2010).

Download citation


  • Virtual Machine
  • False Positive Detection
  • Virtual Address
  • Packed Sample
  • Automate Analysis System