API Monitoring System for Defeating Worms and Exploits in MS-Windows System

  • Hung-Min Sun
  • Yue-Hsun Lin
  • Ming-Fung Wu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4058)


Worms and Exploits attacks are currently the most prevalent security problems; they are responsible for over half of the CERT advisories issued in the last three years. To initiate an infection or intrusion, both of them inject a small piece of malicious code (ShellCode) into software through buffer or heap overflow vulnerabilities. Unlike Unix-like operating systems, ShellCodes for Microsoft Windows system need more complex steps to acquire Win32 API calls from DLL file (Dynamic Load Library) in Microsoft Windows. In this paper, we proposed an effective API monitoring system to get rid of worms and exploits attacks for the Microsoft Windows without hardware support. We address the problem by noticing that ShellCodes need the extra complex steps in accessing Win32 API calls. Through the API monitoring system we purposed, we can successfully stop the attacks made by worms and exploits. Moreover, the efficiency of Win32 API Calls hooking and monitoring system can be improved. Incapability to disassemble and analysis the protected software processes are overcome as well.


Worm Protection System Security API Hooking 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anton, B.: “ Process-wide API spying - an ultimate hack”, CodeProject website (February 2005), Available:
  2. 2.
    Anton, B.: Kernel-mode API spying - an ultimate hack, CodeProject website (February 2005), Available:
  3. 3.
    Michel, B.: Introduction to Shellcoding - How to exploit buffer overflows, Tigerteam’s website (2004), Available:
  4. 4.
    Jesse, C., Rabek, R., Khazan, I., Scott, M., Robert, L., Cunningham, K.: Detection of Injected, Dynamically Generated, and Obfuscated Malicious Code. In: Proc. of 2003 ACM workshop on Rapid Malcode (October 2003)Google Scholar
  5. 5.
    Shannon, C., Moore, D.: The spread of the Witty worm. Security & Privacy Magazine 2(4), 46–50 (2004)CrossRefGoogle Scholar
  6. 6.
    Barrantes, E.G., Ackley, D.H., Palmer, T.S., Stefanovic, D., Zovi, D.D.: Randomized instruction set emulation to disrupt binary code injection attacks. In: Proc. of 10th ACM Conf. Comp. and Comm. Sec.—CCS 2003, pp. 281–289. ACM Press, New York (2003)CrossRefGoogle Scholar
  7. 7.
    Levy, E.: Worm propagation and generic attacks. Security & Privacy Magazine 3(2), 63–65 (2005)CrossRefGoogle Scholar
  8. 8.
    Hunt, G., Brubacher, D.: Detours: Binary Interception of Win32 Functions, Microsoft corp. research website (1999), Available:
  9. 9.
    Ivo, I.: API hooking revealed, CodeProject website (February 2005), Available:
  10. 10.
    Richter, J.: Programming Applications for Microsoft Windows, 4th edn. (2001)Google Scholar
  11. 11.
    Riordan, J., Wespi, A., Zamboni, D.: How To Hook Worms. Spectrum 42(5), 32–36 (2005)CrossRefGoogle Scholar
  12. 12.
    Pincus, J., Baker, R.: Beyond stack smashing: recent advances in exploiting buffer over runs. Security & Privacy Magazine 2(4), 20–27 (2004)CrossRefGoogle Scholar
  13. 13.
    Pietrek, M.: Inside Windows: An In-Depth Look into the Win32 Portable Executable File Format. MSDN Website (2002), Available:
  14. 14.
    Sachin, R.S.: Need for Rebasing a DLL. Code Project website (March 2005), Available:
  15. 15.
    Sachin, R.S.: Need for Binding an Executable to DLLs. Code Project website (March 2005), Available:
  16. 16.
    Payer, U., Teufl, P., Lamberger, M.: Hybrid Engine for Polymorphic Shellcode Detection. In: Second Int. Conf. of 2005 Intrusion and Malware Detection and Vulnerability Assessment (July 2005)Google Scholar
  17. 17.
    Kaplan, Y.: API Spying Techniques for Windows 9x, NT and 2000. From website of teaching API Hooking and Monitoring (June 2004), Available:
  18. 18.
    The MetaSploit Project, ShellCode Archive, MetaSploit Project official website (November 2004), Available:
  19. 19.
    Microsoft Corp., A detailed description of the Data Execution Prevention (DEP) feature in Windows XP Service Pack 2 and Windows XP Tablet PC Edition 2005, Microsoft Corp’s support website (February 2005), Available:
  20. 20.
    The NTInternals. net team, Undocumented Functions for Microsoft Windows NT/2000, NTInternals. net website (November 28, 2004), Available:
  21. 21.
    Phrack Inc., History and Advances in Windows ShellCode. In Phrack Magazine (November 2004), Available:
  22. 22.
    Smiler, The Art of Writing ShellCode, FreeGnu’s personal blog (June 2004), Available:
  23. 23.
    The ShellCode. org, The ShellCode Writing, ShellCode. org website. (November 2004), Available:

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Hung-Min Sun
    • 1
  • Yue-Hsun Lin
    • 1
  • Ming-Fung Wu
    • 1
  1. 1.Department of Computer ScienceNational Tsing-Hua UniversityHsinchuTaiwan

Personalised recommendations