On Improving the Accuracy and Performance of Content-Based File Type Identification

  • Irfan Ahmed
  • Kyung-suk Lhee
  • Hyunjung Shin
  • ManPyo Hong
Conference paper

DOI: 10.1007/978-3-642-02620-1_4

Part of the Lecture Notes in Computer Science book series (LNCS, volume 5594)
Cite this paper as:
Ahmed I., Lhee K., Shin H., Hong M. (2009) On Improving the Accuracy and Performance of Content-Based File Type Identification. In: Boyd C., González Nieto J. (eds) Information Security and Privacy. ACISP 2009. Lecture Notes in Computer Science, vol 5594. Springer, Berlin, Heidelberg

Abstract

Types of files (text, executables, Jpeg images, etc.) can be identified through file extension, magic number, or other header information in the file. However, they are easy to be tampered or corrupted so cannot be trusted as secure ways to identify file types.In the presence of adversaries, analyzing the file content may be a more reliable way to identify file types, but existing approaches of file type analysis still need to be improved in terms of accuracy and speed. Most of them use byte-frequency distribution as a feature in building a representative model of a file type, and apply a distance metric to compare the model with byte-frequency distribution of the file in question. Mahalanobis distance is the most popular distance metric. In this paper, we propose 1) the cosine similarity as a better metric than Mahalanobis distance in terms of classification accuracy, smaller model size, and faster detection rate, and 2) a new type-identification scheme that applies recursive steps to identify types of files. We compare the cosine similarity to Mahalanobis distance using Wei-Hen Li et al.’s single and multi-centroid modeling techniques, which showed 4.8% and 13.10% improvement in classification accuracy (single and multi-centroid respectively). The cosine similarity showed reduction of the model size by about 90% and improvement in the detection speed by 11%. Our proposed type identification scheme showed 37.78% and 31.47% improvement over Wei-Hen Li’s single and multi-centroid modeling techniques respectively.

Keywords

file type identification byte frequency distribution cosine similarity Mahalanobis distance linear discriminant cluster analysis 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Irfan Ahmed
    • 1
  • Kyung-suk Lhee
    • 1
  • Hyunjung Shin
    • 2
  • ManPyo Hong
    • 1
  1. 1.Digital Vaccine and Internet Immune System Lab Graduate School of Information and CommunicationAjou UniversitySouth Korea
  2. 2.Department of Industrial and Information Systems EngineeringAjou UniversitySouth Korea

Personalised recommendations