Using Approximate Matching to Reduce the Volume of Digital Data

  • Frank Breitinger
  • Christian Winter
  • York Yannikos
  • Tobias Fink
  • Michael Seefried
Conference paper
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 433)


Digital forensic investigators frequently have to search for relevant files in massive digital corpora – a task often compared to finding a needle in a haystack. To address this challenge, investigators typically apply cryptographic hash functions to identify known files. However, cryptographic hashing only allows the detection of files that exactly match the known file hash values or fingerprints. This paper demonstrates the benefits of using approximate matching to locate relevant files. The experiments described in this paper used three test images of Windows XP, Windows 7 and Ubuntu 12.04 systems to evaluate fingerprint-based comparisons. The results reveal that approximate matching can improve file identification – in one case, increasing the identification rate from 1.82% to 23.76%.


File identification approximate matching ssdeep 


  1. 1.
    H. Baier and C. Dichtelmuller, Datenreduktion mittels kryptographischer Hashfunktionen in der IT-Forensik: Nur ein Mythos? DACH Security, pp. 278–287, September 2012.Google Scholar
  2. 2.
    F. Breitinger, K. Astebol, H. Baier and C. Busch, mvhash-b – A new approach for similarity preserving hashing, Proceedings of the Seventh International Conference on IT Security Incident Management and IT Forensics, pp. 33–44, 2013.Google Scholar
  3. 3.
    F. Breitinger and H. Baier, Security aspects of piecewise hashing in computer forensics, Proceedings of the Sixth International Conference on IT Security Incident Management and IT Forensics, pp. 21–36, 2011.Google Scholar
  4. 4.
    F. Breitinger and H. Baier, A fuzzy hashing approach based on random sequences and Hamming distance, Proceedings of the Conference on Digital Forensics, Security and Law, 2012.Google Scholar
  5. 5.
    F. Breitinger and H. Baier, Similarity preserving hashing: Eligible properties and a new algorithm mrsh-v2, Proceedings of the Fourth International ICST Conference on Digital Forensics and Cyber Crime, 2012.Google Scholar
  6. 6.
    A. Broder, On the resemblance and containment of documents, Proceedings of the International Conference on the Compression and Complexity of Sequences, pp. 21–29, 1997.Google Scholar
  7. 7.
    L. Chen and G. Wang, An efficient piecewise hashing method for computer forensics, Proceedings of the First International Workshop on Knowledge Discovery and Data Mining, pp. 635–638, 2008.CrossRefGoogle Scholar
  8. 8.
    P. Deutsch and J. Gailly, ZLIB Compressed Data Format Specification Version 3.3, RFC 1950, 1996.Google Scholar
  9. 9.
    J. Kornblum, Identifying almost identical files using context triggered piecewise hashing, Digital Investigation, vol. 3(S), pp. S91–S97, 2006.CrossRefGoogle Scholar
  10. 10.
    J. Kornblum, ssdeep (, 2013.
  11. 11.
    National Institute of Standards and Technology, National Software Reference Library, Gaithersburg, Maryland (
  12. 12.
  13. 13.
    V. Roussev, Data fingerprinting with similarity digests, in Advances in Digital Forensics VI, K. Chow and S. Shenoi (Eds.), Springer, Heidelberg, Germany, pp. 207–226, 2010.CrossRefGoogle Scholar
  14. 14.
    V. Roussev, An evaluation of forensic similarity hashes, Digital Investigation, vol. 8(S), pp. S34–S41, 2011.CrossRefGoogle Scholar
  15. 15.
    C. Sadowski and G. Levin, SimHash: Hash-Based Similarity Detection, Technical Report UCSC-SOE-11-07, Department of Computer Science, University of California Santa Cruz, Santa Cruz, California (, 2007.
  16. 16.
    K. Seo, K. Lim, J. Choi, K. Chang and S. Lee, Detecting similar files based on hash and statistical analysis for digital forensic investigations, Proceedings of the Second International Conference on Computer Science and its Applications, 2009.Google Scholar
  17. 17.
  18. 18.
    D. White, Hashing of file blocks: When exact matches are not useful, presented at the Annual Meeting of the American Academy of Forensic Sciences, 2008.Google Scholar
  19. 19.
    C. Winter, M. Schneider and Y. Yannikos, F2S2: Fast forensic similarity search through indexing piecewise hash signatures, Digital Investigation, vol. 10(4), pp. 361–371, 2013.CrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2014

Authors and Affiliations

  • Frank Breitinger
    • 1
    • 2
  • Christian Winter
    • 3
  • York Yannikos
    • 3
  • Tobias Fink
    • 1
  • Michael Seefried
    • 1
  1. 1.Darmstadt University of Applied SciencesDarmstadtGermany
  2. 2.Center for Advanced Security Research DarmstadtDarmstadtGermany
  3. 3.Fraunhofer Institute for Secure Information TechnologyDarmstadtGermany

Personalised recommendations