The 1998 Lincoln Laboratory IDS Evaluation

A Critique
  • John McHugh
Conference paper

DOI: 10.1007/3-540-39945-3_10

Part of the Lecture Notes in Computer Science book series (LNCS, volume 1907)
Cite this paper as:
McHugh J. (2000) The 1998 Lincoln Laboratory IDS Evaluation. In: Debar H., Mé L., Wu S.F. (eds) Recent Advances in Intrusion Detection. RAID 2000. Lecture Notes in Computer Science, vol 1907. Springer, Berlin, Heidelberg


In 1998 (and again in 1999), the Lincoln Laboratory of MIT conducted a comparative evaluation of Intrusion Detection Systems developed under DARPA funding. While this evaluation represents a significant and monumental undertaking, there are a number of unresolved issues associated with its design and execution. Some of methodologies used in the evaluation are questionable and may have biased its results. One of the problems with the evaluation is that the evaluators have published relatively little concerning some of the more critical aspects of their work, such as validation of their test data. The purpose of this paper is to attempt to identify the shortcomings of the Lincoln Lab effort in the hope that future efforts of this kind will be placed on a sounder footing. Some of the problems that the paper points out might well be resolved if the evaluators publish a detailed description of their procedures and the rationale that led to their adoption, but other problems clearly remain.


Evaluation IDS ROC Analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • John McHugh
    • 1
  1. 1.CERT® Coordination Center, Software Engineering InstituteCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations