Are Performance Differences of Interest Operators Statistically Significant?
The differences in performance of a range of interest operators are examined in a null hypothesis framework using McNemar’s test on a widely-used database of images, to ascertain whether these apparent differences are statistically significant. It is found that some performance differences are indeed statistically significant, though most of them are at a fairly low level of confidence, i.e. with about a 1-in-20 chance that the results could be due to features of the evaluation database. A new evaluation measure i.e. accurate homography estimation is used to characterize the performance of feature extraction algorithms.Results suggest that operators employing longer descriptors are more reliable.
KeywordsFeature Extraction Homography McNemar’s Test
Unable to display preview. Download preview PDF.
- 1.Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1615–1630 (2005)Google Scholar
- 3.Saag, M.S., Powderly, W.G., Cloud, G.A., Robinson, P., Grieco, M.H., Sharkey, P.K., Thompson, S.E., Sugar, A.M., Tuazon, C.U., Fisher, J.F., et al.: Comparison of amphotericin B with fluconazole in the treatment of acute AIDS-associated cryptococcal meningitis. New England Journal of Medicine 326(2), 83–89 (1992)CrossRefGoogle Scholar
- 9.Valgren, C., Lilienthal, A.: SIFT, SURF and seasons: Long-term outdoor localization using local features. In: Proceedings of the European Conference on Mobile Robots (ECMR), pp. 253–258 (2007)Google Scholar
- 10.Clark, A.F., Clark, C.: Performance Characterization in Computer Vision A Tutorial (1999)Google Scholar
- 12.Abdi, H.: Bonferroni and Šidák corrections for multiple comparisons. Sage, Thousand Oaks, CA (2007)Google Scholar