Evaluation Framework

Abstract

The shape of an object is an important feature for image and multimedia similarity retrievals, it conforms to the way human beings interpret and interact with the real world objects. The shape representation of objects can therefore be used for their indexing, retrieval, and as a similarity measure. However, as a consequence of uncertainty, shape representation techniques sometimes work well only in certain applications, and their performance highly depends on the quality of the technique used to represent shapes. Therefore, in this chapter, we address key methodological issues for the evaluation of shape retrieval methods with real data and under different scenarios. We describe several metrics and criterions to evaluate and compare the effectiveness and robustness of a given shape retrieval method. The metrics can be used to compare the efficiency of the techniques in terms of: 1) retrieval accuracy in terms of recall and precision, 2) storage cost for their indices, 3) computation cost to generate the shape signature, 4) computation cost to compute the similarity between two different shape signatures, 5) sensitivity to the presence of noise in the database, and 6) sensitivity to the alternative ways of identifying the boundary points of a shape, 7) support for different query types, and 8) impact of human perception.