Images Annotation Extension Based on User Feedback

Abstract

In this paper, we propose a probabilistic graphical model for images annotation extension. The aim is to extend the annotations of a small subset of images to a whole dataset. Therefore, this subset is used to learn the parameters of our model, which is based on multinomial and Gaussian mixture distributions. Our model allows combining efficiently visual and textual characteristics. Since the performance of our system depends on the quality of the learning, we integrate the user in the loop to improve the annotation quality and minimize the laborious manual annotation effort at three levels. The first level is related to the learning set. We perform a kind of learning in learning. More precisely, we propose a way to annotate semi-automatically images used in the learning. We introduce an iterative loop where annotations are automatically extended and some corrected manually by the user. In this way we reduce the tedious effort of manual annotation. In the second level, after the annotation extension and during a retrieval step a user can correct or add labels to some images. These images with their new labels are introduced progressively to the system and used to relearn incrementally the model. In the third level, we propose an active learning of our model to select the most informative data to improve the quality of learning and reduce manual effort.