A Visual Interactive Environment for Making Sense of Experimental Data

  • Marco Angelini
  • Nicola Ferro
  • Giuseppe Santucci
  • Gianmaria Silvello
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8416)


We present the Visual Information Retrieval Tool for Upfront Evaluation (VIRTUE) which is an interactive and visual system supporting two relevant phases of the experimental evaluation process: performance analysis and failure analysis.


Failure Analysis Information Retrieval System Result List Optimal Ranking Ranking Evaluation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Angelini, M., Ferro, N., Järvelin, K., Keskustalo, H., Pirkola, A., Santucci, G., Silvello, G.: Cumulated Relative Position: A Metric for Ranking Evaluation. In: Catarci, T., Forner, P., Hiemstra, D., Peñas, A., Santucci, G. (eds.) CLEF 2012. LNCS, vol. 7488, pp. 112–123. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  2. 2.
    Angelini, M., Ferro, N., Santucci, G., Silvello, G.: Visual Interactive Failure Analysis: Supporting Users in Information Retrieval Evaluation. In: Proc. 4th Symposium on Information Interaction in Context (IIiX 2012), pp. 195–203. ACM Press, New York (2012)Google Scholar
  3. 3.
    Angelini, M., Ferro, N., Santucci, G., Silvello, G.: Improving Ranking Evaluation Employing Visual Analytics. In: Forner, P., Müller, H., Paredes, R., Rosso, P., Stein, B. (eds.) CLEF 2013. LNCS, vol. 8138, pp. 29–40. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  4. 4.
    Angelini, M., Ferro, N., Santucci, G., Silvello, G.: VIRTUE: A Visual Tool for Information Retrieval Performance Evaluation and Failure Analysis. Journal of Visual Languages & Computing (in print, 2014)Google Scholar
  5. 5.
    Di Buccio, E., Dussin, M., Ferro, N., Masiero, I., Santucci, G., Tino, G.: To Re-rank or to Re-query: Can Visual Analytics Solve This Dilemma? In: Forner, P., Gonzalo, J., Kekäläinen, J., Lalmas, M., de Rijke, M. (eds.) CLEF 2011. LNCS, vol. 6941, pp. 119–130. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  6. 6.
    Harman, D.K.: Some thoughts on failure analysis for noisy data. In: Proc. 2nd Workshop on Analytics for Noisy unstructured text Data (AND 2008), p. 1. ACM Press, New York (2008)CrossRefGoogle Scholar
  7. 7.
    Järvelin, K., Kekäläinen, J.: Cumulated Gain-Based Evaluation of IR Techniques. ACM Transactions on Information Systems (TOIS) 20(4), 422–446 (2002)CrossRefGoogle Scholar
  8. 8.
    Savoy, J.: Why do Successful Search Systems Fail for Some Topics. In: Proc. 2007 ACM Symposium on Applied Computing (SAC 2007), pp. 872–877. ACM Press, New York (2007)Google Scholar
  9. 9.
    Voorhees, E.: Evaluation by Highly Relevant Documents. In: Proc. 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001), pp. 74–82. ACM Press, New York (2001)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Marco Angelini
    • 2
  • Nicola Ferro
    • 1
  • Giuseppe Santucci
    • 2
  • Gianmaria Silvello
    • 1
  1. 1.University of PaduaItaly
  2. 2.“La Sapienza” University of RomeItaly

Personalised recommendations