ezDL: An Interactive IR Framework, Search Tool, and Evaluation System

  • Thomas Beckers
  • Sebastian Dungs
  • Norbert Fuhr
  • Matthias Jordan
  • Georgios Kontokotsios
  • Sascha Kriewel
  • Yiannis Paraskeuopoulos
  • Michail Salampasis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8830)


ezDL is an open-source IR frontend system supporting proactivity, higher level search activities, the digital library life cycle, and collaboration of searchers. The ezDL framework is based on an extensible, service-oriented architecture, with user clients running on the desktop, in a browser or as a smartphone app. For performing user-centered evaluations, ezDL has a builtin evaluation mode that addresses many of the major challenges inherent in setting up evaluation tasks and tracking user activity during the experiments.

Currently, ezDL is employed in three major application areas. For searching computer science literature, it connects to several different digital libraries. In the medical domain ezDL provides literature search for general practitioners, as well as allowing for retrieval of medical images, including 3D data. PerFedPat is an application of ezDL in the patent retrieval domain comprising tools for supporting the International Patent Classification, faceted navigation of results, clustered views of patent search results and cross lingual retrieval.


interactive search system framework user studies 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Adams, S.: Using the international patent classification in an online environment. World Patent Information 22(4), 291–300 (2000)CrossRefGoogle Scholar
  2. 2.
    Allan, J., Callan, J., Collins-Thompson, K., Croft, B., Feng, F., Fisher, D., Lafferty, J., Larkey, L., Truong, T.N., Ogilvie, P., et al.: The Lemur toolkit for language modeling and information retrieval (2003)Google Scholar
  3. 3.
    Amershi, S., Morris, M.R.: Cosearch: A system for co-located collaborative web search. In: Proceedings of the Twenty-sixth Annual SIGCHI Conference on Human Factors in Computing Systems, CHI 2008, pp. 1647–1656. ACM, New York (2008)CrossRefGoogle Scholar
  4. 4.
    Aswani, N., Beckers, T., Birngruber, E., Boyer, C., Burner, A., Bystron, J., Choukri, K., Cruchet, S., Cunningham, H., Dedek, J., Dolamic, L., Donner, R., Dungs, S., Eggel, I., Foncubierta, A., Fuhr, N., Funk, A., Garcia, A., de Herrera, S., Gaudinat, A., Georgiev, G., Gobeill, J., Goeuriot, L., Gomez, P., Greenwood, M., Gschwandtner, M., Hanbury, A., Hajic, J., Hlavácová, J., Holzer, M., Jones, G., Jordan, B., Jordan, M., Kaderk, K., Kainberger, F., Kelly, L., Kriewel, S., Kritz, M., Langs, G., Lawson, N., Markonis, D., Martinez, I., Momtchev, V., Masselot, A., Mazo, H., Müller, H., Pecina, P., Pentchev, K., Peychev, D., Pletneva, N., Pottecher, D., Roberts, A., Ruch, P., Samwald, M., Schneller, P., Stefanov, V., Tinte, M.A., Uresová, Z., Vargas, A., Vishnyakova, D.: Khresmoi professional: Multilingual semantic search for medical professionals. In: ACM SIGIR Workshop on Health Search and Discovery: Helping Users and Advancing Medicine (2013)Google Scholar
  5. 5.
    Bangor, A., Kortum, P., Miller, J.: Determining what individual sus scores mean: Adding an adjective rating scale. Journal of Usability Studies 4(3), 114–123 (2009)Google Scholar
  6. 6.
    Bates, M.J.: Idea tactics. Journal of the American Society for Information Science 30(5), 280–289 (1979)CrossRefGoogle Scholar
  7. 7.
    Bates, M.J.: Information search tactics. Journal of the American Society for Information Science 30(4), 205–214 (1979)CrossRefGoogle Scholar
  8. 8.
    Bates, M.J.: Where should the person stop and the search interface start? Information Processing and Management 26(5), 575–591 (1990)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Beckers, T., Dungs, S., Fuhr, N., Jordan, M., Kriewel, S.: ezDL: An interactive search and evaluation system. In: SIGIR 2012 Workshop on Open Source Information Retrieval (OSIR 2012) (August 2012)Google Scholar
  10. 10.
    Beckers, T., Fuhr, N.: User-oriented and eye-tracking-based evaluation of an interactive search system. In: 4th Workshop on Human-Computer Interaction and Information Retrieval (HCIR 2010) @ IIiX 2010 (2010)Google Scholar
  11. 11.
    Beckers, T., Fuhr, N.: Search system functions for supporting search modes. In: 2nd European Workshop on Human-Computer Interaction and Information Retrieval (EuroHCIR) @ IIiX 2012 (August 2012)Google Scholar
  12. 12.
    Beckers, T., Korbar, D.: Using eye-tracking for the evaluation of interactive information retrieval. In: Geva, S., Kamps, J., Schenkel, R., Trotman, A. (eds.) INEX 2010. LNCS, vol. 6932, pp. 236–240. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  13. 13.
    Bierig, R., Gwizdka, J., Cole, M.: A user-centered experiment and logging framework for interactive information retrieval. In: Understanding the user - Workshop in Conjuction with SIGIR (2009)Google Scholar
  14. 14.
    Brooke, J.: SUS: A quick and dirty usability scale. In: Usability Evaluation in Industry, pp. 189–194 (1996)Google Scholar
  15. 15.
    Chau, M., Wong, C.H.: Designing the user interface and functions of a search engine development tool. Decision Support Systems 48(2), 369–382 (2010)CrossRefGoogle Scholar
  16. 16.
    Fafalios, P., Kitsos, I., Marketakis, Y., Baldassarre, C., Salampasis, M., Tzitzikas, Y.: Web Searching with Entity Mining at Query Time. In: Salampasis, M., Larsen, B. (eds.) IRFC 2012. LNCS, vol. 7356, pp. 73–88. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  17. 17.
    Fafalios, P., Salampasis, M., Tzitzikas, Y.: Exploratory patent search with faceted search and (configurable) entity mining. In: Proceedings of the Integrating IR Technologies for Professional Search (in conjuction with ECIR 2013) (2013)Google Scholar
  18. 18.
    Fuhr, N., Klas, C.-P., Schaefer, A., Mutschke, P.: Daffodil: An integrated desktop for supporting high-level search activities in federated digital libraries. In: Agosti, M., Thanos, C. (eds.) ECDL 2002. LNCS, vol. 2458, pp. 597–612. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  19. 19.
    Golovchinsky, G., Diriye, A., Dunnigan, T.: The future is in the past: Designing for exploratory search. In: Proceedings of the 4th Information Interaction in Context Symposium, IIIX 2012, pp. 52–61. ACM, New York (2012)Google Scholar
  20. 20.
    Hall, M.M., Katsaris, S., Toms, E.: A pluggable work-bench for creating interactive ir interfaces. In: Proceeding of the the 3rd European Workshop on Human-Computer Interaction and Information Retrieval, EuroHCIR 2013 (2013)Google Scholar
  21. 21.
    Hanbury, A., Belle, W., Lawson, N., Dolamic, L., Pletneva, N., Samwald, M., Boyer, C.: Prototype of a first search system for intensive tests. Deliverable D8.2, Khresmoi (2012)Google Scholar
  22. 22.
    Harter, S.P.: Online information retrieval: concepts, principles, and techniques. Academic Press Professional, Inc., San Diego (1986)Google Scholar
  23. 23.
    Jansen, B.J., Pooch, U.: Assisting the searcher: utilizing software agents for web search systems. Internet Research: Electronic Networking Applications and Policy 14(1), 19–33 (2004)CrossRefGoogle Scholar
  24. 24.
    Klas, C.-P., Gövert, N., Fuhr, N.: Distributed agents for user-friendly access of digital libraries. In: Searching for Information: Artificial Intelligence and Information Retrieval Approaches, Savoy Place, London WC2R 0BL, UK, vol. 17, pp. 1–3. IEE Informatics (1999) ISSN 0963-3308Google Scholar
  25. 25.
    Klas, C.-P., Kriewel, S., Schaefer, A.: Daffodil - Nutzerorientiertes Zugangssystem für heterogene digitale Bibliotheken. dvs Band (2004)Google Scholar
  26. 26.
    Kopidaki, S., Papadakos, P., Tzitzikas, Y.: STC+ and NM-STC: Two Novel Online Results Clustering Methods for Web Searching. In: Vossen, G., Long, D.D.E., Yu, J.X. (eds.) WISE 2009. LNCS, vol. 5802, pp. 523–537. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  27. 27.
    Kriewel, S.: Unterstützung beim Finden und Durchführen von Suchstrategien in Digitalen Bibliotheken. PhD thesis, University of Duisburg-Essen (2010)Google Scholar
  28. 28.
    Kriewel, S., Fuhr, N.: Evaluation of an adaptive search suggestion system. In: Gurrin, C., He, Y., Kazai, G., Kruschwitz, U., Little, S., Roelleke, T., Rüger, S., van Rijsbergen, K. (eds.) ECIR 2010. LNCS, vol. 5993, pp. 544–555. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  29. 29.
    Kriewel, S., Klas, C.-P., Schaefer, A., Fuhr, N.: Daffodil - strategic support for user-oriented access to heterogeneous digital libraries. D-Lib Magazine 10(6) (June 2004),
  30. 30.
    Markonis, D., Donner, R., Holzer, M., Schlegl, T., Dungs, S., Kriewel, S., Langs, G., Müller, H.: A visual information retrieval system for radiology reports and the medical literature. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds.) MMM 2014, Part II. LNCS, vol. 8326, pp. 390–393. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  31. 31.
    Muno, S.: Entwicklung eines Android-Frontends fü ezDL. Diplomarbeit. Universität Duisburg-Essen (2013)Google Scholar
  32. 32.
    Muno, S., Beckers, T., Kriewel, S.: Konzeption und Implementierung einer Android-App für das ezDL-System. In: Proceedings of the IR Workshop at Lernen, Wissen, Adaption (LWA 2013) (October 2013)Google Scholar
  33. 33.
    Nottelmann, H., Fuhr, N.: Evaluating different methods of estimating retrieval quality for resource selection. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval, SIGIR 2003, pp. 290–297. ACM, New York (2003)CrossRefGoogle Scholar
  34. 34.
    Nuray, R., Can, F.: Automatic ranking of information retrieval systems using data fusion. Information Processing & Management 42(3), 595–614 (2006)CrossRefzbMATHGoogle Scholar
  35. 35.
    Ounis, I., Amati, G., Plachouras, V., He, B., Macdonald, C., Johnson, D.: Terrier information retrieval platform. In: Losada, D.E., Fernández-Luna, J.M. (eds.) ECIR 2005. LNCS, vol. 3408, pp. 517–519. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  36. 36.
    Paepcke, A.: Digital libraries: Searching is not enough–what we learned on-site. D-Lib Magazine 2(5) (1996),
  37. 37.
    Pharo, N., Beckers, T., Nordlie, R., Fuhr, N.: Overview of the inex 2010 interactive track. In: Geva, S., Kamps, J., Schenkel, R., Trotman, A. (eds.) INEX 2010. LNCS, vol. 6932, pp. 227–235. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  38. 38.
    Salampasis, M., Giachanou, A.: Multilayer collection selection and search of topically organized patents. In: Proceedings of the Integrating IR Technologies for Professional Search (in conjuction with ECIR 2013), pp. 48–56 (2013)Google Scholar
  39. 39.
    Salampasis, M., Paltoglou, G., Giahanou, A.: Report on the clef-ip 2012 experiments: Search of topically organized patents. In: Forner, P., Karlgren, J. (eds.) CLEF (Online Working Notes/Labs/Workshop) (2012)Google Scholar
  40. 40.
    Salton, G., Buckley, C.: Term-weighting approaches in automatic text retrieval. Information Processing & Management 24(5), 513–523 (1988)CrossRefGoogle Scholar
  41. 41.
    Schaefer, A., Jordan, M., Klas, C.-P., Fuhr, N.: Active support for query formulation in virtual digital libraries: A case study with DAFFODIL. In: Rauber, A., Christodoulakis, S., Tjoa, A.M. (eds.) ECDL 2005. LNCS, vol. 3652, pp. 414–425. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  42. 42.
    Shokouhi, M., Si, L.: Federated search. Found. Trends Inf. Retr. 5(1), 1–102 (2011)CrossRefGoogle Scholar
  43. 43.
    Tacke, A., Kriewel, S.: Strategische Suchunterstützung auf Makro- und Mikroebene. In: Proc. IR Workshop at Lernen, Wissen, Adaption (LWA 2013) (Oktober 2013)Google Scholar
  44. 44.
    Tacke, A., Kriewel, S.: Strategic search support on macro and micro level. In: Datenbank-Spektrum (2014)Google Scholar
  45. 45.
    Tran, V.T., Fuhr, N.: Markov modeling for user interaction in retrieval. In: SIGIR 2013 Workshop on Modeling User Behavior for Information Retrieval Evaluation (MUBE 2013) (August 2013)Google Scholar
  46. 46.
    Tran, V.T.: Entwicklung einer Unterstützung für Pearl Growing. Masters thesis, University of Duisburg-Essen (2011)Google Scholar
  47. 47.
    Tran, T.V., Fuhr, N.: Quantitative analysis of search sessions enhanced by gaze tracking with dynamic areas of interest. In: Zaphiris, P., Buchanan, G., Rasmussen, E., Loizides, F. (eds.) TPDL 2012. LNCS, vol. 7489, pp. 468–473. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  48. 48.
    Tran, V.T., Fuhr, N.: Using eye-tracking with dynamic areas of interest for analyzing interactive information retrieval. In: Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM (August 2012)Google Scholar
  49. 49.
    Vijvers, W.G.: The international patent classification as a search tool. World Patent Information 12(1), 26–30 (1990)CrossRefGoogle Scholar
  50. 50.
    Virzi, R.A.: Refining the test phase of usability evaluation: How many subjects is enough? Hum. Factors 34(4), 457–468 (1992)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Thomas Beckers
    • 1
  • Sebastian Dungs
    • 1
  • Norbert Fuhr
    • 1
  • Matthias Jordan
    • 1
  • Georgios Kontokotsios
    • 2
  • Sascha Kriewel
    • 1
  • Yiannis Paraskeuopoulos
    • 2
  • Michail Salampasis
    • 2
  1. 1.Information EngineeringUniversity of Duisburg-EssenDuisburgGermany
  2. 2.Information and Software Engineering, Vienna University of TechnologyAustria

Personalised recommendations