Usability Evaluation of the Digital Library DanteSources
- 2.2k Downloads
In this paper we present DanteSources, a Digital Library of Dante Alighieri’s primary sources, i.e. the works of other authors that Dante cites in his texts. Currently, this information is scattered in many books, making it difficult for the scholars to retrieve it and also to produce a systematical overview of the cultural background of Dante. In order to overcome this problem, an ontology expressed in RDF/S was developed to represent this knowledge. Once the ontology had been defined, we populated it with the data included in authoritative commentaries to Dante’s works. We stored the resulting RDF graph into a Virtuoso triple store. Finally, on top of this graph, we developed DanteSources, which allows users to extract and display the knowledge stored in the knowledge base in the form of charts and tables. In this paper we present the results of a survey to collect suggestions and comments from end-users on their interactions with DanteSources in order to evaluate its usability.
KeywordsDigital libraries Digital libraries usability Semantic web Human-computer interaction
In the domain of humanities, scholars analyze old literary works in order to derive several kinds of knowledge. One important kind of such knowledge regards the primary sources, i.e. the works of other authors that an author makes reference to in his texts. Usually, scholars encode this knowledge in natural language, and report it in commentaries, which the readers use to understand several aspects of the literary works. However, encoding knowledge in natural language does not allow using computers to make inferences that could be useful to scholars, for example about the distribution of the primary sources and of the authors cited in a work.
Our study is part of the “Towards a Digital Dante Encyclopedia” project, an Italian National Research project supporting scholars, amongst others, in formally expressing the knowledge about primary sources present in Dante’s works and more generally in literary texts. The goal of our work is to represent this kind of knowledge by (i) creating an ontology providing a formal representation of the terms required for expressing knowledge about the primary sources of an author’s works, and (ii) developing a semantic digital library based on the ontology that allows scholars to make inferences on the collected data and visualize them in a friendly and easy way.
In the Digital Humanities literature, there are ontologies focusing on different aspects of textual information. Each of these ontologies represents a set of possible interpretations of the source text(s). Up to now, an ontology for representing knowledge about primary sources of literary texts has not been developed. In the previous stages of the project, we developed an ontology  using the RDF language  to represent Dante’s works and the knowledge about their primary sources. Then, we populated the ontology with the knowledge extracted from some authoritative commentaries, obtaining an RDF knowledge base. On top of this knowledge base, we developed DanteSources1, a focused Digital Library (DL) endowed with web services that allow visualizing information on Dante Alighieri’s primary sources in form of charts and tables. In this paper we present the results of a survey, developed using Google Docs, to collect suggestions and comments from end-users on their interactions with DanteSources in order to evaluate its usability.
The paper is structured as follows: Sect. 2 reports a review of related works; in Sect. 3 the functionalities of DanteSources are described. Section 4 presents the results of the evaluation and a discussion of them. Finally, in Sect. 5 we report our conclusive remarks.
2 Related Works
In the last thirty years several major Digital Humanities projects about Dante Alighieri and his works were published online. Most of them focused on offering digital editions of Dante’s works with commentaries, textual search, and multimedia.
The Dartmouth Dante Project (DDP)2 was launched in 1985 with the aim of publishing the full searchable text of the Divine Comedy and several of its commentaries in digital format . On top of the DDP in 2013 the web application Dante Lab3 was developed. This application allows the concurrent visualization of the original text of the Comedy, some translations i.e. English, French and German, and more than 75 commentaries, each of them fully searchable. The Princeton Dante Project4 includes the full searchable text of the Divine Comedy, Dante’s minor works, several commentaries and multimedia resources . The World of Dante5 is a multi-media research tool that contains the whole text of the Divine Comedy with a basic semantic annotation that identifies people, places, deities and structures, and connections to digital objects such as images and music. However, the semantic knowledge is not represented through a formal ontology and is not available in a machine-readable format. An interactive timeline of Dante’s life and relevant maps are also available . Digital Dante6 offers the full text of all Dante’s works with commentaries, illustrations and recorded readings. It also features a subproject called Intertextual Dante7, the first digital attempt at connecting passages of Dante’s works with the corresponding fragments of cited primary sources. However, the project is limited to the works of the Roman poet Ovid . Dante Online8 includes the full text of all Dante’s works, a biography with an interactive timeline, and a database listing hundreds of manuscripts of the Divine Comedy with bibliographic information. Danteworlds9 is a multimedia project built around the Divine Comedy, featuring illustrations by historical and contemporary artists, audio lectures and study resources. More recently, two major Italian research projects were published: DanteSearch10 and Dante Medieval Archive (DaMA)11. DanteSearch is a complete lemmatization, grammatical and syntactic annotation of Dante’s works allowing users to perform morphological and syntactic queries on the full text of the author’s works. On the other hand, DaMA is a digital archive containing the full text of Dante’s works, commentaries and several primary sources in XML-TEI format. With the advent of the Semantic Web, several projects were started with a focus on the application of the new semantic technologies (RDF, OWL12) to the Humanities. For instance, González-Blanco et al.  describe a semantic model to connect repertoires of poetic writings. Vitale  presents an ontology for the 3D visualization of cultural heritage; Lana et al.  describe an ontology for annotating geographical places in texts. In this context, we developed DanteSources DL, which uses the technologies of the Semantic Web to represent the knowledge included in the works of Dante Alighieri, focusing on primary sources.
3 The Digital Library DanteSources
Finally, for what concerns the thematic areas, clicking on one name the user obtains the following information for each thematic area: the primary sources included in that area, their authors and the book, chapter and paragraph (or poem) of the Dante’s work where the thematic area is cited.
- (1)An autocomplete menu where the user can type the title of the cited work, the author, or the thematic area;
An alphabetically ordered list in which the user can select the title of the primary source, the author, or the thematic area.
The data regarding the distributions are not only available for an entire Dante’s work, like Convivio, but also for its subparts like books, chapters or poems. Indeed, by clicking on one bar in the chart representing the distribution of the information onto a Dante’s work, it is possible to visualize information about its subparts. For example, it is possible to visualize the distribution of the selected information on all the books of Convivio and also its distribution on each chapter of the second book of Convivio. Figure 4 shows three views of the distribution of a primary source on three different levels: all Dante’s works (a), one particular Dante’s work (b), a subpart of a Dante’s work (c).
explicit, if the reference is explicitly made by Dante, e.g. “As the Philosopher says at the beginning of the First Philosophy”, where the Philosopher is Aristotle and the First Philosophy is Metaphysics;
strict, if the reference is indicated by a scholar and refers to a specific work, e.g. “SI MANUCA: it is the bread of the angels, the manna as called in the Old Testament (Ps. 77, 25 Panem angelorum manducavit homo)”;
generic, if the reference is indicated by a scholar, and refers to a concept (e.g. Medieval comments to Aristotle’s works).
As for the second group of queries, by clicking on the bars of the chart, the data regarding the distribution of the types of reference are available both for an entire Dante’s work and for its subparts, i.e. books and chapters.
4.1 The Online Survey
The survey included 28 questions about the main aspects regarding the usability of the DL. After a brief overview to identify the sample, we focused our attention on the features of the DL: the navigation usability, the readability of the charts and tables, the usability of the search functionalities, the satisfaction of the results obtained doing a query. A specific attention was put on the effective responsiveness of the layout and on the use of the DL on different devices, e.g. PCs, smartphones, tablets. Questions were presented through multiple choice and text area. The same questions required a personal judgment on selected features. A Likert 5-scale values from 1 (totally negative) to 5 (totally positive) was used to express the opinion.
We collected suggestions and comments from 26 users; some of them did not answer all the questions. Our sample included 18 females and 10 males. 82 % of the participants were in the range of 31–50 years old, 14 % were people between 10 and 30 years old and 3 % were in the range of 51–65 years old.
25 % of the users were students, 20 % were researchers, 12 % were university professors, 4 % were school teachers and the 37 % of the users did not specify their profession. Concerning technological skills, 51 % declared to regularly use smartphones with android system, 18 % an iPhone or iPod touch, 11 % an iPad, 11 % a tablet with Android system, 7 % of the users responded that they never use mobile devices. 100 % of the users said that they use the Internet at least once per day to search information about their interest fields.
As far as the readability of the obtained charts is concerned, 51.9 % of the users assigned score 5, 25.9 % score 4, 14.8 % score 3, and the remaining 7,4 % score 2. No one chose score 1. In particular, the users expressed a positive opinion on the possibility to export in CSV format all the data visualized into the charts and on the possibility of visualizing ten results at a time using the scrollbar. On the other hand, some comments reported the continuous use of the scrollbar as a limit, when the results are many.
Then, we asked to evaluate the tables which the user can access from each chart and to leave a textual comment to describe the main observed problems. Regarding the charts about primary sources, several users reported the usefulness and the completeness of the knowledge presented in the tables. Some users highlighted the lack of text visualization for the cited fragment within the text of the primary source. One user said that it is not intuitive that clicking on the title of the primary sources in the chart, it is possible to visualize a table. This possibility is described in natural language below the chart but this user suggested that it could be more useful having it above. As to the tables regarding the cited authors, the majority of the comments reported that the only missing information is the textual fragment of the primary sources.
For what concerns the tables about the thematic areas, many comments reported the lack of a textual explanation on how the thematic areas were defined, as well as the absence of the textual fragment of the primary sources.
Regarding the position of the information on the screen, 25.9 % of the users assigned score 5, 48.1 % score 4, 22.2 % score 3, 3.7 % score 2. No one expressed a totally negative opinion (score 1). The main reported problem regards the presence of too much blank space between the search forms and the results, which forces the users to vertically scroll the page. Furthermore, some comments reported that the explanations about the functionalities of the charts should have a more relevant position on the page.
Only 18.5 % of the users accessed DanteSources from a mobile device. The used mobile devices were: smartphone with Android system, iPhone, iPad. 57.1 % of this 18.5 % declared that interaction on a mobile device was very easy and clear (score 5), the remaining 42.9 % assigned score 4. No problems were reported in the comments.
Regarding the data obtained through the search functionalities, 61.5 % of the users said that they obtained all the information they needed, while 34.6 % reported that they obtained a lot of information, although they would like to receive additional data. Only one user said that s/he did not retrieve the information s/he needed. This last user specified that s/he is a linguist and for this reason s/he would need to other types of information, like Named Entities, parsing of the texts and collocations. Unfortunately, our project was focused not on the syntax but on the semantics of the primary sources. Many users expressed the desire to visualize the text of the fragments of the primary sources as well as, on request, the entire text of the primary sources and of the Dante’s works. Other comments suggested visualizing the entire text of the commentaries. Unfortunately, they are protected by copyright, so it is not possible to show them.
We asked the user about the major problems encountered using DanteSources. Several users reported that they did not have significant problems, few users suggested making the information more compact on the screen and one user said that the interactive image on the homepage is not intuitive to use.
Finally, the users left suggestions to improve the DL. The three main suggestions regarded (i) the possibility of visualizing the texts of the cited fragments, as well as the entire texts of the primary sources and of the Dante’s works; (ii) the introduction of the possibility to directly visualize the information on the primary sources written by a specific cited author; (iii) the enrichment of the knowledge base with other Dante’s works, especially with the Divine Comedy.
reducing blank spaces in the layout, especially between the search forms and the results and moving the explanation of additional features of the charts in a more evident position;
visualizing the fragments of the primary sources and, upon request, the whole texts of the primary sources and of the Dante’s works;
visualizing the data shown in the charts also in the form of tables or lists;
making available the corresponding works when the users search for a specific author;
enriching the knowledge base with other Dante’s works.
Regarding the (a) suggestion, we have planned to reduce the blank spaces and modify the position of the explanation of the charts’ additional features. About the (b), we have already developed a function that for each Dante’s fragment allows visualizing the fragment of the corresponding primary source. Up to now we implemented this function for the primary sources included in DaMA, since this archive contains several primary sources coded in XML-TEI format. Regarding the suggestion to also visualize the entire texts of Dante’s works, these are already preset in a dedicated page of the DL as external links to DaMA. However, the users didn’t find them easily. For this reason, we have planned to add this links also in the pages of the search results. Concerning the suggestion (c), the users did not consider the possibility to export the data in CSV format in order to have the results in form of simple table. We supposed that the CSV could be a format not very well known and used in the Humanities, so we have thought to replace the current label “Export in CSV” into a more clear label like “Export as a table (in CSV format)”. About request (d), when a user searches for a specific cited author, we have decided to add a feature that shows a table of that author’s works. Regarding the suggestion (e), at this moment we are working to add Rime  in our knowledge base.
In this paper we have presented a survey conducted on the interaction with the DanteSources Digital Library via PCs and mobile devices to collect suggestions and comments on possible usability issues observed by the users.
We have had a total of 26 users. Positive responses on the general interaction have emerged from the survey: more than 60 % of the users have expressed a distinctly positive response. About 80 % have expressed a satisfactory opinion on the navigation (score 4 and 5) and on the readability of charts and tables (score 4 and 5) as well. 100 % of the users who accessed DanteSources from a mobile device have declared that interaction was easy and clear (score 4 and 5). On the other hand, the users have identified some problems and lacks in the interface, in particular: the presence of too much blank space between the search forms and the results, the hidden position of the explanation of the charts’ functionalities, the lack of text visualization for the cited fragments and for the entire texts of the primary sources and Dante’s works. Furthermore, the users have suggested adding a functionality that allows visualizing the list of works when the users search for a specific author. The users have also suggested enriching the knowledge base with other Dante’s works, especially with Divine Comedy. We have planned to modify and enrich DanteSources taking into account the problems highlighted by the users and the given suggestions.
The Dante’s experts who have taken part in our evaluation have said that having the information about primary sources available, for the first time, in digital format, improves and makes more efficient their searches. Indeed, in their commentaries, scholars typically express knowledge on the primary sources in natural language. This limits scholars in their advances insofar as it prevents automatic inferences of new information that may be useful for their studies. For instance, in the case of references to primary sources, the inferences may concern the total amount of references to a certain work or to an author. Eventually, the digitization of the knowledge about primary sources allows experts to have a comprehensive overview of the data, to visualize it as charts and tables, and to export it to make their own analyses. For this reason, DanteSources allows scholars to explore the dynamics of the multi-faceted culture of Dante in relation to the diverse stages of his biography and to study the evolution in time of Dante’s cultural background.
- 1.Alighieri, D.: Vita nuova. In: De Robertis, D. (ed.) Milano, Napoli, Ricciardi (1980)Google Scholar
- 2.Alighieri, D.: De vulgari eloquentia. In: Tavoni, M. (ed.) Opere, vol. I: Rime, Vita Nova, De vulgari eloquentia. Mondadori, Milano (2011)Google Scholar
- 3.Alighieri, D.: Vita Nova. In: Gorni, G. (ed.) Opere, vol. I: Rime, Vita Nova, De vulgari eloquentia. Mondadori, Milano (2011)Google Scholar
- 4.Alighieri, D.: Rime. In: Giunta, C. (ed.) Opere, vol. I: Rime, Vita Nova, De vulgari eloquentia. Mondadori, Milano (2011)Google Scholar
- 5.Alighieri, D.: Convivio. In: Fioravanti, G. (ed.) Opere, vol. II: Convivio, Monarchia, Epistole, Egloghe. Mondadori, Milano (2014)Google Scholar
- 6.Alighieri, D.: Monarchia. In: Quaglioni, D. (ed.) Opere, vol. II: Convivio, Monarchia, Epistole, Egloghe. Mondadori, Milano (2014)Google Scholar
- 7.Bartalesi, V., Meghini, C.: Using an ontology for representing the knowledge on literary texts: the Dante Alighieri case study. Seman. Web, 1–10 (2015, Preprint)Google Scholar
- 8.Erling, O., Mikhailov, I.: RDF support in the Virtuoso DBMS. GI-Edition-Lecture Notes in Informatics (LNI), vol. P-113 (2007). ISSN: 1617-5468Google Scholar
- 9.González-Blanco, E., Seláf, L., Del Rio Riande, M.G., Martínez Cantón, C.I., Martos Pérez, M.D.: Building a metrical ontology as a model to link digital poetic repertoires (2014)Google Scholar
- 10.Hollander, R.: The Dartmouth Dante Project. Quad. dʼitalianistica 10(1–2), 287–298 (1989)Google Scholar
- 12.Lana, M., Ciotti, F., Magro, D., Peroni, S., Tomasi, F., Vitali, F.: Annotating texts with ontologies, from geography to persons and events (2014)Google Scholar
- 13.Manola, F., Miller, E., McBride, B.: RDF primer. W3C Recommendation 10(1–107), 6 (2004)Google Scholar
- 15.Van Peteghem, J.: Digital readers of allusive texts: ovidian intertextuality in the ‘Commediaʼ and the digital concordance on ‘Intertextual Danteʼ. Humanist Stud. Digital Age 4(1), 39–59 (2015)Google Scholar
- 16.Vitale, V.: An ontology for 3D visualisation of cultural heritage (2014)Google Scholar