1 Introduction

When readers visit libraries to browse through bookshelves, they are exposed to a physical experience which enables the possibility of finding items by way of serendipitous encounters [33]. Studies have found that more than half of library visitors when browsing shelves find items that are around the items they are searching for [25], exposing them to the discovery of new materials. However, physical libraries face situations where sometimes their collections cannot be browsed physically [3], making it necessary to consider novel ways to keep providing support for such experiences of browsing and discovery.

One of the main features of physical library stacks is that they have a clearly defined sorting order. Commonly, materials are found sorted by thematic areas, which makes it easy for readers to discover similar items, but in most occasions, it is the only way in which these items are arranged. This is in contrast to digital libraries, which have advantages such as giving readers the possibility of rearranging collections based on their interests [19]. By having items in a digital space, library stacks could be rearranged by bibliographic attributes, without losing some physical features that the browsing experience affords, such as providing spatially mediated and contextualized access to the neighbors of a particular book.

The advent of virtual reality (VR) technology has made possible the implementation of virtual spaces for a myriad of applications, among these book access in libraries [16]. Given that VR brings the possibility of allowing its users to experience environments that cannot be easily recreated in the physical world, this project sets out to explore the implications of browsing and searching in library shelves using multimodal features within a virtual environment. Since browsing through library bookshelves already involves navigating through physical as well as informational space, and can be a somewhat confusing activity, some design guidelines to support book encounters while browsing digital shelves have been proposed [35] and taken into consideration for this project.

Given that the affordances of different input modalities within virtual environments have been previously analyzed [27], and that recently speech has been identified as an effective mechanism for searching within virtual environments [4], voice commands were implemented to enhance the browsing and searching experience within the environment. This project also draws design cues from the understanding of the behavior of readers when they browse physical shelves [26], and attempts to apply them in a virtual reality context.

2 Related Work

Several approaches to the digital representation of library collections have recently been implemented. Among them are Stacklife [17], which is a representation of an infinite stack of books from the Harvard Library collection, and Virtual Bookshelf from ExLibris [14], used to present library items based on their physical shelf representations. In the past, other digital visualization tools such as libViewer [30] have been used to display different book metadata attributes in virtual environments. The Search Wall [13] is also an approach aimed at kids that uses tangible elements to interact with digital items on a projected screen. Other projects aimed at encouraging book discovery are Bookfish [29], which is a web application that lets children find books based on their preferences, and Whichbook [28], which lets readers choose between several combinations of categories to discover books that fit their choices.

One of the most similar approaches to this project is the Blended Shelf [23], which is an interface that uses 3D visualization to enable readers to browse a physical collection of books from a library, using the browsing strategies they are already familiar with. This proposal differentiates itself from such projects because it is a fully immersive environment in which readers use different interaction mechanisms, VR controllers and voice commands, to perform queries. Another similar approach is the Digital Bookshelf [2] which makes use of projections and motion sensors to display book collections allowing the readers to browse through them. Other approaches have used desktop and mobile applications to visualize physical shelve contents [21], but miss the multimodal interaction possibilities that a virtual space affords.

The use of immersive environments for libraries has also been widely explored. In one of the earliest studies, Das Neves and Fox [11] evaluated the behavior of users while searching in a VR library and found that clustering techniques helped in the discovery of items within collection topics, and that presenting books in the original physical order from the library is as effective as presenting them reorganized based on queries; however, in the latter case there is less reader movement when browsing, making the task shorter. The metaphors in a three-dimensional space that can be used to arrange books on virtual spaces have also been explored [9], along with the ways in which book browsing and reading are carried out within a virtual environment [10]. Three-dimensional virtual spaces have also been used to display library document data on collaborative interfaces [7], as well as to present digital book collections using immersive hemispherical displays [1]. Also game-like approaches have been used to present library data in three-dimensional settings [8].

The use of voice commands in virtual environments has also been explored, along with the design issues for the integration of speech processing in virtual environments [24]. The voice interaction modality has been previously used to support simulations of the design and maintenance of systems and assemblies in virtual reality [36], and recent approaches have also used speech recognition to aid in maintenance tasks in what are called virtual maintenance simulations, resulting in smoother interactions with the system they are implemented in [18]. The use of voice commands has also been useful for interacting in virtual environments and facilitating room layout tasks [22]. The latter study pointed out that some of the limitations of voice interaction have to do with the fact that users have to be instructed in the commands that are needed to interact with the system beforehand, possibly generating a high cognitive load.

Although there have been few approaches that use immersive virtual environments for library data exploration [1, 8, 11], most do not use enhanced interfaces for browsing. Given that the use of voice interfaces in virtual environments has not been applied for data browsing and searching, this project seeks to investigate how virtual shelf exploration and book discovery in virtual reality can both be enhanced by the implementation of voice commands within a virtual environment.

3 Babel VR

In the Library of Babel short story [6], Argentinian writer Jorge Luis Borges presented a library setting that was made of a labyrinth of hexagonal rooms that librarians traversed endlessly. This fictional library was meant to contain all of the books that had been or will ever be written. The metaphor is powerful today because in the digital age access to materials appears to be unbounded. Thus the design of this project is based on the concept of library shelves that can be browsed in the same endless manner, and becomes a design fiction exercise inspired by Borges’ infinite library. Here we attempt to create a virtual space where readers can access a seemingly endless catalog of books within the comforting finitude of a single room filled with shelves.

One of the goals of Babel VR is to recreate the experience of shelf exploration in a virtual space and find out what it means to browse and search within an environment, in this case through a representation of a library room. According to Bates, browsing implies exploring a visual field, selecting an object from the field, examining it, and acquiring or abandoning it [5]. Thus when readers enter the Babel VR room to initiate the browsing and discovery experience, they encounter virtual stacks which are populated with MAchine-Readable Cataloging (MARC) records from an actual physical library. To enhance the shelf browsing and search experience voice recognition features have been implemented into the system. In the room, readers can also load portions of the catalog onto the shelves and search the stacks by bibliographic attributes such as title, author, topic and location, using voice commands.

3.1 VR Interaction Design

The interaction design cues for Babel VR build on McKay et al’s analysis of the physical act of browsing library shelves [26], using five of the seven guidelines they proposed to design the system as follows:

  • Display a large range of books for browsing: currently the system displays portions of 300 books distributed in 6 shelves from a 10,000 book record sample, which can be traversed with the help of voice commands.

  • Enable multiple points of access to the collection: this feature is reflected mainly in the search functionality where books appear shelved together after a keyword search has been performed, as well as the ability to “pick up” titles using VR controllers.

  • Support zooming capability: as the main setup of the environment is a library aisle, readers can walk freely through the room and move close or away from the shelves.

  • Seamless transitions: readers can freely grab books from the shelves to display their records, and they can put the virtual books back on the shelves.

  • Access to book information: data from the MARC records of the books that are selected is displayed in one of the walls of the virtual environment. This data contains fields such as title, author, topic and location in the physical library.

The two design recommendations that were not considered were the use of place marking, given that there are search features, and the use of visual alternatives during book triage, because there is no non-bibliographic data from the books such as covers or blurbs to individuate books on our virtual shelves.

Figure 1 shows the main interaction flow that readers can follow in the system. First, they have the option to register their email outside of the virtual environment to be able to send to themselves the book records they find interesting. Once in the virtual room they can walk freely around the aisle which contains six shelves with 50 books each. Then with the aid of a VR controller they can grab books from the shelves. Once they have grabbed a book the can see its record information displayed in one of the walls of the room. Each record contains four fields: title, author, topic and location. If readers are interested in the book record, they can send its information to their email just by using a voice command and without interrupting their book browsing activity.

Fig. 1.
figure 1

Babel VR interaction flow diagram

Full size image

If readers are not interested in the book they are inspecting, they can put it back on the shelf or just drop it into the floor (do not try this in physical libraries). Other voice commands that are available in the system are the search commands, where by saying either ‘title’, ‘author’ or ‘location’ plus a desired search keyword, readers can query the loaded collection and the resulting books along with their records will be stacked onto the shelves.

The voice commands that are currently implemented in the system are: the “Next” command, which is used to load sequential sections of the library catalog into the stacks; the “Title” + keyword, “Author” + keyword and “Location” + keyword commands, which allow for searching using keywords that are present in the Title, Author or Location fields from the book records; and the “Send” voice command, which enables readers to send the chosen book record to their email so that they can request the found item from the library at a later time.

One of the drawbacks of voice keyword search is that readers would have to memorize each of the voice commands; that is why they were implemented as single word commands. There can also be errors in speech command recognition due to word pronunciation or noise in the environment [15].

The virtual shelves were designed to resemble the size of typical physical shelves and are laid out in an aisle distribution setup to resemble the way books are arranged in a physical library, to provide a setting which readers are already familiar with. Readers start the virtual experience in the middle of the aisle which has a preloaded section of books from a set of MARC records that are extracted from a collection of books from a real library (Fig. 2). Then they can proceed to interact with the system using voice commands to load content in the shelves and by searching using commands and keywords in the defined bibliographic categories of title, author, topic or location, and by grabbing individual books using the VR controllers, or just “pointing” at them with the controllers.

Fig. 2.
figure 2

A library visitor holding a book in Babel VR

Full size image

3.2 System Architecture

The software architecture of Babel VR is divided into four main parts (Fig. 3). First, the Babel VR client, which loads the virtual reality environment which displays portions of the library catalog data into book representations contained in virtual shelves. The virtual environment was developed using the personal free edition of the Unity game engine [34], and the HTC Vive virtual reality system runs with the help of the SteamVR software [32]. The second element is a Google Cloud-based server where the data from the book records is stored in JSON files which have been generated from actual MARC records. The server uses the Deployd API [12], which runs on a Node.JS server, and is a web interface that provides the library data feeding mechanism, database search functionalities and email client interaction. The third component is the speech to text service from IBM Watson [20] that provides speech recognition functionality for the voice commands through their API. Finally, the fourth component which handles the email sending functionality is SparkPost [31], which provides a service to manage the sending of the book records from the virtual environment to user email accounts.

Fig. 3.
figure 3

Babel VR system architecture

Full size image

The data that is currently being used to populate the virtual library environment is a sample of 10,000 book records extracted from XML MARC records from the Arizona State University (ASU) library catalog.

4 Conclusion and Future Work

This paper presented the Babel VR system, which aims to provide an interface within a virtual environment for readers to browse virtual book shelves and create a space enabling serendipitous encounters with new materials. In the virtual environment readers have the possibility of performing shelf browsing the same way they do it on a physical library but with the added advantage of using voice commands to traverse and search within the stacks, and virtual controllers to access record information from the virtual books.

The main contribution of this project is to provide a virtual reality environment to display book records from an actual library, where readers can browse and search as part of a seamless immersive experience. The foreseen impact of this work is that it could be applied in cases where users wish to browse a library collection remotely because it is not physically accessible or is not stored in stacks at all [3]. Moreover, this work aims to enhance book discoverability in libraries by providing voice-based search commands.

Based on prior findings of visitor browsing behavior in real-world library settings, [26], we intend to conduct a qualitative field study to observe how readers behave within the virtual environment and to evaluate the implemented interaction mechanisms, focusing on voice commands for browsing and searching as well as VR controls for interacting with the virtual books. For the study, a portion of the catalog from the ASU library, which holds a subset of more than 5 million records, will be chosen. After the first study has been carried out, a quantitative study will be performed to compare the actual physical shelf browsing experience with the virtual reality experience, where readers will be provided with the same shelf browsing and searching tasks.

For every reader that tries the system, several metrics related to browsing behavior will be registered, such as the number of books they grab from the shelves, the number of books they drop or put back, and the number of records they choose to send to email. In regard to search behavior, the metrics that will be recorded are number of book searches by title, author or location.

In the future, the act of reading, sampling, and skimming books within the Babel VR virtual environment will be analyzed as well, by extending the virtual library metaphor to allow readers to open the books they grab from the shelves and read from their pages.