Abstract
This paper presents the results of the development of a system called Babel VR, part of a work-in-progress research project aimed at obtaining insights into how a multimodal virtual reality environment can enhance shelf browsing and the discovery of books that exist in a physical library. Babel VR provides a virtual browsing experience of library items which display bibliographic attributes such as title, author, topic and location in a library, aided by the use of voice commands. The representation of physical library shelves enables the possibility of having some affordances of physical browsing within a virtual environment, while at the same time providing readers with a browsing experience enhanced by multimodal features such as the use of voice commands for searching and browsing.
You have full access to this open access chapter, Download conference paper PDF
1 Introduction
When readers visit libraries to browse through bookshelves, they are exposed to a physical experience which enables the possibility of finding items by way of serendipitous encounters [33]. Studies have found that more than half of library visitors when browsing shelves find items that are around the items they are searching for [25], exposing them to the discovery of new materials. However, physical libraries face situations where sometimes their collections cannot be browsed physically [3], making it necessary to consider novel ways to keep providing support for such experiences of browsing and discovery.
One of the main features of physical library stacks is that they have a clearly defined sorting order. Commonly, materials are found sorted by thematic areas, which makes it easy for readers to discover similar items, but in most occasions, it is the only way in which these items are arranged. This is in contrast to digital libraries, which have advantages such as giving readers the possibility of rearranging collections based on their interests [19]. By having items in a digital space, library stacks could be rearranged by bibliographic attributes, without losing some physical features that the browsing experience affords, such as providing spatially mediated and contextualized access to the neighbors of a particular book.
The advent of virtual reality (VR) technology has made possible the implementation of virtual spaces for a myriad of applications, among these book access in libraries [16]. Given that VR brings the possibility of allowing its users to experience environments that cannot be easily recreated in the physical world, this project sets out to explore the implications of browsing and searching in library shelves using multimodal features within a virtual environment. Since browsing through library bookshelves already involves navigating through physical as well as informational space, and can be a somewhat confusing activity, some design guidelines to support book encounters while browsing digital shelves have been proposed [35] and taken into consideration for this project.
Given that the affordances of different input modalities within virtual environments have been previously analyzed [27], and that recently speech has been identified as an effective mechanism for searching within virtual environments [4], voice commands were implemented to enhance the browsing and searching experience within the environment. This project also draws design cues from the understanding of the behavior of readers when they browse physical shelves [26], and attempts to apply them in a virtual reality context.
2 Related Work
Several approaches to the digital representation of library collections have recently been implemented. Among them are Stacklife [17], which is a representation of an infinite stack of books from the Harvard Library collection, and Virtual Bookshelf from ExLibris [14], used to present library items based on their physical shelf representations. In the past, other digital visualization tools such as libViewer [30] have been used to display different book metadata attributes in virtual environments. The Search Wall [13] is also an approach aimed at kids that uses tangible elements to interact with digital items on a projected screen. Other projects aimed at encouraging book discovery are Bookfish [29], which is a web application that lets children find books based on their preferences, and Whichbook [28], which lets readers choose between several combinations of categories to discover books that fit their choices.
One of the most similar approaches to this project is the Blended Shelf [23], which is an interface that uses 3D visualization to enable readers to browse a physical collection of books from a library, using the browsing strategies they are already familiar with. This proposal differentiates itself from such projects because it is a fully immersive environment in which readers use different interaction mechanisms, VR controllers and voice commands, to perform queries. Another similar approach is the Digital Bookshelf [2] which makes use of projections and motion sensors to display book collections allowing the readers to browse through them. Other approaches have used desktop and mobile applications to visualize physical shelve contents [21], but miss the multimodal interaction possibilities that a virtual space affords.
The use of immersive environments for libraries has also been widely explored. In one of the earliest studies, Das Neves and Fox [11] evaluated the behavior of users while searching in a VR library and found that clustering techniques helped in the discovery of items within collection topics, and that presenting books in the original physical order from the library is as effective as presenting them reorganized based on queries; however, in the latter case there is less reader movement when browsing, making the task shorter. The metaphors in a three-dimensional space that can be used to arrange books on virtual spaces have also been explored [9], along with the ways in which book browsing and reading are carried out within a virtual environment [10]. Three-dimensional virtual spaces have also been used to display library document data on collaborative interfaces [7], as well as to present digital book collections using immersive hemispherical displays [1]. Also game-like approaches have been used to present library data in three-dimensional settings [8].
The use of voice commands in virtual environments has also been explored, along with the design issues for the integration of speech processing in virtual environments [24]. The voice interaction modality has been previously used to support simulations of the design and maintenance of systems and assemblies in virtual reality [36], and recent approaches have also used speech recognition to aid in maintenance tasks in what are called virtual maintenance simulations, resulting in smoother interactions with the system they are implemented in [18]. The use of voice commands has also been useful for interacting in virtual environments and facilitating room layout tasks [22]. The latter study pointed out that some of the limitations of voice interaction have to do with the fact that users have to be instructed in the commands that are needed to interact with the system beforehand, possibly generating a high cognitive load.
Although there have been few approaches that use immersive virtual environments for library data exploration [1, 8, 11], most do not use enhanced interfaces for browsing. Given that the use of voice interfaces in virtual environments has not been applied for data browsing and searching, this project seeks to investigate how virtual shelf exploration and book discovery in virtual reality can both be enhanced by the implementation of voice commands within a virtual environment.
3 Babel VR
In the Library of Babel short story [6], Argentinian writer Jorge Luis Borges presented a library setting that was made of a labyrinth of hexagonal rooms that librarians traversed endlessly. This fictional library was meant to contain all of the books that had been or will ever be written. The metaphor is powerful today because in the digital age access to materials appears to be unbounded. Thus the design of this project is based on the concept of library shelves that can be browsed in the same endless manner, and becomes a design fiction exercise inspired by Borges’ infinite library. Here we attempt to create a virtual space where readers can access a seemingly endless catalog of books within the comforting finitude of a single room filled with shelves.
One of the goals of Babel VR is to recreate the experience of shelf exploration in a virtual space and find out what it means to browse and search within an environment, in this case through a representation of a library room. According to Bates, browsing implies exploring a visual field, selecting an object from the field, examining it, and acquiring or abandoning it [5]. Thus when readers enter the Babel VR room to initiate the browsing and discovery experience, they encounter virtual stacks which are populated with MAchine-Readable Cataloging (MARC) records from an actual physical library. To enhance the shelf browsing and search experience voice recognition features have been implemented into the system. In the room, readers can also load portions of the catalog onto the shelves and search the stacks by bibliographic attributes such as title, author, topic and location, using voice commands.
3.1 VR Interaction Design
The interaction design cues for Babel VR build on McKay et al’s analysis of the physical act of browsing library shelves [26], using five of the seven guidelines they proposed to design the system as follows:
-
Display a large range of books for browsing: currently the system displays portions of 300 books distributed in 6 shelves from a 10,000 book record sample, which can be traversed with the help of voice commands.
-
Enable multiple points of access to the collection: this feature is reflected mainly in the search functionality where books appear shelved together after a keyword search has been performed, as well as the ability to “pick up” titles using VR controllers.
-
Support zooming capability: as the main setup of the environment is a library aisle, readers can walk freely through the room and move close or away from the shelves.
-
Seamless transitions: readers can freely grab books from the shelves to display their records, and they can put the virtual books back on the shelves.
-
Access to book information: data from the MARC records of the books that are selected is displayed in one of the walls of the virtual environment. This data contains fields such as title, author, topic and location in the physical library.
The two design recommendations that were not considered were the use of place marking, given that there are search features, and the use of visual alternatives during book triage, because there is no non-bibliographic data from the books such as covers or blurbs to individuate books on our virtual shelves.
Figure 1 shows the main interaction flow that readers can follow in the system. First, they have the option to register their email outside of the virtual environment to be able to send to themselves the book records they find interesting. Once in the virtual room they can walk freely around the aisle which contains six shelves with 50 books each. Then with the aid of a VR controller they can grab books from the shelves. Once they have grabbed a book the can see its record information displayed in one of the walls of the room. Each record contains four fields: title, author, topic and location. If readers are interested in the book record, they can send its information to their email just by using a voice command and without interrupting their book browsing activity.
If readers are not interested in the book they are inspecting, they can put it back on the shelf or just drop it into the floor (do not try this in physical libraries). Other voice commands that are available in the system are the search commands, where by saying either ‘title’, ‘author’ or ‘location’ plus a desired search keyword, readers can query the loaded collection and the resulting books along with their records will be stacked onto the shelves.
The voice commands that are currently implemented in the system are: the “Next” command, which is used to load sequential sections of the library catalog into the stacks; the “Title” + keyword, “Author” + keyword and “Location” + keyword commands, which allow for searching using keywords that are present in the Title, Author or Location fields from the book records; and the “Send” voice command, which enables readers to send the chosen book record to their email so that they can request the found item from the library at a later time.
One of the drawbacks of voice keyword search is that readers would have to memorize each of the voice commands; that is why they were implemented as single word commands. There can also be errors in speech command recognition due to word pronunciation or noise in the environment [15].
The virtual shelves were designed to resemble the size of typical physical shelves and are laid out in an aisle distribution setup to resemble the way books are arranged in a physical library, to provide a setting which readers are already familiar with. Readers start the virtual experience in the middle of the aisle which has a preloaded section of books from a set of MARC records that are extracted from a collection of books from a real library (Fig. 2). Then they can proceed to interact with the system using voice commands to load content in the shelves and by searching using commands and keywords in the defined bibliographic categories of title, author, topic or location, and by grabbing individual books using the VR controllers, or just “pointing” at them with the controllers.
3.2 System Architecture
The software architecture of Babel VR is divided into four main parts (Fig. 3). First, the Babel VR client, which loads the virtual reality environment which displays portions of the library catalog data into book representations contained in virtual shelves. The virtual environment was developed using the personal free edition of the Unity game engine [34], and the HTC Vive virtual reality system runs with the help of the SteamVR software [32]. The second element is a Google Cloud-based server where the data from the book records is stored in JSON files which have been generated from actual MARC records. The server uses the Deployd API [12], which runs on a Node.JS server, and is a web interface that provides the library data feeding mechanism, database search functionalities and email client interaction. The third component is the speech to text service from IBM Watson [20] that provides speech recognition functionality for the voice commands through their API. Finally, the fourth component which handles the email sending functionality is SparkPost [31], which provides a service to manage the sending of the book records from the virtual environment to user email accounts.
The data that is currently being used to populate the virtual library environment is a sample of 10,000 book records extracted from XML MARC records from the Arizona State University (ASU) library catalog.
4 Conclusion and Future Work
This paper presented the Babel VR system, which aims to provide an interface within a virtual environment for readers to browse virtual book shelves and create a space enabling serendipitous encounters with new materials. In the virtual environment readers have the possibility of performing shelf browsing the same way they do it on a physical library but with the added advantage of using voice commands to traverse and search within the stacks, and virtual controllers to access record information from the virtual books.
The main contribution of this project is to provide a virtual reality environment to display book records from an actual library, where readers can browse and search as part of a seamless immersive experience. The foreseen impact of this work is that it could be applied in cases where users wish to browse a library collection remotely because it is not physically accessible or is not stored in stacks at all [3]. Moreover, this work aims to enhance book discoverability in libraries by providing voice-based search commands.
Based on prior findings of visitor browsing behavior in real-world library settings, [26], we intend to conduct a qualitative field study to observe how readers behave within the virtual environment and to evaluate the implemented interaction mechanisms, focusing on voice commands for browsing and searching as well as VR controls for interacting with the virtual books. For the study, a portion of the catalog from the ASU library, which holds a subset of more than 5 million records, will be chosen. After the first study has been carried out, a quantitative study will be performed to compare the actual physical shelf browsing experience with the virtual reality experience, where readers will be provided with the same shelf browsing and searching tasks.
For every reader that tries the system, several metrics related to browsing behavior will be registered, such as the number of books they grab from the shelves, the number of books they drop or put back, and the number of records they choose to send to email. In regard to search behavior, the metrics that will be recorded are number of book searches by title, author or location.
In the future, the act of reading, sampling, and skimming books within the Babel VR virtual environment will be analyzed as well, by extending the virtual library metaphor to allow readers to open the books they grab from the shelves and read from their pages.
References
Almeida, R., Cubaud, P., Dupire, J., Natkin, S., Topol, A.: Experiments towards 3D immersive interaction for digital libraries. In: Pan, Z., Aylett, R., Diener, H., Jin, X., Göbel, S., Li, L. (eds.) Edutainment 2006. LNCS, vol. 3942, pp. 1348–1357. Springer, Heidelberg (2006). https://doi.org/10.1007/11736639_168
Aslan, I., Murer, M., Primessnig, F., Moser, C., Tscheligi, M.: The digital bookshelf: decorating with collections of digital books. In: Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, pp. 777–784. ACM (2013)
ASU Library, Books are moving out of Hayden Library. https://lib.asu.edu/librarychannel/books-are-moving-out-hayden-library. Accessed 28 Mar 2019
Badam, S.K., Srinivasan, A., Elmqvist, N., Stasko, J.: Affordances of input modalities for visual data exploration in immersive environments. In: 2nd Workshop on Immersive Analytics (2017)
Bates, M.J.: What is browsing-really? A model drawing from behavioural science research (2007)
Borges, J.L.: The Library of Babel. Collected fictions (1998)
Börner, K., Feng, Y., McMahon, T.: Collaborative visual interfaces to digital libraries. In: Proceedings of the 2nd ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 279–280. ACM (2002)
Christoffel, M., Schmitt, B.: Accessing libraries as easy as a game. In: Börner, K., Chen, C. (eds.) Visual Interfaces to Digital Libraries. LNCS, vol. 2539, pp. 25–38. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-36222-3_3
Cubaud, P., Stokowski, P., Topol, A.: 3D metaphors to access a digitalized library. In: Virtual Reality International Conference 2002, Laval, France (2002)
Cubaud, P., Stokowski, P., Topol, A.: Binding browsing and reading activities in a 3D digital library. In: Proceedings of the 2nd ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 281–282. ACM (2002)
Das Neves, F.A., Fox, E.A.: A study of user behavior in an immersive virtual environment for digital libraries. In: Proceedings of the Fifth ACM Conference on Digital Libraries, pp. 103–111. ACM (2000)
Deployd. The simplest way to build and API. http://deployd.com. Accessed 28 Mar 2019
Detken, K., Martinez, C., Schrader, A.: The search wall: tangible information searching for children in public libraries. In: Proceedings of the 3rd International Conference on Tangible and Embedded Interaction, pp. 289–296. ACM (2009)
Ex Libris. The Virtual Bookshelf. https://www.exlibrisgroup.com/the-virtual-bookshelf. Accessed 28 Mar 2019
Ferracani, A., Faustino, M., Giannini, G.X., Landucci, L., Del Bimbo, A.: Natural experiences in museums through virtual reality and voice commands. In: Proceedings of the 2017 ACM on Multimedia Conference, pp. 1233–1234. ACM (2017)
Hahn, J.F.: Virtual Reality Library Environments. American Library Association, Chicago (2017)
Harvard University. Stacklife. http://stacklife.harvard.edu. Accessed 28 Mar 2019
He, Z., Lv, C., Peng, D., Yu, D.: A speech recognition-based interaction approach applying to immersive virtual maintenance simulation. In: 2017 Second International Conference on Reliability Systems Engineering (ICRSE), pp. 1–5. IEEE (2017)
Hinze, A., McKay, D., Vanderschantz, N., Timpany, C., Cunningham, S.J.: Book selection behavior in the physical library: implications for ebook collections. In: Proceedings of the 12th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 305–314. ACM (2012)
IBM. Watson Speech to Text. https://www.ibm.com/watson/services/speech-to-text. Accessed 28 Mar 2019
Jervis, M., Masoodian, M.: Visualization of physical library shelves to facilitate collection management and retrieval. In: Proceedings of the 5th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, pp. 133–138. ACM (2013)
Kefi, M., Hoang, T.N., Richard, P., Verhulst, E.: An evaluation of multimodal interaction techniques for 3D layout constraint solver in a desktop-based virtual environment. Virtual Reality 22, 1–13 (2018)
Kleiner, E., Rädle, R., Reiterer, H.: Blended shelf: reality-based presentation and exploration of library collections. In: CHI’13 Extended Abstracts on Human Factors in Computing Systems, pp. 577–582. ACM (2013)
Kranzlmüller, D., Reitinger, B., Hackl, I., Volkert, J.: Voice controlled virtual reality and its perspectives for everyday life. Itg Fachbericht 168, 101–108 (2001)
McKay, D., Smith, W., Chang, S.: Lend me some sugar: borrowing rates of neighbouring books as evidence for browsing. In: 2014 IEEE/ACM Joint Conference on Digital Libraries (JCDL), pp. 145–154. IEEE (2014)
McKay, D., Chang, S., Smith, W.: Manoeuvres in the dark: design implications of the physical mechanics of library shelf browsing. In: Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval, pp. 47–56. ACM (2017)
Miner, N.E.: Using voice input and audio feedback to enhance the reality of a virtual experience (No. SAND–94-0670C; CONF-9406136–1). Sandia National Labs., Albuquerque, NM (United States) (1994)
Opening the Book. Whichbook. https://www.whichbook.net. Accessed 28 Mar 2019
Pearce, J., Chang, S.: Exploration without keywords: the bookfish case. In: Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future of Design, pp. 176–179. ACM (2014)
Rauber, A., Bina, H.: A metaphor graphics based representation of digital libraries on the world wide web: using the libviewer to make metadata visible. In: Proceedings of the Tenth International Workshop on Database and Expert Systems Applications, pp. 286–290. IEEE (1999)
SparkPost. SparkPost: Email Delivery Service for Developers and Enterprises. https://www.sparkpost.com. Accessed 28 Mar 2019
Steam. SteamVR. https://store.steampowered.com/steamvr. Accessed 28 Mar 2019
Thudt, A., Hinrichs, U., Carpendale, S.: The bohemian bookshelf: supporting serendipitous book discoveries through information visualization. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1461–1470. ACM (2012)
Unity. Unity Personal. https://store.unity.com/products/unity-personal. Accessed 28 Mar 2019
Waugh, S., McKay, D., Makri, S.: ‘Too much serendipity’: the tension between information seeking and encountering at the library shelves. In: Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval, pp. 277–280. ACM (2017)
Zhao, W., Madhavan, V.: Virtual assembly operations with grasp and verbal interaction. In: Proceedings of the 2006 ACM International Conference on Virtual Reality Continuum and Its Applications, pp. 245–254. ACM (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Lemos, J., Finn, E. (2019). Babel VR: Multimodal Virtual Reality Environment for Shelf Browsing and Book Discovery. In: Stephanidis, C., Antona, M. (eds) HCI International 2019 – Late Breaking Posters. HCII 2019. Communications in Computer and Information Science, vol 1088. Springer, Cham. https://doi.org/10.1007/978-3-030-30712-7_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-30712-7_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30711-0
Online ISBN: 978-3-030-30712-7
eBook Packages: Computer ScienceComputer Science (R0)