Abstract
We present an educational application of virtual reality that we created to help students gain an in-depth understanding of the internal structure of crystals and related key concepts. Teachers can use it to give lectures to small groups (10-15) of students in a shared virtual environment, both remotely (with teacher and students in different locations) and locally (while sharing the same physical space). Lectures can be recorded, stored in an online repository, and shared with students who can either review a recorded lecture in the same virtual environment or can use the application for self-studying by exploring a large collection of available crystal structures. We validated our application with human subjects receiving positive feedback.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
Crystals are ubiquitous in nature and represent a key element in several scientific fields such as mineralogy, chemistry, physics, metallurgy, geology, biology, and medicine [8]. A crystal is defined by its unit cell and its lattice geometry. The former specifies the arrangement of atoms in the material and represents the minimum crystal size possible. The latter describes how the unit cells are orderly arranged in three dimensions to make up the crystal structure. Unit cells lie in layers or planes that are identified by their Miller indices, a triplet of integer values (hkl) that determine the vector normal to the plane. Figure 1 shows, for the simple sodium chloride crystal (NaCl), (a) its unit cell, (b) unit cells positioned on the lattice geometry, the planes corresponding to (c) Miller index 100 (the one having x is the vector normal to the plane), and (d) Miller index (111).
Sodium Chloride (NaCl) (a) unit cell; (b) unit cells on the crystal lattice; (c) the plane corresponding to the 100 Miller index; and (d) the plane corresponding to the 111 Miller index. Images have been created using nanoHUB Crystal Viewer [16]
Crystals form complex three-dimensional structures that can be difficult to visualize using traditional two-dimensional representations like images in books or flat displays. Accordingly, students usually find it difficult to understand the spatial arrangement of the atoms and to master how some key concepts, like Miller indices, map on complex crystal structures. Physical artifacts (made out of plastic, metal or wood) are more effective than two-dimensional images; however, they are bulky to store, time-consuming to build and modify. There are several commercial and open source molecular visualizers (e.g., Crystal Viewer [16]) and some of them support virtual reality head sets (e.g., [18]). Academia has also been very active in this area with studies focused on applying computer graphics to achieve more accurate depictions of the chemical compounds and their properties while also reducing the rendering speed [44]. Several applications also deal with teaching crystals-related concepts to students [9]. However, most visualizers are single-user and the ones that have multi-user support focus on collaborative tasks rather than teaching [14, 31].
In this paper, we present an educational application of virtual reality we created to help students understand crystal structures either (i) by attending online lectures that take place in a shared virtual environment or (ii) by exploring a library of crystals structures. Immersive virtual environments have been shown to help students grasp more clearly the 3D shapes of objects [62, 63]. Existing tools mainly focus on self study (e.g. [9]) and collaborative problem solving (e.g., [14, 31]). In contrast, our application has been designed specifically as a support for online university lectures with a focus on the student-teacher class interaction and after-class activities. Teachers can use it (i) to give online lectures to small groups (10-15) of students in a shared virtual environment, and (ii) to record lectures that students can later watch in the same virtual environment. Students can use it as a self-study tool that let them explore a library of crystal structures and recorded lectures. We designed the application in collaboration with professors who, over the last decade, have been teaching crystallography lectures for the“Solid State Physics” course at Politecnico di Milano. Our colleagues helped us identify the functionalities useful to teachers and students during online lectures, for recording a lecture, and for after-class activities. They provided us with an extensive list of actions that users should be able to perform when interacting with crystal structures. Finally, since cost and ease of use are essential factors, they asked us to support inexpensive and user-friendly devices. To the best of our knowledge, this is the first educational crystallography application to provide support for online lectures in a virtual environment to small student classes for low-cost stand-alone devices.
We evaluated our application with 30 human subjects during two events, one involving engineering students, one involving people attending a public scientific dissemination event. The evaluation focused on usability, the effectiveness in conveying the spatial arrangement of atoms inside crystal structures, the fundamental concepts of crystal structures (like Miller indices), and a qualitative assessment of issues related to motion sickness. Subjects received a brief introduction about how the experience would unfold and how to use the controller. Next, they wore the headset and attended a brief virtual reality lecture covering fundamental crystallography concepts and the available functionalities. After the lecture, subjects were allowed to explore the library of crystals on their own. The experience lasted for about 15 minutes and, at the end, we asked subjects to complete an anonymous online form with 27 questions asking for (i) basic background information; (ii) feedback regarding usability, comfort, and educational value of the application using Likert scale [35] or True/False values; and (iii) general suggestions and comments using open questions. Overall, we received very positive feedback suggesting that our application can be an effective mean to convey the spatial arrangement of atoms inside crystals and student would enjoy and benefit from having it as a support to crystallography lectures.
The paper is organised as follows. In Section 2, we provide an overview of the recent research on molecular visualisers and virtual reality applications for teaching crystals-related topics. We describe the application in detail in Section 3 and its high-level software architecture in Section 4. In Section 5, we present the results of an evaluation we carried out both with engineering students and general public. Finally, in Section 6, we draw some conclusions and outline future works.
2 Related work
Virtual reality enables users to experience situations that would either be impossible in real life (like doing a space walk [55]), too dangerous (like training as a firefighter [11, 12, 70]), or too expensive [75]. Several studies have shown that skills gained in virtual simulations may be successfully transferred to real life situations [42, 53] and retained over a longer period of time [10]. Accordingly, virtual reality has been applied to training and learning in a broad spectrum of domains including industrial processes and security [38, 40], aviation safety [10], cultural heritage [3, 4], military training [54, 69], Internet of Things [77] surgery [1, 59, 65, 71], geography [39], safety procedures [10], and social interactions [36, 37, 64, 72, 73]. Virtual reality has also been widely applied to education and a thorough survey of the research in such broad area is outside the scope of this paper. Accordingly, we refer the interested reader to the recent surveys for a more complete overview of this area [21, 24, 28, 52, 61].
In this section, we present a brief overview of the most relevant works on the visualization and teaching of chemical structures published over the last ten years. Table 1 summarizes the works we selected organized by type of structure and purpose. Research in this area focuses either (i) on biology, thus on the visualization and simulation of protein structures (Table 1, column Biology) or (ii) on the visualization of crystal structures (Table 1, column Crystallography). In the former case, data are represented using the Protein Data Bank (PDB) format [6], in the latter, data are represented using the Crystallographic Information File (CIF) [7]. The goal is usually the scientifically accurate visualization of such complex structures, the simulation of interactions, or the support for teaching (see the rows in Table 1).
2.1 Biology - visualization
Interactive visualization of molecular structures dates back to a time when computer did not exist [20] and, as such, it represents one of the oldest branches of data visualization. Kozlíková et al. [32] presents an extensive overview of representation models, rendering, visualization, and simulation for structural biology. The review discusses the major technical challenges while it does not consider virtual reality nor augmented reality. Most of the visualization tools in this area focus on scientific accuracy and typically support flat displays like for example, PyMOL [60], cellVIEW [43], and Chimera [48]. Some tools support virtual reality headsets like the recent versions of VMD [29] and ChimeraX [49], the evolution of Chimera [48]. Molecular Rift [45] integrates virtual reality with Microsoft Kinect V2Footnote 1 to enable hand-based interactions; Caffeine [57] uses a more expensive CAVE-based approach. Fewer tools exploit augmented reality like for example ChemPreview [76], based on Meta 1Footnote 2, and [44] based on Microsoft HololensFootnote 3 to reproduce proteins. Interestingly, in both cases, the authors reported being constrained in several circumstances by the platforms hardware limitations.
2.2 Biology - simulation
Some tools integrates accurate visualization with scientific simulations of molecular interactions; accordingly, they might also be included in the previous category. Férey et al. [19] designed a virtual reality framework dedicated to immersive and interactive molecular simulations that also integrates haptic devices for manipulation. Other tools let users collaborate. For example, Lee et al. [33] built a collaborative system for the visualisation of biomolecular structures to overcome the spatial and temporal limitations of different workplaces. It employed back-projection displays and polarised glasses for stereoptic viewing. Connor et al. [14] designed a virtual reality framework for molecular dynamics that allowed local cooperation among users in the same room. These would wear a visor tethered to a computer while simulations and synchronization happened on a remote server. Kingsley et al. [31] created a collaborative virtual reality application for the visualization of macromolecular structures of drugs. Their goals is the democratization of drug discovery tools to enable people with different backgrounds join in and quicken the field. O’Connor et al. [46] developed a multi-user virtual reality framework, called Narupa, that enables groups of researchers to cohabit a real-time simulation environment to manipulate the dynamics of complex molecular structures. Deeks et al. [17] applied Narupa to test different protocols for docking to the SARS-CoV-2 Main Protease. Juarez-Jimenez et al. [30] present a framework combining molecular dynamics simulation with virtual reality to explore large amplitude conformational changes in protein structures.
2.3 Biology - teaching
All the systems discussed so far can also be used for teaching and therefore could be broadly considered also in this category. However, only few tools have been designed specifically for teaching right from the start. Coan et al. [13] created two virtual reality laboratories (one focused on DNA and collagen, the other one on hemoglobin). Students access one of the labs and initially read a short powerpoint presentation; next they are presented with short tasks (e.g. determine the dimensions of the three DNA molecules using a measuring tool). At the end they receive a questionnaire about the learned concepts. Similarly, Bennie et al. [5] designed a real-time interactive molecular dynamics simulations in virtual reality that included a set of three short tasks that students had to complete (e.g., rearrangement, unbinding, and docking in specific scenarios). Dai et al. [15] developed a virtual reality framework with body tracking to help students better understand coordination chemistry and molecular orbitals. All these tools implement single-user experiences and reported positive feedback from students although none of them presented the data nor the results of an experimental evaluation like we do in this paper.
2.4 Crystallography - visualization
There are fewer tools for the visualization of crystals than the ones available for biomolecules. Furthermore, most of them use flat displays like RasMol [58] and OpenRasMol [47], Crystal Viewer [16], JmolFootnote 4, and [25]. Indeed, some tools support virtual reality. Drouhard et al. [18] proposed a visualiser for materials science using the Oculus RiftFootnote 5 head mounted display. VRChem is another virtual reality tool to visualize complex material science molecules developed for a master thesis project [50]. Other tools exploits augmented reality [41, 66, 67]. For example, Swamy et al. [66, 67] and Mansoor [41] developed two applications to visualise stereoisomers and crystal structures. The former relied on mobile phones and was based on VuforiaFootnote 6; the latter used the more expensive Microsoft Hololens. Both explained their choice to use augmented reality as a way to reduce the cognitive load required by imagining complex spatial structures in order to ease the learning process. More recently, Zakharov et al. [74] discussed the potential benefits of using virtual and mixed reality in studying the geometry of the crystal lattice; however, the authors did not implement any tool.
2.5 Crystallography - simulation
NOMAD VR [22, 23] is probably the most important project in this category and it comprises applications running on a wide variety of virtual reality platforms, from CAVE, to several models of head-mounted displays, to mobile-based ones like Google CardboardFootnote 7. It is an open source project for the visualization and the simulation of interactions between molecules and materials with a long list of features, whose availability varies based on the hardware available.
2.6 Crystallography - teaching
As noted before, all the visualization and simulation tools might be used to support class activities. As such, all the crystallography tools considered so far could be included in this category. There are however few tools that have been specifically developed for teaching. Quishpe-Armas et al. [51] designed virtual laboratory for tethered headsets that a student can use to learn the 3D spatial arrangements of crystals that are necessary to micro-electro-mechanical systems (MEMS). Tarng et al. [68] designed a virtual reality material science teaching module aimed at instructing students on shape memory alloys. Caro et al. [9] did a pilot study to examine whether virtual reality could improve the performances of students with respect to a traditional paper-based lesson. Accordingly, they built a virtual reality learning module about crystal structures aimed at students of a basic material science and engineering course. In [9], they report the results of a four questions form that was presented to 7 students (4 using the virtual reality tool, 3 using the paper-based learning). Although, the sample size did not allow them to draw general conclusions, the authors noted that students using the application answered better when spatial reasoning was required. Interestingly however, students who used the paper-based module had higher scores when questions needed remembering prior knowledge. Note that, these tools for teaching crystallography were mainly designed as single-user experience focused on a set of learning activities to be completed [51, 68]. In contrast our framework was designed as a virtual reality class experience with the focus both on (i) a live teacher-student multi-user interaction and (ii) offline single-user experience involving the viewing of recorded lectures and the free exploration of the teaching material.
3 The virtual reality crystallography application
We created the application as a tool to support the studying and teaching of crystal structures. Teachers can use it as a virtual classroom or a recording studio to produce study material that students can access in the same virtual environment. Students can improve their understanding of crystal structures by attending live online lectures in an interactive virtual environment or by exploring an interactive library of crystals structures and recorded lectures.
3.1 Requirements
At first, we extensively interviewed our colleagues that have taught crystallography at Politecnico di Milano over the last ten years. They demonstrated the tools they have been using and highlighted their limitations. They asked us to design an application (i) that would provide accurate three-dimensional visualization and manipulation of crystals; (ii) that instructors could use for teaching; (iii) students could use to deepen their understanding and review study material. Our colleagues specified a list of interactions (e.g., move, zoom, and rotate, apply symmetries and cuts) and asked to have the option to record both the live lectures with students and after-class material. Being cost an essential factor for a broad adoption of the application, we were asked to support inexpensive devices.
We discussed various options including augmented reality using smartphones, Microsoft Hololens, and virtual reality using the inexpensive and stand-alone head mounted display Oculus Go. At the end, we agreed to use virtual reality because of several limitations of augmented reality. Firstly, augmented reality using smartphones would still display the crystals on a flat surface; in contrast, virtual reality enables a realistic three-dimensional view of the crystals. Secondly, virtual reality provides a potentially infinite world that can be used to place the crystals; while with augmented reality, crystals might overlap with real objects (e.g., furniture) and accurate positioning would require high-end devices equipped with LidarFootnote 8 cameras. Furthermore, virtual reality headsets isolate from the outside world, thus improving focus and limiting environmental distractions. Finally, even though we decided to use Oculus Go, one of cheapest virtual reality head-mounted displays on the market, it is way cheaper and has more computational power when compared to current augmented reality devices.
3.2 Preliminary evaluation
We carried out a preliminary evaluation of an initial prototype with our colleagues teaching crystallography. The qualitative evaluation mainly centered around (i) the scientific accuracy of the educational content, and (ii) the usability of the interface. We focused on our colleagues since they would have been in charge of introducing the application to students during their courses and showcasing it. Thus, it was essential for us that they would approve its educational content, find it comfortable, and consider it a valid support for their curricular activities. We planned to include students in the next and more thorough evaluation (see Section 5). Given the qualitative nature of the evaluation, we did not hand out a questionnaire but simply recorded the opinions and suggestions. The evaluation highlighted some minor issues regarding the zooming and panning of crystals which could overlap with the interface, which we fixed. It also gave teachers the opportunity to ask for additional features like for instance, (i) the option to visualize crystals without bounds, using uniform colors, with the size of atoms proportional to their atomic weight; (ii) a section devoted to epitaxial crystals [2]. All the features were included in the final version.
3.3 Access modes
The application can be accessed as a teacher to organize online lectures (in teacher-student mode) and to record material that students can watch later (using the recording mode); or as a student to access recorded material, and to explore the library of available crystals (in student-mode).
Teacher-student mode
Teachers are the only users that can create a virtual classroom and start a lesson. They start it as a shared environment with a unique identifier and timestamp that students can join by selecting it from a list of the available classes. Teachers can check the list of connected participants so that, once all the students are present, the lesson can begin. Note that an authentication is requested upon accessing the application to identify the users’ role (teacher or student). During a lecture, teachers are the only ones allowed to interact with the structures using all the functionalities made available on their virtual desk shown in Fig. 2a. Students are spectators and can only watch the manipulations applied to the crystals while listening to the explanation. Virtual classes can be organized remotely, with students and teacher in different locations, or locally, with everybody staying in the same room. When teaching remotely, the headset microphone is active and captures the teacher voice that is sent to the speakers embedded in the headsets of the other participants.
Recording mode
Teachers can record lectures (i) online, while teaching to a class or (ii) offline, while preparing material for a lecture. In both cases, the interactions with the crystals are done in the same way through the desk (Fig. 2a), but all the manipulations applied on the crystal and the recorded voice memos are saved on an online server. After lectures are uploaded to the server, they receive a unique identifier including the date and time of the recording; students can then access and replay the recorded lectures at any time.
Student mode
Students can use the application to watch available recordings or to explore the library of crystal structures by themselves. In order to apply the different manipulations, students use the same desk (Fig. 2a) professors use in the Teacher-Student Mode.
3.4 Interface
We evaluated several interface models during a series of preliminary sessions with teachers. At the end, based on their feedback, we decided to use a desk metaphor (Fig. 2a). The desk is positioned over a circular platform surrounded by void and everything users can do is activated by a widget on the desk—either a button, a switch, or a discrete slider. Teachers and students found this setup more familiar and made them feel more at ease in the virtual environment. The circular platform and the fixed desk channel users towards the desk, implicitly cutting off all other possible moves (which would be impracticable anyway).
The desk is flat to provide the widest field of view over the crystal structures (initially, we experimented with vertical transparent panels but they occluded the view too much). In addition, the interactive elements are positioned to minimize the need for moving the head and grouped based on the type of interaction. Commands for switching between views of the crystals (like, the molecule, the unit cell, the lattice, and Miller index planes) are positioned on the right side of the desk. Commands used more often (like panning, rotating, and scaling) are placed in the center. Commands for modifying the overall view (like, switching the bonds on and off, or using a uniform color and representation of atoms) are position on the left side of the desk.
Figures 2 and 3 show images taken from a session. As the application starts, users can choose to access an existing class, the recorded lectures, or the library (Fig. 2b). When the library is selected, users are taken to the menu of available structures (Fig. 2c) that are identified by a unique id (e.g., 1538617 for diamonds and 1100118 for iron). The list of available crystals was specified by our colleagues. Crystals are stored using the Crystallographic Information File (CIF) [26, 27], a standard text file format developed by the International Union of Crystallography (IUCr).Footnote 9 The Crystallography Open DatabaseFootnote 10 provides access to thousands of structures in CIF format that can be potentially included in our application, making it a general-purpose tool for crystallography. In the example, when selecting the diamond structure, users are positioned at the desk and the simplest building block of a diamond is shown (Fig. 3a); they can visualize its unit cell (Fig. 3b) and the unit cells on the diamond lattice geometry (Fig. 3c). Crystals can be rotated and the visualization of the bonds (the tubes connecting the atoms) can be switched off (Fig. 3d). Structures can also be visualized using uniform colors and with the size of atoms proportional to their atomic weight (Fig. 3e). Users can get an inside view of the crystal (Fig. 3f); in this case users can look around but cannot move or rotate the crystal since this would cause vection which would result in motion sickness. With a button of the controller the inside crystal view is turned off and the users is returned at the desk. This transition is implemented by the typical fade off/fade on transition used in virtual reality teletransportation to avoid vection or motion sickness. Finally, users can examine planes corresponding to different Miller indexes (Figs. 3g and 3h).
4 Architecture
The application was designed for stand-alone and inexpensive head mounted displays. Both the initial prototypes and the first version of the applications were developed for Oculus GoFootnote 11.
Oculus go
was the first low-cost stand-alone virtual reality head-mounted display (HMD). It has a single 5.5-inch LCD display with a resolution of 1280×1440 pixels per eye and a refresh rate of 72 or 60 Hz, depending on the application. Its lenses provide a field of view of about 101 degrees. Input is provided with a wireless controller that behaves like a laser pointer. Oculus Go provides a non-positional 3-degrees-of-freedom tracking, making it capable of seated or static-standing activities.
Software architecture
The application was developed using UnityFootnote 12 and the Photon EngineFootnote 13 for online multi-user support. Photon offers dedicated servers, authentication integration, matchmaking and in-game communication. It also provides executables that users can use to run their own server locally. The application relies on a client-server architecture consisting of a single server and multiple clients. Teachers connect to a lobby where they can create a new lecture and invite students to join. During the lecture, all the teacher actions and comments are sent to the server that forwards the information to the students’ clients in real-time. Figure 4 shows the interaction workflows for teachers and students to start and access a virtual lecture. At first, clients connect to a server hosted on the Photon Cloud or on a local server. Next, clients can request the list of all the available lessons, create a new one (if they assigned a teaching role) or join an existing one. Servers keep lectures separated so that clients can interact only with other clients connected to the same lesson. Note that, Recording Mode does not involve any real-time communication since students can access the available lectures later, at any time, by accessing the platform in Student Mode, which also does not involve any real-time interaction with other users. Figure 5 shows the high-level software architecture of our application. The Core implements the platform-independent functionalities such as the creation, visualization and manipulation of crystals, access to the libraries models. Because of the relative large number of headset models and the rapid evolution of the technology, we encapsulated the interaction with our application in an abstraction layer that facilitates porting it to new devices. The layer maps the available functionalities to the actual platform used. Thus, we can potentially support multiple platforms, with different interaction patterns and input devices, by writing platform-specific modules that manage the input and call the functionalities exposed by the Abstraction Layer. Note that, we implemented our own abstraction layer since other solutions like Unity XR Interaction ToolkitFootnote 14 is still in preview.
5 Experimental evaluation
The evaluation was carried out during two different events, one involved engineering students, one was carried out during a public scientific dissemination event.
5.1 Setup of experiments
We gave subjects a brief introduction to the experience, described its purpose, how it would unfold, and how to use the controller. Next, they wore the headset and started the application as students attending a brief crystallography lecture (that is, in Teacher-Student mode) focused on the fundamental crystallography concepts. During the lecture the subjects were introduced to the available functionalities. At the end of the lecture, subjects were allowed to try explore the library of available crystals on their own (in Student Mode). Overall the experience lasted for about 15 minutes and, at the end, subjects were asked to complete an online form to evaluate their experience. The form included 27 Likert-scale [35] questions reported in Table 2, which extends the IBM Usability Questionnaire [34]. It was specified that, when answers ranged from 1 to 5, 1 meant “I completely disagree” or “Very poor” and 5 meant “I completely agree” or “Very good”. For all the questions, we report the percentage of answers in each category as well as the overall mean and standard deviation values. We had 30 subjects (21 males and 9 females), mostly between 18 and 30 years old (only two were between 30 and 50 years old), and 9 of them had previous knowledge about crystallography (Fig. 6a). 15 subjects did not play video games; 7 played video games less than 2 hours per week; 4 played from 2 to 4 hours per week; 2 played from 4 to 7 hours per week; 2 subjects played for more than 7 hours per week (Fig. 6b); 9 subjects had some previous knowledge about crystallography, the remaining 21 had none (Fig. 6c).
5.2 Experimental results
The questionnaire asked subjects to evaluate the experience in terms of usability, comfort, interface, and educational value.
Usability
The subjects rated the usage of the virtual environment as cozy and intuitive (Fig. 7). Moving in the virtual environment was easy (Fig. 7a Q4 4.43 ± 0.62) as well as using the various functionalities (Fig. 7b Q5 4.6 ± 0.61). Subjects felt comfortable in the virtual environment (Fig. 7c Q6 4.4 ± 0.61) and could understand within (what they perceived as) a reasonable time what they could do in the environment (Fig. 7d Q7 4.77 ± 0.42). Subjects rated very high both the usability of the hand-held device (Fig. 7e Q9 4.37 ± 0.71), of the environment (Fig. 7f Q10 4.57 ± 0.62), and the usability of the elements within the environment that can be activated/deactivated (Fig. 7g Q11 4.2 ± 0.70). Their evaluation of the experience, as a whole, was very good (Fig. 7h Q12 4.67 ± 0.47). In the comments section of Q8, two subjects stated they initially had problems interacting with the application because they did not know how to use the controller. However, after some additional training, the problems were solved.
Comfort
Most of the subjects felt comfortable during the whole experience (Fig. 8a Q14 4.43 ± 0.56), 20% of the subjects felt uncomfortable during part of the experience (Fig. 8b Q15 80% True). Some subjects reported some dizziness during the experience: 20% felt comfortable initially but later did not feel well (Fig. 8c 20% True); 10% felt initially uncomfortable but later in the experience did well (Fig. 8d 20% True). The answers to Q18 showed that dizziness was mainly experienced after seeing crystals rotating during the lecture (in Teacher-Student mode) when subjects did not have control over the environment. Interestingly, two subjects experienced some dizziness when looking down and not seeing their body. Other two wrote that the headset was not very comfortable to wear and easily went out of focus. This is a common issue with headsets that do not allow to change the interpupillary distance, like the Oculus Go, since people that vary significantly from the average head size tend to suffer the misalignment with the lenses.
Interface
In terms of intuitiveness, subjects reported that the meaning of the panels and the buttons on the desk was easy to understand (Fig. 8e Q19 4.07 ± 0.85 and Fig. 8f Q20 4.17 ± 0.86) as well as it was the effect these had on the displayed crystal (Fig. 8g Q21 4.13 ± 1.02). The high value of standard deviation in Q21 was mainly due to people who had no prior knowledge in crystallography and had problems in understanding what effect symmetries and cuts had on the visualization.
Educational value
Almost all the subjects believed that the experience could enrich traditional lectures at high-school and university level (Fig. 9a Q23 4.77 ± 0.63) and stated that they would appreciate class activities with this type of support (Fig. 9b Q24 4.77 ± 0.43). Overall, they greatly enjoyed the experience (Q25 4.87 ± 0.35). In the open question box, From the open questions we received some interesting suggestions: two subjects proposed to add the name of the crystal on the desk, so that people do not forget which crystal they chose; some suggested to add a fake body showing legs and feet when looking down; one suggested to lower the lighting. Table 3 summarizes all the results of the evaluation while Table 4 reports the raw data collected from the questionnaires.
6 Conclusions and future work
We presented an educational application we designed to help teachers and students involved in crystallography classes at Politecnico di Milano. Teachers can use it to organize online lectures in a shared virtual environment, and to record material that students can later watch in the same virtual environment. Students can use it to attend online virtual classes, to watch recording in the same virtual environment, or to explore a library of available crystals. While existing applications focus on the visualization of complex crystal structures, our application was designed specifically as a support for online university lectures, with a focus on the student-teacher class interaction and after-class activities. We evaluated the application with engineering students and people attending a scientific dissemination event, receiving positive feedback both in term of usability, comfort, and engagement.
As future developments, we plan to introduce the possibility for students to interact with the professor by raising a virtual hand and being able to take control of the presentation desk (similarly to what is done in many video conferencing tools). The development tool and the networking middleware we used (Unity and Photon) already support these functionalities. However, the feature has not been implemented yet since teachers feared it might be too difficult to manage in virtual reality. As teachers will gain more experience in online virtual classrooms, we are confident that they might be more willing to interact with students more dynamically. We also plan to introduce a gamified mode with quizzes and simple tasks, that might help students’ self-assessment on specific topics. Finally, we have ported the application to the Oculus QuestFootnote 15 headset family and we are evaluating the introduction of controller-free interactions, similar to what analyzed in [56], that Oculus Quest models now support.
Notes
References
Baheti A, Seshadri S, Kumar A, Srimathveeravalli G, Kesavadas T, Guru K (March 2008) Ross: Virtual reality robotic surgical simulator for the da vinci surgical system. In: 2008 Symposium on haptic interfaces for virtual environment and teleoperator systems, pp 479–480
Ballabio A, Bietti S, Scaccabarozzi A, Esposito L, Vichi S, Fedorov A, Vinattieri A, Mannucci C, Biccari F, Nemcsis A, Toth L, Miglio L, Gurioli M, Isella G, Sanguinetti S (Nov 2019) Gaas epilayers grown on patterned (001) silicon substrates via suspendedge layers. Sci Rep 9(1):17529
Bekele MK, Pierdicca R, Frontoni E, Malinverni ES, Gain J (March 2018) A survey of augmented, virtual, and mixed reality for cultural heritage. J Comput Cult Herit. 11(2)
Bekele MK , Pierdicca R, Frontoni E, Malinverni ES, Gain J (March 2018) A survey of augmented, virtual, and mixed reality for cultural heritage. J Comput Cult Herit 11(2):7:1–7:36
Bennie SJ, Ranaghan KE, Deeks H, Goldsmith HE, O’Connor MB, Mulholland AJ, Glowacki DR (Nov 2019) Teaching enzyme catalysis using interactive molecular dynamics in virtual reality. J Chem Educ 96(11):2488–2496
Berman H, Henrick K, Nakamura H (Dec 2003) Announcing the worldwide protein data bank. Nat Struct Mol Biol 10(12):980–980
Bernstein HJ, Bollinger JC, Brown ID, Gražulis S, Hester JR, McMahon B, Spadaccini N, Westbrook JD, Westrip SP (Feb 2016) Specification of the crystallographic information file format, version 2.0. J Appl Crystallogr 49(1):277–284
Borchardt-Ott W (August 2011) Crystallography. Springer Science & Business Media 3rd edition
Caro V, Carter B, Dagli S, Schissler M, Millunchick J (Oct 2018) Can virtual reality enhance learning: A case study in materials science. In: 2018 IEEE Frontiers in education conference (FIE), pp 1–4
Chittaro L, Buttussi F (2015) Assessing knowledge retention of an immersive serious game vs. a traditional education method in aviation safety 04. Vis Comput Graph IEEE Trans 21:529–538
Clifford RMS, Hoermann S, Marcadet N, Oliver H, Billinghurst M, Lindeman RW (Sep. 2018) Evaluating the effects of realistic communication disruptions in vr training for aerial firefighting. In: 2018 10Th International conference on virtual worlds and games for serious applications (VS-games), pp 1–8
Clifford RMS, Jung S, Hoermann S, Billinghurst M, Lindeman RW (March 2019) Creating a stressful decision making environment for aerial firefighter training in virtual reality. In: 2019 IEEE Conference on virtual reality and 3d user interfaces (VR), pp 181–189
Coan HA, Goehle G, Youker RT (2020) Teaching biochemistry and molecular biology with virtual reality — lesson creation and student response. Journal of Teaching and Learning, 14(1)
Connor M, Deeks H, Dawn E, Metatla O, Roudaut A, Sutton M, Glowacki B, Sage R, Tew P, Wonnacott M, Bates P, Mulholland A, Glowacki D (2018) Sampling molecular conformations and dynamics in a multi-user virtual reality framework. Sci Adv, vol 4. 01
Dai R, Laureanti JA, Kopelevich M, Diaconescu PL (Oct 2020) Developing a virtual reality approach toward a better understanding of coordination chemistry and molecular orbitals. J Chem Educ 97(10):3647–3651
Daniel F, Mejia GK, Chu Y (Aug 2019) Crystal viewer lab (new interactive front end)
Deeks HM, Walters RK, Barnoud J, Glowacki DR, Mulholland AJ (Dec 2020) Interactive molecular dynamics in virtual reality is an effective tool for flexible substrate and inhibitor docking to the sars-cov-2 main protease. J Chem Inf Model 60(12):5803–5814
Drouhard M, Steed CA, Hahn S, Proffen T, Daniel J, Matheson M (Oct 2015) Immersive visualization for materials science data analysis using the oculus rift. In: 2015 IEEE International conference on big data (big data), pp 2453–2461
Férey N, Delalande O, Grasseau G, Baaden M (2008) A vr framework for interacting with molecular simulations. In: Proceedings of the 2008 ACM symposium on virtual reality software and technology, VRST ’08, pp 91–94, New York, NY, USA, Association for Computing Machinery
Francoeur E (2002) Cyrus levinthal, the kluge and the origins of interactive molecular graphics. Endeavour 26(4):127–131
Freina L, Ott M (April 2015) A literature review on immersive virtual reality in education State of the art and perspectives. In: Roceanu I, Moldoveanu F, Trausan-Matu S, Barbieru D, Beligan D, Ionita A (eds) Proceedings of the 11th international scientific conference on elearning and software for education bucharest April 23 -24 2015, pp 133–141
García-Hernández RJ, Kranzlmüller D (2017) Virtual Reality Toolset for Material Science: NOMAD VR Tools. In: De Paolis LT, Bourdot P, Mongelli A (eds) Augmented Reality, Virtual Reality, and Computer Graphics. Lecture Notes in Computer Science, pp 309–319, Cham. Springer International Publishing
García-Hernández RJ, Kranzlmüller D (2019) Nomad vr: Multiplatform virtual reality viewer for chemistry simulations. Comput Phys Commun 237:230–237
Greenwald SW, Kulik A, Kunert A, Beck S, Fröhlich B, Cobb S, Parsons S, Newbutt N, Gouveia C, Cook C, Snyder A, Payne S, Holland J, Buessing S, Fields G, Corning W, Lee V, Xia L, Maes P (2017) Technology and applications for collaborative learning in virtual reality. In: Smith BK, Borge M, Mercier E, Lim KY (eds) 12th international conference on computer supported collaborative learning, CSCL 2017 Philadelphia, Pennsylvania, USA, June 18-22, 2017 International Society of the Learning Sciences
Gruber C, James A, Berchtold JT, Wood ZJ, Scott GE, Alghoul Z (2020) Interactive unit cell visualization tool for crystal lattice structures. J Chem Educ 97(7):2020–2024
Hall SR, Allen FH, Brown ID (1991) The crystallographic information file (CIF): a new standard archive file for crystallography. Acta Crystallographica Section A 47(6):655–685
Hall SR, McMahon B (eds) (2006) International tables for crystallography volume g definition and exchange of crystallographic data. Springer, New York
Hite R, Childers G, Jones MJ (2018) Review of Virtual Reality Hardware Employed in K-20 Science Education. Springer, Berlin
Humphrey W, Dalke A, Schulten K (1996) VMD–Visual molecular dynamics. J Mol Graph 14:33–38
Juárez-Jiménez J, Tew P, O’Connor M, Llabrés S, Sage R, Glowacki D, Michel J (2020) Combining virtual reality visualization with ensemble molecular dynamics to study complex protein conformational changes. J Chem Inf Model 60(12):6344–6354
Kingsley LJ, Brunet V, Lelais G, McCloskey S, Milliken K, Leija E, Fuhs SR, Wang K, Zhou E, Spraggon G (2019) Development of a virtual reality platform for effective communication of structural data in drug discovery. J Mol Graph Model 89:234–241
Kozlíková B, Krone M, Falk M, Lindow N, Baaden M, Baum D, Viola I, Parulek J, Hege H-C (2017) Visualization of biomolecular structures: State of the art revisited. Comput Graph Forum 36(8):178–204. https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.13072.
Lee J, Kim JI, Kang LW (2012) A collaborative molecular modeling environment using a virtual tunneling service, August 2012. J Biotechnol Biomed 2012:546521. Publisher: Hindawi Publishing Corporation
Lewis JR (1995) Ibm computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. Int J Hum-Comput Interact 7(1):57–78
Likert R (1932) A technique for the measurement of attitudes. Arch Psychol, (140):1–55
Lourdeaux D, Afoutni Z, Ferrer M-H, Sabouret N, Demulier V, Martin J-C, Bolot L, Boccara V, Victeams RL (2019) A virtual environment to train medical team leaders to interact with virtual subordinates. In: Proceedings of the 19th ACM international conference on intelligent virtual agents, IVA ’19, pp 241–243, New York,NY, USA, ACM
Lugrin J-L, Latoschik ME, Habel M, Roth D, Seufert C, Grafe S (2016) Breaking bad behaviors: A new tool for learning classroom management using virtual reality. Front ICT 3:26
Lv Z, Chen D, Lou R, Song H (2020) Industrial security solution for virtual reality. IEEE Internet Things J, pp 1–1
Lv Z, Li X, Li W (2017) Virtual reality geographical interactive scene semantics research for immersive geography learning. Neurocomputing 254:71–78. Recent Advances in Semantic Computing and Personalization
Manca D, Brambilla S, Colombo S (2013) Bridging between virtual reality and accident simulation for training of process-industry operators. Adv Eng Softw 55:1–9
Mansoor B, Makki M, Al-Thani D (2018) Use of mixed reality tools in introductory materials science courses
Mantovani F, Castelnuovo G (2003) The sense of presence in virtual training: Enhancing skills acquisition and transfer of knowledge through learning experience in virtual environments 01
Muzic ML, Autin L, Parulek J, Viola I (2015) cellVIEW: a tool for illustrative and multi-scale rendering of large biomolecular datasets. Eurograp Workshop Vis Comput Biomed 2015:61–70
Müller C, Krone M, Huber M, Biene V, Herr D, Koch S, Reina G, Weiskopf D, Ertl T (2018) Interactive Molecular Graphics for Augmented Reality Using HoloLens. J Int Bio, 15(2)
Norrby M, Grebner C, Eriksson J, Boström J (2015) Molecular rift: Virtual reality for drug designers. J Chem Inf Model 55(11):2475–2484. PMID: 26558887
O’Connor MB, Bennie SJ, Deeks HM, Jamieson-Binnie A, Jones AJ, Shannon RJ, Walters R, Mitchell TJ, Mulholland AJ, Glowacki DR (2019) An open-source multi-person virtual reality framework for interactive molecular dynamics: from quantum chemistry to drug binding arXiv:1902.01827
OpenRasMol (2009). http://www.openrasmol.org. Accessed 5 Aug 2022
Pettersen EF, Goddard TD, Huang CC, Couch GS, Greenblatt DM, Meng EC, Ferrin TE (2004) Ucsf chimera—a visualization system for exploratory research and analysis. J Comput Chem 25(13):1605–1612
Pettersen EF, Goddard TD, Huang CC, Meng EC, Couch GS, Croll TI, Morris JH, chimerax TEF (2021) Ucsf structure visualization for researchers, educators, and developers. Protein Sci 30(1):70–82
Pietikäinen O (2018) Vrchem: A molecular modeling software for virtual reality; vrchem: Virtuaalitodellisuuteen perustuva molekyylimallinnusohjelmisto. G2 pro gradu, diplomityö, 2018-02-27
Quishpe-Armas JA, Cedeño-Viveros LD, Meléndez-Campos J, Suárez-Mora CA, Camacho-Leon S (2015) An immersive 3d virtual learning environment for analyzing the atomic structure of mems-relevant materials. Procedia Comput Sci 75:413–416. 2015 International Conference Virtual and Augmented Reality in Education
Radianti J, Majchrzak TA, Fromm J, Wohlgenannt I (2020) A systematic review of immersive virtual reality applications for higher education design elements, lessons learned, and research agenda. Comput & Educ 147:103778
Ragan ED, Sowndararajan A, Kopper R, Bowman DA (2010) The effects of higher levels of immersion on procedure memorization performance and implications for educational virtual environments. Presence: Teleoperators and Virtual Environments 19(6):527–543
Rashid F (2017) Use of vr technology and passive haptics for manpads training system
REWIND (2017). https://rewind.co/portfolio/bbc-home-vr-spacewalk/. Accessed 12 Nov 2020
Salomoni P, Prandi C, Roccetti M, Casanova L, Marchetti L, Marfia G (2017) Diegetic user interfaces for virtual environments with hmds: a user experience study with oculus rift. J Multimodal User Interfaces 11(2):173–184
Salvadori A, Frate G, Pagliai M, Mancini G, Barone V (2016) Immersive virtual reality in computational chemistry: Applications to the analysis of qm and mm data. Int J Quantum Chem 116:07
Sayle A, Milner-White EJ (1995) Rasmo: biomolecular graphics for all. Trends Biochem Sci 20(9):374–376
Schijven M, Jakimowicz J, Broeders I AMJ, Tseng L (2005) The eindhoven laparoscopic cholecystectomy training course - improving operating room performance using virtual reality training: Results from the first e.a.e.s. accredited virtual reality trainings curriculum 10. Surg Endosc 19:1220–6
Schrödinger Inc (2017) Pymol
Scott E, Soria A, Campo M (2016) Adaptive 3d virtual learning environments—a review of the literature. IEEE Trans Learn Technol 10(3):262–276
Sharma S, Chen W (March 2014) Multi-user vr classroom with 3d interaction and real-time motion detection. In: 2014 International conference on computational science and computational intelligence, vol 2. pp 187–192
Shibata T (2019) Virtual reality in education: How schools use vr in classrooms. In: Bagnara S, Tartaglia R, Albolino S, Alexander T, Fujita Y (eds) Proceedings of the 20th congress of the international ergonomics association (IEA 2018), pp 423–425, Cham, Springer International Publishing
Stinson C, Bowman D (2014) Feasibility of training athletes for high-pressure situations using virtual reality, 04. IEEE Trans Vis Comput Graph 20:606–15
Ström P, Hedman Leif, Särnå L, Kjellin A, Wredmark T, Felländer-Tsai L (2006) Early exposure to haptic feedback enhances performance in surgical simulator training: a prospective randomized crossover study in surgical residents, 10. Surg Endosc 20:1383–8
Swamy N, Dasgupta C (2018) Investigating the nature of students’ reasoning in connecting molecular structures of stereoisomers with their physical properties using an ar app. In: 2018 IEEE Tenth international conference on technology for education (t4e), pp 53–60
Swamy N, Dasgupta C, Murthy S (2018) Stereochem: Augmented reality 3d molecular model visualization app for teaching and learning stereochemistry. 2018 IEEE 18th International Conference on Advanced Learning Technologies (ICALT), pp 252–256
Tarng W, Chen C-J, Lee C-Y, Lin C-M, Lin Y-J (2019) Application of virtual reality for learning the material properties of shape memory alloys, 02. Appl Sci 9:580
Taupiac J, Rodriguez N, Strauss O, Rabier M (March 2019) Ad-hoc study on soldiers calibration procedure in virtual reality. In: 2019 IEEE Conference on virtual reality and 3d user interfaces (VR), pp 190–199
Vichitvejpaisal P, Yamee N, Marsertsri P (July 2016) Firefighting simulation on virtual reality platform. In: 2016 13Th international joint conference on computer science and software engineering (JCSSE), pp 1–5
Wu J, Chen G, Chang C, Lin C (July 2016) Development of virtual-reality simulator system for minimally invasive surgery (mis) using fractional-order vascular access. In: 2016 SAI Computing conference (SAI), pp 1257–1259
Yan Z, Lv Z (2020) The influence of immersive virtual reality systems on online social application. Appl Sci, 10(15)
Yun H, Park S , Ryu J (2019) Exploring the influences of immersive virtual reality pre-service teacher training simulations on teacher efficacy. In: Kevin Graziano, editor, Proceedings of Society for Information Technology & Teacher Education International Conference 2019, pp 2112–2116, Las Vegas, NV, United States, March 2019. Association for the Advancement of Computing in Education (AACE)
Zakharov PV, Vdovin RS, Markidonov AV, Kochkin AS, Vdovin AS (2020) Virtual and mixed reality in the study of the geometry of the crystal lattice, apr. J Phys Conf Ser 1515:022001
Zhao J, Klippel A (March 2019) Scale - unexplored opportunities for immersive technologies in place-based learning. In: 2019 IEEE Conference on virtual reality and 3d user interfaces (VR), pp 55–162
Zheng M, Waller MP (2017) Chempreview: an augmented reality-based molecular interface. J Mol Graph Model 73:18–23
Zhihan Lv (Jul 2020) Virtual reality in the context of internet of things. Neural Comput Applic 32(13):9593–9602
Funding
Open access funding provided by Politecnico di Milano within the CRUI-CARE Agreement.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
(MP4 118 MB)
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Stella, E., Agosti, I., Di Blas, N. et al. A virtual reality classroom to teach and explore crystal solid state structures. Multimed Tools Appl 82, 6993–7016 (2023). https://doi.org/10.1007/s11042-022-13410-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-022-13410-0