Haptic cues as a utility to perceive and recognise geometry

Abstract

Research has been conducted on how to aid blind peoples’ perceptions and cognition of scientific data and, specifically, on how to strengthen their background in mathematics as a means of accomplishing this goal. In search of alternate modes to vision, researchers and practitioners have studied the opportunities of haptics alone and in combination with other modes, such as audio. What is already known, and has motivated research in this area, is that touch and vision might form a common brain representation that is shared between the visual and haptic modalities and through haptics learning is active rather than passive. In spite of extensive research on haptics in the areas of psychology and neuropsychology, recent advances and rare experiences in using haptic technology have not caused a transfer from basic knowledge in the area of haptics to learning applications and practical guidelines on how to develop such applications. Thus motivated, this study investigates different haptic effects, such as free space, magnetic effects and the bounded box when blind people are given the task of recognising and manipulating classes of 3D objects with which they have varying familiarity. In parallel, this study investigates the applicability of Sjöström’s guidelines on haptic applications development and uses his problem classification to capture knowledge from the experiments. The results of this study show that users can easily recognise and manipulate familiar objects, albeit with some assistance. There is an indication that users completed tasks faster and needed less assistance with magnetic effects. However, they were not as satisfied with this mode. While the results of this study show that haptics have the potential to allow students to conceptualise 3D objects, much more work is needed to exploit this technology to the fullest. Objects with higher complexity are difficult for students, and, in their opinion, the virtual objects (as presented) leave much room for improvement. Sjöström’s error taxonomy proved useful, and four of five sub-guidelines tested were confirmed to be useful in this study.

1 Introduction

Visualisation is becoming increasingly important as a way for people to understand complex information. Visualisation is concerned with ‘exploring data and information in such a way as to gain understanding and insight into data’ [1]. For the purpose of the discussion of blind peoples’ perception of scientific data, a more useful definition might be ‘the binding (or mapping) of data to representations that can be perceived. The types of bindings could be visual, auditory, tactile etc., or a combination of these’ [2]. A user interface built by merging these modes as inputs and outputs is called a multimodal interface [3]. The haptic mode, i.e. a mode that supports the tactile and kinaesthetic sense, has been studied to a limited extent, as an aid for disabled people. One motivation for research on multimodal user interfaces, including haptics for blind people, is that neurological and neuropsychological studies have shown that visual and haptic brain representations are so similar that they might be mutual between modalities [4–6]. Another strong incentive for using haptics is that active learning is more effective than passive learning, i.e. the power of experiencing through touch is more powerful than simply watching [7]. A number of software tools have been developed to help students learn geometry [8]. A research study examining what determines teachers’ intentions to use such software shows that the expected usefulness and teachers’ beliefs in their own technological proficiency determines their intended and actual use [9]. Such software is expected to become more widely used. Research in mathematics education has shown that new pedagogical models are needed for learning geometry. Specifically, more emphasis should be placed on actively exploring 3D objects with the aid of computer technology [10]. Finally, it has been recognised that further research on 3D geometry in virtual reality is much needed to help develop students’ spatial skills [11].

Naturally, the above motivations apply to all students. However, studying how blind people recognise or perceive geometrical objects has different applications, with orientation and mobility training being perhaps the most important applications for the visually impaired [12–14]. Orientation and mobility instruction needs to include the teaching of abstract terms such as point, line, parallel to, above, under, besides and perimeter familiarisation, all of which can be practised in a haptical virtual environment [14]. Using different technologies, researchers have explored blind peoples’ abilities to recognise 3D objects for over a decade. Early studies using a haptic device having three degrees of freedom (i.e. up/down, left/right and forwards/backwards) showed that users perceive larger simple objects such as a cube or a sphere more accurately than smaller objects, but that users may not understand more complex objects. It was encouraging to discover that most users quickly learn strategies for how to explore virtual objects [15]. Later, several studies showed the potential of haptics to let visually impaired users explore graphically represented computer information [16–22]. Using a haptic device with 6 degrees of freedom, Magnusson et al. examined whether visually impaired users were able to describe three-dimensional objects that were created in a virtual environment and whether they could understand the objects. The results of that study were promising, as the percentage of persons that recognised a single 3D geometrical object was 80%. However, when asking users to rebuild a model containing three different objects on a 3 × 3 grid, they found it more difficult than the recognition task, and only 39% were able to reconstruct the model perfectly. Users of this study had good success in identifying complex objects such as a vase, piano, communications satellite, guitar and a sword, with results ranging from 79% (vase) to 96% (communication satellite) [23].

Exploiting the knowledge that is gathered from basic research in haptics and visualisation, a number of virtual learning environments have been developed, at least as prototypes, e.g. in physics, biology and astronomy [24–26]. The potential of haptics has been shown in experiments, but further investigation is required to design applications in a real-life context [27]. It remains to be researched where and how this potential can be best applied and how other human senses can be utilised in order to allow blind individuals to interact with a computer in the most effective way. Studying to what extent people are able to conceptualise a simple geometric object with its geometric properties and to manipulate a device in a virtual environment will help when developing applications for blind students.

Design guidelines have been one of the outcomes of research on usability. For example, there are guidelines that are suggested for designing haptic user interfaces [28, 29]. An example is Hale’s guidelines, which are built on human physiological, psychophysical and neurological foundations but do not focus on visually impaired users. Sjöström’s guidelines, on the other hand, do focus on visually impaired users, but since their publication, not much research has been conducted on validating the guidelines, nor is there much evidence of their use in practice. With regard to standardisation, the working group TC159/SC4/WG9 has been developing guidance for haptic and tactile interactions, which will be published in the framework of ISO 9241-900 [30, 31]. Their work on these guidelines is based on guidance from over 40 papers, e.g. those of Hale, Sjöström (mentioned above) and Oakley et al. [32], as well as on 10 ISO standards. The ISO 9241-900 framework includes six items, which cover various aspects of haptic and tactile aspects, including ergonomics, evaluations and devices. Currently, one of these items has been published as a standard (ISO 9241-920), which provides guidance on the design and evaluation of hardware, software and combinations of hardware and software interactions, including the design/use of tactile/haptic inputs and outputs, with general guidance on their design and use [33].

The objectives of the study described in this article are as follows:

• 1. To investigate the effects of haptic cues on how accurately users perceive information without visualisation.

• 2. To explore the use of haptic cues for blind students to learn geometry.

More specifically, this study focuses on the subject of geometry in mathematics and explores how haptic cues can be utilised to perceive and recognise 3D objects. A simple virtual environment has been developed, which includes several types of 3D objects of varying complexity and haptic effects, presented to users through a haptic peripheral. The next section describes the background of this research, namely assistive technology for blind students learning geometry, and specifically describes the use of haptics for this purpose. Section 3 covers in more detail the research questions that are addressed in this article. Section 4 describes the geometry objects that are examined in the experiments with haptic modes, discussed in Sect. 5. Section 6 reports the results of the research study. Section 7 provides a summary and explanation of the results, identifies their limitations and compares them to related work. Finally, Sect. 8 concludes the article by discussing the impact of the results and suggesting future work.

2 Background

Mathematics as a discipline forms the basis for science and engineering, but its mastery is one of the greatest challenges for blind students. Many blind students have difficulty developing mathematical skills, because mathematical concepts and notations are represented graphically; this limitation has motivated various solutions that utilise information technology, as described in survey articles [34–36]. One of the most difficult challenges has been the teaching of concepts that involve three-dimensional objects, which are concepts found in problems at all levels of mathematics. These concepts are often difficult for students to understand, even though 3D objects are most often represented in 2D drawings. Visual virtual environments that help students to learn geometry have been suggested [37], and with the advent of haptic technologies, touch and kinaesthetics have been used to enhance students’ experience of 2D [22] and 3D objects at the high school and university levels [38, 39].

There have been several studies within the educational and technical disciplines on how to solve problems that blind students have with perceiving mathematical forms or equations. For example, Sribunruangrit et al. [40] developed the Braille Box to access graphical information on a computer. This tool allows users to explore a figure such as a black line on a white background and to distinguish colours by sound. The problem is not only that of recognising the objects, but of being able to link them to the mathematical formulas. Karshmer et al. [41] tackled the problem of reading and writing mathematics, especially with respect to complex equations, by developing tools to translate the Nemeth Braille Code, which is used by blind students to read and write mathematics, into LaTex and vice versa. Although such tools have been developed, the most commonly used solution to teach visually impaired students about functions or graphs is to manually create them with rubber bands on a cork composition board that is mounted with a rubber mat that has been embossed with a grid of 1/2-inch squares. Another solution is to print out embossed graphs on paper. For geometric objects, teachers describe the object verbally and use real objects that can be touched.

The potential use of haptic technologies for helping blind users to understand mathematics has been studied by researchers. For example, Sjöström developed several applications, including graphical representations of mathematical information. He investigated how blind people orient and navigate the virtual haptic environment, explored possible solutions to access the graphical user interface through haptical interactions and studied how different types of graphical information such as maps, floor plans and pictures could be accessed through haptic interaction. As a result of testing a prototype application with blind users, he presented guidelines for haptic interaction design [28]. The goals of his guidelines are the following: (1) to develop a usable, efficient, learnable and user-acceptable design of products, (2) to evaluate existing designs and (3) to assist users in providing feedback from their experiences. In the guidelines, he classified haptic interaction design into five aspects: (1) virtual objects, (2) navigation and overview in the virtual environment, (3) context of virtual objects, (4) modalities and (5) learning of the programming environment.

Yu et al. [42] studied haptic interfaces for blind people in a much narrower area. Focusing on graph representations of mathematical and business data, they combined force-feedback and synthesised speech and non-speech audio to improve access to graphs and tables for blind users. They also compared two different force-feedback devices: PHANToM™ from SensAble Technology Inc. and WingMan Force-Feedback mouse from Logitech™ [43]. They investigated the practical use of haptic interactions and developed a web-based haptic application to create bar and line graphs with a WingMan™ mouse. Yu et al.’s solution of a popular web-based application with a low-cost device such as WingMan is one example of providing a real-life application. When investigating the role of haptics in multimodal interfaces (synthesised speech, non-speech sound and haptics), the results showed that the role of haptics depends on the information representation. Moreover, ‘the nature of the application and the dimensions of the information being presented determine the amount of haptic input and the way it is used’ [42, p. 123].

3 Research questions

3.1 Investigating the effect of haptic cues on the accuracy of information perception

To use the haptic sense correctly in a multimodal interface, it is important to examine how accurately users are able to perceive virtually presented information through the haptic device [19]. This examination is accomplished by presenting to the user simple familiar objects such as a cube or sphere, using the different types of haptic effects, specifically free space, bounded box and magnetic effects. Thus, two questions are formulated to reach the first research objective of investigating the effect of haptic cues on the accuracy of information perception.

Question 1: Using haptic sensory cues, are people able to conceptualise the geometric properties of simple familiar objects such as a cube or sphere?

Magnusson et al. examined whether blind users were able to identify objects with small details such as a piano or chair. The results of their study were that people had difficulty identifying small details such as the legs of a piano. When using simpler geometric objects as test samples, their results showed that 80% of the test participants were able to identify the objects [23]. Similarly, this study presents to the users simple and familiar objects such as spheres and cubes, but with different haptic effects and reference points. Whereas the participants in this study describe the properties of the objects verbally, Magnusson and her colleagues asked the test participants to identify the geometric objects by allowing them to select the real object [23]. Because their test participants chose the real object, which they were able to touch with both hands, it is assumed that participants might have been able to guess the object rather than identify the objects correctly. Thus, there might be data errors in identifying the properties of objects accurately in their study. To provide an independent learning solution to blind students, it is important that they do not guess at the haptically represented information.

Question 2: Will different haptic effects of the same geometric object improve the perception of haptically presented information?

Earlier research has demonstrated that it may be difficult to find objects in a virtual world. For example, Tuominen et al., in developing a tool to teach students astronomy, have designed the objects so that it is easy to navigate between different microworlds. It should be noted that finding objects in a virtual world is a recognised problem in virtual environments research [11]. This study has motivated research on the different effects of objects. The effects studied are an object in free space, an object in a bounded box and an object with a magnetic effect.

The application developed for this study adopted the following six suggestions from Sjöström’s guidelines, which were subsequently tested (see Table 1): (1) use rounded corners rather than sharp corners, (2) consider different representations to enhance different properties, (3) make sure that the models are haptically accurate and work without vision, (4) be aware that the orientation of the object matters, (5) provide well-defined and easy to find reference points in the environment and (6) avoid objects with small and scattered surfaces. To provide information using haptic sense optimally, it is necessary to test different effects on the same objects and to understand the level of perception. The advantage of virtual haptic force-feedback over a real object is that effects can be controlled by the application, and more information can be sent to the users to improve the perception of the geometric properties. By applying Sjöström’s guidelines to a prototype, it was examined whether different effects improve haptic interaction for blind users.

Table 1 Sjöström’s Guidelines to be validated

3.2 Exploring the use of haptic cues for geometry learning

The third question asked in this article attempts to explore the use of haptic cues for geometry learning. The study explores what type of information can be presented haptically so that blind students can learn geometry autonomously.

Question 3: Do haptics have the potential to allow students to conceptualise 3D geometry information?

To answer this question, several objects of different complexities are presented to participants. The study explores whether the participants can identify them correctly, how easily, and with how much confidence. In addition to familiar objects, such as the cube and sphere, less familiar objects such as cylinders and cones and the least familiar objects such as the torus and rotated cube were introduced to participants.

4 3D geometry objects

In research studies on perception, with an emphasis on visual, tactile or haptic aspects, different objects are presented to respondents. As described previously, these objects are sometimes real-life objects, such as forks or scissors. In other cases, geometry objects of varying familiarity or complexity are used ([44, 45]). Patel and Brian [46] presented a set of 12 Geons that were based on [47]. Furthermore, they suggested and were able to verify a complexity measure, where a sphere is level one, a cone is level 2, and more complex objects would be in level 3. Instead of considering the simplicity of objects, it has been stated that we choose the visual interpretation most likely to be true. Whereas those terms, i.e. complexity and likelihood, have traditionally considered to be opposite, there is an ongoing discussion on whether they can be seen as equivalent [48]. In this study, we use a term, related to likelihood, labelled ‘familiarity’. The prototype application used in this study presents two simple geometrical objects, the sphere and the cube, in three different ways: (a) objects residing in free space with the default force effect, (b) objects residing in a closed space (i.e. a bounded space with virtual walls) with the default force effect and (c) objects with magnetic effects residing in free space. In addition to the three representations of each sphere and cube, the following geometric objects are presented: a cone residing in a bounded space with a default force effect, a cylinder residing in the bounded space with a default force effect, a torus residing in the bounded space with a default force effect and a cube in a rotated position residing in a bounded space with a default force effect. These objects are described further below.

When an object resides in a free space as seen in Fig. 1, the user feels the object when a collision occurs; otherwise, the user can move the stylus around in the virtual space freely and even off the screen space. When objects reside in the bounded space, as seen in Fig. 2, the user is able to move the stylus only inside it. The area is bounded by five walls; top, bottom, back, right and left. It is constructed like a box shape except for the front wall. The box shape is open at the front so that the user moves the stylus as if they insert it into the box. The five walls restrict the movement of the stylus.

Fig. 1

A sphere and a cube in free space

Fig. 2

A sphere and a cube in bounded space

Objects that are less familiar to users, i.e. a cone and a cylinder (see Fig. 3), and those objects that are thought to be least familiar to them, such as a torus and rotated cube, are also tested in the bounded box (see Fig. 4).

Fig. 3

A cone and a cylinder in bounded space

Fig. 4

A rotated cube and a torus in bounded space

A third condition is when the special force effect is applied. The graphics of the objects are the same as in Fig. 1. The user can move the stylus freely until it collides with an object or objects. As soon as the collision occurs, the stylus is stuck to the object, and the user is able to move the stylus only on its surface.

The prototype application used in the study was developed with OpenGL for graphical rendering and GHOST SDK for haptic rendering. The default force effect was calculated using GHOST SDK. The equipment used in this study was the PHANToM™ Desktop Device from SensAble Technology Inc.

5 Experiment with Haptic Mode

5.1 Research methods

This section describes the experiment, including the instruments, participants, the test environment, tasks to be tested, evaluation measurement and experimental results. The experiment was followed by a structured interview. Because the experiment did not allow users to use the visual cue to interact with the computer, it was assumed that the level of the participants’ concentration might be high and that the conventional think-aloud method [49] often used in usability evaluations might interrupt their concentration [50]. Instead, a reflection method was adopted, but with one modification, namely that the participant could ask for assistance during the task. Normally, during a usability test session, the tester speaks as little as possible and does not engage in conversation with the test participant. According to Hanna et al. [51], children tend to ask for help when they are not certain what they are doing. Similarly, blind students usually receive constant assistance from teachers during classes at school. They are used to asking questions whenever they are uncertain about what is happening. Therefore, this experimental method allowed participants to ask questions during the session. Test participants performed certain tasks given by test personnel to conceptualise the virtual information provided.

In addition to observing the user performing tasks, the experiment collected subjective data through short interviews after the completion of each task and the entire test. Test participants answered a pre-test questionnaire, to collect background information about their computer use and computer game experience, consisting of after-task and after-test questionnaires. Finally, a short interview was conducted immediately after each task and after completion of the test, to extract detailed and specific feedback.

The test assessed potential hindrances and benefits of haptic devices for blind users through performing the following tasks: (a) identify different shapes of geometrical objects and (b) manipulate the device to touch different parts of objects. Such assessments measure how well and to what degree of accuracy users are able to conceptualise haptically presented information. The after-task questionnaire asked specific questions about the details of the users’ performances and perceptions (see the “Appendix”).

5.2 Design of the experiment

The experiment is a 2 by 3 factorial within-subject experiment. Table 2 shows for which combinations the data were collected. The following were the dependent variables: time to complete a task, time to touch the locations of objects: top, bottom, left, right, front and back, ease of identification and manipulation, confidence in control and reality of objects.

Table 2 Design of experiment

5.3 Quality characteristics and metrics

The experiment collected data based on the usability metric described in the standard ISO 9241 [52]. Usability requires effectiveness, efficiency and satisfaction, which aid users in achieving goals in a context. In addition to these aspects, the experiment collected data that relates to the accuracy of the information perception. Accuracy of information perception relates to the target goals of the task; accuracy means a correct and complete response. To measure accuracy of information perception, the following data were used: answers when asked to identify object shapes, completeness achieved when placing a stylus on a virtual object and the level of realism in the virtual reality set-up. Efficiency reflects the level of effort that it takes to accomplish a task. To measure efficiency, the following data were used: time to complete the task, time to place the stylus at certain positions on the virtual object, and data that were collected from the questionnaire to measure how easy it was to identify objects and to manipulate the device. Effectiveness relates to accuracy and completeness of the achieved goals. The number of completed tasks is used to measure effectiveness. Satisfaction relates to the comfort and overall acceptability of the task and was measured with answers from the after-test questionnaire (Q6 and Q8) and user observations. Table 3 provides a summary of the quality characteristics and metrics.

Table 3 Summary of quality characteristics and metrics

To validate Sjöström’s guidelines (see Table 1), question 1 in the after-task questionnaire was used in addition to question 2 for tasks T1–T6 (for more details, see Sect. 5.5) and questions 1, 2 and 5 in the after-test questionnaire.

5.4 Participants and testing environment

Five test participants were selected in the age range from 11 to 55 years old. All, but one of them, were students. Two participants were sighted and participated in the test blindfolded, while the rest of the participants were blind from childhood. Four of five participants had experience in playing interactive computer games with special computer controllers such as joysticks. Three of these four participants played games on a daily basis. All the participants had computer experience and used computers every day for their study, work and entertainment. None of the participants had previous experience with a haptic device comparable with the device used in the experiment.

The selection of a test place is especially important when testing with blind people, because they usually have difficulty in feeling comfortable in new places. Because the majority of participants were school-aged children, the testing site was carefully selected to make them feel comfortable, either at home or at school.

5.5 Task scenarios

Before the experiment took place, test participants were given a general overview the test environment, and the flow of the test sessions was explained. The participants received short training to learn how to manipulate the haptic device, and there was a brief discussion about the general concept of a virtual environment, a haptic device and haptic interaction.

During the experiment, the participants were asked to carry out ten tasks (Table 4) targeted to answer the three research questions. The tasks were divided into three groups, addressing familiar, less familiar and least familiar objects. In tasks T1–T6, familiar shapes of a sphere and a cube were examined. In tasks T7–T8, less familiar shapes of a cone and a cylinder were examined, and then the least familiar shapes, a torus and a rotated cube were examined (tasks T9–T10). In each task, the participant was requested to identify the shape and to place the stylus on the top, bottom, front, back, right and left sides of the objects. The final column of Table 4 shows which of Sjöström’s guidelines could be validated in the corresponding tasks.

Table 4 Tasks to be tested

6 Results

6.1 Identifying an object (tasks T1–T6)

In this section and the next four sections, tasks that involve familiar objects are discussed. All the participants were able to finish the task of identifying an object with various amounts of help. The average time to identify the objects and the standard deviations are presented in Table 5. The times vary greatly between tasks, the sphere with a magnetic effect taking the least amount of time, then the cube in bounded space and the sphere in free space taking the third shortest time. The cube in free space took the longest on average and had the largest standard deviation. For example, to identify a cube in free space, the shortest performance time was 30 s, while the longest performance time was 420 s (14 times longer).

Table 5 Average time to identify an object

All of the participants understood the following features of objects before test personnel provided assistance: a sphere was something round and a cube was something with corners. Although all of the participants accomplished the task of identifying the object within a certain time with assistance from test personnel, the first response was often different. Table 6 shows such responses.

Table 6 Users’ response for identifying object (Tasks T1-T6)

6.2 Level of assistance (tasks T1–T6)

As stated previously, all the participants needed some assistance from the test personnel while accomplishing the tasks. The degree of assistance provided is evaluated subjectively, based on the audio tape review and user observations and is labelled with L (low), M (medium) and H (high). The degree of weighted assistance was the highest with the objects in free space but the lowest with the magnetic effect. A bar chart summarising the assistance that was provided for the different effects is shown in Fig. 5.

Fig. 5

Assistance provided by effect

Examining whether the type of the effect or the type of the object had an impact on the assistance, it was discovered that the participants needed less assistance when manipulating objects with the magnetic effect, and they needed more assistance with the free space effect. This result is supported by the non-parametric measure, the Mann–Whitney U test of two independent samples, which resulted in a value of 26.5, N = 20 and showed a marginal significance of p = 0.060.

6.3 Perceived difficulty in identifying objects

After completing tasks T1 through T6, i.e. identifying the sphere and cube objects under different conditions, the users were asked to rank the different effects as free space, bounded box or magnetic effects. The question was the following: ‘Rank with number one to three, to indicate the ease of identifying the objects’, where number one indicates the easiest and number three indicates the most difficult. This question was designed to evaluate which effects helped the participants to identify the objects. The results are summarised in Fig. 6. For the cube, three users ranked the space and bounded space first, but only one user ranked the magnetic effect first. Three users ranked the magnetic effect last. For the sphere, three users ranked the free space and the magnetic effect first. Two users ranked the free space third. Taking both objects together, most users ranked the magnetic effect third and the free space first. This subjective analysis is not consistent with the results of the objective data, i.e. time performance, which showed that it took participants the least amount of time to finish tasks with the magnetic effect and they required the least assistance.

Fig. 6

Ranking by effect

Looking at the difference in the perceived difficulty across the effects, a Chi-square analysis does not show a significant difference in preference between the different effects. However, when dividing the group according to objects additionally, for the cube there is an indication that the participants preferred as easier the free space and preferred the bounded box over the magnetic effect and disliked as most difficult the magnetic effect and bounded box more than the free space.

6.4 Difference in performance between effects or objects

The average times for examining the spheres (M = 80.5 s, SD = 17.0, N = 15) are lower than for the cubes (M = 105.7 s, SD = 30,3, N = 15), and the average times for the magnetic effect (M = 69.3 s, SD = 28.5, N = 10) are much lower than for the free space (M = 123.0 s, SD = 37.8, N = 10) and somewhat lower than for the bounded box (M = 87.0, SD = 21.5, N = 10). Figure 7 shows the performance of the individual conditions. The magnetic effect did take less time on average compared with free space for both types of objects. For those conditions, it seems easier to identify the sphere. The time to identify the sphere in a bounded box is longer than when using the magnetic effect and to identify the sphere in a bounded box takes only slightly longer than to identify it in the free space. However, for the cube, it takes less time in the bounded box than in the free space and the magnetic effect environment.

Fig. 7

Means of time by effect and object

A repeated measure analysis, within-subjects, shows that the combination of object effect contributes significantly to the time that it takes to identify an object, with F(2,8) = 4.90, p < 0.05, sphericity assumed (with Mauchly’s W = 0.899, p > 0.05) and partial Eta squared η² = 0.551.

6.5 Pointing to positions on objects

All of the participants performed the tasks of pointing to a specific position on the objects within 30 s, except for three instances of 180. After the completion of task T2, all of the participants accomplished this task within 2 s, except for in one instance of 120. Table 7 shows the mean times for identifying the different locations. An analysis of variance did not show that the effects contributed to the time it took to identify the different positions of the objects. Figure 8, showing the means for the different effects, illustrates that the participants took longer to identify the locations in free space, but that there was very little difference in performance between the other two effects.

Table 7 Average times to identify locations

Fig. 8

Means of the time to identify the different locations of the objects

6.6 Results from the questionnaires after the tasks and after the test

Table 8 shows a summary of the after-task questionnaire with average scaled points. The values are rated with a 5-point Likert-type scale, where five indicates strong agreement. After-task question Q1 measures the level of ease in identifying shapes. Question Q2 measures the level of ease to manipulate a device, i.e. locating different places on the objects. Question Q3 concerns the level of the user’s confidence when manipulating a device and question Q4 the level of realism in the virtual environment. Table 8 shows that participants are very positive regarding the ease of identifying objects and locating positions, and they are confident in manipulating the objects. However, they are not as satisfied with the level of realism when using the device, for which the mean is 2.83 (M = 2.83, SD = 0.91). No difference between these questions with respect to the objects or the effects was observed.

Table 8 Summary of after-task questionnaire

After all the tasks were completed, participants answered four questions (Ate-Q1 to Ate-Q4) on their overall experiences. When asked whether it was easier to identify the shapes of some objects than others, participants answered positively (M = 4.4, SD = 0.894). Participants felt that it was equally easy to point to locations of different objects (M = 2.0, SD=1.414); they felt that the device might be moderately useful to learn about geometrical objects (M = 3.6, SD = 1.14) and that it was moderately easy to control the device (M = 3.8, SD = 1.304).

6.7 Unfamiliar objects in a bounded box (tasks T7–T10)

The aim of tasks T7–T10 is to ask participants to identify unfamiliar objects, such as a cylinder, cones, a rotated cube and a torus. Those objects have been divided into two categories, those that are less familiar and those that are the least familiar. Table 4 shows the average time to identify the less familiar objects, consisting of cylinders and cones in bounded space (tasks T7 and T8). Participants were able to identify the following features of the less familiar objects. A cone has a narrow top and a round surface that becomes larger as the stylus is moved to the lower part of the object, and it has a flat surface at the bottom. A cylinder has a flat surface on top and a round surface below the top.

An analysis of variance of the time according to the object, taking into account all of the data for the effect of the bounded box, showed that the object does not have a significant effect on the time. For the bounded box, the objects were divided into two categories in a variable called familiarity, which reflected familiar (sphere and cube) and less familiar (cylinder and cone). The descriptive statistics of these groups are the following: for familiar objects, M = 87.0, SD = 68.04 and for less familiar objects, M = 62.5, SD = 44.30, showing that the mean is higher for the familiar objects compared to the less familiar objects within the bounded box. However, an analysis of variance showed that the derived variable, familiarity, does not have a significant effect on time.

All of the participants failed to identify a cube in a rotated position and a torus in a bounded space, even with assistance from test personnel. In the case of a torus shape, none of the participants had any idea of what it was, but they identified the following features. A cube in a rotated position was identified as a house, a pyramid, a triangle or a diamond. According to the participants, the rotated cube has features that are similar to shapes such as a house roof, a pyramid or a diamond, and a torus has some round features.

Several participants also experienced some level of realism even though they did not feel objects as realistically as touching with their fingers. However, the study showed that the accuracy and detail of the virtual information that they perceived is lacking in haptic-only interaction. Participants were not able to understand the orientation of an object (rotated cube) and extra information presented around the object (objects in the bounded space). For blind students to use haptics to learn geometry independently, greater accuracy of information perception is required: a sphere must not be of an egg shape, and a cube should not be a rectangular box.

6.8 An analysis of problems

The summary of user observations (Table 9) during the test session was classified based on the problem classification of haptic interaction developed by Sjöström [28]. He classified problems into three categories depending on their background: (a) problems related to touch interaction, in real life and in virtual reality, also compared to other senses, (b) problems related to virtual interaction of any type and c) problems related to discrete point interaction haptics. These problem categories were further divided into five categories in a matrix-like manner: (1) Object, (2) Navigation and Overview, (3) Context, (4) Multimodality and (5) Learning the interaction method and specific programs. Because Sjöström deemed the five categories (1–5) to be necessary as a prerequisite for being able to work efficiently in a virtual environment, this study chose to rely only on it and viewed the former classification as supplementary.

Table 9 Observed problems in haptic interaction

The problem taxonomy proved complete, with no additional categories necessary. As with most subjective classifications and as the reader can verify, a single problem can seldom be uniquely classified into a single category. Nonetheless, the problem categorisation is useful for a better understanding and an overview of the participants’ problems. The number and nature of the problems discovered by the participants of this study unsurprisingly rely on the nature of the virtual environment that is developed. For example, context is provided to the participant through the similarity of the objects examined and the effects provided; multimodality is limited, because of the emphasis on haptic and no other senses such as audio or vision, and the navigation and overview are limited to one object at a time. Many problems are expected to be attributed to the learning category, because all of the participants were novices in haptic technology. The next Section explains in more details the guidelines that were applied in the virtual environment’s development and how effective they were.

6.9 Analysis of guidelines

The study tested a subset of Sjöström’s guidelines [28] (see Table 1), to examine whether the different effects and presentation of objects that Sjöström suggested helped participants to understand virtual information better. Four of the five items covered below are sub-guidelines of Sjöström’s first guideline, Elaborate a virtual object design of its own, and the fifth one (V-5) is a sub-guideline of the guideline Facilitate navigation and overview. Sjöström’s other three guidelines are not discussed further here because they apply more to fully working applications rather than to individual prototypes for research purposes such as the one presented here. They are the following: Provide contextual information, utilise all available modalities and support the user in learning the interaction method and the specific environments and programs. Research on visual object recognition has clearly shown that the context of the objects matter, i.e. it matters what other objects are close to the object that is to be identified; furthermore, functional groupings of objects affects perception [53]. However, the only context that can be provided to the user in the work discussed here is that most of the objects are abstract geometrical solids. Thus, although it is important to validate these guidelines, such validation is outside the scope of this article. Furthermore, this research study only utilised a single modality, and although the interaction method and the specific environments and programs were clearly explained, no formal intervention, allowing us to discriminate the results, was performed.

V-1:

Use rounded corners rather than sharp corners.

 :

When participants attempted to identify the cube, the sharp corners were keys for them to understand the object.

V-3:

Make sure that the models are haptically accurate and work without vision.

 :

The magnetic cube did not have haptically accurate effects when participants moved the stylus quickly. The force was not calculated in real time, and thus, when the stylus was moved quickly, the participants did not feel the right force and became confused. Hence, the haptic model needs to send an accurate force in a natural way. Much research has been accomplished to develop better algorithms to support creating haptically correct effects [54].

V-4:

Be aware that the orientation of the object matters.

 :

A rotated cube was used to examine this guideline. The participants experienced three cubes with different haptic representations before they tested the rotated cube. The rotated cube is similar to the cube in bounded space, except for its orientation. All the participants failed to understand the orientation of the object. They understood only a portion of the object. A new experiment is required to determine how to support users in understanding the orientations of objects.

V-5:

Provide well-defined and easy to find reference points in the environment.

 :

To test this guideline, magnetic effects on objects were added. Magnetic effects on the sphere increased the level of perception. Four participants accomplished the task to understand the magnetic sphere in less time than the spheres in free space or bounded box. Three of four participants accomplished this task within 30 s. Three participants rated the magnetic sphere as the easiest sphere to understand, and only one person rated this sphere as the most difficult one. However, there may be some variability depending on the object, because the magnetic cube was not the easiest object to understand.

 :

Another test to study this item was to represent the object in a bounded box. Bounded space restricts the area in which participants can move the stylus. If they touch the wall of the bounded area, they feel force and cannot move any farther. If they move the stylus beyond the screen space, a vibrating force is provided as feedback. These features made it easier for participants to navigate in the virtual space. The bounded box served well as a reference point.

 :

Instead of adding a magnetic effect or a bounded box around an object, as has been done in this study, it is possible to add friction to enhance the users’ perception of the object. Adichi et al. [55] showed that an additional friction effect helped users to trace the surface of the virtual object. The above evidence shows that extra effects on the object enhance perception.

V-6:

Avoid objects with small and scattered surfaces

 :

Normally, objects have specific details to represent their features. As an example of an object with a small surface, the participants tried the torus shape. The torus shape is a doughnut-shaped ring. This object has more detailed features than a sphere and a cube, but it is not as complex as a grand piano or the chair that Magnusson and her colleagues tested [23]. All the participants failed to recognise the torus and had no idea what this object was.

Evidence, analysed using Sjöström’s guidelines, indicates that people are able to understand some geometric properties of objects. However, when the accuracy of the information that is understood and the details of the object are concerned, haptic-only interaction still has limitations for the purpose of recognising objects. The results of the limited study of the five sub-guidelines prove that they are indeed mostly useful. The only one that contradicts the obtained results is the first sub-guideline discussed above, i.e. that of using rounded corners rather than sharp ones.

7 Discussions of the results

Table 10 summarises the results of the study by an overview of its metrics, i.e. the effectiveness of the tasks, their efficiency and the participants’ satisfaction.

Table 10 Overview of results of effectiveness, efficiency and satisfaction

Objective 1: To investigate the effect of haptic cues on the accuracy of information perception

Question 1: Using haptic sensory cues, are people able to conceptualise the geometric properties of simple familiar objects such as a cube or sphere?

This study shows that participants are able to conceptualise geometric properties of familiar objects such as a cube or sphere, albeit with some assistance. They could equally identify cubes and spheres and did not require more assistance with either of the objects. Additionally, participants could easily locate different locations of objects. Finally, they said that it was easier to identify some objects than others, but it was equally easy to identify the locations of the two objects.

Question 2: Will different haptic effects of the same geometric object improve the perception of haptically presented information?

Users need less assistance with magnetic effects than with free space. It also takes participants significantly less time to identify objects with magnetic effects than with other effects. Participants found it moderately easy to control the device. They liked controlling the objects in free space the most, but found it marginally easier to control the magnetic effect. Hence, there seems to be a conflict between what the participants liked and how they performed.

Objective 2: To explore the use of haptic cues for geometry learning

Question 3: Do haptics have the potential to allow students to conceptualise abstract geometry information?

Participants had difficulty when identifying the least familiar objects. Considering the objects in bounded box only, there was not a significant difference in the time that it took to identify familiar and less familiar objects. Participants found that it would be moderately useful to learn about geometrical objects. Participants were not as satisfied when asked about the level of realism using the device.

Overall, the users were quite successful at manipulating the device and identifying the objects. Beforehand, the users were expected to find it easier to manipulate the objects with the magnetic effects than in free space and the bounded box, but the participants of the study seemed less satisfied with the magnetic effect and the bounded box than with the free space. It is likely that this result arises from the users preferring to be active participants and preferring to be unconstrained in their actions, which is consistent with the findings from previous studies [7]. It was also expected that the time to identify an object using the magnetic effect and the bounded box would be longer. In a study by Jay et al. [56], similar to the one described in this article, a magnetic effect is used to guide users around an object, and a spring is used to guide them from one node to the next. It was noticed by Jay et al. that, in experiments with different haptic and auditory cues, asking blindfolded sighted users to identify structures of spheres and cubes, the authors find that haptic cues aid in structure recognition but that audio cues do not. Haptic cues combined with audio cues enabled participants to identify nodes and structure the most quickly and the nodes more accurately. An observation from the results of this study, which is supported also by the study by Jay et al., is that Miller and Zeleznik’s guiding principle [57], which states that the motions that a user is undertaking should support but should never control a user’s input, holds true.

The fact that the users did not have any difficulty in identifying four of the six objects and that, for the bounded box, there was no significant difference in the time that it took, confirms that the haptic cue for representing 3D objects has a significant potential. On the other hand, the users’ inability to identify the most complex objects, a torus and a rotated cube, could be due to a combination of the inherent complexity of the objects and the users’ unfamiliarity with them. Allowing the users to compare to real objects could have mitigated their unfamiliarity but, at the same time, could have skewed the results, as mentioned at the onset of the paper. It is well known that tasks become easier if they are broken down into components. Testing whether such a breakdown could help subjects to identify unfamiliar objects, Ehrich et al. [58] found that the synthesis of haptically sensed shapes is based on simple geometric objects and that this sensing can be strongly influenced by expectations.

The use of haptics for presenting geometry objects can be expanded to learning subjects other than mathematics. For example, Minogue et al. [59] assessed whether students using visual and haptic interaction gained more knowledge in their concepts of animal cells compared to those students who were only assisted by visual interaction. While there was not a significant difference in the knowledge increase of the visual plus haptic group over the visual group only, the study found that students in the former group were less frustrated and less disoriented.

The main limitation of the current study is the small number of participants. To make up for this limitation, the participants were asked to perform several tasks. The effect of this limitation is in the quantitative analysis. Especially where no significant difference could be found, e.g. in the time taken to identify familiar versus unfamiliar objects, it cannot be excluded that such a difference could be found if more cases were studied. Another disadvantage is that the users were not allowed to evaluate difficult objects in free space or magnetic effects; they only evaluated complex objects in a bounded box. Finally, it must be carefully considered that the software of the haptic arm determines how the users experience the objects. Hence, some of the difficulties that were encountered may be attributed to the models that are programmed for the device, and hence, a less than optimal representation was offered to users.

8 Conclusions and future work

The contributions of this article are the following:

• a. Validation of haptic guidelines. Guidelines on haptic user interfaces have been put forward by Sjöström, based on past research. However, these guidelines have not been applied or validated since.

• b. Validation of the usefulness of haptic taxonomy for expressing user problems. The results can raise the awareness of researchers and practitioners with respect to haptic taxonomy.

• c. Systematic research into the haptic perception of 3D objects, using three different effects: magnetic, bounded box and free space. The results provide an improved understanding of the different effects on haptic perception.

• d. Research into the haptic perception of real objects has been extensive. It has researched participants’ responses to objects of varying complexity and familiarity. This article transfers this knowledge to the haptic perception of virtual objects, which has not been previously accomplished in the literature.

The answers to the three research questions stated at the beginning of the article are summarised below, with ideas for further work.

Question 1: Using haptic sensory cues, are people able to perceive geometric properties of simple familiar objects such as a cube and sphere?

The conclusion is that users are able to perceive geometric properties of familiar objects, albeit with some assistance. This study showed that users could identify simple objects and could easily touch different parts of the objects. This study allowed users very little training; however, it may be reasonable to investigate further what effect such training could have. While this study has not analysed what type of assistance users needed, future work could analyse this issue further, with the intention of providing the user with the identified computer-aided assistance.

Question 2: Will different haptic effects of the same geometric object improve the perception of haptically presented information?

There is an indication that users were faster and needed less assistance with the magnetic effect. Users were fastest when identifying objects with the magnetic effect and needed the least assistance, but they were not as satisfied with it as with the other effects. Future work would include combining the magnetic effect and the bounded box approaches.

Question 3: Do haptics have the potential to allow students to conceptualise abstract geometry information?

When considering the haptic peripheral for learning geometry, the present study shows that it has some limitations in that detailed information is difficult to understand and the accuracy of information perception is not good enough to provide independent learning material for blind students, although they are able to understand some geometric properties. While the results of the study show that haptics have the potential for allowing students to conceptualise 3D objects, much more work is needed to exploit this technology to the fullest. The more complex objects are difficult for students, and, in their opinion, the virtual objects as presented leave much room for improvement.

Attempting to make an impact in as complex a field such as haptics technology for blind students is a marathon at best. The results of this article will hopefully convince readers that a theory-based discussion of the design of haptic cues merits attention and requires further studies. As was so well-stated by Minogue and Jones [27], there are formidable barriers to the widespread use of haptics in education. These barriers can be characterised as perceptual, technological and methodological. With regard to the perceptual barriers, this study concludes that different haptic effects for more complex objects should be examined further, e.g. a combination of magnetic effects and a bounded box may be useful. Using the magnetic effect requires more training than is provided in the trial, and the results of the questionnaire showed that users may feel too constrained using the magnetic effect. If the bounded box shows as good a performance as the magnetic effect, then it may be more palatable to users. In relation to the second barrier, the technological one, the results of the study show that more advanced techniques to implement haptic cues are necessary to enhance realism. Finally, to overcome the methodological barriers, this study has applied and evaluated guidelines, thus attempting to translate research into applications development. Nevertheless, this study shows some ways of translating geometry objects to a learning environment, in which it should especially be investigated how much training students need to manipulate the haptic peripheral more confidently.

Appendix

1.1 After-task questionnaire

There are five scales to grade your response. Answer 1 indicates ‘Strongly disagree’, and 5 indicates ‘Strongly agree’. You will grade your responses by choosing a number from 1 to 5.

Q1:

It was easy to recognise the object.

Q2:

It was easy to point out the specific place of the object.

Q3:

I was confident in working with the machine.

Q4:

I was able to recognise the object as easily as touching it with my fingers.

1.2 After-test questionnaire

For questions 1–4, please rate your answer with a number between 1 and 5.

1 indicates strongly agree, and 5 indicates strongly disagree.

Q1:

For some objects, it was easier to identify the shape than others.

Q2:

For some objects, it was easier to locate the stylus than others.

Q3:

This machine might be useful for learning about 3D shapes.

Q4:

It was easy to control the machine.

Please tell us your opinions.

During this test, you experienced feeling some objects in a virtual environment.

Q5:

If you felt each object differently, what made you feel an object better or worse than the other object?

Q6:

Do you feel that you have controlled the machine well? If not, what made you feel that way?

Q7:

Do you think that this machine is useful for some activities you perform with a computer?

Q8:

Please tell us your emotions while you were trying to accomplish each task. Was it fun, frustrating, boring or interesting?