Keywords

1 Introduction

Over the last few decades, Information and Communication Technologies (ICT) have assumed an increasingly important role in the teaching of mathematics and science. Computers in the classroom have become an indispensable tool for supporting teaching and learning [14]. Innovations in ICT have made computing a ubiquitous phenomenon where devices such as computers, tablets, and smart phones are widely adopted in our daily lives as well as in educational activities.

Advances in computing and multimedia have enabled students to visualize and engage with mathematical concepts that were not possible with earlier systems or with the traditional resources such as textbooks. There are several kinds of software applications that can be used to aid math education [1]. Main types of mathematics education software that are popularly used by practitioners are predominantly Dynamic Geometry software, spreadsheets and Computer Algebra Systems (CAS) [5]. Many pedagogical environments for math education have been developed, such as Cinderella (www.cinderella.de), Geometer’s Sketchpad (www.keypress.com/sketchpad), Cabri geometre II+ (www.cabri.com), and GeoGebra (www.GeoGebra.org), sampled among many other applications. This study focuses on GeoGebra, because it is a free Dynamic Geometry Software (henceforth DGS) that also provides basic features of a Computer Algebra System to bridge the gap between math domains such as geometry, algebra and calculus [4] and is freely available at www.geoegebra.org. This software combines geometry, algebra and calculus into a single and easy package for teaching and learning mathematics from elementary to university level [6].

According to Hohenwarter and Preiner [6], GeoGebra appears to be a user-friendly software that can be operated intuitively and does not require advanced skills to get started. It is easily accessible from home as well as from school via multiple platforms. Students can practice, do homework, prepare for their lessons and revise from home. It also supports multiple languages and is a great asset for classrooms that have multilingual learners. They can create and share their constructions by using the GeoGebraWiki tool or use the templates provided to customize the tool for their learning needs. There is a user forum where students and teachers can share ideas and discuss math problems [7].

Domènech [4] stated that students encounter many types of difficulties when learning mathematical concepts and solving such problems often require coordinated reasoning over symbolic expressions and visualizations. Although students can face structural and visualization problems when learning geometry, developing deductive reasoning skills can be considered as the biggest challenge for the students. In particular, students may have difficulty moving from geometry based on shallow visual properties to a geometry based on a deeper understanding of the structural patterns that bring together primitive objects such as points and lines for constructing more complex geometric representations [4]. For example, students can observe how changing the radius of a cylinder changes the side area both graphically and symbolically in an environment like GeoGebra. In other words, students can observe the implications of a visual action on the quantities and vice versa, which will help them understand the relationships among different ways to represent the same mathematical concept. Realization of such connections among different representations is considered as an indication of deep learning of mathematical concepts, and dynamic geometry software has the potential to stimulate and facilitate the development of such deep level of understanding.

The realization of such benefits depend on to what extent the interface effectively supports students to construct and manipulate dynamic representations. Systems such as GeoGebra requires users to add and manipulate basic primitive constructs such as points, angles, lines, and circles. These primitives need to be combined in specific ways to construct even more complex objects that are typically used in the math classroom, and combining objects often involve specific interface actions such as selecting two points to combine with a line, or dragging a point to change its coordinates. Although these interface actions are based on traditional mouse-based gestures used for desktop applications, learning appropriate use of the features for building math representations may not necessarily be a trivial matter for the students. Consequently, usability issues involved with the design of the interface elements and gestures acting on them have important educational consequences. However, systematic usability studies of primitive interface elements provided by dynamic geometry tools are not widely covered in the literature. Existing evaluations tend to focus more on pedagogical aspects, with the exception of a few studies focusing on usability concerns.

To the best of our knowledge, there are two main studies focusing on the usability of similar DGS software. In the first study, Hohenwanter and Lavizca [8] evaluated the difficulties encountered by the participants while using basic GeoGebra tools through a questionnaire. This study was carried out with the participation of 44 mathematics teachers, where they were asked to rate the GeoGebra tools from 0 = very easy to 5 = very difficult. According to the results, Hohenwanter and Lavizca classified the tools in terms of their difficulty of use, and stated that “easy-to-use” tools can be discovered via individual experimentation at home or school, the “middle” difficulty tools should be demonstrated by the instructor, and the “difficult-to-use” tools require both guidance and planned exercises to master their use [8]. The second study was conducted by Konterkamp and Dorhman [9], where they used a prototype of Cinderella to investigate the affordances of multi-touch screens for constructing geometric objects. In particular, they evaluated how Cinderella supports multi-touch features. Their findings pointed out that existing interfaces do not adequately take advantage of multi-touch features and function primarily as an extension of pen-based single touch interfaces [9].

Given the recent proliferation of mobile DGS applications and their increasing potential for widespread use, this study aims to explore to what extent the recently released table version of the popular GeoGebra system makes effective use of the affordances of mobile user interfaces. In particular, the purpose of this study is to explore the effectiveness of the iPad version of GeoGebra, and to suggest interaction design ideas to improve upon the detected usability issues.

2 Methodology

In this study, we conducted a usability evaluation of the recently released iPad version of GeoGebra by using a Tobii X2-60 eye tracker mounted mobile stand. The study aims to identify usability issues in the present mobile interface in an effort to explore ways to improve students’ engagement with geometric reasoning by constructing, manipulating and reflecting upon geometric objects. Based on the ISO 9241-11 [2] definition of usability, the evaluation focused on efficiency, effectiveness and satisfaction dimensions.

2.1 Design of the Study

Participants.

Our sample included 10 participants who are graduate students at a local university, which presents one of the target user groups for the developers of dynamic geometry software. The average age of the participants was 26.5 (range: 24–31). The majority of the participants were female (7 female, 3 male). Subjects from several different specialty areas in field participated. 5 of the participants had an Engineering Sciences background, 3 participants were enrolled in Basic Sciences (Math & Statistics), and two participants were majoring in Educational Sciences. Before the experiment participants were asked to rate their basic math and computer skills between 1–9. On average the participants rated their computer skills and basic math skills as 7.8 and 6.2 respectively. None of the participants had any prior experience with GeoGebra or a similar dynamic geometry application (Fig. 1).

Fig. 1.
figure 1

GeoGebra Tablet Interface

Full size image

Materials, Apparatus and Software.

The first data collection instrument employed is a scaled down version of the System Usability Scale (SUS) questionnaire developed by John Brooke from Digital Equipment Corporation [3]. This scale allows researchers to carry out a quick and practical usability evaluation of a given user interface. As Sauro argued SUS can be used on very small sample sizes (as few as two users) and still generate reliable results [10].

In this study, a Tobii X2-60 eye tracker was used together with a special stand for mobile devices (Fig. 2). This eye tracker device can track both eyes of the participants at a rate of 60 Hz to gather information about the duration and the location of their gaze movements by using infrared cameras [13]. The Tobii X2-60 is a small and portable eye tracker, so it can be used for usability studies conducted with different interfaces such as laptops, mobile devices, as well as real word interfaces and TV screens [12].

Fig. 2.
figure 2

Average time spent and mean number of fixations during visual search and construction episodes across all tasks.

Full size image

2.2 Data Collection Procedure

During the usability study, a set of geometric construction problems that could be completed within an hour were given to each subject (Table 1). The tasks were adapted from the tutorials prepared by the Virtual Math Teams (VMT) project team [11]. Before the experiment, frequently used GeoGebra tools while solving geometry problems were introduced together with an example for constructing an equilateral triangle by using dynamic circles. The tasks we used in this study were related to the use of basic features of the system. We did not expect the users to reach perfect solutions to the given problems in this study. We just sought to find out if they could construct an acceptable dynamic geometry presentation and, if yes, how much effort they put in constructing it.

Table 1. Tasks of the study
Full size table

2.3 Data Analysis

In this study a mixed methods approach was employed for the analysis of usability issues involved with the tablet version of GeoGebra. After the experiments, data gathered from the eye-tracker were analyzed quantitatively. For this analysis, areas of interest (AOI) over the GeoGebra interface were defined, and then various eye tracking measures were extracted such as time to total visit, mouse click count, percentage of time spent on an AOI, number of fixations prior to first fixation on an AOI, percentage of participants who fixated the target at least once by using the Tobii Studio Software. In addition to this, a task analysis was conducted to guide the statistical analysis of the eye tracking data. The overall SUS rating for the Geogebra mobile system is calculated based on the participants’ responses to the questionnaire they filled after the experiment. Finally, participants were asked open-ended questions after the experiment about the difficulties they had while using the GeoGebra mobile interface.

3 Results

3.1 SUS Evaluation

Table 2 summarizes the SUS scale ratings of the participants collected after the experiment. The SUS score (47.0) was found to be below average.

Table 2. SUS scores of the study
Full size table

3.2 Task Analysis

The analysis of task performance is carried out in 3 steps. First, overall measures of accuracy and completion times are provided for all tasks. Next, the analysis is elaborated further via a hierarchical task analysis, where the sequence of actions performed by subjects in each task is compared with respect to expected solution steps. Finally, the analysis is further developed with eye tracking measures, which aim to provide further insights regarding the attentional resources participants used while attempting the construction tasks.

First, we identified the number of correctly solved and unsolved cases for each task. All participants were able to complete tasks 1, 2 and 4. Participants seemed to struggle the most with tasks 5 and 6. One participant failed to complete task 3. On average subjects took more time to complete tasks 5 and 6. The length of the interquartile range is also higher for tasks 5 and 6, which indicate a higher level of variability among participants as compared to other tasks. Since the task completion values were not normally distributed, a non-parametric Friedman’s ANOVA test was used for statistical comparison. Friedman’s ANOVA showed that there is a significant difference among the tasks in terms of their completion times, χ2 = 24.19, p < .01. Follow up pair-wise comparisons with Wilcoxon Signed Rank tests found that the difference between tasks 1 and 4 (z = −2.70, p < .01), 1 and 5 (z = −2.19, p < .05), 1 and 6 (z = −2.37, p < .05), 2 and 3 (z = 2.19, p < .05), 2 and 4 (z = 2.80, p < .05), 2 and 5 (z = 2.20, p < .05), 2 and 6 (z = 2.37, p < .05), 3 and 5 (z = 2.20, p < .05), 4 and 5 (z = 1.99, p < .05), 4 and 6 (z = −2.37, p < .05) were statistically significant.

In order to further analyze the task performance of the participants, transcripts that capture the sequential list of actions performed by each participant while attempting the geometric construction problems from the eye tracking videos. These transcripts capture a short description of each move, its time-stamp or time duration, the total number of fixations and the average fixation duration logged during that move. Each line of action in the transcript is classified into three basic categories; visual search, construction actions, and actions that indicate failure or repair. Visual search refers to those segments where the user visually scans the interface without tapping on any items, indicating that he/she is searching for the relevant system features. Construction actions refer to drawing new objects such as adding a point, line, etc., Repair or failure actions include cases when the user performs an undo, erases an existing part of the dynamic drawing, or decides to quit the task. A total number of 1373 action descriptions were categorized.

When we investigate the transcripts, we see that participants spent 22 % of their total time on searching for relevant drawing features that they may use to solve the task at hand, 58 % of their total time while constructing drawings, and 20 % on repairing or erasing existing parts of a drawing.

For each segment categorized as visual search or construction, average fixation duration and the number of fixations were also recorded as indicators of efficiency and cognitive workload. Since undo and erasing actions took on average small amount of time, those segments were not subjected to fixation analysis. Figure 2 shows the distribution of average time duration for each visual search and construction action. A two way ANOVA analysis showed that the average time spent on visual search was significantly higher than average time spent on construction actions, F(1,1013) = 10.093, p < 0.05. There was also a significant interaction effect, F (5, 1013) = 2.655, p < 0.05. This is due to the fact that the time spent on visual search is especially higher than construction in tasks 3 and 6, which indicates that subjects had more difficulty finding related drawing features during these tasks. There are also cases such as tasks 1 and 5 where visual search and construction actions had similar average time.

Figure 2 shows the distribution of total fixation counts for each segment type across all tasks. A 2-way ANOVA showed that the visual search segments have significantly higher number of fixations as compared to construction segments, F(1,1008) = 13.472, p < 0.01. The difference was particularly high for tasks 3 and 6, which suggest that subjects searched the interface more vigorously in these tasks. The interaction of segment type and task was also significant, F(5,1008) = 2.280, p < 0.05, which is due to the fact that some tasks such as 1 and 4 had almost equal mean fixation counts for search and construction segments.

Figure 3 shows the distribution of average fixation duration values observed in search and construction segments for all tasks. A 2-way ANOVA conducted on average fixation duration values showed a significant effect of segment type, F (1, 1008) = 9.372, p < 0.01. Construction segments have higher average fixation values than visual search segments. The interaction effect was not significant, F (5, 1008) = 0.991, p > 0.05, so the pattern of relationship is preserved across different tasks. This suggests that the fixations that guide the construction of dynamic figures tend to elicit higher average duration values than fixations that guide the search process.

Fig. 3.
figure 3

The distribution of average fixation duration values in each segment type across all tasks

Full size image

4 Discussion

Our first research question is concerned with the usability issues involved with the tablet version of GeoGebra. Our main findings regarding this question are discussed under the sub-titles of effectiveness, efficiency and satisfaction.

4.1 Effectiveness

In order to evaluate the effectiveness of the GeoGebra interface, we focused on the task accuracy statistics. Tasks 1, 2 and 4 were accurately completed by all participants, whereas tasks 3, 5 and 6 were not accurately completed by 1, 4 and 3 of the participants respectively. Tasks 3, 5 and 6 included additional problem solving steps, whereas other tasks could be considered more routine application of the drawing tools. However, the main goal of GeoGebra is to help students explore geometry through building such dynamic constructions, so we believe such tasks are still relevant within the context of a usability study.

The highest number of failures in the experiment occurred while participants attempted Task 5. There were two important issues in Task 5. The first one was to draw 3 parallel lines by using the parallel line tool. The other was to construct an equilateral triangle. The problem mostly faced by the participants who used the parallel line tool was for them to put points instead of drawing a line. To use the parallel line tool in GeoGebra, it was necessary to choose another line firstly and then to draw another line parallel to it. In other words, after clicking the parallel line tool, firstly the line targeted would be clicked and then the screen would be clicked to draw the desired parallel line. The participants who did not know at first that they needed another line faced this problem, and then formed a line to solve this problem, but this time they went on putting points as they did not know the order of clicking. Although the drawing of an equilateral triangle, which is another element of the task, was easy for some participants, it proved difficult for others. Unable to notice that an equilateral triangle could be drawn by using the regular polygon tool, the participants lost time using the other polygon tools.

4.2 Efficiency

We considered the time and the number of steps each task took as indicators of efficiency in this study. The box-plot in Fig. 3 shows the distribution of completion times measured in seconds for each successfully completed task. The box-plot shows that on average subjects took more time to complete tasks 5 and 6. The length of the interquartile range is also higher for tasks 5 and 6, which indicate a higher level of variability among participants as compared to other tasks.

When the number of steps it took participants to complete each task was examined, we found that Task 6 took the most number of steps as compared to all other tasks, which on average took 40 steps to complete. While completing this task, most of the participants had difficulty forming circles with the same radius that intersect with each other. To intersect them accurately, they were required first to draw a circle with center A and passing through point B, and then click on point B to select it as the center of the second circle and then select point A to let the new circle pass through A. Participants ended up drawing two intersecting circles that did not share the same radius. Another case in which the participants had difficulty was to form new intersecting points on these intersecting circles. Most of the participants searched for the intersection tool for a long time. It did not occur to them that the point tools could be in the subtitle. Considering the task and the way the tool is used, they seemed to ignore the possibility that the point tools would be in the subtitle, and so they looked for them at different parts of the interface for a long time. Participants also had problems with using the intersect tool. This was probably because the expected way to use the intersect tool requires users first to click on one of the circles and then on the other circle. Those who tried to put the intersecting point immediately without doing these two steps process failed. Another problem frequently faced by the participants was the accidental deletion of all existing constructions on the screen. The users, therefore, had to redraw the circles which they had already formed previously.

4.3 Satisfaction

Participants’ satisfaction with the tool was investigated through the SUS scale results as well as the post interview comments obtained via open-ended interview questions after the experiment. Table 2 summarizes the SUS scale ratings of the participants for the Tablet version of GeoGebra as 47.0, which was below average according to Sauro’s criterion [8]. This suggests that there are important usability issues limiting users’ overall satisfaction with the interface.

Participants’ post interview comments provided further insights into their level of satisfaction with the GeoGebra interface. For instance, Participant 2 reported that she was bored because of not being able to do what she has done in GeoGebra in one time. Noticing the input bar and finding “double click to open input bar” message was difficult. Moreover, she stated that she could not put the points in specific places with precision. The necessity of selecting the targeted object or the drawing tool at every time was boring. The undo button sometimes undoes everything to the initial state. Similarly, participant 3 stated that typing a function into the input bar makes it harder to select the points that are not in the axes and the use of the Erase tool was not very clear. Participant 4 reported that priorities can be given while two different things such as point or line are input. Participant 5 stated menus in which some geometry shapes are provided together with text descriptions of the functions was not user-friendly. Moreover, he also commented that input area where functions are written was hard to see. Participant 6 reported that she could not get used to touch pad since her hands sweat. She could not understand what the slider tool really does. She emphasized she could not draw a triangle by entering 3 in the Regular Polygon tool as input. Moreover, she stated that it was not easy to understand what can be done in the input bar. Participant 7 also commented that the Input bar was not user friendly. Furthermore, she reported it was hard for her to draw 2 intersecting circles and it was so difficult to mark a desired point. Participant 8 stated that Pictures (Icons) do not indicate the purpose of the button. She suggested that any help of the system will make it easier to be used. Participant 9 reported that while he was drawing a polynomial curve, it takes time to find that the function should be inserted in the input area.

4.4 Suggestions for Improvement

Our usability analysis highlighted some of the difficulties users faced when they were given the task of making specific constructions with minimal supervision. Some of the issues we found could be detrimental to the broader educational adoption of the tool, so we also aimed to suggest some interface design improvements that may help mitigate some of the usability issues we found out.

A peculiar issue our participants had with the angle tool was to put the angles in the desired part of the drawing. In the current interface you need to click on three points in either clock-wise or counter-clockwise direction to mark an internal or external angle. Alternatively, the system could allow users to select the location of the angle with a hand gesture similar to how we draw angles on paper by drawing a small arc connecting two existing line segments. In this new feature, after the user selects the angle button, he will draw a short arc touching on both segments between which the angle should appear. Until the user lifts his finger from the screen, the system can display a visual feedback by highlighting the line segments implicated and the anticipated area where the angle will appear. Such a feature would simplify defining angles by eliminating the need to identify 3 points in a specific order, and providing a more naturalistic method based on a familiar with drawing.

Given the limited display size, it is generally difficult to make precise drawing or editing actions on a tablet window. There are existing solutions for guiding text editing movements over mobile interfaces where users can hold and drag to open a zooming lens, which allows them to make more precise movements such as inserting the carat or highlighting a specific portion of the text. Intersecting geometric objects, placing points at specific places may benefit from such guidance as well.

Participants also experienced difficulty with managing text input via interfaces such as the Input Bar and the Redefine window. The fact that GeoGebra is making use of the standard keyboard of iPad seems to be contributing to this issue. A keyboard that is optimized for the math notation expected by GeoGebra would make such interfaces easier to use for the users.

Problems faced by the participants with moving and deleting existing objects on the interface motivated another possible design improvement. In the current interface, before moving an object one needs to click on a button to switch the cursor into the selection mode, which requires an additional step almost always forgotten by the users. In such cases users ended up moving only a part of the construction, which distorted the overall structure, and required the use of the undo button. Since the moving operation requires clicking on and dragging the object, when the user clicks on the object for a brief amount of time, a context menu may open (e.g. like a right click does on most operating systems) which may present a few alternative actions such as Move or Delete.

5 Conclusion

In this study we conducted a usability evaluation of the recently released iPad version of GeoGebra, which is one of the first dynamic geometry applications that allow users to construct and view dynamic figures on mobile devices. Overall, we aimed to identify if GeoGebra is effectively taking advantage of touch-based gestures to support the construction of dynamic geometry objects, and to explore in what ways the interface can be explored to make abstract geometry concepts more tangible for students.

Our findings suggest that the mobile version primarily replaced the function of the mouse in the desktop version with the finger, and did not take advantage of the gestures supported by the multi-touch screens of new generation tablet computers. GeoGebra should take better advantage of the unique affordances of multi-touch features, such as the ones proposed by Konterkamp [9] in the context of dynamic geometry over smart boards, where users can draw a line by using two fingers at the same time, or draw a circle by having one finger as the center together with an arc gesture circling around the center. We believe recognizing such gestures will contribute to the overall usability of the GeoGebra mobile interface and its further adoption by the educational community. In addition to this general lack of support for multi-touch actions, we identified other usability issues and offered possible interface design ideas that may help users overcome such issues.

The existing GeoGebra platform is mathematically very versatile and comprehensive, and it holds great potential for transforming math education at all levels starting from elementary school to graduate level education. However, making those versatile features difficult to access will inevitably hamper the general adoption of the tool and the effective use of dynamic constructions for math learning. We hope that the open source community supporting the development of GeoGebra will address the issues identified in this study in the near future.