Keywords

1 Introduction

Collaborative learning is an effective educational method that plays an essential role in theories such as Constructivism and Knowledge Building [13, 18]. Studies have shown that collaborative creative work in small groups can significantly improve learning when supported by concept maps [11]. Concept maps are tools to organize, represent, and structure knowledge relationships in a graphical format. Although useful in collaborative environments, most applications for the development of concept maps are designed for personal or small tablet computers, which can limit student communication in a team setting. In addition, the use of these applications usually requires training periods that may reduce the time allotted for regular learning activities.

Studies suggest that large horizontal interactive surfaces can effectively promote collaboration and face-to-face communication [2, 6, 14]. These surfaces often provide multi-touch interfaces that implement common hand gestures for basic operations such as zoom, pan, and rotate. In this paper, we present a software tool for developing concept maps that combines tangible interactive controls with multi-touch and multi-user capabilities. Tangible controls provide a richer and more direct interaction mechanism than plain visual representations on the computer screen normally do [3–5, 20]. Multi-touch surfaces make interactions more natural and intuitive.

Our interactive surface is designed around an optical infrared frame mounted on a commercial large screen high-definition TV set. The optical frame supports the simultaneous detection of a large number of contact points, which allows the use of physical objects (which can be 3D printed) to reproduce various touch patterns, similar to those performed by explicit hand gestures on the table. These patterns are then recognized by our system and used to identify what tangible element is being used as well as to calculate its position and orientation on the surface. As a result, specific functionalities can be assigned to physical elements based on its position and orientation on the surface (i.e. to simulate the physical interaction with a rotating knob).

Our application uses tangible elements as a method to manage information locally on an interactive surface and as a natural mechanism to exchange information between different surfaces to support distributed collaborative design work. A preliminary usability evaluation shows that the application is intuitive and provides sufficient functionality and resources to build large concept maps of relatively high complexity.

2 Distributed Concept Maps

Tabletop systems offer a large interactive surface on which several users can work collaboratively and communicate face-to-face. In this regard, some researchers have used tabletops to create concept maps in collaborative digital work spaces [9].

Although the work space of a tabletop system is large, it is difficult for more than three or four users to work simultaneously in a comfortable and productive manner. In collaborative work scenarios with many users, it is possible to define a distributed model where a set of tabletop devices connect to a common digital work space. In [16], an implementation of this model is described. The model allows the extension of the local collaborative workspaces that are established around a physical interactive surface, into a distributed setting that allows the interconnection of remote devices. Therefore, a common virtual workspace is generated, similar to the concept of cloud. The interaction with the system is based on Natural Interfaces. Unlike other solutions, our technology allows designing complex schemes of classroom orchestration in a natural and simply manner.

A software application for the construction of concept maps was developed based on the distributed collaborative work space model [16]. The system allows the construction of concept maps in this digital space, so each tabletop connected to the system acts as an interactive window to the common space (Fig. 1). The system provides a natural interface that minimizes learning curves and allows the addition of new information through BYOD mechanisms (Bring Your Own Device). For example, users can use their smart phones to send text and images to the system or by e-mail or through the social network Twitter.

Fig. 1.
figure 1

Representation of a distributed collaborative work space

Full size image

In collaborative environments, the exchange of information between groups is especially important. Our system facilitates these tasks by implementing operations such as copy/cut/paste, which propagate hierarchically through the directed graph defined by the concept map. Thus, a user can copy part of a concept map by selecting the root node of a subtree and clicking the appropriate option on the context menu provided by the system. By using zooming and panning operations, users can “virtually” move over to the working area of a different group of users and perform a “paste” operation.

However, this mechanism can become confusing to some users in environments where several working groups are distributed in a room. In this scenario, where users carry their devices around the room, the fact that information can only move virtually without involving a physical displacement of users, may be perceived as unnatural.

These impressions are reflected in the usability test performed in [16]. The results of this study (using a Likert [8] scale of 5 values) are shown in Table 1. While results were positive in all cases, teamwork (question 6) received the lowest score. Subsequent interviews with users who participated in the experiment revealed difficulties in tasks related to the exchange of information among different working groups.

Table 1. Evaluation test questions and results from previous study [16]
Full size table

The integration of new mechanisms based on physical manipulators of information can simplify the exchange and manipulation of digital information between working cores, making it more natural for users.

3 Tangible Interface

A Tangible User interface (TUI) is a user interface where users can interact with digital information through a physical medium. Natural interfaces give digital information a physical form, which simplifies user interaction by becoming metaphors for handling physical objects and materials (actions that are performed instinctively).

TUIs have been widely used in teamwork and collaborative learning environments [3–5, 20]. In our previous work, we used tangible items over tabletop systems to improve collaborative work in teams [1]. Our system used tangible elements to represent conceptual map nodes, while transitions were represented graphically by the tabletop. Similar systems were described in [12, 19]. For instance, in [19], both nodes and transitions are represented digitally, and tangible manipulators are used as tools to perform actions (create and move nodes, create and label links, etc.). Other studies use tabletops for the implementation of concept maps without the use of tangible items [9]. From the standpoint of collaborative work, results are satisfactory at the local level, where multiple users work simultaneously over a single interactive surface.

In this paper, we propose the use of tangible manipulators to implement information exchange operations in a distributed scenario between different tabletop devices of different working groups. Therefore, the transfer of information becomes a physical action, similar to what happens in a traditional work environment, where the physical transfer of information involves the movement of users.

3.1 Using Tangibles to Exchange Information

The information exchange between tabletops using tangible elements is implemented as cut/copy/paste operations. For these tasks, two tangible marks are used, the first one associated with the ‘cut’ operation and the second one associated to the ‘copy’ operation. The two operations are similar. Only ‘cut’ deletes information from the source tabletop. In both cases, the information is temporarily stored in the clipboard (supported by the application server). Placing a ‘cut’ tangible on a concept map node will copy the node and its hierarchically dependent child nodes to the clipboard and delete it from the source working area. The ‘copy’ operation is similar, but the original information will not be deleted from the source. By placing the ‘tangible manipulator’ on a different tabletop system (or on a different location on the same tabletop), the hierarchical structure stored in the clipboard will appear, with the root node directly under the manipulator.

3.2 Additional Operations with Tangibles

Tangible manipulators are also designed for zooming and panning operations. For zooming, the tangible manipulator must be placed on the tabletop surface and rotated. Clockwise rotation implies a positive zoom, while counterclockwise rotation implies a negative zoom. Similarly, by moving the tangible associated with the panning operation, it is possible to move the displayed area of the work plane.

4 Implementation Details

4.1 Tabletop System

Since the appearance of the first tabletop devices, several types of solutions and technology implementations have been used [10, 17]. The use of infrared optical frames [1] allow to convert any flat screen into a multi-touch tabletop device. Currently, the affordability and availability of this type of frames has increased significantly, even for very large screens formats, reaching high resolutions and high rates (>=200 Hz) in the detection of touches. The type of tangible interaction described in this paper is not common in optical frame based tabletops.

4.2 Tangible Marks

In our previous work [15], we developed a system of tangible manipulators based on the use of physical objects that were completely passive. The position and orientation of these manipulators could be identified and calculated in real time when placed on the working surface to reproduce various touch patterns, similar to those performed by specific hand gestures on the interactive table.

An example of the developed tangible passive manipulators (corresponding to id’s 2, 3 and 5) is shown in Fig. 2. If necessary, the physical marks can be labeled according to its functionality. The basic element consists of a cylindrical base with three pins, which describe an isosceles triangle. This basic element corresponds to the handler with ID = 0. Since the three pins form a triangle, the element is stable on the surface of the tabletop and its circumcenter defines the XY position of the manipulator. Additionally, the orientation of the mark (an isosceles triangle) can be easily determined.

Fig. 2.
figure 2

Example of passive tangible manipulators placed on an optical frame surface

Full size image

The codification of the different IDs is performed by adding additional pins between the two pins that define the short side of the isosceles triangle according to a binary encoding in base 2 arrangement. Therefore, the number of different manipulators can be 2n, where ‘n’ is the maximum number of pins that can be placed between the two pins that define the short side of the triangle. This number depends on the resolution of the optical frame and size (radius) of the manipulator. An example of eight possible encodings using three pins is shown in Fig. 3.

Fig. 3.
figure 3

Codification of eight manipulators using three pins

Full size image

Once the markers are placed on the interactive surface, a set of single touches are detected, which are treated as point clouds and segmented into clusters. Subsequently, for each cluster the system calculates in real time the XY position of its centroid, its orientation, and ID, and encapsulates this information in respective TUIO frames [7]. The set of points detected on the interactive surface to the situation presented in Fig. 2 is illustrated in Fig. 4.

Fig. 4.
figure 4

Points detected on an interactive surface from the three tangible marks shown in Fig. 2

Full size image

4.3 System Architecture

The Distributed Concept Map system is an application that consists of a server and several local nodes (tabletop systems) connected via a TCP/IP network (Fig. 5).

Fig. 5.
figure 5

Architecture of the distribute concept maps with tangible elements interface

Full size image

Both local and server applications were implemented using the Unity engine, which provides extensive support for developing graphical and networking applications. The entire virtual workspace is managed by the server and each client has a view of it. Any action performed by a client is transmitted to the server, which immediately updates the scene and, in return, sends the changes back to the clients. The operation is similar to a network video game, where a common virtual world is stored on a server, and each player connects to it as a client.

Using the API provided by the optical frame manufacturer, a daemon process was developed to continually sample touches on the interactive tabletop and return them as XY point coordinates. With this information, a middleware application computes the number of marks placed on the surface as well as their respective IDs, position and orientations, and encapsulates this information in TUIO frames that are sent via UDP to the local application. All points that are not classified into any cluster (and thus do not belong to any mark) are sent as single points in single TUIO “point” frames. This feature allows compatible interaction with the multi-touch table using both tangible marks and hand gestures.

5 Preliminary Usability Evaluation

A preliminary test was conducted to study the impact of tangible manipulators in a collaborative work scenario where eight students distributed in two tabletops worked on the construction of concept maps on a common theme. The session was conducted by teachers to encourage situations where both groups of students needed to exchange knowledge, and experiences to build a complete concept map. Such situations caused the physical movement of students and information. The questionnaire and results of the 5-point Likert scale used in the evaluation are shown in Table 2.

Table 2. Questions and results of the preliminary usability evaluation test using tangibles
Full size table

6 Conclusions and Future Work

Based on the results of the preliminary usability test, the use of tangible manipulators has been shown beneficial in tasks related to the exchange and management of information between different workgroups. Comparing these results with those obtained in our first usability test (Table 1), we can observe an improvement in question 6.

From interviews with students who participated in the first test, it was concluded that the low scores obtained in the question 6 were mainly due to the difficulty of exchanging information between groups. In this regard, it appears that the addition of tangibles for handling information simplifies this task and makes it more natural. This conclusion is reinforced by the scores from question 9, which confirm that tangible markers are easy to use for information exchange activities. Moreover, a natural flow of students between the two work groups was observed during the experiment. The virtual information transfer between the two tabletops mimics the physical transfer of information media in traditional work environments.

The use of tabletop systems implemented with optical frame technology combined with the use of the proposed passive tangible manipulators, results in a tool with a reasonable price/performance ratio that has proven useful in distributed collaborative learning environments. In the future, more comprehensive testing usability evaluations involving a larger number of users and tabletop stations will be conducted.