Introduction

Large touchscreen interfaces offer various opportunities for collaboration between co-located learners. These opportunities exist not only when the interface is shared but also when two or more co-located interfaces are networked together allowing the transfer of materials between them (Kharrufa et al. 2013; Kreitmayer et al. 2013). The successful integration of multi-touch tables into usable computer-supported collaborative learning (CSCL) activities can present major challenges both technically and for teacher orchestration of lessons (Dillenbourg and Evans 2008). However, such integration also presents valuable pedagogical opportunities.

Collaborative small group learning activities have the potential to bring substantial learning gains for the participants, with collaboration bringing greater gains than completing activities individually (Barron and Darling-Hammond 2008; O’Donnell et al. 2013). There is a clear role for appropriate technologies to support collaborative learning; technology enables configurations for group work and types of task which would not otherwise be possible. CSCL theory and research emphasises the role of technology in supporting learner participation, knowledge acquisition (information) and knowledge creation (knowledge objects and social practices) (Lipponen et al. 2004). Many studies in this field emphasise the importance of face-to-face interaction and the design of tasks to optimise the participation of learners (Stahl et al. 2014).

The growing interest in face-to-face collaboration by learners engaged in mutual negotiation of meaning (both procedural and factual) inspired the creation of the SynergyNet projectFootnote 1 at Durham University. This project was funded by the UK research councils EPSRC and ESRC to identify important technical and pedagogical challenges in the CSCL domain (Higgins et al. 2012). Important technical challenges addressed during the project included the development of teacher orchestration tools. These tools enabled seamless transition between individuals, groups and interactive whiteboards, termed the private, shared and public spaces, respectively, by Dillenbourg and Evans (2008).

SynergyNet also identified the potential for groups to share information with other groups, building on the work of Everitt et al. (2006). In this case, the mechanism developed did not involve sharing between personal devices such as computers. Rather, SynergyNet developed tools to allow sharing seamlessly between multi-touch tables within a lab-classroom setting (Hatch et al. 2011).

The original SynergyNet study focused on schoolchildren aged 10–11 years old, working in collaborative groups nested within a lab-classroom. Data were collected on each group as tasks were completed simultaneously, as well as on the arrangement of the whole class and the teachers’ behaviour whilst supporting the collaborative activity. Compared to tasks completed using paper-based equivalents, learners showed improved uptake of ideas via the tasks completed using multi-touch tables (Mercier et al. 2016). Groups using the multi-touch tables engaged in more sophisticated reasoning, spending more time on problem-focused talk rather than procedural talk (Higgins et al. 2012). Further studies highlighted the importance of the division of roles and showed different patterns of organisational and intellectual leadership in groups using multi-touch tables (Mercier et al. 2014), the importance of classroom layout for task completion (Mercier et al. 2014), the differences in teacher decision making process when shifting from group to whole-class dialogue (Joyce-Gibbons 2016), the development of adaptive expertise among group members (Mercier and Higgins 2013) and the affordances of the tables to structure representations of reasoning processes (Mercier and Higgins 2014). SynergyNet studies have also been concerned with how the use of multiple large touchscreen interfaces impacts collaborative education in single environments, i.e., individual classrooms (Basheri and Burd 2012; Mercier et al. 2017).

However, the principal limitation of the original SynergyNet study was that the research was only ever conducted within the lab rather than in a real-world school environment. This study presents the challenges encountered in developing the SynergyNet framework in authentic classrooms; in particular, it details how this can be achieved when dealing with the technical challenges of the school’s infrastructure. As such it presents the first step in ongoing research agenda, to test the utility of the tables in authentic classroom settings.

A key advantage of collaborative education across multiple locations is allowing students with a potentially diverse range of knowledge, backgrounds and skills to work together (Kizilcec 2013). In this context, the opportunities for educational collaboration across multiple locations afforded by co-located interfaces via large networks, both open and closed, need to be explored (Daradoumis and Marquès 2000). Such a move would allow users at both locations to collaborate using the touchscreen interface, offering the opportunity for a range of novel gestures to intuitively instigate specific actions which can be beneficial for younger users (Kim et al. 2007). One such gesture is the `flick’ motion (Reetz et al. 2006) which is typically used for the movement of materials about an environment with minimal effort. This gesture can aid collaboration, allowing users to transfer materials to each other without needing to enter each other’s interaction space.

This work is contextualised by the substantial ongoing reform of ICT education in England and Wales. It also intersects the wider technology-enhanced learning agenda (and related scrutiny on the digital competencies of all teachers). A new computing curriculum was established in England in September 2014 (Brown et al. 2014), as well as emerging ICT curriculum reform in Wales (Arthur et al. 2013).

This paper details the innovative technical work undertaken to support a trial that investigated collaborative interactions using large touchscreen interfaces in two separate geographic locations. A valuable case study of the technical aspects of implementing a novel research trial using multiple interactive and networked devices in real-world school environments is presented. This case study specifically discusses the key technical challenges encountered and their solutions.

Meeting the challenges of synchronous remote collaboration

The original SynergyNet studies explored learning by developing tools to be used explicitly in a lab setting where groups of students were co-located and activities were well supported by team members at hand with technical expertise. This paper relates the technical challenges which the team faced and overcame as they attempted to introduce SynergyNet into multiple classroom settings simultaneously. This study built upon features already present in the framework (e.g., `Network Flick’ and the Mysteries tasks). However, it also required some development in other areas to enable the framework to be deployed over the Internet rather than over a local area network (LAN). The section describes the organisation of the study. It describes the key features of SynergyNet already present in the framework from previous studies.

Technical set-up

The physical set-up of the study involved a single multi-touch table in a quiet room in each location. Each table was a Samsung SUR40, with a rectangular screen mounted on its legs. Three chairs were arranged around one of the two longer edges of the rectangular multi-touch table. A tablet computer was placed perpendicular to the table on the opposite edge, facing the chairs. Figure 1 shows the set-up with groups facing each other in each school. A session had a group of three participants working together at a table in each school. They communicated via Skype, running on a tablet computer in front of each group (thus the groups regarded themselves as being opposite each other). In total 24 learners participated in the study. Participants were aged 9–10 and organised in four groups of three children in each location. Each session lasted for approximately 20 min. Situated in the North East of England and in South Wales, the two participating schools were approximately 300 miles apart. Despite their distance, the two schools share a common industrial heritage. They were located in villages which had prospered when their local economy was based on coal mining. More recently, both areas have experienced significant post-industrial socio-economic trauma with the closure of the mines and de-industrialisation. Both schools were located in similar socio-economic circumstances reflecting this decline. The head teachers of both schools expressed concern that impact on the children living in these former mining communities has led to a dislocation with their environment as they grow up with little or no first-hand knowledge of mining, although are aware of the cultural heritage of the community (as are their families).

Fig. 1
figure 1

Groups from school 1 and school 2 facing each other via Skype

Full size image

The mysteries tasks

The Mysteries task selected represented an attempt to engage the learners in a collaborative activity which would give them the opportunity to share their understanding of mining and how this related to their own area. They had to investigate a 10-year old boy injured in a mining accident. Based on the available evidence they were asked to jointly arrive at an explanation of what had actually happened to cause his injuries and then to discuss who was to blame for these (Leat and Higgins 2002). Collaboration was encouraged through the task design. Groups in each school only had half of the clues each. To view all the clues, they had to share with the other group via the network flick gesture.

This and other Mysteries tasks had been used in several previous SynergyNet studies (Mercier and Higgins 2013; Joyce-Gibbons 2016) where students use a selection of clues, to solve a problem or come to a conclusion about the open questions as shown in Fig. 2. The SynergyNet frameworkFootnote 2 allowed any item in its supported applications to be moved and manipulated by users through common multi-touch gestures. It also supported communication between multiple interfaces via a network connection.

Fig. 2
figure 2

The SynergyNet Mysteries app with task content used in our trial

Full size image

The framework supports several different methods of transferring materials between instances of its apps. The use of networked interfaces allowed media-based content to be easily shared between multiple users, allowing both intra- and inter-group collaboration as shown in earlier SynergyNet studies (Mercier 2014; Mercier and Higgins 2014). The `Network Flick’ gesture allows learners working at each table to flick content, thereby transferring content from one table to another by using a flicking gesture (Reetz et al. 2006). Existing versions of the Mysteries task are supported by several apps for the SynergyNet framework. The key innovation of this study in the development of the Mysteries tasks is the utilisation of the flick gesture in interactions between tables for the first time.

This required the creation of a new app which would manage the loading of the materials in a dynamic way and allow them to be transferred through the network gesture. Figure 2 shows the app built for the trial with content relevant to the task. This content was dynamically generated from a structured text file (XML format) ready for transfer via the network flick mechanism. Content could include text snippets from a structured XML file and media from any other files in a folder. Although video and audio clips are supported by the app, only text snippets were used in this study to reduce potential confounding factors. The dynamic content system allowed the app to be easily configured so that each site would have a different selection of clues for each Mysteries task, forcing them to share content with the students at the remote location.

Network flick

The network flick gesture is built upon the physical metaphor of pushing items around on a low-friction surface (such as ice). It is now regarded as a common feature of many mobile and touchscreen devices (Moyle and Cockburn 2005). A small amount of initial effort can allow the item on which the force is being exerted to travel a great distance. Friction can be applied to decelerate flicked objects over time. This is useful for both helping users keep control of items. They do not continue to travel indefinitely at high speeds after being flicked, making them difficult to grab and stop. The behaviour of items on the interface also better matches their real-world equivalent. Again this links back to the metaphor of sliding objects on ice.

The networking element of this gesture appears when users flick a content item in the direction of the interface to which they wish to send content. The item travels to the side of the initial interface. It then disappears and reappears on the target interface as shown in Fig. 3. When the item arrives on the target interface, the framework uses its predetermined knowledge of the interface locations to ensure the item appears in view from the appropriate direction of the source interface. This aids users in easily identifying from where newly arrived content items have been sent when interfaces are co-located. The use of a flicking gesture not only informs users on recipient interfaces of the origin of transferred items but also creates an intuitive way of sending and sharing content. Users can select their recipient through the direction of the flick and thus initiate the transfer simply by moving the item in that direction and releasing it. This is intended to reduce the cognitive load required to move content between interfaces. The potential benefit of this approach, by having a simple method to move content between interfaces, users will be better able and willing to share items due to smaller barriers to sending and receiving. Previous studies have investigated the human–computer interaction aspects of the network flick gesture but none of these have used it in a classroom environment (Coldefy and Louis-dit-Picard 2007). Typically, classroom-based studies with SynergyNet have utilised a remote teacher interface to orchestrate the transfer of content (Hatch and McNaughton 2011).

Fig. 3
figure 3

The use of the network flick gesture to transfer content between two example co-located interfaces

Full size image

Technical challenges

Having outlined the features of the existing SynergyNet framework which was possible to adapt to facilitate synchronous remote collaborative learning activities, this section addresses those features which presented a barrier to doing so and how these barriers were overcome. Initially, SynergyNet was designed for co-located collaboration, incorporating architecture which was not able to support remote working. This section outlines the challenges faced in terms of the framework, the school setting and mobile connectivity.

Co-location to non-co-location

The SynergyNet framework was developed to be as adaptable and extensible as possible. However, this had previously not included functioning across multiple environments in separate locations. The main priority of early development had been use in a single lab-classroom. Consequently, several issues in the design of both the framework and the apps became apparent when meeting the challenge of working between multiple sites in the current project.

The majority of SynergyNet features were developed on the assumption that users would be located in the same physical environment and thus be able to see each other’s screen directly. This informed the design of visual elements, such as the menus. For example, the menu used by teachers to load and transfer content in previous studies assumed that the teacher was able to see pupil’s interfaces, so that they could choose which group’s work to display to the class. As this was not the case for this study, the in-app menu system was removed to avoid potential disruptions through its use. A second issue was the inbuilt assumption of the network flick gesture about its use: users will know where their target interface is in relation to them. However, this knowledge can also be made available to users even when their target interface is in a remote location. The implications of not directly seeing the target interface when performing a network flick gesture are the key foci of this new strand of work and will be discussed in terms of the networking challenges and content transfer behaviour.

Network challenges

Overview

The use of single environments in the past has meant that the framework was always deployed across a single local area network (LAN) in all studies. Many of the features are not immediately capable of functioning across the wider internet.

Despite the original intention of using SynergyNet solely across LAN, a large amount of the framework’s networking capability was built in such a way that it could easily function across the internet. The use of third-party libraries such as OpenFireFootnote 3 (a real-time collaboration (RTC) server that uses XMPP/Jabber (the widely adopted open protocol for instant messaging) and HazelcastFootnote 4 (an open source in-memory data grid for simplified distributed computing, written in Java) means that the framework can use several different protocols across wider networks for communicating messages to each other.

However, the transfer of media content between instances of the framework was an original feature which was solely limited to use in a LAN. The framework uses a shared network location as a form of cache, which was beneficial in the past as it was simple to implement and gave teachers a single location to collect the files used in a lesson for use in future lessons. The Mysteries app only sends data when first discovering other instances of the app and when a content transfer occurs. The first time any content is sent it is moved to the shared area on the network. Afterwards, all subsequent transfers point back to the same item in the shared area, treating it like a cache.

The secure use of a shared network location across a wider network, such as the internet, is inadvisable without precautions due to data security and latency of such a technique. Development work took place prior to the trial taking place to allow the framework, to function over the internet as the two locations used would not be able to share a typical LAN.

Virtual private networking

Many of the networking features (including the network flick gesture) are dependent on the devices being on the same local network. When working with different locations and networks there are a number of technical issues to overcome. Some of these issues are general; others are specific to working in educational environments. It became apparent that the most effective approach would be to emulate a LAN across a wider network (i.e., the internet). This solution was decided upon as it was more practical given the available time and resources compared to supporting direct communication across the internet. The latter would have required several of the framework’s key networking features to be completely re-engineered.

The most effective way to emulate a LAN in this type of environment was to utilise a virtual private network (VPN). A VPN provides many of the features of a local network that SynergyNet requires, such as shared folders, whilst still supporting communication across the internet. Though various VPN solutions are available, both free and paid-for services, Zero-TierFootnote 5 was selected to provide network virtualisation services. This low-cost VPN solution was able to support the necessary networking features over the internet. Two table-top machines were required to be connected to the same network in the trial (i.e., one touchscreen table-top interface for each site). It was not necessary to use a more feature-rich VPN. The tablets used in the trial for the Skype did not need to be on the same network as the table-top interfaces and could function over the internet. Given the sensitivity associated with conducting research in a school—in particular the fact that the participants are vulnerable persons—security of the connection was a major consideration. The trial did not transmit any private data between the two sites. Therefore, the level of security afforded by the Zero-Tier VPN was considered to be adequate both technically and in the ethical permissions granted for the project to be conducted.

School security

An unexpected problem was encountered with the use of Zero-Tier on the school networks. It was originally intended that all devices would connect to the internet via the schools’ own network infrastructure. However, during preparation for the trial it was discovered that the firewalls at both of the schools were explicitly blocking outbound VPN traffic relating to Zero-Tier. When tested in other environments which have strict firewall policies (e.g., university networks), the Zero-Tier VPN had operated as expected. However, the school firewalls were locked down much more than anticipated. Both school’s networks limited the permitted outbound connections allowed through the firewall to a great degree.

Such stringent measures, prompted by e-safety concerns and wider protection issues in a school environment, have been a feature technical infrastructure for some time. However, there has been a concerted effort for local education authorities not to indiscriminately “block and lock” networks without justification (Welsh Government 2015). Despite this policy of more focused and strategic security, the SynergyNet framework could not operate when connected to the Zero-Tier VPN through the schools’ networks. Adding exceptions to the firewalls would have allowed the VPN connections to function correctly. However, this was not feasible due to administrative issues with approval via the school and local education authority in any short-to-medium timeframes. Therefore, an alternative method of connecting to the internet was required.

Mobile networking and connectivity

An alternative method of connecting to the internet allowing the table-top interfaces to function was identified using pay-as-you go 4G mobile dongles. These were connected via USB to the two table-tops to allow them to connect to the internet through a relatively fast mobile connection. Due to the geographical locations of each school it proved difficult to find acceptable coverage using a single mobile network so the interfaces at both sites were connected to the internet via two different 4G providers. This had no noticeable impact or latency on the connection and allowed the instances of SynergyNet on both table-tops to interact quickly and with minimal data loss. For example, there were no noticeable instances of failed transmissions of messages between the instances resulting in flicked items not arriving on their target device.

Although 4G mobile connections may not be as secure as using the wired school network, data to and from the VPN were end-to-end encrypted ensuring they were secure. Furthermore, task design ensured no sensitive or personal data were transmitted between the tables.

Multi-casting

Prior to the current project, SynergyNet had used multicast service discovery as a method of automatically finding other running instances of the framework on a network. However, many VPNs do not support the User Datagram Protocol (UDP) that SynergyNet employs. To accommodate this constraint, the framework was modified to work entirely through Transmission Control Protocol (TCP). However, this change meant that SynergyNet could no longer automatically detect other instances using multicast service discovery. Therefore, at least one instance would need to be informed of the IP address of the other instance. To support this new functionality, a user interface was developed which allowed for the user to specify participating device’s IPs on the network. This update was initially a change at the app level but has since been integrated into the framework so that all apps on the latest version of SynergyNet can work over VPNs more easily.

Content transfer behaviour

The tablet computer showing each group of participants the other group via Skype was placed on the long table edge opposite to them. The tables were configured so that flicking items towards the tablet (and therefore the other group ‘facing’ them) would transfer these items to the remote table. This was achieved by configuring the tables so that they were virtually positioned opposite each other, with the sides on which the tablets were placed being parallel to each other. This meant each table operated virtually as if the other was co-located with it. With students passing items towards their ‘window’ to the other team, the metaphor of physically pushing items towards another interface was maintained. Virtually positioning the tables opposite each other also meant that the entire edge of the tables with the tablet on could be used as the target for network flicks. If the two tables were configured to use their real-life locations in relation to each other, the size of the boundary where a network flick could be trigged (i.e., that points to the remote table) would be less than a pixel at the distance between the two locations.

Another benefit to this configuration of the tables being placed virtually opposite relates to behaviour of the transfer. Normally a realistic effect is employed with time taken for items to transfer and is calculated by the time it would take to travel from the gap between the interfaces at the speed of the item before it leaves its source interface (ignoring deceleration). This is useful for co-located interfaces as it reinforces the metaphor of real objects being pushed around the environment. Clearly this behaviour would not be suitable with the interfaces using their real-world locations in the trial as it would take a long time even with the fastest flick gesture the interfaces are capable of identifying. Therefore, the virtual distance between the tables was set to be as small as possible. This meant the flick behaviour had the effect of items travelling the shorter virtual distance from one interface to the other when flicked in a direction with a trajectory that would eventually intercept the nearby virtual representation of the remote table.

Limitation: data collection

Screen recording software was installed on both tablets allowing both sides of the video conferencing software used for communication between the groups to be recorded. In addition to this, stand-alone cameras were used at both locations to record in high-definition the interactions in and around the table-top interfaces.

Previous SynergyNet studies have been informed by detailed data collection of touch frequency and location during activities. These data were collected on separate lab-based infrastructure which was not possible to replicate in the current study. SynergyNet uses OpenGL (a cross-language, cross-platform API for rendering 2D and 3D vector graphics) for its visual output. Video capture from the table-top screens was problematic because, using a secondary device that mirrored the output of the table would have affected the visual output configuration and impacted on the performance of the device. This in turn would adversely affect the intuitive operation of the tables. This was considered an unnecessary risk to the study for the potential data collected. Future research will explore data capture solutions which do not risk inhibiting the performance of the tables.

Conclusions

Firstly, the process of preparing, running and collecting data from this project has demonstrated that SynergyNet can be used successfully to facilitate collaboration beyond physically co-located groups. It can move out of the lab and into schools where groups can collaborate in separate locations. The innovations and adaptations described above now allows the framework, and any of the applications it supports, to be used across multiple sites for the first time. This will facilitate further research trials in collaboration between remote sites using natural user interfaces. Furthermore, the project has established that the ‘flicking’ of content between these geographically remote groups can allow effective and timely sharing of content leading to successful problem-solving activities. This was only possible, however, due to the successful adaptation of the SynergyNet framework’s networking features to work through a VPN.

These innovations have identified and attempted to resolve or mitigate a number of important technical considerations when using the equipment in a real-world school environment rather than a lab. Many of these technical challenges have been based around networking and connectivity and balancing these with valid institutional concerns and precautions regarding e-safety and child protection. Team members regularly informed key stakeholders (in each school these included the Headteacher, the Child Protection Officer, the class teacher and parents) of their activities and the steps they were taking to avoid and manage possible risks to participant safety. The use of 3G and 4G mobile network connections to avoid the difficulties of locked-down school networks may prove to be useful in future studies, but this overlooked potential issues around latency and security. In the study described above, no personal or sensitive data were sent through the network connection, and all information was encrypted via the VPN. However, the use of a secure wired school’s network is preferable where feasible, providing improved connectivity and resilience. This may require significant preparation, testing and approval processes before research studies can be conducted in situ. In addition, although the trial used a set of specific multi-touch table devices, the portability of the SynergyNet framework potentially allows for it to be run on a range of different devices. This affords the opportunity to deploy similar trials into a wider range of consumer devices in the future, with little reliance on the bespoke and expensive hardware available.

With the changes to the framework made as part of the trial, SynergyNet (and associated pedagogic practices) is now better suited for use across the internet where a VPN is set up appropriately. This allows for the tables used in the trial to be re-deployed anywhere where there is an acceptable 3G/4G signal or with an alternate network connection where the network is configured appropriately.

Finally, this project has shown the potential of the SynergyNet software in educational settings, both co-located and at remote locations. It suggests that, despite a number of technical barriers that have had to be overcome, the software framework provides the potential for innovative pedagogic practice and collaboration which would otherwise be impossible with conventional digital devices and associated resources.