An Intelligent Support System to Help Teachers Plan Field Trips

Abstract

Field trips enrich learning programs with out-of-school activities that can bring gains in students’ academic content knowledge and personal growth. However, they are a source of anxiety for teachers because of the bureaucracy, pedagogy, etc., risks they imply. To address this issue, we propose FieldTripOrganizer, a field trip planner based on the mixed-initiative approach aimed at increasing teachers’ autonomy and motivation in designing educational tours. The key aspects of our application are (i) the simultaneous provision of information filtering and automated scheduling support while the user designs the field trip, and (ii) the visual annotation of places and activities to show whether they can be included in the itinerary without violating its time constraints. Different from current tour planners, these functions enable the user to be in full control of the design process, delegating the system to manage difficult and burdensome tasks such as consistency checks and itinerary optimization. We evaluated FieldTripOrganizer in the use case of organizing a science field trip. In a preliminary user study involving 18 science teachers, our application turned up to be superior to a baseline tour planner in both usability and user experience. Moreover, the teachers declared that it was helpful, motivated them, and reduced their anxiety during the design of the field trips.

Introduction

As discussed by Arık and Bodzoğan (2022), the experiences acquired outside the school are as important to students as those gained in the classroom to form critical thinkers with creative thinking skills, who can solve problems using scientific processes. Thus, several types of out-of-school learning environments, such as natural ones, and science/sports/art centers might be used to enrich the learning programs. However, organizing out-of-school learning is much more complex than traditional classroom lessons, and teachers can be overwhelmed by the organization and feel anxiety (Arık, 2022; Kisiel, 2007). Therefore, they rarely make use of field trips’ full educational potential (Bravo et al, 2022; Finchum, 2013; Kisiel, 2007). Arık (2022) studied teachers’ anxiety levels when planning trips to out-of-school learning environments and found that it has “bureaucracy-related”, “safety risks-related”, “harm-related” and “pedagogy-related” causes. Several qualitative studies focused on how to support teachers in organizing field trips. For example, Kisiel (2005) studied how to improve their motivation, and Martin and Sewers (2003) provided guidelines for planning activities. Indeed planning a field trip is a complex task involving different steps, from the identification of the places and activities to be performed to the finalization of bureaucratic aspects such as payment. However, to the best of our knowledge, no automated support has been developed so far in this direction.

Most research on Artificial Intelligence in Education focuses on Technology Enhanced Learning (Manouselis et al, 2014) and on the development of Learning Management Systems to support the access to materials, contents, and information, promoting students’ active participation in building their own knowledge; e.g., see (Frost and McCalla, 2021; Carbone et al, 2021). In the present work, we address a complementary issue, i.e., helping teachers to extend learning support by selecting and organizing out-of-school activities for student classes, which complement in-class work. We investigate the potential of intelligent systems to alleviate the burden of selecting and planning the activities to be performed. Specifically, we propose to combine information filtering and automated scheduling to assist the teacher in the design of the field trip, depending on the didactic objectives and the spatial/time constraints to be satisfied, and in the generation of the itinerary.

Most personalized tour planners elicit preferences and constraints from users and then autonomously generate optimized itineraries without involving them in the design process. Users can only critique the proposed solutions and ask for new ones; for instance, see Lim et al (2018) and Herzog and Wörndl (2019). Differently, we aim at empowering teachers to organize field trips without limiting their freedom of choice in the selection of places to visit, and activities to be carried out. For this reason, we exclude the automated planning approaches where the system leads the construction of the itinerary. In contrast, we want to promote the user’s creativity in the planning phase by (i) providing information filtering functions aimed at supporting the identification of relevant options and (ii) delegating the system to perform burdensome and difficult tasks, like the evaluation of whether the options selected by the user fit in the time constraints of the tour to be organized. From this perspective, we investigate the following research questions:

• RQ1. How does the simultaneous provision of information exploration support and automated scheduling help teachers during the organization of a field trip? How does it motivate teachers? How does it impact their level of anxiety and their autonomy?

• RQ2. How can an automated scheduler help teachers having diverse technological expertise organize a field trip for a student class?

• RQ3. During the organization of a field trip, how important is keeping teachers aware of the feasibility of their choices in the context of the temporal constraints to be satisfied?

To answer these questions, we developed FieldTripOrganizer, a web-based, mobile application that helps teachers schedule custom field trips by applying mixed-initiative interaction (Horvitz, 1999) to enable collaboration between the user and the system during the design phase. Our application makes this task fully interactive by empowering the user to control each design step of an itinerary that involves Points of Interest (PoIs) to be visited and, different from existing tourist guides, activities to be carried out. FieldTripOrganizer enables the teacher to filter both PoIs and activities by their domains of interest, e.g., naturalistic, archaeological, and so forth. Moreover, it supports the selection of the difficulty level of the activities by specifying which school level is required to perform them. Furthermore, it supports the user’s awareness and decision-making: following Shneiderman et al (2016)’s Golden Rules for Interface Design, the application enables the user to incrementally plan the field trip by visually annotating the items that are consistent with the specified preferences to make the user aware of whether they fit in the time constraints of the itinerary under construction. The visualization of the options’ feasibility (based on heuristic reasoning, and on an underlying itinerary generation service that computes the schedule taking the shortest time) prevents the user from selecting items that will break the temporal constraints of the tour and empowers her/him to interactively modify the plan, e.g., by removing some items to make others fit in while being aware, at each step, of how much time the partial tour takes.

As a testbed for the evaluation of FieldTripOrganizer, we selected a use case involving a science laboratory in the area of Ivrea, a city located in the North-West of Italy. We carried out a user study involving 18 science teachers of different school levels, from preschool to high school. The reason for this choice is that the out-of-school activities are particularly relevant to science programs where it is crucial providing students with practical experiences, such as visiting natural environments and performing field experiments, in an out-of-school learning vision (Eshach, 2007; Dumitru, 2018). Specifically, nature field trips support learning science from early childhood to secondary education (Dillon et al, 2016; Pasquier and Narguizian, 2006; Tal and Morag, 2009) by creating practical activities and experiences for the students (Heras et al, 2020). In this way, they provide a multisensory environment where to connect abstract content developed in the classroom with the real world, improving the understanding of knowledge and producing strong long-term memories.

In the user study, which involved 18 science teachers, we compared FieldTripOrganizer with a baseline tourist trip organizer app that offers the main functions provided by recent tour planners. The results of this preliminary experiment provide insights into our research questions and show that FieldTripOrganizer outperformed the baseline as far as both usability and user experience are concerned. Specifically, the participants declared that our application was helpful, motivated them, and reduced their anxiety during the organization of the field trips. This encourages the adoption of mixed-initiative planning to develop applications that support field trip organization.

The remainder of this paper is structured as follows. In “Related Work”, we outline the state of the art. “Field Trip Organization” describes FieldTripOrganizer and the model underlying it. “User Study” describes the user test we carried out and its results, which we discuss in “Discussion of Results”. “Implications” describes the implications of our work. “Limitations and Future Work” reports the limitations of our work and our future plans. “Ethical Issues” presents the ethical issues and “Conclusions” concludes the paper.

Related Work

We position our work in the research about tourist trip planning because, even though the scheduling of school field trips includes both the visits to places and the execution of activities, it has aspects in common with the organization of a tourist trip. In the Tourist Trip Design Problem (TTDP) (Vu et al, 2022), the user expresses preferences for the trip to be designed, such as the most relevant categories of PoIs to be visited, and constraints like the available budget and the arrival/departure dates and locations. Given this input, an algorithm solves the problem by generating a plan that includes the visit to a set of places that maximize the user’s utility with respect to the specified conditions.

The organization of a field trip has aspects in common with the solution of the TTDP but strongly benefits from a mixed-initiative interaction with the user. In fact, a teacher planning a field trip for a student class might pursue teaching goals that require her/his intervention in the selection of the places and activities to be carried out. However, similar to the use cases explored in the traditional research about mission planning (Bresina and Morris, 2007) and work-centered design (Butler et al, 2007), or more recent approaches to customer support management (Convertino et al, 2017) and mixed-initiative creative interfaces (Deterding et al, 2017), the system can play a key role in supporting user awareness of the objects under development. In our case, this concerns the consistency of the trip schedule and the feasible options, considering spatial displacements and time constraints regarding the steps of the plan, which are difficult and burdensome computations. We thus need to integrate planning support with an assessment of the feasibility of the available options to effectively help field trip organization.

Another difference from the TTDP is that the visit to some places (e.g., a laboratory) might include the execution of one or more activities, such as scientific experiments or observations that add a further specification level to the selection of the places to be visited. In other words, PoIs must be modeled as complex entities and their relevance to the user might also depend on the activities they offer. This adds the preferences about the features of the “components” of PoIs to the tour and item preference levels identified by Pugacs et al (2017).

Earlier trip planners generated optimal schedules satisfying users’ requirements (Gavalas et al, 2014) without involving people in the selection of the places to be visited. For instance, DailyTRIP (Gavalas et al, 2011) and CT-Planner4 (Kurata and Hara, 2013) build personalized itineraries by taking in input spatial and temporal constraints, preferences for the categories of PoIs, and contextual data such as weather conditions and distance between places. Some tour planners extend the information used to generate personalized trip plans with further types of user preferences (Borrás et al, 2014). For instance, Hti and Desarkar (2018) identify the places to be included in a travel package by also considering their popularity and the fraction of times they are visited when the starting location of the tour is the same as the one chosen by the user. TripBuilder (Brilhante et al, 2015) mines common travel patterns to suggest trip plans. Dadoun et al (2019) exploit location embeddings from a database of check-ins to recommend next trips by taking the user’s previous visits into account. PersTour (Lim et al, 2018) analyzes the duration of the user’s past visits to learn an individual model of her/his interests and adapt the tour schedule accordingly. Moreover, context-aware tourist recommender systems generate personalized itineraries based on both preferences and rich context parameters, such as the user’s mood (Braunhofer and Ricci, 2016, 2017).

Most of these systems fully control the generation of tour plans. Even though some of them enable the user to criticize the presented solution, e.g., asking for a more economical one, they react by proposing fully built alternatives. Some systems, such as City Trip Planner (Vansteenwegen et al, 2011), let the user ask to remove or include specific PoIs but they do not involve her/him in the composition of the whole itinerary. As discussed by Massimo and Ricci (2022), the autonomous generation of travel plans, based on machine learning techniques, is challenging because next-PoI-recommendation is more difficult than next-item-recommendation, which has been successfully applied in product catalogs. Specifically, in tourism, there is no clearly defined catalog of recommendable items, and diverse types of tourists have different travel habits, which cannot be mined in an indiscriminate way. Starting from these considerations, we conclude that an effective solution to field trip organization is the involvement of the user in the plan generation process, rather than the management of post-hoc evaluations, or revisions on the system proposals.

Recent tourist recommender systems provide richer services, such as complete travel packages that include places, travel solutions, and accommodations (Chaudhari and Thakkar, 2020), they use information extracted from Location-Based Social Networks for personalization (Sánchez and Bellogín, 2022) and they introduce nudging techniques to guide people towards sustainable tourism (Merinov et al, 2022). However, most of these systems assume a fixed interaction model where the user specifies preferences and spatial-temporal constraints and receives complete solutions satisfying such requirements. Analogously, most current commercial trip planners fail to support the collaborative design of tour plans because they fully control the generation of the solutions. For instance, Klarna.Trips¹ plans an itinerary in the area selected by the user, who can only revise it by adding or removing PoIs. Moreover, these systems are conceived for organizing a holiday and not for planning a school trip; e.g., see Roadtrippers². Only in some cases, as in wanderlog³, the user has full control of the selection of PoIs but (s)he is also responsible for scheduling the tour because the system does not offer this function. Our work differs from the previously described ones as follows:

• 1. We want to support the scheduling of both the visit to places and the activities they offer, treating PoIs as complex entities to be included in the field trip.

• 2. The mixed-initiative planning of the field trip is our primary goal. We want to help teachers filter places and activities depending on individual interests while being aware of which ones satisfy the constraints of the field trip. In this context, FieldTripOrganizer supports decision-making by graphically highlighting the feasibility of the available options to prevent scheduling mistakes. The application supports the design of itineraries by combining information filtering and graphical annotations to make users (teachers) aware of which PoIs and activities are consistent with the tour constraints.

Different from navigators such as Google Maps⁴ and personalized route planners like (Quercia et al, 2014; Gavalas et al, 2015, 2017; Verma et al, 2018; Arnaoutaki et al, 2021), FieldTripOrganizer does not focus on fine-grained routing, dynamic recalculation, and reactive itinerary adaptation (Stavrakis et al, 2020; Horvitz et al, 2007) because it supports the design of the field trips rather than their execution.

Some trip planners target visitor groups. For instance, INTRIGUE (Ardissono et al, 2003) generates tours that are consistent with the needs of heterogeneous user groups, including impaired people. TourRec (Herzog et al, 2018) supports the planning and execution of a flexible trip that enables tourists to split into subgroups to visit different PoIs, and Herzog and Wörndl (2019) extend TourRec to dynamically elicit user preferences through critiquing of the tour plan, or of its PoIs, following the research path of critique-based conversational recommender systems (Jannach et al, 2021). Our work differs because field trips target a whole class of students moving together under the teacher’s supervision. This also means that we are not affected by the group decision-making dynamics discussed by Delic et al (2018).

From the perspective of out-of-class activities, our work supports a faceted search (Hearst, 2006) of PoIs and activities based on filters to specify the domain of interest and the required school level.

Field Trip Organization

The OPENALPLAB project in which we developed FieldTripOrganizer is aimed at establishing a network of laboratories to enhance out-of-school education in alpine and pre-alpine zones of Piedmont, a region in the North West of Italy. One of the target areas includes Ivrea city and a teaching laboratory that operates on the San Michele lake (next to Ivrea city) and on the Alice one (located about 20 Kms from the town). The areas of interest of the project need to be preserved by creating a network of researchers, administrators, and local stakeholders capable of sharing a multidisciplinary knowledge framework, with a view to dissemination and public engagement also aimed at environmental protection. In this context, FieldTripOrganizer can support teachers of any school level in the organization of field trips for visiting cultural places and carrying out educational activities.

Domain Knowledge

For applicability purposes, FieldTripOrganizer exploits an external data source to retrieve information about places and activities supporting exploration and itinerary generation. At the setup time, domain experts are expected to populate this data source. For the management of the domain information, we made two design decisions:

• To make FieldTripOrganizer suitable to organize tours focused on different topics, such as science and arts, we based it on a generic data model that abstracts from the details of the types of entities to be considered (e.g., laboratories vs. museums, or scientific experiments vs. lectures) and only specifies the “Domains of interest" of the entities, which can be used at runtime to filter the items relevant to the user’s search.

• To support domain experts in information authoring, we decided to leverage as a data source a Content Management System (CMS) offering a user-friendly interface to upload and revise data, suitable for non-technical people. Specifically, we selected Omeka-S⁵, which supports a semantic specification of domain knowledge, expressed as an OWL (OWL Services Coalition, 2004) ontology, and offers flexible APIs to be queried by external applications.

We defined the data model of our application as the ontology shown in Fig. 1, which specifies that the entities to be handled are “Items” representing places (PoIs), activities, or objects. The ontology also specifies that items can be related to each other either through a specification relation (“is-a”) or usage (“use_in”) and location (“is_in”) relations.

• The “Item” concept describes the common attributes of Points of Interest and activities. These are the title, geo-location, description, list of available images, opening hours, and expected duration of the interaction with the item, e.g., the time needed to visit a PoI or to perform an activity. Items have a list of “Domains of interest” that describe the types of interest they satisfy: for example, historical, archaeological, architectural, and so forth.

• Both “PoI” and “Activity” are sub-concepts of “Item”. Moreover, a PoI can include some elements of type “Activity” representing the activities it offers (“is_in”). The presence of this relation makes it possible to treat PoIs as entities that can aggregate one or more activities, different from the typical atomic elements of tourist applications. The activities have more attributes: “School levels” describes their level of difficulty. Currently, we propose a coarse-grained classification, i.e., for all students, starting from kindergarten, from primary school, and so forth. “Type” specifies whether the activities are carried out in a laboratory or outdoors. “Setup time” describes the expected time to prepare for the activity; by default, it is 10 minutes for all the activities.

• The “Object” concept represents the tools used to carry out the activities (“used_in” relation), as well as other objects that are located (“is_in”) in the PoIs. For instance, they can be instruments such as microscopes, or objects like pictures, and stones exhibited in a place.

Fig. 1

FieldTripOrganizers’ ontology representing Points of Interest and activities

The FieldTripOrganizer Application

FieldTripOrganizer is a web-based application that can be used from a laptop, tablet, and mobile phone to set up a field trip. The user initially specifies the intended duration of the field trip, and whether it has to be scheduled in the morning or afternoon.⁶ Then, (s)he starts an interaction loop to lead the itinerary specification process by applying filters to visualize the relevant types of items (by geographical area, the domain of interest, and school level) and incrementally adding items to the tour. During this loop, the application keeps the user aware of how much time is available to extend the itinerary, given the expressed constraints. Moreover, it graphically annotates items to show how much time they take, which options can be added, and which ones fail to fit into the schedule, considering the time needed to move from place to place. In this way, the user can figure out how the partial schedule built so far might be revised by adding or removing items to improve the satisfaction of the didactic objectives to be fulfilled.

Fig. 2

User interface to select the items of the field trip. If a PoI or activity is selected, as “Lago di Alice”, the semaphore is not shown. In the image, we superimposed the circles with letters to identify the graphical components described in the paper

Figure 2 shows the user interface of FieldTripOrganizer supporting the selection of items for the field trip:

• The left section of the page shows the spare time to add PoIs and activities to the itinerary (letter A in the Figure - “Tempo a disposizione rimanente” - 2 hours and 20 minutes). Moreover, it shows the filters that the user can jointly apply to select the relevant items: by geographical area (B -“Seleziona un’area”), by the domain of interest (C -“Interessi luoghi”), by school level (D - “Difficoltà attività”), and by type (E - “Tipologia attività”).

• The right section presents the PoIs that satisfy the filters applied by the user, supporting both a list view of places and their visualization on a geographical map (“Visualizza mappa”, at the top of the section):

  • For each PoI, the application shows the title (e.g., “Lago di Alice” - letter F), an image, a short description, the domains of interest (“INTERESSI”), and the time required to visit the place (“Tempo di visita”). Moreover, it displays a green or red semaphore depending on whether the item can be included in the itinerary without exceeding the available time or not. The semaphore is intended to guide the user to select feasible items; if the user disregards this indication, causing a scheduling error, the application shows a message that explains the problem.

  • If a PoI offers any activities, the application displays them below it. For each activity, it shows the title (e.g., “CAMPIONAMENTO DEL PLANCTON DA IMBARCAZIONE” - letter G), the time needed for its execution split into duration of the activity, and setup time (30 minutes execution + 10 minutes setup), the semaphore, the difficulty level (“Livello difficoltà”), the type (“Tipologia”), the domains of interest, and a button to visualize more information about it (“Più dettagli”).

  • Next to each PoI and each activity, there is a button to add it to (or remove it from) the itinerary.

Below the filters, the left section of the user interface shows the places and activities that the user has already selected (letter H). We decided to visualize these items as a simple list to maintain the order of presentation during the incremental design of the itinerary. However, each time an item is added or removed, FieldTripOrganizer invokes an itinerary generator to estimate the time span of an optimal plan that includes the selected items and the displacements from place to place. Thus, the application can update the spare time in the top-left portion of the page.

Fig. 3

Visualization of a finalized itinerary in Alice Lake

Finalization of the field trip

When the user is satisfied with the selected PoIs and activities, (s)he can press the button “Crea itinerario” (Create itinerary) located both at the top and in the bottom right portions of the page to finalize the field trip. In turn, the application refreshes the user interface to show the complete itinerary, as in Fig. 3:

• The left section of the page shows the total time span of the itinerary (letter I - “2 ORE e 5 MINUTI”, i.e., 2 hours and 5 minutes) and the buttons to book the field trip with the local team that will assist the students during its execution; we skip these details that are out of the scope of the present work. Below (L), the application shows the schedule, with a card for each PoI or activity that summarizes its data and makes it possible to visualize its details (“Visualizza dettagli”). The cards might be separated by the indication of how much time is needed to move from one place to another; moreover, a lunch break is added if the trip takes more than half a day.

• In the right section, the application displays the places on a map.

It can be noticed that FieldTripOrganizer also offers some pre-made itineraries that teachers can select, avoiding the design phase. Those itineraries are offered on the home page of the application and are represented exactly as those that users dynamically compose. In the present paper, we overlook this aspect because we focus on field trip planning support. For the same reason, we leave administrative aspects related to paying services apart, which are out of the scope of the present work and can be addressed by integrating external APIs in the system.

Scheduling Support

Itinerary Generation

FieldTripOrganizer exploits the VROOM⁷ open-source Vehicle Routing Problem solver to manage the instances of the Travelling Salesman Problem (TSP, (Lenstra and Kan, 1975)) that arises while the user iteratively designs the itinerary by adding/removing PoIs and activities. VROOM optimizes the schedule of the PoIs and activities included in the partial itinerary to estimate its temporal extension. This is needed to make the user constantly aware of the spare time available to add further items and is used when the field trip is finalized, to show the sequence of steps of the itinerary to be booked.

Different from the Tourist Trip Design Problem (TTDP) mentioned in “Related Work”, which aims at maximizing user utility by adding activities to the trip while satisfying given constraints and preferences, the TSP consists in finding the best order to visit a set of locations, given a matrix containing their mutual distances. This reflects the fact that FieldTripOrganizer separates information filtering from itinerary optimization.

The problems we submit to VROOM are actually a slight extension of the classic TSPs for two reasons: firstly, locations are items of Fig. 1 and can thus be either activities or PoIs. Secondly, we associate a cost to individual items, and to the connections between different items. Specifically, to take into account displacements between PoIs as well as the activities to be performed at each PoI, we consider time as the quantity to be minimized:

• Displacements take times according to a pre-populated symmetric, zero-diagonal matrix TD containing times td(i, j) to move from PoI i to PoI j. In the current version of FieldTripOrganizer, we instantiated this matrix with constant values that we retrieved from Google Maps, assuming that people walk from one place to the other. As discussed in “Limitations and Future Work”, we plan to make this matrix adaptive to take contextual factors into account; e.g., the age of the student class, which might determine quicker or slower walking, and the use of some transportation means.

• Each PoI p(i) has a temporal extension tp(i) that models its visit time. For example, the visit to Lago di Alice in Fig. 2 takes 60 minutes.

• Each activity a(i, j) performed at PoI p(i) has a temporal extension ta(i, j) that includes its duration and setup time. For instance, in the same figure, “CAMPIONAMENTO DEL PLANCTON DA IMBARCAZIONE” takes 30 minutes + 10 minutes for its setup.

Given the current set of user-selected PoIs $P = \{p(1), \ldots , p(m)\}$ and the sets of activities $A(i) = \{a(i,1), \ldots , a(i,n_i)\}$, for each PoI p(i), FieldTripOrganizer invokes VROOM REST APIs to find an order $\pi = (p(\pi _1), \ldots , p(\pi _m))$ for  the PoIs that minimizes the total time employed to visit the places and carry out the activities. To generate the optimal itinerary, VROOM minimizes the following formula:

$$\begin{aligned} \sum _{i=1}^m tp(i) + \sum _{i=1}^{m-1} td(p(\pi _i),p(\pi _{i+1})) + \sum _{i=1}^{m} \sum _{j=1}^{n_i} ta(i,j) \end{aligned}$$

• The first term is the sum of the PoIs’ temporal extensions.

• The second one is the sum of the displacement times between two consecutive PoIs according to the execution order $\pi$.

• The third term is the sum of the temporal extensions of all the activities A(i) selected by the user at each PoI p(i). It can be noticed that, if no activity has been selected for a PoI p(i), then the upper limit $n_i$ in the internal summation is 0, i.e., p(i) is skipped in this term.

To invoke VROOM, our application passes the following data: the TD matrix; the set $P = \{p(1), \dots , p(m)\}$ of PoIs, and the associated activities A(i) that the user has selected; the maximum time span $T_{MAX}$ (s)he has specified to complete the field trip. If this temporal constraint is violated, e.g., because the user has included in the itinerary an item having a red semaphore, VROOM returns an error and a list of unassigned activities, which is useful to explain the problem in the user interface of the application.

Evaluation of Items’ Feasibility

Given the partial itinerary that the user has built at a certain step of interaction with FieldTripOrganizer, we might employ the itinerary generator to compute the feasibility of the items presented by the application. This approach would enable the system to annotate them precisely, reflecting their feasibility in an optimized schedule but is computationally expensive. We thus decided to apply a heuristic approach to address this problem. Starting from the partial itinerary, PI, we estimate the feasibility of each item (a PoI, or an activity) as follows:

• Let $T_{NOW}$ be the time span estimated by VROOM for PI, and t(item) the time associated with item. We compute t(item) as follows:

  • If item is a PoI p(i), $t(item) = tp(i)$ because it represents a new place to be visited.

  • If item is an activity a(i, j) offered by a PoI p(i) that is already included in PI, $t(item) = ta(i,j)$ because the activity should be carried out in a place whose time of visit has already been considered.

  • If item is an activity a(i, j) offered by a PoI p(i) not yet included in PI, we add the visit time of the place. Therefore, $t(item) = tp(i) + ta(i,j)$.

• Let td(i, j) be the displacement time between p(i) and the PoI p(j) of PI that, according to the displacement matrix TD, takes the minimum time to reach p(i). This hop takes time 0 if p(i) is already part of PI.

Then, the application estimates the time span of a revised itinerary that also includes item by adding item’s temporal extension, and the time needed to move from p(j) to p(i) and back to p(j):

$$\begin{aligned} T^*_{NEXT} = T_{NOW} + t(item) + 2 \times td(i,j) \end{aligned}$$

If $T^*_{NEXT} \le T_{MAX}$, the semaphore of item must be green, red otherwise.

This approach might generate some false negatives because it does not optimize the schedule of the revised itinerary; therefore, it might overestimate the displacement time needed to accommodate item into the itinerary. However, VROOM is invoked immediately after the user has selected a new item and thus the next heuristic estimation of items’ feasibility relies on a refreshed, optimized schedule.

User Study

To answer our research questions, we carried out a user study in the use case of science teachers organizing a science field trip. We aim at understanding how users perceived the interaction with the FieldTripOrganizer application.

Method

We compared FieldTripOrganizer with a baseline, Klarna.Trips tour organizer⁸ that covers most of the functions offered by the tour planners described in “Related Work” and has several functions in common with our application. For instance, it provides category filters on items, enables the user to select both places and activities, and shows the proposed itinerary as a timeline. The main differences are that Klarna.Trips does not enable the user to design the field trip from scratch nor does it relate the activities to the PoIs offering them. Moreover, instead of preventing the selection of items that do not fit in the overall schedule, it highlights the time conflicts caused by the user’s selections after they have occurred. In the user study, we aimed at collecting information about the usability of the two applications, and the user experience while interacting with them by means of a task experiment and three questionnaires. The task experiment consists of designing a field trip using the former or the latter application.

Measures and Material

We tested the usability of FieldTripOrganizer and Klarna.Trips using a translated (Italian) version of the System Usability Scale (SUS) (Brooke, 1996), a reliable tool to measure the usability of a system. SUS consists of 10 statements to be answered in the $\{$Strongly disagree, Disagree, Neither agree nor disagree, Agree, Strongly agree$\}$ scale that we mapped to [1, 5]; Table 1 reports these statements.

Moreover, we collected information about the participants’ experience with the two applications during the task experiment by observing their interaction behavior (see below), by means of the think-aloud protocol (Jääskeläinen, 2010), and by asking people to answer the 4-item post-task questionnaire⁹ of Table 2 immediately after they completed each task. The respondents were asked to answer each statement of this questionnaire using the same scale as SUS. The post-task questionnaire also included a space for free comments.

We implemented the questionnaires as Google forms for ease of distribution and data collection purposes (Courage and Baxter, 2005).

Table 1 Post-task SUS questionnaire results

Procedure

We recruited the participants through mailing lists and social networks: in the invitation message, we specified as inclusion criterion being a science teacher who works in an Italian school, which we consider important for participants to be able to express true and informed preferences and needs, given that we instantiated FieldTripOrganizer on science field trips in Piedmont, Italy. Moreover, we declared that the collected data would have been processed anonymously for scientific purposes only, in accordance with European current legislation. We did not provide any monetary or other incentives as a reward for participating in the study.

For the purpose of the user study, we applied a within-subjects approach. We managed each treatment condition as an independent variable and each participant received both treatments. During the test, we counterbalanced the order of the tasks to reduce the effects of practice and fatigue, as well as the impacts of result biases. We did not impose any time limits to the execution of the tasks to leave participants free to explore the presented information and select the items to be included in the field trips without any time pressure.

The participants carried out the test, one at a time, in a virtual Zoom¹⁰ room in the presence of one of the authors of this paper. During the task experiments, the researcher observed the activities they carried out, taking notes about their interaction with the applications and writing down the interesting comments they made. Moreover, the researcher measured the duration of the tasks. In detail, we organized the user study according to the following steps:

• 1. The participants were invited to join the virtual room to take part in the test. The researcher explained the goals of the study and invited them to read the informed consent¹¹ and give their explicit agreement to participate in the study by checking for approval the checkbox of the final statement of the consent. To preserve the users’ privacy, the researcher did not collect their names or any other identifying data. Moreover, (s)he invited them to share the screen while performing the tasks and to hide it while answering the questionnaires.

• 2. The participants filled in a questionnaire concerning demographic information (gender, age), their expertise in using ICT, and work-related questions, i.e., what level of school they worked in, whether they had ever organized a school trip in the past, and if they wanted to organize one in the future.

• 3. They used FieldTripOrganizer and Klarna.Trips, in a counterbalanced order, to carry out the task experiments, which consisted in organizing a field trip for a class of students. We decided to ask participants to perform the same task twice, albeit focusing on different geographical areas (Ivrea City, and Alice Lake), so that they could describe their experience with the two applications in exactly the same work context.

• 4. Immediately after having used one of the systems, the participants filled in the SUS questionnaire to provide feedback about its usability. Moreover, they answered the post-task questionnaire of Table 2 to assess the following dimensions: (i) level of support, (ii) level of autonomy, (iii) level of anxiety, and (iv) level of motivation in organizing a field trip induced by the system.

Participants

The restriction on science teachers who had worked in an Italian school, and the management of synchronous experiments that required setting specific appointments with people, made the recruitment of participants difficult, ending up with a sample size equal to 18. Thus, we interpret the results of the user study described in the following sections as initial evidence to be enforced through further experiments.

The participants were (13 female, 5 male, 0 not-binary, and 0 not declared) science teachers working in the preschool, primary, middle, and high school levels to collect feedback concerning diverse contexts of teaching in Italy.

Most participants are 31-40 years old (14) while few people belong to the other age ranges: 21-30 (1), 41-50 (1), 51-60 (1), and over 60 (1). Three of them work in preschool, 5 in primary school, 3 in middle school, and 7 in high school. The distribution of people’s expertise in using ICT technologies is as follows: advanced (7), intermediate (8), low (3), and null (0). The majority of the subjects (13) stated that they had never been involved in organizing any school trips; only 5 people had some experience in the past. However, 17 participants declared that they wished to organize this type of program in the future, and only one reported not being interested in planning any field trips.

Results

The participants of the user study used an average of 20 minutes to complete the whole test with task experiments, and questionnaires included.

Evaluation of Systems’ Usability

Table 1 shows the results of the SUS questionnaire (statements Q1-Q10). FieldTripOrganizer SUS scored 79.47/100 (Brooke, 1996). If we refer to the Sauro-Lewis curved grading scale (Lewis and Sauro, 2018), which turns the average value of the SUS into grades, our application obtained an A- score. In comparison, the baseline Klarna.Trips scored 34.44/100 which corresponds to the D score.

While SUS was only intended to measure the perceived ease-of-use, (Lewis and Sauro, 2009) showed that it provides a global measure of user satisfaction in the sub-scales of learnability (items Q4 and Q10) and usability (the other statements). In this respect, FieldTripOrganizer obtained a learnability value equal to 82.64/100: this means that users did not have to put much effort the first time they used the application. Moreover, its usability value is 71.18/100, which denotes good usability. Differently, Klarna.Trips obtained a learnability value equal to 53.47/100; in other words, it’s not so easy for users to learn how to interact with the application the first time they use it. Moreover, its usability value, 29.69/100, denotes poor usability.

FieldTripOrganizer obtained better results than the baseline in all the statements of the SUS questionnaire. We conducted a post-hoc comparison using a Mann-Whitney test which showed a statistical significance of the difference between the results on all the statements. In detail, the participants would like to frequently use FieldTripOrganizer (mean value M=4.) to plan field trips (Q1) with a large difference in the results. Regarding the ease of use and complexity of the system (Q2, Q3, Q4, Q8) FieldTripOrganizer is evaluated as easier to use than the baseline during the interaction with it. Moreover, participants evaluated our application as easier to learn to use (Q7, Q10) and felt more confident (Q9) in using it. Furthermore, they thought that the functions of the system are better integrated into FieldTripOrganizer than in Klarna.Trips (Q5).

Table 2 Post-task questionnaire results concerning the systems’ functionality

Evaluation of Systems’ Functionality

Table 2 shows the results of the users’ experience with the functionality of the two systems (Q11-Q14). Based on the initial results of this user study, FieldTripOrganizer received better evaluations than Klarna.Trips in all the questions and thus for all the dimensions of analysis:

• Support in organizing trips (Q11). Most participants (66.67%) claimed that our application is useful as a support to organize a field trip (ratings $\ge 4$, M=3.89). Only one person claimed that it was useless. In comparison, Klarna.Trips was rated very low (ratings $\le 3$, M=1.61) by 17 participants out of 18.

• Level of autonomy (Q12). Neither of the applications was considered sufficient to make the teacher completely independent in planning the field trip. However, FieldTripOrganizer obtained better results than Klarna.Trips (M=3.33 vs. M=1.56): 38.89% of users rated this aspect $\ge 4$ against the 5.6% of the baseline system. 50% of the people gave FieldTripOrganizer a medium score (3), meaning that the help of external bodies is considered necessary. This finding also emerged in the free-text comments, discussed later on, where some people claimed that external bodies should be involved in managing the transfer to the location of the field trip.

• Level of anxiety (Q13). One of the goals of FieldTripOrganizer was to mitigate teachers’ anxiety in the organization of field trips. According to the answers, the goal was reached in 55.5% of the cases, with ratings $\ge 4$ (M=3.39). Only 1 person stated that the application did not work for this goal; however, in comparison, this happened in the 77.78% of cases when using Klarna.Trips (M=1.89).

• Motivation in organizing a field trip (Q14). FieldTripOrganizer motivated teachers in organizing science field trips in 77.78% of cases: 15 users gave a rating $\ge 4$ (M=4.06). Differently, Klarna.Trips received a high score from a single participant (M=1.94).

Usability Analyzed according to People’s Declared Expertise in using ICT

We further analyzed the results by splitting the participants’ group into users who stated to be advanced ICT users (7 people) and users with low/intermediate expertise (11 people).

Table 3 shows the results of the SUS questionnaire on the two splits. Both groups evaluated our application better than the baseline: FieldTripOrganizer’s SUS score is 74.29/100 with the advanced users, and 72.95/100 with the intermediate/low ones, while Klarna.Trips’s one is 33.21/100 with the advanced users, and 35.22/100 with the intermediate/low ones. However, if we consider the balance between the SUS scores, we notice that the advanced users evaluated FieldTripOrganizer slightly higher than the intermediate/low ones (74.29/100 vs. 72.95/100), while the opposite holds for Klarna.Trips (33.21/100 vs. 35.22/100).

The answers to the individual statements of the SUS questionnaire can explain the previous findings. FieldTripOrganizer obtained better results than Klarna.Trips for all the statements on both user groups. However, Q1 shows that the intermediate/low group would have liked to frequently use our application to organize field trips (M=4.18) more than the advanced one (M=4.00). This is probably due to the fact that intermediate/low users particularly appreciated its support during the organization of the field trip. Differently, advanced ones gave less emphasis to the help provided by the application because they are more autonomous, being used to interact with different websites and sources to search for information.

Regarding the ease of use and complexity of the system (Q3, Q4, Q8), the intermediate/low users evaluated FieldTripOrganizer as more difficult to use than the advanced ones. Moreover, the former felt less confident (Q9) and had to learn more things (Q10) than the latter to interact with the application. These results are coherent with the idea that people unfamiliar with ICT might be challenged by it. Consistently, these users believed with less strength than the advanced ones that most people would learn to use FieldTripOrganizer very quickly (Q7). However, they found the application less unnecessarily complex than the advanced users (2.00 vs. 2.43). This is unexpected, given the other answers they provided. However, their evaluation might have been influenced by the fact that they really appreciated the system’s support, which prevented them from using different websites to find places and activities suitable for their students.

Table 3 Post-task SUS questionnaire results split by participants’ expertise in using ICT

Table 4 Post-task questionnaire results concerning the systems’ functionality, split by participants’ experience in using ICT

Functionality Analyzed according to Users’ Declared Expertise in Using ICT

Similarly, we analyzed the answers to the Q11-14 statements based on the differences in users’ declared expertise in using ICT; see Table 4.

• Support in organizing trips (Q11). The intermediate/low users perceived FieldTripOrganizer as slightly more useful as a support than the advanced ones (M=3.91 vs. M=3.86). We explain this finding with the fact that the intermediate/low users could rely on the information filtering and awareness support provided by the application to avoid mistakes and automatically solve at least part of the problems in the design of the field trip.

• Level of autonomy (Q12). Analogously, it seems that FieldTripOrganizer increases the level of autonomy of the people having intermediate/low expertise more than that of the advanced ones (M=3.36 vs. M=3.29). As previously discussed, this might be due to the fact that expert teachers are able to search for information by themselves; thus, they might give less emphasis to the help provided by our application than the other participants.

• Level of anxiety (Q13). The reduction of anxiety is stronger in intermediate/low users than in advanced ones (M=3.45 vs. M=3.29). Also in this case, we attribute the result to the information finding and partial automation support offered by FieldTripOrganizer, which offers a single place to find most of the data for the field trip organization. We believe that experienced people pose less emphasis on this aspect because finding content on the web is not a problem for them.

• Motivation in organizing a field trip (Q14). The motivation is the dimension where the difference between intermediate/low and advanced users seems more evident: M=4.27 for medium-low expertise teachers and M=3.71 for the expert ones. This means that, even though some people might have been challenged by FieldTripOrganizer’s user interface because of their lack of skills, they appreciated the functionality of the system.

Free-text Comments

In this section, we analyze the written comments provided by the participants in the free-text questions of the post-task questionnaires¹² to investigate the pros and cons of FieldTripOrganizer and Klarna.Trips. We identified the main themes emerging from the collected answers through Thematic Analysis (Braun and Clarke, 2006):

• Usefulness of being aware of the spare time to add items to the itinerary. Most participants considered the awareness of the remaining time while organizing the field trip as very useful. They declared that they appreciated it a lot in FieldTripOrganizer. For example, U14 claimed “I liked the interactivity with the application that allowed me to add or remove activities based on the time left”, and U16 wrote “It was helpful to know that I still had an hour and a half left over from the planned activities in case the time got longer”. Some people missed this functionality in Klarna.Trips: “It would be helpful to have timely information about the time each activity takes in relation to the total time” (U10). Moreover, a user (U5) suggested that it would be interesting to tailor the time required to visit a place or carry out an activity based on students’ school level: “I suggest paying attention to the time allocated to the various activities because it can vary when organizing trips with preschools to when organizing trips with secondary schools”.

• Mixed-initiative planning. The participants particularly appreciated the mixed-initiative interaction model we propose. For example, U13 declared “In general, I really liked the automatic planning function, so that you don’t have to build the itinerary manually, but still have the ability to select the activities to do”. Moreover, U14 wrote “I liked the interactivity with the application that allowed me to add or remove activities based on the time left”. Coherently, participants considered Klarna.Trips as less useful because it proposes an itinerary and, only after that, it lets the user add or remove any places from it. For example, U7 claimed “Since the system already proposes an itinerary that I can change, I feel less free, because I am influenced by its proposal”. However, for both applications, some participants declared that they still need external bodies such as booking agencies to organize the transfer to the locations of the field trip.

• Make the user constantly aware of the characteristics of the items/activities. The participants especially appreciated the fact that FieldTripOrganizer describes the features of the PoIs and of the activities, supporting an informed decision at every step of the trip construction. U3 wrote “In particular, I appreciated the fact that the system explicitly and immediately expressed the characteristics of the item concerning content, type, and level of complexity so that I could easily understand whether it was suitable for my class or not”. For what concerns Klarna.Trips, users commented that the system is less useful because it does not provide this type of support; thus, they had to understand on their own if the activities suited the students’ school level or not: “It was difficult and not immediate to figure out whether or not the activities were suitable for my target audience, it required a lot of effort on my side” (U13); “I did not always succeed in understanding if an activity was suitable for my students” (U17).

• Put into practice the lessons previously presented in the classroom. Connecting to out-of-school learning, some participants considered FieldTripOrganizer useful to put into practice the contents previously explained in the classroom. For example, U4 declared “In general, I find it useful to plan activities for a field trip to deepen the topics covered in the classroom through practical activities and site visits. In this way, I can turn what the students see in books into reality”. Moreover, U12 stated “Very nice application, especially because there are activities to do that are suitable for the students. This allows me to explain the theoretical part in the classroom and then implement the lesson in practice”.

• Take lunch breaks into account. Regarding both applications, some participants declared that it would be nice to take the activities related to lunch into account. For example, U6 claimed that “It would be useful to integrate other types of information, such as cafes or places to have lunch or picnic areas”.

Researchers’ Observations of Participants’ Behavior

With FieldTripOrganizer, the participants used the filters on the school level and domains of interest of the activities, showing attention to these functions both to find options that suit their student group and as tools to reduce information overload. For instance, U2 said “I select the primary school level so there is not too much information to explore”. Some participants also considered the duration of the activities as an indicator of the depth in which they are carried out. For example, U6 said “As this activity takes 5 minutes, it is probably not performed in great depth. Therefore, it may not be too difficult for my students”.

Nobody added to the itineraries any activities or places having a red semaphore, i.e., exceeding the spare time of the field trips. This suggests that the graphical annotation of items with colors describing their feasibility provides users with efficacious feedback for decision-making.

Moreover, the participants added and removed activities and PoIs while exploring the list of available options to make the school trip coherent with the topic they wanted to deepen with the students. Furthermore, their comments provide evidence that they kept an eye on the time left, trying to include into the schedule as many relevant activities as possible. For example, U9 said “I will remove this activity that takes too much time so that we can visit two other places”. In contrast, the geographical map was not that much used, probably because the presentation of PoIs grouped by geographical area favored the selection of places that are close to each other, and thus their displacement was not that relevant. Moreover, U5 said “You should pay attention to the time required for the activities and to move from one place to another because when you deal with kids probably the time required is higher” meaning that as future work there is a need to personalize the required time based on the school level. Furthermore, U7 said that people with a low ICT level could be challenged in using the application. However, this observation is mitigated by the fact that, according to the user experience results, low-ICT users declared that they would like to frequently use the system (statement Q1 of Table 3).

In comparison, the participants criticized the poor interactivity of Klarna.Trips, which limits the user control to adding or removing places from an already built itinerary. Moreover, they confirmed the fact (already observed in the written comments) that they had to understand on their own if the places or activities presented by the system suited the school level of their students.

Discussion of Results

In this section, we discuss how the initial results concerning the participants’ answers to the questionnaires, free-text comments, and the observations collected by the researcher during the user study answer our research questions.

• RQ1. How does the simultaneous provision of information exploration support and automated scheduling help teachers during the organization of a field trip? How does it motivate teachers? How does it impact their level of anxiety and their autonomy?

  • The findings suggest that the simultaneous provision of information exploration support and automated scheduling helps teachers during the organization of a field trip. In this respect, the participants’ feedback and observed behavior showed that the filters provided by FieldTripOrganizer are important because they empower the teacher to reduce information overload not only on a topic basis but also by considering the students’ school level. Moreover, the other key function is the high-level information-awareness support provided by the combination of how much spare time is available to include items in the itinerary, and whether the available options fit into the schedule or not. This type of support enables teachers to have control of the situation and focus on the aspects they repute important, without bothering to consider other details such as the scheduling and optimization of the itinerary, which they can delegate to the application.

  • The answers to the questionnaires suggest that the support offered by FieldTripOrganizer motivates teachers with diverse expertise in using ICT applications to organize field trips, and that it reduces their level of anxiety. We believe that the key factor in this respect is the mixed-initiative support offered by the application, which takes care of solving some technical and possibly annoying aspects of the itinerary organization, while putting the user in control of the main decisions to be made, i.e., of selecting relevant PoIs and activities for the student class.

  • The findings also show that the support offered by the application increases teachers’ level of autonomy in organizing a school trip, but that external bodies are still needed to finalize the travel aspects.

• RQ2. How can an automated scheduler help teachers having diverse technological expertise organize a field trip for a student class? Participants’ answers to the questionnaires, free comments, and observed behavior suggest that teachers appreciate a system like FieldTripOrganizer. However, this type of application specifically helps people having medium/low expertise in ICT by decreasing their level of anxiety in the management of organizational details. As previously discussed, the key factor is the mixed-initiative support that helps design the field trip without limiting the user’s freedom of choice.

• RQ3. During the organization of a field trip, how important is keeping teachers aware of the feasibility of their choices in the context of the temporal constraints to be satisfied? As previously discussed, according to the free-text comments provided by the participants of the study, and to the researcher’s observations, the continuous feedback about the consistency of PoIs with the time constraints of the field trip is crucial and appreciated.

Implications

To the best of our knowledge, FieldTripOrganizer is the first application supporting mixed-initiative field trip design. This brings a first practical implication, i.e., teachers can use this system to select and schedule out-of-school activities for their students. Having said this, we can identify some further practical implications, which we summarize in the following:

• An important one is applicability. Even though we tested FieldTripOrganizer in the organization of science field trips, it might be used to plan out-of-school activities in other topics that can be combined with practical activities to be done outside the classroom, such as history, archaeology, and so forth. Moreover, it might be used for organizing less interactive indoor visits, for example to an art museum. As discussed in “Domain Knowledge”, this flexibility derives from the generic domain model we adopted, which does not make reference to specific types of entities, but only to their domains of interest, i.e., to a property of the “Item" concept. By extending this property with new values, new types of places and activities can be managed by the application without introducing new data types that would require a revision of its implementation.

• FieldTripOrganizer could also be used outside the school context. For instance, a family with young children might use it to figure out which cultural activities are available in a geographical area and schedule a tour by selecting the places to visit and the activities to carry out. Especially in the case of families with kids, the school-level filter helps reduce the number of available options and only consider activities that are suitable for them.

• While FieldTripOrganizer can support the organization of articulated field trips, it might also favor simpler ones, such as the visits to places nearby the school, by enabling teachers to discover relevant activities and schedule them in the available time span. This might increase out-of-school learning because teachers would not probably need to delegate external bodies for support, and they could rely on our application for the setup of the itinerary, moving with students on foot, or using local transportation.

From a future perspective, the increase in out-of-school learning could favor the development of new labs for students and increase the collaboration between schools, laboratories, and museums.

Limitations and Future Work

The work we presented has the following limitations regarding the user study:

• As discussed in “Method”, the user study involved a small participant sample (size 18) because of the complexity of the recruitment task, in which we selected science teachers who had previously worked in an Italian school and we planned synchronous experiments in which both the participant and a researcher were present. The results we obtained have thus to be interpreted as initial findings to be substantiated with further experiments. By contrast, the management of synchronous experiments enabled us to collect richer feedback from these participants than what usually happens in an online experiment assisted by an automated system. First, we noticed that the presence of the researcher was an incentive for participants to pay attention to what they were doing; therefore, they carried out the test very carefully. Second, the researcher could collect detailed information about their behavior and experience, more than what they spontaneously declared. Anyway, to obtain stronger results, in our future work, we plan a larger experiment where we will further analyze the user experience with FieldTripOrganizer.

• Another limitation concerns the assessment of ICT skills to split the sample of participants into novices and experts and analyze user experience accordingly. To date, this type of information is mainly collected through self-reports and subjective evaluations for the sake of convenience (Kaarakainen et al, 2018). Similarly, in our user study, the participants self-reported their own ICT skills; thus we cannot be certain that the declared value is the actual one. Indeed, people might have overestimated it for social desirability reasons (Phillips and Clancy, 1972). In future experiments, we will have to find a way to assess the real value, for example, by means of a performance-based test (Siddiq et al, 2016).

As far as the functions provided by FieldTripOrganizer are concerned, we point out the following limitations:

• Currently, the application supports the search for relevant PoIs and activities by domains of interest and required school level, assuming that the user selects the geographical area to be visited. In our future work, we plan to support this type of search by extending FieldTripOrganizer with a function to search items by keyword, as we did in our past work on OnToMap (Mauro et al, 2020). The data model we defined supports this type of search, which might be based on the analysis of the title and description of items to select those relevant to specific search queries.

• FieldTripOrganizer does not support the execution of the itinerary in real-time, e.g., modifying the plan while the student group is working or moving around, because it exploits a static matrix to estimate the displacement time between places; see “Scheduling Support”. However, the application is based on a loosely-coupled architecture that can be extended to support the dynamic revision of the schedules by feeding the VROOM itinerary generator with real-time data provided by a navigator such as Google Maps via API.

• The geographical distribution of the PoIs considered in our user study made it possible to manually group them in different zones for their visualization in the user interface of FieldTripOrganizer. For the purpose of applicability, we plan to apply clustering techniques that, based on the geographical distance between places, will automatically define the areas that the system should present.

Ethical Issues

In planning the user study we complied with literature guidelines on controlled experiments¹³ (Kirk, 2013). As described in “User Study”, before starting the test, participants had to read an informed consent form describing the nature of the tasks to be performed and their rights (see https://bit.ly/3lwUGrn) and confirm that they had read and understood their rights by filling in the form. Every participant was given the same instructions by the researcher assisting the experiment.

During the user study, we did not collect participants’ names, nor any data that could be used to identify them: we worked with anonymous codes (U1, ..., U18) that the researcher attributed to people immediately before they started the test.

The user study presented in this paper exploited the FieldTripOrganizer application to extend a set of experiments devoted to evaluating information exploration support in recommender systems. The project grounding such experiments has been approved by the Ethics Committee of the University of Torino (Protocol Number: 0421424).

Conclusions

We presented the FieldTripOrganizer application to help teachers in an active organization of field trips. The key aspects are (i) the simultaneous provision of information filtering and automated scheduling support while the user designs the field trip, and (ii) the visual annotation of places and activities to show whether they can be included in the itinerary without violating its time constraints. Different from current tour planners, these functions enable the user to build an itinerary from scratch while being in full control of the design process, delegating the system to manage consistency checks and optimize the itinerary path.

We tested FieldTripOrganizer in a user study involving 18 teachers who compared it to a recent tour planner to create science field trips. Our application was superior to the baseline in both usability and user experience. Moreover, the participants of the user study declared that it was helpful, motivated them, and reduced their anxiety during the design of the field trip. Even though these findings have been collected in a small user study, they encourage the adoption of the application to plan field trips for out-of-school learning.