Keywords

1 Introduction

The contemporary travel industry offers tourists a range of options to acquire travel itineraries. The proliferation of Social Media has enabled tourists to use others’ experience to develop their own itineraries [1]. The advancements of Smart Tourism and automation support them with apps that offer real-time customisation and personalisation [2]. However, these tools still require substantial time and effort to find a relevant itinerary, understand it and adapt it to individual needs. More recently, ChatGPT allows tourists to generate texts for various purposes, including for travel itineraries [3, 4]. While ChatGPT is a quick, easy-to-use, and innovative tool, the quality of the ChatGPT-generated content for travel itineraries remains unclear. This paper aims to provide a preliminary evaluation of the ChatGPT-generated travel itineraries quality compared to human-created itineraries offered by travel experts. The study suggests that ChatGPT can be used for efficiently drafting preliminary travel itineraries. Such itineraries need validation and further planning to ensure a positive travel experience.

2 Literature Review

2.1 The Value of a Travel Itinerary

A travel itinerary is a subset of points of interest (POI), organised sequentially to optimise travel experience within available tourist time [1]. Tourist experience from a destination is constructed as a sum of individual occurrences (aka “experiences”) in specific contexts [5]. Thus, tourists are primarily interested in exploring attractions and engaging in other available activities at a chosen destination. A relevant choice of POIs and a convenient schedule would contribute to a positive tourist experience. Such factors as too many planned attractions, inadequate time dedicated to each of the POIs, and time and effort invested in commuting between them can prevent tourists from satisfying their needs while facing losses of time, energy, or money [1]. The tourist experience from a destination largely depends on the choice of the itinerary [6]. Tourists are in search of tools which can help them in optimising their travel experience.

2.2 The Quality of a Travel Itinerary

Planning a high-quality travel itinerary is a challenging task for tourists [7]. Initially, the quality of a travel itinerary was directly associated with the high density of attractions [1]. Taylor et al. [1] define a “good” travel itinerary as one that maximises the value of the POIs subset while being manageable within the available travel time and budget. An itinerary need to include useful planning information (e.g., locations, opening hours, distances, time for each POI, costs), which is conveniently presented to tourists. It should also be personalised, i.e., to be relevant to the tourists’ individual needs [2]. The development a travel itinerary should take a holistic approach.

Wang & Strong [8] proposed a holistic framework for tourism content. An itinerary content should have high intrinsic quality. The POIs description, and the information required for planning, should be up-to-date, objective, reputable and believable [2]. The itinerary content should have high representational quality, i.e., to be consistent, concisely structured and presented to enable ease of use [9]. The itinerary should be adapted to real-time tourist context and to have exhaustive information for them [2]. Finally, the itinerary content should be accessible to all target tourists (Fig. 1).

Fig. 1.
figure 1

Conceptual Framework for Travel Itinerary Quality

Full size image

2.3 ChatGPT for Travel Itinerary

ChatGPT can be used in all stages of the trip by both tourism managers and tourists. ChatGPT (Generative Pre-training Transformer) is a case of generative AI, which aims to predict the likelihood of typical human word sequences, thereby creating texts that resemble natural human speech [10]. It can generate ideas for a visit to a destination and outline the main attractions and activities by day based on a predetermined length of stay in the destination [3, 4]. It can also form a short description of the attraction/activity and the main reasons why it should be visited/performed. ChatGPT can used to automatically generate an itinerary-like text to substitute a multistage planning.

The quality of the ChatGPT-generated texts quality varies. The data ChatGPT used is until 2021, hence its itinerary might be outdated. Moreover, ChatGPT is known for hallucination and invention of facts [10]; hence, its itinerary might not be factually correct. While ChatGPT has been tests for some tasks (e.g. passing exams, generating marketing text [10]), its potential for developing travel itineraries remains underexplored.

3 Methodology

To assess the quality of a ChatGPT-generated itinerary, the study compares the ChatGPT outcome quality towards expert-develop itineraries. The study analyses 3 different case studies: an iconic destination (Vienna, Austria), a secondary-level country destination (Plovdiv, Bulgaria), and a tertiary country-level destination (Spetses, Greece). First, the study created a baseline for comparison of the ChatGPT results with the information that is openly and easily available to tourists. The study collected travel itineraries from travel websites by using “3-day travel itinerary” + “destination” as search terms. Only the content published/updated at reputable website or a reputable author in 2023 was retained. 6 out of 17 Viennese itineraries and 3 out of 6 Plovdiv itineraries were selected as high-quality human-generated itineraries. For the case of Spetses, the search did not identify a published 3-days itinerary, which reflected a common trend of a single night stay at the island. Therefore, for Spetses, the study used the published suggestions for travel activities for this island. The content of itineraries was evaluated by the two researchers with the expertise in itinerary planning to ensure their quality. Second, ChatGPT-4 was used to generate 3-day itineraries for the abovenamed destinations: Vienna [11], Plovdiv [12], Spetses [13]. The quality of the ChatGPT texts was evaluated by the authors against the human-made itineraries.

4 Findings and Discussion

Table 1 present the evaluation of the human- and ChatGPT-generated itineraries. The intrinsic quality of the ChatGPT-generated itinerary can be characterised as questionable. ChatGPT generates objective facts about tourist attractions. ChatGPT officially acknowledges that its training data was until 2021. Its information might not reflect recent changes in a destination’s offer. Nevertheless, the generated page includes the date of the text-generation, which creates a false impression of the itineraries being up to date. However, no fake information was identified. On the other hand, ChatGPT fails compared to human-generated itineraries because it only includes the permanently existing travel attractions (aka museums and monuments). Other types of POIs (temporal exhibitions, restaurants for lunch/dinner, stops, toilets, timeframes, etc.) are not included, making the itinerary less useful. Importantly, the ChatGPT-generated itineraries look similarly believable to the human-generates ones. Considering that in some cases ChatGPT positions itself as a reputable and accurate tool [10], the itinerary might be falsely perceived by non-experts as accurate. To ensure travel experience, ChatGPT-generated itinerary requires validation prior to usage.

Table 1. 3-days selected travel itineraries
Full size table

The analysis of contextual quality of ChatGPT-generated itineraries provides contradicting results. ChatGPT can provide a personalised outcome. Thus, standardised expert-developed itineraries do not offer a 3-day itinerary for Spetses. ChatGPT produced a 3-day travel itinerary for this destination. This provides tourists with the response to their exact request. However, this outcome does not reflect optimal travel experience, taken into consideration by experts. Importantly, in comparison to human-made itineraries, ChatGPT fails to provide exhaustive content. It generates a minimalistic description of a POI, which limits the itinerary’s usefulness and prevents tourists from making informed decisions. Tourists need to check the relevance of the whole itinerary and each POI to ensure positive travel experience.

The representational and accessibility quality of ChatGPT-generated itineraries is high. The POIs information is consistent and concise. The website is up-to-date and mobile-friendly, which makes it accessible from multiple devices and adjustable to individual needs. While this meets high usability standard, this might affect tourists’ perceptions on believability of the content. The ease of use and the accessibility of ChatGPT-generated content can benefit in planning accurate and relevant itineraries.

5 Conclusion

ChatGPT demonstrates potential in generating relevant ideas for activities and attractions in a destination. For popular destinations, it can create factually correct and relatively feasible itineraries without “hallucinating”. However, it only includes permanently available attractions and ignores other types of POIs, thereby, diminishing the usefulness of the itinerary. The itineraries also lack sufficient details to provide value. At the same time, for a small destination ChatGPT can create an itinerary that is not otherwise available from experts. ChatGPT can be used as a first rather than the last or the only point in the travel inspiration and planning stage.

The main limitation of the paper is the small sample size, although it focuses on three destinations with different characteristics. Future research may involve more destinations and validation of ChatGPT-generated itineraries by travel experts for more reliable results. Future research may focus on the practical challenges travel companies face in the implementation of ChatGPT and generative AI in their operations, and tourists’ trust is AI-generated travel itineraries. Research can also shed light on the potential automatability of other tasks (beyond travel itinerary development) at travel agencies.