1 Introduction

The benefits of nature to the public are widely recognised. For example, a recent EU report [1] concludes that

There is robust scientific and practice-based evidence that nature can contribute to addressing health and social challenges that EU citizens are facing

In the commercial domain, and particularly the mobile app market, there are a growing number of smartphone apps that support interaction in nature, for example apps designed to support hiking in nature [2]. Furthermore, there is a growing research community interested in investigating the potential ways that digital technology such as smartphones can be used to support users’ engagement with nature [3] and the tensions and challenges that arise. A key challenge for interaction design is to explore and uncover technology designs that are both playful and unobtrusive to the nature experience.

In this article, we contribute to this exploration by presenting findings from our in-the-wild observation study of a digital display deployment designed to support the nature experience of visitors to a valley in the heart of the English Lake District. The growing research literature focussed on the interaction design issues associated with public and situated digital displays highlights a number of key challenges and opportunities. One significant challenge is that of display blindness [4], a phenomenon whereby passers-by tend not to notice or give their attention to digital displays. Another is interaction blindness [5] in which even when noticed, a user may assume that a digital display does not support interaction. Both of these phenomena arose through researcher observations of real-world digital display deployments (and further discussion of these issues is presented in Sect. 2.1).

The digital display deployment described in this article relates to a particular type of display known as a projection augmented relief model (PARM) [6]. These displays feature high-resolution physical relief models of landscapes enhanced through the digital projection of high resolution maps, imagery and animation. PARM displays aim to inform and educate whilst exploiting the engaging and tactile qualities of physical models. Our research aim was to observe how visitors would notice and interact with the PARM display in the particular deployment setting.

Physical models have long been an option for landscape representation in the context of planning and landscape architecture where public consultation is required [7]. Many options for ‘3D displays’ exist [8], aiming to immerse the viewer in a modelled landscape, many being classed as ‘non-autostereoscopic’ in that they convey stereo effects using viewing aids such as virtual reality (VR) headsets. Where a broader landscape context needs to be communicated, for example showing patterns across a region rather than within a local scene, then first-person immersion is less appropriate and an aerial perspective can offer a more effective spatial frame of reference. Relief models have been recognised as powerful tools for giving very natural overviews of landscape for hundreds of years in contexts such as military training [9] and to aid orientation for visitors as with the Swiss alpine models [10]. The production of physical models is now easier than ever thanks to more freely available digital terrain data and 3D fabrication technology so the relief model is now a viable option for certain types of landscape visualisation especially when augmented through projection as with PARM. Where landscape overviews are required, PARM therefore presents an alternative to visually immersive yet physically detached forms of visualisation like VR and as with traditional relief model displays allows for a shared experience supported by direct discussion and gesture.

A physical relief model augmented with projection could be considered a tangible display, benefitting from the generic and engaging power of solid models as seen across a range of disciplines [11]. Solid models of all kinds offer kinaesthetic interactions that include the ability of the viewer to effortlessly change their perspective on the model to appreciate its structure in three dimensions. In the case of landscape models, this can help the viewer build up an understanding of the spatial interrelationships between geographical features, whether these are parts of a settlement or mountains and valleys without the need to interpret symbols on a map.

The spatiality associated with people’s movements to and around displays featuring physical landscape models would be considered part of the ‘spatial interaction’ theme within the framework for tangible interaction described by Horneker and Buur [12]. Many visitor centres around the world feature landscape models as part of their interpretation strategy, usually coloured to reflect the land cover and often allow visitors to highlight places of interest. In some cases routes, points of interest (PoIs) and a ‘You Are Here’ (YAH) location can be highlighted by buttons which illuminate bulbs set into the model as in Fig. 1a. Where places of interest are distributed across a larger model, the buttons can be arranged around the edge of the model in appropriate locations, often accompanying small interpretation panels as in Fig. 1b, requiring physical movement to explore the content.

Fig. 1
figure 1

Landscape models in visitor centres. a Mount Rainier National Park, Washington, USA. b Lake Mead National Recreation Area Visitor Centre, Nevada, USA

Full size image

Physical landscape models also offer potential for direct interactions with parts of the model in reference to the features in the real world which they embody, for example by pointing or tracing. In this sense, models can act as tools for discussion about landscape but there would rarely be an expectation for any kind of output response from the input gestures. There are however types of landscape display designed to be manipulated by hand and to provide response to such inputs by projecting new surface information. The Illuminating Clay project [13] for example allowed a clay landscape model to be deformed through touch resulting in new contours, water flow and other surface parameters being projected onto the model in response. These and similar sand-based models allow a very direct link between input action and output response which offers users a very intuitive spatial mapping when exploring the task space [14].

PARM displays, as with Illuminating Clay, use projection onto landscape models, but here, the models are solid and so cannot provide the same kind of interaction outputs. In many PARM displays, the delivery of information has been achieved through a passive sequence, or organised into themed sections each triggered by interaction via buttons or a touch screen. Users do however consistently interact with the models themselves in a very tactile way either by exploring the surface texture or by making more conscious references to places of interest if part of a group. This has been particularly evident in PARM displays when the user’s current location is within the modelled area, with a typical interaction beginning with touching the model accompanied by a comment such as ‘look, we are here’.

In this research, we follow a technology probe approach [15] using a simple PARM display to act as a You Are Here model in quite an explicit way, exploring the user’s spatial interaction in a general sense but also to inspire ideas for more direct modes of interaction. The in-the-wild [16] study featured a 9-month deployment involving close collaboration with the Lake District National Trust (NT), a conservation charity responsible for managing a significant portion of the land and most of the public bridleways in the Lake District. The Lake District is located in Cumbria in the North of England and was awarded UNESCO World Heritage Site status in July 2017, with the NT as one of the key partners involved in preparing the bid.

Our main contact with the NT throughout this research has been the Trust’s World Heritage Site Programme Manager. In working with the NT, we have co-designed the content featured on the PARM to communicate the work of the NT (e.g. its conservation activities) and to engage and orient visitors to the Langdale valley which is situated in the heart of the English Lake District National Park. The site for our deployment of the PARM display within the Langdale valley was a pub owned by the NT called Sticklebarn. Unlike the majority of NT properties, Sticklebarn would not be considered a visitor attraction itself, but rather a place to take refreshments whilst enjoying the landscape within which the pub is set. The PARM display consisted of a projection-enhanced relief model and a monitor set into a custom-made console as shown in Fig. 2. The NT provided details of a 2 km hiking trail through the Langdale valley beginning and ending at Sticklebarn. The trail passes through a number of key PoIs that highlight the diverse cultural landscape of Langdale and the land management and conservation activities carried out by the NT.

Fig. 2
figure 2

The PARM display deployment

Full size image

The remainder of this article is structured as follows. Section 2 describes background and related work and includes sub-sections on PARM (Sect. 2.1), public and situated displays (Sect. 2.2) and the emerging body of research focussing on the design of digital technologies to support engagement with natural places (Sect. 2.3). Section 3 describes our design of the PARM display and the strong involvement of the NT in supporting its deployment and the co-design of content. Next, in Sect. 4, we present details of our 3-day observation study and its findings including the apparent expectation of visitors for direct interaction with the PARM. A discussion of our findings is presented in Sect. 5, and this is followed by our concluding remarks (Sect. 6).

2 Background and related work

In the following sections, we present background and related work associated with three key areas: PARM (Sect. 2.1), interaction design related to situated displays (Sect. 2.2) and finally, the role of digital technologies in supporting engagement in natural places (Sect. 2.3).

2.1 Projection augmented relief models

The projection augmented relief model (PARM) technique involves a situated display featuring a physical relief model (Fig. 3a) textured from above through projection (Fig. 3b), sometimes coupled with an adjacent monitor to display ancillary information.

Fig. 3
figure 3

The projection augmented relief model (PARM) technique. a Physical relief model painted white for projection. b PARM configuration featuring monitor for ancillary information

Full size image

PARM could be considered a specific subset of the projection augmented (PA) model technique [17] where physical objects are dynamically textured through projection. Tangible 3D tabletop displays [18] represent tangible interfaces where physical objects can be moved around, triggering responses in the form of information being projected onto the objects and the table. The ‘Illuminating Clay’ project described earlier was an example of a representation of landscape being textured through projection, later developing into a tool to explore landscape analysis principles [19], although here, it was a malleable surface rather than a solid model. Using a similar technique, augmented reality sand boxes [20] have been adopted by many schools as engaging tools to support spatial literacy teaching. PARM differs from these techniques in that displays feature solid high-resolution models of real landscapes created from a range of digital terrain data. The landscape model can neither be moved nor manipulated yet acts as a tangible display that features detailed place-specific content based on maps, imagery and animation projected onto the model with ancillary information displayed on a screen.

The potential affordances of PARM were described in [6], based on the development of a number of trial installations. The authors gained practical experiences of building and deploying subsequent PARM installations beginning with a display at the Wordsworth Trust museum in Grasmere, Cumbria, UK. This deployment sought to engage visitors with a collection of manuscripts in the gallery which related to William Wordsworth’s childhood memories of events in certain places in the landscape which inspired later poetry. The ‘Spots of Time’ PARM display (Fig. 4a) featured a number of projected sequences highlighting places in the landscape that visitors could initiate via a touchscreen, whilst they heard extracts from the poem being read via an audio shower. Experiences of using the touchscreen, model, screen and audio shower suggested there was a need to simplify subsequent displays to help visitors focus on the content. One such display was the PARM deployment at Southwell workhouse, Nottinghamshire, UK (Fig. 4b), an NT property, which offered visitors background information relating to the role of the workhouse by showing seasonal movements of people from the surrounding landscape to and from the workhouse. The display featured at the start of the visitor’s trajectory through the building and aimed to connect some of the content to objects and stories elsewhere in the property. This display featured a single content sequence on a continuous loop, with ancillary information projected to the side of the model.

Fig. 4
figure 4

Previous PARM deployments. a The Wordsworth Trust gallery, Grasmere, Cumbria, UK. b The Southwell workhouse, Nottinghamshire, UK

Full size image

The PARM displays discussed so far aimed to place certain objects or themes relevant to the building where the display was housed into a broader landscape context. As such, the focus was to help with visitor ‘interpretation’ within a curated cultural heritage space. The geographical area covered by the displays was large and connecting more directly with features of interest in the immediate surroundings was less of a concern.

There is great potential for PARM to operate at a more local level, promoting themes of interest that can be explored immediately outside. In this way, PARM could operate as a media-rich form of You-Are-Here map, attempting to engage people with their natural surroundings.

YAH maps are often stylised in some way, whereby a graphic visualisation is used to highlight a particular facet or facets of the terrain covered by the map (e.g. in order to highlight the locations of PoIs or the path of a walking trail). In the case of traditional YAH maps, this visualisation is static and fixed. The PARM approach allows for a series of PoIs, perhaps along a trail, to be introduced to the viewer in stages, or selectively, along with some additional information. This along with the natural frame of reference offered by the raised relief model therefore represents a powerful tool for promoting local exploration.

2.2 Public and situated displays: interaction design challenges

The Langdale PARM display can be categorised as a situated display and according to Weiser’s categorisation [21] of ubicomp technologies can be considered as a yard-scale display. There is a growing research literature focussed on situated displays and associated interaction design, and this understanding has clear relevance to the PARM display.

One key interaction design challenge associated with displays situated in public or semi-public settings [22] is the need to entice users to engage with the display. In the seminal work by Brignull and Rogers [23], which involved the deployment of a large situated display placed in two authentic social gatherings, the authors observed that users were reticent to interact with the display because of the potential for social embarrassment. In this same research, the authors propose the notion of activity spaces for considering the actions of users in moving from passive bystander to direct interaction with the display.

A further issue associated with public display deployments is that of display blindness [4] whereby potential users can fail to notice displays assuming that they are some form of digital signage associated with advertising, or they actively look at them, but choose to ignore them [24].

In a recent analysis of the issues around display blindness presented in [5], the authors note that current display blindness and engagement studies have focussed largely on traditional flat panel displays. The PARM does not take this form and consequently appears less likely to be dismissed or ignored for being advertising-based digital signage. In their work on curious objects [25], the authors describe their 1-day exploratory study into the potential of attaching so-called curiosity objects to interactive displays in order to mediate interaction, and this approach showed promising results in terms of reducing the blindness problem. Given the novel and salient appearance of the PARM display within the Sticklebarn setting, it is likely that the curiosity of visitors played a significant role in enticing them to engage with the PARM display.

Another interaction issue is that of interaction blindness. This refers to the problem of users not necessarily realising that a public display supports interactivity. As discussed more fully in the findings of our observation study (Sect. 4.2), we actually observed the opposite of this phenomenon with some users expecting an interaction capability, such as direct touch interaction, to be supported by the PARM display. The 3D terrain surface of the PARM appears to afford a physicality that encourages users to touch the display in order to trace contours with their fingers (as illustrated in Fig. 2.) and in this respect supports a form of tangible interaction, though no objects are moved or manipulated as such [26]. When considering the potential of such tangible interaction in the context of public visualisation displays, Claes and Vande Moere describe how “tangible interaction can elicit different forms of engagement and generate more and deeper kinds of insights, when compared to traditional public display media” [27]. In [28], the authors discuss their formative studies of a tangible map designed to serve as an interactive centrepiece within the MIT campus and which combines a tangible user interface (created with 3D-printed buildings) and dynamic spatial information displayed on a flat screen beneath the building models. In this case, the buildings represented a set of well-defined and visually distinct PoIs which had been made responsive to touch. This differs from many of the landscape surfaces used in PARM displays which represent continuously variable terrain surfaces which do not have discrete physical PoIs; therefore, any expectation of interactivity from the viewer is harder to predict.

In Sect. 5, we return to the aforementioned design challenges and insights when describing our observations of visitors engaging with the PARM display in the social space of the Sticklebarn.

2.3 Digital technologies supporting engagement in natural places

There is a growing body of research activity focussing on the potential of digital interactive technologies to support people’s engagement with natural places in an unobtrusive manner [3, 29] and to facilitate outdoor recreation [30] and play [31]. One of the seminal research projects to focus on the use of digital technology to facilitate engagement with nature was the ambient wood project [32]. Other examples include the GreenHat Mobile Augmented Reality System, supporting students in learning about biodiversity [33], the MobileGIS system which forms one of the case studies described in [34] and the research presented by [35] which studies the ways in which fifth-grade students interacted with nature using mobile technology during a nature hike. The aforementioned examples all utilise mobile screens but in research on so-called nature-technology hybrids [36], the use of such technologies is deliberately resisted because of the potential ‘captivating hold of the screen’. With a similar design goal in mind, Robinson describes his research on developing prototypes that support so-called Eyes-off interaction in [37]. In [38], the authors present seven themes to support technology design in natural places in ways that ‘provoke integration of the natural and computational worlds’. One such theme is that of revealing and receding which suggests that ‘design must simultaneously fade into the background and provoke seeing natural places differently’. This theme relates strongly to the design of the Langdale PARM display which aims to provoke visitors into seeing the different facets of Langdale that are not immediately visible, such as the valley’s farming heritage and the on-going land management activities that have a direct impact on the valley’s appearance. This theme also resonates with the analysis presented in [39] which suggests that: ‘The proliferation of mobile apps brings into sharp relief the power of digital technologies to disrupt, and therefore reveal, aspects of our experience of the natural world’.

3 Design and installation of the PARM display and associated content

In this section, we describe our development of the PARM display and the stages of co-design relating to the overall objectives, the content and the configuration of the display. The design decisions involved in creating this technology probe are considered particularly relevant because the site for deployment was not a conventional location for displaying landscape interpretation material, and there was a collective shared interest in exploring public engagement in such a context.

3.1 Early discussions with the National Trust

The research presented in this paper followed on from an earlier involvement with the NT as part of a wider research and development project, exploring the potential of digital technology to engage visitors to the Lake District. In more detail, the Trust’s Visitor Experience manager had supported the co-design of a mobile app-based locative media experience for visitors to the Lake District’s Borrowdale valley [40]. The positive feedback received from this research led to an e-mail introduction in November 2016 from the NT’s Programme Manager responsible for their World Heritage Site bid. In his e-mail, the programme manager summarised his approach as follows:

“... I am developing a series of valley-based walks/itineraries that tell the World Heritage Site/cultural landscape story. Each valley is able to showcase different aspects of the WHS’ Outstanding Universal Values – Identity, Inspiration and Conservation”

During the first face-to-face meeting with the programme manager, he detailed the aims of the trust with respect to attaining World Heritage Site status and his interest in using digital technology to support valley-based walks. He had produced a document detailing one such walk (and associated itinerary) for the Langdale valley.

The 2 hour circular walking route (which started at the Sticklebarn Inn) was presented as an annotated map (Fig. 5) and featured a number of key PoIs highlighting the diverse cultural landscape of Langdale and the land management and conservation activities carried out by the NT.

Fig. 5
figure 5

Annotated image provided by the NT showing the recommended hiking trail

Full size image

The authors considered the capabilities of a PARM display to be well-suited for promoting the walk from the Sticklebarn and during a later meeting, the authors offered to demonstrate how the route could be visualised using such a display. The demonstration took place at the Sticklebarn in February 2017. Note that in this article, we focus on the PARM display rather than the smartphone app which was also developed to support the walk.

The demonstration session was attended by both authors, the programme manager and the NT’s operations manager for Langdale. To demonstrate the PARM concept, a small rig (Fig. 6) was used, showcasing the possibilities for various projection styles onto an existing relief model surface representing a different part of the Lake District landscape, with associated information being displayed on an integrated monitor. A digital mock-up of what a physical relief surface centred on the NT’s walk might look like it was also used to help plan the exact spatial extent of a physical relief model to use in the final display. The reaction of the NT staff was overwhelmingly positive and discussions quickly turned to the requirements for a PARM display if deployed at Sticklebarn.

Fig. 6
figure 6

Meeting in Sticklebarn with National Trust Staff to demonstrate and discuss the PARM display and potential deployment locations, February 2017

Full size image

Given the available spaces at Sticklebarn, it was agreed that a reasonably small footprint for a physical display would be necessary. In order to test public reaction to using this kind of display, it was decided to initially adopt a simple passive display using a relief surface model, a monitor and projector. There would be no interface to control the content which would be shown as two synchronised displays, where a projection onto the model would have accompanying content on an adjacent monitor. In terms of the model area, there was a requirement to present the detail of a walking route in the valley but also the spatial context of that route. To determine the geographical coverage required the location in the landscape of the broad features that were of interest from that walking route was located on the map and compared to the initial mock-up. The resulting coverage area for the relief surface represented a square on the ground covering just less than 4.5 km across. The combination of spatial extent and level of detail required suggested that a cartographic scale of around 1:10,000 would be suitable, with no vertical exaggeration, resulting in a relief surface model 45 cm across.

In order to create the relief surface model, a digital surface model (DSM) derived from airborne radar was deemed suitable as the recognisable craggy relief, as seen in Fig. 7, would be represented more so than a model derived from contour lines and would be seen by many to characterise this landscape. This DSM was cropped and processed within ArcGIS software, exported as a mesh model, and then the physical relief model was carved from robust model board using CNC milling. The model was sprayed white suitable to enhance the projection effect and mounted at a height of 80 cm on a purpose built console containing a monitor and a projector.

Fig. 7
figure 7

The Langdale surface model

Full size image

The fidelity of the projection is also important and should match or exceed that of the physical features represented on the model in order to create a holographic effect. Where the vertical extent of the model is relatively large as is the case here when modelling a small area in upland terrain, then the imagery needs to be warped to fit the physical model. This is achieved by rendering the projected images or maps from the perspective of the projector’s vantage point, using a scale model of the display in CAD.

3.2 The co-design of content for the PARM display

In order to implement projections onto the relief surface synchronised with a descriptive narrative running on an adjacent monitor, it was agreed to use PowerPoint as the authoring environment, which would also allow the stakeholder to easily modify the content.

The demonstration contained examples of various categories of projection effect which enabled the NT programme manager to make more informed suggestions as to possible projections onto the relief surface for each particular theme. To summarise, the categories of projection effect were:

  1. 1.

    Animated illumination effects mimicking changing sun angle.

  2. 2.

    Static illumination effects mimicking a light source on the relief model surface. A variant of this can suggest a cone of vision or ‘viewshed’ if wanting to highlight the area that can be seen from a vantage point.

  3. 3.

    Spotlight effects illuminating an area of land but with no hard boundary.

  4. 4.

    Highlighting map features where points, lines or areas are emphasised through colour, style or brightness.

  5. 5.

    Animated map features to highlight movements including progress along a route.

  6. 6.

    Animated atmospheric or environmental conditions including flooding, snowfall and cloud shadow effects.

The co-production of content began with the programme manager producing a second version of the walk comprising concise notes as to the intended messages for each slide along with some initial ideas for projection which were discussed with the authors. Examples of notes from three PoIs along the walk follow, showing summaries of the NT programme manager’s initial ideas for the intended message at each point, and the projection onto the relief surface model, with any ideas for the monitor referred to as the ‘second screen’.

3.2.1 Example 1: the Neolithic axe factory

Summary of message

There is evidence of Neolithic stone axe manufacturing from around 4000 years ago in an area up on the hillside. These high-quality axe heads were prized across the country, and they were exported from Langdale.

Projector animation

“At PoI, highlight the higher scree slopes where the axe factory was roughly. On 2nd screen show Neil’s reconstruction of axe?”

The location to be highlighted here was approximate, but there was a location recognised as the centre where many axes had been found. A static campfire-like illumination effect was chosen to suggest a centre of activity but with a fuzzy boundary (Fig. 8a).

Fig. 8
figure 8

Examples of projection content styles. a Lighting effect. b Highlighted map features. c Spotlights

Full size image

3.2.2 Example 2: footpath restoration

Summary of message

Towards the head of the valley, there are popular paths up to the summits of Pike of Blisco and Bow Fell. Due to the large numbers of walkers combined with weather erosion, these paths are in a terrible condition, often leaving large scars on the fellsides. The National Trust and its partners initiated a scheme to repair the paths and the adjacent land. Today, upland paths across the Lake District are maintained by the Fix the Fells partnership, with National Trust rangers and over 100 ‘lengthsmen’ volunteers and donations are welcomed.

Projector animation

“Show the paths on the map; 2nd screen could show before/after picture; pictures of fell top path workers.”

There was a requirement here to show many known footpaths in an accurate fashion, with the programme manager using the phrase ‘on the map’ so suggesting a mechanism to highlight this set of map features achieved by using a bright colour line symbol with a darkened aerial imagery behind (Fig. 8b).

3.2.3 Example 3: the view from Old Dungeon Ghyll hotel

Summary of message

The Old Dungeon Ghyll was originally a farm and an inn. In the early 1900s, the farm was bought by historian and academic George Trevelyan who gave it to the National Trust. Trevelyan continued buying farms as they came available, donating them to the National Trust. His aim was to conserve the valley’s farming heritage and to protect public access rights. In 1949, the old bar changed to the Climbers’ Bar and became a hub for climbing clubs from across the country. Dungeon Ghyll Force behind the inn is the setting for William Wordsworth’s pastoral poem ‘The Idle Shepherd-Boys’.

Projector animation

“pics of old climbers/old tavern? Highlight all the GMT bought farms in the valley. 2nd screen pics of GT and the other Trust properties?”

There are several possible themes attached to this location including farm purchases, early rock climbing and Wordsworth’s literature. It was agreed to keep it simple and have one message which was the legacy of Trevelyan’s farm purchases but even here, there were options for portraying this which were discussed at a follow-up meeting. Map symbols to highlight the farm locations would appear distorted by the often rugged nature of the relief surface so a series of soft spotlight effects were animated in a sequence to reveal the five farms within the study area (Fig. 8c).

The overall intention of the display was to raise awareness of NT activities but without detail, instead making reference to learning more via the in-field experience provided by an accompanying mobile app. A sequence was developed which began with an illumination effect representing sunrise over the valley (Fig. 9a) followed by cloud shadows moving over aerial photography. The information on the monitor presented a narrative description of the area and each stage of the suggested walking route. Movement along the route between each PoI was animated with a bright blue dot over aerial imagery, whilst on the monitor the viewer’s attention was directed downwards with an arrow and the text ‘Walk the Langdale trail’ (Fig. 9b). As the animated dot reached each PoI the location highlighted over the model using a combination of map data, imagery and illumination effects as appropriate. For example, in some circumstances, well-defined areas could be highlighted such as the medieval ‘ring garth’ wall around the valley floor (Fig. 9c).

Fig. 9
figure 9

Further projection techniques. a Sunrise effect. b Focus on model whilst animated dot follows trail. c Highlighting areas on the model

Full size image

A full summary of the content on both monitor and relief surface projections at each PoI is shown in Table 1.

Table 1 Content for Langdale PARM at each point of interest (PoI)
Full size table

3.3 Installation of the Langdale PARM display

There were several options for locating the PARM display within the Sticklebarn building. After reviewing the possibilities, it was agreed that the initial location would be on the upper floor of the building on route to an overspill seating area and open plan activities space (Fig. 10).

Fig. 10
figure 10

Plan of Sticklebarn showing the placement of the PARM probe alongside the generalised visitor flows

Full size image

Some locations received quite bright sunlight through windows, so to ensure the projections were always effective, a full-sized 3000-lm projector was used. The display would be set up each day by a number of different members of the bar staff, so setup was made automatic after the projector and the PC were switched on. The display was in a busy public setting whose primary function was a pub so the whole display, including the relief surface model, needed to be robust. The display was not monitored by staff, so the projector was partially enclosed in a housing to prevent it being switched off, though during the initial installation, it became clear that this housing was getting too hot and so the display was taken away, modified and re-installed the following week. This latter point highlights another benefit of the technology probe approach, namely field testing the technology [15].

After several weeks, following observation of visitor flows, and in consultation with the manager of Sticklebarn, the display was placed on the ground floor as shown in Fig. 10. The display was visible to visitors as they walked through the main entrance and was en-route to the toilet, side entrance and upper level with additional seating. Adjacent to the display was a shelf unit containing leaflets and local information.

4 Observation study

In this section, we describe our observation study of the PARM display deployment which had the research aim of gaining an understanding for how visitors to Sticklebarn would notice and interact with the display. Furthermore, we wanted to investigate the extent to which phenomena such as display blindness would be evident and the extent to which visitors would be enticed to move between different activity spaces and stages of interaction according to the audience funnel framework.

4.1 Procedure

In order to reduce possible observer effects, the observation was conducted in a non-intrusive manner with both authors sitting at a table situated approximately 5 m from the display as shown in Fig. 10. On this occasion, we were unable to employ video recording because the manager of the Sticklebarn was not comfortable with this and its use was not imperative for achieving the goals of the observation study.

Our use of the observation method followed a semi-structured approach with data capture supported through a coding sheet that was informed by the schemes employed by previous observation studies investigating public display use seen in [4, 23] and the analysis presented in [5]. Our coding aligned most closely with the audience funnel model [4] because the PARM display was effectively deployed along a thoroughfare (with people passing the display on their way to the Sticklebarn toilets or the upper seating area) although the public bar area also acted as a social setting in common with the settings studies in [4]. Consequently, we included code categories for passing-by, viewing & reacting, subtle interaction, direct interaction, multiple interaction and follow-up action. As noted in [4], the direct interaction code is common to the coding used in [23] and the viewing & reacting and subtle interaction codes approximately map onto the focal and peripheral awareness codes. These coding categories formed columns across the coding table to record the stages of engagement for each visiting individual or group, along with fields for comments at each stage. The coding sheet also allowed un-categorised observations to be recorded and in this respect, our approach was semi-structured. As suggested in [5], we took note of group behaviour, e.g. we noted when a group member actively beckoned or fetched other members of his or her group to look at the PARM.

In addition to updating the coding sheet, the proximity of the researchers to the PARM display enabled the researchers to hear comments made by those visitors interacting with the display and notes of comments pertinent to the interaction were made on the coding sheet.

The study took place over 3 days, 24–26 July 2017 during the school summer holidays. The observations started at noon and across the 3 days lasted 15 h. The weather was favourable for the first 2 days of the study but poor for the final day on which day the Sticklebarn was noticeably less busy.

4.2 Findings

In total, 211 visitors passed by the display and a demographic breakdown is presented in Table 2.

Table 2 Demographic breakdown of visitors passing by the display
Full size table

In describing their audience funnel framework [4], Müller et al. note that “the framework includes different (not necessarily sequential) phases, where at each transition between phases only a certain percentage of the audience can be retained.” In this observation study, we noted that of the 211 visitors that passed by the display, 167 (79%) clearly noticed it (e.g. were observed glancing at the display). The relatively high percentage of visitors noticing the display would indicate that in this particular setting, the relief surface display did not suffer from the display blindness phenomenon.

Of these 167 visitors, we observed 113 (68%) giving the display their focal attention and carrying out a subtle interaction (e.g. pausing to have an extended look at the display or gesturing towards it). Of these individuals, 78 (69%) went on to directly interact with the display (e.g. by touching it). Consequently, 47% of visitors transitioned from noticing the display to direct interaction (whilst 70% transitioned from subtle interaction to direct interaction).

Over the 3 days, a notable observation was the apparent enthusiasm shown by children for the surface model. We observed children noticing the display and eagerly pointing it out to their parents. Indeed, for children, the conversion rate for transitioning from noticing the display to direct interaction rises to 69% (whilst the conversion rate for transitioning from subtle interaction to direct interaction rises to 80%).

A common action observed in groups was for at least one member to trace a path with their finger whilst describing a route or particular PoI visited along a route. For example, on one occasion, we observed a woman explaining to a child:

“that’s stickle tarn, and that’s where we went today, that’s blea tarn, we came up here”

The woman was seen gesturing towards the corresponding PoIs on the PARM display during this explanation and in the sense, the PARM can be viewed as a useful tool for discussion that effectively supported the nature experience by providing a map view that clearly attracted the attention of the child whilst the woman gestured towards the relevant part of the PARM.

Another notable observation was an expectation for interaction capability. In more detail, when directly interacting with the relief display (e.g. touching a PoI), we observed visitors looking at the adjacent monitor apparently expecting it to update with related information. On one occasion, we observed a child playfully attempting to use gestures such as a pinch and zoom and on another, we observed and noted a female visitor saying:

“wow I like this.. If you touch it does it tell you what it is?”

In this sense, our observations revealed use that appears to conflict with the so-called interaction blindness phenomenon and this finding is discussed further in the following section.

5 Discussion

In this section, we initially consider the potential limitations of our study given the data capture method employed before going on to discuss the findings of our research within the context of existing research insights relating to public interaction with situated displays. Lastly, in this section, we discuss our current work and future plans for enhancing the PARM display to support direct interaction.

The motivating aim behind our PARM deployment and associated observation study was to gain insight into how visitors would notice and interact with the display in the social setting of Sticklebarn. In particular, we were interested in the extent to which visitors would be enticed to move between stages of interaction and the extent to which phenomena such as display blindness would be evident. In common with many of the studies that have investigated public interaction with situated displays, we have used naturalistic observation as our key method of data capture. It is important to consider the potential biases and limitations of our approach. One potential bias associated with the observation method is that of observer bias, and this was to some extent mitigated by our use of two observers agreeing on observations. In describing their view on what might be regarded as the ideal reporting of observation studies related to public display deployments and display blindness [5] includes average glance time as a metric to be measured. Whilst this metric can be obtained from analysing the video footage of a public deployment, the NT stakeholders involved in our deployment did not consider the capture of video footage in the Sticklebarn setting to be appropriate.

Another potential limitation relates to the representativeness of the visitors observed during our observation study. When considering this aspect, it is important to note that the 3 days of observation occurred during the school summer holidays (see Sect. 4.1) and consequently, the proportion of families with children was likely higher than would have been the case outside of this period. Another important issue to note is that there was no control over whether or not any of the visitors observed had already encountered the display (given that the display had already been deployed for several months before the observation period).

In considering the Sticklebarn deployment setting against that of other public display deployments, it is worth noting that Sticklebarn represents a more protected, semi-public, space than that of a city deployment for example. Consequently, some of the key external factors that challenge fully public deployments such as deliberate vandalism to a display [41] were less of a factor. The display blindness phenomenon is typically associated with city-wide deployments where members of the public might expect to be subject to displays focussed on advertisements. That said, the Sticklebarn is not a conventional NT property, where visitors come to learn about a building and its surroundings, so people would not necessarily be looking to see interpretation material. During our observations, we found that the display did not appear to suffer from the display blindness phenomena with 79% of visitors passing the display over the 3 days clearly noticing it.

It is certainly the case that a screen-only display may have struggled to attract visitor attention in the same way as the PARM display which appeared to provide the important curiosity factor as noted by [42] in motivating passers-by to take notice of the display. Indeed, this curiosity factor can be attributed to the fact that a relief model offers something different and when enhanced through projection by maps and imagery creates an unusual holographic effect that appears to be highly compelling.

Whilst the PARM display was clearly eye-catching, analysis of observations also revealed that 68% of those noticing the display moved on to some form of subtle interaction, and of those 69% transitioned to some form of direct interaction. Consequently, 47% of visitors transitioned from noticing the display to some form of direct interaction. The conversion rate for children was particularly high with 69% of those noticing the display transitioning to some form of direct interaction. When children were part of a passing group, a very common observation was for one of them to notice the display, approach it, touch it and then encourage others in the group to come across. Often, this was followed by an adult member of the group pointing things out on the model, typically including their current location followed by tracing out routes/paths with their finger. Whilst we observed this explicit ‘beckoning’ behaviour, we would differentiate this from the honey-pot effect discussed in [23] which appeared less prevalent during our observations. The final deployment location was in a relatively high traffic area (being passed by visitor’s en-route to the toilets) but was placed in an area that was not overlooked by a lot of seating. Had the placement been more overlooked then this could have resulted in people being more self-conscious about interacting with the display. This could in part explain why we did not observe the phenomenon of social embarrassment discussed in [23] and another factor could be the prevalence of young children interacting with the display.

Given the effectiveness of PARM for capturing the attention of passers-by, the capability to play media-rich sequences and the powerful frame of reference of a relief model, it offers a promising option for displaying information about a visitor’s surroundings. The requirement to provide visitors with additional information relating to the landscape around them was suggested by the Operations Manager for the National Trust who commented:

“one of the things we ask them about is do you want more … information on History .. there has been a pull away from interpretation panels but visitors are saying that it has almost gone too far.... If this is a way of potentially getting some of that information out … it’s really powerful.”

Attempting to convey information about a number of themes or places relating to the surrounding landscape using PARM can lead to long sequences of content. An ongoing research aim is to explore techniques to allow visitors to trigger small sections of content by making the display interactive. As discussed in Sect. 2.1, one of the interaction design challenges associated with digital display deployments is that of interaction blindness whereby a potential user may assume that a digital display does not support interactivity. However, our observations revealed a significant percentage of those noticing the display went on to attempt direct interaction by touching the display. The first option for interactivity to be explored therefore is directly touching the model itself.

In order to further explore the potential of supporting touch-based direct interaction in an intuitive manner, we are currently undertaking a series of pilot studies based upon the Wizard-of-Oz technique [43]. People’s apparent natural inclination to touch relief models however only works when the models are small enough to be touched. This is at odds with one of the properties of traditional relief models in that they tend to be quite large, allowing the combination of detail and broader landscape context and giving the sense of visual immersion. Larger models also encourage physical movement around them to allow the landscape to be seen from many perspectives. Any attempts to make the actual landscape surface interactive may not therefore be scalable but could be explored as an additional option for those developing interpretation strategies where space is limited and where a manageable number of recognisable PoIs are featured. We intend to explore this option by embedding a series of sensors in the surface model in a way similar to that implemented on the MIT campus map [28]. Unlike the campus model, however, the PoIs may not be discrete physical features on the model; therefore, we will experiment with alternative ways to signpost interactive areas on the model through projection as part of the Wizard-of-Oz studies.

The second option for interactivity to be explored is to use buttons around the edge of the model, either when the model itself is too large to support direct touch or when content sequences do not relate directly to discrete PoIs. This kind of interactivity is common in conventional painted relief models of the type shown in Fig. 1 although is typically used to highlight PoIs using light bulbs embedded within the model. Sometimes, this is a one-to-one mapping but occasionally, one button can illuminate a series of PoIs or a route, or occasionally where an area is to be highlighted then overhead spotlights are used. One affordance of PARM is that through projection, it can deliver a much richer level of content. Rather than being limited to highlighting PoIs in the model, it can allow more complex spatial distributions to be visualised across the whole surface, which might involve sequences of maps, lighting effects and animated features. We are therefore exploring the use of buttons to trigger sequences of projected content in situations where the narratives relate more to themes rather than particular PoIs. The setting for these investigations is a visitor centre in a nearby area of the Lake District where we are again working with National Trust staff, along with staff from the Lake District National Park Authority. As part of this development, we are planning to support more complex narratives relating to the World Heritage Site status of the Lake District. There will be more information to convey than was the case with the Langdale PARM and so the intention is to use a series of buttons in order to allow the user to initiate the various sequences that form the overall narrative. This will allow an assessment of the added value of button-based interactivity in terms of retention of interest and also engagement with the learning objectives of the display. The nature of interactivity should add a degree of playfulness, exploiting the apparent success of PARM in converting the attention of children into more direct interaction.

6 Concluding remarks

In this article, we have presented findings from an in-the-wild study of a PARM deployment in Langdale, Cumbria, co-designed with the Lake District National Trust team responsible for landscape interpretation at the ‘Sticklebarn’ site which housed the display.

The technology probe approach has been effective in field testing the PARM technology over a longitudinal period (9 months) and has demonstrated the technology to be reliable and robust to failure in the Sticklebarn setting.

Our 3-day observation study (n = 221) aimed to provide empirical understanding of how visitors would notice and interact with the PARM display. The placement of the PARM display supported a high level of visitor flow. Using the audience funnel framework, our observations paid particular attention to the extent to which visitors transitioned between the various activity zones associated with the framework. Our observations revealed that 79% of visitors noticed the PARM display and that, of these, 68% transitioned to giving the display their focal attention and that, of these, 69% transitioned to direct interaction with the display.

Our observations revealed a particularly strong level of playful engagement with children and for this group, the overall rate for transitioning from noticing the display to some form of direct interaction was 69% (as opposed to 47% for the visitor group as a whole).

In terms of supporting the nature experience, our observations revealed significant use of the model as a tool for discussion with visitors gesturing to relevant parts of the PARM display whilst explaining or commenting on PoIs visited or routes taken as part of a group discussion.

The widespread use of tracing and pointing gestures, some with an apparent expectation for the display to respond contrasts to the phenomenon of interaction blindness, as discussed in the literature.

Whilst touch-based interaction on the model could be explored for developing PoI-based You-Are-Here style displays the potential of PARM lies in the capability to deliver animated media-rich sequences that are not constrained to PoIs but relate to the whole landscape. Further work will explore this using the simple button-based interaction common with traditional relief models in visitor centres. This will be part of a series of studies aiming to explore ways of leveraging the apparent affordance of PARM displays to support direct touch-based interaction.

Our research suggests that when placed in a suitable public setting, the physical nature of PARM displays, coupled with media-rich information sequences choreographed between relief model and screen, can prove compelling to visitors, encouraging passers-by to notice and interact with the display. As such, PARM displays are a promising tangible display technology and an attractive alternative for those developing landscape interpretation strategies designed to engage visitors with the local environment.