Understanding visitor interaction with a projection augmented relief model display: insights from an in-the-wild study in the English Lake District
Projection augmented relief models (PARMs) are a promising tangible display technology for assisting users in orientating themselves within the represented landscape. The production of physical models is now easier than ever thanks to more freely available digital terrain data and 3D fabrication technology. When placed in a public setting, such as a visitor centre, the physical nature of such displays coupled with digital surface projection is compelling, enticing passers-by to notice and interact playfully with the display. In this article, we describe findings from our in-the-wild study of a PARM display designed to engage and orient visitors to the rural landscape of a remote valley in the heart of the English Lake District. The deployment has involved close collaboration with the Lake District National Trust, and the results of our 3-day observation study (n = 221) contribute to the growing research community seeking to explore and uncover technology designs that are both playful and unobtrusive to the nature experience. Our research also contributes to the literature on public and situated digital displays and, in particular, understandings of visitor behaviour as considered through the so-called audience funnel framework. Our observations revealed that a significant portion (79%) of visitors noticed the PARM display and that, of these, 68% transitioned to giving the display their focal attention. We also observed an apparent expectation for the PARM display to support direct tangible interaction (such as pointing gestures) which contrasts to the phenomenon of interaction blindness discussed in the literature.
KeywordsProjection augmented relief model Visitor engagement Landscape interpretation Audience funnel
In the commercial domain, and particularly the mobile app market, there are a growing number of smartphone apps that support interaction in nature, for example apps designed to support hiking in nature . Furthermore, there is a growing research community interested in investigating the potential ways that digital technology such as smartphones can be used to support users’ engagement with nature  and the tensions and challenges that arise. A key challenge for interaction design is to explore and uncover technology designs that are both playful and unobtrusive to the nature experience.
There is robust scientific and practice-based evidence that nature can contribute to addressing health and social challenges that EU citizens are facing
In this article, we contribute to this exploration by presenting findings from our in-the-wild observation study of a digital display deployment designed to support the nature experience of visitors to a valley in the heart of the English Lake District. The growing research literature focussed on the interaction design issues associated with public and situated digital displays highlights a number of key challenges and opportunities. One significant challenge is that of display blindness , a phenomenon whereby passers-by tend not to notice or give their attention to digital displays. Another is interaction blindness  in which even when noticed, a user may assume that a digital display does not support interaction. Both of these phenomena arose through researcher observations of real-world digital display deployments (and further discussion of these issues is presented in Sect. 2.1).
The digital display deployment described in this article relates to a particular type of display known as a projection augmented relief model (PARM) . These displays feature high-resolution physical relief models of landscapes enhanced through the digital projection of high resolution maps, imagery and animation. PARM displays aim to inform and educate whilst exploiting the engaging and tactile qualities of physical models. Our research aim was to observe how visitors would notice and interact with the PARM display in the particular deployment setting.
Physical models have long been an option for landscape representation in the context of planning and landscape architecture where public consultation is required . Many options for ‘3D displays’ exist , aiming to immerse the viewer in a modelled landscape, many being classed as ‘non-autostereoscopic’ in that they convey stereo effects using viewing aids such as virtual reality (VR) headsets. Where a broader landscape context needs to be communicated, for example showing patterns across a region rather than within a local scene, then first-person immersion is less appropriate and an aerial perspective can offer a more effective spatial frame of reference. Relief models have been recognised as powerful tools for giving very natural overviews of landscape for hundreds of years in contexts such as military training  and to aid orientation for visitors as with the Swiss alpine models . The production of physical models is now easier than ever thanks to more freely available digital terrain data and 3D fabrication technology so the relief model is now a viable option for certain types of landscape visualisation especially when augmented through projection as with PARM. Where landscape overviews are required, PARM therefore presents an alternative to visually immersive yet physically detached forms of visualisation like VR and as with traditional relief model displays allows for a shared experience supported by direct discussion and gesture.
A physical relief model augmented with projection could be considered a tangible display, benefitting from the generic and engaging power of solid models as seen across a range of disciplines . Solid models of all kinds offer kinaesthetic interactions that include the ability of the viewer to effortlessly change their perspective on the model to appreciate its structure in three dimensions. In the case of landscape models, this can help the viewer build up an understanding of the spatial interrelationships between geographical features, whether these are parts of a settlement or mountains and valleys without the need to interpret symbols on a map.
Physical landscape models also offer potential for direct interactions with parts of the model in reference to the features in the real world which they embody, for example by pointing or tracing. In this sense, models can act as tools for discussion about landscape but there would rarely be an expectation for any kind of output response from the input gestures. There are however types of landscape display designed to be manipulated by hand and to provide response to such inputs by projecting new surface information. The Illuminating Clay project  for example allowed a clay landscape model to be deformed through touch resulting in new contours, water flow and other surface parameters being projected onto the model in response. These and similar sand-based models allow a very direct link between input action and output response which offers users a very intuitive spatial mapping when exploring the task space .
PARM displays, as with Illuminating Clay, use projection onto landscape models, but here, the models are solid and so cannot provide the same kind of interaction outputs. In many PARM displays, the delivery of information has been achieved through a passive sequence, or organised into themed sections each triggered by interaction via buttons or a touch screen. Users do however consistently interact with the models themselves in a very tactile way either by exploring the surface texture or by making more conscious references to places of interest if part of a group. This has been particularly evident in PARM displays when the user’s current location is within the modelled area, with a typical interaction beginning with touching the model accompanied by a comment such as ‘look, we are here’.
In this research, we follow a technology probe approach  using a simple PARM display to act as a You Are Here model in quite an explicit way, exploring the user’s spatial interaction in a general sense but also to inspire ideas for more direct modes of interaction. The in-the-wild  study featured a 9-month deployment involving close collaboration with the Lake District National Trust (NT), a conservation charity responsible for managing a significant portion of the land and most of the public bridleways in the Lake District. The Lake District is located in Cumbria in the North of England and was awarded UNESCO World Heritage Site status in July 2017, with the NT as one of the key partners involved in preparing the bid.
The remainder of this article is structured as follows. Section 2 describes background and related work and includes sub-sections on PARM (Sect. 2.1), public and situated displays (Sect. 2.2) and the emerging body of research focussing on the design of digital technologies to support engagement with natural places (Sect. 2.3). Section 3 describes our design of the PARM display and the strong involvement of the NT in supporting its deployment and the co-design of content. Next, in Sect. 4, we present details of our 3-day observation study and its findings including the apparent expectation of visitors for direct interaction with the PARM. A discussion of our findings is presented in Sect. 5, and this is followed by our concluding remarks (Sect. 6).
2 Background and related work
In the following sections, we present background and related work associated with three key areas: PARM (Sect. 2.1), interaction design related to situated displays (Sect. 2.2) and finally, the role of digital technologies in supporting engagement in natural places (Sect. 2.3).
2.1 Projection augmented relief models
PARM could be considered a specific subset of the projection augmented (PA) model technique  where physical objects are dynamically textured through projection. Tangible 3D tabletop displays  represent tangible interfaces where physical objects can be moved around, triggering responses in the form of information being projected onto the objects and the table. The ‘Illuminating Clay’ project described earlier was an example of a representation of landscape being textured through projection, later developing into a tool to explore landscape analysis principles , although here, it was a malleable surface rather than a solid model. Using a similar technique, augmented reality sand boxes  have been adopted by many schools as engaging tools to support spatial literacy teaching. PARM differs from these techniques in that displays feature solid high-resolution models of real landscapes created from a range of digital terrain data. The landscape model can neither be moved nor manipulated yet acts as a tangible display that features detailed place-specific content based on maps, imagery and animation projected onto the model with ancillary information displayed on a screen.
The PARM displays discussed so far aimed to place certain objects or themes relevant to the building where the display was housed into a broader landscape context. As such, the focus was to help with visitor ‘interpretation’ within a curated cultural heritage space. The geographical area covered by the displays was large and connecting more directly with features of interest in the immediate surroundings was less of a concern.
There is great potential for PARM to operate at a more local level, promoting themes of interest that can be explored immediately outside. In this way, PARM could operate as a media-rich form of You-Are-Here map, attempting to engage people with their natural surroundings.
YAH maps are often stylised in some way, whereby a graphic visualisation is used to highlight a particular facet or facets of the terrain covered by the map (e.g. in order to highlight the locations of PoIs or the path of a walking trail). In the case of traditional YAH maps, this visualisation is static and fixed. The PARM approach allows for a series of PoIs, perhaps along a trail, to be introduced to the viewer in stages, or selectively, along with some additional information. This along with the natural frame of reference offered by the raised relief model therefore represents a powerful tool for promoting local exploration.
2.2 Public and situated displays: interaction design challenges
The Langdale PARM display can be categorised as a situated display and according to Weiser’s categorisation  of ubicomp technologies can be considered as a yard-scale display. There is a growing research literature focussed on situated displays and associated interaction design, and this understanding has clear relevance to the PARM display.
One key interaction design challenge associated with displays situated in public or semi-public settings  is the need to entice users to engage with the display. In the seminal work by Brignull and Rogers , which involved the deployment of a large situated display placed in two authentic social gatherings, the authors observed that users were reticent to interact with the display because of the potential for social embarrassment. In this same research, the authors propose the notion of activity spaces for considering the actions of users in moving from passive bystander to direct interaction with the display.
A further issue associated with public display deployments is that of display blindness  whereby potential users can fail to notice displays assuming that they are some form of digital signage associated with advertising, or they actively look at them, but choose to ignore them .
In a recent analysis of the issues around display blindness presented in , the authors note that current display blindness and engagement studies have focussed largely on traditional flat panel displays. The PARM does not take this form and consequently appears less likely to be dismissed or ignored for being advertising-based digital signage. In their work on curious objects , the authors describe their 1-day exploratory study into the potential of attaching so-called curiosity objects to interactive displays in order to mediate interaction, and this approach showed promising results in terms of reducing the blindness problem. Given the novel and salient appearance of the PARM display within the Sticklebarn setting, it is likely that the curiosity of visitors played a significant role in enticing them to engage with the PARM display.
Another interaction issue is that of interaction blindness. This refers to the problem of users not necessarily realising that a public display supports interactivity. As discussed more fully in the findings of our observation study (Sect. 4.2), we actually observed the opposite of this phenomenon with some users expecting an interaction capability, such as direct touch interaction, to be supported by the PARM display. The 3D terrain surface of the PARM appears to afford a physicality that encourages users to touch the display in order to trace contours with their fingers (as illustrated in Fig. 2.) and in this respect supports a form of tangible interaction, though no objects are moved or manipulated as such . When considering the potential of such tangible interaction in the context of public visualisation displays, Claes and Vande Moere describe how “tangible interaction can elicit different forms of engagement and generate more and deeper kinds of insights, when compared to traditional public display media” . In , the authors discuss their formative studies of a tangible map designed to serve as an interactive centrepiece within the MIT campus and which combines a tangible user interface (created with 3D-printed buildings) and dynamic spatial information displayed on a flat screen beneath the building models. In this case, the buildings represented a set of well-defined and visually distinct PoIs which had been made responsive to touch. This differs from many of the landscape surfaces used in PARM displays which represent continuously variable terrain surfaces which do not have discrete physical PoIs; therefore, any expectation of interactivity from the viewer is harder to predict.
In Sect. 5, we return to the aforementioned design challenges and insights when describing our observations of visitors engaging with the PARM display in the social space of the Sticklebarn.
2.3 Digital technologies supporting engagement in natural places
There is a growing body of research activity focussing on the potential of digital interactive technologies to support people’s engagement with natural places in an unobtrusive manner [3, 29] and to facilitate outdoor recreation  and play . One of the seminal research projects to focus on the use of digital technology to facilitate engagement with nature was the ambient wood project . Other examples include the GreenHat Mobile Augmented Reality System, supporting students in learning about biodiversity , the MobileGIS system which forms one of the case studies described in  and the research presented by  which studies the ways in which fifth-grade students interacted with nature using mobile technology during a nature hike. The aforementioned examples all utilise mobile screens but in research on so-called nature-technology hybrids , the use of such technologies is deliberately resisted because of the potential ‘captivating hold of the screen’. With a similar design goal in mind, Robinson describes his research on developing prototypes that support so-called Eyes-off interaction in . In , the authors present seven themes to support technology design in natural places in ways that ‘provoke integration of the natural and computational worlds’. One such theme is that of revealing and receding which suggests that ‘design must simultaneously fade into the background and provoke seeing natural places differently’. This theme relates strongly to the design of the Langdale PARM display which aims to provoke visitors into seeing the different facets of Langdale that are not immediately visible, such as the valley’s farming heritage and the on-going land management activities that have a direct impact on the valley’s appearance. This theme also resonates with the analysis presented in  which suggests that: ‘The proliferation of mobile apps brings into sharp relief the power of digital technologies to disrupt, and therefore reveal, aspects of our experience of the natural world’.
3 Design and installation of the PARM display and associated content
In this section, we describe our development of the PARM display and the stages of co-design relating to the overall objectives, the content and the configuration of the display. The design decisions involved in creating this technology probe are considered particularly relevant because the site for deployment was not a conventional location for displaying landscape interpretation material, and there was a collective shared interest in exploring public engagement in such a context.
3.1 Early discussions with the National Trust
“... I am developing a series of valley-based walks/itineraries that tell the World Heritage Site/cultural landscape story. Each valley is able to showcase different aspects of the WHS’ Outstanding Universal Values – Identity, Inspiration and Conservation”
During the first face-to-face meeting with the programme manager, he detailed the aims of the trust with respect to attaining World Heritage Site status and his interest in using digital technology to support valley-based walks. He had produced a document detailing one such walk (and associated itinerary) for the Langdale valley.
The authors considered the capabilities of a PARM display to be well-suited for promoting the walk from the Sticklebarn and during a later meeting, the authors offered to demonstrate how the route could be visualised using such a display. The demonstration took place at the Sticklebarn in February 2017. Note that in this article, we focus on the PARM display rather than the smartphone app which was also developed to support the walk.
Given the available spaces at Sticklebarn, it was agreed that a reasonably small footprint for a physical display would be necessary. In order to test public reaction to using this kind of display, it was decided to initially adopt a simple passive display using a relief surface model, a monitor and projector. There would be no interface to control the content which would be shown as two synchronised displays, where a projection onto the model would have accompanying content on an adjacent monitor. In terms of the model area, there was a requirement to present the detail of a walking route in the valley but also the spatial context of that route. To determine the geographical coverage required the location in the landscape of the broad features that were of interest from that walking route was located on the map and compared to the initial mock-up. The resulting coverage area for the relief surface represented a square on the ground covering just less than 4.5 km across. The combination of spatial extent and level of detail required suggested that a cartographic scale of around 1:10,000 would be suitable, with no vertical exaggeration, resulting in a relief surface model 45 cm across.
The fidelity of the projection is also important and should match or exceed that of the physical features represented on the model in order to create a holographic effect. Where the vertical extent of the model is relatively large as is the case here when modelling a small area in upland terrain, then the imagery needs to be warped to fit the physical model. This is achieved by rendering the projected images or maps from the perspective of the projector’s vantage point, using a scale model of the display in CAD.
3.2 The co-design of content for the PARM display
In order to implement projections onto the relief surface synchronised with a descriptive narrative running on an adjacent monitor, it was agreed to use PowerPoint as the authoring environment, which would also allow the stakeholder to easily modify the content.
Animated illumination effects mimicking changing sun angle.
Static illumination effects mimicking a light source on the relief model surface. A variant of this can suggest a cone of vision or ‘viewshed’ if wanting to highlight the area that can be seen from a vantage point.
Spotlight effects illuminating an area of land but with no hard boundary.
Highlighting map features where points, lines or areas are emphasised through colour, style or brightness.
Animated map features to highlight movements including progress along a route.
Animated atmospheric or environmental conditions including flooding, snowfall and cloud shadow effects.
The co-production of content began with the programme manager producing a second version of the walk comprising concise notes as to the intended messages for each slide along with some initial ideas for projection which were discussed with the authors. Examples of notes from three PoIs along the walk follow, showing summaries of the NT programme manager’s initial ideas for the intended message at each point, and the projection onto the relief surface model, with any ideas for the monitor referred to as the ‘second screen’.
3.2.1 Example 1: the Neolithic axe factory
Summary of message
There is evidence of Neolithic stone axe manufacturing from around 4000 years ago in an area up on the hillside. These high-quality axe heads were prized across the country, and they were exported from Langdale.
“At PoI, highlight the higher scree slopes where the axe factory was roughly. On 2nd screen show Neil’s reconstruction of axe?”
3.2.2 Example 2: footpath restoration
Summary of message
Towards the head of the valley, there are popular paths up to the summits of Pike of Blisco and Bow Fell. Due to the large numbers of walkers combined with weather erosion, these paths are in a terrible condition, often leaving large scars on the fellsides. The National Trust and its partners initiated a scheme to repair the paths and the adjacent land. Today, upland paths across the Lake District are maintained by the Fix the Fells partnership, with National Trust rangers and over 100 ‘lengthsmen’ volunteers and donations are welcomed.
“Show the paths on the map; 2nd screen could show before/after picture; pictures of fell top path workers.”
There was a requirement here to show many known footpaths in an accurate fashion, with the programme manager using the phrase ‘on the map’ so suggesting a mechanism to highlight this set of map features achieved by using a bright colour line symbol with a darkened aerial imagery behind (Fig. 8b).
3.2.3 Example 3: the view from Old Dungeon Ghyll hotel
Summary of message
The Old Dungeon Ghyll was originally a farm and an inn. In the early 1900s, the farm was bought by historian and academic George Trevelyan who gave it to the National Trust. Trevelyan continued buying farms as they came available, donating them to the National Trust. His aim was to conserve the valley’s farming heritage and to protect public access rights. In 1949, the old bar changed to the Climbers’ Bar and became a hub for climbing clubs from across the country. Dungeon Ghyll Force behind the inn is the setting for William Wordsworth’s pastoral poem ‘The Idle Shepherd-Boys’.
“pics of old climbers/old tavern? Highlight all the GMT bought farms in the valley. 2nd screen pics of GT and the other Trust properties?”
There are several possible themes attached to this location including farm purchases, early rock climbing and Wordsworth’s literature. It was agreed to keep it simple and have one message which was the legacy of Trevelyan’s farm purchases but even here, there were options for portraying this which were discussed at a follow-up meeting. Map symbols to highlight the farm locations would appear distorted by the often rugged nature of the relief surface so a series of soft spotlight effects were animated in a sequence to reveal the five farms within the study area (Fig. 8c).
Content for Langdale PARM at each point of interest (PoI)
Relief surface projections
Try the Langdale Trail app
Would you like to discover more about the Langdale valley?
This app-based walk will help you explore more…
Fade in route of trail
Fade in You Are Here arrow
Learn about sustainable hydro power
Illuminate Stickle Tarn, then stream, then location of hydroelectric station
Dramatic Stickle Tarn
Lighting effect around Tarn
Learn about farming heritage…
Fade in elements of land cover map
Learn about the ‘Ring Garth’, a medieval farm wall
Highlight wall. Valley bottom within wall aerial photo, area outside wall is darkened aerial image
Learn about Neolithic axe making
Camp fire illumination effect to highlight axe making area in warm light
Find out how upland paths are being restored
Path network highlighted as bright yellow routes on darkened aerial image
Discover the legacy of George Trevelyan
Stage in spotlight effect on five farms over darkened historical map
Learn about the value of hay meadows
Highlight four fields in yellow over darkened aerial image
Fly above tranquil Blea Tarn (video plays)
Illumination effect mimics area visible over Blea Tarn
Learn why planting Juniper helps.
Fuzzy torchlight effect over area of new planting
3.3 Installation of the Langdale PARM display
Some locations received quite bright sunlight through windows, so to ensure the projections were always effective, a full-sized 3000-lm projector was used. The display would be set up each day by a number of different members of the bar staff, so setup was made automatic after the projector and the PC were switched on. The display was in a busy public setting whose primary function was a pub so the whole display, including the relief surface model, needed to be robust. The display was not monitored by staff, so the projector was partially enclosed in a housing to prevent it being switched off, though during the initial installation, it became clear that this housing was getting too hot and so the display was taken away, modified and re-installed the following week. This latter point highlights another benefit of the technology probe approach, namely field testing the technology .
After several weeks, following observation of visitor flows, and in consultation with the manager of Sticklebarn, the display was placed on the ground floor as shown in Fig. 10. The display was visible to visitors as they walked through the main entrance and was en-route to the toilet, side entrance and upper level with additional seating. Adjacent to the display was a shelf unit containing leaflets and local information.
4 Observation study
In this section, we describe our observation study of the PARM display deployment which had the research aim of gaining an understanding for how visitors to Sticklebarn would notice and interact with the display. Furthermore, we wanted to investigate the extent to which phenomena such as display blindness would be evident and the extent to which visitors would be enticed to move between different activity spaces and stages of interaction according to the audience funnel framework.
In order to reduce possible observer effects, the observation was conducted in a non-intrusive manner with both authors sitting at a table situated approximately 5 m from the display as shown in Fig. 10. On this occasion, we were unable to employ video recording because the manager of the Sticklebarn was not comfortable with this and its use was not imperative for achieving the goals of the observation study.
Our use of the observation method followed a semi-structured approach with data capture supported through a coding sheet that was informed by the schemes employed by previous observation studies investigating public display use seen in [4, 23] and the analysis presented in . Our coding aligned most closely with the audience funnel model  because the PARM display was effectively deployed along a thoroughfare (with people passing the display on their way to the Sticklebarn toilets or the upper seating area) although the public bar area also acted as a social setting in common with the settings studies in . Consequently, we included code categories for passing-by, viewing & reacting, subtle interaction, direct interaction, multiple interaction and follow-up action. As noted in , the direct interaction code is common to the coding used in  and the viewing & reacting and subtle interaction codes approximately map onto the focal and peripheral awareness codes. These coding categories formed columns across the coding table to record the stages of engagement for each visiting individual or group, along with fields for comments at each stage. The coding sheet also allowed un-categorised observations to be recorded and in this respect, our approach was semi-structured. As suggested in , we took note of group behaviour, e.g. we noted when a group member actively beckoned or fetched other members of his or her group to look at the PARM.
In addition to updating the coding sheet, the proximity of the researchers to the PARM display enabled the researchers to hear comments made by those visitors interacting with the display and notes of comments pertinent to the interaction were made on the coding sheet.
The study took place over 3 days, 24–26 July 2017 during the school summer holidays. The observations started at noon and across the 3 days lasted 15 h. The weather was favourable for the first 2 days of the study but poor for the final day on which day the Sticklebarn was noticeably less busy.
Demographic breakdown of visitors passing by the display
5 to 15
16 to 35
36 to 50
In describing their audience funnel framework , Müller et al. note that “the framework includes different (not necessarily sequential) phases, where at each transition between phases only a certain percentage of the audience can be retained.” In this observation study, we noted that of the 211 visitors that passed by the display, 167 (79%) clearly noticed it (e.g. were observed glancing at the display). The relatively high percentage of visitors noticing the display would indicate that in this particular setting, the relief surface display did not suffer from the display blindness phenomenon.
Of these 167 visitors, we observed 113 (68%) giving the display their focal attention and carrying out a subtle interaction (e.g. pausing to have an extended look at the display or gesturing towards it). Of these individuals, 78 (69%) went on to directly interact with the display (e.g. by touching it). Consequently, 47% of visitors transitioned from noticing the display to direct interaction (whilst 70% transitioned from subtle interaction to direct interaction).
Over the 3 days, a notable observation was the apparent enthusiasm shown by children for the surface model. We observed children noticing the display and eagerly pointing it out to their parents. Indeed, for children, the conversion rate for transitioning from noticing the display to direct interaction rises to 69% (whilst the conversion rate for transitioning from subtle interaction to direct interaction rises to 80%).
“that’s stickle tarn, and that’s where we went today, that’s blea tarn, we came up here”
The woman was seen gesturing towards the corresponding PoIs on the PARM display during this explanation and in the sense, the PARM can be viewed as a useful tool for discussion that effectively supported the nature experience by providing a map view that clearly attracted the attention of the child whilst the woman gestured towards the relevant part of the PARM.
“wow I like this.. If you touch it does it tell you what it is?”
In this sense, our observations revealed use that appears to conflict with the so-called interaction blindness phenomenon and this finding is discussed further in the following section.
In this section, we initially consider the potential limitations of our study given the data capture method employed before going on to discuss the findings of our research within the context of existing research insights relating to public interaction with situated displays. Lastly, in this section, we discuss our current work and future plans for enhancing the PARM display to support direct interaction.
The motivating aim behind our PARM deployment and associated observation study was to gain insight into how visitors would notice and interact with the display in the social setting of Sticklebarn. In particular, we were interested in the extent to which visitors would be enticed to move between stages of interaction and the extent to which phenomena such as display blindness would be evident. In common with many of the studies that have investigated public interaction with situated displays, we have used naturalistic observation as our key method of data capture. It is important to consider the potential biases and limitations of our approach. One potential bias associated with the observation method is that of observer bias, and this was to some extent mitigated by our use of two observers agreeing on observations. In describing their view on what might be regarded as the ideal reporting of observation studies related to public display deployments and display blindness  includes average glance time as a metric to be measured. Whilst this metric can be obtained from analysing the video footage of a public deployment, the NT stakeholders involved in our deployment did not consider the capture of video footage in the Sticklebarn setting to be appropriate.
Another potential limitation relates to the representativeness of the visitors observed during our observation study. When considering this aspect, it is important to note that the 3 days of observation occurred during the school summer holidays (see Sect. 4.1) and consequently, the proportion of families with children was likely higher than would have been the case outside of this period. Another important issue to note is that there was no control over whether or not any of the visitors observed had already encountered the display (given that the display had already been deployed for several months before the observation period).
In considering the Sticklebarn deployment setting against that of other public display deployments, it is worth noting that Sticklebarn represents a more protected, semi-public, space than that of a city deployment for example. Consequently, some of the key external factors that challenge fully public deployments such as deliberate vandalism to a display  were less of a factor. The display blindness phenomenon is typically associated with city-wide deployments where members of the public might expect to be subject to displays focussed on advertisements. That said, the Sticklebarn is not a conventional NT property, where visitors come to learn about a building and its surroundings, so people would not necessarily be looking to see interpretation material. During our observations, we found that the display did not appear to suffer from the display blindness phenomena with 79% of visitors passing the display over the 3 days clearly noticing it.
It is certainly the case that a screen-only display may have struggled to attract visitor attention in the same way as the PARM display which appeared to provide the important curiosity factor as noted by  in motivating passers-by to take notice of the display. Indeed, this curiosity factor can be attributed to the fact that a relief model offers something different and when enhanced through projection by maps and imagery creates an unusual holographic effect that appears to be highly compelling.
Whilst the PARM display was clearly eye-catching, analysis of observations also revealed that 68% of those noticing the display moved on to some form of subtle interaction, and of those 69% transitioned to some form of direct interaction. Consequently, 47% of visitors transitioned from noticing the display to some form of direct interaction. The conversion rate for children was particularly high with 69% of those noticing the display transitioning to some form of direct interaction. When children were part of a passing group, a very common observation was for one of them to notice the display, approach it, touch it and then encourage others in the group to come across. Often, this was followed by an adult member of the group pointing things out on the model, typically including their current location followed by tracing out routes/paths with their finger. Whilst we observed this explicit ‘beckoning’ behaviour, we would differentiate this from the honey-pot effect discussed in  which appeared less prevalent during our observations. The final deployment location was in a relatively high traffic area (being passed by visitor’s en-route to the toilets) but was placed in an area that was not overlooked by a lot of seating. Had the placement been more overlooked then this could have resulted in people being more self-conscious about interacting with the display. This could in part explain why we did not observe the phenomenon of social embarrassment discussed in  and another factor could be the prevalence of young children interacting with the display.
“one of the things we ask them about is do you want more … information on History .. there has been a pull away from interpretation panels but visitors are saying that it has almost gone too far.... If this is a way of potentially getting some of that information out … it’s really powerful.”
Attempting to convey information about a number of themes or places relating to the surrounding landscape using PARM can lead to long sequences of content. An ongoing research aim is to explore techniques to allow visitors to trigger small sections of content by making the display interactive. As discussed in Sect. 2.1, one of the interaction design challenges associated with digital display deployments is that of interaction blindness whereby a potential user may assume that a digital display does not support interactivity. However, our observations revealed a significant percentage of those noticing the display went on to attempt direct interaction by touching the display. The first option for interactivity to be explored therefore is directly touching the model itself.
In order to further explore the potential of supporting touch-based direct interaction in an intuitive manner, we are currently undertaking a series of pilot studies based upon the Wizard-of-Oz technique . People’s apparent natural inclination to touch relief models however only works when the models are small enough to be touched. This is at odds with one of the properties of traditional relief models in that they tend to be quite large, allowing the combination of detail and broader landscape context and giving the sense of visual immersion. Larger models also encourage physical movement around them to allow the landscape to be seen from many perspectives. Any attempts to make the actual landscape surface interactive may not therefore be scalable but could be explored as an additional option for those developing interpretation strategies where space is limited and where a manageable number of recognisable PoIs are featured. We intend to explore this option by embedding a series of sensors in the surface model in a way similar to that implemented on the MIT campus map . Unlike the campus model, however, the PoIs may not be discrete physical features on the model; therefore, we will experiment with alternative ways to signpost interactive areas on the model through projection as part of the Wizard-of-Oz studies.
The second option for interactivity to be explored is to use buttons around the edge of the model, either when the model itself is too large to support direct touch or when content sequences do not relate directly to discrete PoIs. This kind of interactivity is common in conventional painted relief models of the type shown in Fig. 1 although is typically used to highlight PoIs using light bulbs embedded within the model. Sometimes, this is a one-to-one mapping but occasionally, one button can illuminate a series of PoIs or a route, or occasionally where an area is to be highlighted then overhead spotlights are used. One affordance of PARM is that through projection, it can deliver a much richer level of content. Rather than being limited to highlighting PoIs in the model, it can allow more complex spatial distributions to be visualised across the whole surface, which might involve sequences of maps, lighting effects and animated features. We are therefore exploring the use of buttons to trigger sequences of projected content in situations where the narratives relate more to themes rather than particular PoIs. The setting for these investigations is a visitor centre in a nearby area of the Lake District where we are again working with National Trust staff, along with staff from the Lake District National Park Authority. As part of this development, we are planning to support more complex narratives relating to the World Heritage Site status of the Lake District. There will be more information to convey than was the case with the Langdale PARM and so the intention is to use a series of buttons in order to allow the user to initiate the various sequences that form the overall narrative. This will allow an assessment of the added value of button-based interactivity in terms of retention of interest and also engagement with the learning objectives of the display. The nature of interactivity should add a degree of playfulness, exploiting the apparent success of PARM in converting the attention of children into more direct interaction.
6 Concluding remarks
In this article, we have presented findings from an in-the-wild study of a PARM deployment in Langdale, Cumbria, co-designed with the Lake District National Trust team responsible for landscape interpretation at the ‘Sticklebarn’ site which housed the display.
The technology probe approach has been effective in field testing the PARM technology over a longitudinal period (9 months) and has demonstrated the technology to be reliable and robust to failure in the Sticklebarn setting.
Our 3-day observation study (n = 221) aimed to provide empirical understanding of how visitors would notice and interact with the PARM display. The placement of the PARM display supported a high level of visitor flow. Using the audience funnel framework, our observations paid particular attention to the extent to which visitors transitioned between the various activity zones associated with the framework. Our observations revealed that 79% of visitors noticed the PARM display and that, of these, 68% transitioned to giving the display their focal attention and that, of these, 69% transitioned to direct interaction with the display.
Our observations revealed a particularly strong level of playful engagement with children and for this group, the overall rate for transitioning from noticing the display to some form of direct interaction was 69% (as opposed to 47% for the visitor group as a whole).
In terms of supporting the nature experience, our observations revealed significant use of the model as a tool for discussion with visitors gesturing to relevant parts of the PARM display whilst explaining or commenting on PoIs visited or routes taken as part of a group discussion.
The widespread use of tracing and pointing gestures, some with an apparent expectation for the display to respond contrasts to the phenomenon of interaction blindness, as discussed in the literature.
Whilst touch-based interaction on the model could be explored for developing PoI-based You-Are-Here style displays the potential of PARM lies in the capability to deliver animated media-rich sequences that are not constrained to PoIs but relate to the whole landscape. Further work will explore this using the simple button-based interaction common with traditional relief models in visitor centres. This will be part of a series of studies aiming to explore ways of leveraging the apparent affordance of PARM displays to support direct touch-based interaction.
Our research suggests that when placed in a suitable public setting, the physical nature of PARM displays, coupled with media-rich information sequences choreographed between relief model and screen, can prove compelling to visitors, encouraging passers-by to notice and interact with the display. As such, PARM displays are a promising tangible display technology and an attractive alternative for those developing landscape interpretation strategies designed to engage visitors with the local environment.
The authors are grateful for the time and effort put in by the team at the National Trust, Cumbria, UK, in particular Alex McCoskrie, the World Heritage Site Programme Manager. Data used to create the physical relief models shown in Figs. 3, 4, 5, 7, 8, 9, and 10 is the NEXTmap digital terrain model product from Intermap Technologies. This work was supported by the Economic and Social Research Council [grant number ES/M500598/1].
- 1.Brink P, Mutafoglu K, Schweitzer J-P, Kettunen M, Twigger-Ross C, Baker J, Kuipers Y, Emonts M, Tyrväinen L, Hujala T, Ojala A (2016) The health and social benefits of nature and biodiversity protection. A report for the European Commission (ENV.B.3/ETU/2014/0039). Institute for European Environmental Policy, LondonGoogle Scholar
- 2.Colley A, Napari E, Häkkilä J (2017) A survey of mobile apps to support hiking, The 2nd workshop on NatureCHI - Unobtrusive User Experiences with Technology in Nature, at MobileHCI '17, September 04, 2017, Vienna, AustriaGoogle Scholar
- 3.Häkkilä J, Cheverst K, Schöning J, Bidwell NJ, Robinson S, Colley A (2016) NatureCHI: unobtrusive user experiences with technology in nature. In: Proceedings of the 2016 CHI conference extended abstracts on human factors in computing systems (CHI EA ‘16). ACM, New York, pp 3574–3580Google Scholar
- 4.Müller J, Wilmsmann D, Exeler J, Buzeck M, Schmidt A, Jay T, Krüger A (2009) Display blindness: the effect of expectations on attention towards digital signage. In: Proceedings of the 7th international conference on pervasive computing (pervasive '09). Springer-Verlag, Berlin, pp 1–8Google Scholar
- 6.Priestnall G, Gardiner J, Durrant J, Goulding J (2012) Projection augmented relief models (PARM): tangible displays for geographic information In: Proceedings of Electronic Visualisation and the Arts (EVA 2012) 180–187Google Scholar
- 7.Cureton P (2016) Strategies for landscape representation: digital and analogue techniques. Routledge, London, New YorkGoogle Scholar
- 10.Niederoest J (2002) Landscape as a historical object: 3D reconstruction and evaluation of a relief model from the 18th century. Int Arch Photogramm Remote Sens Spat Inf Sci XXXIV, Part 5/W3Google Scholar
- 11.de Chadarevian S, Hopwood N (2004) Models: the third dimension of science. Stanford University Press 464 ppGoogle Scholar
- 12.Hornecker E, Buur J (2006) Getting a grip on tangible interaction: a framework on physical space and social interaction. In: Proceedings of ACM CHI 2006 conference on human factors in computing systems 2006. pp 437–446Google Scholar
- 13.Piper B, Ratti C, Ishii H (2002) Illuminating clay: a 3-D tangible interface for landscape analysis, Proceedings of the conference on human factors in computing systems (CHI ‘02), MinneapolisGoogle Scholar
- 15.Hutchinson H, Mackay W, Westerlund B, Bederson BB, Druin A, Plaisant C, Beaudouin-Lafon M, Conversy S, Evans H, Hansen H, Roussel N, Eiderbäck B (2003) Technology probes: inspiring design for and with families. In Proc. CHI 2003, ACM, 17–24Google Scholar
- 16.Rogers Y (2011) Interaction design gone wild: striving for wild theory, interactions. 18(4). https://doi.org/10.1145/1978822.1978834
- 17.Dalsgaard P, Halskov K (2011) 3D projection on physical objects: design insights from five real life cases. In Proc CHI. ACM Press, pp 1041–1050Google Scholar
- 22.Agamanolis S (2003) Designing displays for human connectedness. In: O'Hara K, Perry M, Churchill E, Russell D (eds) Public and situated displays: social and interactional aspects of shared display technologies. Kluwer, pp 309–334Google Scholar
- 23.Brignull H, Rogers Y (2003) Enticing people to interact with large public displays in public spaces. In In proc. of interact ‘03, IOS Press, 17–24Google Scholar
- 24.Dalton N, Collins E, Marshall P (2015) Display blindness? Looking again at the visibility of situated displays using eye tracking. In Proceedings of CHI ‘15, ACMGoogle Scholar
- 25.Houben S, Weichel C (2013) Overcoming interaction blindness through curiosity objects. In Proc. CHI'13 EA, ACM, pp 1539–1544Google Scholar
- 26.Hornecker E (2011) The role of physicality in tangible and embodied interactions. interactions 18, 2 (March 2011), 19–23Google Scholar
- 28.Walker W, Sung H, Ong CK, Casalegno F (2017) Exploring spatial meaning with a tangible map. In: Proceedings of the 6th ACM international symposium on pervasive displays (PerDis ‘17). ACM, New York Article 6, 8 pagesGoogle Scholar
- 30.Jones MD, Anderson Z, Häkkilä J, Cheverst K, Daiber F (2018) HCI outdoors: understanding human-computer interaction in outdoor recreation. In: Extended abstracts of the 2018 CHI conference on human factors in computing systems (CHI EA ‘18). ACM, New York, Paper W12, 8 pages. https://doi.org/10.1145/3170427.3170624 CrossRefGoogle Scholar
- 32.Rogers Y, Price S, Fitzpatrick G, Fleck R, Harris E, Smith H, Randell C, Muller H, O’Malley C, Stanton D, Thompson M, Weal M (2004) Ambient Wood: designing new forms of digital augmentation for learning outdoors. In: Proceedings of interaction design and children. ACM Press, pp 3–10Google Scholar
- 33.Kimiko R, Agogino A (2013) Off the paved paths: exploring nature with a mobile augmented reality learning tool. Int J Mob Hum Comput Interact 5:2Google Scholar
- 34.Adams A, Fitzgerald E, Priestnall G (2013) Of catwalk technologies and boundary creatures. ACM Trans Comput Hum Interact 20(3) Article 15, 34 pagesGoogle Scholar
- 36.Edwards L, Coulton P, Chiasson M (2015) Walking off the garden path: a design journey. In: Proceedings of the 2nd Biennial Research Through Design Conference, 25-27 March 2015, Cambridge, UK, Article 11Google Scholar
- 37.Robinson S (2012) Eyes-off physically grounded mobile interaction. PhD thesis. Accessed: http://cs.swan.ac.uk/~cssimonr/projects/phd/Simon-Robinson-PhDThesis.pdf
- 39.Coyne R (2014) Nature vs. smartphones. interactions 21, 5 (September 2014), 24–31Google Scholar
- 40.Cheverst K, Gregory I, Turner H (2016) Encouraging visitor engagement and reflection with the landscape of the English Lake District: exploring the potential of locative media. Retrieved from the NatureCHI 2016 workshop: http://www.naturechi.net/2016/p11-Cheverst.pdf
- 41.Mäkelä V, Sharma S, Hakulinen J, Heimonen T, Turunen M (2017) Challenges in public display deployments: a taxonomy of external factors. In: Proceedings of the 2017 CHI conference on human factors in computing systems (CHI '17). ACM, New York, pp 3426–3475. https://doi.org/10.1145/3025453.3025798 CrossRefGoogle Scholar
- 43.Kelley JF (1985) CAL – a natural language program developed with the OZ paradigm: implications for supercomputing systems. First international conference on supercomputing systems (St. Petersburg, Florida, 16–20 December 1985), ACM, New York, pp 238–248Google Scholar
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.