Introduction

Recent technological advances, especially relatively low-cost electroencephalography (EEG) data acquisition equipment and open-source analytical tools, make possible the adoption of EEG outside its traditional clinical or neuroscience context into new domains of research. This paper argues that EEG is a complicated but essential addition to our present toolkit for the understanding of how people operate within their environments and, in turn, how their environments affect them. As emphasised in the editorial of a recent special issue on applications of mobile EEG in the International Journal of Psychophysiology: “the new generation of mobile EEG devices available soon will enable us to monitor human brain activity in action, in real-life environments. We firmly believe that this will help to boost the external validity of psychophysiology research, and uncover new areas of application” (Vos and Debener 2014, p.2). The aim of this paper is to introduce mobile EEG and how it is already being applied in various spatial questions: spatial cognition, architecture, urban design, planning, environmental psychology and spatial analysis.

We focus our discussion on the study of processes that are related with urban behaviour and, in particular, the understanding of human movement in the city. The present arsenal of qualitative and quantitative methods for the study of urban movement involves a wide range of methods: from GPS tracking data, to observation studies, surveys or self-reports, small-scale experiments and shadowing individuals, analysis of video and CCTV, social media content (e.g., from Twitter), telecommunications metadata and more. Using such data sources, we can now observe behaviour at large scales, with sample sizes that far exceed the limits of any targeted study. Analysing the movement of people in the city, can offer various insights, for example about how the city is used or the spatiotemporal movement of different groups. These are relevant to traditional questions of transport, planning and design, but also as part of various smart-city efforts, which posit urban sensing as a shift to more efficient management of existing resources and infrastructure (cf Kitchin 2014; Townsent 2013). These news kinds of data and data analysis methods, have wider applications to the understanding the cognitive mechanisms underlying urban movement itself (Manley et al. 2015), what kinds of environments encourage walking (Middleton 2011), or the health benefits of cycling (Woodcock et al. 2014), with the implications for behaviour change, advocacy, design and policy to create better urban environments.

Unravelling our Relationship with the Environment

How does a stroll on a busy street differ from a walk in the park? How do we know what direction to take towards a destination? What are the psychological influences of our everyday environments, from at home or office, or the streets of our commute? Environmental psychophysiology studies the relationship and the transactions between humans and their environments; how, for example, the presence of green space, the amounts of noise or air pollution, the design of spaces, may affect levels of health, stress hormones, concentration and attention, wellbeing or productivity. A variety of behavioural and self-reported measures, such as surveys, can be employed to study people’s experiences, cognitive and psychological processes, but there are phenomena which are not directly accessible to the researchers.

However, psychological and cognitive processes may be manifested in the brain or body (for example, the responses of the autonomic nervous system), and increasingly, this is measurable. According to Cacioppo et al. (2000) the basis of environmental psychophysiology is precisely that “psychological events are assumed to have a physiological referent; that is, the mind is viewed as having a physical substrate” (Cacioppo et al. 2000:12). For example, with the aim of assessing the impact of environmental stressors, such as noise, we can ask people to self-report their perception of noise, evaluate their cognitive abilities (e.g., respond to a working memory task) or capture biomarkers of arousal and stress such as cortisol and skin conductance. Parsons and Tassinary highlight that “the psychophysiological approach complements other research approaches, and that complement is especially important when the psychological processes of interest are not completely available to or accurately represented by conscious recollection and behavioural observation” (Parsons and Tassinary 2002:174). These methods are not mutually exclusive and form a set of complementary methods that can be applied together.

In this context, EEG, a measure of brain function and activity, as well as the responses of the autonomic nervous system (heart rate, skin conductivity, endocrinological levels) are studied as a window to cognitive and mental states or related psychological phenomena or to study the effects of these phenomena on physiology.

The paper is organised into three sections. Not assuming familiarity with the topic, the first section is intended as an orientation and introduces the basic principles of EEG in a historical, technological and technical context, including a note on data acquisition and analysis. Second, we present key applications of EEG to ‘spatial questions’; for example, to studying the neural basis of spatial cognition, or the psychological effects of environments. Finally, we present selected exploratory research projects using EEG with the experience of space in real-world contexts, and introduce some important methodological considerations.

What is EEG?

EEG measures the electrical signals produced by the post-synaptic activity of large neuronal assemblies firing in coherence from locations in the higher layers of the cortex (for a comprehensive introduction see Buzsáki 2006). The power of EEG signals ranges around few microvolts - μV (Misulis 2007; da Silva and F 2009). EEG usually refers to signals measured by placing electrodes on the scalp (therefore named scalp-EEG), but EEG signals can also be recorded by placing electrodes under the skull (on the surface of the cortex, hence electrocorticography or ECoG) or by inserting very thin electrodes deep in the brain (intracranial EEG or iEEG). ECoG and iEEG are rarely used in humans (only as part of brain surgery), but are frequently used in animal neuroscience studies that we will refer later. In contrast, scalp-EEG—or simply EEG—is a safe, non-invasive neuroimaging method, which is also relatively comfortable for participants.

EEG signals are dynamic and non-stationary and one way to analyse is according to their frequency and power, in several rhythms or bands: alpha (8–13 Hz), beta (13–30 Hz), gamma (30+ Hz), delta (0–4 Hz), theta (4–8 Hz). Increased power in each of these bands is commonly associated with different brain states: e.g., alpha activity increases when subjects’ eyes are closed or when they are relaxed, beta increases with alertness or cognitive demands, delta is associated with deep sleep and theta is associated with drowsiness (Misulis 2007). Activity in these frequency bands can also be task-related, for example, as we will discuss, theta oscillations are also associated with navigational tasks.

Signal Acquisition and Analysis

An EEG system consists of a number of electrodes, ranging from 3 up to 256 depending on aims of study, which are attached to the scalp spread over the brain areas under examination (e.g., visual or prefrontal cortex, temporal or parietal lobes). Two of the electrodes act as a reference, and EEG is measured as the voltage difference between each electrode and the references. Ideally, reference electrodes are positioned on sites far from brain activity, such as the nose, the ear lobe or the mastoid process, while additional electrodes are sometimes positioned near the eyes to measure the ocular muscle signals. The placement of electrodes on the head usually follows the International 10–20 system (with variations for higher density of electrodes), a standardised positioning system internationally agreed in 1957 to ensure the consistency of placement regardless of anatomical differences in head size or shape (Jasper 1958). Good conductivity (i.e., low impedance) between the scalp and the electrode is achieved using a conductive medium (usually conductive gels), while new systems sometimes use wet (saline solution) or entirely dry electrodes to make devices more user friendly and target commercial applications. EEG systems operate in very high sampling rates (128–2000 Hz) for two reasons: first to capture different frequency bands satisfying the Nyqvist theorem (a signal must be captured at a sampling frequency at least twice the signal under investigation), and second, to register fast neural phenomena that occur only for a fraction of second after a stimulus (e.g., under 200 milliseconds).

EEG analysis comprises of a set of signal processing methods that aim to distinguish noise (extraneous signal) from the oscillations reflecting cognitive function. Electrical noise, also referred as artifacts, can be of biological origin, for example due to stronger electrical signals elicited by muscle movement due to blinks, eye- or head-movements. Non-biological artifacts are due to the movement of electrodes on the scalp, or other sources of electrical noise, such as interference from the mains or other stray electromagnetic signals. For the analysis of EEG signals, artifacts have to be filtered (removed or corrected), then the signal is split in segments of equal duration, called epochs, and then epochs are compared in terms of frequency and power over time across different experimental conditions (e.g., different kinds of stimuli).

There are many signal processing techniques to evaluate EEG, relative to the various experimental paradigms. For example, a commonly used paradigm is the event related potential (ERP), which involves the analysis of signals derived during a time locked event (e.g., presentation of an image) and the comparison of signal changes between different conditions. Other methods regard the synchronisation and desynchronisation of the signals across different locations, the clustering of signals etc. The aim of these analyses is firstly to observe a difference in the signal that correlates with a particular experiment condition, stimulus or behaviour, and also to estimate the most likely origin of this signal.

Brain imaging involves methods such as fMRI (functional Magnetic Resonance Imaging), MEG (Magneto-encephalography) or PET (Positron Emission Tomography) and others, each employed appropriately according to the brain area and neural phenomenon investigated. For example, while EEG has excellent temporal resolution of less than 1 millisecond, it is usually restricted to the study of neural activity from areas close to the surface of the skull (the cerebral cortex)Footnote 1 (da Silva and F 2009), although novel 3d source-reconstruction techniques can partially address this issue. fMRI on the other hand, has the spatial resolution to study systems situated deep in the brain, such as the hippocampus or the amygdala, in a non-invasive fashion. However, fMRI or MEG equipment is bulky, some of it very noisy and imposes posture constraints for participants (e.g., fMRI requires subjects to lie still and horizontal for the duration of the experiment). EEG is less restrictive, it is lightweight, it can be wireless, and with new analytical tools becomes increasingly tolerant to head and body movements, even allowing participants to walk freely.

From Clinical Diagnosis to Consumer Electronics

The first human EEG was recorded in 1929 by the neurophysiologist Hans Berger, who was studying epilepsy (Swartz and Goldensohn 1998). Until the previous decade, EEG was primarily used in a clinical context to observe patients’ brain function, identify abnormalities or diagnose epilepsy (Gilliam et al. 1999). Traditional EEG systems required large equipment and complicated cable arrangements, therefore restricting its application beyond the laboratory. In mobile EEG, the signal amplifier is small (approximately the size of a deck of cards), located near the head, and transfers data wirelessly to a recording or monitoring device, e.g., a laptop. This eliminates the long electrode leads which were a source of complexity and signal noise, and participants can be involved either static tasks (seated, screen-based) or active tasks (walking, moving) that are closer to natural behaviour.

The first mobile (ambulatory) EEG equipment was developed to monitor patients exiting the hospital (e.g., after an epileptic seizure) and already in the 1980s 72 or even 96-h continuous mobile EEG recordings were possible using magnetic tape-recorders (Askamp et al. 2014). Research-grade mobile EEG, with a high number of electrodes and high sampling rates, have become available from most major manufacturers, and have spurred new applications and increasingly mobile research experiments. Commercial mobile EEG devices are also increasingly available, intended for the consumer and gaming industry, and with a cost at a fraction of research equipment, enable experimental applications beyond the laboratory. Usually commercial mobile EEG are more limited in terms of number of electrodes, targeting only brain areas that are of interest to applications such as gaming or neurofeedback.

Several studies have compared commercial EEG against higher-grade devices, to establish their advantages and limitations and if they are suitable for research purposes. Duvinage et al. (2013) compared Emotiv EPOC, a popular device, against higher-grade devices, in terms of signal quality, performance in standard tests such as auditory ERP (Debener et al. 2012; Duvinage et al. 2013; Vos and Debener 2014) or visual tasks (Longo et al. 2014). The results suggest that while research-grade systems are superior, Emotiv’s accuracy is sufficient for several kinds of experiments. Further, Debener et al’s study (2012) suggests that a few modifications on the electrodes (as in Fig. 1) can increase signal quality and overall stability, especially for mobile experiments.

Fig. 1
figure 1

Participant during a mobile EEG experiment, wearing an EEG headset and carrying a backpack with the recording equipment

Full size image

In parallel, the development of software tools has increased the available recording systems, from computers to smartphones and tablets, and a variety of operating systems. The Smartphone Brain Scanner project has one of the first to integrate signal acquisition and analysis in an Android application (Stopczynski et al. 2014) while new devices now come with cross-platform capabilities and software tools (SDKs), to be discussed in the last section. In terms of EEG signal analysis there is a variety of peer-reviewed and open-source software to analyse data after the experiment (i.e., offline), most notably EEGLab, a plugin for Matlab with a very active community (Delorme and Makeig 2004) as well as proprietary software.

EEG signal processing in real-time is also feasible, partly as a result of research on Brain Computer Interfaces (BCI). BCI refers to “communication system[s] in which messages or commands that an individual sends to the external world do not pass through the brain’s normal output pathways of peripheral nerves and muscles” (Wolpaw et al. 2002, p. 769). Interest in this field comes from efforts for the restoration of function for patients with neuromuscular disorders, where a BCI system can analyse EEG data in real-time to gauge the user’s intent and trigger an adequate response from the system. As these systems become more robust, BCI “will leave the laboratory and show robust performance in real-life environments, such as at patient bedside, at patient home environments for long-term monitoring, or as a neuro-rehabilitation training tool, for prosthesis and wheelchair control, or for the monitoring of mental states and attention of healthy individuals at various workplaces” (Vos and Debener 2014, p.1). BCI is relevant for spatial research too, as it provides tools to infer mental states or the user’s intent in real-life situations, giving users new kinds agency over their environment, as it is discussed later.

A simple kind of BCI is applied in neurofeedback applications, in which raw EEG signals are classified into mental and emotional states to control some feedback towards the user. Most commercial EEG devices come with some kind of classification of arousal or attention under various names (e.g., Emotiv provides ‘excitement’, ‘engagement’ and other metrics). Although the exact EEG classification algorithms are often not disclosed or peer-reviewed due to their status as industrial intellectual property, these metrics could make analysis more accessible to users without a neuroscience background. For example, studies have used Emotiv EPOC’s Affectiv metrics, to register complex cognitive processes such as insight (Cernea et al. 2011) and some of the studies discussed later have found these metrics useful for psychological assessment of environmental experiences (Aspinall et al. 2013), discrimination of environmental images (Roe et al. 2013), and other fields such as experimental art, market research or What place for EEG?” computer games. Nevertheless, there is scope to develop peer-reviewed frameworks for the classification of EEG measured in the field, in order to transform mobile EEG from a laboratory to fieldwork device for design, planning and environmental psychology.

What Place for EEG?

This section provides an overview of how EEG can be employed to explore fundamental questions about spatial behaviour, to understand the neural correlates of navigation, assess the interaction the psychological effects of environments or even create environments that respond to our mental states and intentions.

The Neural Basis of Spatial Cognition

Navigating the city on the way home, reading the map if disoriented, or remembering where we left the keys are only three examples of our ability to learn the locations of objects, predict their position from a different perspective, or flexibly reason about the optimal way to get to a location. This ability to think and reason about space is referred to as Spatial Cognition and involves “perceiving spatial information from multiple sensory cues, and maintaining spatial representations in short- and long-term memory, and using and manipulating these representations to guide navigational behaviour” (Wolbers and Hegarty 2010). Spatial cognition abilities vary between individuals (Hegarty et al. 2002), but also between different scales of space (Montello 1993) and here we focus on what Montello (1993) calls the ‘environmental scale’: large-scale environments, such as buildings and cities, that cannot be observed from a single location. Understanding spatial cognition abilities of people has applications in cartography, the prediction of spatial behaviour, wayfinding assistance (from signage to assistive systems), route-choice (Manley et al. 2015), and estimation of activity spaces or choice of transport mode (Garling 1989; Golledge 1999; Kitchin and Blades 2002).

There is extensive research about the underlying neural systems and mechanisms that enable spatial cognition, or what explains differences observed between individuals. One particularly important brain area is the hippocampal formation and the hippocampus – a brain structure located deep in the brain in the Medial Temporal Lobe. The hippocampus is critical for the formation of new memories and creating mental representations of space. Studies in animals (usually rats or rodents) have identified groups of ‘spatial’ cells, which contribute to an awareness of position, location and orientation. Using single-unit recordings of firing neurons in the rat hippocampus, O’Keefe and Dostrovsky (1971) discovered a kind of neurons that ‘fire’ only when the animal is at a particular location of its environment—and suitably coined the term ‘place cells’. Other neurons, called grid cells, fire in regular and repetitive patterns as the animal moves to ‘provide a spatial metric’ contributing to awareness of self-motion, while others are sensitive to the animals orientation (for a recent review see Spiers and Barry 2014). These groups of cells operate in tandem in a process described by Burgess as “multiple parallel systems using representations with different frames of reference, processing different aspects of the spatial scene, using different learning rules, and supported by a corresponding set of neural systems” (Burgess 2008). Information from large populations of all these distinct kinds of cells is integrated to create coherent representations of the environment, where the animal or person is, what direction she is facing and where their target is.

Studies in humans have corroborated the importance of the hippocampus in spatial memory. For example, it is required to memorise and recall large street networks and without it navigation is severely impaired or impossible (Spiers and Maguire 2007). The volume of the hippocampus correlates with better performance when learning a new environment (Maguire et al. 2003; Schinazi et al. 2013) or resolving a demanding spatial reasoning task, such as finding the shortest path or taking a new shortcut (Howard et al. 2014). Conversely, the opposite cognitive process, such as the effortless retracing of highly familiar route, does not rely on the hippocampus and activates a neighbouring structure instead, the Caudate Nucleus (Hartley et al. 2003). In humans, neural activity in Medial Temporal Lobe (MTL) systems, such as the hippocampus, is usually studied in a non-invasive way (usually fMRI) and the data are compared with data from direct-cell recordings on animals, as strong similarities across many species allow for comparisons (Burgess 2008).

EEG During Navigation

EEG cannot yet be used to study the deep Medial Temporal Lobe structures, but has been used to understand aspects of spatial cognition. Theta frequency oscillations have been related with encoding and retrieval of spatial information in the animal hippocampus (O’Keefe and Burgess 1999) or, in humans, theta activity under the frontal midline area—above the forehead—has been found to reflect working memory load working, episodic memory (Mitchell et al. 2008), or navigation task difficulty and performance (Bischof and Boulanger 2003; Kober and Neuper 2011). For example, the difference between navigating actively and following passively is reflected in different EEG patterns over the frontal midline areas, both in the frequency of short bursts of theta activity (Bischof and Boulanger 2003) and an increase in the power of theta frequency oscillations (Havranek et al. 2012). These increases in theta activity reflect the state of active navigation, which requires use of mental representations of space to choose the correct path, but also reflect the complexity of the environment and occur near decision points when the navigator has to decide whether they are making a correct turn, for example—often in situations “when new hallways come into view” (Bischof and Boulanger 2003). The power of theta oscillations during a navigational task correlate with participants’ accuracy (Kober and Neuper 2011), suggesting that EEG could be used, for example, as a measure of the difficulty a person is facing in navigation tasks, or to understand differences in performance between different groups.

Navigation strategies may rely only on landmarks (e.g., “turn left after the bakery”); on knowing the route; or on having a flexible, map-like representation, which allows the navigator to find novel shortcuts. The first strategy is often associated with poor navigation performance and getting lost, as it is easy to become disoriented without a landmark in view. EEG studies reveal different oscillation patterns when landmarks of interest (targets) are in view, versus irrelevant ones (Weidemann et al. 2009). In another study, increased EEG responses in the view of landmarks were associated with taking longer paths in the virtual environment—showing poorer navigational performance (Kober and Neuper 2011). Understanding how individuals process and use landmarks is important to understand how mental representations of space may use landmarks as a network ‘anchors’ (e.g., Couclelis et al. 1987; Wiener and Mallot 2003) or, in a planning context, how landmarks contribute to ‘legible’ urban forms (Lynch 1960).

The Importance of Mobile and Situated Experiments

Largely, spatial cognition studies in humans dissociate physical movement from navigation, since fMRI is not portable and studies are necessarily conducted in Virtual Reality (VR). Participants explore or navigate in ‘environmental surrogates’, created upon artificial or real environments (video walkthroughs, 3D simulations). The advantage is that cognitive processes can be isolated, the environment can be manipulated according to the experiment design (e.g., constrain areas, regulate speed, situate landmarks) and, confounding factors can be controlled with precision; for example, the experimenter can limit variation in environmental attributes, such as building form, colour, texture or street activity and also luminosity, weather, noise or traffic. This facilitates replication of the same conditions between experiments and subjects.

The VR approach has been criticised on the basis that exploring, learning and navigating in a virtual environment is radically different to the conditions of real environment (Taube et al. 2013). Neural activity between moving and static conditions seems to differ: in animal studies, theta oscillations are attenuated during VR, and Taube et al. (2013) conclude that “it is not known whether the amplitude and characteristics of this theta [oscillations] are the same as during active locomotion” (Taube et al. 2013). This may be due to major perceptual differences because in VR the person is immobile. Therefore the sense of movement relies only on visual flow from the screen, with a lack of self-motion information from vestibular or proprioceptive cues (perception of acceleration or of moving one’s own body). Comparing VR and active locomotion, Waller and Greenauer (2007) found minor differences in overall performance, but VR participants were less accurate in estimating bearing to locations. This reinforces the idea that the integration of turning cues relies on body-based senses beyond the purely visual. Although humans and animals are capable of navigating in virtual reality almost as well as in the real world, we do not know if the involvement of systems or neural activity is the same. Mobile EEG makes studies possible that join free movement, perception and cognition in the laboratory (Gramann et al. 2014), and it enables cognitive and navigational experiments entirely in the real world.

Summarising, together with other neuroimaging methods, mobile EEG is used to investigate how neural processes of spatial cognition, movement and navigation unfold together. This allows researchers to compare brain activity observed in laboratory and naturalistic experiments, studying how navigators perform in the multisensory conditions of a real building or city. Experiments in naturalistic environments, however, are not without methodological issues, which will be outlined in the last section.

Understanding the Psychological Effect of Environments

Walking on a busy street or walking through a park can be two very different experiences; being in different environments may have different psychological outcomes. The psychologist Roger Ulrich asked whether “encounters with some types of common environments have restorative effects, while other everyday settings hamper or even work against recovery?” (Ulrich et al. 1991). In environmental psychology, ‘restorative theory’ posits that environments predominantly composed of vegetation (and/or water)—broadly termed ‘natural’—have positive psychological outcomes on people, increasing mental capacity, reducing and ‘immunising’ against fatigue or stress. Several explanations have been proposed to explain the restoration of attention or stress, on the basis of reduced attentional demands, aesthetic preference or evolutionary dispositions (Ulrich et al. 1991; Parsons and Tassinary 2002; Roe and Aspinall 2011). A variety of methods have been used to explore the restorative theory hypothesis, from self-reports to the measurement of psychophysiological indicators including heart-rate (HR), skin conductance (SC), EEG, cortisol levels, and even crowdsourcing (with the Mappiness app, as discussed in Mackerron and Mourato 2013). One of the early studies to measure EEG of participants watching images of natural, urban or water dominated scenes, found that natural scenes elicit increased power at the alpha frequency band, which is considered a sign of relaxation (Ulrich 1981). Results from more studies using other psychophysiological indicators (heart rate, skin conductance etc.) agree that people generally recuperate faster from stressors and/or are immunised against future stressor tasks if they attend to stimuli dominated by vegetation (Parsons et al. 1998; Ulrich et al. 1991). These studies were based in the laboratory, ‘simulating’ experience with video walkthroughs or image slideshows.

To explore these questions in a more naturalistic way and increase the ecological validity of the findings (see Parsons et al. 1998 for a discussion), studies can also be carried in situ, using various methods to assess the psychological state of individuals and the effects of different environments on it. For example, self-reports have been used to assess mood and restoration following walks in different environments (Roe and Aspinall 2011), while another indicator of stress is the levels of salivary cortisol. Ward Thomson and colleagues (2013) measured changes in diurnal levels of salivary cortisol concentration as a psychological indicator of stress, and established a positive effect of living in proximity to green space of residents in several deprived communities in Scotland. A major disadvantage of this method is the requirement of frequent saliva samples from participants. In addition, salivary cortisol levels change slowly through the day and as a result are “not a very good measure for capturing stress reduction in short-time nature exposure experiments for all types of participants” (Tyrväinen et al. 2014). Due to the low frequency of sampling (e.g., before and after a walk), this method is not sensitive to short-term aspects of spatial experience, such as the effect of different streets, crossing junctions, or the effects of the social and architectural environment.

Direct measurement of psychological indicators, combined with the tracking of movements in various environments offers the ability to assess the psychological implications of different environments. The first to combine GPS tracking and psychological measurement (skin conductance) in situ was the new-media artist Christian Nold (2009), in a series of participatory art experiments in various cities to produce an ‘emotional cartography’ or biomapping. A pioneer in this sense, Nold followed a qualitative approach, using geo-located arousal levels to record and visualise the experience of urban environments (e.g., stress near intersections)—but also as a prompt, asking users to discuss and explain what triggered different responses; a kind of quantified ethnography.

While Nold had to build an ad-hoc device, today skin conductance measuring devices are integrated in wristbands and ‘smart-watches’. Although such scientific equipment is still expensive, commercial devices that incorporate heart-rate sensors and other indicators are increasingly affordable, capitalising on ‘quantified-self’ movement and a broader interest in measuring one’s own patterns. This trend may mean that geo-located psychophysiological data becomes increasingly available to researchers, opening a window into the relationship between activity, mobility and mental state.

In this context, although mobile EEG remains more intrusive than wearing a wristband, it will further enhance our ability to register and monitor mental and cognitive states in natural conditions. Motivated by the affordances of consumer EEG, researchers have explored whether they are informative in the context of environmental preferences, validated in situated, naturalistic studies (Mavros et al. 2012; Roe et al. 2013; Aspinall et al. 2013). In the laboratory study, participants looked at 15 images of various environments and the EEG-derived metrics of Emotiv’s ‘Affectiv Suite’ registered different responses evoked by natural and urban environments (Roe et al. 2013). For the naturalistic experiment, 12 participants wearing an EEG headset were asked to walk along a designated route that took them through calm, low-traffic, busy heavy-traffic and green (park) urban parts of central Edinburgh (the paradigm is described in Mavros et al. 2012). This pilot found evidence of lower frustration, engagement and arousal, and higher meditation when moving into the green space zone—and higher engagement when moving out of it (Aspinall et al. 2013).

Albeit a small sample, the results of these pilot studies were in line with theories of restorative influence of natural environment and demonstrated the potential of mobile EEG and emotion classification to tap into the emotional and cognitive states of people who are engaged in naturalistic tasks, like walking in urban environments. Emotional responses to natural or urban environments were different between the passive experiment (watching images on a screen in the laboratory) and the active experiment (walking through outdoor environments). This observation prompts more research on the use of virtual reality or environmental surrogates versus real-world environments, to study psychological experiences of space. Both of these studies were experimental, establishing protocols for further research in this domain, but effects need to be tested in larger samples, in different environmental contexts and with different groups of people. This will allow us to understand the psychological effects of environments in a more readily generalizable way to inform design, planning, and architecture.

Mood, Mobility and Place (MMP), is a project that continues EEG research into the restorative qualities of natural versus urban environments, with a particular focus on older people, aged 65+. MMP is a three-year research project, funded by UK’s Engineering and Physical Sciences Research Council (EPSRC), that investigates the relationship between the quality of the design of places and the effect they have on older people’s outdoor mobility patterns. The project “builds on evidence that how we experience environments influences our mood and, in turn, our willingness to be active. Mood affects people’s judgments and actions. Well-designed places with good ambience are more likely to engage us and restore our ability to stay alert and be active”.Footnote 2 MMP involves several methodologies, including participatory design, exploring architectural and urban design solutions for the living/home environment and also the use of mobile EEG to study the effects of mobility and exposure (through walking in urban environments) for older people.

This raises the question of how the ideas of restorative environments can link with urban design and policy. Roe and Aspinall (2011) compared the outcome of one-hour walks in a rural environment between people with good or poor mental health, and after controlling for factors like walking patterns and general health, found the walk to be particularly beneficial for those with poor mental health. As they emphasize, such findings are important because, even though the benefits of exercise against depression are recognised, “there is little evidence to guide doctors on what types of locations might maximise the mental health benefits of exercise” (Roe and Aspinall 2011, p. 112). In other words, studies that can provide evidence—and data—on the psychological effects of environments, can enhance our ability to recommend and implement changes. Another example is a recent study on the health benefits and risks of urban cycling in London. Woodcock et al. (2014) combined data about air quality, physical activity rates, accident rates and other health measures in a single model, and argue that after controlling for risk factors, cycling is still beneficial. This study would not have been possible without data about public health, air pollution in different parts and streets of London, and estimates of street usage derived from London’s cycle-hire infrastructure. These two studies illustrate how targeted studies and relevant sources of data, transform the ability to assess the role environments play in mental or physical health. The set of urban design interventions aiming to make cities more pedestrian friendly and ‘walkable’ (Gehl 2013), require an understanding of multiple factors of pedestrian experience.

Thus, in an urban design context, mobile EEG could be used to understand how individuals experience urban environments, aiding the design process by indicating to architects, urban designers and planners how spaces function affectively (i.e., the effect of the constructed environment on mood). According to the MMP project, the ability of mobile EEG to unveil brain processes is regarded as a “potential tool for evaluating environmental design interventions”. Another approach is to apply mobile EEG to assess how spaces function or where intervention is needed to improve the quality of the urban environment and infrastructure.

This is a novel field of experimentation for architects and planners. Collins et al. (2014) explored these questions as part of experimental urban design workshops for students and citizen scientists at Columbia University’s Graduate School of Architecture and the Van Alen Institute in New York. Working with citizen groups that are often underrepresented in the public debate could further raise awareness and encourage more planning efforts. Our project was a collaboration with the Guide Dogs for the Blind Association (Guide Dogs) and the Future Cities Catapult, as part of the Cities Unlocked project,Footnote 3 to conduct a pilot study in the use of mobile EEG to better understand how individuals with visual impairments experience pedestrian mobility in their everyday life (Mavros et al. 2015). Preliminary findings have suggested differences between the emotional experiences of sighted and visually impaired participants, and suggest that EEG can be used as a diagnostic tool to localise areas that impact individuals in a negative way.

Our current research further explores the application of mobile EEG to explore the urban experiences of different users, study the interactions between pedestrians’ emotional state and environmental factors, and investigate how mental states influence spatial decision-making. A recent pilot study explored whether taking quiet, low-traffic streets or major streets, busy with shops, pedestrian and traffic has a different effect on pedestrians. In particular, the study looked at the interaction between pedestrians’ emotional state, as an effect of environmental factors such as air-quality and air-pollution (Freudenthal 2015) or ambient sound and noise-pollution (Puustinen-Hopper 2015). Finally, following this previous work, our current research projects investigate the interaction between the emotional state of individuals with their environmental perception and the influence of the environment on their spatial decision-making process. Through these different strands of research we have aimed to resolve the technical issues behind the concurrent measurement of neural, behavioural and environmental data, and provide new insights on how cities influence the health, experience and behaviour of people.

Closing, mobile EEG, and the other psychophysiological indicators, are tools to study perception and cognition in natural conditions, assess the psychological effects of environments, to answer questions about mental health in the context of urban mobility, and inform relevant debates—from care to wellbeing to the design of urban infrastructure.

Applications

Assessing Mobility, Modelling Transport

Could our understanding of urban phenomena be enhanced by information about the neural underpinning of the cognitive processes necessary for navigation, or the psychological effects of mobility? Several studies suggest that to understand urban movement and route choice, it is important to study the interaction between the cognitive mechanisms underlying navigation, the psychological state of pedestrians and the effect of the urban experience. For example, although it is often assumed that people take the shortest path, evidence points to the opposite. People use a variety of criteria during their route-choice (Golledge 1995), in a spatial decision-making process and the route choice of pedestrians, cyclists and drivers is situated, opportunistic and involves heuristics as well as cognitive biases. Heuristics and route-choice criteria may aim to minimise the distance to destination, or balance factors such as safety, the pleasantness or aesthetic experience of the routes. The complexity of routes (number of turns), the bearing to the destination (Conroy Dalton 2003; Turner 2009) or the prior knowledge of locations and areas (Manley et al. 2015; Wiener and Mallot 2003) also influence which routes are finally chosen. Often this process is not conscious. Hölscher et al. (2011), asked participants “to describe a route for oneself, or to describe a route for an addressee” and then walk it, and found that participants deviated from their own plans, changing route strategies during the walk opportunistically. As Jennie Middleton concludes in her study of everyday walking behaviours, “the routinized and embodied aspects of experience that cannot be reduced to ‘rational’ intention. This is significant in policy terms as an area might be considered more ‘walkable’ if a pedestrian is able to walk on ‘autopilot’ and the flow of their movement is uninterrupted by an awareness of their corporeal planes of experience” (Middleton 2011, p. 591). Foltête and Piombini (2007, 2010) also found consistent deviations from the shortestInterest, excitement or quietness can be important factors for walkers; for example, a street lined with shops creates a different experience than a long wall. Analysis of Minicab routes from London (Manley et al. 2015) provides a large-scale demonstration that shortest path cannot explain how people actually navigate.

A major challenge is how to capture and evaluate those environmental factors that have an impact pedestrian experience. Noise, air pollution or traffic levels can be measured using sensors, but others like safety or aesthetic quality can be harder to quantify. One such approach is crowdsourcing, asking users to provide input or classify urban environments, to explore whether stated preferences for environments is linked with how cities operate or to improve routing algorithms. Place Pulse (Salesses et al. 2013) and Urban gems (Quercia et al. 2014) asked users to classify the attractiveness of images, the former to compare and rank how cities perform and the latter to create a routing algorithm that takes the user through more scenic areas of cities. The combination of mobile EEG with established psychophysiological, and qualitative methods can be used to study people’s experience and behaviour. In a recent pilot by Skroumpelou et al. (2015), mobile EEG was used to explore how the affective qualities of the urban environment influence the perception of distance between two locations, which depends on knowledge of the area but may also be biased by other factors such as attention and emotion. New direct measures, such as EEG, will provide better understanding of cognitive processes at play during navigation will lead to better models of mobility and of the psychological experience evoked by different environments and conditions.

Towards Responsive Environments

How could our environment respond, adapt or anticipate some of our needs? In an architectural context, we can imagine spaces that reconfigure automatically according to the mental, emotional or physical state of their users. Recent kinetic architectural installations harness brain oscillations to actuate architectural mechanisms and reconfigure spaces. A mobile EEG system means that users can freely move and explore the space, while the space adapts and responds to their state, in a kind of ambient or spatial feedback. The project ‘Cerebral Hut’ by the Turkish architect Güvenç Ozel that was presented at the Istanbul Design Biennial,Footnote 4 responds to the user by changing its colour and size of the space. ‘Furl’, a project by Bijing Zhang & Francois Mangion at the Bartlett School of Architecture,Footnote 5 combined EEG and robotic structures that control the spatial components, with a vision “that our environments begin moving and responding to our very thoughts” (Magnion 2014). In a similar context, exploring how digital tools can be used to mediate empathy as part of the project CEDE (Creating and Exploring Digital Empathy), we have designed a system to control the ambient lighting of a room implicitly, depending on the user’s emotional state, measured with an Emotiv EEG paired through our software with the Phillips HUE lights (Mavros 2014). The choice of colours was designed to reflect distinct emotional states, based on James Russell’s circumplex theory of affect (Russell, 1980). Recently, Phillips announced a partnership with Emotiv, to develop home automation and assistive applications based on Emotiv’s BCI products, to give “more independence to patients with amyotrophic lateral sclerosis (ALS)”, including tools to control TV displays, ambient lighting and Philips own Medical Alert System. In this way, the environment could become sentient, anticipating or reconfiguring itself in response to the user’s needs, intentions or emotional states.

While such applications of EEG are still at a stage of experimentation, such case studies highlight firstly the progress in the ease of use of commercial EEG systems but also the possibilities for architectural exploration. The spatial qualities of the immediate environment can be attuned to the user needs (sensing) and adjust (respond) according to a preprogramed scheme. Such frameworks raise questions about agency and intentionality—would we actually desire to inhabit a space that mutates constantly following all the subtle variation in our brain states? On the other hand, such applications of the technology can be of particular interest to particular user groups, from education, using multisensory environments for science dissemination, to healthcare, for example responsive lighting or space could be used for various patient groups, to enrich or enable different ways of interaction with the environment.

Methods

For the closing section of this paper, we outline key methodologies, as well as data collection challenges currently faced in mobile EEG research. Experimental protocols have to be adapted for the uncontrolled (and uncontrollable) conditions of real-world environments and mobile experiments face particular challenges—for example, to establish the accurate timing when a participant responds to a set of stimuli or environments.

Experiment Designs

During navigation experiments participants are asked to move from an origin to a destination, simulating naturally occurring conditions. Depending on their aim, experiment designs may seek to control the amount of exposure to an environment or, manipulate the wayfinding or decision-making processes of navigation. In Wiener et al.’s (2009) taxonomy, wayfinding tasks can be further classified according to several parameters, including prior familiarity with the environment, use of navigation aids (maps or signage) or having a target destination or not. The experiment route can be predetermined, devised to pass through different environments and locations of interest, such as landmarks, junctions, urban areas, parks or other. According to Wiener et al.’s (2009) taxonomy this is as a path following paradigm and is appropriate when exposure to the same environment is needed, and allows data analysis in a between-subjects design, as all participants go through the same environments. For example, studies have examined how spatial learning occurs (Ishikawa and Montello 2006), compared how learning differs depending on input from the senses (Waller and Greenauer 2007), or the psychological outcomes of different urban environments (Roe and Aspinall 2011; Aspinall et al. 2013).

Navigation studies, whether in ‘real’ environments or virtual reality, give participants agency over their routes, to replicate the cognitive processes of natural behaviour, wayfinding and spatial decision-making. For example, the ‘situated planning’ paradigm discussed earlier (Hölscher et al. 2011) involves a naturalistic task, as participants were asked to walk towards a non-visible destination in the city of Freiburg but were free to decide, change and refine their exact route. According to the same taxonomy, this could be described as directed wayfinding with path planning, since the destination provides a direction in a familiar environment, but individuals are free to plan their paths. This approach allows researchers to examine the wayfinding heuristics as well as the influence of different environmental variables.

Mixed designs combining elements of the above two methods have also been used to elicit or study people’s navigational abilities. Schinazi et al. (2013) directed participants through two predefined routes and asked them to identify novel shortcuts, to corroborate that the size of the hippocampus correlates with spatial learning. In the Soho navigation neuroimaging study by Howard et al. (2014), participants watched a video-walkthrough in Soho (London), which followed a set of predetermined routes, but participants were prompted to ‘think’ (plan) their routes towards various destinations during the experiment, as if they were, in fact, navigating themselves. This approach provided a tightly controlled route with predetermined events (deviations, blocked streets etc.) while tapping into the cognitive processes of spatial reasoning in a way that is similar to a real environment. In addition, it had the benefit of replicating environmental conditions and navigational demands across participants to compare differences in their responses. Variations of these three paradigms, together with appropriate performance and experience measures, can be used to tap into the emotional experiences as well as the cognitive processes during real-world experiments.

Mobility and Synchronisation

Finally, an important aspect of experiments is the precise and accurate synchronisation of neural activity, stimuli, and external events. An experiment involves intentionally occurring events, such as the presentation of instructions or stimuli, recording of participant responses and behavioural data (gestures, eye-tracking or other), which must be time-locked with neural recordings. This way, researchers can be confident whether the fast-changing oscillations are triggered by external or experiment-related events.

Meanwhile, non-controlled –and possibly confounding– factors have to be registered as well. Even a short walk in the city involves traffic, crossing junctions, avoiding obstacles, negotiating crowds and other incidents, like loud noises or accidental stumbles, which may also influence the participants’ behaviour or emotional state. These need to be taken into account later, during data analysis, to distinguish, subset or exclude sections of the data in a meaningful manner, for example excluding verbal interactions or reactions to noise. In a mobile context, location is also an important parameter, and for mobile experiments GPS records are included in the data collection. As neural and psychophysiological data are captured continuously, researchers need to register precise and accurate timestamps for the beginning and the end of the experiment, interactions, interruptions and any other incidents.

In laboratory experiments, researchers usually choose specialised proprietary software or develop bespoke software (usually MATLAB or Python), and open-source software tools have been developed to handle precise time-synchronisation, such as the Lab Streaming Layer library, which allows the combination of data streams from many devices concurrently.Footnote 6 Mobile EEG experiments have an extra layer of complexity, as the procedure has to be adapted for moving participants, data collection is relayed to lightweight laptops or tablets, and smartphones are used for the task/stimulus presentation using wireless connections. So far researchers address these issues ad-hoc, developing several recent open-source projects. The SmartphoneBrainscanner Android platform (Stopczynski et al. 2014) is developed to work with the Emotiv EPOC hardware and allows real-time data acquisition, filtering and visualisation of brain activity on an off-the-shelf Android tablet.

As part of our recent research, we have developed mobile applications (apps) for the popular Android platform, to geo-annotate behavioural observations, and deliver instructions or wayfinding aids (maps) to participants in a controlled and synchronised way. The Soho Navigation App, consisting of one Android OS and one Windows OS application, was developed in order to handle a complex and tightly controlled navigation experiment in the ‘wild’ (Mavros et al. 2014), and it enables the precise replication of a previous fMRI study by Howard et al. (2014). It provides the functionality to control events (providing instructions, presenting stimuli and tasks, and registering participant responses) using a standard smartphone and tablet. The novelty is that it relies on the experimenter to control the experimental flow, and send instructions (trigger events) to the participant, as GPS within city areas is not always reliable to estimate participant location with accuracy. In order to capture behavioural observations that can be time locked with the EEG, Skroumpelou (2014) developed a custom application for the Android platform, that allows the experimenter to geo-annotate the experiment with various preconfigured events, such as when the participants starts, stops, talks or other, with a precision of milliseconds. The ‘Routes’ app, allows the presentation and usage monitoring of a map and wayfinding instructions, providing a log with the participants path, as well as all their interactions with the map (pan, zoom) and their duration. The ‘Presenter’ app allows the delivery of experiment instructions, integrating presenting questions and storing responses, collecting response times along with location data.

All of the above apps are open-source and available to other researchers to use and expand.Footnote 7 Mobile neural and behavioural experiments have the barrier of a complexity borne of using a variety of hardware and software tools. To address this challenge, researchers have combined commercial and ad-hoc solutions with bespoke software, which tap in the rapidly changing capabilities of mobile technologies.

Discussion

This paper has provided a review of what mobile EEG is, how it is used to understand spatial cognition and the psychological effects of space and even dynamically transform it, as well as the basic research principles employed in such studies. Mobile EEG is a research methodology that requires tightly controlled experiments and complicated analytical tools, but it is increasingly used outside of a clinical or research context to monitor brain and cognition in real world. Combined with neuroimaging studies in the laboratory it contributes to our understanding of the neural processes that enable spatial perception and cognition. Further, mobile EEG is being used to unravel the psychological transactions between people and the environment and gauge the effects of urban and natural environments on the emotional state of pedestrians. Such approaches seek to understand how the urban environment influences our wellbeing and how better urban design could mitigate negative emotional experiences, or even evoke positive ones.

In this context, mobile EEG could be integrated in the research on cities as a tool to understand or assess the impact of environments on individuals, and target interventions. Different studies have already tapped into the affordances of this method to explore how the type and quality of urban environments influence particular user groups, such as older or visually impaired people. Insights gained from such studies could reinforce debates on cities and the built environment, which seek to understand how to create places that encourage mobility, empower citizens according to their own needs, and support their wellbeing. In this sense, EEG, a non-invasive technique of measuring the brain activity of individuals directly, could support a user-centric, inclusive approach to understanding life in cities.

Our current research aims to develop the tools and paradigms for the use of mobile EEG in an urban context, to understand cognitive and psychological aspect of urban behaviour. Current experimental work involves studies on the emotional experience of different groups of pedestrian (sighted and visually impaired), the impact of environmental factors, such as noise and air-pollution in pedestrian comfort or how pedestrian decision-making unfolds in the city.