Cartographies of Rest: The Spectral Envelope of Vigilance

For Hubbub, the anthropologist Josh Berson designed a field study with the aim of integrating physiological, phenomenological and environmental measures with place and time, and of giving study participants a chance to reflect on the experience of monitoring their movements. Here, Josh outlines how treating rest as a phenomenon with a spectral envelope emerged as a natural way to emphasize the non-linear interaction of all these factors.

the energy of the vibration gives us the sound's loudness. But of course there is more to sound than pitch and loudness. We have no trouble distinguishing two sounds, equal in pitch and loudness, made by two different voices or instruments. The same is true of another oscillatory phenomenon, social rhythms of activity. To describe either phenomenon we need more than the fundamental frequency. We need the spectral envelope.
In signals processing (the analysis of the waveform properties of a signal, that is, any phenomenon that varies in intensity in time or space, such as ambient sound pressure, respiratory volume in the lungs or the illuminance in different parts of a photograph) spectral envelope refers to characterizations of variation in the mix of frequencies present in a signal over time. Measures of a signal's spectral envelope offer a concise way to describe features of the signal that emerge from the interaction of frequency, energy and time. For acoustic signals -sounds -spectral envelope is associated with those perceptual features we subsume under timbre: all that is neither pitch nor loudness, things like 'brightness', 'hoarseness', 'smoothness' and 'roughness'. Timbre is that dimension of sound as a sensory phenomenon that allows us to distinguish among different voices and instruments making what, from the perspective of loudness and pitch contour, is the same sound. i As Hubbub unfolded, I came to think of the task of Cartographies of Rest, the strand of the project that Dimitri Nieuwenhuizen of LUSTlab 1 and I led, as one of characterizing the spectral envelope of vigilance.
On the surface, our self-imposed brief could not have been simpler: Build an instrument to measure social rhythms of rest and its opposites in the wild. Make it participatory. Make it scalable. But responding to the brief turned out to be tricky. First, we were trying to bring together two strands of thinking with a long history of mutual distrust in anthropology: a 'constructivist' view that prioritizes the juxtaposition of incompatible ways of making sense of the world and a 'positivist' view that stresses the reduction of research hypotheses to a manageable number of measures that can be sampled efficiently in a population. Second, we were trying to bring together incompatible approaches to research: that of social science, which stresses the contextualizability, reliability and validity of measures with reference to past work (and which demands rigorous safeguards against the risk of adverse outcomes for respondents/participants) and that of human-centred design, which favours rapid iterability and user engagement over the rigour of results. Third, we were trying to operationalize a feature i See Chap. 18. of human life -restedness -that had not been adequately operationalized before, and do it in a way that would allow us to draw in a broad range of participants who would stick with us for weeks or months at a time. Finally, we were determined to break out of the paradigm that has defined mobile sampling studies up till now. This paradigm is one in which what is of interest is what is happening inside participants' bodies (as the ever-expanding number of self-tracking apps makes clear) -never what is happening in their environment or at the interface between body and world. This last challenge is where the concept of a spectral envelope comes in.
As the project went on, we came to focus more and more on sound, but my use of 'spectral envelope' here is not just about sound. Rather, what I'm trying to capture is that temporal and spatial periodicities in the collective habits of activity and mood in a population represent an outcome of how individuals interact, in dyads and groups, with one another and with a host of environmental cues, many anthropogenic, most coming under one of three broad headings: sound, light and movement. We can no more describe social rhythms of activity and rest in terms of a linear summation of individual behaviour than we can describe the timbre of a busy urban scene -recall Shoreditch and Shibuya -as a linear summation of the fundamental frequencies of the individual voices and vehicles within hearing. Instead we must look for more supple ways of capturing the many-to-many interactions among moving bodies and between moving bodies and world. ii

Vigilance
Why vigilance, as opposed, say, to activity, alertness, wakefulness, awareness or arousal? I started thinking of the phenomenon we were trying to track as vigilance in 2013, when I began turning a set of lectures I'd been giving on 'circadian selfhood' into what became the chapter on 'clocks' in my monograph Computable Bodies. 2 An article comparing the phenomenology and pathophysiology of mania and attention deficit hyperactivity 3 led me to a case report from the mid-1980s describing 'disturbed vigilance in mania'. 4 The report details two cases in which an individual with a history of episodic mania was admitted to hospital showing classic signs of mania -motor agitation, expansive mood and insomnia. In both cases, within minutes of eye closure, electroencephalography (EEG) showed 'sleep spindles', the characteristic brain electrical marker of the early stages of sleep. Observations of this type, difficult to reconcile with the phenomenology of mania, went largely ii See Chaps. 4, 6, 7, 12. unappreciated for decades but have lately been rediscovered. The hyperactivity and sensation-seeking characteristic of mania and attention deficit hyperactivity disorder (ADHD) represent, Hegerl and colleagues hypothesize, an effort on the part of the individual to create a stimulating environment to counteract the dangerous implications of depressed endogenous vigilancethat is, the dangers of not being aware of your environment.
Harvey, in her review of the role of circadian rhythm anomalies in mood disorders, cites the same case report and stresses the need to see circadian regulatory dysfunction as a potential cause, rather than simply a consequence, of mood dysregulation. 5 That is, disruptions in rhythms of rest and activity are not simply a symptom of an underlying mood disorder: these disruptions may be the underlying phenomenon. The recent dramatic rise in the incidence of recurring non-episodic mood disturbances, especially in young people, 6 may represent, in part, an outcome of dramatic changes in our social and material environment that make it difficult to establish a robust alternation of activity and rest. The timbre of our activity rhythms is getting noisier.
I came to feel that 'activity', 'arousal' and 'alertness' did not go far enough in describing what we were trying to capture in Cartographies of Rest. Our target was not simply motoricity or activity in the sense of 'activity tracker', the increasingly popular accelerometer-based devices that have become a metonym for the migration of self-tracking technologies out of the psychiatric clinic and into the realm of consumer wellness. 7 Rather, what we were after was the relationship between motoricity and attention. This is what I mean by vigilance.
In other words, we needed to decompose rest into at least two components. More than that, we needed to find ways of sampling these components that would allow the most diverse range of people possible -diverse in background and life history and diverse in attentional-motoric styleto provide data targeting roughly the same underlying phenomenon while at the same time allowing them to rely on their own, mutually divergent rubrics for assessing what that phenomenon entails in terms of bodily experience. 'Activity' in the sense of 'activity tracker' means something like 'whole-body motoric tonus'. This was way too coarse-grained to use as a proxy for 'activity' in the sense of 'rest-activity rhythms'.
Vigilance comes with its own conceptual baggage. The term has another, more common, sense: vigilance in the sense of preparedness or threat readiness, 8 an attunement to demands for attention and social synchronization in our perisomatic space. The two senses -preparedness and attentional-motoric tone -are related; for instance, a surplus of threat in one's environment will tend to produce a heightened attentionalmotoric tone, not just in the moment but over time. Another way to put it would be to say that state vigilance -vigilance in the momentcontributes, over time, to heightened trait vigilance.
It is difficult to operationalize phenomena such as vigilance in a manner sufficiently supple for anthropological purposes, more difficult still to study them in the wild and to formulate a comparative framework, that is, to imagine a continuum of differentiated registers of motoricity and attention. 9 Ideally, you'd plot the spectral envelope of vigilance in a population rather than simply measure changes in individual activity level over time.
This was exactly what I wanted Cartographies of Rest to do.

'near SenSing' and a PerSPectiVal View of Urban SPace
In our first pilot study, in June 2015, we sought to integrate standard physiological measures of restedness into our sampling platform. We started with heart rate variability, since reduced heart rate variability represents a widely validated measure of stress and fatigue, and a number of vendors have brought to market inexpensive, consumer-grade devices, most using a bracelet form factor, that promise to measure heart rate using ulnar photoplethysmography -optical sensing of dilation of an artery in the wristwith strong correlation with the results you'd get using a chest strap. Heart rate bracelets proved a dead end for a bunch of reasons. For one, the wearer's movements distort the signal, and smoothing the signal to filter out motion artefacts makes the data worthless. Once the data have been smoothed, it's only valid down to a resolution of about one minute, and this is as fine-grained as device vendors will let you see. But to compute heart rate variability you need instantaneous resolution, that is, the time between successive heartbeats. A more serious problem was that self-tracking measures focus exclusively on events unfolding inside users' bodies. If you rely on these measures, you implicitly adopt a model in which the key determinants of well-being are things internal to the individual. But we wanted to incorporate social and environmental measures into our model. More than that, we wanted to make the research process itself participatory.
Participants in our first pilot -nine design students at the University of the Arts Utrecht -said that what they'd really like to have had was an opportunity to annotate their responses with audiovisual data. So in the second pilot, in November 2015 in Berlin, we included just such a feature, asking participants to include one photo and one audio sample every time Handset views from the Cartographies of Rest app. Once a day, users receive a notification asking them to comment on their state of being. To respond, they trace a path through a two-dimensional 'mood space'. The mood space is reliable at the individual level but provides some play for individual differences in the phenomenal coding, the 'feel', of physiological processes. While the user responds to the prompt, we record background sound. Users can review their own data and see how their data fall in the broader panel of users. Future iterations will include groups and channels that allow users to view data from just a subset of the total population of users.
we polled them. The audio samples in particular turned out to be surprisingly varied and revealing.
By pairing self-report data (see Fig. 11.1) with ambient environmental data, we could reorient mobile sampling methods away from their fixation on the autonomous body and toward a richer picture of how states of being are made through the ongoing encounter of bodies with other bodies and their shared environment. That is, we could start to treat our participant-users as social and spatio-temporal anchor points from which to look out into the world. I came to think of the technologies we were experimenting with not as technologies of self-tracking but of near sensing, by analogy with remote sensing. 10 Once you break out of the mindset that mobile sampling is fundamentally about individual participants, higher moments in the signal that emerge from interactions among participants and between participants and their environment become more salient -that is, it becomes more natural to view the research question not in terms of the sum of individual participants' fundamental frequencies (of activity or mood or vigilance or whatever other measure) but in terms of the spectral envelope of the entire panel of participants taken as a unit.
In Haruki Murakami's Hear the Wind Sing, first published in 1979, the anonymous narrator pursues an experiment in self-awareness remarkably similar to those of contemporary self-trackers. He rapidly becomes disillusioned: I believed in all seriousness that by converting my life into numbers I might be able to get through to people. That having something to communicate could stand as proof I really existed. Of course, no one had the slightest interest in how many cigarettes I had smoked, or the number of stairs I had climbed, or the size of my penis. When I realized this, I lost my raison d'être and became utterly alone. 11 The 'proof' to others that we really exist, the substance of our presence as social beings, resides not in our habits as individuals but in our collective life. To measure rest or any other dimension of well-being, we need to look beyond the skin envelope of the individual body.