Data Work: How Energy Advisors and Clients Make IoT Data Accountable

2.1 Energy, sustainability and sensor data

Our work stands apart from the mainstream of energy-related work in HCI, which has largely focused on Energy Consumption Feedback (ECF) to raise awareness and encourage people to change their behaviour (DiSalvo et al. 2010; Pierce and Paulos 2012). ECF has been criticised for ignoring the situated practices in which energy consumption is embedded (Strengers 2011), and this provides the premise to ‘go and look’ at such practices that motivates our own work. It resonates too with recent work that has called for “opportunities to study people’s practices that include the everyday use of IoT technologies” (Robertson and Wagner 2015). Coupled to this, advances in computing and sensor technology have led to the development of novel applications that enable, for example, the measurement of air quality (Jiang et al. 2011; Kim and Paulos 2010), occupancy (Scott et al. 2011), and CO2 data (Jacobs et al. 2013). The same advances have enabled us to prototype our own interactive sensor-based system. A common research focus has also been the design and evaluation of interactive systems to help make sense of energy data. In our design, we have been inspired by prior work that has studied visualisations, annotations and other means to inspect and make sense of sensor data (Costanza et al. 2012).

Less prevalent, but equally as informative is research that pays attention to collaborative energy-related practices. Dillahunt’s studies of low-income rented properties has shed light on social issues that prevent energy-related improvements, for example, such as lack of control and ownership (Dillahunt et al. 2009), conflicts between landlords and tenants (Dillahunt et al. 2010), and lack of connectedness in the community (Dillahunt and Mankoff 2014). Studying a workplace setting, Bedwell et al. (2016) have also highlighted the ways in which employees collaboratively manage energy consumption. Our work is also informed by and speaks to research concerned with the relationship between sensor data and its intelligibility in a domestic context (e.g., Chetty et al. 2010; Dong et al. 2015; Pousman et al. 2008). Complementing this, Human-Data Interaction has recently been proposed as an emerging field that acknowledges “the inherently social and relational character of data” (Crabtree and Mortier 2015). Our fieldwork pays attention to the various forms of situated and practical reasoning that people collaboratively apply when articulating sensor data, such as reasoning about place, time, people, practices and events (Tolmie et al. 2016).

2.2 Work-practice

Our research has a particular focus on work-practice (Button and Harper 1995) and draws on fieldwork to inform and shape systems design. Work-practice studies are traditionally associated with the workplace and paid labour. However, as Crabtree et al. (2009) point out, work-practice is a generic feature of human interaction and collaboration wherever it occurs. Work-practice spans and blurs traditional boundaries and lends itself well to the study of advisors’ work, which takes place not only in offices but in clients’ homes, and is not-for-profit in nature. Related work in non-profit workplace settings has, for example, examined information management (Merkel et al. 2007), coordination and awareness (Stoll et al. 2010), participatory design with community groups (Merkel et al. 2004), fundraising (Goecks et al. 2008), and volunteer coordination (Voida et al. 2012).

In this paper we draw on previous ethnomethodological studies of the work-practices implicated in the conduct of not-for-profit energy advice-giving work (Fischer et al. 2014, 2016) to shape the design of an IoT system that supports interaction and collaboration between energy advisors and their clients. The system has been extended, redeployed in client’s homes, and subject to further field study, the results of which are presented in this paper. In this respect our work is also related to technology deployments in the home which, as Tolmie and Crabtree (2008) point out, is often oriented to by household members as something done to them rather than done for them. Our work seeks to exploit technology to deliver beneficial outcomes for clients through supporting professional practice. In doing so it trades on and further unpacks the inherently collaborative character of the work that advice-giving turns upon.

2.3 Articulation work

The collaborative work of energy advice-giving turns upon the ‘articulation’ of sensor data (Fischer et al. 2016). Articulation work is foundational to CSCW and has its origins, as Schmidt and Bannon (1992) note, in sociology and the interactionist studies of work done by Anselm Strauss (1985). Strauss recognised that collaborative action involves ‘a supra type of work’, which Schmidt and Bannon characterised as the ‘overhead cost’ of collaboration. The overhead cost consists of making work or action in the round accountable to participants. Without this it is impossible for actors to ‘mesh’ their actions together and thus pull off the collaborative endeavour they are engaged in. Importantly, as Strauss made perspicuous, this is often provided for through the construction of ‘accountability systems’. An accountability system may be a simple paper form or a complex computational system. Whatever the case, articulation work orients us to understanding how the collaborative action and interaction implicated in technology use is made accountable to the parties involved in doing it, which in turn provides insights for the development of collaborative systems. Our work thus seeks to understand how the IoT is articulated and made accountable in the interactions between advisors and their clients.

3.1 Energy advice and in-home visits

The energy advisors involved in our research are employees of the Centre for Sustainable Energy (CSE), a not-for-profit charity based in Bristol, UK. Practically, the project aimed at leveraging the IoT to support and enhance the provision of in-home energy advice to their clients. In-home visits involve the most vulnerable of CSE’s clients and include sick children, the elderly, disabled, and infirm. This cohort is routinely affected by compound issues to do with education, employment, personal finances, health and bodily ability. The winter months can be particularly problematic, when the proportion of (already low) income needed to be spent on fuel to keep warm rises. To complicate matters, vulnerable people often live in rented housing in poor condition, which they lack the funds to improve. The energy advisors’ work involves diagnosing the causes of high bills and health risks (e.g., damp and mould), recommending material and behavioural improvements, and reporting to third parties to make the case for improvements on their client’s behalf (e.g. landlords, councils, and energy suppliers).

3.2 Formative ethnographic findings

An earlier ethnographic study of advisors’ work practices (Fischer et al. 2014) informed the development of an initial ‘seed prototype’, a previous version of the IoT system presented in this paper. The purpose of the study was to identify opportunities for technology support, in particular for sensors to be installed in client’s homes and digital representations of sensor data to be provided to the advisors to help identify the causes of problems and improve their ability to tailor advice to clients. With respect to sensors, it was found that electricity and natural gas sensors would be insufficient for these purposes. Sensing issues such as dampness, mould, and cold require ambient environmental data, such as temperature and humidity sensing. For example, low temperatures and high levels of humidity may lead to mould growth and may affect health.

Prior work also showed the need for graphical visualisations that advisors could show to clients. Simple line charts were deemed the preferred type in design workshops with advisors. In addition, prior findings suggested that CSE’s clients have lower access to broadband and digital devices, in line with statistics that show that 42% of low-income households in the UK do not use the Internet (Dutton and Blank 2013). As a result, we initially experimented with a self-contained (3G mobile network-based) infrastructure. However, this proved unreliable and we opted to make broadband access a requirement for participation in the study following advice from CSE that they are seeing more and more client households equipped with broadband.

The initial seed prototype allowed us to place a simple sensor kit, which packaged a temperature, humidity and light sensor in a single networked device - in client’s homes and to furnish advisors with data ‘print outs’ prior to in-home visits. This enabled the advisors to develop an initial understanding of the client’s problems and their potential causes, and to identify energy-related issues and topics for discussion during in-home visits. Ethnographic study of the seed prototype in use provided detailed insight into the collaborative work involved in articulating the data visualisations generated by the sensor kit. This ‘data work’ involved advisors and clients working together to ‘unpack’ the indexical character of simple line charts – i.e., making data visualisations accountable to local activities and events. In turn, this work provided the basis for advisor and client to formulate situationally appropriate remedial actions (Fischer et al. 2016).

Deployment and study of the seed prototype gave rise to a number of new design requirements: a) the system should support multiple temperature and humidity sensors, and enable data collection from multiple rooms; b) the system should incorporate outdoor temperature in order to disambiguate indoor temperature fluctuations, and CO2 in order to disambiguate occupancy; c) the system should enable flexible data visualisation across multiple data sources, including filtering, highlighting, and zooming in on interesting periods and sections of the data; d) the system should convert raw electricity and gas measurements for selected periods of time into monetary values; e) the system should allow advisors to annotate the data in order to support pre-home visit ‘rehearsal’ of the data and in situ ‘performance’ of data-driven advice.

3.3 Co-designing the CharIoT system

In response to the requirements that emerged from prior work (Fischer et al. 2016), researchers and advisors undertook a co-design process to create an interactive IoT system through a series of iterative prototyping workshops. This process began with a half-day design workshop, in which the requirements gathered from previous work were shaped into more concrete ideas for developing a working interactive IoT system. This was followed by the development of wireframe interface ‘walkthroughs’ on paper, which were developed by the researchers and presented to the energy advisors as part of a second half-day workshop.

The second workshop led to a more detailed critique by energy advisors of the specific functionality of the system, as they began to imagine making use of the proposed interactive IoT system as part of their existing energy advice-giving practices. The refined requirements gathered from analysis of data from the second workshop informed the development of a second set of wireframe prototypes for the interactive system, which would allow energy advisors to organise, deploy and monitor multiple sensors across multiple homes. These wireframes formed the basis of a third half-day workshop to critically analyse the way in which advisors might interact with sensors and sensor data as part of the energy advice process. The outcomes of the final design workshop were used to refine the system specification and resulted in CharIoT, an extend IoT system consisting of the following sensor kit (Figure 1):

• A Raspberry Pi based hub, which connects to the client’s broadband via Ethernet and receives radio signals from the wireless sensors and relays data to an online system.

• Off-the-shelf, battery-powered wireless sensors to monitor temperature and humidity.

• Custom-built, battery-powered wireless sensors to monitor temperature, humidity, and CO2.

Open image in new window

In addition to the sensor kit, the CharIoT system also provides an interactive web app, which includes:

• A configuration utility, which allowed advisors to assign sensors to households and add basic information about households to the system.

• A dashboard, which provides an overview of all sensor kit deployments and shows the most recent readings and battery levels for all sensors.

• A data viewing tool (Figure 2), which was designed to enable advisors to view and annotate sensor data, and to support articulation of the data between advisors and clients.

Open image in new window

This beta-level system was then given to energy advisors to deploy in their own homes for testing, allowing them to get a feel of what it is like to have real data recorded and visualised. The workshop following this beta testing focused on understanding the ways in which advisors accounted for the data recorded in their homes and what issues if any they anticipated would arise as part of a live system deployment with real clients in the wild. Further refinements made to the system as a result of the beta testing included room-level filtering, PDF export, visual improvements, and general bug fixing. A final system was given to the advisors along with a training session on how to set up and deploy sensors.

4.1 Advisors’ role

In this deployment we worked closely with four advisors. While their main responsibility continued to be providing advice during in-home visits, their role also involved managing the end-to-end process of deployments, which included three home visits and office work. Tasks included recruiting participants, installing the sensor kit, visually inspecting and annotating the sensor data using the interactive system, providing advice to clients drawing on the data, and wrapping up the deployments. The three advisors split up the 10 deployments opportunistically (by availability) between them, but generally attempted to look after each case for the whole duration of the deployment.

4.2 Participants

Ten homes were recruited to take part in our study after ensuring participants met CSE’s recruitment criteria. Two households subsequently dropped out. Out of the eight remaining households, three households had people over 70 years of age, four had children below the age of five, all eight were low-income households (less than £16,000 per year, compared to the median national household income of £26,000), six reported their homes were colder than they preferred in winter, three reported problems with damp and mould, five reported they struggle or sometimes struggled to pay their fuel bills, and four homes had people with illnesses made worse by the cold. The latest UK government statistic report that 2.38 million households were in fuel poverty in 2014 (DECC 2016).

4.3 Home visits and workshops

Deployment, use and study of the system was facilitated by the advisors through a sequence of three home visits. First, advisors conducted an installation visit to place the sensors in the participant’s home (one sensor was typically placed outside, and up to four were distributed in different rooms within the home). After about two weeks, an advice visit was done in which advisors drew on the interactive system to work through perceived problems together with clients. After a further one to two weeks, the sensor kit was collected in a final visit, and the participants were asked about their experience and whether they did anything differently as a result. In addition, advisors and researchers conducted two workshops. In order to prepare for the advice visit, a rehearsal workshop was conducted, where the data from homes were reviewed. Further, a debrief workshop was held at the end to reflect on the deployments.

4.4 Data collection and analysis

We treated the deployments as opportunities and subject matter for fieldwork. To capture the collaborative work involved in doing in-home visits, a fieldworker accompanied the advisor and recorded their interactions with clients. Data captured includes about 20 hours of audio and video material, along with fieldnotes. The audio data was transcribed and analysed, taking into account the fieldnotes. Our analytic orientation is ethnomethodological and seeks to identify the methodical ways in which the members of a setting naturally and accountably order their activities in interaction (Garfinkel 1991). Two experienced fieldworkers worked through the transcripts using the ‘horizontal/vertical slicing’ technique (Crabtree et al. 2012) to surface the sequential organisation of interaction and drill down into the methodical ways in which the interactive IoT system was articulated in practice by those who were party to its use.

5.1 Anticipation

Anticipating data is the first job of work involved in making the technology work in situ. It is done by administering a questionnaire during an in-home interview with the client. In the first deployment (Fischer et al. 2016), the questionnaire was used to profile the property (e.g., fuel type, heating system, appliances), the occupants (e.g., number, type and age of people living in the home), their everyday routines (e.g., how they use the heating system, dry their clothes, ventilate the home, etc.), and to establish the client’s main concerns (e.g., damp and mould, high bills, cold home, etc.). This ‘contextual data’ helped the advisors make subsequent sense of the data produced by the sensor kit.

As a result of the first deployment, advisors amended the questionnaire to gather more contextual data in order to better understand the indexical character of sensor data when rehearsing for the advice visit. Thus, a new section was added to the questionnaire to capture property information (age, type and wall type), energy efficiency measures, and specific issues with condensation, damp, mould and draught. It was further amended to capture more detail about the occupants, particularly health conditions exacerbated by cold and damp, such as asthma, and whether or not anyone relied on any electrical medical equipment. The questionnaire was also amended to capture not the just the type of heating system and the make and model of the boiler and thermostat, but also the settings used by the client, whether or not they used a programmable timer and if so how they used it, and whether they were using any secondary heating sources such as electric heaters. Further detail was also captured about the occupants’ cooking habits (e.g., how often they typically used the hob, oven, and kettle), and a checklist to capture any frequently used electrical equipment. The locations sensors were placed in were also noted for future reference.

As the above vignette makes visible, the introduction of sensors into the home is ‘occasioned’ (Zimmerman and Pollner 1970), and occasioned in a number of ways that make the introduction accountably reasonable. Thus, we can see that the current interaction between the advisors and client is occasioned by prior contact between the client and CSE, which warrants the advisors being in the client’s home ‘here and now’; the warrant being that “your home is colder than you’d like it to be” and that there are “some difficulties with damp and mould”. It is this warrant that occasions the elicitation of contextual data and results in the articulation of a specific problem “in our bedroom”. This, in turn, occasions the proposal to “put the sensors in” and “test” how “humid the room gets and the temperature”, both of which “can lead to damp and mould”. In home after home we see the same methodical procedures at work in the articulation of contextual data and the occasioning of warrants, problems, and proposals making the introduction of the sensing kit an accountably reasonable thing to do.

It is clear, then, that sensors come to be situated in specific locations with reference to the particular problems that occasion their introduction into the home; that there is a reflexive relationship between the articulation of problems, which warrant proposals being made to introduce sensing into the home, and the actual placement of sensors. A sensor is not, and cannot, be placed just anywhere. Rather, there is a close coupling between just where a sensor is placed and the reasons that accountably motivate its introduction. It is also plain to see that in the course of situating sensors advisors seek to understand clients’ problem management practices (e.g., constant wiping down). There is more to eliciting contextual data than simply filling in a form then, and there is more to situating sensors than coupling them to problems. Sensors are also placed to help the advisors understand the impact of domestic routines on the home and the client’s problem(s). Thus sensors are placed in locations that enable the advisors to understand occupancy patterns (through CO2 sensing), heating patterns (through temperature sensing) and the impact of routine activities such as cooking and bathing (through humidity sensing), etc.

As the vignette makes visible, situating multiple sensors requires the advisor and researcher, and indeed anyone who might be doing this work, to “check” that the sensors are working. This is done by looking for “readings” on the hub, which turns not only on technical knowledge of sensor communications (e.g., waiting for a refresh) but also on the situated particulars of ‘just this’ installation. Thus, checking that the sensors are working also turns upon pairing readings with sensors (denominated by two-letter IDs) and sensors with locations, which as the above vignette shows is occasionally problematic. The problem is resolved by working through the mapping and matching sensor IDs to locations.².

Thus, having installed the sensor kit, and checked that it is working, the advisor and client make specific arrangements to “look at the data” and for the advisor to “offer some advice” to the client “particularly around the mould issue.” Eliciting contextual data is more than a matter of simply completing a questionnaire then. When we look to see what’s done in the doing (Crabtree et al. 2012) of filling in the questionnaire, we see that the articulation of contextual data warrants the doing of a technical job of work that methodically provides a) for the introduction of IoT technology into the home, b) for situating it in particular locations with reference and respect to specific problems, and c) for the future use of data generated by the sensing kit to address those problems. We note too, that the improvements made to the contextual data capture instrument are indicative of an effort towards more systematic elicitation of contextual data or metadata. Metadata provides crucial information on the indexical relationship of sensor data to the sites and practices of its production. Without metadata then, it would be very difficult for advisors to make sense of the data, or to use it in a meaningful way within the subsequent provision of energy-related advice.

5.2 Rehearsal

The vignette makes it visible that the advisors first need to “simplify” multi-sensor datasets to identify remarkable patterns, such as temperature or humidity fluctuations, and thus “get an idea” of what the problems are in a particular home. The simplification is done methodically by filtering out the noise created by multiple sensor feeds, focusing down on single data sources, and noting down remarkable occurrences (e.g., that the temperature is quite low, or that humidity is variable and problematic).

As this vignette makes visible, the correlation of multiple data sources enables advisors to infer particular patterns. That, for example, the inhabitants of the home have “been away” from home because CO2 readings are “stable” and it’s plain to see that there’s “not a lot happening”. Conversely, correlating multiple data sources enables advisors to infer that and when people are at home as the reading starts to peak alongside a “temperature increase”, an occupancy pattern that is confirmed by “putting in the gas” to see if it “coincides” with CO2 and temperature, which it does. Taken together, single data sources enable the advisors to identify particular classes of problem in the home, and multiple data sources allow them to infer the patterns of human action that are implicated in their production. Thus, and for example, the correlation of multiple data sources enables an advisor to identify issues implicated in remarkable patterns seen through a single data source (such as occupancy and temperature fluctuations).

A further methodical feature of data work is found in the way that advisors earmark remarkable patterns and associated data for discussion in the advice visit. This was one of the main practices we sought to support digitally by means of the interactive system. However, this has turned out to involve not just the interactive system, but additional notes on paper, as this advisor explains to a colleague.

• Vignette 7a. ‘Noting’ remarkable patterns

• A: So, I’d probably go about putting in a note there (pointing at screen, Figure 3), annotating it as an area to explore with them [the client]. And I would try to use a kind of notation system, writing down an actual note on paper with the time and roughly what I would say. Just so I know where to go back on here (pointing at screen) - an indication, like as simply as possible.

Open image in new window

The notation system involves both digital and physical annotations. The former is provided for through the interactive system. The latter is provided for through a bespoke accountability system: the “home visit sheet”. The sheet not only indexes the digital annotation, making it easy to locate specific parts of the dataset during an in-home visit and associated queries, it also lays out and defines an order of business to be addressed during the home visit (occupancy one, two, three, and so on). Thus, the home visit sheet is a coordinating device used methodically to order interaction between advisor and client. It provides a situationally-specific schedule of work that parses the overall dataset produced by the sensing kit and surfaces particular issues that need to be worked through with the client.

The “checklist” formalises a professional practice for looking at the data, orienting advisors to “anything out of the ordinary” - maximum and minimum data points, differences, variations, fluctuations in and between rooms, etc. Finding “anything out of the ordinary” turns upon the various orders of knowledge and reasoning that the advisors possess and exploit in looking at and reading the data.

Formulating potential advice turns upon sensor data (e.g., temperature readings), technical knowledge (e.g., of normal and abnormal heating cycles), common-sense knowledge (e.g., that abnormalities could, in this case, be caused by small room size), and contextual knowledge (e.g., of the client’s problems and practices). The combination of sensor data with these different orders of knowledge and reasoning enables advisors to go beyond offering general advice and provide situationally specific advice instead. Thus, and for example, an advisor can offer a client “tips” on using the heating system more efficiently such as “making use of the room thermostat”. Such fine-grained tips are provisional, however. They may resolve the client’s problems, but whether they do so or not has yet to be ratified.

To sum up, the ‘rehearsal’ stage of data work involves articulating problems and their potential solution. This is done by simplifying the dataset to identify ‘remarkable’ patterns in the data and correlating data sources to identify issues potentially implicated in their production. The work turns upon the use of technical, common-sense and contextual knowledge and the design and use of methods for ‘noting’ remarkable patterns and assembling ‘reference data’. Reference data parses the overall dataset and surfaces ‘anything out of the ordinary’, which enables advisors to formulate ‘tips’ that may resolve the problem situation. Reference data is indexed through the production of a home visit sheet, which allows advisors to quickly locate relevant data and defines a situationally-specific schedule of work ordering subsequent interaction between advisor and client in the performance of data work.

5.3 Performance

This vignette makes visible the methodical way in which advisors go about articulating a complex dataset produced by multiple sensors. Just as in rehearsal they filter the dataset to focus down on a single data source “to begin” with, though this time the filtering is driven by items earmarked on the home visit sheet. In this particular case the data source is related to the primary problem that affects the client’s home: temperature, which is indicative of potential damp and mould problems. The vignette also makes it visible how in articulating the data from a single source advisors explain what the data visualisation “shows”, thus offering an orienting description – “the purple one is the living room, and the grey one the kitchen… olive one the upstairs bedroom,” etc.

The data allows the advisor to articulate particular patterns (e.g., average temperatures throughout the home), and in turn to assess these (e.g., that they are “good”), and to relate them to normal expectations (as prescribed, for example, by general health guidelines). Visible discrepancies between the two enables the advisor to propose potential remedial action (e.g., “to heat a bit more”). Multiple data sources are introduced into the interaction to drill down into the problem (e.g., humidity in the upstairs bedroom) and drive home the advice: temperature is low, humidity high, so increasing the temperature is going to decrease the humidity levels. While multiple data sources enable deeper articulation of the client’s problems and their causes, this vignette also makes it plain to see that the advice they enable may also be problematic: e.g., increasing the heating in a low-income household.

That is not to say that nothing can be done about a client’s problem and advisors routinely go beyond data work to work through potential solutions (Fischer et al. 2016). In this particular case, the advisor and client left the data behind and went to inspect the damp and mould in the upstairs bedroom. In doing so the advisor articulated a range of options for managing the problem, including frequent wiping down, leaving the window ajar to increase ventilation, opening the curtains to let the sun warm the room, putting a reflective panel behind the radiator to increase heating efficiency, having the landlord check the roof insulation above the problem area, and turning the heating off when no one is in the house (a pattern also made visible by multiple data sources, particularly temperature, gas and CO2).

Overall, what is evident in the fieldwork observations is that the articulation of complex multi-sensor datasets frames and guides advice giving in important respects. In order to deal with the complexity of these datasets, advisors filter down to single data sources related to items earmarked for discussion and in doing so reflexively articulate and explain what the data is about and what it shows the client. Perceived problems are initially articulated with reference to a single data source, with multiple data sources being subsequently drawn upon to drill down into and elaborate problems. While client’s problems are often obvious, in that they already know what they are, multi-sensor datasets allow advisors to understand and articulate their causes (e.g., that low temperatures and high humidity are at the root of a particular damp and mould problem) and offer tailored advice to remedy the situation. The tailoring of advice is done with respect to the client’s circumstances and may also be informed by multi-sensor datasets, which is to say that such data is not only drawn on to articulate problems but also to articulate viable local solutions.

5.4 Reflective workshop

The advisors found value in the interactive system and appreciated the ability it gave them to “immediately identify” and “flag” potentially problematic issues.

The use of multiple sensors has been “beneficial” to professional practice, enabling the advisors to both identify material problems with the home’s infrastructure (e.g., things that need to be fixed) as well as issues related to the use of that infrastructure (e.g., poor temperature control).

The perception that the sensor kit and interactive system helped clients engage with issues and “believe” what the advisors were saying underscores findings from our previous work, which suggests that IoT data can and does play a constructive role in building a trusting relationship between advisor and client.³.