1 Introduction

Shape-changing interfaces are complex, tangible interactive objects, surfaces and spaces allowing rich computational experiences and physicalization of information. Examples include mobile phones that bend at the corners to alert you to a call or a text [39], table top surfaces that raise up to pass you your tablet [9], and the organic, liquid metal art installations that move in ever increasing complexity [24]. Shape-changing interfaces allow dynamic shape as physical input and output, as well as supporting other sensory interaction, and herald the next stage in computing hardware. At present the field is largely technology driven, with end-user applications emerging from the affordances of the available platform. Despite the diverse range of prototypes researchers have developed [55], there are no formal methods, guidelines, or tool-kits specifically developed for shape-changing interface development – with the exception of rapid prototyping, often using modular devices [20].

For existing systems, User-Centered Design ensures that the tasks, needs, and context of end-users drive and reflect upon the development of a new system, but we cannot presently apply this to shape-changing interfaces. This is because current shape-changing interface design is not targeted at solving a particular problem, or trying to design specific hardware or interaction – we are purely striving for innovation. In addition to this, possible end users are not aware of shape-change as a technology, what it can do, and the range of available hardware is difficult to demonstrate due to issues of location, portability or safety. In the cases where users are given the opportunity to interact with a shape-changing interface prototype, they may also become “trapped” into thinking about shape-change as being of one particular form (e.g. actuated pins [6]). Much of the work on shape-change appears to pick from different areas of existing design processes, but does not seek to employ them as a specific methodology during the research process: for example, building hardware is often the first step in exploring shape-changing interfaces, and is then followed by a short usability study for whichever application best suits the platform [54, 58]; or studies might focus on user-ideation or co-design for non-specific products [5, 56]. The reason behind this may be that, traditional, sequential processes (e.g. planning, user research, user evaluation, information architecture) may not fit exactly with emergent hardware which does not already have a predefined role.

At this early stage, we hope to utilise readily available methods to provide a baseline of requirements for shape-changing interfaces. By focusing on a practical start-point – requirements generation – we believe we can begin to adapt and build formal design process for this exciting technology from the ground up. Following the generation of requirements, we can model these to form an overview of the field and its possibilities. This paper therefore contributes: (1) A practical, readily available approach for requirements generation for shape-changing interfaces; (2) A 50 participant study that demonstrates the validity of this approach to generate requirements for this novel technology; (3) A thematic analysis of the generated user requirements; (4) A shape-change stack model to support practitioners in design requirements-gathering activities for shape-changing interfaces, intended to provide a cohesive resource for those building and testing shape-changing interfaces with the view to their eventual adoption.

2 Related Work

Generating requirements for devices that do not yet exist is an exciting challenge. To address this, we suggest utilising accessible techniques in order to inform and engage potential end-users about shape-changing interfaces. Understanding current applications and approaches to shape-change – alongside these already validated techniques – can assist in beginning to design and develop formal processes for these novel devices.

2.1 Shape-Changing Interfaces

Shape-changing interfaces are an exciting subset of tangible computing hardware, with the potential to not only use 3D form in input/output, but to also allow for this form to be manipulated by the user, and sometimes, self actuated. Examples include (but are not limited to): interactive tabletop surfaces such as InFORM [9] or the elastic-pin hybrid Tablehop [49]; rapid prototyping devices like ShapeClip [20]; mobile phones like WhammyPhone which explores bendable audio interaction [14] or Reflex which uses the bend action to enhance reading on small displays [54]; furniture such as the unexpectedly actuating CoMotion bench [17]; or even public installations like Aegis Hyposurface [16] and Protrude/Flow [24]. Although a large number of works focus on hardware development, there also exist examples which periodically review, critique and ponder the state of the field of shape-change, notably Rasmussen et al. [44], Coelho and Zigelbaum [3], and recently, Sturdee and Alexander [55] but these offer general overviews of interactive capabilities or material properties rather than advice on building these novel devices (although the latter suggests material properties may be a helpful starting point for a design framework). Additionally, research touches upon particular aspects of design for these devices such as emotional or anthropometric content [27], vocabularies for design [45] or constructing form language for shape-change [61], but there still exists a gap in the literature for a consolidated framework or process for the building and development of shape-changing interfaces.

2.2 Requirements for Shape-Change

Requirements are the things a system should have in order to function and fulfil the needs of the user [26]. They can be gathered in a systematic (as with software requirements engineering) [52] or informal manner, with a preference for the former. However requirements engineering has not yet been directly applied to shape-change, and there are no existing parameters. Stakeholders (in this case potential users) have no existing schema for shape-change, so first the concept and structures must be communicated and the solution space expanded. For shape-change, the easiest way is to demonstrate and allow interaction with existing hardware, but this is not practical given constraints such as geographical location and accessing multiple devices from different research labs. The second challenge is how to capture the stakeholder responses as you cannot simply interview the user about their experiences with a product that does not exist, and that they have not used. To overcome these barriers, we might explore the possibility of using creative methods [38] to elicit early stage requirements in three ways: by employing an efficient method of communication [62] to describe the state-of-the-art of shape-change; by creating opportunities for interaction [41]; and by using an accessible method of information production [2].

2.3 Accessible Techniques for Design Requirements Generation

The techniques described below enable us to address the challenge of generating design requirements for shape-change.

Video. High-quality video is often produced alongside published work in order provide quick explanations of a hardware or system concept, without reading the accompanying text. It can also be used to inform, communicate or explore concepts [62]. To a large percentage of users, receiving information in this way is a normal, accessible part of smart (and other) device interaction (such as YouTube or Vimeo) [10]. For actuated prototypes, a realistic rendering style was found to be the optimal way of communicating a concept to users in a study of shape-changing phones [40], whereas Gong et al. [15] suggest that high quality videos depicting novel hardware allow users to “suspend their disbelief” and make judgements about how useful a prospective technology might be. Videos have also been used within studies in combination with low-fidelity prototypes to generate high-level comments [34] which suggests that combining this media with other techniques may yield useful results. In context, video also enables us to present work that we do not have access to, due to geographical, or other constraints, and is an apt method to communicate high-level concepts.

Low Fidelity Prototypes. Low-fidelity prototypes are quick mock-ups of designs or devices allowing concept testing without committing to an expensive or lengthy build, making them ideal in the requirements-gathering stage of the design process [48]. In HCI, concepts such as paper-prototyping [51] and rapid-prototyping [20] are often used, examined and critiqued for their role in research. For shape-change, the difficulty in creating low-fidelity prototypes is mirrored by the range of technically complex hardware and interactive capabilities of the high-fidelity, working prototypes. By thinking about the materiality of shape-change however, we can emphasise its tangible nature in a simple, easy-to-build manner. Schmid et al. suggests and tests a form-first approach using glass objects to generate ideas for tangible interfaces [50]. The reasoning behind using low-fidelity, white box, prototypes to explore shape-change are twofold: (1) Existing prototypes are often bulky, heavy, expensive and situated in laboratories across the globe; (2) White box prototypes allow for the presentation of matter in a consistent way (i.e. all the same size, colour) so that participants are unbiased by incidental details. Examples include: Kwak’s Repertory Grid Study [27] where a variety of actuated white box prototypes were created to explore the expressive and emotional qualities in shape-change; Petrelli’s work on tangible interfaces which looked at the psychological affect inherent with concepts of shape and haptic interaction [41]; and, Winther et al. who generated white box prototypes following an exploration of form language for shape-changing interfaces [61]. The recent shift in HCI toward the importance of materials [11] also suggests there are benefits in considering our most simple, tactile interactions, (e.g. Atkinson et al.’s consideration of natural, gestural interaction with soft materials [1]). White box prototypes are helpful, as humans may require material anchors to be, by their nature, “sketchy” in order to facilitate cognitive processes [35].

Sketching and Storyboards. Sketches are often seen as low-fidelity – they are rough ideas that welcome opinion and modification [7], and also explain concepts that are hard to suggest with words [12]. Sketching has long been part of the user-centred design process, and Buxton’s book Sketching User Experiences has actively encouraged and enhanced researcher engagement with this format [2]. Sketches are also cheap to produce, and are an inclusive way of generating output as they require only access to a pen and paper – and have additional cognitive benefits [13]. Sketching also has an established place in the design of user interfaces (UI), giving rise to computer-based UI design programs which either utilise, or appear to be sketches and storyboards [18, 28] and can be annotated or embedded with metadata [19]. Storyboards used in the design process can also lead to more effective design [60] or help communicate research findings [22]. There is already a precedence for using storyboards to generate requirements, whether via sketched, computationally enhanced outputs [19] or the traditional, hand drawn versions [57], and participatory sketching has also been shown to help generate requirements for real-world interaction (e.g. elevators) [59]. Storyboards and comics are also already used in HCI within areas such as software engineering [60] and cyber-security [30, 63], and are accessible, quick-to-produce medium for proposing future scenarios. Finally, sketching in direct application to shape-change has already been employed in the user ideation process [56], as a way of exploring the design of interactions for shape-changing devices [45] and even to examine a futuristic material [23].

3 Method

To generate requirements for shape-changing interfaces, we asked participants to experience the materiality of shape-change [55] with white-box prototypes (Fig. 1), and then sketch ideas, diagrams and scenarios. We took inspiration from the described techniques, and Read et al.’s approach to exploring organic user interfaces (comics and material prototyping used in combination to communicate ideas about materiality and change [46]) – for shape-change we can blend parts of the analytic and design stages where feasible. Shape-change is diverse in its materiality and potential interaction range, so to attempt to represent and explain this technology in a simple, single step would be prohibitive. We used a combination of videos to inform and educate, low-fidelity, white-box prototypes to enable exploration and basic interaction, and sketched output in the form of diagrams and storyboards to both assist in explanation and later, interpretation (the diagrams play the role of annotation or metadata for the storyboards [18]). The desired outcome of this process is to “reduce the distance” between the researcher and the end user [37] and create a meaningful collaboration.

3.1 Study Overview

The study was a five-stage process lasting between 40 min to 1 h, including explanation, questions and feedback: (1) Introduction to shape-change using video material from existing research; (2) Exploration and interaction with white box prototypes; (3) Idea generation; (4) Idea elaboration and diagram creation; (5) Storyboarding and scenario generation. The participant output was collected for coding and analysis.

3.2 Participants

Fifty participants (24 male/26 female) were recruited using social media, email, or from volunteering after observing the study set-up directly in shared social/study spaces. For example, 8 participants responded to a pre-organised workshop call hosted internally within the university, another 12 participated after approaching the study team voluntarily in a shared postgraduate student study and social space, and 8 were invited via email and snowball sampling to participate whilst the study was set up in a general-use study-hub meeting room on the university campus. Participants with diverse social and professional backgrounds took part (of the 50, 22 were not involved in academic research or study, and 28 were either students or university staff). Those participants who were not affiliated with the university were recruited via email and word of mouth from existing social and professional relationships of the study team, and encompassed backgrounds as diverse as call-centre worker, marketing manager and retiree. The age range of participants was 21–69, (mean 49).

3.3 Video Material

Participants were shown 7 videos relating to existing research, chosen for quality, specific actuation type and related to the material properties of white box prototypes. The videos served to inform those taking part about the state-of-the-art in shape-change research, and introduce the concepts of materiality in prototyping. The chosen works were: Physical Telepresence [29] (actuated interface); Protrude, Flow [24] (liquid interface); Lightcloth [21] (paper/cloth interface); ReFlex [54] (bendable interface); Paddle [43] (foldable interface); Claytric Surface [36] (malleable interface); Obake [4] (elastic/inflatable interface).

3.4 White Box Prototypes

We created 7 white box prototypes reflecting the materiality of a range of existing shape-changing interfaces representative of current functional prototypes within the field [55], one of which also demonstrated the ability of these interfaces to change state via jamming [8] – see Fig. 1 for categories and images. By utilising white-box prototypes spanning a range of materials, we can communicate the intended interaction of shape-changing interfaces in a simple, portable manner.

Fig. 1.
figure 1

White-box prototypes based on shape-change categories from [55], with examples of how interaction occurs for each: 1. Foldable; 2. Bendable; 3. Paper/cloth; 4. Elastic/inflatable; 5. Actuated; 6. Liquid; 7. Malleable; 8. Jamming.

Full size image

3.5 Ideation, Elaboration and Storyboarding

Participants were asked to explore the white box prototypes through touch and comparison, then write down ideas for applications, hardware, surfaces or spaces that would benefit or enhance their own lives in some way (e.g. in work, hobbies, social contexts). This was based on the process employed in Sturdee et al.’s public ideation study [56]. Following this, they were asked to choose their favourite idea and expand on it via sketching a diagram and writing notes about – for example – how it works, the user base, interaction, and so on. Finally, the chosen idea was put into context within a storyboarded scenario of use (see Fig. 3).

4 Analysis

Fifty participants generated 255 ideas and corresponding sketches for shape-changing interfaces, applications, surfaces and spaces (mean 5.1). They then selected one idea to elaborate upon (n = 50). The majority of chosen ideas were indicative of shape-changing hardware (43/50) rather than specific applications for a generic device, although most would allow for multiple applications. Two of the chosen ideas did not specifically address shape-changing technology so were not used in the analysis.

4.1 Generating Design Requirements

To elicit requirements from the data, four HCI researchers (to limit bias: one independent from the study, one uninvolved in data collection) coded the data using open coding [53] and affinity diagramming with post-its [33]. Initially, one data set was chosen at random and examined by all researchers, who then generated post-it notes suggesting requirements, interactive properties, context and possible implications for the technology. The group then split into pairs and worked on the data, and these pairs were rotated. Requirements were extracted in several ways (see Figs. 2 and 3 for an example of participant data): directly through notation on the diagrams and storyboards (e.g. device is \(20\times 20\) cm, device is portable); from examining the interaction (device has furry texture, folding and closure must be possible); from the proposed output (mimics organic form); and from context (device has therapeutic purpose). The implications of the technology then arose from the idea of having access to a lifelike, adaptable non-organic representation of a pet – e.g. decline in pet ownership. A guide to the coding and requirements generation is overlaid on Fig. 2.

Fig. 2.
figure 2

Diagram & scenario for Kitten Everywhere app, annotated to show how requirements were generated. For example, bottom right – the kitten’s paw is overlapping the person’s hand and reconnecting with the interface surface – closure [47].

Full size image

Mid-way through the process we examined emergent themes and categories, and it became clear that there was more than simple hardware/software requirements and basic human factors. The post-it notes were then recategorised under specific titles (for example) scale, interaction, portability, multi-sensory, device dependent properties, context of use and so on. The remainder of the initial data analysis was then completed (with further categories emerging) before the next stage, where the clear hardware/software requirements and interaction types (based on [44]), the physical properties were temporarily separated off, and the complex, operational properties were recoded entirely by the group. Finally, all the categorisations and themes were cross-referenced with the original data and recoded where necessary to create multiple categorical levels, then proofed and the entire dataset digitised and checked for errors. The complete user-generated data set can be downloaded as an appendix.

4.2 Results

Analysis of the 255 participant idea sketches produced 506 coded items across three categories – Requirements (333), Applications (104) and Implications (69) – with multiple sub-categories. An item refers to all text-based outputs from the coding process. Items from Requirements and Applications were synthesised to produce the stack model in Fig. 4 by using top level categories for each and ordering them logically for a prototype build process. From those items falling under the Requirements category, we provide those most frequently occurring so as to give an overview of how people appear to think about shape-change (Sect. 5), and show a categorisation of the top level themes which enables a stacked requirements model (Sect. 6). Finally, we analyse individual findings and current works to demonstrate the validity of the stack model (Sect. 7).

5 Frequently Occurring Requirements

In total, 333 requirements were generated, across 5 top level categories (Input, Output, Construction & Assembly, Control Systems, and Interactions & Behaviour) which directly relate to the top level headings for the stack model in Fig. 4. The following text contains the highest frequency sub-themes emerging from the analysis (in order of highest frequency), suggesting specific, perhaps essential, requirements for the design of shape-changing interfaces. Examples of sections of original data relating to the categories below are shown in Fig. 3.

Fig. 3.
figure 3

Examples of shape-changing interface relating to frequently occurring requirements: (1) Photo album for blind; (2) Battle armour; (3) Drinkable tablet.

Full size image

Between Device Communication (Interactions & Behaviour). This finding suggests that our current devices will co-exist with their newer, physically dynamic counterparts. Concepts such as up-scaling of film or image from 2D to physical 3D are explored. Shape-change is also expected to communicate between devices (e.g. Photo album for the blind – Fig. 3(1)).

Rigidity (Construction & Assembly). Varying the material qualities of a device was communicated by the jamming box, and was used in conjunction with other material categories to create behaviours in which a device or application moves between states depending on context of use. This variation is especially important in generic devices where multiple uses are anticipated (e.g. interface which becomes a playable guitar).

Strength (Construction & Assembly). Shape-change is not only expected to display data, provide comfort or simulate environments, it is expected to be load-bearing in architecture, move boulders with the tap of an application and support multiple bodies as a sofa, car, or podium. To this end, the materials and construction used to create shape-change must be physically robust (e.g. flooring) for the intended application.

Modularity (Construction & Assembly). Modularity not only refers to the ability of identical devices to communicate with each other, but for parts of other types of material surface to be removed, used, and reintegrated, or for components of shape-change such as actuators to communicate with each other (e.g. toy blocks).

Portability (Construction & Assembly). Many of the participant ideas were categorised as portable object-scale devices, suggesting the need for novel batteries or charging methodologies such as using body movement (kinetic), solar, or wireless. In the requirements, portability emerged as a distinct theme (e.g. armour – Fig. 3(2)).

Multi-sensory Input and Output (Input & Output). Users are expecting interfaces to have deformation as an interaction technique, and for this technology to also employ the full range other human senses in their application and design, emphasising the organic potential of shape-change (e.g. drinkable computer – Fig. 3(3)).

Organic Movement (Interactions & Behaviour). By attributing natural and humanistic qualities to range of movement, shape-change crosses into the territory of Artificial Intelligence, or the mimicking of life. Organic movement in shape-change links to comfort and sensitivity, reflecting a positive behaviour (e.g. prosthetics).

Device Personalisation (Interactions & Behaviour). For planar, screen based devices, personalisation usually takes the form of a physical accessory such as a screen protector, or amendments to the display or applications. These amendments can also be attributed to shape-changing devices, but additionally we might control the shape, texture and even how it connects with our bodies (e.g. adaptive training shoes).

6 Toward a Design Requirements Model

Our synthesis of requirements of shape-changing interfaces revealed five overarching categories, each at a different level of abstraction. Together, they logically fit into a stacked layer model (Fig. 4) describing levels of requirement and implementation in shape-changing interfaces, and differentiating between physical/operational characteristics. Under the top-level categories, we also identified eight sub-categories of requirements. The top-level categories are outlined below alongside the Implications category which was major theme arising from the data not directly connected to design requirements.

Fig. 4.
figure 4

The stacked model of implementation, based on the requirements generated during the study – see Sect. 5 for frequently occurring requirements.

Full size image

6.1 Applications and Context

Applications are defined as the specific use envisaged for the device (e.g. battle armour in Fig. 3(3)), and “apps” (such as Kitten Everywhere in Fig. 2). Context applies to where, when, what and why of using the application (e.g. at home, working hours, provide remote massage, travel less for work). Just over a fifth of the items produced during coding were application and context related. Often the context also dictates the application, and this is especially the case for generic, catch-all devices. Poupyrev et al. [42] reviewed potential uses for actuated, tangible interfaces ten years ago and suggested applications and areas based on the literature at that time, creating five categories: Aesthetics; Information Communication; Mechanical Work; Controls – Data Consistency; and, People to People Communication. In the intervening 10 years however, the range of devices and applications has grown and the five categories remain relevant, but can be blended into the overarching contextual categories generated from more recent work [56]. The application areas generated in this work can be mapped directly onto those found in Sturdee et al. [56] (with the exception of the infrastructure category), with some exact application ideas being repeated such as responsive computational flowers, remote massage, or actuated storybooks for children. However we also propose here a wider generic category where the properties of the device or surface are used for multiple use-cases.

6.2 Interactions and Behaviour

Interaction refers to the relationship the user has with the device and includes the type of interaction, e.g. specific behaviours identified during coding (squash-to-delete) but also the interaction the device has with other technology (between device communication). Behaviour encompasses software actions – what the device does (switches between planar/3D output). Eighty-four items fall into this category. Interaction for shape-changing interfaces is perhaps best classified by Rasmussen [44] who developed the framework of Direct, Indirect and No Interaction. The requirements for basic interaction in this model also offer other options: anti-interaction (the device actively avoids or puts off, interaction), encouraging interaction, and device-to-device communication, covering interaction between shape-changing devices and also between shape-change and current mobile devices and computers. In addition to these high level categories, the behavioural aspects of shape-change are explored with regards to how the device or surface acts or moves to initiate the programmed output, and whether you can personalise your device.

6.3 Control Systems

This layer covers the Software Implementation for the shape-changing device, outlining how the device puts the hardware features into use in programmed, pre-set features and attributes. This is the least dense layer with 20 items, and includes requirements such as physical shape-memory and must have user recognition software. This can be thought of as the interface operating system.

6.4 Construction and Assembly

The physical requirements that are unrelated to sensory input and output are contained in this layer, it contains specific information on hardware and appearance of the device (such as size and portability) as well as non-functional requirements (washable, lightweight). It also contains the Hardware Implementation which includes information such as integral camera or low latency over internet. This layer contains over a third of the requirements and over a fifth of the total items.

6.5 Input and Output

This layer describes input and output sensors for shape-change, incorporating multi-sensory information in addition to visual and shape output, and specific information such as GPS, speed, texture, temperature and air pressure. Despite a tendency toward two-way multi-sensory interaction for generic shape-changing devices, bi-directionality was not seen as an essential quality for application specific shape-change, which contradicts previous work which suggests this is an overarching feature of tangible interfaces [3].

6.6 Implications

Sixty-nine implications were generated from the context and use cases implied by the applications. An implication, in this format is a possible direct result or reaction deriving from the adoption and use of a shape-changing technology. The Implications were realised both from the application ideas that were generated, but also from specifics within the scenarios and diagrams that the participants created. They are categorised into: Positive (of benefit to the majority of the population, such as faster recovery from debilitating injury, improved well-being, and sustainability); Neutral (not clearly mapped onto a specific target group or are cannot be categorised, such as “more money for cinemas” or “consequences of shape-changing AI left to own decisions”) and Negative (of negative benefit to the majority of the population, such as removing the need for labouring work, or reducing human contact). The possibility of AI taking jobs from people is already a hot topic, and shape-change could enable a far greater obsolescence.

7 Using the Requirements Model

We envision that the stack model can be of use for researchers, to be used as a reference list showing the relationship between different parts of the stack, to ensure users consider requirements at all levels (important because the decisions at one layer effect the layers above and below). It could also help categorise and synthesise many requirements together, and function as a tool to direct users to think about certain types/areas of requirements during early design phases. Designers could also use it to ensure they have considered input at all levels (as incorrect assumptions at one level can propagate), and technologists can use it to understand the impact of their technology decisions (bottom layers) on the upper layers. Within the existing framework of shape-change, it could also be used to assist researchers to better understand existing prototypes – however, we envisage further development before it can be fully adopted by the community.

7.1 Applying the Requirements Model to the Dataset

The overarching requirements and subcategories are distributed across the stack, with a slight bias toward the multi-sensory, bottom layer of the stack (input/output). Software implementation (Control Systems) is the most under-represented layer, perhaps due to the participant sample we used (non-computer background), but this omission could be addressed by asking users to specifically think within the stack system. Looking at the participant data in relation to the stack model, we could re-analyse the diagrammatical and storyboard data and identify extra requirements that the user may not have explicitly thought about during the study. In the case of the drinkable tablet (Fig. 3(3)) which we know has variable rigidity and is safe to ingest we might now return to the under-represented software implementation and extract the information that if it is a tablet then it also runs apps, and we can see from the sketch that it also supports a 2D planar screen, meaning it should be backward-compatible. The demonstrates that the stack model therefore can support directed thinking for the researchers by identifying specific areas and therefore eliciting further requirements. Observation of the participants during the study suggests that a mixture of top-down and bottom-up approach is used to create the data: either where an application idea was realised via consideration of what hardware would be required to achieve that goal, or beginning with an idea relating to a specific hardware type based on the white box prototypes, and moving up the stack for application and contextual design. For example, the drinkable tablet idea (Fig. 3(3)) came directly from the liquid interface prototype (Fig. 1(6)), blended with the idea of variable rigidity (Fig. 1(8)), and although in context of a science fiction story, the practical information remains relatable. Following the ideation and diagram creation, the context in which a drinkable tablet would be helpful was then realised – in this case the spy conceals their data by ingesting it – and a scenario based on this idea was drawn.

7.2 Application to Existing Prototypes

We applied a retrospective analysis of the stack model to two existing, contrasting, prototypes to confirm the layers fit with current research, and to get an idea of where build focus was. The choice of prototype was made by examining the methodological processes of existing works, focusing on: Decision-making – what was the initial inspiration or background to the work?; Technology used – was it existing, based upon other prototypes, or a novel implementation; Context – what need does the work fulfil?; Behaviour – what form of shape-change does the prototype exhibit?; and, Inputs/Outputs – how is the prototype controlled and what display mechanism does it have? By querying existing research using the stack model as a basis it is possible to retrospectively apply its principles.

Table 1. The stack model for shape-change applied retrospectively to Morpho-Tower [24] and Shape-Clip [20].
Full size table

Kodama’s ferrofluid works [24] are born of the desire to emulate organic movement and create art, therefore the focus in the stack is on application, and the interactive/behavioural qualities: a top-down perspective (Table 1, left). Conversely, for ShapeClip [20] the focus was on the hardware: building a bidirectional actuation device that was low cost, modular and easy to use (Table 1, right). This suggests a focus on the lower two layers of the stack (bottom-up), with omissions in detailing interaction behaviours. Both papers refer to future hardware improvements, applications or use-cases (e.g. advertising, sound equaliser; third skin) but do not consider the longer-term implications for their projects (this loosely supports the notion that this research is very technology driven, and not focused on long term adoption [25]). This is not a criticism of existing research, but identifies the requirements focus for different types of shape-change, and where researchers have concentrated their efforts. It also demonstrates the difference between application (Morpho Tower) and utility (ShapeClip).

7.3 Limitations and Additions

During the coding, it became clear that the aesthetic and emotional aspects of shape-change were largely overlooked, relegated to resulting from – or being incidental to – the device or application itself. Participants tended to focus on the practical aspects of shape-change, which usually started with an application idea (top of the stack) rather than a hardware type, although some items (e.g. shape-changing, responsive flowers) were built with aesthetics in mind, and the non-functional and functional requirements were built around that notion. The lack of focus on aesthetics may suggest two things: That design for aesthetically pleasing objects is a given; or that the desire and design for aesthetics occurs further down the stack – for example, comfort and beauty may be built into the construction phases of implementation (which makes sense if the purpose of the device is as a furniture provider). In terms of emotional content or outputs, these are more likely be an implication of, or bound up in, the type of application (such as a virtual physical pet to provide company). However, Kwak et al. [27] noted that the behaviour of his prototypes was implied by their actuation, and gave rise to emotional content e.g. stubbornness or feeling hopeful. This links to the Interactions & Behaviour category within the stack, within which a number of the behavioural themes related to organic movement, meaning emotions would be built in after the software implementation cycle. The stack model could thus be adapted to add a subcategory of emotional content, bound up in the design of organic movement (one of the minor themes from the coding process), whereas aesthetics would become a category within Construction & Assembly.

8 Discussion

The stack model and analysis suggests that the methodology has the potential to expand our understanding of how these devices will be built, and applies an organisational structure to the development process; whereas the implications generated can stimulate discussion about future adoption of physically dynamic interfaces. Our methods fulfil their intended purpose: we communicated complex, novel technology to non-expert end-users who were able to generate detailed outputs from which researchers could elicit requirements. The commonly occurring sub-themes also threw up some novel requirements for shape-change that (to our knowledge) have not been documented. Between device communication within shape-change is not a new concept, but these requirements also consider integration of shape-changing interfaces into existing technological structure – e.g. having an integrated USB port, or sending planar data to be up-scaled. Device personalisation has been addressed in shape-change, but only in attribution of types of actuation to different mobile phone notifications [39], when personalisation could potentially also involve texture, form, complex organic movement and multi-sensory experiences – that shape-change should be multi-sensory is another under-explored facet, especially given that the only essential output is change in form. Finally, although many of the requirements are pre-existing in other technology development processes, the way the stack model addresses these requirements, and how the extra dimensions of movement, organic behaviours (and so on) are integrated makes the result specific to the application of shape-changing technology. The fact that we are producing similarities with other work also means that we are in a good place to begin formalising process and practice for shape-change – design requirements needed to be approached in a tailored way to shape-change, and we have appropriately managed this process.

8.1 Methodological Reflection

Due to the range of materials, interaction and applications possible, we did not use a specific scenario or hardware type (e.g. shape-change for mobile gaming/a music app for a malleable device). Given the nature of our approach, we felt that choosing one application or problem would ask the researcher to arbitrarily define that issue, and bias the process toward one kind of shape-change. By asking users to define their own problem and solution, we explored the nature of the process in way that still provided focus. During introduction and ideation, participants found the videos helpful, though two found it difficult to relate what they had seen to the boxes which did not self-actuate. Others became quite excited by the boxes, choosing to have one next to them so they could return to a concept, or taking part of the material (malleable) with them to work through interactions. The jamming prototype box was seen as the most engaging, as the material concept was novel and the transition between states illustrated how a shape-changing interface might move between material properties, but also because it was in context of the other boxes. Another observation was one of the hedonic qualities of material interaction [44], some participants enjoyed touching the materials, and focused upon how the pleasant sensations could be utilised. Removal of either of the stages would be detrimental, as establishing technological context sets the scene and explains what, but providing tangible, low-fidelity examples encourages participants to ask questions and suggest improvements about how without focusing on single use-cases.

The sketching and storyboarding process allowed the contextualisation of ideas in an easily understandable, visual output. Some participants expressed anxiety during the task, but were reassured by focusing on the ideas rather than producing high quality artwork. Sketching even had a positive effect: “I haven’t drawn in years, I had forgotten how much fun it was”. Given the success engaging with end-users for shape-change, we envisage this process being used in the early stages of new research. In future work, the technique could be applied using a single white box prototype, and specific application, and to explore the next stages of developing shape-changing interfaces – functional prototype development/interface design), and the stack model evaluated in the early development of shape-changing interfaces.

8.2 Implications for Adoption?

As well as approaching user-centred design from the standpoint of shape-changing interfaces, we are also attempting to consolidate research in this area, and encourage the organised advancement of specific interfaces. HCI as a field has been accused of an unfocused attitude toward research, rarely developing topics so that they enter the mainstream [25], or criticised for focusing on short-term utility [32], so by offering up a methodology to engage with possible end users and suggest constructive avenues to pursue we attempt to counteract this view. As an extension to this, some researchers suggest reaching even further into the future to explore not only the adoption of technology, but the implications of that adoption [31] – how might domesticating technology affect people in both positive and negative ways? An unexpected but welcome side effect of the requirements-gathering process was the generation of implications surrounding the adoption of shape-change. This allows researchers to focus on potentially interesting build-concepts but to also ask should we?

9 Conclusion

We investigated both the application of visual methods to explore complex technology, and also the value of providing structure to the design and development phase of shape-changing interfaces. Novel requirement themes emerged such as the need for integration into existing technology, device personalisation and between device communication, alongside those already existing in technology. The process used has provided a helpful means of accessing thoughts on pre-existing technology in novice users, and may assist others in researching along these lines by using sketching and lo-fidelity prototyping – capitalising on the engaging and hedonic qualities of shape-change. Not only this, but the formal guidelines for design and development of shape-change could help researchers at the beginning of their work in prototype development. We also imagine that the technique of image generation and analysis requires further work to consolidate it as a viable technique in HCI research, perhaps in the context of other types of prototype or interface design.

Despite the focus of participants on the practical and not aesthetic aspects, aesthetics can be built in by designers and developers once the practicalities of the build are concrete – and the same can apply to emotionality. These particular elements of shape-change are evident in other work in the field, e.g. Kwak et al. [27] and Rasmussen et al. [44], so by integrating our work with that of others in future research, we can formulate a ’world-view’ of shape-change – made stronger still by examining existing processes in commercial design and engineering. This may help the field towards adoption rather than becoming trapped in a cycle of rapid development without fixed end-goals. The overall implications of this work relate not only to the eventual adoption of shape-change technology, but how this might change the way in which we consume technology in the future.

To summarise – shape-changing interfaces are complex, emergent technologies for which it is difficult to apply pre-existing processes – to address this, we used non-expert users to generate design requirements for these devices by sketching and diagramming their thoughts after working with video and white-box prototypes. The process produced multiple levels of design requirements which were discerned from open coding of the resulting images. These were categorised and analysed to form a stack model for shape-changing interfaces, which can be applied to future work in platform development. The findings demonstrate new ways of approaching the design of shape-changing interfaces and the continuing development of these highly complex computational experiences. Finally, we also considered the practicalities of adoption, and the long term implications of shape-change.