Building a Soft Machine: New Modes of Expressive Surfaces
- 1.7k Downloads
This paper will investigate how the distinct role of the textile designer can enrich the design process in HCI. It will advocate embodiment as a design methodology by focusing on a subjective, visceral engagement with material and physical computing using tacit textiles expertise. This theoretical premise is explored drawing on the fields of soft robotics, organic user interfaces and transitive materials for the fabrication of a responsive textile composite. The research uses a range of theoretical references to support its concepts of design thinking and computational materiality and deploys the methodological process of autoethnography as a qualitative system for collecting and evaluating data on the significance of textile thinking. This research concludes that there are insights gained from the creative practice experimental methods of textile thinking in HCI that can contribute to the commercial research and development field in wearable technology.
KeywordsMaterial Expressive surfaces Aesthetics Soft composite Soft robotics Wearables
The human body and its component parts are, within a consumer electronics cultural setting, frequently referred to as ‘body real estate’ . It is commodified, objectified and compartmentalized by companies and research laboratories eager to commercialize and exploit the ‘hot trend’ of ‘Wearables’  - smart watches, bands and headgear embedded with sensing capabilities, which quantify, measure and calculate. As an exhibitor at the International Consumer Electronics Show (CES 14, 15) and Consumer Electronics Week (CE 13), the author witnessed an emerging commercial interest from fashion and entertainment organizations seeking convergences. The majority of ‘wearables’ on display face issues of market unacceptability and statistical data designed in the context of industrial engineering design practices. The consequences, as exposed by the wearable technology trend of wristwear, are systems that fail to acknowledge the complexity of being in this world .
Technologies acting at the interface of the body in HCI, offer an opportunity to extend, explore and re-define user-experience. Yet the objectified standardization commonly found in a Cartesian-inspired design of human-machine integration can isolate us within a fixed chain of procedures and overlook opportunities to develop embodied, sensitive and nuanced methods of expression.
The embodiment theories of Maurice Merleau-Ponty  provide an alternative to René Descartes’ dualism of a radical separation of mind and body – thus offering a holistic lens to view and design a ‘soft machine’ through the active participation of the lived body in different stages of the design process. Moreover, digital critic Jaron Lanier argues in favor of re-designing machines by developing and articulating self-expression .
Since 2010, there have been key developments in the arena of turning wearable computing into forms that are more expressive. Increasingly, textile designers with an affinity towards technology and its multi-sensory capabilities are re-branding as ‘Materials-experience designers’, ‘Trans-disciplinary designers’ and ‘Materiologists’ as ways and means to describe their emerging position. It is within the commercial wearable technology field itself that we can now interrogate developments. Gaps identified include a lack of sensory ‘material’ awareness and experimentation. ‘Cute Circuit’, one of the leading players in expressive wearable technology has found success as a platform to popular culture through celebrity endorsement and soft electronics . However, a focus on advancing purely the technology (miniaturization of LED components embedded into garments) can result in a limited visual and sensory language. ‘The Unseen’, in contrast, employ a materials-led approach of ‘Material Alchemy’ experimenting with a range of responsive printed inks, such as heat-sensitive (thermo chromic), and wind-sensitive (piezochromic) on such natural materials as feathers and leather . Despite this, the brand defines its practice in opposition to the computational due to hard, clunky components and mainstream associations with consumer-led wearable technology.
This paper will position a relationship between both material and computational practices. Thinking through physical prototyping and reflexive practice is at the core of the textile designer’s work. Therefore, an engagement with the textile practice allows the exploration and experimentation of computational design through material, allowing the maker to express their tacit knowledge. Conversely, the computational, rather than being just a collection of clunky components used to measure and quantify, can be applied to ‘textiles thinking’ as a tool for experiential, interactive and aesthetic capabilities.
Subsequently, novel ways of designing the soft machine are now considered by contesting user-led design in favor of design through exploration. Firstly, through a contextual case study of the author’s work on wearable technology and secondly through a series of small-scale composite experiments in ‘visceral softness.’
1.2 Case Study of the Author’s Work
In 2010, the author founded ‘Rainbow Winters’, a brand seeking to express the emotive and aesthetic expression of technology . The sound-reactive ‘Thunderstorm Dress’ (2010), as an example, turned the wearer into a living Thunderstorm with an aim to create ‘visual music’.
John McCarthy et al. introduced the experiential as a type of user experience within HCI . The ‘visceral’ visual connection to experience, rather than usability, meant that the prototype reached a wider audience (fashion industry  and mainstream news ) than was usually prevalent amongst traditional function-led wearable technology. The ‘color-sensitive’ Garden dress developed at the Royal College of Art (2014); Craft-based techniques such as hand printing and embroidery were appropriated alongside a soft circuit, micro-controller and sensor as a method to blend the physical and the digital.
The physical experience of making each component of the prototype led to new awareness on the limitations of expression in current technology such as the LEDs and Electroluminescence, with their one dimensional, emissive and binary expressional capabilities. Yet, this binary method is alien for textile designers who work in a distinctive way using explorative, unplanned, non-linear, experimental, unknown and unforeseen outcomes. The relationship between the senses and arts practice can be defined by Sarah Pink’s sensory ways of knowing .
Could therefore, an inherently subjective and reflective qualitative design methodology, such as autoethnography, which explicitly seeks to evoke the experience of the maker, be placed within HCI?
2 Related Work
2.1 Material Lens HCI
Framed within a material lens, this paper intends to translate embodiment as a design methodology by engaging the human body beyond the conventional screen-pointer interface. The application of computational components to flexible, soft substrates blurs the boundaries between the physical and the digital, thus transforming our notion of materiality.
Several researchers have discovered the potential of Soft Materials within HCI  to perform as a convergence tool between the physical and digital. Anna Vallgårda et al. , and Robles and Wiberg  articulate a new vocabulary in computational material design by advocating a material strategy and position aesthetics as a driver for novel computational materiality . Whilst Frankjaer and Gilgen define the Soft User Interface [SUI] as technology ingrained within flexible materials .
We can, therefore, infer that there is a requirement for design to appropriate its own material language and methodologies in the research and development process of advanced materials – especially, if designers may concentrate on attributes such as form, expression and aesthetic. The Design-Science, Technology, Engineering, Maths model [D-STEM] proposed by Toomey et al.  calls on designers to apply intuitive and non-intuitive methods in the development of smart materials as part of their materials toolbox. An experiential understanding of these novel materials gives the designer new knowledge to understand innate characteristics and affordances . Ylva Fernaeus et al. for example, introduces ‘soft hard-ware’ as ‘electronic components, coating, and shells built from soft, malleable materials’ .
Under the wider term of ‘Programmable Materials’, the emerging fields of Transitive Materials , Organic User Interfaces , and Soft Robotics  can redefine our understanding of both computation and interaction .
‘Transitive Materials’ unite physical materials, and their unique and diverse properties, with the computational . These novel substrates fused with computational properties offer an opportunity to explore interaction through adaptive surface properties  stimulating both visual and tactile senses, which may not be apparent in flat interfaces with emissive light changes.
Organic User Interfaces (OUI) can be defined as a user interface with a non-flat display. Holman et al. use the textiles analogy of knitwear to ask what computers would look like if designed with a delicate sensibility  and explore the physical materiality of paper as being malleable and possessing haptic, visual cues.
Traditionally robots are comprised of inflexible, hard material and joints. The emerging field of Soft Robotics uses flexible materials such as soft elastomers embedded with sensing and actuating properties .
Rolf Pfeiffer et al. advocate embodiment as a methodology in bio-robotics; embodiment becomes an ‘enabler for cognition’ , especially as soft robotics use flexible and soft materials which require manipulation . Further, composite structures are used as a technique in soft robotics to combine the mechanical properties of different materials . Pneumatic networks, for example (networks of small channels in elastomeric materials)  can open design opportunities within wearables due to their flexible and conformable nature.
Until now, soft robotic applications have been limited to function-led products such as a pneumatic glove for hand rehabilitation . Within the textile domain soft robotics remains under-explored even though both subjects focus on the innate characteristics of ‘soft’- stretchy, malleable, fluid, tender, manipulated, adaptable, sensorial, sensation and highly tactile.
2.2 Textiles Thinking
What assumptions can we challenge for textile design? Its ubiquity and lack of critical design discourse hide an effective methodology, which is ripe for development, to re-define user experience within HCI. By moving the machine into the realm of ‘soft’ - adaptable, shape-shifting and ubiquitous textiles, this paper will pose the question: ‘why does ‘soft’ matter?’
This study is based on the premise that the textile designer can appreciate and respond to complex emotional and aesthetic influences. Claire Pajaczkowska argues that an embodied relationship with material stimulates a distinctive type of thinking . As illustrated by Textile designer, Rachel Phillpott, whose practice is characterized as a playful manipulation of materials driven by embodied knowledge with the potential to expand from an anticipated guideline in creating new inventions .
Sensibility within the context of re-defining user-experience has been represented by McCarthy et al. as sensory fluency . This particular sensitivity to material experience detailed as delicacy, subtlety, intuition, feeling, responsiveness and perceptiveness, is embedded within the tacit knowledge of the textile designer who makes use of a range of sensory clues.
Textile theorist Elaine Igoe contests Nigel Cross’s non-acknowledgement of methods used by designers who have a close understanding and unique relationship with materials (textiles, fashion, jewellery, ceramics, glass). The designers reviewed by Cross in developing ‘Designerly ways of thinking’  all share a similar background (industrial, product, engineering design) and apply user-led design methodologies which focus on functionality and practical applications. Igoe asserts that the textiles design process needs to be articulated to validate a critical awareness of its distinctive type of design thinking with its own ‘specific methodologies’ . This utilization of tacit textiles knowledge is transformed to create surfaces, which are aesthetic, haptic and engage directly with the subjectivity of the maker .
Textiles sensibility needs to enter a dialogue with HCI to enrich the discourse in wearable technology and as a method and process of innovation within a growing commercial arena, which needs further validation. In this light, McCarthy et al. align with Igoe in placing equal importance on sensation and cognition . Materials can play an essential part in brand positioning therefore, qualitative perceptual qualities such as tactility, visual impact and user experience play a role . Donald A Norman’s framework for analyzing design on an emotional level constitutes the following: Design evokes an emotional response at three different levels: the visceral, the behavioral and the reflective . Behavioral design relates to functionality and usability favored by traditional HCI user-experience. The ‘visceral’ relates directly to our experiential and sensorial experience where the material rules supreme - physical features (look, feel, sound) prevail (shape, form, physical feel and texture).
Industrial design can find design processes used in the more affective disciplines such as textiles, fashion and theatrical design challenging. Norman characterizes emotions as fundamental to fashion  yet translating a mood, expression or feeling is inherently subjective and not quantifiable; film-makers for example do not start out their process with earnest user studies and insights. Therefore, the nature of industrial design and the design methods with a pre-disposition towards use and usability mean expressive and emotional methods are often overlooked. This user-led approach to design, which Cross advocates tends to reduce people to grouped commonalities, which this research argues, would by default, reduce expressional capabilities. Norman aligns with this approach claiming that user-centered design is not always appropriate for either visceral or reflective design .
2.3 Affect and Visceral Engagement with Material
This research frames the design of the soft machine within the ‘Visceral’ layer of emotional design as charted by Norman . Visceral is a theme often found in the context of theater and film for the creation of ‘affect’. Youn-Kyung Lim et al., for their part, explore emotional experience in interaction design - using Norman’s definition of ‘Visceral’ as ‘perceptually-induced reactions’ which relate directly to our physical senses .
Positioning textiles as an interface for the experiential, can we imagine a ‘visceral soft system’ that engages our body and its sensory experience: optical, acoustic, olfactory, tactile, thermal, kinaesthetic, proprioceptive, visceral and cross-modal? An experiential textiles interface could be described as translating one subjective experience into another, a human transitional experience. However, in traditional computation the computer engineers build the tools that are invariably technically driven  and designers/artists are thus limited to using these tools.
Framing the design of the soft machine within the ‘visceral’ layer of emotional design offers an opportunity for expressive surfaces to be fabricated through programmable matter. Norman speaks of future machines as layered systems of affect . The textiles designed through ‘visceral softness’ are designed for purpose, to translate a ‘visceral experience’. They cannot be replicated through standard off-the-shelf materials but go through a range of textile processes by the designer.
Textiles have the potential, within the field of HCI, to offer magical, functional, ubiquitous, accessible, intimate, curious and multi-dimensional behaviors. McCarthy et al. for example, re-examine user-experience as felt experience through the concept of ‘enchantment’. They question how designers can use this tool in the design of interactive systems within HCI to build affective attachments through the sensuality of the artefact, a playful regard of material and the potential of its transformational qualities . Ling Yao et al. have developed a technical framework for pneumatically actuated composites illustrating for example dynamic texture change and composite sensing layering . A function-led approach of developing potential application such as a shape changing mobile is employed. However, the point of departure in this paper is located within ‘sensory making’ as described by Pink with particular attention to sensory perception and experience . We build on Yao’s framework by fabricating interactive pneumatic and fluidic surface design within specific ‘textiles thinking’ methods. Developing the ‘sensory engagement’ layers of the composite through experimentation with surface design, color, texture and pattern.
3 Soft Responsive Composite Textile
Through a textile design composite, we can now introduce the term ‘visceral softness’ and advocate an approach that builds the ‘machine’ on the principles of embodied interaction.
The ground-work for the ‘textile thinking’ experimental process within HCI is laid out as a detailed set of instructions in the context of Autoethnography, a distinctly subjective iteration of traditional design thinking models that includes human experience  as well as the embodied experience of materials. We draw on Pink’s notion that sensory participation is a reflexive and experiential process . By seeking out directly subjective, tacit knowledge, the author is thus positioned as a researcher and participant in a method that encompasses human experience, experience of materials, as well as problem solving.
The instructional method serves to translate tacit textile knowledge into an explicit and reproducible process. Further, we wish to build on an emerging methodology within HCI to translate the experiential; Kristina Hook for example uses autoethnography as a tool to convert embodied experiences into new knowledge .
3.2 Composite Experiments: Visceral Softness
This initial stretchable composite prototype uses off-the-shelf materials as a ‘technology blue-print’ using a 3 V air motor, 12 V peristaltic pump, sound sensor and proximity sensor. The composite layers comprise of (1) Active texture and (2) Fluidic Actuation. To move the composite into an autonomous soft prototype, we propose in the next iteration to include the layers of (3) soft circuit and (4) battery. The ‘sensory engagement’ layers (active texture and fluid actuation) combine traditional textile craft techniques used in print and mixed media (surface disruption, layering and interruption).
This materials-engaged and explorative method of working builds upon emerging approaches, which consider the embodied and experiential value within HCI as opposed to a function-orientated, user-centred approach. Examples include Heekyoung Jung’s ‘materiality understood through exploration rather than use’ [41, 42], Felicia Davis’s, ‘research through material’ , and Coelho et al. fusion of new technologies with craft as a basis for manifesting unexpected design opportunities .
3.3 Instructional Method for Building a Soft Machine
These instructions have been adapted from the ‘Soft Robotics’ Toolkit, an open-source resource to contribute to the advancement of soft robotics from the Whitesides Research Group, Harvard University . In our process, we use a metaphor of cooking, with a range of options to be adapted for each layer.
Required materials: silicon (Eco-flex 00-30), 3 V pneumatic mini pump, 12 V peristaltic pump, 1 mm I.D x 2 mm O.D clear translucent silicone tubing (for fluidic layer but size is interchangeable), 1.5 mm I.D x 3 mm O.D polyethylene tubing (active texture layer), non-woven fabrics Vilene (Lutradur 30 gm).
Optional materials: silicone pigments, colored powders, fibers (in different colors, we used fibers also found in applications such as prosthetic make-up, model-making garden fiber), 3D printed molds for elastomer channels.
Tools: scissors, pins, cutting knife, metal baking tray, disposable gloves, wooden stir sticks, soldering iron, latex gloves, mixing/measuring cups.
Optional: sublimation printer, heat-press, 3D printer for molds.
Step One: Inspiration
Devise a mood board (a collection of textures, colors and images) to capture thoughts, glimpses, nuances and impressions. This experiment used for example macro-photography to explore surface through organic forms, texture, and the secret enchantments of the natural world. Textile designers often use found materials or pictorial experiences to convey a tone or atmosphere and will embed physical tactile references and unconventional materials into their experimental processes to create brand new forms. This mood board will be the blueprint for feeding into the ‘sensory-engagement’ layers of the composite. Images and thoughts expressed in a quick, instinctive, visceral manner.
The composition of the mood-board invited a story of interaction to investigate experiential textiles creating novel interactive expressive surfaces. Suggestive ideas started to emerge such as skin and water gasping, breathing acoustics, swarm, bass, beat.
Step Two: Electronic System.
Step Three: Composite Fabrication
Sublimation print to a non-woven polyester (Lutradur 30). Heat from a soldering iron can be used to distress the material and re-imagine surface qualities, structures and textural qualities.
Mix part A and part B Ecoflex into a measuring cup equally. Spread a very fine layer over the printed fabric samples. Cure for four hours.
Pour a layer of EcoFlex on a baking tray (layer 1), pigments and fibers can be mixed to create a variety of tensile strengths. Mixed-media layering involves freely combining any and all kinds of materials that you want to use in creating the composite. Painterly methods of layering, dripping, pouring, and glazing enable expressivity and the multi-layer structures evoke a tactile quality where human-made imprecision is a desired part of the result (Fig. 3).
Active texture layer: Coat the edges of (layer 1) with a small amount of Ecoflex to bond and place on the cured printed fabric sample (this forms as an adhesive) (Fig. 5). Cure for four hours. Use a pin to insert a hole for tubing into layer 1.
- Fluidic layer: fix silicon tubing onto the printed fabric sample with EcoFlex. Here we use fluids as a display surface as an alternative to compressed air (Fig. 4).
Step Four: Testing.
Together with the aforementioned Design-STEM model proposed by Toomey et al., this method calls for designers to assert both a sensory and a technical fluency in the development of smart materials. In a commercial context this would mean that the designer, rather than accepting a submissive role in STEM developments, would instead move ‘to one of active influence and effect from within this arena’ . ‘Textiles thinking’ is discovered through a direct visceral engagement with physical materials and experimental technology to uncover subjective capabilities such as imagination, touch and material manipulation.
3.4 Future Development
Soft, stretchable electronics have been defined by Rus et al. as the next stage in the growth of soft robotics and material machines , which our next development would hope to exploit.
The results obtained from the literature survey and autoethnographic approach to designing a soft responsive composite, indicate that this may only be the beginning of a discourse into how ‘visceral material exploration’ contributes to the design discourse of Wearables. An evaluation strategy will be constructed to survey key stakeholders including textile designers, R&D departments and commercial tradeshow platforms such as CES. Alongside autoethnographic data, this will provide further evidence that design usability can be expanded to include experience, advocating a material based approach to constructing the soft machine.
The author would like to thank Clive Hudson at Programify for support with electronic circuit fabrication and coding.
- 1.Dvorak, J.: Moving Wearables into the Mainstream, p. 107. Springer, Heidelberg (2008)Google Scholar
- 2.Marketsandmarkets.com,: Wearable Technology Market by Product - 2020| MarketsandMarkets. http://www.marketsandmarkets.com/Market-Reports/wearable-electronics-market-983.html
- 3.Damasio, A.: Descartes’ error, p. 249. Vintage, New York (2006)Google Scholar
- 4.Merleau-Ponty, M.: Phenomenology of Perception. Humanities Press, New York (1962)Google Scholar
- 5.Lanier, J.: You Are Not a Gadget, p. 191. Alfred A. Knopf, New York (2010)Google Scholar
- 6.CUTECIRCUIT: tshirtOS. http://cutecircuit.com/tshirtos/
- 7.Seetheunseen.co.u: T H E U N S E E N. http://seetheunseen.co.uk
- 8.Rainbowwinters.com: Rainbow Winters – homepage. http://www.rainbowwinters.com
- 9.McCarthy, J., Wright, P.: Technology as Experience. MIT Press, Cambridge (2004)Google Scholar
- 10.Steele, C.: PCMag’s Coco Rocha Explores CE Week’s Wearable Tech. http://uk.pcmag.com/consumer-electronics-reviews-ratings/14462/news/pcmags-coco-rocha-explores-ce-weeks-wearable-tech
- 11.Walker, R.: Fashion’s big brands follow the money to join the wearable tech revolution. http://www.theguardian.com/technology/2015/feb/14/fashion-phones-wearable-technology
- 12.Pink, S.: Situating Sensory Ethnography: From Academia to Intervention. In: Doing Sensory Ethnography. pp. 7–22 (2009)Google Scholar
- 14.Vallgårda, A.: A material strategy: exploring material properties of computers. Int. J. Des. 4, 3 (2010)Google Scholar
- 15.Robles, E., Wiberg, M.: Texturing the “material turn” in interaction design. In: Proceedings of the Fourth International Conference on Tangible, Embedded, and Embodied Interaction - TEI 2010 (2010)Google Scholar
- 16.Vallgårda, A., Redstorm, J.: Computational composites. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI 2007 (2007)Google Scholar
- 17.Frankjaer, T., Gilgen, D.: Wearable Networks, Creating Hybrid Spaces with Soft Circuits. In: Marcus, A. (ed.) DUXU 2014, Part II. LNCS, vol. 8518, pp. 435–445. Springer, Heidelberg (2014)Google Scholar
- 18.Toomey, A., Kapsali, V.: D-STEM: a design led approach to STEM innovation. In: A Matter of Design: Making Society through Science and Technology (2014)Google Scholar
- 19.Fernaeus, Y., et al.: Touch and feel soft hardware. In: Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction - TEI 2012 (2012)Google Scholar
- 20.Coelho, M., et al.: Programming reality. In: Proceedings of the 27th International Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA 2009 (2009)Google Scholar
- 22.Jacob, R., et al.: Reality-based interaction. In: Proceeding of the Twenty-Sixth Annual CHI Conference on Human Factors In Computing Systems - CHI 2008 (2008)Google Scholar
- 26.Pfeifer, R., et al.: How the body shapes the way we think, pp.19, 30. MIT Press, Cambridge (2007)Google Scholar
- 28.Polygerinos, P., et al.: Towards a soft pneumatic glove for hand rehabilitation. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (2013)Google Scholar
- 29.Pajaczkowska, C., Jefferies, J., et al.: The handbook of textile culture, making known, The Textiles Toolbox- Psychoanalysis of Nine types of Textile Thinking, p. 79. Bloomsbury Academic (2015)Google Scholar
- 30.Philpott, R.: Engineering opportunities for originality and invention: the importance of playful making as developmental method in practice-led design research. Studies in Material Thinking (2013)Google Scholar
- 32.Cross, N.: Design Thinking, p. 67. Berg, Oxford (2011)Google Scholar
- 34.Schifferstein, H., Wastiels, L.: Sensing Materials. Materials Experience, pp. 15–26 (2014)Google Scholar
- 35.Norman, D.: Emotional design, p. 63, 93, 97. Basic Books, New York (2004)Google Scholar
- 37.Berzowska, J., Bender, W.: Computational expressionism, or how the role of random () is changing in computer art. In: Proceedings of SPIE 3644. Human Vision and Electronic Imaging, vol. 3644 (1999)Google Scholar
- 38.Yao, L., et al.: PneUI. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology - UIST 2013 (2013)Google Scholar
- 39.Ellis, C., et al.: Autoethnography: an overview. Forum Qual. Sozialforschung / Forum: Qual. Soc. Res. 12, 1 (2010)Google Scholar
- 40.Höök, K.: Transferring qualities from horseback riding to design. In: Proceedings of the 6th Nordic Conference on Human-Computer Interaction Extending Boundaries - NordiCHI 2010 (2010)Google Scholar
- 41.Jung, H., Stolterman, E.: Digital form and materiality. In: Proceedings of the 7th Nordic Conference on Human-Computer Interaction Making Sense Through Design - NordiCHI 2012 (2012)Google Scholar
- 42.Davis, F.: The textility of emotion. In: Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition - C&C 2015 (2015)Google Scholar
- 43.Softroboticstoolkit.com: Design. http://softroboticstoolkit.com/book/pneunets-design
- 44.Bartels-mikrotechnik.de: Bartels Mikrotechnik GmbH – Home. http://www.bartels-mikrotechnik.de