Keywords

1 Introduction

Affordance is a ubiquitous term in interaction design whose– very rough and high-level–meaning designers seem to have an understanding of, but whose precise semantics has been the subject of intense refinement, discussion, and disagreement [7, 8, 12, 14, 18, 20–22, 26, 30–32, 34]. Regardless of the specific semantics, awareness of, and manipulation of, affordances is used in practice by designers. In addition, specific interpretations of “affordance” may be used for analysis of the properties of an artefact, for example–as suggested in [18] for uncovering usability problems in finished products or prototypes, and hence employed in re-design. In a different vein of research, the interaction and design opportunities for objects that are working, broken down or otherwise damaged, have recently been the subject of intense scrutiny with researchers starting to explore the design space around objects that are already broken, or even deliberately designing both physical and virtual objects with broken or decayed parts [15–17, 19, 24, 27]. The aim of this paper is to explore and analyze the implications of a designer–deliberately or inadvertently–breaking one or more affordances of an object—as opposed to breaking the object itself. We posit that among the many extant definitions of affordance, only some are operational in the sense that they allow for non-trivial analysis, and experimental exploration, of breakage, and we take our departure in the notion of instrumental technology affordance [18] and its constituent elements of handling affordance and effecter affordance. Through analysis of the difference between breaking affordances of physical and virtual, and through two explorative experiments, we reach two conclusions: (A) that systematically breaking the different facets of instrumental affordances, either by thought experiment or by prototyping, can be a useful tool for analyzing and refining interface design, and (B) that there is an untapped design space to be explored in breaking affordances as part of deliberate design of objects that are not intended to be ephemeral; and that this design space is distinct from one where brokenness is a quality of the object. Space constraints prevent us from treating all the reasonable variations of the difficult concepts of affordance and brokenness, from treating the notions of physical and virtual as anything but mutually exclusive. We are convinced that our analysis, and our assessment of the possibilities of the design space of brokenness, may be extended to encompass many of these variations.

1.1 Related Work

Previous studies have investigated user interaction and coping with broken (hardware) interfaces [27], investigating continued use and re-purposing of damaged artefacts, and have explored design possibilities [13]. Ikemiya and Rosner [15] investigate design strategies using deliberate wear and degradation of objects to explore how to use breakage and wear to inform and cultivate design practices. Jackson and Kang [16], and Kang et al. [17] investigate interactions with broken and discarded technologies by letting artists explore disassembly, reassembly and design, and having people playfully interact with the resulting artworks. However, none of these studies analyze the strategies of users when interacting with specific broken affordances, or the differences between broken affordances in physical and virtual objects. A related vein of research considers digital heirlooms and virtual objects with which users have emotional attachment, e.g. Kirk and Sellen [19] and Odom et al. [24], but consider mostly data (e.g., electronic diaries or photographs) rather than technological mediators such as programs, making for a very different set of affordances; in this vein of research, Gulotta et al. mention the mostly untapped design space of using wear or decay on virtual objects [11], but from an aesthetic perspective rather than treating breakdown or obsolescence of affordances. De Souza, Prates and Carey [6] consider the actions of users missing affordances (essentially equivalent to users not perceiving affordances intended by designers, and equivalent to the defective signifiers treated later in this paper) and declining affordances (deliberately not acting on affordances intended by designers). Finally, Oshlyansky et al. [25] investigate the effect of culture on perception of and interaction with affordances, but do not consider deliberate breakage.

2 Affordances in HCI and Other Disciplines

The psychologist J.J. Gibson first coined the term “affordance” [9, 10] as part of an ecological alternative to cognitive perception. Cognitive perception posits that an individual only has direct access to sensations, and that those sensations are integrated with memories, which in turn build up symbolic representations of the environment and its potential for goal-oriented action [7]. Ecological perception, on the other hand, suggests that the environment is not merely a physical environment. In ecological perception, an individual collects information from a meaning-laden environment, which provides, amongst other things, affordances. Gibson wrote that “The affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or ill” [10, p. 127]. Several post-Gibsonian definitions were proposed in ecological perception (see, e.g., [5, 29, 32]).

2.1 Affordances in Human-Computer Interaction

Norman applied the term “affordances” to everyday artefacts in his book “The Psychology of Everyday Things” [21] as part of an attempt to understand how humans manage in a world with thousands of objects, many of which we encounter only once. In later clarification, Norman calls his original affordances “perceived affordances” [22]. In this view, roughly, “affordances are the fundamental”, actual properties of an object that define how it can be physically interacted with. These properties are relational properties of the environment, that exist independently of whether a particular user perceives them or not. In contrast, “Perceived affordances” are actions that a user perceives to be possible and meaningful. These perceived affordances inhere solely in the user, but can be “encouraged” or suggested through good design. Under this–strict–view of a perceived affordance, it is not possible for an external agent to create or add a perceived affordance, as these depend solely on the user’s perception of the situation and not what is actually possible. Gaver explores the notion of affordances and the role of perception and context [7], separating affordances from their perception, and accounts for ‘complex’ affordances through exploration and sequential/nested affordances. To Gaver, the perception of affordances is determined in part by the observer’s culture, social setting, experience and intentions [7]. After Norman and Gaver, many authors have attempted to clarify or re-conceptualize the notion of affordance in technology and interaction (a sample: [7, 14, 20–22, 26, 29–32]). We briefly treat some of the most influential of these. McGrenere and Ho detail the differences between Gibson’s and Norman’s notions of affordance and argue for a grading of the presence of affordances rather than a binary view where the affordance is either there or not, and advocate separating affordances from the perception of them. Bærentsen and Trettvik suggest using activity theory for conceptualizing affordances and advocate using a much more general and encompassing notion of activity than earlier work. Turner [31] suggests expanding the notion of affordance to cover both the classic Gibsonian notion of affordance and the context of use. Conceived from a socio-cultural approach inspired by Vygotsky [35] and theoretically grounded in mediated action, Kaptelinin and Nardi argue that Gibson’s concept of affordances is purposefully limited in scope; they hold that the limited focus of Gibson’s affordances fits into his conceptual framework of ecological psychology, a framework which is difficult to work within for HCI [18].

2.2 The Instrumental Affordances of Kaptelinin and Nardi

Kaptelinin and Nardi identify tool usage as a particularly problematic area of the Gibsonian view in which tools are presented as part of the animal-environment system. Kaptelinin and Nardi [18] find this insufficient for three reasons (i) Gibson does not recognize that the facets of a tool may be independent of one another, particularly in technology, (ii) Gibson’s affordances do not deal with the social aspects that may occur through use and production of tools, and (iii) in Gibson’s view, tool usage is unrelated to the capabilities of the actor.

In the mediated action perspective, technology (tools) can be a mediational means between an actor and the environment, in the pursuit of a goal. If we think of technology as an instrument, Kaptelinin and Nardi suggest that such an instrument offers two related facets: (a) possibilities for interacting with the technology, handling affordances; and (b) possibilities for using the technology to cause an effect on an object, effecter affordances. These two facets define Kaptelinin and Nardi’s instrumental technology affordances: possibilities for acting through technology on an object. For example: a pair of scissors offers an instrumental affordance of cutting, consisting of a handling affordance (scissor handle) and an effecter affordance (the blade of the scissor). This concept extends to virtual technologies as well. For example: a scroll-bar offers dragging either horizontally or vertically (handling affordance), and this typically scrolls an object of interest, e.g. by moving what part of something is visible appropriately (effecter affordance). More generally, software widgets are comprised of these two facets: a handling affordance for interacting with the widget, and an effecter affordance which is the consequence of interacting with the widget.

Kaptelinin and Nardi distinguish between instrumental affordances as defined above, auxiliary affordances “determined by the embeddedness of a technology in ‘webs of mediators’ typical of real-life uses of technology [4], and learning affordances. Examples of auxiliary affordances are maintenance and aggregation affordances concerned respectively with the maintenance of objects and the possibility of actions afforded by several objects acting in concert (e.g., a mobile phone connected to a headset). Learning affordances are the affordances of objects by which users understand how to act on instrumental and auxiliary affordances, for example tooltips, standardized icons, or embedded manuals in software.

We stress that the notion of instrumental affordance is operational in the following sense: the separation of instrumental and auxiliary affordances, and further subdivision of instrumental affordance into handling and effecter affordance allows for separate design considerations to be made to each of the constituent parts. We believe that this clear subdivision (a) more readily affords analysis and possibilities of re-design than the psychology-focused approaches of both the earlier Gibsonian notions and the later refined work by several authors (see [20, 30] for related discussions), (b) is better specified and well-grounded than the ground-breaking, but early, approaches of Norman [21–23] and Gaver [7], and (c) complements earlier operationally-oriented approaches to interface design that did not specifically focus on affordances, for instance [3].

3 Making Perception Explicit

Whereas perception does not play an explicit role in the model of Kaptelinin and Nardi, the user’s perception of instrumental affordances becomes a tricky issue when devising an operational definition of breaking (see Sect. 4); we now briefly discuss and make the issue of perception explicit. It is fruitful to consider the typical timeline involved in the activation of an affordance: the user needs to perceive the presence of the affordance (and, consciously or not identify it correctly), whereupon the user activates the handling affordance, which subsequently activates the associated effecter affordance, and finally the user receives feedback. Thus, temporally:

Perceive and understand signifier(s) → Activate handling affordance → (effecter affordance) → Receive feedback

The parentheses around effecter affordance above are a reminder that the effecter affordance need not be directly perceivable to the user. We stress that the above simplified sequence can be replaced by more refined models of interaction, for example by Norman’s classic Stages of Action model [21]. The word “signifier” here means the cues to the user about the entire instrumental affordance; more fine-grained notions exist, for example Vermeulen et al., though not working within Kaptelinin and Nardi’s framework, consider signifier to–roughly–mean cues about the handling affordance and feed-forward to mean cues about the effecter affordance [33]. For some affordances, there may be no perceivable difference between some of the steps above: for example, in a soda vending machine there is the instrumental affordance of seeing a button with the desired soda and recognizing it as such (signifier), pushing the button (handling affordance) and receiving a soda bottle in the dispenser tray (effecter and feedback). As an aid for better user-centered design, and as a guide for exploring the design space, we argue that all steps should be considered separately, echoing McGrenere and Ho [20].

Following Norman, we stress that signifiers are distinct from affordances; indeed, Norman argues strongly that the notion of affordance should be replaced by signifier, leading to better design. In the mediated action perspective of Kaptelinin and Nardi, signifiers do not seem to play a role insofar as they are not part of the reconceptualization of the notion of affordance [18], and the issue of the “Role of perception” is relegated to something that “can be an outcome of learning” [18, Table 2].

From a design perspective, there is an obvious incentive to explicitly consider signifiers: signifiers tell the user that a specific affordance is present, based on learning and context; and as we make clear in the discussion on breaking affordances in Sect. 4, there is ample reason to consider signifiers as distinct phenomena from instrumental affordances when exploring the design space unfolding when breaking affordances.

While signifiers serve as cues that an affordance is present, and is perceived by the user prior to acting on the affordance, affordances may also have one or more feedbacks serving as cues that the intended action was performed (i.e., that the affordance indeed was present when the action was taken). For some technology artefacts, in particular physical ones, feedbacks associated to affordances may be obvious: when pressing a selector button on a soda vending machine (the handling affordance), the desired soda bottle drops to the dispenser tray (the effecter affordance, and the feedback). For some affordances, feedback may occur long after the affordance has been acted upon (e.g., submitting a physical reimbursement form to a university bursar, not receiving any receipt, and only observing the reimbursement appear on a back account some time later). For affordances of virtual objects, feedbacks for simple interface affordances such as scrollbars or widgets for closing windows, may also be tightly bound to the effecter affordances: the window scrolls, or closes. However, interaction with modern devices are replete with instrumental affordances whose handling affordance is simple (e.g., a button), but whose effecter requires separate feedback: consider the instrumental affordance of submitting an online form by pressing a button labeled “submit”; the form may be submitted and end in the right place on a system elsewhere in the world, but the user has no way of knowing unless some explicit feedback is given (e.g. a confirmation page appears, a confirmation email is sent to the user’s account, etc.).

4 Breaking Affordances

We now propose an operational notion of brokenness and describe how it can be used to understand the actions and propensities of users interacting with technological affordances.

4.1 What Is Brokenness?

We define a technology affordance to be broken if (i) the designer intended the affordance, and the user either (iia) perceives the presence of the affordance but the affordance is not present, or (iib) perceives that the affordance was supposed to be present, but is not present (the user may or may not discover the brokenness after attempting to activate the affordance).

As affordances in the mediated action perspective may depend on cultural or other contexts, different users may perceive distinct affordances. We thus view the property of being broken as relative to the context, and relative to the particular user. An example: a user perceiving an obviously smashed “on” button on a TV, sees a broken (handling) affordance. A user perceiving an unbroken “on” button that fails to turn on the TV when pressed sees a broken (effecter) affordance. The above definition of broken is absolute and binary: the affordance is completely absent if broken (and completely present if unbroken). For many affordances, in particular aggregate affordances, it may be fruitful to consider grades of brokenness: For instance, Schaub et al. [27] show that many users still use smartphones with cracks in the display for viewing and interaction, and that more severe cracks negatively affect the kind, and the quality, of interactions more than less severe ones. However, a full discussion and exploration of these issues is beyond the scope of this paper.

Note that we consider brokenness to be a quality of a technology affordance intended by a designer (hence, not of natural objects as in the Gibsonian view, and not of unintended or emergent affordances). These restrictions in scope are due to our focus on the operational aspect of affordance to designers and to reduce the number of special cases to be considered. For example, if we removed “the designer intended the affordance” from the definition, the perceived presence of an affordance where there is none can be due to a defective signifier (e.g., a graphical widget with the visual appearance of a button, but with no underlying functionality at all); this is an interesting phenomenon, but beyond the scope of the paper.

4.2 Breaking Handling and Effecter Affordances Separately

In an instrumental affordance, the handling affordance can–informally–be “shared” between several instrumental affordances with different effecter affordances: (opening) the lid of a physical mail box may afford dropping of envelopes to adults and dropping of firecrackers to wayward children. More subtly, there is a definitional choice of whether to allow the theoretical possibility of distinct instrumental affordances having different handling affordances, but sharing an effecter affordance.

For example, highlighting a specific entry field in a virtual text entry form can typically be done by tabbing through fields (if the window is already highlighted), or by selecting the field with a pointing device–two distinct instrumental affordances with distinct handling affordances, but accomplishing the same action. While breaking a handling affordance breaks the instrumental affordance that it is part of, several instrumental affordances may have effecter affordances accomplishing the exact same goal, and a user may often still complete the desired action by switching to a different instrumental affordance of the same object (e.g., use a keyboard shortcut or a drop-down menu instead of pressing a button). If a user acts using two distinct instrumental affordances of the same object with a result indistinguishable to the user, we say that the two effecter affordances are equivalent, see Fig. 1. To avoid confusion due to inconsistent language, we also say that two handling affordances are equivalent if they are indistinguishable to the user (but their associated effecter affordances may be different), see Fig. 2.

Fig. 1.
figure 1

Two instrumental affordances with equivalent effecter affordances but distinct handling affordances: opening the print dialog of a word processor, using a keyboard shortcut (left) and by pressing the left mouse button (right). In both cases, the effecter affordance is a print window.

Full size image
Fig. 2.
figure 2

Two instrumental affordances with equivalent handling affordances but distinct effecter affordances. Left: handing a piece of paper to someone. The handling affordance is grasping the paper, and the effecter affordance the giving of paper. Right: tearing a piece of paper. The handling affordance is grasping the paper, and the effecter affordance is the paper being torn.

Full size image

The reader may deem the notion of “equivalent handling affordances” unnecessary: in most mediating technology objects, two (visually or otherwise) similar handling affordances are cognitively associated with distinct instrumental affordances unless the user is given explicit cues that it is not the case. For example, two distinct, but identical-looking, buttons are for accomplishing two different things, but a single button (or a mailbox lid) with two distinct effects is cognitively a single entity, and it may seem to be useless semantics to talk about handing affordance of opening a mailbox lid to “be” two distinct, but equivalent affordances.

The notion of “equivalence” becomes clearer when considering what happens if handling or effecter affordances are broken: For many simple mediating tools, breaking the handling affordance also breaks the effecter affordance: consider a hammer where the handle is burned off–the hammer no longer affords hammering. However, for more complex tools, the handling affordance may appear broken upon activation, but the effecter affordance could still be present: when pushing a button on a touch-sensitive keyboard, it may not depress, but the touch sensor could still send the appropriate signal and activate the effecter affordance; if in doubt, the user may often use another input device (say, a pointing device) to accomplish the same effect as pushing the button (i.e., act on an affordance with effecter equivalent to the one activated by the button push). For complex instrumental affordances, the effecter affordance of an instrumental affordance in general is easy to break without breaking the handling affordance: for a software object, a button may work, but the intended effect, e.g. submission of a form, may not occur.

4.3 “Breaking” Perception: Defective Signifiers and Feedbacks

In investigating the actions of users, and in particular the design space associated to, breaking affordances, we argue that signifiers and feedbacks have an important role to play independently of affordances: Namely that signifiers and feedback can be changed to signal that an affordance is broken (in the above sense), or can them-selves be defective in several ways without the associated affordance being broken. Consider the instrumental affordance of submitting a data form by clicking on the ’Submit’ button. The handling affordance is pressing the button (by a mouse click), and the effecter affordance is the (supposed, but invisible to the user) submission of the data. There is at least one signifier present: The widget suggesting a button with the word “submit” is a signifier of the handling affordance. There are–at least–the following scenarios where the signifier or feedback is somehow defective, but the associated instrumental affordance is affected in different ways: (a) the button no longer visually depresses, and so does not alert the user to having been pressed /activated correctly. But the button action still activates correctly (handling and effecter affordances work, feedback is defective); (b) the computer responds, on pressing the submit button, with an ‘Error’ beep, as if the user is trying to press or activate something they cannot. But the button action still activates correctly (handling and effecter affordances work, feedback is defective). (c) The button is greyed out from the beginning, i.e. the signifier introducing the button as clickable is defective from the get-go. But the button action still activates correctly (handling and effector affordances work, signifier is defective). (d) The button is visually marred (e.g., by “cracked” graphics or an unintended visual artifact such as a red “X” across the button), but the button still activates correctly (handling and effector affordances work, signifier is defective). (e) There is no follow-up result from clicking the ‘Submit’ button, alerting the user of the success or failure of their submission (handling and effecter affordances work, feedback is defective). As the above example shows, signifiers and feedback can be defective independently from brokenness of handling and effecter affordances. The philosophical question of whether an affordance exists if the user does not perceive its existence is beyond the scope of this paper, and we have consequently deftly defined brokenness in such a way that if a user does not perceive an affordance to be there, then it cannot be broken. The technology may fail to work in an absolute sense, but this does not matter to the particular user if she was never conscious of the possibility of using the technology for a particular purpose. Hence, a sufficiently defective signifier prevents an affordance from being broken.

We posit that there is a very useful distinction between considering the user’s perception of an affordance (the signifier(s) and feedback) and the user’s possibility to carry out an action (the affordance). Whereas the notion of “broken” can be applied in an absolute sense to affordances (they “work”, or “do not work”), the question of the user’s perception is more subtle, hence our use of the word “defective” for signifiers and feedback instead of “broken”. For example, a signifier can be missed by the user, or refer to another affordance entirely, or misdirect leading to accomplishing the wrong goal. “Breaking” a signifier in an absolute sense–i.e., either the user perceives the affordance or not–is less interesting as an analysis tool than breaking an affordance; rather it is an important question of good design, which we do not treat here.

5 The Affordances of Objects with Broken Affordances

For some simple tools, their instrumental affordances are tightly coupled with their identity: a hammer that cannot be used for hammering is not a hammer (and for most hammers, the affordance of being used for hammering, is the only intended affordance). For complex tools, say a car, or a word processor, the designers intended a multitude of instrumental affordances for accomplishing subtasks and interacting with the tool. Some high-level aggregate affordances may still be closely tied with the identity of the object (a word processor that cannot be used for writing is not a word processor), but low-level instrumental affordances may potentially be broken without the object itself losing its identity. Thus, an object may have broken affordances without a user deeming the object itself broken. Objects with broken affordances afford–at least–repair, maintenance, or replacement, though this may be highly dependent on the object and user: decay of objects may be viewed as positive and engender affection and memory, also for virtual objects such as photos [19, 24]. Furthermore, they may afford repurposing where the object is changed or combined with other objects to form a new object with distinct affordances, and exploration where the inside of complex technological objects are investigated by the user [16].

In addition, objects with broken affordances may afford repurposing, to be used for something else or having some of its affordances that were already there take the foreground, even though they were not what the object was originally produced for, as already remarked by Bærentsen and Trettvik [2]. For example, an old CRT TV with a broken screen may be used as a table, or a polystyrene packing box cracked in two may be used as two bathtub toys.

5.1 Physical Versus Virtual Objects

Physical objects may have emotional significance that is harder to find for virtual objects: a blanket that has been worn thing and no longer affords warmth or insulation may have belonged to a beloved family member. This is also true for virtual objects such as photos or video [24]. A difference between physical and virtual objects, in particular of tools, is the permanence and common irreplaceability of physical objects: a well-crafted saw, or a vintage car, may have belonged to a loved one, and retained for sentimental value even if they have lost key affordances, as may a leaky teapot made by a pre-schooler; and the objects are very hard to replace.

Similarly, physical objects are often repurposable, but software tools that have broken affordances are much harder to repurpose: even an expert user or programmer will be hard pressed to repurpose a program (though parts of the program code may be re-used). We hold that there are two primary reasons for this. The first is simply the intangibility of virtual objects: if users cannot experience an object with a full sensory apparatus, the objects will in general fail to be repurposable: if a user cannot feel, or hear, or smell an object, fewer possibilities for creative reuse present themselves. The second reason is cultural and holds equally for many quickly replaceable technological physical artefacts: objects have a very limited lifespan before obsolescence–when today’s high-schooler passes away in 70 years, her grandchildren may keep her old yearbook as a family heirloom, but probably not her currently favorite smartphone. A key difference between virtual tools such as programs, and physical tools, is that they are often maintained actively through patching or releases of new versions, and lend themselves well to rapid prototyping: adding an instrumental affordance with an effecter equivalent to an existing one (e.g., adding a quick-access button to an action otherwise hidden deep in layers of menu) can often be done, and tested, quickly, as opposed to (high-fidelity) physical prototyping. However, for highly complex and non-modular software, testing of a quickly-produced prototype can be highly cumbersome.

6 Experimental Exploration

To challenge our analysis above, and to explore the actions of users and designers when faced with broken affordances, we facilitated an explorative workshop with usability experts and artists, and devised a user study where users were faced with combinations of broken handling and effecter affordances, and with defective signifiers and feedback. The scenarios used in the workshop and the software and questions used for the user study can be obtained by contacting the authors.

6.1 Explorative Workshop

Two usability consultants from industry, and a visual designer and a visual artist were recruited for a workshop. All were professionals making their primary income from their profession. Participants were remunerated by standard rates for consultancy work (the usability consultants), respectively by standard rates for expert participants (visual designer and artist). Participants were not informed about the purpose of, or tasks to be covered in, the workshop prior to its beginning, and were only informed about the purpose in the debriefing following the workshop. Scenarios were prepared for the workshop, consisting of a description and an associated physical artefact, software mockup or image of a physical artefact (Fig. 3). The scenarios were divided into: (i) 8 goal-oriented scenarios, describing an artefact and what was broken about it, along with a particular goal to attempt to achieve, and (ii) 4 open scenarios, describing an artefact and encouraging users to both find out what was wrong with it, and what they would like to use the artefact for. The first half of the workshop involved goal-oriented scenarios, the second half open scenarios. The first scenario was to be treated and reflected upon individually. For the remaining scenarios, participants were divided into pairs consisting of a usability consultant and visual designer or visual artist. At the conclusion of each of the two halves of the workshop, a facilitated plenary discussion was held. At the conclusion of the workshop, a facilitated discussion was held concerning desired behaviors of more complex software system with broken affordances. The workshop was recorded on video and audio devices and later coded by the authors.

Fig. 3.
figure 3

Foreground: two physical artefacts from the workshop: a conference room chair with defective backrest (breakage behind the leather flap visible in the picture center), and a plastic bottle with a hole near the bottom (indicated by red marker outline on the bottle in the center-left). Background: the workshop lab.

Full size image

Results.

We identified a number of recurring themes in the response to, and discussion of, the scenarios, all of which supported our original analysis: Firstly, simple physical objects with broken affordances afford repair by the user: A hole in a bottle can be plugged, a chair with a faulty backrest can be mended if the user has the skill, or can simply be placed against a wall. Secondly, some physical objects with broken affordances afford repurposing: A chair with a faulty backrest can be made into a footstool, a bottle with a hole can be made into a bird feeder.

For the goal-oriented scenarios, the amount of workarounds and tinkering with physical objects was context-dependent (this observation was explicitly articulated several times by the participants): for example, a cardboard ballot box with tape across the slit could be easily used by either removing the tape, cutting a hole in the side, or removing the entire lid, but all participants agreed that they would not tinker with a wooden ballot box with a padlock. Likewise, virtual objects with broken affordances were treated in a manner entirely different from the physical objects: all participants first attempted to verify that it was not themselves who had made an error (this was most evident in the text entry box with a greyed-out “submit” button). Virtual objects were clearly treated as tools that were replaceable (participants faced with a program with broken affordances all suggested using a different program to accomplish scenarios), reinforcing the view of affordances as facets of mediated human action. In contrast, physical objects were treated as having a variety of roles: tools (to be repaired or replaced by similar tools), objects of affection (a broken coat hanger could have been made a cherished family member), or repurposable objects (often completely replacing the original affordances of the object with new ones).

Asked to provide scenarios – explicitly unconstrained by their knowledge of current technology constraints – where virtual objects with broken affordances could be retained (as opposed to worked around, replaced, or reinstalled) participants volunteered that this would only happen if the object had emotional significance, for instance if it was a program that they had themselves contributed to in some way.

6.2 Goal-Oriented User Study

To investigate user interaction with instrumental affordances where signifiers, handling affordances, effecter affordances, and feedback could be broken separately, we conducted a small user study concerned with accomplishing a simple task using interface metaphors that would be immediately recognizable to users, but deliberately decontextualized: the user was placed in front of a screen and asked to interact with a program clearly not embedded in any standard software they used (i.e., not browser-based, with color scheme and icon placement clearly distinct from the usual suite of software they were used to). This setup allowed us to put the users in a position where they would recognize the standard WIMP metaphors (e.g., dragging-and-dropping, or WASD keyboard mappings), but such that the ambient software of the OS did not afford obvious alternatives to complete the task. A classic shape-sorter toy was used for the study. In a shape-sorting toy, the user is presented with a number of differently-shaped holes and a number of shapes to be put into the “right” holes; the holes are shaped such that only a single shape can fit into a given hole, but occasionally smaller objects may fit into holes intended for larger objects. The layout of the shape-sorting task is shown in Fig. 4. The user is presented with one task at a time, with an option to skip to the next task by clicking a ‘Next’ button. At the beginning of each task, a green-colored shape is provided to the user at the bottom of the screen. One or more white shapes appear at the top of the screen, and the user can drag the green-colored shape into a white shape, and if it is the correct shape then the task is completed, a popup box appears stating, ‘Correct!’, and the program advances to the next task. Moving the green shape into a shape where it does not fit, or anywhere else, will prompt a popup box stating, ‘Incorrect!’. All shapes can be put in a hole whose geometry and area would allow it (e.g., all shapes except the oval can be dropped in the circular hole). We defined five variations of the ‘default task’:

Fig. 4.
figure 4

The shape-sorter task. Top left: the green oval affords dragging to one of the white holes. Top right: feedback for correctly solving a task. Bottom left: Broken signifier: the green oval still affords dragging and dropping to the oval hole, but the text above the hole reads “this is not an oval”. Bottom right: Broken feedback: the hexagon is dropped correctly on top of the white hexagon, but the software responds with ‘Incorrect!’. The task has actually been completed correctly, so the software moves on to the next task (not depicted) (Color figure online).

Full size image
  1. 1.

    Broken handling affordance, broken effecter: The green shape is impossible to move by any means.

  2. 2.

    Broken effecter, defective feedback, intact handling affordance: Despite moving the green shape onto the correct target shape, the task does not complete.

  3. 3.

    Broken handling affordance, intact effecter: The green shape shows no visual indication of being moved, but still moves.

  4. 4.

    Defective feedback, no broken affordance: The task will display ‘Correct!’ instead of ‘Incorrect!’ and vice versa.

  5. 5.

    Defective signifier, no broken affordance: Text intended to mislead the user will be displayed on the screen, such as ‘This is not an oval.’ above the white oval, ‘Triangle.’ above an empty space where the triangle would normally be, and ‘Pentagon??’ close to where the hexagon would usually be.

Variations 1 and 2 present an unsolvable task. In such a case, the only option the participant had to advance was to press the ‘Next’ button. The first 5 tasks were always chosen to be the ‘default task’ with nothing broken, and a randomized list of 60 variations were chosen for a total of 65 tasks. 6 participants were recruited from a graduate programme in game design (all male, age 23-27, median 24). Each participant received a short written instruction prior to the study, was provided a laptop with the shape-sorter toy program open. Each participant received a short oral instruction, and a facilitator was present to answer general questions about the study, but to provide no help with the interface. Participants were made explicitly aware that the program would only proceed to the next task by itself (i.e. without the user pressing the ‘Next’ button), if they had correctly solved the task. Participants were asked to think aloud, and complete a short questionnaire after the study asking for (free, user-defined) categorization of the various problems encountered and a free-text description of what they would normally do if a program exhibited these kinds of problems, and were asked to rank their self-defined types of errors from most to least annoying. An audio device recorded the utterances of each participant. The study took between 45 and 60 min per participant.

Results.

By coding participants’ think-aloud utterances and examining the error counts for the task variations, we observed the following: P1-5 all attempted to use arrow keys several times in Variations 1 and 3 (broken handling affordances); P6 did not try using the keyboard at any point in time during the study. Erroneous feedback (Variation 4) when the user correctly/incorrectly solved a task caused frustration and confusion amongst all participants. All participants initially indicated that they blamed themselves for solving the task wrong, not understanding that in fact the feedback was incorrect and they had correctly solved the task. P1,P3,P5 and P6 realized that the program would occasionally lie to them or give them wrong feedback, while P2 and P4 considered all feedback to be correct, and faults to lie with them self and not the program. P1-4 and P6 considered the tasks with a broken handling affordance to be most annoying, all of them likening it to a program freezing. P2 and P4 proclaimed that they would rather receive incorrect feedback, or have the program simply not work (read: broken effecter), rather than not be able to manipulate the elements of the software at all. The part(s) of an affordance broken in the different task variations had effects as follows: (i) broken handlers (variations 1 and 3) caused immediate frustration for several participants, (ii) defective feedback (variations 2 and 4) prompted participants to try a wide range of different actions to solve the task. When only defective feedback was present, but the task still worked (variation 4), there was little frustration amongst the participants. When defective feedback was coupled with a broken task (variation 2), several participants became frustrated very quickly, more than in variations 1 and 3. Participants displayed widely varying levels of effort with regards to the number of different actions attempted upon a non-obvious or unsolvable task: some participants attempted dozens of different combinations of keyboard and mouse input, while others almost immediately stopped trying.

7 Discussion: Implications and Opportunities for Design

The user studies, in conjunction with the theoretical treatment of Sects. 2, 4 and 5, give rise to several intriguing observations:

  • From a design perspective, there are several design guidelines that may be gleaned. Specifically: (A) Create several instrumental affordances with equivalent effecter affordances (but with different handling affordances and signifiers). This entails ensuring several ways to accomplish the same goal. A classic instance of this is to provide keyboard shortcuts to supplement point-and-click interfaces. For OS designers, an example is to allow access to a bricolage of clients rather than chaining the user to a specific application (e.g., allowing several browsers on a smartphone). As mentioned by all participants in our workshop, this was a preferred strategy when using web interfaces. (B) Couple handling and effecter affordances tightly (also advocated by Kaptelinin and Nardi [18]): the outcome of a user action should be clear. In the user study, all participants were confused when they received no visual feedback upon trying to drag a shape (i.e., a broken handling affordance). This confusion, in some cases, led to completely incorrect assumptions about how the program worked and how input affected it. (C) Do not remove expected interaction possibilities, even if no effecter affordance is intended by the designer. If a user expects to be able to drag-and-drop a file to move it and, for some reason, it cannot be moved (i.e., a broken effecter from the user’s perspective), let the user drag-and-drop instead of “freezing” the file or returning an error message when clicking the file. If possible, give feedback after the drop that the action could not be completed. (D) Make the background logic of virtual objects with affordances accessible to the user, if possible. This is a difficult task, and may not lend itself well to the current generation of users. Workshop participants stated clearly that if they could program, they would like to examine the source of the page to ascertain whether they had made an error, or more closely inspect what the problem could be (supposedly, instead of looking for alternative instrumental affordances sharing the same effecter). Note that (A)-(D) vindicate, and provide additional support for, existing design heuristics. There is one novelty evident in (A) and (C), namely that the explicit focus on handling and effecter affordances may provide the designer with a specific terminology and mental model to work with.

  • While we have focused on user experience, e.g., by considering user’s utterances in the experimental explorations, it is hard to draw firm conclusions for the implications for usability: in the goal-directed user study, several users rank the brokenness of different parts of an instrumental affordance on a self-devised scale of annoyance; but this says little about the effectiveness or efficiency with which they solve tasks. We expect that effectiveness will be hampered by, say, a broken effecter if the designer’s intent matches that of the user; but the impact on effectiveness of, say, breaking a handling affordance compared to breaking an effecter may be small even though users may report it as more annoying.

  • We have studied artefacts where the notion of brokenness was absolute and, for most users, not subject to discussion. Our definition of brokenness concerns the designer’s intent, but a broken object may add to the user experience by virtue of its brokenness, especially in cases where there is discrepancy between the intent of the designer and the needs of the user. Consider the typical SMS app found on smartphones; this app often comes with an auto-correcting language dictionary that cannot be easily disabled. If this functionality were to break, users who find the dictionary to be annoying might see this as a benefit, not a problem.

  • We have argued that the reconceptualization of the notion of affordance by Kaptelinin and Nardi is “operational” in a sense that the earlier notions of Norman and Gaver (and the original ecological perspective of Gibson) are not. We hold that both user studies completely vindicate this view: Participants clearly reacted differently, and with different coping strategies when signifiers and feedback, respectively handling and effecter affordances, were independently defective or broken. We believe that this fact alone makes the case for a rich notion of affordance (in the style of Kaptelinin and Nardi) to be used for design purposes (see also [20]).

8 Conclusions and Perspectives

We have argued that the notion of instrumental technology affordances of [18] is “operational” in the sense that the notion of deficiency or breakdown of various components of an affordance, and the consequences, is much more clearly elucidated than in much of the early literature; however, work in activity theory, notably [1, 2], while taking its inspiration in work closely aligned with [18], could conceivably be used to perform a structured study of the concept and implications of brokenness of affordances. Similarly, recent attempts to give detailed accounts of cognition in interaction with technology could be used for a similar purpose [28], as may more fine-grained analyses of the roles of signifiers and feedback [33]. We have used this operational view to analyze the interplay between notions of brokenness for affordances in both physical and virtual objects, and outlined the differences between these. Through analysis and explorative experimental work we have identified opportunities for concrete identification, and circumvention or improvement, of usability problems, using systematic breaking of instrumental affordances. A novelty of this approach is that breaking handling or effecter affordances separately forces the designer to consider the constituent parts of each affordance (and how malfunction of each part may lead to different coping strategies by the user), and may lead to new design choices or opportunities by re-appropriation or alternative use of objects or interfaces. We welcome longitudinal field studies “in the wild” of designers, artists, and developers using brokenness as a means of expression, and analysis. As noted in Sect. 7, the naïve notion that “broken = bad” may be challenged due to the discrepancy between the intent of the designer and the needs of the user; we expect that this challenge may be used creatively. Finally, we have posited that exploration of the design space of wear and brokenness of objects, as put forth in recent work [13, 15–17], would be informed by both systematic and playful breaking of facets of the affordances (e.g., breaking of handling affordances, not effecter affordances, and vice versa) of objects. The payoff, we believe, is a further sensitization of designers of the consequences when affordances break down and objects are repurposed, repaired, or complemented by affordances of other objects.