1 Introduction

Over the last years, the context of design has started to include products that are highly interactive and able to respond and adapt to our behavior. However, the miniaturization and increased amount of computing technology implemented in these products will eventually lead to enormous amounts of data that need to be processed. Therefore, research is necessary to determine what behavioral data offer the most interesting insight into these processes, and designers have the responsibility to design feedback that is tailored to these situations. Thus, not only data representation is moving toward the background as described in calm technology [1] but products are also starting to measure all kinds of sensory input in the background.

Products with embedded sensors to measure behavior may address functional properties such as a closet that supports in choosing clothes in different occasions [2], or it may be applied to communicate or to measure concrete values, such as taken steps by a pedometer. Behavioral biometric profiling has also been investigated in security applications as people have very specific means of behavior [3]. Research is also being conducted on how products could measure implicit meanings underlying these functional behaviors, for example to predict emotions. Wensveen et al. [4] for example developed an alarm clock that measures emotions while people move the twelve sliders that are used to set the wake-up time.

While behaviors may be used as a means to measure affect, the way that these behaviors or interpretations need to be fed back to the user remains unclear. In terms of emotions, the affective loop is defined as a feedback mechanism that continuously adapts to the behavioral expression of the user [5]. Höök et al. [6] have explored the affective loop in various interactive systems such as eMoto, which explored how bodily expressions could add emotional loading to mobile messages. Another exploration of the affective loop was in a personal logging system [7], in which day-to-day experiences were logged in a diary on which users could reflect.

The vision behind the affective loop is that the user first expresses an emotion through physical interaction involving the body, for example, through gestures or manipulations of an artifact. Consequently, the system (or another user through the system) responds by generating an affective expression, using for example, colors, animations, or haptics, which in turn affects the user (both mind and body) to respond and step-by-step feel more and more involved with the system [6]. In this vision, the first step reflects the expression of the user toward the product. The second describes the feedback mechanism, in which the product invites the user to perform a new action to keep her engaged in the interaction. This presentation of information can be defined as a means of feed forward. In the affective loop, the feedback appears to gradually evolve into feed forward to achieve the third step, in which the user is engaged in a feedback and feed forward process of interaction that gradually changes her expression and thus the dynamics of her movement. The product continuously adapts to the input of the user.

The affective loop provides a clear vision on how the product could respond to emotional interaction, such as nervous behaviors with a pen that could be considered as implicit expressions of stress [8]. However, all projects exploring the affective loop rely on the cognitive skills of the user and do not present feedback that addresses the perceptual motor skills. According to Djajadiningrat et al. [9], products should exploit the sensory richness and action potential of physical objects in relationship to perceptual motor skills for which the affective loop could also address the perceptual motor skills of the user. In the current paper, we propose a new means of feedback for the affective loop that does not necessarily rely on the cognitive skills of the user.

1.1 Inherent feedback

To improve the sensory richness and action potential of physical objects in interactive products, Wensveen et al. [10] propose the concept of inherent feedback. Inherent feedback originates from a learning psychology concept called intrinsic feedback. This concept is defined as: “feedback which is given as a natural consequence of the action; the feedback is intrinsic to the action” [11]. However, intrinsic applies to both things and people. To emphasize that this feedback relates to products, Wensveen et al. [10] refer to it as inherent feedback because it is about a quality that exists in something as a permanent, essential, or characteristic attribute.

In relation to human-product interaction design, Wensveen et al. [10] defined inherent feedback as the information that a product returns to the user, while she interacts with it and that addresses primarily her perceptual motor skills. In inherent feedback, there is a tight coupling between action and feedback, because the feedback offered by the product is a natural consequence of the action. The concept of inherent feedback is inspired by how mechanical products respond to user’s interaction. In most mechanical products, the appearance, the action possibilities, the action and the function are naturally coupled, allowing for intuitive interaction. The concept is illustrated through the interaction with a pair of scissors. The scissors provide visual, auditory and tactile feedback during cutting, which is a direct consequence of the user’s action, e.g., when the user is in a hurry, the cuts are imprecise and hurried.

While in product design this type of feedback is normally treated as a by-product of the choice of controls, e.g., each button has its specific touch and feel, in interactive products designers need to consider how interaction with these products can remain natural. Due to the miniaturization of technology and digitalization of products, the natural and mechanical sound, touch and feel in the interaction with these products are disappearing. For example, a button on a touch screen does not require any physical displacement or does not click when it is pushed down. This makes it difficult to perceive whether the button has been pushed or not when the product does not respond to an action due to a software failure.

To translate this coupling to interactive products, Wensveen et al. [10] propose the Frogger framework. The Frogger framework suggests that feedback is coupled naturally when user’s action and product reaction coincide in time, location, direction, modality, dynamics and expression. However, the power of inherent information does not have to end at providing natural feedback during the action. When an action possibility offers inherent feedback, even after the action has ceased, it can blend into inherent feed forward to guide further actions toward intended functionality, thus creating a feedback loop. If an emotional experience is added to this type of interaction, it could develop an affective feedback loop in product interaction.

1.2 Inherent feedback during product interaction to influence affect

Given the fact that emotions are expressed through behavioral actions and the intention is to use these object manipulations as an input for an interactive product, first, implicit communication of an emotion needs to be defined. Consequently, the way that a product could recognize these emotions needs to be explored. Having defined how emotions are expressed, the product will have to respond to it by providing feedback in such a way that the emotion can be affected.

We propose a method for designing an affective feedback loop in product interaction (see Fig. 1), which initiates when the user experiences an emotional event (a), subsequently the user expresses her emotional experience through her motor system (b). This behavior serves as input for an interactive product (c), which interprets her behavior and responds to it by changing its own behavior (d). This shift in product behavior affects how the user experiences the product (e) and will generate a shift in the users’ perception of the product (f). Initially, the product will trigger a behavioral response of the user, changing her action, but it could finally result in a reflective response that makes the user aware of her emotional status and affects that emotion (g). Subsequently, the different steps proposed in this affective feedback loop in product interaction process will be described by looking at both the user and the product. The concept of the affective feedback loop in product interaction will be illustrated by means of affecting the perception of stress to stimulate relaxation.

Fig. 1
figure 1

Buildup of the affective loop for product interaction with the letters in the figure depicting emotional event (E), emotional experience (X), action (A), input (I), interactive product (IP), output (O) and perception (P)

Full size image

1.2.1 Perception of stress

In an affective loop experience, the user experiences an emotional event that affects her emotional experience (Fig. 1a), and this experience triggers various bodily processes such as physiological, psychological and behavioral changes.

1.2.2 Expression of stress

In the context of product interaction, the first aspect that is addressed is the expression of motor behavioral responses to the emotional situation by means of behavioral expressions of the hands (Fig. 1b). Object interactions such as fidgeting for example are related to restlessness [12]. Research regarding such object movements is required to determine what movements are associated with certain emotions.

1.2.3 Product sensor

The product within the system in which the affective feedback loop will be integrated is an interactive product requiring input that it can interpret. Therefore, after having defined the type of behaviors that are expressed during the emotional situation, these behaviors need to be translated into information that the product can sense (Fig. 1c). Thus, adequate sensors need to be defined that can generate input for the product, based on the movements of the user. Furthermore, design considerations are required on how the user will actually perform the implicit interactions. The product has to be designed in such a way that a user can express behaviors via it, without consciously being aware of it, and sensors can sense these behaviors.

1.2.4 Product actuator

In Fig. 1d the interactive product translates the input that it receives from the user, who is feeling the emotion, into a response. Thus, first the feedback needs to be defined, and secondly actuators need to be developed that enable this feedback, which should be inherent. Therefore, it has to be embedded in the product that is manipulated in order to coincide in location. Furthermore, to coincide in modality, the feedback could address the tactile sense, for example, through haptic feedback.

1.2.5 Interaction and influencing the emotional situation

Finally, as illustrated in Fig. 1f, one has to look at how the interaction will affect the perception of the emotional situation. The experience of a product by the user can be separated into different parts. Norman and Ortony [13] described three emotional responses to a product: visceral, behavioral and reflective. The visceral response is a perceptually based reaction, in which the user automatically evaluates the perceptual properties of objects and classifies them quickly without any need of evaluation (e.g., feeling disgust when touching a moldy orange). On the other hand, the behavioral response is an expectation-induced reaction. It still is a subconscious and automatic response, but in contrast to the visceral response, it builds on past experience (e.g., pleasure when touching a furry ball). Finally, the reflective response is a conscious interpretation of a situation, resulting in an intellectually induced reaction (e.g., feeling pride after having solved a puzzle).

When designing for the affective feedback loop in product interaction, the visceral experience of the product needs to be addressed first; the product needs to be comfortable and pleasant to hold, while interacting with it (Fig. 1e). When the product notices that the user experiences an emotion, the product will create awareness by responding on the behavioral level (Fig. 1f). It is expected that a change in the behavioral expectations, such as failure of the product, will generate a strong emotional response [13]. Creating this awareness in a positive sense will be the first step in influencing the emotion, which means that a reflective response occurs. The user becomes aware of the experience and tries influencing the emotion by changing her behavior (Fig. 1g).

2 Implementation

The emotional event that was selected to affect the experience is a stressor. As mentioned before, stress is an emotion that generally provokes nervous behaviors resulting in object manipulations [12]. To evaluate how the affective feedback loop can be integrated in product interaction, the six steps were implemented in a prototype after a series of design iterations (e.g., [14]). A pen was selected to illustrate the concept because a study on nervous habits by Hansen et al. [15] found that 63% of college students “play with pens and pencils.” Furthermore, rocking and rolling movements with a pen were previously defined as indicators of stress [8]. In addition, the pen appeared to be an inconspicuous object that is of general use and fits the context as it can unobtrusively sense implicit behaviors.

2.1 Prototype

The final prototype has a tip that can be rolled and changes in friction while rolling, which serves as a means of haptic feedback. Furthermore, the pen senses rocking movements and the freedom of movement of a ball in the top of the pen can be modified to provide inherent feedback on this movement.

The shape of the pen is very much influenced by the physical constraints imposed by the size of the hardware components and because the pen was required to be wireless to facilitate manipulations. The pen is controlled by an Arduino, communicates wirelessly via Bluetooth and is powered by 3 AAA batteries. The tip of the pen is fixed to an axis, which ends in a magnetic disk and rolls freely within a magnetic coil. When the coil is activated, it attracts the magnetic disk, thereby increasing the friction and counteracting the rolling movement. Two infrared sensors measure the speed and direction of the rotating disk on the axis. In the top of the pen, a second coil controls the freedom of movement of an 8-mm-diameter magnetic steel ball. Four Hall sensors measure the frequency by which the ball moves, while the user rocks the pen (see Fig. 2).

Fig. 2
figure 2

Components of the pen prototype employed in the experiment

Full size image

The body of the pen is made out of aluminum and slightly conical. Both the tip and the upper part of the pen are resin prints. The tip of the pen has a linear tread pattern for grip and to support the rolling movement and a pencil lead for writing.

2.1.1 Input

The rotation of the tip of the pen as well as the movement of the magnetic ball served as input for a program that was developed in Cycling Max/MSP. Roll was measured by calculating the rotation of the tip through the amount and direction of change in sensor input data from the infrared encoders, which were measured at 20 Hz. Rock was calculated by measuring the movement of the ball, which was obtained from the amount of change in magnetic field sensed by the Hall effect sensors.

2.1.2 Output

Movement was calibrated by setting 0 for no movement and 100 for maximum possible movement, which was determined by rocking the pen device as fiercely as possible. Through a process of trial and error, the output settings for the coil in the upper part and the coil were defined. The final setting was that first short movements, i.e., below 10 for roll and below 7 for rock, were filtered out to prevent the pen from being too responsive. After filtering, the rotation input between 10 and 40 was converted linearly to a response of the pen from 0 to maximum friction. All changes in rotation above 40 were automatically set to the maximum amount of friction. For the fixation of the magnetic ball, the values of input were set between 7 and 15, again higher values were automatically set to the maximum amount of fixation.

3 Experiment

To evaluate whether inherent feedback to implicit behavioral expressions with a pen can affect the emotional experience, an experiment was conducted that addressed the following three questions:

  • Is it possible to differentiate emotional situations based on how the prototype is manipulated?

  • Does the inherent feedback provided by the prototype stimulate the user to change behavior?

  • Does the inherent feedback provided by prototype affect the emotional experience?

3.1 Participants

Thirty-four students (17 female, 17 male; mean age 20.2 ± 1.9 years) participated voluntarily in this experiment. One participant was excluded from all analyses because his session was disturbed and he misinterpreted the order of the experiment. They were informed that the experiment was about observing their hand movements while watching a set of images.

3.2 Procedure

Two groups of participants were exposed to two different conditions. In one condition, the pen only measured movement (Pen 0), and in the second condition, the pen provided haptic feedback on the movement (Pen 1). Upon arrival, students were seated at the desk where the experiment would take place (see Fig. 3) and were introduced to the procedure. Once they agreed to participate in the study, they were requested to sign an informed consent form after which the experiment started by filling in the short version of State Anxiety Inventory (SAI) developed by Marteau and Bekker [16], which is described below. Consequently, participants viewed three sets of eight digitized photographs selected from the International Affective Picture System [17].

Fig. 3
figure 3

Experimental setup

Full size image

After each set of eight pictures, participants had 2 min to complete the short version of the SAI and to conduct a memory task. The memory task had to ensure that they would use the pen in between the sessions and consisted of describing one of the previously shown pictures, in at least 20 words. After describing the picture of the final set, participants had to complete one last short version of the SAI and a questionnaire related to the pen. At the end of the experiment, participants were interviewed as to whether they had noticed something from the pen. Subsequently, all participants were debriefed, and the real purpose of the experiment was explained, i.e., comparing the two conditions of the pen. The total duration of the experiment was of approximately 25 min (see Table 1).

Table 1 Overview of task procedure
Full size table

3.3 Physiological measurement

Cardiac activity has been known to be under autonomous control and to be primarily sensitive to mental stress [18]. In the current experiment, inter-beat intervals (IBIs) were recorded by means of the Heartmath Emwave ear clip. Heart rate (HR) was calculated by dividing the sum of every ten previous inter-beat intervals by ten. A mean HR was calculated for every 48 s (8 × 6 s) of picture presentation and for every 120 s of short version SAI and image description. HR measures were fully omitted for four participants, and for two other participants, the last and the last two sessions, respectively, were omitted due to failures in the measuring equipment. The mean frequencies of the movements, rock and roll, were calculated for the picture presentation and for every short version SAI and image description, by dividing the total amount of movement during these periods by 48 and 120 s, respectively.

3.4 Subjective questionnaires

As in the current experiment, the targeted emotions are related to stress the shortened version of the State Anxiety Inventory (SAI) by Spielberger et al. [19] was selected. The SAI is an often-employed measure for stress and anxiety and is more specific in terms of stress measurement than the SAM questionnaire [20], which is a general measure for valence and arousal. As the long version would have been too cumbersome for the duration of the experiment, the short version developed by Marteau and Bekker [16] was employed. The short version consists of six selected items from the SAI: “I am tense”; “I feel upset”; “I am worried”; “I feel calm”; “I am relaxed”; and “I feel content.” The participant can respond to these questions on a four-point scale indicating: “Not at all”; “Somewhat”; “Moderately so”; “Very much so.”

The questionnaire related to the pen was meant to evaluate the pen and was similar to the short version of SAI, but it was setup in such a way that the question items were preceded by “The pen made me…” or “The pen made me feel…” instead of “I am” or “I feel….” Both questionnaires were transformed into single number items by summing the scores for calm, relaxed and content and deducting the scores for tense, upset, worried. This provided values between −9 (highest anxiety) and 9 (lowest anxiety).

3.5 Affective pen

Both groups of subjects received the same pen, with which two conditions were tested. One group received the pen in which actuation was turned off (Pen 0). The pen only measured behavior but did not respond by means of inherent feedback. The latter group of participants received the pen that responded to the object manipulators rock and roll by providing haptic feedback (Pen 1). The pen was activated before starting the experiment, and participants were requested to hold the pen in their hands throughout the whole experiment. It was mentioned that incorporated sensors would measure their hand movements.

3.6 Task

To induce arousal, participants were conditioned with pictures from the International Affective Picture System [21]. Visual conditioning has been employed in experiments both in affective computing [22] as well as in experiments addressing behavioral expressions of affect [23]. To ensure that subjects would hold the pen throughout the experiment, participants had to perform a memory-writing task, which is often employed as a stressor (e.g., [24]), with the affective pen.

The set of images employed in this experiment was the same set of images as employed in the experiment by Coombes et al. [23]. Each set represented an affective category that was matched or polarized according to valence and arousal norms (pleasant, unpleasant, neutral). Categories were differentiated according to valence (pleasant: 7.2 ± 1.8; unpleasant: 2.5 ± 1.6; neutral: 5.0 ± 1.3), and pleasant and unpleasant categories were matched for arousal and chosen so as to distinguish valenced categories from neutral (pleasant: 5.3 ± 2.2; neutral: 3.2 ± 1.9; unpleasant: 5.7 ± 2.1). Images were presented for a period of 6 s on an Apple PowerBook G4, and the participant started the experiment by clicking the “Proceed” button with the mouse. Two different orders were randomized: pleasant, neutral, unpleasant; and unpleasant, neutral, pleasant.

4 Results

In this result section, the effect of the picture presentation and the memory task will be evaluated first. Consequently, the experimental results are presented in three steps. First, the question will be addressed whether it is possible to differentiate emotional situations based on how the prototype is manipulated. Secondly, the way that the prototype stimulates the user to change her behavior through inherent feedback will be assessed. And finally, it will explore whether inherent feedback to implicit behavior by means of the affective pen succeeded in affecting the emotional experience.

4.1 Effect of the stressor

To determine the effect of the pictures on the emotional experience, a Repeated Measures General Linear Model (RM GLM) analysis was conducted on the scores of the short version of the SAI for the different conditions. A significant effect (F = 13.2) was found for the subjective evaluation, with a higher level of state anxiety measured during the unpleasant condition (see Table 2).

Table 2 Results of the RM GLM on the responses to the short version of the SAI at the start, after pleasant, neutral, and unpleasant images and at the end of the experiment
Full size table

A RM GLM was conducted on the measures for HR during the conditions in which the images from IAPS were presented, as well as for the questions that were presented directly after the image presentation denoted by Q. No significant effect was found for HR over the different conditions (see Table 3).

Table 3 Results of the RM GLM comparing the users’ mean HR during pleasant, neutral and unpleasant images and during the questionnaire after images
Full size table

4.1.1 Discussion on effect of the stressor

The subjective evaluation of the images by means of the SAI showed that scores for the unpleasant images were significantly lower than the scores for pleasant and neutral images. The fact that no differences were found between pleasant and neutral images could be attributed to the fact that a different questionnaire was employed. While in general the SAM questionnaire [20] is used in these experiments, the SAI has fewer possibilities for neutral answers. Furthermore, the SAI is more explicit in asking how the participant feels, whereas the SAM is a translation of the experience that a subject has while viewing the images.

No significant differences were found for HR across the different conditions. This could be attributed to the fact that the picture presentation was too short to elicit any changes in HR. In the current experiment, only 8 pictures were shown per condition, while other experiments that found differences in HR used larger samples of pictures [25, 26].

In general, when normal subjects view valenced images, HR appears to decelerate, with the largest decelerations observed while viewing unpleasant images [17, 20, 26]. The observed decreases in HR, although not significant, during the unpleasant scenes could be attributed to the fact that HR deceleration is larger during observation of stimuli depicting mutilations, injuries or blood [20, 27], as some of the pictures in the current sample did.

4.2 Effect of emotional experience on behavior

Having explored the effect of the images, the first subquestion that will be evaluated is whether the pen can recognize different emotional experiences. Therefore, a one-way ANOVA was conducted for the amount of rocking and rolling movements during the conditions in which the images from IAPS were presented, as well as for the questions that were presented directly after the image presentation.

No significant differences were found in frequency of rocking and rolling movements between the different conditions (see Table 4). This could have been caused by the fact that the pen provided feedback, and this influenced the behavior. Therefore, an independent samples t-test was conducted to evaluate whether there were any differences between the two types of pens in terms of behavior.

Table 4 Results for one-way ANOVA comparing the users’ movements, rock and roll, during images as well as during questionnaire after images for the conditions pleasant, neutral and unpleasant
Full size table

Overall participants did not show different behaviors with the two types of pen, with the exception for the rolling movement during unpleasant images. Subjects with the pen that provides feedback showed a significantly higher frequency in rolling movements when they were presented with unpleasant images (F = 15.8).

When looking at the graphs depicting the mean frequencies of the movements, rock and roll, over the conditions during image presentation as well as during questionnaire after image presentation for pleasant, unpleasant and neutral, one can observe a flattening out of the rock movement. While the ball in the pen that does provide feedback is moved less when there is more movement, the ball is moved more frequently in the pen that provides feedback when there is little movement. For the rolling movement, the two pens also show a similar pattern. However, the rolling behaviors occur more frequently in the pen that provides inherent feedback (see Fig. 4a, b).

Fig. 4
figure 4

a Graph depicting the mean frequency of the movement rock per condition. b Graph depicting the mean frequency of the movement roll per condition

Full size image

4.2.1 Discussion on the effect of emotional experience on behavior

Although no significant differences in movement were observed for the two pen types, which could be attributed to the small sample size, the graphs show two interesting trends. The inherent feedback to the rocking movement appears to reduce the movement when there is too much movement, whereas it stimulates movement when there is too little. On the other hand, the inherent feedback to the rolling movement does not reduce the frequency of movement, but seems to stimulate it. Participants might have been triggered to play with the pen as this feedback was much more apparent and experienceable than the feedback on the rocking movement. However, only three participants responded that they had noticed the haptic feedback provided by the pen.

4.3 Influencing the emotional experience

Thus, the inherent feedback did not significantly modify the behaviors, but some changes were found when looking at the differences in behavior in a more qualitative way. To evaluate whether these differences in behavior also affected the emotional experience, the two groups were compared on both subjective and physiological experience. A Mann–Whitney, nonparametric test showed no significant differences between the subjective evaluations of both conditions, and no significant differences for the evaluations of the pen. An independent samples t test comparing the HR for the two groups (see Table 5) showed a significantly lower HR for the participants that used the pen that responded to their behavior when compared to the participants that used the non-responsive pen (t = −2.5). To evaluate whether these differences in HR were also observed per condition, independent samples t-tests were conducted to explore the differences between the two groups considering HR yielding no significant differences.

Table 5 Results for independent samples t-tests comparing the users’ HR for the different pen groups
Full size table

5 Discussion

In this paper, we explored how products could respond to behavioral expression of emotion by building on the concept of the affective loop described by Höök [5] as well as on the concept of inherent feedback in tangible interaction [9]. It combined these two concepts, with the aim of affecting the experience of an emotional situation through product interaction. The concept of the affective feedback loop in product interaction was explored in the context of stress by means of a pen prototype that measured how the user manipulated it to determine her level of affect. First, a behavioral response of the user toward the pen was expected; secondly, a shift in how the user interacted with the pen; and finally, a reflective response influencing the emotional experience of the user.

5.1 Affective and behavioral response

Significant differences were found in the subjective evaluation of the stressors, but no significant differences were observed in the frequency of manipulation of the pen during the different conditions or throughout the whole experiment. This could indicate that the pen cannot detect differences in emotional experience from behavior. However, the presented images did not have an effect on the heart rate either. As the images did not have any effect on the physiological experience, which may be attributed to the limited time frame of this experiment, this may also be the reason that no behavioral change was observed.

The fact that no differences in behavior were found also suggests that the feedback did not have an effect on the emotional experience. However, subjects in the group with the pen that responded to behavior had a different visceral experience. Despite the increased torque imposed on the rolling movement, which would normally result in increased HR [28], a decrease in HR was found for the participants that used the pen that provides feedback. The lower HR could indicate that participants in this group were in a more relaxed state [29, 30]. However, this conclusion needs to be taken with caution as no baseline was taken for HR. Future experiments should provide more evidence for this preliminary conclusion. In these experiments, more robust measures for cardiovascular activity could be taken such as an electrocardiogram.

5.2 Reflective response

In the presented experiment, no differences in subjective experience of the stressor while holding the different pens were found. A reason could be that only subjective evaluations for measuring perceived anxiety were taken and no measures for boredom. Furthermore, no differences were found in the experience of the two pens, which could also be attributed to the fact that subjects hardly experienced any differences in the behavioral response of the pen. Only three of seventeen experienced the (designed) feedback on rotation, and none experienced the feedback on movement.

Sundström et al. [6] suggested that in order to achieve the affective loop, both mind and body have to be affected to make the user feel more and more involved in the interaction with the product. However, the reflective part was not observed in the current experiment since no differences were found in the subjective evaluation of the conditions. The fact that users did not experience a reflective response could indicate that the user was not aware of her actions because the pen became an extension of the hand [31]. In this case, the user was unaware of holding the pen, and while she was manipulating it due to her restlessness, it became an extension of her body.

5.3 Unaware interaction

The user thus expressed restlessness by manipulating the pen as a means to cope and counteract the overactivity of the heart. When the pen changed its behavior, the user had to adapt her manipulations, and because the manipulations were a consequence of her physiological reaction, the heart had to adapt to the new situation. One could argue that since the changes in the object are perceived through the body, the body responds to the changes in the object without the need to affect the cognitive perception of the object as was expected by Sundström et al. [6] in their exploration of the affective loop.

The results of the current research suggest that a product can also change the perception of a stressor by only affecting the visceral experience and without requiring any cognitive or reflective processing. Therefore, the concept of the affective feedback loop in product interaction as it was presented in the introduction needs to be adapted to a situation where the stressor directly affects the behavioral experience, and the object affects the behavioral response without the user cognitively experiencing it (see Fig. 5). The user is unaware of both the interaction and the feedback. However, the interaction with the product has a clear influence on the visceral experience of the user. One could address this interaction, in which the user is neither aware of what she is doing nor of how the product responds as unaware interaction.

Fig. 5
figure 5

Left model as presented in the introduction and right revised model

Full size image

Although the results of this experiment offer some interesting speculations as pointed out above, the limitations should also be considered. Because the changes in movement of the pen could not be measured due to the interference of the actuators, it is not possible to measure how the behavior was affected by the feedback. A future experiment could incorporate a baseline for behavior measurement. In addition, considering that effects were only measured over the whole experiment, the duration of such experiment can be considered as a determinant factor. In future experiments, a longitudinal setup will probably be preferred involving new requirements in terms of size and reliability of the prototype.

5.4 Implications for design

Although there are various methodological issues that may be argued considering the current experiment and its findings, there are some aspects that are worth further research. In the currently described project, it was observed that the user was not aware of her actions while interacting with the pen. In addition, she did not perform these actions with a certain goal in mind, indicating that there are product interactions of which the user is not continuously aware. Although no effects were found on how subjects cognitively experienced the two pens, significant differences were found between the visceral experiences of both pens. This changes the preconception of this research, as the first idea was that one could affect the emotional experiences by bringing the subconscious processes to the foreground, whereas the emotional experience can be affected while all processes remain in the background. It appears that product feedback can be experienced only viscerally, without the user being cognitively aware of it. When the user is influenced viscerally over a longer period of time, this could eventually influence her emotionally. This proposes an interesting challenge for designers since by implementing the affective feedback loop, they can design for unaware interaction by offering inherent feedback to subconscious product interactions.