Keywords

1 Introduction

Touch brings instinct and abundant emotions to people. Different emotions by touch like shaking hands among friends, embracing from lover and mother’s caressing convey different meanings of pleasure. Emotions by touch, however, could also arise unpleasant feelings: the pain of being hurt, pricking and burning sensation resulted of overcooling or overheating [1]. Touch is featured with abundant emotions which can largely affect users’ emotion and experience.

There has been a long history for the research on haptics and haptic interactions. Since 1970s, four types of tactile receptors was found [2] and there were related researches on tactile psychophysics [3]. A set theories of function relation describing external physical stimuli and haptic psychological intensity has been established and consolidated. In terms of emotion, Russell et al. proposed “a circumplex model of affect” [4] in 1980s, and it innovated quantitative research on emotions. Picard et al. introduced the concept of “affective computing” [5] in 1997. It combined the emotion and human-computer interaction, striving for the intellectualization and harmonization of human-computer interaction.

Nowadays, it is yet still a huge challenge for haptics-based interaction as well as related user experience research. Currently, operations and feedbacks on haptic interactions are still relatively limited, such as the contact of touch screen: tow, tapping, force touch, and the haptic vibration or force feedback. Most of the haptic interactions reply on glass and other display screen or touch tablet made of smooth material, which lacks of richer touch information. In the process of interaction, the whole industry basically shows “vision as the main channel, touch and audition as assist” interaction mode and does not refer to the emotions that touch brings. The feeling of immersion need be enhanced.

Aimed at affective haptics interaction and design, firstly, from the human (user) perspective, this paper discusses the research methods of human emotional physiology and the feedback scheme of haptics signals. Secondly, from inside-out, this paper analyzes current affective haptics based on human emotional feedback characteristic, including a variety of affective haptic device and haptic signals that express a more abundant emotional belongings to increase user affective experience. Last but not least, based on previous human and device analysis, we explore the emotional experience design, and expect that haptic interaction and immersive experience can be enhanced.

2 Human Emotional Feedback on Haptic Signals

2.1 Emotion Evaluation Methods

People has complicated emotional activities. Generally speaking, there are two ways to evaluate emotions in terms of affective research: classification evaluation and dimension evaluation. Hevner [6] posed 67 types of emotions that can be described and divides them into 8 categories in 1935. This method, however, lacks of effective connection for different categories. Therefore, in the field of affective haptics, dimension evaluation is often used. Russell introduced “A Circumplex Modal of Affect” [4] in 1980, which put all human emotions into two dimensional plane rectangular coordinate system by “valance” and “arousal”. As Fig. 1 shows:

Fig. 1.
figure 1

A circumplex modal of affect [4], redraw by authors

Full size image

In this emotional model, X-axis stands for valence and Y-axis stands for arousal. both dimensions are linear. If the value is bigger, it will be farther away from the center point and reach deeper emotion. In contrast, if it is closer to the center point, it will be emotionally lower. The zero-point stands for human normal status. The four points between two coordinate axes in the Fig. 1 (Excitement, Contentment, Depression and Destress) are the common effect of valence-arousal. More detailed 15 emotions mapping see [7]. Thus, human emotion can reflect on a point in the valence-arousal space.

Circumplex model of affect is a concise, direct and effective emotional evaluation method. Compared to diversified classification of emotions, this model uses orthogonality of valence-arousal to establish emotional space. Besides, in terms of this model, some researchers have been trying to extent more dimensions apart from valence and arousal, including the dimension of “dominance” [8]. The experiment, however, showed that the dominance degree shows a certain correlation with the other dimensions, which is not supposed to be a third independent dimension. Therefore, most current emotional evaluation is mainly using valence-arousal method. Based on this evaluation method, any external signals can affect people and map the emotional state to a point in valence-arousal plane so as to establish a connection of physical signals and human emotional feedback.

2.2 Emotion Evaluation Methods

Based on the previous emotion evaluation methods and the circumplex model of affect, the other important question is how to accurately measure human emotion and correlate it to the valence-arousal space. Normally, there are subjective and objective methods in terms of human affective measure. Subjective method means subject gives subjective opinions after trying out external signals. In fact, most affective haptics experiments follow the idea: subjects feel a certain type of haptic signal (such as vibration, friction) and give a score in the valence-arousal space subjectively to establish the connection of haptics signals and emotional feedback. Bradley and Lang from university of Florida introduced “Self-Assessment Manikin, SAM” [9]. They demonstrated a gradual process of “pleasure-misery” and “arousal-sleepiness” by showing manikin images. As Fig. 2 shows:

Fig. 2.
figure 2

Self-assessment manikin [9]

Full size image

This self-evaluation model uses 5 manikins in each dimension to show different status at different degrees. It also inserts four middle status between two adjacent manikins so that each dimension has 9 status to be selected. The experiment shows that this model is effective and there is a better balance of discrimination and complexity in terms of emotion experiment, which can be employed in a wide range of applications, not only for affective haptics. but also for visual image and voice emotion experiments. [10, 11].

The subjective method can directly reflect subjects’ emotions and the experiment is relatively simple. In this type of psychology and affect experiments, however, subjects cannot be accurate enough to feel their own thinking and emotion in some situations. Or for some reasons, they try to give objective answers after thinking and modifying. these are some problems that subjective method may exist.

Compared to subjective measure method, the objective method tries to get a more accurate and objective emotional response by objective observation and some equipment. The observation and identification of facial expression, voice, body language can speculate human emotional status, while galvanic skin response, heart rate and the unmyelinated afferents can precisely inspect human emotion status from the perspective of biology and neurology.

In terms of the arousal dimension, there is a significant correlation between the extracted value of galvanic skin response and heart rate and the arousal dimension of emotion, which means stronger emotion, larger signal value and vice versa. By doing so, we can measure arousal emotional state objectively. In terms of the valence dimension, the neuroscience found out from an experiment in recent years that a movement of stroking in the human hairy skin was usually linked to pleasant touch sensation. Meanwhile, low-threshold unmyelinated mechanoreceptors (C-tactile) responded most vigorously [12]. It can be inferred that C-tactile afferents show positive correlation with pleasant sensation.

Furthermore, C-Tactile experiment discussed the relations between caressing speed and the degree of pleasant sensation. As Fig. 3a shows, the swing device made reciprocating movement at specific speed. The soft brush on the tail stroke the dorsal of forearm to create stroking motion.

Fig. 3.
figure 3

C-tactile experiment and analysis of speed-valence relations [12]

Full size image

In terms of the relation of C-Tactile and quantitative pleasure, the average release frequency and functional relations between pleasure sensation and stroking speed are showed as Fig. 3b and c. Two curves show similarities and are presented as inverted “U”. The middle part of the curve shows stronger pleasant sensation. While the speed is too slow or too fast, the sense of pleasure will decrease. Therefore, the experiment shows that when the stroking speed of haptic signal received on the hairy skin is between 110 cm/s and the pressure force is low, the pleasant sensation feels strongest [12].

The research above reveals the scheme of human haptic feedback, which can be the foundation for design theory and design instruction of affective haptic device and haptic interaction design.

3 Affective Haptic Device Analysis

According to human haptic emotion feedback scheme, based on the current hardware and technology conditions, some haptic devices have been developed in order to explore the connection of haptics and emotion as well as provide hardware basis for haptic interaction design.

3.1 Tactor and Related Device

Tactor, or vibration motor, has been usually used as haptic feedback related applications in human-computer interaction. Lots of smart or wearable devices, including mobile phone, smart watch and smart wristband put tiny tactor inside. When it gets touch with skin, human haptic receptors will receive vibrotactile signals in order to remind or recall user’s attention. Normally, some parameters of the tactor can be adjusted, including vibrotactile amplitude, duration, frequency and envelope frequency. For one single tactor, Yoo [13] explored the relation between these four tactor parameters and human affective sensation. In this experiment, a tactor was attached behind a cellphone making different combinations of these four parameters. Participants were asked to held the phone and use the valence-arousal space with bipolar Likert scale to log the affective response. In terms of the arousal, vibrotactile amplitude and duration showed positive correlation with arousal. Increasing vibration frequency, under some situations, would strengthen arousal. Envelope frequency showed no obvious connection with arousal. In the respect of valence, vibrotactile amplitude and duration as well as envelope frequency showed negative correlation with valence. Increasing vibration frequency, in some situation, could increase valence [13].

To step further, under one tactor condition, increasing arousal (such as vibrotactile amplitude and duration) will greatly decrease valence, reflecting that participants tend not to receive relatively strong reminding signals. When the vibration frequency increases from 60 Hz to 200 Hz, the valence and arousal of emotion will increase. It is decided by the characteristics of human haptic receptors. Human skin is most sensitive when the vibration frequency is between 200–300 Hz [14] and during this scope, people feel more comfortable [13]. So, the above conclusion can guide haptics interaction design: in terms of haptic signal design, the vibration frequency between 200–300 Hz is often used. A more appropriate intensity of the haptic signal should be at the point of just awakening and avoid over-strong.

Currently the combination of multiple tactors is a popular research field. There are some haptic illusions in haptics conception due to not only the peripheral physiology, but human central nervous mechanisms as well, For example, a row of tactors are attached on the skin and follows a specific vibration duration and interval. Then the subject will integrate these discrete vibrations into a continuous apparent motion [15].

According to this haptic characteristics, van Erp and Huisman made use of apparent motion to imitate consecutive haptic signals [16] and explored what types of parameters of the tactors can generate relatively pleasant sensation. As Fig. 4a shows, four tactors were arranged in a row and distributed around the forearm. Tactors vibrated in one direction to imitate continuous moving sensation. In the low arousal scenario, when the equivalent speed of movement was 10 cm/s, the pleasant sensation felt strongest, as Fig. 4b shows. This result matches the conclusion of Sect. 2.2, which confirms the best pleasant sensation happens at 1–10 cm/s speed.

Fig. 4.
figure 4

Experimental setup and analysis of velocity-perceived pleasantness relations [16]

Full size image

This research conclusion is very important for haptic signal design. Compared to single tactor scenario, the matrix configuration of multiple tactors can convey more abundant signals (such as vibration position, moving directions, frequency etc.). Meanwhile, designers need take haptic emotional factors into consideration. It would remarkably increase the experience of haptic interaction if the user receives pleasant sensation of haptic feedback or interaction.

3.2 Tactile Display

Tactile display has a wide range of concept. Any device that can generate haptic signals and “display” them to uses belongs to tactile display. Strictly speaking, the tactor is included, but this section put more emphasis on the system-based device. In this field, Biamchi et al. explores a fabric-based softness display [17] (shown in Fig. 5) and a novel fabric-based tactile display [18] (shown in Fig. 6).

Fig. 5.
figure 5

A fabric-based softness display [17]

Full size image
Fig. 6.
figure 6

Fabric-based tactile display [18]

Full size image

The device in picture 5 contains a soft fabric that is contacted with human finger. The two ends of the soft fabric can adjust the tension in real-time though two motors to change the level of fabric softness. The other “caress-like” device in picture 6 contains a soft fabric strip that can make reciprocating movement on the skin and generate the affective sensation. Both devices made experiments and analysis about the parameters of fabric tension, fabric quality and stroking speed. They shared the results in common: the increase of tension led a higher pressure and the subject valence would decrease. This conclusion is the same as the influence of tactor’s amplitude, which is that people tend not to receive relatively strong signals. When both devices made a relatively slow stroking, the subject responded with stronger valence. Therefore, the haptic stroking speed (110 cm/s) is not only suitable for the large area of skin surface movement but also for the relative movement of a certain place in the skin.

3.3 Affective Haptics of Touch Materials

In the field of tangible interaction, it is very important to choose a propriate type of material. Different materials and surface texture will lead to different effect, especially user emotional status. Drewing [19] explored the emotional connection of touching 47 different types of materials. In this experiment, 47 materials are put in different jars respectively. Users cannot see, but can only touch it by hand to guess what it is, describe it by adjectives and respond the emotional feedback by valence-arousal space. After analyzing these adjectives and related emotional responses, it was found that the adjectives to describe object has certain correlation with emotion. In terms of valence, the material roughness and valence had relatively weak negative correlation, which meant that if an object was rougher, the valence would be lower. While for the dimension of arousal, there was no related adjectives found.

There are abundant natural materials. In the haptic interaction process, single material information shows limited influence to people emotion. In the haptic interaction design, we need to combine material and a series of interaction activities as well as scenarios so as to benefit user’s emotional state, and better serve any possible interaction purpose.

4 Affective Haptic Interaction Design

In Sect. 2, the human emotional and physiological research provide the important fundamental support on affective haptic interaction design. In Sect. 3, the review on affective haptic device reveals the usability of current affective haptics in different perspectives, such as single tactor, haptic display or haptic emotional feedback of natural materials. Based on emotions in human-computer interaction and the state of the art of affective haptics, this chapter is to discuss haptic interaction design principles and instructions from design point of view.

4.1 Follow the Foundation of Haptic Physiological Characteristics

Compared with the sense of vision and audition, the physiological characteristic of the sense of touch decides a relatively low information capacity in terms of information transfer rate during haptic interactions. Therefore, in terms of interaction design, we need to put haptic related operation and feedback to a reasonable place, and any haptic interaction activities should be based on haptic physiological characteristics.

In terms of haptic signal design for the function of reminding and awakening, we need to consider user’s environment and interaction scenario. It would be a good suggestion that the intensity of haptic vibration could be self-adjusted based on the “golden” line where the user’s real time attention threshold is, not only meeting the functional requirement of interaction, but hitting the pleasant sensation as well. Currently, some haptic devices such as force feedback chair, joystick and wearables can introduce haptic caress or stroking sensation of pleasure, as illustrated in previous section. Considering the haptic semantic information including vibration position and direction that haptic signals provide, we would match the appropriate vibrotactile intensity as well as stroking speed to increase pleasant sensation from both the information transfer and emotion perspectives.

4.2 Increase the Haptic Immersive Experience

One of the characteristics of haptic feeling and interaction is ambient and immersive experience. When a user is touching an object, feeling the zero-distance of himself with this device, he or she will enhance this immersive experience. So, affective haptic factors under the enhanced immersive condition will be magnified and deeply influence user’s emotion and interaction experience. From the other perspective, “user experience” emphasizes the user subjective feeling during interaction, while this subjective feeling largely reflects user’s emotional state. Thus, the design of emotion is significant to increase haptic interaction experience.

4.3 Multimodal Interaction by the Sense of Vision, Audition and Touch

As mentioned before, the sense of touch has relatively low information capacity. In terms of our daily human-computer interactions, from the perspective of author, the proportion of the channel of vision, audition and touch is roughly 7:2:1. Therefore, the interaction research of haptics is not haptic-only, but the interaction of multimodal. For example, in the scenario of virtual reality (VR), users can put wearable device to have immersive experience and users’ fingers and body can also feel haptic signals corresponded with the scenario. Although there might be different information in terms of various channels, user’ emotion state is consistent and on top of each single modal. Therefore, designers need to combine haptic emotion with the emotions of vision and audition, offer a holistic design solution and avoid emotional incongruity of each channel to create a unified and coordinated emotional experience.

5 Concluding Remarks

This paper explores the affective haptics and interaction design. By the literature review of human emotional feedback on haptic signal and affective haptic devices, the haptic interaction design principles and instructions are explored. In conclusion, haptic interaction design cannot be separated with the research and application of emotion. Based on the physiological characteristics of human haptics and emotional feedback, designers need to involve haptic signals that satisfy the user’s emotional needs into the multimodal interaction scenario. Meanwhile, designers also need to refer to the state of the art of affective haptic devices, focus to the detailed interaction scenarios and real needs of the users to offer better affective experience.