Keywords

1 Introduction

A negative body perception or misconception can cause an elevated risk of eating disorders, isolation, and mental disease [16, 29, 34]. Such distortions of one’s own body image affect one’s way of interacting with others and with the environment, negatively impacting one’s emotional state [29, 57]. Thus, being able to manipulate this perception through technology can open up the opportunity for novel and more effective therapies for people with body perception disorders, and allow for new design explorations of virtual body augmentation [1, 37]. Research in neuroscience [20, 56] has shown that our brain holds several mental models of one’s own body appearance, known as body perceptions, necessary for successful interactions with the environment. These studies have demonstrated that these body perceptions are continuously updated in response to sensory inputs received from outside and inside the body [21]. Our study suggests an opportunity to design sensory feedback oriented toward changing one’s BIP.

Fig. 1.
figure 1

Graphical representation of the concept of BIP, which refers to the picture we have in our mind of the shape, size, and form of our body.

In this paper, we investigate for the first time how olfactory feedback can be used and combined with auditory feedback to change a person’s BIP. The influence of multisensory feedback on body perception has been investigated since the early 1990s [47]. Of all the human senses, vision, touch, and proprioception are the ones most studied with respect to their impact on BIP [1, 27, 38], with audition recently joining the effort [46, 50]. Many studies have exploited visual-tactile stimulations to induce body illusions, in which the participant can embody an avatar with a different appearance (e.g., different weight [40], body mass index [39], body shape [37], skin colour [38]). Sound, in particular, has been used to manipulate the perception of one’s body appearance and weight [50]. However, an important sense that has been largely unexplored from a BIP perspective is that of smell, which is surprising given the ubiquity of smell (i.e., scents surround us even if we are not aware of them [33]).

While the effects of smell on BIP have not been investigated, we see a growing interest in the integration of the sense of smell in the design of technology [13] and in several emerging HCI studies [12, 22, 25]. These emerging efforts are prompted by the proliferation of scent-delivery systems and interfaces (e.g., [12, 13, 23, 30, 36]) that enable the scientific investigation of the role of smell in various contexts. We contribute to this emerging research and design space by investigating the effect of different scent stimuli on a person’s BIP. We particularly draw upon the rich knowledge generated in multisensory perception research and crossmodal correspondences [11, 49]. We investigate the relationships between scent and body shape, and furthermore the relationship between scent, sound, and BIP.

In summary, our work offers contributions on three levels: (i) insights into the relationship between smell & BIP, (ii) the integration of scent & sound to change BIP, and (iii) implications regarding the ability to modify our BIP through multisensory stimuli.

2 Related Work

In this section, we discuss related works that have explored the concept of BIP, the use of technology and sensory feedback to change BIP, and the unknown effect of scents based on crossmodal correspondences research.

2.1 Body Image Perception (BIP)

Our brain holds at least three types of mental body representations: body image, body schema, and peripersonal space. With the last term, we refer to the space surrounding our body, where the body can grab and manipulate objects, integrating visual, motile, and proprioceptive feedback [20]. The second term, body schema, refers to the representation of the different parts of the body in space [21]. The first and relevant term for our work, body image, defines how we picture our body [43], or in other words, how we perceive what our body looks like. The term “body image” was coined in 1935 by Slade [47], who defined it as “the picture we have in our minds of the size, shape and form of our bodies, and our feelings concerning these characteristics and our constituent body parts” (Fig. 1). This definition specifies two relevant components: body size estimation (as a perceptual task) and feelings towards the body (i.e., tactile, visceral, sensorial, postural, and emotional experience) [47]. More recently, studies have shown that our BIP is continuously updated according to the feedback received from our actions or to the action exerted on our body (e.g., the role of visual, tactile and proprioceptive cues to induce embodiment) [21, 28, 43]. There are many studies in neuroscience and HCI that have used these findings to investigate embodiment and body perception in virtual reality (i.e., VR) [48]. However, there is no research on the concept of BIP and its perceptual modulation through the sense of smell. Before we highlight the potential around smell, we review relevant research on the change of BIP through sensory stimulation.

2.2 Changing BIP Through Vision, Touch, and Sound

The perception people have of their BIP does not always match with reality. According to Garner and Garfinkel [16], there are two types of BIP distortion: (i) perceptual (i.e., body size distortion: a misperception of one’s own size and shape), and (ii) cognitive (i.e., body dissatisfaction: feeling uncomfortable with one’s own body).

Nowadays, as a result of technological progress, the concept of body misperception can be widely studied and reproduced by carefully designing sensory feedback that changes BIP [24, 37, 41]. Visual, tactile, and proprioceptive are the stimuli most used to change and modulate BIP [9]. In addition, Tajadura-Jiménez et al. [52, 54, 55] showed that it is possible to induce body illusions through sound stimuli. These studies showed that people’s perception of the length of their limbs (arms [54], legs [52], fingers [55]) can be altered by modifying the sound feedback that those limbs make as they are moved in the environment. Most relevant for our work, Tajadura-Jiménez et al. [50, 51] showed that changing the sound frequency of participants’ footsteps could make participants feel lighter or heavier. They built a device able to change the perceived sound of a participant’s footsteps in real time [50]. Shifting the frequency spectra of the footstep’s sound was found to alter the perceived body weight. More specifically, shifting the sound of the footsteps to lower frequencies resulted in the perception of a heavier body, while shifting the sound to higher frequencies resulted in the perception of a lighter body. The results showed that the induced changes in BIP also caused changes in walking behaviour. The same device was used with patients affected by complex regional pain syndrome who suffer from strong BIP distortions [51]. The above studies show how controlled manipulation of sensory stimuli contributes to changes in a person’s BIP. Such knowledge does not yet exist for the sense of smell.

2.3 The Unknown Effect of Smell on BIP

The sense of smell has not yet been studied in relation to BIP. This could be because of its invisibility and dissociation from any specific action performed by the body. To date, the only sensory feedback that has been demonstrated to influence BIP is associated with an action performed by or on the body, or a representation of it (e.g., the movement of our hands [55], the sound of our footsteps [50]). With respect to our sense of smell, we are possibly less aware of our own scent (due to olfactory adaptation [8]) and what an action may smell like. Given the immediacy and ubiquity of smell and its importance for our daily life and emotional well-being, including its unknown effect on BIP, we deem it worthwhile to investigate in order to open up future research and design directions.

Several studies have investigated multisensory associations between smell and other modalities [29], adding new knowledge to crossmodal correspondence research. This research provides evidence for an association between scents and visual feedback during grasping movements. For example, some research has shown that participants opened their fingers wider when the scent evoked large objects (e.g., an orange), compared to a scent evoking smaller-sized objects (e.g., an almond) [5, 6]. Other research has shown an association between scents and colours (e.g., between strawberry scent and red colours, or spearmint scent and turquoise [2, 10, 35, 42]). Similarly, some scents have been shown to be associated with the classical “Bouba-Kiki” paradigm [19, 44], where the sound of “Bouba” is associated with a rounded shape (i.e., visual) and “Kiki” is associated with a spiky shape. Hanson-Vaux et al. [19] found that lemon and pepper were associated with spiky shapes, while raspberry and vanilla with rounded ones.

Furthermore, multisensory associations have also been reported between sound and the sense of smell, which are more integrated into the perception than is commonly assumed. This was pre-figured in 1857 by Piesse [18], who wrote: “Scents, like sounds, appear to influence the olfactory nerve”. Later, Grimshaw and Walther-Hansen [18] proved that sound can influence our ability to interiorise scents, and they built an olfactory device that produces scents instead of sound waves, called a “Smeller”. Visually similar to an organ, the “Smeller” could play “smellodies” instead of melodies [18]. It was shown, also, that different scents were matched with different sound pitches, such as fruity scents with high- pitched sounds [2, 11]. These results point to a subconscious semantic correspondence between smell and sound. In our work, we used this existing but under-exploited association between smell and sound.

3 Exploration of Combined Smell and Sound on BIP

Based on the research on sensory stimuli and BIP, we explored the unknown effect of smell on users’ BIP. We did this through two user studies. As previously done for other types of feedback, we first ran Study 1 to investigate if there is an association between scent stimuli and the body through the crossmodal correspondence between scent and shapes (building on the “Bouba-Kiki” paradigm described above). We presented a body silhouette (Fig. 2b) as a representation of the body shape while perceiving a scent.

Following the confirmatory results from Study 1, in Study 2 we examined the multisensory impact of combined scents and sound on BIP, following the findings from prior work on BIP adaptation [2, 11] and inspired by the idea that “what is essential in the sensuous-perceptible is not that which separates the senses from one another, but that which (...) unites them among themselves” (Von Hornbostel, The Unity of the Senses, 1927/1950, p. 214) [49, 51]. We exploit this unification of the senses to investigate the multisensory effects on BIP through combining two devices: first, a scent-delivery device that allows full control over scent delivery [50]; second, a device that is able to change the sound of users’ footsteps in real time, similar to that described in [12]. In the following section, we describe each of the two studies in detail.

4 Study 1: Scents and Body Shape Associations

The first user study aimed to validate scent-shape associations reported in prior work [19, 49] and establish an understanding of scent and body shape associations. More specifically, we investigated whether thin and thick body silhouettes (Fig. 2b) are perceptually linked with different scents.

4.1 Experimental Design

Based on prior work on crossmodal correspondences between scents and shapes [19], we selected vanilla and lemon scents. Both scents are described as pleasant scents and are correlated with rounded and spiky shapes respectively (building on the “Bouba-Kiki” paradigm [19, 49]). We added an unpleasant scent (i.e., civet - a pungent animal scent) because it has been found that pleasantness and unpleasantness are respectively associated with curved and spiky shapes [44]. More importantly, the unpleasant scent was used to remove the confound of valence on the perception of body shapes. In addition, we used water as a neutral olfactory stimulus (i.e., there is nothing like air as a neutral scent). To represent visually the shape of the body, we used 2D body silhouettes (www.makehuman.en) (Fig. 2b). We also captured possible relationships between the selected scents and sounds (i.e., low- and high-pitched sounds), with two audio tracks played and other crossmodal correspondences (i.e., pleasantness and temperature using the two pairs of words pleasant-unpleasant and cold-hot) to achieve a more comprehensive overview of the topic.

Overall, the experiment followed a within-participant design and was composed of four blocks (one for each scent and the neutral condition) of 10 trials each. The experimental blocks were presented in a randomised order.

4.2 Setup and Procedure

The study took place in a quiet room. An information sheet and a consent form were provided to the participants upon their arrival. Participants sat in front of a desk with a computer screen while wearing headphones. We delivered the olfactory stimuli through a 3D-printed nozzle (diameter 3.5 cm) that was positioned at 1 m distance from the participant. Throughout the study, participants were asked to rest their chin on a chinrest to keep the distance consistent between participants (details in Fig. 2(a). The scent-delivery device used (described in [12]) allows full control over scent delivery and regulates its exact timing: as previously tested with respect to scent lingering, the residual effect of released scent is no longer perceived after 10 s using 1 Bar pressure (we delivered scents every 24 s at 1 Bar pressure). The device is composed of 4 electrovalves (4 mm Solenoid/Spring pneumatic valve) that regulate the air passage (on-off) from a tank of compressed air. The tank (70 l/s, maximum pressure of 8 Bar) supplies airflow through 4 mm plastic pipes linked through electrovalves and which open into four small glass bottles that contain 2.5 ml of three undiluted natural essential oils (lemon, vanilla, and civet) and the same amount of water (neutral condition). Before each of the four blocks, participants were informed of the upcoming delivery. After that, they were asked to rate the perceived scent using a Visual Analogue Scale (VAS, measures used below). The study lasted 30 min and was approved by the University Ethics Committee.

Fig. 2.
figure 2

(a) Setup Study 1. A: nozzle to deliver scents (positioned 1 m distance); B: tank of compressed air; C: electrovalves; D: glass bottles to contain essential oils and water. During the experiment participants rested their heads on a chinrest. (b) VAS capturing participants’ responses on five main items, with A representing the body silhouettes.

4.3 Measures Used

To capture participants’ feedback on the associations, we used two questionnaires (i.e., the VAS, post association task questionnaire) and some open feedback questions. To rate all the measurements, we used continuous scales from 0 to 1. Participants were asked to enter the answers by moving the mouse to the desired point on the scale. The distance from the beginning of the scale to the point was computed and used as a score (i.e., a number with two decimals).

Scent Association Task with Visual Analogue Scale (VAS). We gathered participants’ feedback on five pairs of scent associations and attributes (Fig. 2b): (A) thin vs thick body silhouettes; (B) spiky vs rounded shapes [19]; (C) low vs high pitch [59]; (D) pleasantness vs unpleasantness [44]; (E) cold vs hot [60]. The VAS was presented on a screen. All the attributes (Fig. 2b) were randomised and presented in a counterbalanced order. Every participant was exposed to ten trials, half of the trials with the order of anchors left-right and the other half with the reversed order, right-left.

Post Association Task Questionnaire. After completing all the VAS for each scent condition, participants filled in a post association task questionnaire. This questionnaire focused on the emotions elicited by the scent stimuli and the liking and perceived intensity of the scent. To capture the emotional effect we used Self-Assessment Manikins (SAM) to rate valence, arousal, and dominance [4]. For liking and intensity ratings, we used two 0–1 continuous scales (from 0 = “not at all” to 1 = “very much” and from 0 = “very weak” to 1 = “very strong”).

Subjective Feedback and Experiences of the Scent Stimuli. Following the post task questionnaire, we delivered each scent one more time and asked participants to label the scent typing its name, if recognised, alongside any other descriptors they could think of.

4.4 Participants

An a priori statistical power analysis was performed for sample size estimation in G*Power, using a repeated measures ANOVA with 40 trials (10 VAS items \(\times \) 4 scents). A power of 0.95, an \(\alpha \) level of 0.05, and a medium effect size (f = 0.25, \(\eta \)p2 = 0.06) require a sample size of approximately twelve participants. We recruited fourteen people to take part in the study (ten males, Mage = 29.64, SD = ±5.26). They had normal or corrected-to-normal vision and hearing and no olfactory impairments (e.g., allergies, cold).

4.5 Results

Here we present the results of the analyses of the VAS scores and the post association task questionnaire, alongside participants’ subjective responses on elicited emotions, liking, and perceived intensity of the scents. We opted for the use of one-way repeated measures multivariate analysis of variance (MANOVA) due to the within-participant study design and because it allows exploration of the factor interactions across dependent variables. Furthermore, MANOVA is quite robust to moderate deviations from normality [31].

Fig. 3.
figure 3

Bar charts representing (from left to right) the same five items shown in Fig. 2b. X = scents, Y = Means (from .2 to .7).

Scent Association Results Based on the VAS. We ran a MANOVA, with one independent factor (scent) at four levels (lemon, vanilla, civet, and water). The results shown in Fig. 3 report a significant difference for all the attributes based on different scents (F(15, 132.908) = 4.64, p < .001, Wilk’s \(\lambda \) = 0.313, \(\eta \)p2 = .35). As shown in Fig. 3, results illustrate a significant effect of scent on body silhouettes ((F(3,52) = 10.688 p < .001 \(\eta \)p2 = .381); pairwise comparisons with Bonferroni correction showed significant differences (p < .05) between lemon and vanilla, and lemon and civet scent conditions. With respect to shapes, we found a significant effect for scent (F(3,52) = 11.210 \(p\,{<}\,.001\) \(\eta \)p2 = .393); pairwise comparisons with Bonferroni correction showed significant differences (p < .05) between lemon and vanilla, lemon and water, and lemon and civet scent conditions. Moreover, concerning sound, a significant effect was found (F(3,52) = 5.605 p = .002 \(\eta \)p2 = .244); pairwise comparisons with Bonferroni correction showed significant differences (p < .05) between lemon and vanilla, and lemon and civet scent conditions. Also for liking, we found a significant difference (F(3,52) = 11.417, p < .001, \(\eta \)p2 = .347); pairwise comparisons with Bonferroni correction showed significant differences (p < .05) between lemon and civet, vanilla and civet, and water and civet scent conditions. Finally, a significant effect on temperature was shown (F(3,52) = 6.495, p = .001 \(\eta \)p2 = .273); pairwise comparisons with Bonferroni correction showed significant differences (p < .05) between lemon and vanilla, and lemon and civet scent conditions.

In summary, the results suggest that lemon is associated with thin silhouettes, spiky shapes, high-pitched sounds, and coldness. Vanilla is associated with thick silhouettes, rounded shapes, low-pitched sounds, and warmth. Civet is associated with thick silhouettes, rounded shapes, low-pitched sounds, and warmth. Moreover, as expected, both lemon and vanilla were considered pleasant (Fig. 3). In the following section, we report additional analyses on participants’ emotional responses.

Post Association Task Questionnaire Data. It is known that smell and emotion are strongly correlated. Hence, we present the analysis of the emotional responses in detail. We ran a MANOVA with one independent factor (scent) at four levels (lemon, vanilla, civet, and water) for valence, dominance, and arousal.

The results report a significant difference for all the dimensions based on different scents (F(15, 132.908) = 4.64, p < .001, Wilk’s \(\lambda \) = 0.335, \(\eta \)p2 = .30).

Results showed a significant effect of scent on valence (F(3,52) = 9.632, p < .001, \(\eta \)p2 = .357). Further comparison tests with Bonferroni correction showed a significant difference between lemon and water (p = .021), lemon and civet (p = .001), and vanilla and civet (p = .005). With respect to arousal, we found a significant effect for scent (F(3,52) = 8.240, p < .001, \(\eta \)p2 = .322); comparison tests with Bonferroni correction showed a statistical difference between lemon and water (p < .001). Concerning dominance, the results did not show any significant difference. Overall, these results indicate that both lemon and vanilla scents were perceived positive (resp., M = .0.72, SD = ±0.18; M = 0.60 SD = ±0.24). Civet was rated as negative (M = 0.35, SD = ±0.19). Furthermore, lemon and civet were much more arousing than vanilla and water (resp.; M = 0.69, SD = ±0.21; M = 0.66, SD = ±0.17; M = 0.47, SD = ±0.21; M = 0.32, SD = ±0.29).

Liking, Perceived Intensity, and Subjective Feedback. We ran a MANOVA on the ratings in the post-association questionnaire, with one factor (scent) at four levels (lemon, vanilla, civet, and water).

Results showed a significant effect of scents on liking (F(3,52) = 9.628, p < .001, \(\eta \)p2 = .357). Further comparison tests with Bonferroni correction showed a significant difference between vanilla and civet (p = .011), lemon and civet (p < .001), and lemon and water (p=.022). We also found a main effect of scents on the perceived intensity ratings (F(3,52) = 11.022, p < .001, \(\eta \)p2 = .389). Further comparison tests with Bonferroni correction showed significant differences between lemon and water (p < .001), vanilla and water (p = .023), and civet and water (p < .001). These effects illustrate high liking levels for lemon and vanilla scents and low liking levels for civet (resp., M = 0.72, SD = ±0.19; M = 0.57, SD = ±0.25; M = 0.32, SD = ±0.22). Moreover, lemon and civet were perceived more intensely than water (resp., M = 0.68, SD = ±0.18; M = 0.67, SD = ±0.17; M = 0.25, SD = ±0.26), while vanilla did not differ significantly from the other scent conditions (M = 0.51, SD = ±0.28). In addition, based on the qualitative feedback at the end of the study, despite intersubjective differences, we observed a partial recognition of the scents; in fact, 8/14 participants recognised lemon, 7/14 recognised vanilla or something sweet (e.g., chocolate), 12/14 said air or nothing, and 7/14 described civet as negative (e.g., fertiliser, rubbish).

5 Study 2: Scent-Sound Effect on BIP

Having established a scent-body shape association in Study 1, we further investigated the effect of scents on BIP in Study 2. As a person’s BIP is continuously updated through multisensorial signals [47], we integrated scent with sound stimuli to investigate the multisensory effect on BIP. This is based on the established associations between scents and sounds in prior work [2, 11, 18]. Moreover, as discussed, prior work within HCI has demonstrated that sounds modulated through a novel device can change a person’s perceived body weight [50]. Hence, building on this cross-disciplinary research, we investigated for the first time the multisensory effects on BIP by combining two stimuli: olfactory and auditory.

5.1 Experimental Design

We combined two devices. We used a shoe-based device built similarly to the one used by Tajadura-Jiménez et al. [50]. This device consists of a pair of strap sandals with a stiff rubber sole able to elicit clear and distinctive footstep sounds. Each sandal was equipped with a microphone (Core Sound) that captured the walking sound (Fig. 4a). Both microphones were connected to a small stereo pre-amplifier (SP-24B) which was connected to an equaliser (Behringer FBQ800) that changed the sound spectra. The resulting sound was played back to participants via headphones. We integrated this device with the scent-delivery device used in Study 1. This time participants were standing; hence, we adjusted the scent-output nozzle for the specific setup (details in Setup & Procedure below and Fig. 4a). We were particularly interested in exploring the interaction between scent and sound, and whether scent could increase or affect the sound-driven BIP modulation found by Tajadura-Jiménez et al. [50]. We based our investigation on the existing interactions between olfactory and auditory modalities, which are in general congruently associated with the same types of shapes [7, 11, 49].

Based on the Study 1 results, we selected lemon and vanilla scents, respectively associated with thin and thick body shapes, and water as a neutral condition. Building on prior work [50, 51], we used three sound conditions: (i) a frequency condition in which the frequency of the footstep sounds in the range 1–4 kHz was amplified by 12 dB and those in the range 83–250 Hz were attenuated by 12 dB; (ii) a low frequency condition in which the frequency components in the range 83–250 Hz were amplified by 12 dB and those above 1 kHz were attenuated by 12 dB; (iii) a control condition (no sound manipulation) where participants were provided with their natural footstep sounds equally amplified across frequency bands to study if scents have an impact on BIP when there is no manipulation of the frequency of the produced sounds. We hypothesised that participants would feel thinner and lighter when smelling lemon, and thicker and heavier when smelling vanilla, compared to the neutral water condition. Moreover, we expected that when combining a high-pitched sound with a lemon scent participants would feel even thinner and lighter compared with the sound-only condition. Similarly, when combining a low-pitched sound condition with a vanilla scent, participants were expected to feel even thicker and heavier compared with the sound-only condition.

Overall, this study had a within-participant design with three blocks (one for each scent and the neutral condition) of three trials each (one for each sound condition). The blocks and conditions were presented in a randomised order.

Fig. 4.
figure 4

(a) Setup Study 2. A: nozzle to deliver scents on a tripod (positioned 1 m distance); B: shoe-based device; C: motion-capture sensors. During the experiment, participants walked on the spot and received scent & sound stimuli. (b) 3D body avatar. Real height and weight were typed and kept fixed. Participants were asked to change chest, waist, and hips measures after each trial. The starting points of the sliders were always set in the middle to prevent any bias. The entire body was automatically scaled by the software in function of the manipulation of the parameters.

5.2 Setup and Procedure

The study took place in a quiet room. An information sheet was provided to the participants upon their arrival, containing the details of the experiment (e.g., tasks, devices, measures). They were also asked to sign a consent form. Afterwards, participants were asked to adjust the size of a 3D avatar using a body visualisation tool (www.bodyvisualizer.com) (details in Methods used below and Fig. 4b). Thereafter, the researcher assisted participants with putting on the shoe-based device, the backpack (containing the wires and the amplifier), and a pair of sensors to track their gait (Fig. 4a). Participants were then asked to move onto a wooden board and to walk on the spot without wearing the headphones, just to familiarise themselves with the device and setup. When they felt comfortable, they were asked to put on the headphones and follow the instructions. Participants were instructed to start and stop walking on the spot by vocal commands (following the procedure in [50]). The walking lasted one minute, during which a scent was delivered three times (i.e., t1 = 0 s, t2 = 24 s, t3 = 48 s). In other words, the scent was delivered at the start and then every 24s (accounting for 6 s scent lingering time + 18 s pause). We used the same scent-delivery device as in Study 1. However, in this study the nozzle was mounted onto an adjustable tripod at 1 m distance to adapt the delivery of the scent to the participants’ height (Fig. 4a).

After the walking phase, participants were asked to use the body visualisation tool (i.e., the 3D body avatar, see Fig. 4b) to review the chest, waist, and hip measurements. Finally, they were asked to complete a short questionnaire about their emotions (SAM) and perceived body behaviour (details in Methods Used below). This procedure was repeated for each of the three blocks. After completing each block, participants were asked to rate the scent based on the perceived intensity and liking. At the end, they were provided with a Debriefing Sheet with more details about the study, contact information, and a sign-up option to receive further details (e.g., publication of the results). The study lasted 45 min and was approved by the University Ethics Committee.

5.3 Methods Used

To capture participants’ responses on the scent-sound feedback while walking, we used a combination of subjective and objective measures. To rate body feelings, we used continuous scales from 0 to 1. Participants were asked to enter the answers by moving the mouse to the desired point on the scale. The distance from the beginning of the scale to the point was computed and used as a score (i.e., a number with two decimals).

Body Feelings. To measure the BIP, we needed an adjustable body visualisation tool that went beyond the static use of the body silhouettes from Study 1. Hence, we used a 3D body avatar (Fig. 4b) before and after each walking phase. The researcher chose the gender and fixed height and weight, according to the real ones, without any possibility of later changes. Participants, instead, were able to increase or decrease chest, waist, and hips by matching visually the part of the body in the silhouette to their perceived body. The entire body was automatically scaled by the software in function of the manipulation of a selected parameter. The numerical values associated with the sliders were hidden.

In our study, we only focused on chest, waist, and hips according to the literature on BIP disease, which recognises them as critical regions [15] for the allocentric perception people have of their body. We collected additional feedback on participants’ BIP (following questions used in prior work: [32, 50, 55]). We asked them to rate their level of agreement using continuous scales from 0-“strongly disagree” to 1-“strongly agree” for: (i) sense of agency regarding the footsteps, (ii) vividness of the experience and (iii) feet localization. We inserted this task to ensure they had recognised the amplified sound as their own footstep, as a prerequisite for bodily illusion to take place. We also asked participants to assess their perceived walking speed (slow vs. quick), body weight (light vs. heavy), body strength (weak vs. strong), and body straightness (stooped/hunched vs. straight), as additional information about the BIP, using a continuous scale from 0 to 1.

Post Body Feelings Questionnaire. As in Study 1, we measured participants’ emotional responses on three dimensions (i.e., valence, dominance, and arousal) by using the self-assessment manikins (SAM) [4]. Additionally, we asked participants to rate their liking and perceived intensity of the scents used (respectively from 0-“not at all” to 1-“very much” and from 0-“very weak” to 1-“very strong”).

Behavioural Changes (Gait Sensors). Following the approach in prior work [50], we also asked participants to fit two wearable motion-capture sensors (www.wearnotch.com) on the left calf and thigh to collect the position data to quantify the maximum foot lifting elevation, acceleration, gait speed, and number of footsteps in each condition.

5.4 Participants

An a priori statistical power analysis was performed for sample size estimation in G*Power, using a repeated measures ANOVA with nine trials (three sound conditions \(\times \) three scents). A power of 0.95, an \(\alpha \) level of 0.05, a medium effect size (f = 0.25, \(\eta \)p2 = 0.06), requires a sample size of approximately twenty-two participants. We recruited twenty-two participants to take part in this study (fifteen males, nine females, Mage = 27.96, SD = ±6.08). They reported having normal or corrected-to-normal vision and audition, with no olfactory impairments (e.g., allergies, cold, flu). Out of the twenty-two participants in Study 2, eight participated also in Study 1. However, Study 1 & 2 are focused on different aspects, and in Study 2 there is no effect of experience.

5.5 Results

Here, we present the results of the body feelings questionnaire, alongside the participants’ subjective responses on elicited emotions, liking, and perceived intensity of the scents. We used a MANOVA due to the within-participant design and because it allows exploration of the factor interactions across dependent variables. Furthermore, MANOVA is quite robust to moderate deviations from normality [31]. For the gait data, we used a repeated measures ANOVA since the dependent variables were calculated based on the same data and were interconnected.

Body Feelings Data. Concerning the 3D body visualiser (Fig. 4b) data, we ran a one-way repeated measure MANOVA, with 3 \(\times \) 3 factors (scent and sound), for each variable (hips, waist, and chest). As extra testing, we calculated the difference between the baseline condition, recorded before the beginning of the experiment, and all the others, recorded after each trial. We did not find any significant effects.

For the questionnaire items on the body feelings, we ran a one-way repeated measure MANOVA, at 3 \(\times \) 3 factors (scent and sound). The results report a significant main effect of scents (F(28,352) = 1.63, p = 0.25, Wilk’s \(\lambda \) = 0.784, \(\eta \)p2 = .115). However, there are no main effect of audio (F(28,352) = .356, p = .999, Wilk’s \(\lambda \) = 0.946, \(\eta \)p2 = .028) and no interaction (F(56,686.78) = .507, p = .999, Wilk’s \(\lambda \) = .854, \(\eta \)p2 = .039).

These results were obtained despite the fact that, according to Fig. 5, participants agreed that the sounds they heard were produced by their own body (sense of agency) and they did not find the experience more vivid or surprising than normal (vividness and surprise). The reason to test these items was to make sure people felt all the conditions similarly and, since the means of the variables for each condition do not differ significantly, we can say that the setup worked. Furthermore, we found no effect of sound or scent or their interaction on the level of confidence participants reported when localising their feet. Figure 5 also provides an overview of the additional items relevant to the BIP. The results showed a main effect of scent on straightness (F(2,189) = 5.071, p = .007, \(\eta \)p2 = .051), and further comparison tests with Bonferroni correction showed a significant difference between vanilla and lemon (p = .005).

Results showed a main effect of scent on weight (F(2,189) = 3.645, p = .028, \(\eta \)p2 = .037), and further comparison tests with Bonferroni correction showed a significant difference between vanilla and lemon (p = .030). No main effect of sound and no interaction effects were found for both factors. Further, no significant effects of sound and scent were found for the strength and speed self-reported variables.

Overall, these results indicate that the lemon scent, as we hypothesised, is associated with the feeling of lightness and standing straight; the vanilla scent, instead, is associated with heaviness and being stooped.

Fig. 5.
figure 5

Means (SD) for SAM and body feelings questionnaire data (continuous 0–1 scales). *= significant mean differences.

Post Body Feelings Questionnaire. After each walking phase, we measured participants’ emotional feelings through the three SAM scales. To rate each of the three dimensions we used a 0–1 continuous scale, and participants had to move the slider to the desired point on the scale. We normalised our data with the LOG-transformation and ran a one-way repeated measures MANOVA, with 3 \(\times \) 3 factors (scent and sound).

As demonstrated in Fig. 5, results showed a significant effect of scents on valence (F(2,189) = 6.126, p = .003, \(\eta \)p2=.061), and further comparison tests with Bonferroni correction showed a significant difference between lemon and vanilla (p = .002). No main effect of sound or interactions were shown. A significant main effect of scent on dominance was also shown (F(2,189) = 5.063, p = .007, \(\eta \)p2 = .051), and further comparison tests with Bonferroni correction showed a significant difference between lemon and vanilla (p = .007). No main effect of sound or interactions were shown. A trend of scent on arousal was also shown (F(2,189) = 2.945, p = .055, \(\eta \)p2 = .030), and further comparison tests with Bonferroni correction showed a trend between lemon and vanilla (p = 0.114) and lemon and water (p = .108). Overall, these results indicate that lemon caused participants to feel more positive compared to vanilla and water. Furthermore, lemon conveyed a stronger feeling of control (p = .005) and was more arousing than vanilla (trend).

Gait Sensors Data. For the gait data, we calculated the elevation angle, average of steps, lift velocity and acceleration, inter-step interval, and the number of steps/minutes. We used one-way repeated measure ANOVAs, with 3 \(\times \) 3 factors (scent and sound).

The results of the tests of within-subject contrasts showed a significant main effect of sound on velocity (F(1,18) = 4.673, p = .044, \(\eta \)p2 = .206), and further comparison tests showed a significant difference between the high frequency and low frequency condition (p = .004) (Fig. 6b). A significant quadratic main effect of sound on acceleration was also shown (F(1,18) = 6.790, p = .018, \(\eta \)p2 = .274), and further comparison tests showed a significant difference between the high frequency and low frequency condition (p = .042) (Fig. 6a). We did not find any significant main effect of scent. However, we found a significant interaction effect between sound and scent for acceleration (i.e., increased acceleration was detected during the high frequency condition and when smelling the lemon scent) (F(1,19) = 5.381, p = .032, \(\eta \)p2 = .221).

Overall, these results show an increase in velocity and acceleration in the high frequency condition compared to the low frequency condition (i.e., participants walked faster in the high frequency condition) (Fig. 6a and b). There is also an interaction between lemon scent and sound (p = .032): when smelling the lemon scent instead of vanilla participants walked faster during the high frequency condition and slower during the low frequency condition (Fig. 6a).

Fig. 6.
figure 6

(a) Line chart scent-sound interaction effect on acceleration. X = scents, Y = Means. (b) Line chart main effect of sound on velocity. X = sound, Y = Means.

Liking, Perceived Intensity and Subjective Feedback. Using the ratings from the post association questionnaire, we ran a one-way multivariate ANOVA, with one factor (scent) with three levels (lemon, vanilla, and water).

Results suggest a significant effect of scents on liking (F(2,69) = 10.595, p < .001, \(\eta \)p2 = .235). Further comparison tests showed a significant difference between lemon and water (p < .001) and lemon and vanilla (p = .001). We also found a main effect of scent on the perceived intensity ratings (F(2,69) = 49.126, p < .001, \(\eta \)p2 = .587). Further comparison tests showed significant differences between lemon and water (p < .001), water and vanilla (p < .001), and lemon and vanilla (p = .002). These effects illustrate a liking for lemon scent (M = 0.72, SD = ±0.25) and neutral judgements for water and vanilla scent (resp., M = 0.41, SD = ±0.25; M = 0.44, SD = ±0.28). Moreover, as expected, lemon and vanilla were perceived more intensely than water (resp., M = 0.75, SD = ±0.15; M = 0.51, SD = ±0.29; M = 0.10, SD = ±0.22).

Finally, based on the qualitative feedback and, despite intersubjective differences, we observed a good understanding of the scents; 21/24 participants recognised lemon, 12/24 recognised vanilla or something sweet (e.g., flower), and 23/24 said air or nothing.

6 Discussion

We showed for the first time an association between scents and BIP, and a combined interaction between scents and sounds.

In Study 1 we investigated the existence of scent and body shape associations using 2D body silhouettes (Fig. 2b). We used a VAS to test the relationship between lemon and vanilla, previously associated with spiky and rounded shapes [19, 45, 49], and thin and thick body silhouettes. As both scents were known to be pleasant (positive valence), we added civet as an unpleasant scent (negative valence) to remove the confound of valence. Our results confirmed the association of lemon with thin silhouettes and vanilla with thick ones. These findings contribute new knowledge to the field of crossmodal correspondences for smell. For the first time, our work focused on the body and how we picture ourselves using a simple body silhouette visualisation.

To deepen this line of research, in Study 2 we used an adjustable 3D body visualiser (Fig. 4b) and a questionnaire about body feelings to capture changes on BIP based on the combined scent-sound stimulation. We found, through the questionnaire, that the scent of lemon results in feeling lighter, while the scent of vanilla makes participants feel heavier. When combined with the sounds of a participant’s footsteps, which were modified as in [50] to make participants perceive their body as heavier or lighter, the results showed that the highly arousing scent of lemon enhances the effect of high-pitch sounds related to perceiving one’s body as lighter. We observed also that, captured in the self-report questionnaires, participants tended to feel as if they were walking with a straighter posture when smelling lemon than when smelling vanilla.

Implicit Effect of Scent-Sound Stimulation on Gait. While our study results are based on self-report data, we also captured data on participants’ gaits using sensors attached to participants’ left calves and left thighs. Based on prior work [50], we expected that shifting the pitch of walking footstep sounds would modulate the gait of participants making them adopt a behaviour consistent with having a lighter/heavier body. We hypothesised that the addition of a lemon scent would result in feeling lighter when combined with high-pitched sounds. This would also result in an increased velocity and acceleration of gait. For the vanilla scent, we assumed an association with a feeling of heaviness when combined with low-pitched sounds. This would also result in a decreased velocity and deceleration of gait. Results from Study 2 support these assumptions (i.e., lemon & high pitch are linked with faster gait), despite the explicit walking speed questionnaire showing a significant effect only for scents.

We found an interesting interaction between sound & scents: participants walked faster when smelling lemon than vanilla. The results support the literature that emphasises the multisensory concept of BIP and underlines the importance of unified sensory stimulation [49]. Based on our findings, it seems that sound has a stronger effect on unconscious behaviour (i.e., an action we do not think about) based on the gait data results, whilst scent had a stronger effect on conscious behaviour (i.e., actions we are aware of) based on questionnaire results. This hypothesis might show the novelty of the scent stimulation, which can be consciously processed and reflected in the responses.

Emotional Link to Scent Stimuli and Scent-Sound Interaction. We captured participants’ emotional reactions after each trial. Although the emotional effect of scent and its relation to the perceived body image was not a key focus of our work, the emotional link to scent stimuli and to scent-sound interaction is important to consider in HCI and experience design. We found that the interaction between scents and sound had an emotion-modulation effect. In other words, we explain the fact that lemon scent enhanced the acceleration of gait during the high pitch condition and the deceleration of gait in the low-pitch condition (Fig. 6a) by referring to the concept of dominance (tested through SAM). A heavy feeling is often linked to a sense of strength that could be associated with high levels of dominance and arousal. Based on this assumption, it is possible that the lemon scent, being arousing, may have enhanced the feeling of lightness during the high pitched sound condition, and, at the same time, enhanced the dominance level during the low pitched sound condition. These findings might also be explained by the unequal gender sample (i.e., we tested more male than female participants) and the possible association between the feeling of heaviness, strength, and dominance typical of males. A. Tajadura-Jiménez’s work [53], found that the effect of sound interacted with participant’s actual body weight and masculinity/femininity aspirations, the effect of the high pitch condition was enhanced on people reporting a wish to be more feminine, and the effect of the low pitch condition was enhanced on people reporting a wish to be more masculine. Hence, it would be interesting to investigate how personality characteristics could affect BIP and how smell can have an impact.

Overall, our work opens up a range of future research and design endeavours, ranging from integrating scent and scent-sound stimulation in the therapy of body misperception diseases (e.g., anorexia), and to the creation of novel augmented and embodied experiences (e.g., in VR). Below we discuss the implications of our findings for HCI, highlighting the opportunities for future work.

6.1 Usage Scenarios

Based on our findings, we now explore possible design opportunities. We present three possible scenarios for the use of multisensory stimuli in the context of BIP.

Scenario 1: One possible scenario is linked to the creation of more embodied interaction experiences in VR, designing accurate environments to remove ambient distractors (e.g., focusing on a mirror in VR) to avoid possible objectification of olfactory stimuli in VR. Based on our findings, it would be interesting to study if we can enhance gaming experiences and facilitate embodiment in VR (e.g., [1, 38, 40]) by adding scents and modulating the auditory feedback. We see an opportunity to expand on efforts related to the creation and investigation of body illusions in VR, which can be exploited not only for entertainment scenarios but also for training people with distorted BIP (see Scenario 3) (e.g., body illusion in VR for clinical aims [1, 38]).

Scenario 2: Another usage scenario is linked to interactive clothes and wearable technology. In fact, the world of fashion is expanding its boundaries by exploiting multisensory feedback with different purposes. Examples include TapTap (a wearable system that records touch and play back for emotional therapy) [3], and Scarfy (a scarf-based device which detects the way it is tied and can deliver information through shape-change and vibration) [58]. Looking beyond HCI, we see contemporary artists experimenting with olfactory interaction. As an example, Jenny Tiollotson designed an interactive scent outfit (www.escent.ai/smartsecondskin), a smart second skin dress that mimics the body’s circulation system and emits a selection of scents depending on a person’s mood. Our idea is to use scents not only to manipulate emotional states, and to produce relaxing or activating effects, but also, from our study results, to enhance people’s self-confidence in their appearance and recalibrate distorted feelings of body weight.

Scenario 3: As mentioned at the beginning, the way people perceive their own body does not always match their actual appearance. BIP distortions are prominent in relation to eating disorders (e.g., anorexia, bulimia) and chronic pain. In fact, as the National Association of Anorexia Nervosa and Associated Disorders (ANAD) reports, at least 30 million people of all ages and genders suffer from an eating disorder in the U.S (Le Grange, D., 2012) and eating disorders have the highest mortality rate of any mental illness (Smink, F. E.,2012). One of the reasons for the increasing number of reported cases of eating disorders in the last ten years is the “distorted” body representations in our society (e.g., best exemplified by skinny fashion models) [29]. We see an opportunity to use our work as an inspiration to explore design interventions in therapies for patients with eating disorders, but also generically for those with BIP disorders. Moreover, by being able to influence self-esteem, our findings could be applied to mitigate against sedentary lifestyles -one of the main causes of obesity- which affect one quarter of the adult population and more than 80% of school-going adolescents across the world [14, 26]. In fact, the advantages that the sense of smell and sound provide are their ease of use, their subtlety in everyday life, and their mobility (i.e., ubiquitous settings).

6.2 Limitations and Future Work

Although we can envision several implications of our work, we also need to acknowledge some limitations. Firstly, the group of participants was limited to healthy individuals. Considering our Scenario 3, it is important to extend these findings to a clinical sample with body misperception. In order to achieve this aim, we must take into account the cumbersomeness of our scent delivery system and undertake future work on more flexible and convenient devices and measurements. Additionally, since our subject sample did not allow us to test gender differences, it would be interesting to test if they exist, given the media pressure on women regarding body size [2]. It would also be interesting to explore the sense of heaviness as a sense of strength, as some people may want to feel stronger, and thus then heavier, rather than lighter. Our results suggest a possible effect of this type as already suggested in [50]. Secondly, although we combined self-report and sensor data, the insights are limited as BIP is a very complex concept. Capturing more parameters and carrying out repeated measures in different sessions could increase the confidence in the results found. We are also convinced that qualitative data can contribute to the understanding of BIP and the effects of sensory stimuli. We achieved an initial glimpse of the value of qualitative data in both studies (i.e., qualitative feedback). However, we want to expand on this by applying a psycho-phenomenological or think-aloud approach in future studies, previously used in HCI [60] to help people in verbalising their subjective experiences and understand individual differences. Thirdly, we tested only a small set of scents. This choice was based on literature on crossmodal correspondences [19, 49]. However, we could expand our study by adding different scents with similar characteristics of valence and arousal, such as pepper instead of lemon and lavender instead of vanilla [19]. Finally, it would be interesting to explore the use of scents linked to the human body (e.g., sweat [17]).

7 Conclusion

We investigated if and how the sense of smell and its combination with sound can impact BIP. We presented novel insights into the effect of lemon scent on the feeling of lightness, and vanilla scent on heaviness. This effects were further modulated through high- and low-pitched sounds. Most interestingly, we found that each modality contributed to the modulation of BIP in either more conscious behaviour (self-reports through smell) or unconscious behaviour (tracked through sound). Even more interesting is the interaction between smell and sound showing that arousing scents enhanced the effect of sound by making people feel lighter when hearing a high-pitched sound and possibly stronger when hearing a low-pitched sound. This interaction could be affected by how people would like themselves to be. Hence, the results point to a complex interaction between the modalities and emotional factors triggered by scents and people’s desires in relation to their body. Finally, we discussed our findings in the context of HCI and present three usage scenarios where our work could make a difference in the short-, medium-, and long-term (e.g., creation of new embodied experience in VR, integration with wearable and sensor technologies, interventions in therapies for people with eating disorders).