Advertisement

Noncontact Thermal and Vibrotactile Display Using Focused Airborne Ultrasound

Open Access
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12272)

Abstract

In a typical mid-air haptics system, focused airborne ultrasound provides vibrotactile sensations to localized areas on bare skin. Herein, a method for displaying heat sensations to hands where gloves are worn is proposed. The gloves employed in this study are commercially available gloves with sound absorption characteristics, such as cotton work gloves without any additional devices such as Peltier elements. The method proposed in this study can also provide vibrotactile sensations by changing the ultrasonic irradiation pattern. In this paper, we report basic experimental investigations on the proposed method. By performing thermal measurements, we evaluate the local heat generation on the surfaces of both the glove and the skin by focused airborne ultrasound irradiation. In addition, we performed perceptual experiments, thereby confirming that the proposed method produced both heat and vibrotactile sensations. Furthermore, these sensations were selectively provided to a certain extent by changing the ultrasonic irradiation pattern. These results validate the effectiveness of our method and its feasibility in mid-air haptics applications.

Keywords

Heat sensation Vibrotactile sensation Airborne ultrasound 

1 Introduction

Recently, mid-air haptics technologies have attracted substantial interest because they can produce tactile sensations without any physical contact or the need for wearing any devices. Airborne ultrasound phased arrays (AUPAs) are one of the most practical devices in mid-air haptics. They can produce vibrotactile sensations on bare skin based on acoustic radiation pressure [1, 2]. The stimulus area, a focal point generated by AUPAs, can be down to wavelength, i.e., approximately 8.5 mm at 40 kHz (for typical AUPAs). It can be generated at an arbitrary position and controlled electronically. Complex stimulus patterns, such as the shapes of various objects based on multi-foci [3] and focal points with time-division [4], can be produced via the proper drive control of the transducers in AUPAs.

Most existing studies that employ AUPAs have developed applications that can only produce vibrotactile sensations. The realization of other types of haptic sensations using AUPAs, in addition to vibrotactile sensations, can expand the range of applications of AUPAs and contribute to the evolution of mid-air haptics technologies. In this paper, we propose a method to produce heat sensations using AUPAs, in addition to producing vibrotactile sensations.

The proposed method requires users to wear gloves to produce heat sensations, whereas existing methods in mid-air haptics do not have such requirements. This is a limitation of the proposed method; however, the gloves that are required in the proposed method are ordinary ones, such as cotton work gloves that absorb ultrasound, without any additional devices such as Peltier elements. Additionally, vibrotactile sensations can be produced by changing ultrasonic irradiation patterns. We can find some practical applications where wearing gloves are acceptable. For example, at numerous factories, workers wear cotton work gloves while they work. Our method can be employed to prevent the inadvertent intrusion of workers’ hands into dangerous zones by imparting a hot sensation as a danger alert. Our method can also be applied to surgery support by improving the surfaces of surgical gloves to absorb sound, where the glove remains disposable and battery-less.

Thermal displays in mid-air haptics, such as methods that employ infrared lasers [5] and thermal radiation [6], have been proposed. A method that uses lasers can also display a tactile sensation similar to a mechanical tap when an elastic medium is attached to the skin [7]. A generic comparison of the proposed method with the aforementioned laser-based methods is not straightforward; however, it is certain that the proposed method is the easiest, with no additional cost, with regard to application in a scenario where a worker wearing cotton gloves is already being aided by ultrasound mid-air haptics. A method for providing a cold sensation using AUPAs has been proposed; however, providing warm and hot sensations was beyond the scope of this method [8].

Herein, we report basic experimental investigations regarding the proposed method. We conducted two kinds of experiments: the first involves temperature measurements on the glove and the skin surface when the glove was exposed to ultrasound. The second is a perceptual experiment for confirming that our method provides heat and vibrotactile sensations and that it can display these sensations selectively by changing the ultrasonic irradiation patterns.

2 Proposed Method

Figure 1 shows an illustration of the proposed method. Both heat and vibrotactile sensations are produced at a focal point by AUPAs on a glove. Any type of glove can be used in this method, as long as it possesses sound absorption characteristics. A cotton work glove is used in this study. The production of heat sensations in this method is based on the phenomenon of sound absorption: heat is generated in the focal point on the glove owing to the absorption of sound, which in turn supplies heat to the skin at the contact points of the skin with the glove via the heat conduction. In addition, the generation of vibrotactile sensations utilizes the acoustic radiation pressure of the ultrasound on the glove.
Fig. 1.

Illustration of the proposed method

These sensations can be provided selectively by two modes of irradiation: static pressure (SP) mode where constant-amplitude ultrasound is irradiated and amplitude modulation (AM) mode where the ultrasound is modulated at 150 Hz. The acoustic absorption coefficient of the glove and the acoustic power at the focal point determine the temperature of the glove exposed to ultrasound. In SP mode, the ultrasound generates heat on the glove while inducing no vibrotactile sensations. In AM mode, the modulated ultrasound produces vibrotactile sensations. The modulation frequency of 150 Hz is selected so that the mechanoreceptors are excited efficiently [9]. The glove temperature also rises in AM mode. However, it is possible to adjust the amplitude and duration of the ultrasound to produce only vibrotactile sensations without heat sensation. The irradiation duration and amplitude should be adjusted to generate only vibrotactile sensations.

We confirmed the feasibility of the selective stimulation in the following experiments.

3 Experiments

3.1 Heat Generation on Glove Surface and Skin Surface

First, we measured the temperature elevation of a cotton work glove that is in contact with a human palm. The glove was exposed to ultrasound, and the temperature was measured both on the surface of the glove and the surface of the palm.

Figure 2 shows the experimental setup. We placed a fabric that was cut from a cotton work glove on an acrylic plate with square perforations of 50 mm side lengths. Subsequently, a hand was placed on the fabric to simulate the scenario of a user wearing the glove. We operated two 40 kHz AUPAs comprising 498 transducers (TA4010A1, NIPPON CERAMIC CO., LTD.) such that a focal point was generated on the fabric surface 200 mm above the surfaces of the AUPAs. The center of the focal point was considered as the origin point, (x, y) = (0, 0) mm, for the measurement. A thermography camera (OPTPI450O29T900, Optris) was employed to measure the temperature distribution on the surface of the fabric irradiated with ultrasound. Simultaneously, four thermocouples discretely arranged between the palm and fabric measured the temperature elevation on the skin. A temperature logger (SHTDL4-HiSpeed, Ymatic Inc.) connected to the thermocouples collected the data with a sampling period of 10 ms. The thermocouples were placed at 5 mm intervals in the x-direction, i.e., at (x, y) = (0, 0), (5, 0), (10, 0), and (15, 0) mm. The ultrasonic exposure patterns were SP mode and AM mode at 150 Hz, and the exposure time was restricted to 10 s in order not to induce pain and burn injuries.
Fig. 2.

Photograph of the experimental setup. (a) Complete view of the experimental setup, (b) enlarged view of thermocouples and fabric cut from a cotton work glove.

Figure 3 shows the images of the surface of the fabric that was exposed to the ultrasound over time, from which it can be clearly observed that the proposed method heated the localized area. The variation in the sizes of the focal points obtained for SP and AM modes can be attributed to distance deviation between the fabric and AUPA surfaces caused by the pressing force variation. Although some sidelobes were formed, the obtained temperature distribution is reasonable because the sound distribution on the surface with the focal point is formed in accordance with the sinc function [2]. The measured temperature was higher in the case of SP mode than that in the case of AM mode because the effective acoustic energy of SP mode is double of AM mode.
Fig. 3.

Thermal images of observed area in the cases of (a) SP mode, (b) AM mode at 150 Hz.

Figure 4 shows the measured results with regard to temperature changes on the skin. These results were obtained by matching the initial average temperatures for the two ultrasonic radiation patterns because the initial average temperatures differed by 1 ℃ for the two patterns (SP: 31.125 ℃, AM: 30.125 ℃). The temperature elevation was highest at the center of the focal point (x, y) = (0, 0) mm, and it decreased with an increase in the distance from the center of the focal point, for both radiation patterns. The temperature at (x, y) = (0, 15) mm did not increase for both radiation patterns. Thus, the proposed method heated the surface of the glove locally. Through comparison, it can be seen that the temperature in the case of SP mode was higher than that in the case of AM mode, at every measurement point. These characteristics were inconsistent with the result in Fig. 3. A temperature of 45 ℃, which induces the sensation of pain [10], was achieved within 5.88 s at the earliest, at (x, y) = (0, 0) mm in the case of SP. Thus, the exposure time must be less than that in this setup. Increasing the number of AUPAs can be considered for achieving a faster increase in temperature because as the input acoustic energy increases, the time needed to induce pain will decrease. Although the relationship between the number of AUPAs and temperature elevation needs to be investigated, it is not considered in this study.
Fig. 4.

Skin surface temperature measured by each thermocouple

3.2 Distinguishing Vibrotactile and Heat Sensations

Next, we conducted perceptual experiments to confirm whether or not our method can generate both heat and vibrotactile sensations, as well as that can switch the generated sensation by the SP-AM mode alternation.

Figure 5 shows a photograph of the setup of the perceptual experiments. The experimental setup was the same as that shown in Fig. 2, except for the thermocouples and the temperature logger. Participants placed their hand, equipped with a cotton work glove, on the acrylic plate. We displayed three patterns: SP mode, AM mode (150 Hz), and no irradiation. Each pattern was displayed 10 times, and the total number of displays was 30 times. The order of displays was random. In each trial, one of the three patterns were presented for 5 s. After the presentation time, the participant chose one out of the following four options: “I only felt heat,” “I only felt vibration,” “I felt both heat and vibration,” “I did not feel anything,” as compared to what they felt at the beginning of the presentation time. The presentation time of 5 s was determined based on the result in Fig. 4, to ensure that the sensation of pain was not induced. Furthermore, the participants waited for 10 s after answering before the next trial was started. The participants were made to hear white noise during the experiment to ensure that the sound from the AUPAs did not affect the experimental results. The participants were 9 men and 1 woman, with ages ranging from 23 to 29 years. The average age of the participants was 25.4 years.
Fig. 5.

Photograph of perceptual experiment

Table 1 shows the average results obtained for all participants. These results demonstrate that SP mode caused heat sensations at a rate of 98%. Furthermore, AM mode caused vibrotactile sensations at a rate of 97%, although this was accompanied by heat sensations at a rate of 61%. This could be attributed to the deviation of the initial temperature in each trial owing to residual heat from the previous trial. Thus, the proposed method has the possibility of providing only a vibrotactile sensation by adequate AM irradiation time.
Table 1.

Average results for 10 participants. The values in bracket are standard errors in each result.

Irradiation patterns

Answer

Heat only

Vibration only

Heat & vibration

None

SP mode

98%

(1.3%)

0%

(0%)

1%

(0.9%)

1%

(0.9%)

AM mode

3%

(2.8%)

36%

(7.9%)

61%

(7.8%)

0%

(0%)

No irradiation

0%

(0%)

0%

(0%)

0%

(0%)

100%

(0%)

In conclusion, the proposed method generated both heat and vibrotactile sensations. Additionally, these sensations were selectively generated to a certain extent by using different ultrasonic radiation patterns.

4 Discussion

The perceptual experiment shows that AM mode could provide vibrotactile sensation; however, this mechanism is not obvious. This mechanism has three possibilities, as follows: The first is that the acoustic radiation pressure of ultrasound passing through the mesh of the fabric stimulates skin. The second is that the acoustic radiation pressure vibrates the fabric, which taps skin. The third is that an elastic wave in fabric excited by irradiation ultrasound propagates to the skin. The actual mechanism needs further investigation.

The limit range of displaying heat sensation from the AUPAs is decided by acoustic energy at the focal point. The acoustic energy transmitted from AUPAs is inversely proportional to the square of the distance from AUPAs, assuming that energy attenuation in the air is ignored. The limit range in the setup of this paper is approximately 300 mm assuming that the temperature of the skin surface reaches 40 ℃.

5 Conclusion

In this paper, we proposed a noncontact method for generating heat and vibrotactile sensations using focused airborne ultrasound. The proposed method provides heat sensations to a localized area by irradiating airborne ultrasound to the surface of a hand wearing a glove having sound absorption characteristics. This method also provides vibrotactile sensations by changing the ultrasonic irradiation pattern. It was confirmed that the proposed method locally provided both heat and vibrotactile sensations. In addition, our method generated these sensations selectively to a certain extent by varying the ultrasonic radiation pattern.

References

  1. 1.
    Iwamoto, T., Tatezono, M., Shinoda, H.: Non-contact method for producing tactile sensation using airborne ultrasound. In: Ferre, M. (ed.) EuroHaptics 2008. LNCS, vol. 5024, pp. 504–513. Springer, Heidelberg (2008).  https://doi.org/10.1007/978-3-540-69057-3_64CrossRefGoogle Scholar
  2. 2.
    Hoshi, T., Takahashi, M., Iwamoto, T., Shinoda, H.: Noncontact tactile display based on radiation pressure of airborne ultrasound. IEEE Trans. Haptics 3, 155–165 (2010)CrossRefGoogle Scholar
  3. 3.
    Carter, T., Seah, S.A., Long, B., Drinkwater, B., Subramanian, S.: UltraHaptics: multi-point mid-air haptic feedback for touch surfaces. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, pp. 505–514 (2013)Google Scholar
  4. 4.
    Korres, G., Eid, M.: Haptogram: ultrasonic point-cloud tactile stimulation. IEEE Access. 4, 7758–7769 (2016).  https://doi.org/10.1109/ACCESS.2016.2608835CrossRefGoogle Scholar
  5. 5.
    Meyer, R.A., Walker, R.E., Mountcastle, V.B.: A laser stimulator for the study of cutaneous thermal and pain sensations. IEEE Trans. Biomed. Eng. BME 23, 54–60 (1976).  https://doi.org/10.1109/TBME.1976.324616CrossRefGoogle Scholar
  6. 6.
    Saga, S.: HeatHapt thermal radiation-based haptic display. In: Kajimoto, H., Ando, H., Kyung, K.-U. (eds.) Haptic Interaction. LNEE, vol. 277, pp. 105–107. Springer, Tokyo (2015).  https://doi.org/10.1007/978-4-431-55690-9_19CrossRefGoogle Scholar
  7. 7.
    Lee, H., et al.: Mid-air tactile stimulation using laser-induced thermoelastic effects: the first study for indirect radiation. In: 2015 IEEE World Haptics Conference (WHC), pp. 374–380. IEEE (2015)Google Scholar
  8. 8.
    Nakajima, M., Hasegawa, K., Makino, Y., Shinoda, H.: Remotely displaying cooling sensation via ultrasound-driven air flow. In: 2018 IEEE Haptics Symposium (HAPTICS), pp. 340–343 (2018).  https://doi.org/10.1109/HAPTICS.2018.8357198
  9. 9.
    Hasegawa, K., Shinoda, H.: Aerial vibrotactile display based on multiunit ultrasound phased array. IEEE Trans. Haptics 11, 367–377 (2018)CrossRefGoogle Scholar
  10. 10.
    Jones, L.A., Ho, H.-N.: Warm or cool, large or small? The challenge of thermal displays. IEEE Trans. Haptics 1, 53–70 (2008).  https://doi.org/10.1109/TOH.2008.2CrossRefGoogle Scholar

Copyright information

© The Author(s) 2020

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  1. 1.The University of TokyoKashiwa-shiJapan

Personalised recommendations