Designing of 2D Illusory Tactile Feedback for Hand-Held Tablets
In this paper, we investigate whether the “out of body” tactile illusion can be extended or applied to a relatively large hand-held device such as a tablet for which the hands/fingers would not be in direct contact with the vibration motors. We derived guidelines for applying tactile illusion techniques in 2D space with regards to operational conditions such as the size of the device, holding position, minimally required vibration amplitudes, and the effects of matching visual feedback. For this purpose, a series of exploratory pilot experiments were first conducted in 1D space. Based on the results, a 2D illusory tactile rendering method was devised and tested for its effectiveness. We have found that for a tablet sized device (e.g. iPadmini and iPad), the illusory perception was possible with a rectilinear grid resolution of 5 × 7 (with a grid size of 2.5 cm) with matching visual feedback.
KeywordsFunneling Illusory feedback Vibro-tactile feedback User experience Mobile/Hand-held interaction
Smart hand-held devices nowadays come in many different sizes: from smart phones (10–15 cm in diagonal length) to larger tablet devices and hand-held computers (18–25 cm). There have been increasing interests in improving the user experience on hand-held devices by providing richer and more dynamic feedback through vibro-tactile actuators [6, 11, 12]. To avoid resorting to vibro-tactile arrays which would require a sizable and continuous contact area with the hand or body parts, researchers have proposed to apply tactile illusion techniques so that similar effects can be realized with only a few number of vibration motors [6, 11]. In addition, Miyazaki has demonstrated that tactile illusory feedback could be elicited at the “out of the body” space . Kim has applied the “out of the body” tactile illusory feedback for a small-sized smartphone (9.4 cm in diagonal length), and demonstrated the illusory feedback was indeed applicable and extendable for interaction in 2D screen space using only four vibrators .
Thus, to find out how to apply the “out of body” tactile illusion to tablet devices, we ran a series of pilot exploratory experiments to first observe the effects of the tactile illusion e.g. with regards to various operational conditions such as the size of the device, holding position, minimal vibration amplitudes and effects of accompanying visual feedback for a fair perception of the illusion. The findings and observations would form the basis for designing a proper 2D tactile illusory rendering method for the tablet device. We shortly describe these experiments, report the newly derived guidelines, propose for a 2D tactile illusory rendering method for a tablet, and empirically validate its effectiveness.
2 Related Work
Funneling is one of the major perceptual illusion techniques specifically for vibro-tactile feedback (along with saltation which we do not consider in this work for now). It refers to stimulating the skin at two different locations simultaneously with different amplitudes and eliciting phantom sensations in between the two (i.e. 1D phenomenon) [1, 3]. The intended location of the phantom sensation (ITL) can be changed by modulating the stimulation intensity.
Several researchers have applied this phenomenon to human interfaces, experimenting with different ways of modulating the vibration amplitudes for detailed controlling of the target phantom sensation locations [1, 2, 5, 9, 10]. For example, the SemFeel was a vibro-tactile feedback system for a mobile touch screen device  using five vibrators attached to the backside of the device (4 corners and 1 center). It was capable of producing 10 different perceptible vibration patterns for expressing richer semantic information compared to the usual single vibrator scheme.
Recently, researchers have discovered such phantom sensations can be extended to the “out of body”  (see Fig. 1(a)), thus making it possible to generate phantom tactile sensations as if coming from an external object. This result is appealing in its application possibility to mobile devices. That is, there now is a potential method to make a user feel (phantom) sensations indirectly emanating from the hand-held device (e.g. middle of the display), but supplying actual vibrations only to the “natural” holding finger/hand locations (Fig. 1). Kim demonstrated that the aforementioned phenomenon could be extended to 2D by employing vibrators both horizontally and vertically on a small sized (3.5 inch screen) smart phone . But no general guidelines have been reported for further applications to a variety of today’s hand-held media devices.
Finally, many human computer interfaces also rely on cross modal illusions to enrich user experience by fusing feedbacks of different modalities. For instance, a mere single vibrator based tactile feedback (as adopted by most current smart phones), when combined with other modalities, can induce phantom directional haptic feedback and rich user experience . We posit that the tactile illusion feedback, when combined with other usual modalities such as the visual and aural, can similarly improve the interaction experience even further.
Thus, to summarize, the distinguishing point of our work is that we seek to newly extend funneling to an enlarged 2D space, and identify important design guidelines. This would open doors for a more flexible design of vibro-tactile feedback techniques for a wide variety of hand-held devices.
3 Pilot Experiments: “Out of Body” Funneling Effects Along Longer 1D Separation Span
Due to lack of space, we only provide a very short description of how the exploratory pilot experiments were run. Funneling stimulations were given through vibrators attached to an iPadmini and iPad, along three horizontal (along the top/middle/bottom row) or vertical directions (along the left/middle/right column) using two vibrators attached at the respective ends (i.e. six vibrators used for each case). Sixteen participants were asked to report the existence of any illusory tactile perception and indicate where they came from, if any (along the respective row or column). Various rendering parameters were changed and results observed.
3.1 Perception Resolution and Extent
Our data showed the smallest perceptible linear interval size to be comparable to that of the smart phone case (~1.6 cm) , “regardless of the size of the device (or distance between the vibration motors),” but only so in the middle part (40–60 % of the bands). The perception resolution was notably lower (or discernable grid size “stretched”, e.g. to as long as 7 cm depending on where the device was held) near the horizontal or vertical ends (see Fig. 2). This was more evident in the vertical direction, because the hands were holding the device along the horizontal direction.
3.2 Tactile Rendering Function and Interpolated Stimulation Strength
In the pilot experiment described in Sect. 3.1, we initially tested the three most popular and arguably reliable (in terms of eliciting the phantom sensation from the intended location) stimulation rendering methods for funneling, namely, interpolating the two vibration amplitudes using linear, logarithmic, and tangent functions . Among the three, the tangent based interpolation showed the widest perception band (e.g. 6 intervals vertically and 9 horizontally on an iPad as shown in Fig. 2). However, in the later main experiment, this finding did not carry over to the 2D case. The responses were inaccurate and irregular. This was due to the fact that when tangent-based interpolation was applied in 2D, the interpolated amplitudes at the middle region became relatively too low (compare the two interpolating functions in Fig. 3). Thus, the logarithmic interpolation was chosen to render the tactile vibration amplitude with slightly less effective perception resolution (5 × 7) but with higher accuracy and response regularity.
3.3 Hand Location
4 Main Experiment: Extending “Out of Body” Funneling to Tablets
4.1 Experimental Design and Set up
In the main experiment, we used four single vibrators attached in the four rear corners of the iPad (see Fig. 1). We used the same rendering algorithm as in Kim’s study for the smart phone  but with the few parameters adjusted according to our pilot experiment findings described in the previous section. The main purpose of the experiment was (1) to assess the illusory tactile perception behavior and effectiveness of the newly adjusted rendering algorithm with the larger tablet device and (2) how combining the illusory tactile feedback with the visual would further improve the accuracy and user experience. Two cases of 2D illusory tactile renderings were compared: without any visual feedback (NV), and with visual feedback (VF).
For NV, users were given 2D illusory tactile stimulations on an iPad held in the middle part (e.g. as shown in Fig. 1(d)) at the resolution of 5 × 7 (conservatively chosen based on the results of the pilot experiment). Participants were asked to pinpoint (using a stylus pen) where they felt the tactile feedback to be coming from.
For VF, when the stimulation was given, 5 visual icons (matching feedback), only one of them corresponding to the true intended target location (ITL), were shown and the user had to choose the correct one.
Vibrators were controlled by a voltage input using a PWM signal with an amplitude between 0 to 5 V, which in turn produced vibrations with frequency between 0 and 250 Hz and associated amplitudes between 0 to 2G (measured in acceleration, or 0 to 18 μm in position) respectively.
4.2 Detailed Procedure
5.1 NV Experiment
ANOVA was applied for analyzing the perception resolution illustrated in the horizontal and vertical directions separately for simplicity. Figure 5 shows the cases for the first row and column where 5 and 3 distinct intervals were perceived respectively. Other rows and columns showed similar statistics. This lead us to project that the 2D perception resolution to be about only 3 × 5, lower than the expected (Fig. 2). The graphs in Fig. 5 show a similar pattern of lowered resolution at the periphery of the screen, i.e. elongated/enlarged grid size (e.g. ITLs at 1, 2 and 6, 7 in Fig. 5(a) and 1, 2 and 4, 5 in 5(b)). Also note that the user response was not linear as desired, an artifact of resorting back to using the logarithmic interpolation and a possible interaction among the stimulations in 2D. Thus, we believe that better and linear responses can be obtained by employing vibration motors with larger amplitude capabilities and a “mixed” interpolation technique (e.g. logarithmic + tangent-based). Although the main purpose of NV experiment was observing user behavior for 2D response in 2D tactile feedback, correct responses for each ITL grid were recorded (Fig. 6).
5.2 VF Experiment
6 Conclusion and Future Work
In this paper, we carried out pilot experiments to explore how to extend 2D illusory tactile rendering to tablet devices. Tablet devices exceed in length for which “out of body” tactile illusion is known to work, and the stimulations become more indirect and their interactions in 2D more complex. Based on the experimental findings, we have devised a 2D illusory tactile rendering algorithm and have found that for an iPad sized device, approximately 5 × 7 regular resolution (grid size = ~2.5 cm) of illusory perception was achievable, when a matching visual feedback was provided.
For future work, we will apply our findings to real world examples such as interactive videos and mobile games, and assess whether the UX can be improved by the proposed vibro-tactile rendering method. At the same time, we plan to investigate additional ways to improve the 2D sensation localization accuracy by adjusting various rendering parameters, combining with the aural modality and employing other vibro-tactile illusion techniques such as the saltation and apparent motion.
This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (No. 2011-0030079) and funded by the Forensic Research Program of the National Forensic Service (NFS), Ministry of Government Administration and Home Affairs, Korea (NFS-2015-DIGITAL-04).
- 2.Barghout, A., Kammerl, J., Cha, J., Steinbach, E., El Saddik, A.: Spatial resolution of vibrotactile perception on the human forearm when exploiting funneling illusion. In: Proceeding of IEEE International Workshop on HAVE, pp. 19–23 (2009)Google Scholar
- 4.Geshelder, G.: Psychophysics: The Fundamentals. Lawrence Erlbaum Associates, Mahwah (1997)Google Scholar
- 5.Israr, A., Poupyrev, I.: Control space of apparent haptic motion. In: IEEE World Haptics Conference (2011)Google Scholar
- 6.Kim, Y., Lee, J., Kim, G. J.: Extending “out of the body” saltation to 2D mobile tactile interaction. In: Proceedings of APCHI, pp. 67–74 (2012)Google Scholar
- 7.Kingdom, F., Prins, N.: Psychophysics: A Practical Introduction. Elsevier-Academic Press, London (2010)Google Scholar
- 10.Seo, J., Choi, S.: Initial study for creating linearly moving vibrotactile sensation on mobile device. In: Proceedings of IEEE Haptics Symposium, pp. 67–70 (2010)Google Scholar
- 11.Yatani, K., Truong, K.: SemFeel: a user interface with semantic tactile feedback for mobile touch-screen devices. In: Proceedings of UIST, pp. 111–120 (2009)Google Scholar
- 12.Nokia, Bouncing ball on Nokia N900 (2013). http://bit.ly/x2-bouncingball