One-hand usage of handheld devices results in poor reachability, re-gripping, occlusion and reduced accuracy. This results in poor user experiences, especially with user interactions that require multi-finger usage. Use of deformation gestures can augment touch gestures to extend the limited functionalities experienced in one-handed usage. We present BendSwipe - a combined bend and touch enabled input interaction to perform target zooming on a flexible handheld device. It uses the bend gestures to augment a swipe interaction performed anywhere at the back of the device to perform target zoom-in and out on any area of the display content. This paper presents the details of proposed input interactions, prototype and future directions to evaluate the designs.
- Deformation based interactions
- One-Handed usage
- Bend & touch interactions
Prior research findings show 74% users prefer to use mobile phones with one hand . One handed usage of handheld devices is commonly observed in situational impairments, e.g. walking, sitting and standing where the other hand is busy doing real- world tasks . Hence, the design of suitable input interactions for one-handed usage becomes critical in common scenarios of situational impairment.
The standard handheld devices pose several challenges during one-handed usage including limited reachability , re-gripping of the device , reduced accuracy , and increased occlusion . These problems are primarily attributed to factors like the constantly increasing size of the device, hand-thumb morphology and interactions that demand the attention of both hands to perform certain tasks . This leads to difficult user interactions with handheld devices, especially for the tasks that demand multi-finger and/or multi-step interactions such as target zooming [3, 5]. Target zooming is defined as zoom-in and zoom-out to a specific location of an interface (e.g. top-right & top-left corner, bottom-left & bottom-right corner etc.). These locations often demand multi-step interactions by enabling (a) access to unreachable areas and (b) further panning through diagonal, horizontal and vertical finger movements to achieve a user-defined target in an interface. These tasks are commonly performed in applications such as navigational maps and image gallery.
The problems with one-handed usage are not limited to standard handheld devices, but will also be experienced in future flexible handheld devices . Although extensive research is conducted in improving one-handed interactions on standard handheld devices [1, 3, 9, 10], investigations targeting one-handed interactions for flexible handheld devices is insufficiently explored. These investigations are important as flexible devices offer opportunities of deformations that may result in intuitive and efficient one-handed user interactions. The potential of deformation is further amplified by bringing in touch capabilities as it is useful to offset against the inability to perform bimanual interactions in one-handed usage .
In this paper, we propose new input interactions – a combination of central bend based deformation gestures and touch supported swipe on back side of flexible handheld device to perform target zooming of an image.
2 Design of Proposed Input-Interactions
To illustrate the use case of target zooming using the BendSwipe interaction technique, we use an infographic image and divide it into a 3 × 3 grid (Fig. 1). BendSwipe uses bend and swipe gesture to manipulate the direction of zooming - directing towards specific location of 3 × 3 grid. The gesture involves central bending the flexible device in order to perform a center squeeze and performing a swipe action at the back of the device simultaneously.
2.1 Design Rationale
To reduce occlusion and re-gripping events and enable easy target zooming, we chose a combination of bend gesture and touch supported swipe gesture on backside of flexible handheld device. We chose bend gestures due to its strong preference in current literature [1, 11, 13] which are less time consuming, easy to use, and intuitive; hence aiding in performing zooming tasks easily and quickly. We also chose touch based swipe gesture as common holding position in one-handed usage freed the index finger on the backside of the device . Swiping the index finger in specific direction aides in identifying the zooming direction; hence enabling target zooming. We believe that combination of bend and swipe gesture on backside of flexible handhelds can effectively reduce the current challenges of occlusion, reachability and re-gripping and enable target zooming easily and quickly.
2.2 BendSwipe Walkthrough
Zoom-in and out is triggered by performing a bend and swipe gesture simultaneously. Performing a center squeeze which bends the central vertical axis (X-X’ in Fig. 2) towards the user is called bend-in. A bend-in displays the navigation wheel as displayed in Fig. 2(a). Performing a linear swipe gesture in any direction on the back of the device triggers the zoom-in which triggers in the exact direction of the swipe gesture; hence performing a target zoom-in. The zoom-in is performed as long as a touch is detected at the end coordinate of the swipe gesture. For example, to zoom-in towards the area in grid 8 (Fig. 1) the user needs to perform a bend-in, followed by a swipe gesture in the South-East (SE) direction anywhere on the back of the device.
Performing a center squeeze which bends the central vertical axis (X-X’ in Fig. 2) away from the user is called bend-out. A bend-out displays the navigation wheel as displayed in Fig. 2 (b). Performing a linear swipe gesture in any direction on the back of the device triggers the zoom-out which triggers in the exact direction of the swipe gesture; hence performing a target zoom-out. Similar to zoom-in, zoom-out is performed as long as a touch is detected at the end coordinate of the swipe gesture. For example, to zoom-out towards the area in grid 8 (Fig. 1) the user needs to perform a bend-out, followed by a swipe gesture in the SE direction anywhere on the back of the device.
The flexible prototype is made out of a thin sheet of paper (5.5 size) laminated with plastic sheets on both sides. The plastic coating gives the device the ability to bend and twist easily along with elasticity. A bend sensor is located diagonally (on the backside) to detect bend-in and bend-out of the flexible device (Fig. 3). To detect and track the finger swipe gesture at the back of the device, a deformable and conductive transparent sensor material  is used that is placed on the backside of flexible device. The movement as well as the location of a finger can be followed across the cross-grid sensor array that results in the tracking of finger swipe direction. The bend sensor and the conductive sensor material sends the data to the Arduino microcontroller to control the image size using Processing software. Moreover, the display content is integrated with the device by means of a top projection.
4 Conclusion and Future Work
In this paper, we explored the capabilities of deformation gestures to augment touch gestures in order to extend their functionality for one-handed hand-held usage. We designed a method of target zooming using the central bend gesture and the swipe gesture at the back of the device to overcome the challenges experienced in one-handed usage. These challenges are yet to be explored sufficiently for flexible handheld devices. We presented an input method to zoom-in or out towards any area in a continuous and controlled manner. To evaluate the effectiveness of this interaction technique, we will conduct a comparative evaluation with the standard smartphone. The comparative study will investigate variables (a) task completion time to zoom to several specific target areas and (b) number of re-gripping events while performing the target zooming during the task. We will also measure 4 usability constructs (i) ease of use, (ii) intuitiveness, (iii) ease of learning and (iv) behavioral intention. In addition to the effectiveness of the proposed input interactions, the study can potentially bring some insights on the effectiveness of swipe as a back interaction used along with deformation gesture in one-handed scenario.
Holman, D., Hollatz, A., Banerjee, A., Vertegaal, R.: Unifone: designing for auxiliary finger input in one-handed mobile interactions. In: Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (TEI 2013), pp. 177–184. ACM, New York (2013). doi:10.1145/2460625.2460653
Pascoe, J., Ryan, N., Morse, D.: Using while moving: HCI issues in fieldwork environments. ACM Trans. Comput.-Hum. Interact. 7(3), 417–437 (2000). doi:10.1145/355324.355329
Lai, J., Zhang, D.: ExtendedThumb: a target acquisition approach for one-handed interaction with touch-screen mobile phones. Proc. IEEE Trans. Hum.-Mach. Syst.
Ti, J., Tjondronegoro, D.: TiltZoom: tilt-based zooming control for easy (2012)
Karlson, A.K., Bederson, B.B., Contreras-Vidal, J.L.: Studies in one-handed mobile design: Habit, desire and agility. In: Proceedings of the 4th ERCIM Workshop User Interfaces All (UI4ALL) (2006)
Karlson, A.K., Bederson, B.B., Contreras-Vidal, J.L.: Understanding one-handed use of mobile devices. In: Proceedings of the Handbook of Research on User Interface Design and Evaluation for Mobile Technology (2008)
Karlson, A.K., Bederson, B.B.: ThumbSpace: generalized one-handed input for touchscreen-based mobile devices. In: Baranauskas, C., Palanque, P., Abascal, J., Barbosa, S.D.J. (eds.) INTERACT 2007. LNCS, vol. 4662, pp. 324–338. Springer, Heidelberg (2007). doi:10.1007/978-3-540-74796-3_30
Sarwar, M.S., et al.: Bend, stretch, and touch: locating a finger on an actively deformed transparent sensor array. Sci. Adv. 3(3), e1602200 (2017)
Yu, N.-H., Huang, D.-Y., Hsu, J.-J., Hung, Y.-P.: Rapid selection of hard-to-access targets by thumb on mobile touch-screens. In: Proceedings of the 15th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI 2013), pp. 400–403. ACM, New York (2013). doi:10.1145/2493190.2493202
One-handed mobile interactions. In: Proceedings of the Internet of Things Workshop, OZCHI 2012: Integration, Interaction, Innovation, Immersion, Inclusion, Melbourne, Victoria, Australia, pp. 1–5 (2012)
Ahmaniemi, T.T., Kildal, J., Haveri, M.: What is a device bend gesture really good for? In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2014), pp. 3503–3512. ACM, New York (2014). doi:10.1145/2556288.2557306
Wobbrock, J.O., Myers, B.A., Aung, H.H.: The performance of hand postures in front-and back-of-device interaction for mobile computing. Int. J. Human-Comput. Stud. 66(12), 857–875 (2008)
Girouard, A., Lo, J., Riyadh, M., Daliri, F., Eady, A.K., Pasquero, J.: One-handed bend interactions with deformable smartphones. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 1509–1518. ACM, April 2015
Riyadh, Md.: Exploring tapping with thumb input for flexible tablets. In: CHI 2014 Extended Abstracts on Human Factors in Computing Systems. ACM (2014)
Editors and Affiliations
© 2017 IFIP International Federation for Information Processing
About this paper
Cite this paper
Sorathia, K., Singh, A., Chhabra, M. (2017). BendSwipe: One Handed Target Zooming for Flexible Handheld Display. In: Bernhaupt, R., Dalvi, G., Joshi, A., K. Balkrishan, D., O’Neill, J., Winckler, M. (eds) Human-Computer Interaction – INTERACT 2017. INTERACT 2017. Lecture Notes in Computer Science(), vol 10516. Springer, Cham. https://doi.org/10.1007/978-3-319-68059-0_45
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68058-3
Online ISBN: 978-3-319-68059-0