Towards Enhancing Force-Input Interaction by Visual-Auditory Feedback as an Introduction of First Use

  • Akemi KobayashiEmail author
  • Ryosuke Aoki
  • Norimichi Kitagawa
  • Toshitaka Kimura
  • Youichi Takashima
  • Tomohiro Yamada
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9732)


Force-input interfaces are now attracting research interest and feedback schemes that yield high accurate operation are being investigated. Extrinsic feedback can help the user to operate a device with high accuracy and/or efficiency. However, for learning force control, the effectiveness of multimodal augmented feedback has not been investigated enough. This paper focuses on foam objects for learning force control and describes an experiment conducted to verify the effectiveness of augmented visual and auditory feedback. In the experiment, participants apply pushing force (using their thumb) to a foam cube under different augmented feedback conditions: (1) visual feedback, (2) auditory feedback and (3) visual-auditory feedback. Training and retention tests are conducted. The results show that just five minutes of practice similar to playing a musical instrument can enhance force reproduction skills and make subsequent force inputs more accurate.


Force-input Motor learning Visual-auditory feedback and anticipate force control 


  1. 1.
    Mizobuchi, S., Terasaki, S., Keski-Jaskari, T., Nousiainen, J., Ryynanen, M., Silfverberg, M.: Making an impression: Force-controlled pen input for handheld devices. In: CHI 2005 Extended Abstracts on Human Factors in Computing Systems. pp. 1661–1664. ACM (2005)Google Scholar
  2. 2.
    Kildal, J.: 3D-press: Haptic illusion of compliance when pressing on a rigid surface. In: International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction. ACM (2010)Google Scholar
  3. 3.
    Watanabe, C., Cassinelli, A., Watanabe, Y., Ishikawa, M.: Generic method for crafting deformable interfaces to physically augment smartphones. In: CHI 2014 Extended Abstracts on Human Factors in Computing Systems, pp. 1309–1314. ACM (2014)Google Scholar
  4. 4.
    Sugiura, Y., Kakehi, G., Withana, A., Lee, C., Sakamoto, D., Sugimoto, M., Igarashi, T.: Detecting shape deformation of soft objects using directional photoreflectivity measurement. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, pp. 509–516. ACM (2011)Google Scholar
  5. 5.
    Ronsse, R., Puttemans, V., Coxon, J.P., Goble, D.J., Wagemans, J., Wenderoth, N., Swinnen, S.P.: Motor learning with augmented feedback modality-dependent behavioral and neural consequences. Cerebr. Cortex 21(6), 1283–1294 (2011)CrossRefGoogle Scholar
  6. 6.
    Sigrist, R., Rauter, G., Riener, R., Wolf, P.: Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review. Psychon. Bull. Rev. 20(1), 21–53 (2013)CrossRefGoogle Scholar
  7. 7.
    Ramos, G., Boulos, M., Balakrishnan, R.: Pressure widgets. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 487–494. ACM (2004)Google Scholar
  8. 8.
    Lai, C.H., Niinimaki, M., Tahiroglu, K., Kildal, J., Ahmaniemi, T.: Perceived physicality in audio-enhanced force input. In: Proceedings of the 13th International Conference on Multimodal Interfaces, pp. 287–294. ACM (2011)Google Scholar
  9. 9.
    Kawato, M., Gomi, H.: A computational model of four regions of the cerebellum based on feedback-error learning. Biol. Cybern. 68(2), 95–103 (1992)CrossRefGoogle Scholar
  10. 10.
    Schmidt, R.A., Wrisberg, C.A.: Motor learning and performance: A Situation-Based Learning Approach with Web Study Guide, 4th edn. Human Kinetics Publishers (2007)Google Scholar
  11. 11.
    Schmidt, R.A., Wulf, G.: Continuous concurrent feedback degrades skill learning: Implications for training and simulation. Hum. Factors 39(4), 509–525 (1997)CrossRefGoogle Scholar
  12. 12.
    Doyo, D.: A basic study on the visual feedback in the pushing-power control training (in Japanese). Jpn J. Ergon. 42(4), 227–233 (2006)CrossRefGoogle Scholar
  13. 13.
    Torrey, M.J.L., Ofori, E., Sosnoff, J.J.: Force control under auditory feedback: Effector differences and audiomotor memory. Percept. Mot. Skills 114(3), 915–935 (2012)CrossRefGoogle Scholar
  14. 14.
    Potter, N.L., Kent, R.D., Lindstrom, M.J., Lazarus, J.A.C.: Power and precision grip force control in three-to-five-year-old children: velocity control precedes amplitude control in development. Exp. Brain Res. 172(2), 246–260 (2006)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Akemi Kobayashi
    • 1
    Email author
  • Ryosuke Aoki
    • 1
  • Norimichi Kitagawa
    • 2
  • Toshitaka Kimura
    • 2
  • Youichi Takashima
    • 1
  • Tomohiro Yamada
    • 1
  1. 1.NTT Service Evolution LaboratoriesNippon Telegraph and Telephone CorporationYokosukaJapan
  2. 2.NTT Communication Science LaboratoriesNippon Telegraph and Telephone CorporationAtsugiJapan

Personalised recommendations