A Comparative Longitudinal Study of Non-verbal Mouse Pointer

  • Murni Mahmud
  • Adam J. Sporka
  • Sri H. Kurniawan
  • Pavel Slavík
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4663)


A longitudinal study of two non-speech continuous cursor control systems is presented in this paper: Whistling User Interface (U3I) and Vocal Joystick (VJ). This study combines the quantitative and qualitative methods to get a better understanding of novice users’ experience over time. Three hypotheses were tested in this study. The quantitative data show that U3I performed better in error rate and in simulating a mouse click; VJ was better on other measures. The qualitative data indicate that the participants’ opinions regarding both tools improved day-by-day. U3I was perceived as less fatiguing than VJ. U3I approached the performance of VJ at the end of the study period, indicating that these two systems can achieve similar performances as users get more experienced in using them. This study supports two hypotheses but does not provide enough evidence to support one hypothesis.


Voice-based interface non-verbal vocal input speech recognition cursor control continuous input mouse cursor acoustic gestures 


  1. 1.
    Al-Hashimi, S.: Blowtter: A voice-controlled plotter. In: Proc HCI 2006 Engage, The 20th BCS HCI Group conference in co-operation with ACM, London, England, vol. 2 (2006)Google Scholar
  2. 2.
    Allen, J.: The online reaction time test. On-line (retrieved August 2006) (2002),
  3. 3.
    Barra, M., Cillo, T., Santis, A.D., Petrillo, U.F., Negro, A., Scarano, V., Matlock, T., Maglio, P.P.: Personal webmelody: Customized sonification of web servers. In: Proc 2001 International Conference on Auditory Display (August 2001)Google Scholar
  4. 4.
    Bilmes, J.A., Li, X., Malkin, J., Kilanski, K., Wright, R., Kirchhoff, K., Subramanya, A., Harada, S., Landay, J.A., Dowden, P., Chizeck, H.: The Vocal Joystick: A voice-based human-computer interface for individuals with motor impairments. In: Proc Human Lang Tech Conf./Conf. on Empirical Methods in Natural Language Processing (October 2005)Google Scholar
  5. 5.
    Dai, L., Goldman, R., Sears, A., Lozier, J.: Speech-based cursor control: a study of grid-based solutions. SIGACCESS Access. Comput. 77(78), 94–101 (2004)CrossRefGoogle Scholar
  6. 6.
    Fitts, P.M.: The information capacity of the human motor system in controlling the amplitude of movement. J. of Experimental Psychology 47(6), 381–391 (1954)CrossRefGoogle Scholar
  7. 7.
    Franklin, K., Roberts, J.C.: Pie Chart Sonification. In: Proceedings Information Visualization, pp. 4–9. IEEE Computer Society, Los Alamitos (2003)Google Scholar
  8. 8.
    de Mauro, C., Gori, M., Maggini, M., Martinelli, E.: Easy access to graphical interfaces by voice mouse. Technical report, Universit? di Siena (2001) Google Scholar
  9. 9.
    Hämäläinen, P., Mäki-Patola, T., Pulkki, V., Airas, M.: Musical computer games played by singing. In: Proc of 7th Intl Conf. on Digital Audio Effects, Naples, pp. 367–371 (2004)Google Scholar
  10. 10.
    Harada, S., Landay, J.A., Malkin, J., Li, X., Bilmes, J.A.: The vocal joystick: evaluation of voice-based cursor control techniques. In: Proc ASSETS 2006, Portland, Oregon (2006)Google Scholar
  11. 11.
    Igarashi, T., Hughes, J.F.: Voice as sound: using non-verbal voice input for interactive control. In: UIST 2001. Proc 14th Annual ACM Symp on User Interface Software and Technology, New York, NY, USA, pp. 155–156. ACM Press, New York (2001)Google Scholar
  12. 12.
    Kamel, H.M., Landay, J.A.: Sketching images eyes-free: a grid-based dynamic drawing tool for the blind. In: Proc ASSETS 2002, New York, pp. 33–40 (2002)Google Scholar
  13. 13.
    Kjeldskov, J., Skov, M.B., Stage, J.: Does time heal?: a longitudinal study of usability. In: OZCHI ’05: Proceedings of the 19th conference of the computer-human interaction special interest group (CHISIG), pp. 1–10, Australia (2005)Google Scholar
  14. 14.
    MacKenzie, I.S., Kauppinen, T., Silfverberg, M.: Accuracy measures for evaluating computer pointing devices. In: Proc CHI 2001, pp. 9–16. ACM Press, New York (2001)Google Scholar
  15. 15.
    Manaris, B., McCauley, R., MacGyvers, V.: An intelligent interface for keyboard and mouse control - providing full access to PC functionality via speech. In: Proc. 14th Intl. Florida Artificial Intelligence Research Society Conference, pp. 182–188. AAAI Press, Stanford, California (2001)Google Scholar
  16. 16.
    Mihara, Y., Shibayama, E., Takahashi, S.: The migratory cursor: accurate speech-based cursor movement by moving multiple ghost cursors using non-verbal vocalizations. In: Proc. ASSETS 2005, pp. 76–83. ACM Press, New York (2005)Google Scholar
  17. 17.
    Nakano, T., Goto, M., Ogata, J., Hiraga, Y.: Voice drummer: A music notation interface of drum sounds using voice percussion input. In: Proc. of UIST 2005, pp. 49–50. ACM Press, New York (2005)Google Scholar
  18. 18.
    Sporka, A.J., Kurniawan, S.H., Slavík, P.: Accoustic control of mouse pointer. Universal Access in the Information Society 4(3), 237–245 (2006)CrossRefGoogle Scholar
  19. 19.
    Sporka, A.J., Kurniawan, S.H., Slavík, P.: Non-speech operated emulation of keyboard. In: Clarkson, J., Langdon, P., Robinson, P. (eds.) Cambridge Workshop on Universal Access and Assistive Technology, CWUAAT 2006. Designing Accessible Technology, pp. 145–154. Springer, Heidelberg (2006)Google Scholar
  20. 20.
    Sporka, A.J., Kurniawan, S.H., Mahmud, M., Slavík, P.: Tonal control of mouse cursor (U 3 I): A usability study with the elderly. In: Proc. HCI International 2005, Lawrence Erlbaum Associates, Mahwah (2005)Google Scholar
  21. 21.
    Sporka, A.J., Kurniawan, S.H., Mahmud, M., Slavik, P.: Non-speech input and speech recognition for real-time control of computer games. In: The Proceedings of The Eighth International ACM SIGACCESS Conference on Computers & Accessibility, ASSETS 2006, Portland, Oregon, ACM, New York (2006)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2007

Authors and Affiliations

  • Murni Mahmud
    • 1
  • Adam J. Sporka
    • 2
    • 3
  • Sri H. Kurniawan
    • 4
  • Pavel Slavík
    • 3
  1. 1.International Islamic University Malaysia, P.O. Box 10, 50728 Kuala LumpurMalaysia
  2. 2.Hečkova 8, 14200 Praha 4Czech Republic
  3. 3.Czech Technical University in Prague, Karlovo náměstí 13, 12135 Praha 2Czech Republic
  4. 4.University of Manchester, PO Box 88, Manchester M60 1QDUnited Kingdom

Personalised recommendations