Skip to main content
SpringerLink
Log in

Rotate-and-Press: A Non-visual Alternative to Point-and-Click?

  • Conference paper
  • First Online:
HCI International 2020 – Late Breaking Papers: Universal Access and Inclusive Design (HCII 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12426))

Included in the following conference series:

Abstract

Most computer applications manifest visually rich and dense graphical user interfaces (GUIs) that are primarily tailored for an easy-and-efficient sighted interaction using a combination of two default input modalities, namely the keyboard and the mouse/touchpad. However, blind screen-reader users predominantly rely only on keyboard, and therefore struggle to interact with these applications, since it is both arduous and tedious to perform the visual ‘point-and-click’ tasks such as accessing the various application commands/features using just keyboard shortcuts supported by screen readers.

In this paper, we investigate the suitability of a ‘rotate-and-press’ input modality as an effective non-visual substitute for the visual mouse to easily interact with computer applications, with specific focus on word processing applications serving as the representative case study. In this regard, we designed and developed bTunes, an add-on for Microsoft Word that customizes an off-the-shelf Dial input device such that it serves as a surrogate mouse for blind screen-reader users to quickly access various application commands and features using a set of simple rotate and press gestures supported by the Dial. Therefore, with bTunes, blind users too can now enjoy the benefits of two input modalities, as their sighted counterparts. A user study with 15 blind participants revealed that bTunes significantly reduced both the time and number of user actions for doing representative tasks in a word processing application, by as much as \(65.1\%\) and \(36.09\%\) respectively. The participants also stated that they did not face any issues switching between keyboard and Dial, and furthermore gave a high usability rating (84.66 avg. SUS score) for bTunes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://docs.microsoft.com/en-us/visualstudio/vsto/office-primary-interop-assemblies?view=vs-2019.

  2. 2.

    https://docs.microsoft.com/en-us/visualstudio/vsto/office-solutions-development-overview-vsto?view=vs-2017.

References

  1. Abu Doush, I., Pontelli, E., Simon, D., Son, T.C., Ma, O.: Making microsoft excel™: multimodal presentation of charts. In: Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility, Assets 2009, New York, NY, USA, pp. 147–154. Association for Computing Machinery (2009). https://doi.org/10.1145/1639642.1639669

  2. Apple Inc.: Vision accessibility - mac - apple (2020). https://www.apple.com/accessibility/mac/vision/

  3. Asakawa, C., Itoh, T.: User interface of a home page reader. In: Proceedings of the Third International ACM Conference on Assistive Technologies, Assets 1998, New York, NY, USA, pp. 149–156. ACM (1998). https://doi.org/10.1145/274497.274526

  4. Ashok, V., Borodin, Y., Puzis, Y., Ramakrishnan, I.V.: Capti-speak: a speech-enabled web screen reader. In: Proceedings of the 12th Web for All Conference, W4A 2015, New York, NY, USA. Association for Computing Machinery (2015). https://doi.org/10.1145/2745555.2746660

  5. Ashok, V., Puzis, Y., Borodin, Y., Ramakrishnan, I.: Web screen reading automation assistance using semantic abstraction. In: Proceedings of the 22nd International Conference on Intelligent User Interfaces, IUI 2017, New York, NY, USA, pp. 407–418. Association for Computing Machinery (2017). https://doi.org/10.1145/3025171.3025229

  6. Billah, S.M., Ashok, V., Porter, D.E., Ramakrishnan, I.: Speed-dial: a surrogate mouse for non-visual web browsing. In: Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2017, New York, NY, USA, pp. 110–119. Association for Computing Machinery (2017). https://doi.org/10.1145/3132525.3132531

  7. Brooke, J., et al.: SUS-a quick and dirty usability scale. Usab. Eval. Ind. 189(194), 4–7 (1996)

    Google Scholar 

  8. C. Bell, E., M. Mino, N.: Employment outcomes for blind and visually impaired adults. J. Blind. Innov. Res. 5 (2015). https://doi.org/10.5241/5-85

  9. Evans, D.G., Diggle, T., Kurniawan, S.H., Blenkhorn, P.: An investigation into formatting and layout errors produced by blind word-processor users and an evaluation of prototype error prevention and correction techniques. IEEE Trans. Neural Syst. Rehabil. Eng. 11(3), 257–268 (2003). https://doi.org/10.1109/TNSRE.2003.816868

    Article  Google Scholar 

  10. Freedom Scientific: Jaws ® - freedom scientific (2020). http://www.freedomscientific.com/products/software/jaws/

  11. Gadde, P., Bolchini, D.: From screen reading to aural glancing: towards instant access to key page sections. In: Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility, ASSETS 2014, New York, NY, USA, pp. 67–74. Association for Computing Machinery (2014). https://doi.org/10.1145/2661334.2661363

  12. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Advances in Psychology, vol. 52, pp. 139–183. Elsevier (1988)

    Google Scholar 

  13. Haverty, R.: New accessibility model for microsoft windows and cross platform development. SIGACCESS Access. Comput. 82, 11–17 (2005). https://doi.org/10.1145/1077238.1077240

    Article  Google Scholar 

  14. Jaijongrak, V.r., Kumazawa, I., Thiemjarus, S.: A haptic and auditory assistive user interface: helping the blinds on their computer operations. In: 2011 IEEE International Conference on Rehabilitation Robotics, pp. 1–6. IEEE (2011)

    Google Scholar 

  15. Khurana, R., McIsaac, D., Lockerman, E., Mankoff, J.: Nonvisual interaction techniques at the keyboard surface. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018, New York, NY, USA. Association for Computing Machinery (2018). https://doi.org/10.1145/3173574.3173585

  16. Melnyk, V., et al.: Look ma, no aria: generic accessible interfaces for web widgets. In: Proceedings of the 12th Web for All Conference, W4A 2015, New York, NY, USA. Association for Computing Machinery (2015). https://doi.org/10.1145/2745555.2746666

  17. Morales, L., Arteaga, S.M., Kurniawan, S.: Design guidelines of a tool to help blind authors independently format their word documents. In: CHI 2013 Extended Abstracts on Human Factors in Computing Systems, CHI EA 2013, New York, NY, USA, pp. 31–36. ACM (2013). https://doi.org/10.1145/2468356.2468363

  18. NV Access: Nv access (2020). https://www.nvaccess.org/

  19. Owen, J.M., Petro, J.A., D’Souza, S.M., Rastogi, R., Pawluk, D.T.: An improved, low-cost tactile “mouse” for use by individuals who are blind and visually impaired. In: Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility, Assets 2009, New York, NY, USA, pp. 223–224. Association for Computing Machinery (2009). https://doi.org/10.1145/1639642.1639686

  20. Rastogi, R., Pawluk, D.T., Ketchum, J.M.: Issues of using tactile mice by individuals who are blind and visually impaired. IEEE Trans. Neural Syst. Rehabil. Eng. 18(3), 311–318 (2010)

    Article  Google Scholar 

  21. Soviak, A., Borodin, A., Ashok, V., Borodin, Y., Puzis, Y., Ramakrishnan, I.: Tactile accessibility: does anyone need a haptic glove? In: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2016, New York, NY, USA, pp. 101–109. Association for Computing Machinery (2016). https://doi.org/10.1145/2982142.2982175

  22. Wang, K., Barron, L.G., Hebl, M.R.: Making those who cannot see look best: effects of visual resume formatting on ratings of job applicants with blindness. Rehabil. Psychol. 55(1), 68 (2010)

    Article  Google Scholar 

  23. WebAIM: Screen reader user survey #8 results - webaim (2020). https://webaim.org/projects/screenreadersurvey8/

  24. Wiȩcek-Janka, E., Papierz, M., Kornecka, M., Nitka, M.: Apple products: a discussion of the product life cycle. In: 4th International Conference on Management Science and Management Innovation, vol. 31, pp. 159–164 (2017)

    Google Scholar 

  25. Zeng, L., Miao, M., Weber, G.: Interactive audio-haptic map explorer on a tactile display. Interact. Comput. 27(4), 413–429 (2015)

    Article  Google Scholar 

Download references

Acknowledgments

The work in this paper was supported by NSF Awards: 1805076, 1936027; NIH Awards: R01EY026621, R01EY030085, R01HD097188; and NIDILRR Award: 90IF0117-01-00.

Author information

Authors and Affiliations

  1. Stony Brook University, Stony Brook, NY, 11794, USA

    Hae-Na Lee & I. V. Ramakrishnan

  2. Old Dominion University, Norfolk, VA, 23529, USA

    Vikas Ashok

Authors
  1. Hae-Na Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vikas Ashok
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. I. V. Ramakrishnan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hae-Na Lee .

Editor information

Editors and Affiliations

  1. University of Crete and Foundation for Research and Technology – Hellas (FORTH), Heraklion, Greece

    Constantine Stephanidis

  2. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Greece

    Margherita Antona

  3. Tsinghua University, Beijing, China

    Qin Gao

  4. Chongqing University, Chongqing, China

    Jia Zhou

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lee, HN., Ashok, V., Ramakrishnan, I.V. (2020). Rotate-and-Press: A Non-visual Alternative to Point-and-Click?. In: Stephanidis, C., Antona, M., Gao, Q., Zhou, J. (eds) HCI International 2020 – Late Breaking Papers: Universal Access and Inclusive Design. HCII 2020. Lecture Notes in Computer Science(), vol 12426. Springer, Cham. https://doi.org/10.1007/978-3-030-60149-2_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-60149-2_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-60148-5

  • Online ISBN: 978-3-030-60149-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation