Skip to main content
SpringerLink
Log in

AI-Driven Intelligent Text Correction Techniques for Mobile Text Entry

  • Chapter
  • First Online:
Artificial Intelligence for Human Computer Interaction: A Modern Approach

Part of the book series: Human–Computer Interaction Series ((HCIS))

Abstract

Current text correction processes on mobile touch devices are laborious: users either extensively use backspace, or navigate the cursor to the error position, make a correction, and navigate back, usually by employing multiple taps or drags over small targets. In this chapter, we present two techniques, Type, Then Correct and JustCorrect, that utilize the power of artificial intelligence to improve the text correction experience on mobile devices. All of the techniques skip error-deletion and cursor-positioning procedures, and instead allow the user to type the correction first, and then apply that correction to a previously committed error. We evaluated these techniques in and the results show that correction with the new techniques was faster than de facto cursor and backspace-based correction.

Portions of this chapter are reproduced with permission of the ACM from the following previously published papers [10, 66].

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The model and data processing codes are available at https://github.com/DrustZ/CorrectionRNN.

  2. 2.

    https://android.googlesource.com/platform/packages/inputmethods/LatinIME/.

  3. 3.

    https://github.com/farmerbb/Notepad.

References

  1. Agirre E, Cer D, Diab M, Gonzalez-Agirre A, Guo W (2013) *SEM 2013 shared task: semantic textual similarity. In: Second joint conference on lexical and computational semantics (*SEM), Volume 1: Proceedings of the main conference and the shared task: semantic textual similarity, Association for Computational Linguistics, Atlanta, Georgia, USA, pp 32–43. https://www.aclweb.org/anthology/S13-1004

  2. Apple (2018) About the keyboards settings on your iphone, ipad, and ipod touch. https://support.apple.com/en-us/HT202178. Accessed 22 Aug 2019

  3. Arif AS, Stuerzlinger W (2013) Pseudo-pressure detection and its use in predictive text entry on touchscreens. In: Proceedings of the 25th australian computer-human interaction conference: augmentation, application, innovation, collaboration, Association for Computing Machinery, New York, NY, USA, OzCHI ’13, p 383–392. https://doi.org/10.1145/2541016.2541024

  4. Arif AS, Kim S, Stuerzlinger W, Lee G, Mazalek A (2016) Evaluation of a smart-restorable backspace technique to facilitate text entry error correction. In: Proceedings of the 2016 CHI conference on human factors in computing systems, Association for Computing Machinery, New York, NY, USA, CHI ’16, pp 5151–5162. https://doi.org/10.1145/2858036.2858407

  5. Azure M (2019) Text to speech api. https://azure.microsoft.com/en-us/services/cognitive-services/text-to-speech/. Accessed 25 Aug 2019

  6. Bahdanau D, Cho K, Bengio Y (2016) Neural machine translation by jointly learning to align and translate. 1409.0473

    Google Scholar 

  7. Benko H, Wilson AD, Baudisch P (2006) Precise selection techniques for multi-touch screens. In: Proceedings of the SIGCHI conference on human factors in computing systems, Association for Computing Machinery, New York, NY, USA, CHI ’06, pp 1263–1272. https://doi.org/10.1145/1124772.1124963

  8. Brooke J (2013) Sus: a retrospective. J Usability Studies 8(2):29–40

    Google Scholar 

  9. Cho K, van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using RNN encoder–decoder for statistical machine translation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), Association for Computational Linguistics, Doha, Qatar, pp 1724–1734. https://doi.org/10.3115/v1/D14-1179. https://www.aclweb.org/anthology/D14-1179

  10. Cui W, Zhu S, Zhang MR, Schwartz A, Wobbrock JO, Bi X (2020) Justcorrect: Intelligent post hoc text correction techniques on smartphones. In: Proceedings of the 33rd annual ACM symposium on user interface software and technology, Association for Computing Machinery, New York, NY, USA, UIST ’20, pp 487–499. https://doi.org/10.1145/3379337.3415857

  11. Davies M (2018) The corpus of contemporary American english: 1990-present

    Google Scholar 

  12. Dhakal V, Feit AM, Kristensson PO, Oulasvirta A (2018) Observations on Typing from 136 Million Keystrokes, Association for Computing Machinery, New York, NY, USA, pp 1–12. https://doi.org/10.1145/3173574.3174220

  13. Erk K (2012) Vector space models of word meaning and phrase meaning: a survey. Lang Ling Compass 6(10):635–653

    Article  Google Scholar 

  14. Findlater L, Froehlich JE, Fattal K, Wobbrock JO, Dastyar T (2013) Age-related differences in performance with touchscreens compared to traditional mouse input. In: Proceedings of the SIGCHI conference on human factors in computing systems, Association for Computing Machinery, New York, NY, USA, CHI ’13, pp 343–346. https://doi.org/10.1145/2470654.2470703

  15. Fitzmaurice G, Khan A, Pieké R, Buxton B, Kurtenbach G (2003) Tracking menus. In: Proceedings of the 16th Annual ACM symposium on user interface software and technology, Association for Computing Machinery, New York, NY, USA, UIST ’03, pp. 71–79. https://doi.org/10.1145/964696.964704

  16. Fowler A, Partridge K, Chelba C, Bi X, Ouyang T, Zhai S (2015) Effects of language modeling and its personalization on touchscreen typing performance. In: Proceedings of the 33rd Annual ACM conference on human factors in computing systems, Association for Computing Machinery, New York, NY, USA, CHI ’15, pp. 649–658. https://doi.org/10.1145/2702123.2702503

  17. Frederick BN (1999) Fixed-, random-, and mixed-effects anova models: a user-friendly guide for increasing the generalizability of anova results. Advances in social science methodology, Stamford. JAI Press, CT, pp 111–122

    Google Scholar 

  18. Fuccella V, Isokoski P, Martin B (2013) Gestures and widgets: performance in text editing on multi-touch capable mobile devices. In: Proceedings of the SIGCHI conference on human factors in computing systems, ACM, New York, NY, USA, CHI ’13, pp 2785–2794. https://doi.org/10.1145/2470654.2481385, http://doi.acm.org/10.1145/2470654.2481385

  19. Heafield K (2011) KenLM: faster and smaller language model queries. In: Proceedings of the EMNLP 2011 sixth workshop on statistical machine translation, Edinburgh, Scotland, United Kingdom, pp 187–197. https://kheafield.com/papers/avenue/kenlm.pdf

  20. Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6(2):65–70. http://www.jstor.org/stable/4615733

  21. Holz C, Baudisch P (2011) Understanding touch. In: Proceedings of the SIGCHI conference on human factors in computing systems, Association for Computing Machinery, New York, NY, USA, CHI ’11, pp 2501–2510. https://doi.org/10.1145/1978942.1979308

  22. Inc E (2018) Messagease - the smartest touch screen keyboard. https://www.exideas.com/ME/index.php. Accessed 22 Aug 2019

  23. Inc G (2020) Grammarly keyboard. https://en.wikipedia.org/wiki/Grammarly. Accessed May 2020

  24. Islam A, Inkpen D (2009) Real-word spelling correction using google web it 3-grams. In: Proceedings of the 2009 conference on empirical methods in natural language processing: Volume 3 - Volume 3, Association for Computational Linguistics, USA, EMNLP ’09, pp 1241–1249

    Google Scholar 

  25. Isokoski P, Martin B, Gandouly P, Stephanov T (2010) Motor efficiency of text entry in a combination of a soft keyboard and unistrokes. In: Proceedings of the 6th Nordic conference on human-computer interaction: extending boundaries, ACM, New York, NY, USA, NordiCHI ’10, pp 683–686. https://doi.org/10.1145/1868914.1869004. http://doi.acm.org/10.1145/1868914.1869004

  26. Kim Y, Jernite Y, Sontag D, Rush AM (2016) Character-aware neural language models. In: Proceedings of the Thirtieth AAAI conference on artificial intelligence, AAAI Press, AAAI’16, pp 2741–2749

    Google Scholar 

  27. Komninos A, Nicol E, Dunlop MD (2015) Designed with older adults to supportbetter error correction in smartphone text entry: the maxiekeyboard. In: Proceedings of the 17th international conference on human-computer interaction with mobile devices and services adjunct, Association for Computing Machinery, New York, NY, USA, MobileHCI ’15, pp 797–802. https://doi.org/10.1145/2786567.2793703

  28. Komninos A, Dunlop M, Katsaris K, Garofalakis J (2018) A glimpse of mobile text entry errors and corrective behaviour in the wild. In: Proceedings of the 20th international conference on human-computer interaction with mobile devices and services adjunct, Association for Computing Machinery, New York, NY, USA, MobileHCI ’18, pp 221–228. https://doi.org/10.1145/3236112.3236143

  29. Kristensson PO, Zhai S (2004) Shark2: a large vocabulary shorthand writing system for pen-based computers. In: Proceedings of the 17th annual ACM symposium on user interface software and technology, ACM, New York, NY, USA, UIST ’04, pp 43–52. https://doi.org/10.1145/1029632.1029640. http://doi.acm.org/10.1145/1029632.1029640

  30. Leiva LA, Sahami A, Catala A, Henze N, Schmidt A (2015) Text entry on tiny qwerty soft keyboards. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, Association for Computing Machinery, New York, NY, USA, CHI ’15, pp 669–678. https://doi.org/10.1145/2702123.2702388

  31. Levenshtein VI (1965) Binary codes capable of correcting deletions, insertions, and reversals. Soviet Phys Doklady 10:707–710

    MathSciNet  MATH  Google Scholar 

  32. Limpert E, Stahel WA, Abbt M (2001) Log-normal distributions across the sciences: keys and clues: on the charms of statistics, and how mechanical models resembling gambling machines offer a link to a handy way to characterize log-normal distributions, which can provide deeper insight into variability and probability–normal or log-normal: that is the question. BioScience 51(5):341–352. https://doi.org/10.1641/0006-3568(2001)051[0341:LNDATS]2.0.CO;2. https://academic.oup.com/bioscience/article-pdf/51/5/341/26891292/51-5-341.pdf

  33. Littell R, Henry P, Ammerman C (1998) Statistical analysis of repeated measures data using sas procedures. J Animal Sci 76(4):1216–1231. https://doi.org/10.2527/1998.7641216x

    Article  Google Scholar 

  34. LLC G (2020) Gboard. URLhttps://en.wikipedia.org/wiki/Gboard. Accessed May 2020

  35. MacKenzie IS, Soukoreff RW (2002) A character-level error analysis technique for evaluating text entry methods. In: Proceedings of the second nordic conference on human-computer interaction, Association for Computing Machinery, New York, NY, USA, NordiCHI ’02, pp 243–246. https://doi.org/10.1145/572020.572056

  36. MacKenzie IS, Soukoreff RW (2002) Text entry for mobile computing: Models and methods,theory and practice. Hum-Comput Int 17(2-3):147–198. https://doi.org/10.1080/07370024.2002.9667313. https://www.tandfonline.com/doi/abs/10.1080/07370024.2002.9667313

  37. Mahoney M (2011) About text8 file. http://mattmahoney.net/dc/textdata.html. Accessed May 2020

  38. Mikolov T, Chen K, Corrado GS, Dean J (2013) Efficient estimation of word representations in vector space. arXiv:1301.3781

  39. Ng HT, Wu SM, Wu Y, Hadiwinoto C, Tetreault J (2013) The CoNLL-2013 shared task on grammatical error correction. In: Proceedings of the seventeenth conference on computational natural language learning: shared task, Association for Computational Linguistics, Sofia, Bulgaria, pp 1–12. https://www.aclweb.org/anthology/W13-3601

  40. Ng HT, Wu SM, Briscoe T, Hadiwinoto C, Susanto RH, Bryant C (2014) The CoNLL-2014 shared task on grammatical error correction. In: Proceedings of the eighteenth conference on computational natural language learning: shared task, Association for Computational Linguistics, Baltimore, Maryland, pp 1–14. https://doi.org/10.3115/v1/W14-1701. https://www.aclweb.org/anthology/W14-1701

  41. Ola Kristensson P, Vertanen K (2011) Asynchronous multimodal text entry using speech and gesture keyboards. In: Proceedings of the international conference on spoken language processing, pp 581–584

    Google Scholar 

  42. Palin K, Feit A, Kim S, Kristensson PO, Oulasvirta A (2019) How do people type on mobile devices? Observations from a study with 37,000 volunteers. In: Proceedings of 21st international conference on human-computer interaction with mobile devices and services (MobileHCI’19), ACM

    Google Scholar 

  43. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, Desmaison A, Kopf A, Yang E, DeVito Z, Raison M, Tejani A, Chilamkurthy S, Steiner B, Fang L, Bai J, Chintala S (2019) Pytorch: an imperative style, high-performance deep learning library. In: Wallach H, Larochelle H, Beygelzimer A, d’ Alché-Buc F, Fox E, Garnett R (eds) Advances in neural information processing systems 32, Curran Associates, Inc., pp 8024–8035. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf

  44. Pennington J, Socher R, Manning C (2014) GloVe: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), Association for Computational Linguistics, Doha, Qatar, pp 1532–1543. https://doi.org/10.3115/v1/D14-1162. https://www.aclweb.org/anthology/D14-1162

  45. Řehůřek R, Sojka P (2010) Software framework for topic modelling with large corpora. In: Proceedings of the LREC 2010 workshop on new challenges for NLP frameworks, ELRA, Valletta, Malta, pp 45–50. http://is.muni.cz/publication/884893/en

  46. Ruan S, Wobbrock JO, Liou K, Ng A, Landay JA (2018) Comparing speech and keyboard text entry for short messages in two languages on touchscreen phones. Proc ACM Interact Mob Wearable Ubiquitous Technol 1(4). https://doi.org/10.1145/3161187

  47. Rubio S, Díaz EM, Martín J, Puente J (2004) Evaluation of subjective mental workload: a comparison of swat, nasa-tlx, and workload profile methods. Appl Psychol 53:61–86

    Article  Google Scholar 

  48. Schmidt D, Block F, Gellersen H (2009) A comparison of direct and indirect multi-touch input for large surfaces. In: Gross T, Gulliksen J, Kotzé P, Oestreicher L, Palanque P, Prates RO, Winckler M (eds) Human-computer interaction - INTERACT 2009. Springer, Berlin, pp 582–594

    Google Scholar 

  49. Sears A, Shneiderman B (1991) High precision touchscreens: design strategies and comparisons with a mouse. Int J Man Mach Stud 34:593–613

    Article  Google Scholar 

  50. Sim KC (2010) Haptic voice recognition: Augmenting speech modality with touch events for efficient speech recognition. In: 2010 IEEE spoken language technology workshop, pp 73–78. https://doi.org/10.1109/SLT.2010.5700825

  51. Sim KC (2012) Speak-as-you-swipe (says): A multimodal interface combining speech and gesture keyboard synchronously for continuous mobile text entry. In: Proceedings of the 14th ACM international conference on multimodal interaction, ACM, New York, NY, USA, ICMI ’12, pp 555–560. https://doi.org/10.1145/2388676.2388793. http://doi.acm.org/10.1145/2388676.2388793

  52. Sindhwani S, Lutteroth C, Weber G (2019) Retype: Quick text editing with keyboard and gaze. In: Proceedings of the 2019 CHI conference on human factors in computing systems, ACM, New York, NY, USA, CHI ’19, pp 203:1–203:13. https://doi.org/10.1145/3290605.3300433. http://doi.acm.org/10.1145/3290605.3300433

  53. Soukoreff RW, MacKenzie IS (2004) Recent developments in text-entry error rate measurement. In: CHI ’04 extended abstracts on human factors in computing systems, Association for Computing Machinery, New York, NY, USA, CHI EA ’04, pp 1425–1428. https://doi.org/10.1145/985921.986081

  54. Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. In: Proceedings of the 27th international conference on neural information processing systems - Volume 2, MIT Press, Cambridge, MA, USA, NIPS’14, pp 3104–3112

    Google Scholar 

  55. Vertanen K, Memmi H, Emge J, Reyal S, Kristensson PO (2015) Velocitap: Investigating fast mobile text entry using sentence-based decoding of touchscreen keyboard input. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, Association for Computing Machinery, New York, NY, USA, CHI ’15, pp 659–668. https://doi.org/10.1145/2702123.2702135

  56. Vogel D, Baudisch P (2007) Shift: a technique for operating pen-based interfaces using touch. In: Proceedings of the SIGCHI conference on human factors in computing systems, Association for Computing Machinery, New York, NY, USA, CHI ’07, pp 657–666. https://doi.org/10.1145/1240624.1240727

  57. Wagner RA, Fischer MJ (1974) The string-to-string correction problem. J ACM (JACM) 21(1):168–173

    Article  MathSciNet  Google Scholar 

  58. Weidner K (2018) Hackers keyboard. http://code.google.com/p/hackerskeyboard/. Accessed 22 Aug 2019

  59. Wobbrock JO, Myers BA (2006) Analyzing the input stream for character- level errors in unconstrained text entry evaluations. ACM Trans Comput-Hum Interact 13(4):458–489. https://doi.org/10.1145/1188816.1188819

    Article  Google Scholar 

  60. Xie Z, Avati A, Arivazhagan N, Jurafsky D, Ng A (2016) Neural language correction with character-based attention. ArXiv:1603.09727

  61. Young T, Hazarika D, Poria S, Cambria E (2018) Recent trends in deep learning based natural language processing [review article]. IEEE Comput Intell Mag 13:55–75

    Article  Google Scholar 

  62. Zesch T (2012) Measuring contextual fitness using error contexts extracted from the Wikipedia revision history. In: Proceedings of the 13th conference of the European chapter of the association for computational linguistics, Association for Computational Linguistics, Avignon, France, pp 529–538. https://www.aclweb.org/anthology/E12-1054

  63. Zhai S, Kristensson PO (2003) Shorthand writing on stylus keyboard. In: Proceedings of the SIGCHI conference on human factors in computing systems, Association for Computing Machinery, New York, NY, USA, CHI ’03, pp 97–104. https://doi.org/10.1145/642611.642630

  64. Zhai S, Kristensson PO (2012) The word-gesture keyboard: reimagining keyboard interaction. Commun ACM 55(9):91–101. https://doi.org/10.1145/2330667.2330689

    Article  Google Scholar 

  65. Zhang MR, Wobbrock OJ (2020) Gedit: keyboard gestures for mobile text editing. In: Proceedings of graphics interface (GI ’20), Canadian information processing society, Toronto, Ontario, GI ’20, pp 97–104

    Google Scholar 

  66. Zhang MR, Wen H, Wobbrock JO (2019) Type, then correct: intelligent text correction techniques for mobile text entry using neural networks. In: Proceedings of the 32nd annual ACM symposium on user interface software and technology, Association for Computing Machinery, New York, NY, USA, UIST ’19, pp 843–855. https://doi.org/10.1145/3332165.3347924

  67. Zhang X, Zhao J, LeCun Y (2015) Character-level convolutional networks for text classification. In: Proceedings of the 28th international conference on neural information processing systems - Volume 1, MIT Press, Cambridge, MA, USA, NIPS’15, pp 649–657

    Google Scholar 

  68. Zhu S, Luo T, Bi X, Zhai S (2018) Typing on an invisible keyboard. In: Proceedings of the 2018 CHI Conference on human factors in computing systems, Association for Computing Machinery, New York, NY, USA, CHI ’18, pp 1–13. https://doi.org/10.1145/3173574.3174013

Download references

Author information

Authors and Affiliations

  1. The Information School, University of Washington, Seattle, WA, 98195, USA

    Mingrui Ray Zhang & Jacob O. Wobbrock

  2. The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, 15213, USA

    He Wen

  3. Department of Computer Science, Stony Brook University, Stony Brook, NY, 11794, USA

    Wenzhe Cui, H. Andrew Schwartz & Xiaojun Bi

  4. Grammarly, Inc., San Francisco, CA, 94104, USA

    Suwen Zhu

Authors
  1. Mingrui Ray Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. He Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenzhe Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Suwen Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. H. Andrew Schwartz
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiaojun Bi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jacob O. Wobbrock
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mingrui Ray Zhang .

Editor information

Editors and Affiliations

  1. Google Research (United States), Mountain View, CA, USA

    Yang Li

  2. Advanced Interactive Technologies Lab, ETH Zurich, Zurich, Switzerland

    Otmar Hilliges

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Zhang, M.R. et al. (2021). AI-Driven Intelligent Text Correction Techniques for Mobile Text Entry. In: Li, Y., Hilliges, O. (eds) Artificial Intelligence for Human Computer Interaction: A Modern Approach. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-030-82681-9_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-82681-9_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-82680-2

  • Online ISBN: 978-3-030-82681-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation