Skip to main content
SpringerLink
Log in

Learning Through the Lens of Sketch

  • Chapter
  • First Online:
Frontiers in Pen and Touch

Part of the book series: Human–Computer Interaction Series ((HCIS))

  • 530 Accesses

Abstract

Dr. Tracy Hammond gave a keynote on morning of the second day of the conference. She spoke about how both her research and the field of Sketch Recognition evolved over the last decade. One motivation of her career was to develop algorithms that provide insights into human brain activity and also develop applications that improve human-human communication. Her initial work focused on domain-independent recognition methods, while her current work focuses on developing systems to improve education. She also shows how sketch recognition methods can advance both sketch forensics and activity recognition, providing inspiration as per how this can allow for surprisingly intelligent personalized feedback. This chapter provides a lightly edited transcription of that keynote.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://en.wikipedia.org/wiki/Entropy_(information_theory).

References

  1. Adler, A.D.: Midos: multimodal interactive dialogue system. Ph.D. thesis, Massachusetts Institute of Technology (2009)

    Google Scholar 

  2. Alamudun, F., Yoon, H.J., Hammond, T., Hudson, K., Morin-Ducote, G., Tourassi, G.: Shapelet analysis of pupil dilation for modeling visuo-cognitive behavior in screening mammography. In: Proceedings of the SPIE. SPIE, vol. 9787, pp. 97870M01–97870M13 (2016). doi:10.1117/12.2217670. http://dx.doi.org/10.1117/12.2217670

  3. Alamudun, F., Yoon, H.J., Hudson, K.B., Morin-Ducote, G., Hammond, T., Tourassi, G.D.: Fractal analysis of visual search activity for mass detection during mammographic screening. Med. Phys. 44, 832–846 (2017)

    Article  Google Scholar 

  4. Alvarado, C.J.: A natural sketching environment: bringing the computer into early stages of mechanical design. Ph.D. thesis, Massachusetts Institute of Technology (2000)

    Google Scholar 

  5. Atilola, O., Field, M., McTigue, E., Hammond, T., Linsey, J.: Evaluation of a natural sketch interface for truss FBDs and analysis. In: Proceedings of the 2011 Frontiers in Education Conference, FIE’11, pp. S2E–1–1–S2E–6. IEEE Computer Society, Washington, DC (2011). doi:10.1109/FIE.2011.6142959. http://dx.doi.org/10.1109/FIE.2011.6142959

  6. Atilola, O., Field, M., McTigue, E., Hammond, T., Linsey, J.: Mechanix: a sketch recognition truss tutoring system. In: American Society of Mechanical Engineers (ASME) 2011 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Volume 7: 5th International Conference on Micro- and Nanosystems; 8th International Conference on Design and Design Education; 21st Reliability, Stress Analysis, and Failure Prevention Conference, vol. 7, pp. 645–654. ASME, Washington, DC (2011)

    Google Scholar 

  7. Atilola, O., McTigue, E.M., Hammond, T., Linsey, J.: Mechanix: evaluating the effectiveness of a sketch recognition truss tutoring program against other truss programs. In: 120th American Society for Engineering Education Annual Conference & Exposition (ASEE), June 23–26. ASEE, Atlanta (2013)

    Google Scholar 

  8. Atilola, O., Valentine, S., Kim, H.H., Turner, D., McTigue, E., Hammond, T., Linsey, J.: Mechanix: a natural sketch interface tool for teaching truss analysis and free-body diagrams. Artif. Intell. Eng. Des. Anal. Manuf. 28, 169–192 (2014). doi:10.1017/S0890060414000079. http://journals.cambridge.org/article_S0890060414000079

  9. Atilola, O., Vides, F., Mctigue, E.M., Linsey, J.S., Hammond, T.A.: Automatic identification of student misconceptions and errors for truss analysis. In: 119th American Society for Engineering Education Annual Conference & Exposition (ASEE), 10–13 June 2012. ASEE, San Antonio (2012)

    Google Scholar 

  10. Barreto, L., Taele, P., Hammond, T.: A stylus-driven intelligent tutoring system for music education instruction. In: Hammond, T., Valentine, S., Adler, A. (eds.) Revolutionizing Education with Digital Ink: The Impact of Pen and Touch Technology on Education, pp. 141–161. Springer, Cham (2016). doi:10.1007/978-3-319-31193-7_10. http://dx.doi.org/10.1007/978-3-319-31193-7_10

  11. Bartley, J., Forsyth, J., Pendse, P., Xin, D., Brown, G., Hagseth, P., Agrawal, A., Goldberg, D.W., Hammond, T.: World of workout: a contextual mobile RPG to encourage long term fitness. In: Proceedings of the Second ACM SIGSPATIAL International Workshop on the Use of GIS in Public Health, HealthGIS’13, pp. 60–67. ACM, New York (2013). doi:10.1145/2535708.2535718. http://doi.acm.org/10.1145/2535708.2535718

  12. Bhat, A., Hammond, T.: Using entropy to distinguish shape versus text in hand-drawn diagrams. In: Proceedings of the 21st International Jont Conference on Artifical Intelligence, IJCAI’09, pp. 1395–1400. Morgan Kaufmann Publishers Inc., San Francisco (2009). http://dl.acm.org/citation.cfm?id=1661445.1661669

  13. Brooks, R., Hammond, T., Koh, J.I.: Score improvement distribution when using sketch recognition software (mechanix) as a tutor: assessment of high school classroom pilot. In: 10th Conference on Pen and Touch Technology in Education. CPTTE 2016. CPTTE (2016)

    Google Scholar 

  14. Cherian, J., Rajanna, V., Goldberg, D., Hammond, T.: Did you remember to brush? A noninvasive wearable approach to recognizing brushing teeth for elderly care. In: 11th EAI International Conference on Pervasive Computing Technologies for Healthcare. ICDC (2017)

    Google Scholar 

  15. Choi, H., Hammond, T.: Sketch recognition based on manifold learning. In: Proceedings of the 23rd National Conference on Artificial Intelligence, AAAI’08, vol. 3, pp. 1786–1787. AAAI Press (2008). http://dl.acm.org/citation.cfm?id=1620270.1620353

  16. Choi, H., Paulson, B., Hammond, T.: Gesture recognition based on manifold learning. In: Proceedings of the 2008 Joint IAPR International Workshop on Structural, Syntactic, and Statistical Pattern Recognition, SSPR & SPR’08, pp. 247–256 (2014). Springer, Berlin/Heidelberg (2008). doi:10.1007/978-3-540-89689-0_29. http://dx.doi.org/10.1007/978-3-540-89689-0_29

  17. Cohen, J., Penney, D., Davis, R., Libon, D., Swenson, R., Ajilore, O., … Lamar, M.: Digital clock drawing: Differentiating “Thinking” versus “Doing” in younger and older adults with depression. J. Int. Neuropsychol. Soc. 20(9), 920–928 (2014). doi:10.1017/S1355617714000757

  18. Corey, P., Hammond, T.: Gladder: combining gesture and geometric sketch recognition. In: Proceedings of the 23rd National Conference on Artificial Intelligence, AAAI’08, vol. 3, pp. 1788–1789. AAAI Press (2008). http://dl.acm.org/citation.cfm?id=1620270.1620354

  19. Cummings, D., Fymat, S., Hammond, T.: Sketch-based interface for interaction with unmanned air vehicles. In: CHI’12 Extended Abstracts on Human Factors in Computing Systems, CHI EA’12, pp. 1511–1516. ACM, New York (2012). doi:10.1145/2212776.2223664. http://doi.acm.org/10.1145/2212776.2223664

  20. Cummings, D., Lucchese, G., Prasad, M., Aikens, C., Ho, J., Hammond, T.: Geotrooper: a mobile location-aware system for team coordination. In: Proceedings of the 13th International Conference of the NZ Chapter of the ACM’s Special Interest Group on Human-Computer Interaction, CHINZ’12, pp. 102–102. ACM, New York (2012). doi:10.1145/2379256.2379286. http://doi.acm.org/10.1145/2379256.2379286

  21. Cummings, D., Lucchese, G., Prasad, M., Aikens, C., Ho, J., Hammond, T.: Haptic and ar interface for paratrooper coordination. In: Proceedings of the 13th International Conference of the NZ Chapter of the ACM’s Special Interest Group on Human-Computer Interaction, CHINZ’12, pp. 52–55. ACM, New York (2012). doi:10.1145/2379256.2379265. http://doi.acm.org/10.1145/2379256.2379265

  22. Cummings, D., Prasad, M., Lucchese, G., Aikens, C., Hammond, T.A.: Multi-modal location-aware system for paratrooper team coordination. In: CHI’13 Extended Abstracts on Human Factors in Computing Systems, CHI EA’13, pp. 2385–2388. ACM, New York (2013). doi:10.1145/2468356.2468779. http://doi.acm.org/10.1145/2468356.2468779

  23. Cummmings, D., Fymat, S., Hammond, T.: Reddog: A smart sketch interface for autonomous aerial systems. In: Proceedings of the International Symposium on Sketch-Based Interfaces and Modeling, SBIM’12, pp. 21–28. Eurographics Association, Aire-la-Ville (2012). http://dl.acm.org/citation.cfm?id=2331067.2331071

  24. Cummmings, D., Vides, F., Hammond, T.: I don’t believe my eyes! Geometric sketch recognition for a computer art tutorial. In: Proceedings of the International Symposium on Sketch-Based Interfaces and Modeling, SBIM’12, pp. 97–106. Eurographics Association, Aire-la-Ville (2012). http://dl.acm.org/citation.cfm?id=2331067.2331082

  25. Dahmen, K., Hammond, T.: Distinguishing between sketched scribble look alikes. In: Proceedings of the 23rd National Conference on Artificial Intelligence, AAAI’08, vol. 3, pp. 1790–1791. AAAI Press (2008). http://dl.acm.org/citation.cfm?id=1620270.1620355

  26. Damaraju, S., Seo, J.H., Hammond, T., Kerne, A.: Multi-tap sliders: advancing touch interaction for parameter adjustment. In: Proceedings of the 2013 International Conference on Intelligent User Interfaces, IUI’13, pp. 445–452. ACM, New York (2013). doi:10.1145/2449396.2449453. http://doi.acm.org/10.1145/2449396.2449453

  27. David, J., Eoff, B., Hammond, T.: Coske-an exploration in collaborative sketching. In: Computer Supported Cooperative Work Posters (CSCW), pp. 471–472. Savannah (2010)

    Google Scholar 

  28. Dixon, D., Prasad, M., Hammond, T.: iCanDraw?: a methodology for using assistive sketch recognition to improve a user’s drawing ability (2009)

    Google Scholar 

  29. Dixon, D., Prasad, M., Hammond, T.: icandraw: using sketch recognition and corrective feedback to assist a user in drawing human faces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’10, pp. 897–906. ACM, New York (2010). doi:10.1145/1753326.1753459. http://doi.acm.org/10.1145/1753326.1753459

  30. Eoff, B.D., Hammond, T.: User identification by means of sketched stroke features. In: Proceedings of the 23rd National Conference on Artificial Intelligence, AAAI’08, vol. 3, pp. 1794–1795. AAAI Press (2008). http://dl.acm.org/citation.cfm?id=1620270.1620357

  31. Eoff, B.D., Hammond, T.: Who dotted that ’i’? Context free user differentiation through pressure and tilt pen data. In: Proceedings of Graphics Interface 2009, GI’09, pp. 149–156. Canadian Information Processing Society, Toronto (2009). http://dl.acm.org/citation.cfm?id=1555880.1555916

  32. Field, M., Valentine, S., Linsey, J., Hammond, T.: Sketch recognition algorithms for comparing complex and unpredictable shapes. In: Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, IJCAI’11, vol. 3, pp. 2436–2441. AAAI Press (2011). doi:10.5591/978-1-57735-516-8/IJCAI11-406. http://dx.doi.org/10.5591/978-1-57735-516-8/IJCAI11-406

  33. Goldberg, D.W., Cockburn, M.G., Hammond, T.A., Jacquez, G.M., Janies, D., Knoblock, C., Kuhn, W., Pultar, E., Raubal, M.: Envisioning a future for a spatial-health cybergis marketplace. In: Proceedings of the Second ACM SIGSPATIAL International Workshop on the Use of GIS in Public Health, HealthGIS’13, pp. 27–30. ACM, New York (2013). doi:10.1145/2535708.2535716. http://doi.acm.org/10.1145/2535708.2535716

  34. Green, M.G., Caldwell, B.W., Helms, M., Linsey, J.S., Hammond, T.A.: Using natural sketch recognition software to provide instant feedback on statics homework (truss free body diagrams): assessment of a classroom pilot. In: 2015 ASEE Annual Conference and Exposition. Seattle (2015). doi:10.18260/p.25007. https://peer.asee.org/25007

  35. Hammond, T.: Automatically generating sketch interfaces from shape descriptions. In: Proceedings of the 4th Annual MIT Student Oxygen Workshop. MIT (2004)

    Google Scholar 

  36. Hammond, T.: Enabling instructors to develop sketch recognition applications for the classroom. In: 37th Annual Frontiers In Education Conference – Global Engineering: Knowledge Without Borders, Opportunities Without Passports, FIE’07, pp. S3J–11–S3J–16 (2007). doi:10.1109/FIE.2007.4417930

  37. Hammond, T.: Ladder: a perceptually-based language to simplify sketch recognition user interface development. PhD Doctoral Dissertation, Massachusetts Institute of Technology (MIT), Cambridge (2007). Advisor: Randall Davis, 495 pp.

    Google Scholar 

  38. Hammond, T.: Simplifying sketch recognition UI development. In: Grace Hopper Celebration of Women in Computing. GHC, Orlando (2007)

    Google Scholar 

  39. Hammond, T.: Workshop – integrating sketch recognition technologies into your classroom. In: 38th Annual Frontiers in Education Conference, FIE 2008, pp. W2C–1 (2008). doi:10.1109/FIE.2008.4720505

    Google Scholar 

  40. Hammond, T.: Dialectical creativity: sketch-negate-create. In: Studying Visual and Spatial Reasoning for Design Creativity, pp. 91–108. Springer Netherlands, Dordrecht (2015). doi:10.1007/978-94-017-9297-4_6. http://dx.doi.org/10.1007/978-94-017-9297-4_6

  41. Hammond, T., Davis, R.: Tahuti: a geometrical sketch recognition system for UML class diagrams. In: Technical Report SS-02-08: Papers from the 2002 Association for the Advancement of Artificial Intelligence (AAAI) Spring Symposium on Sketch Understanding. AAAI, Menlo Park (2002)

    Google Scholar 

  42. Hammond, T., Davis, R.: Ladder: a language to describe drawing, display, and editing in sketch recognition. In: Proceedings of the 18th International Joint Conference on Artificial Intelligence, IJCAI’03, pp. 461–467. Morgan Kaufmann Publishers Inc., San Francisco (2003). http://dl.acm.org/citation.cfm?id=1630659.1630728

  43. Hammond, T., Davis, R.: Automatically transforming symbolic shape descriptions for use in sketch recognition. In: Proceedings of the 19th National Conference on Artificial Intelligence, AAAI’04, pp. 450–456. AAAI Press (2004). http://dl.acm.org/citation.cfm?id=1597148.1597222

  44. Hammond, T., Davis, R.: Debugging shape definitions for use in sketch recognition. In: MIT Lab Abstract. MIT, Cambridge (2004)

    Google Scholar 

  45. Hammond, T., Davis, R.: Shady: a shape description debugger for use in sketch recognition. In: AAAI Fall Symposium on Making Pen-Based Interaction Intelligent and Natural (AAAI). AAAI, Arlington (2004)

    Google Scholar 

  46. Hammond, T., Davis, R.: Ladder, a sketching language for user interface developers. Comput. Graph. 29(4), 518–532 (2005)

    Article  Google Scholar 

  47. Hammond, T., Davis, R.: Interactive learning of structural shape descriptions from automatically generated near-miss examples. In: Proceedings of the 11th International Conference on Intelligent User Interfaces, IUI’06, pp. 210–217. ACM, New York (2006). doi:10.1145/1111449.1111495. http://doi.acm.org/10.1145/1111449.1111495

  48. Hammond, T., Davis, R.: Creating the perception-based ladder sketch recognition language. In: Proceedings of the 8th ACM Conference on Designing Interactive Systems, DIS’10, pp. 141–150. ACM, New York (2010). doi:10.1145/1858171.1858197. http://doi.acm.org/10.1145/1858171.1858197

  49. Hammond, T., Eoff, B., Paulson, B., Wolin, A., Dahmen, K., Johnston, J., Rajan, P.: Free-sketch recognition: putting the chi in sketching. In: CHI’08 Extended Abstracts on Human Factors in Computing Systems, CHI EA’08, pp. 3027–3032. ACM, New York (2008). doi:10.1145/1358628.1358802. http://doi.acm.org/10.1145/1358628.1358802

  50. Hammond, T., Gajos, K., Davis, R., Shrobe, H.: An agent-based system for capturing and indexing software design meetings. In: Proceedings of International Workshop on Agents In Design, WAID. MIT, Cambridge (2002)

    Google Scholar 

  51. Hammond, T., Logsdon, D., Paulson, B., Johnston, J., Peschel, J., Wolin, A., Taele, P.: A sketch recognition system for recognizing free-hand course of action diagrams. In: Innovative Applications of Artificial Intelligence, pp. 1781–1786. Atlanta (2010). http://www.aaai.org/ocs/index.php/IAAI/IAAI10/paper/view/1581

  52. Hammond, T., Logsdon, D., Peschel, J., Johnston, J., Taele, P., Wolin, A., Paulson, B.: A sketch recognition interface that recognizes hundreds of shapes in course-of-action diagrams. In: CHI’10 Extended Abstracts on Human Factors in Computing Systems, CHI EA’10, pp. 4213–4218. ACM, New York (2010). doi:10.1145/1753846.1754128. http://doi.acm.org/10.1145/1753846.1754128

  53. Hammond, T., O’Sullivan, B.: Recognizing free-form hand-sketched constraint network diagrams by combining geometry and context. In: Proceedings of the Eurographics Ireland, pp. 67–74. Eurographics, Dublin (2007)

    Google Scholar 

  54. Hammond, T., Paulson, B.: Recognizing sketched multistroke primitives. ACM Trans. Interact. Intell. Syst. 1(1), 4:1–4:34 (2011). doi:10.1145/2030365.2030369. http://doi.acm.org/10.1145/2030365.2030369

  55. Hammond, T., Prasad, M., Dixon, D.: Art 101: learning to draw through sketch recognition. In: Proceedings of the 10th International Conference on Smart Graphics, SG’10, pp. 277–280. Springer, Berlin/Heidelberg (2010). http://dl.acm.org/citation.cfm?id=1894345.1894383

  56. Hammond, T., Valentine, S., Adler, A., Payton, M.: The Impact of Pen and Touch Technology on Education, 1st edn. Springer Publishing Company, Incorporated (2015)

    Book  MATH  Google Scholar 

  57. Hammond, T.A., Davis, R.: Recognizing interspersed sketches quickly. In: Proceedings of Graphics Interface 2009, GI’09, pp. 157–166. Canadian Information Processing Society, Toronto (2009). http://dl.acm.org/citation.cfm?id=1555880.1555917

  58. Herot, C.F.: Graphical input through machine recognition of sketches. In: Proceedings of the 3rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH’76, pp. 97–102. ACM, New York (1976). doi:10.1145/563274.563294. http://doi.acm.org/10.1145/563274.563294

  59. Hilton, E.C., WilliFord, B., Li, W., McTigue, E., Hammond, T., Linsey, J.: Consistently evaluating sketching ability in engineering curriculum. In: 4th ICDC, International Conference on Design and creativity. ICDC (2016)

    Google Scholar 

  60. Hong, Y., Vollmer Dahlke, D., Ory, M., Goldberg, D., Cargill, J., Kellstedt, D., Pulczinski, J., Hammond, T., Hernandez, E.: Development of icanfit: a mobile device application to promote physical activity and access to health information among older cancer survivors. In: 142nd APHA Annual Meeting and Exposition, 15–19 Nov 2014. APHA (2014)

    Google Scholar 

  61. Johnston, J., Hammond, T.: Computing confidence values for geometric constraints for use in sketch recognition. In: Proceedings of the Seventh Sketch-Based Interfaces and Modeling Symposium, SBIM’10, pp. 71–78. Eurographics Association, Aire-la-Ville (2010). http://dl.acm.org/citation.cfm?id=1923363.1923376

  62. Kaster, B.L., Jacobson, E.R., Hammond, T.A.: Sssousa: automatically generating secure and searchable data collection studies. In: International Workshop on Visual Languages and Computing. VLC/DMS, Redwood City (2009)

    Google Scholar 

  63. Kaul, P., Rajanna, V., Hammond, T.: Exploring users’ perceived activities in a sketch-based intelligent tutoring system through eye movement data. In: ACM Symposium on Applied Perception (SAP’16). SAP (2016)

    Google Scholar 

  64. Kebodeaux, K., Field, M., Hammond, T.: Defining precise measurements with sketched annotations. In: Proceedings of the Eighth Eurographics Symposium on Sketch-Based Interfaces and Modeling, SBIM’11, pp. 79–86. ACM, New York (2011). doi:10.1145/2021164.2021179. http://doi.acm.org/10.1145/2021164.2021179

  65. Kim, H.H., Taele, P., Seo, J., Jeffrey, L., Hammond, T.: A novel sketch-based interface for improving children’s fine motor skills and school readiness. In: Proceedings of the International Symposium on Sketch-Based Interfaces and Modeling, SAP, pp. 1–10, 2016

    Google Scholar 

  66. Kim, H.H., Taele, P., Valentine, S., Hammond, T.: Developing intelligent sketch-based applications for children’s fine motor sketching skill development. In: 2014 International Conference on Intelligent User Interfaces (IUI) Workshop on Sketch: Pen and Touch Recognition. IUI, ACM, Haifa (2014)

    Google Scholar 

  67. Kim, H.H., Taele, P., Valentine, S., McTigue, E., Hammond, T.: Kimchi: a sketch-based developmental skill classifier to enhance pen-driven educational interfaces for children. In: Proceedings of the International Symposium on Sketch-Based Interfaces and Modeling, SBIM’13, pp. 33–42. ACM, New York (2013). doi:10.1145/2487381.2487389. http://doi.acm.org/10.1145/2487381.2487389

  68. Kim, H.H., Valentine, S., Taele, P., Hammond, T.: Easysketch: a sketch-based fine motor skill recognizing educational interface for children emerging technology research strand. In: Workshop on the Impact of Pen & Touch Technology on Education (WIPTTE). WIPTTE, College Station (2014)

    Google Scholar 

  69. Kim, H.H., Valentine, S., Taele, P., Hammond, T.: Easysketch: a sketch-based fine motor skill recognizing educational interface for children emerging technology research strand. In: Hammond, T., Valentine, S., Adler, A., Payton, M. (eds.) The Impact of Pen and Touch Technology on Education, 1st edn., pp. 35–46. Springer Publishing Company, Incorporated (2015)

    Chapter  Google Scholar 

  70. Lara-Garduno, R., Leslie, N., Hammond, T.: Smartstrokes: Digitizing paper-based neuropsychological tests. In: Hammond, T., Valentine, S., Adler, A. (eds.) Revolutionizing Education with Digital Ink: The Impact of Pen and Touch Technology on Education, pp. 163–175. Springer, Cham (2016). doi:10.1007/978-3-319-31193-7_11. http://dx.doi.org/10.1007/978-3-319-31193-7_11

  71. Li, W., Hammond, T.: Recognizing text through sound alone. In: AAAI Conference on Artificial Intelligence (2011). http://www.aaai.org/ocs/index.php/AAAI/AAAI11/paper/view/3791

  72. Li, W., Hammond, T.: Using scribble gestures to enhance editing behaviors of sketch recognition systems. In: CHI’12 Extended Abstracts on Human Factors in Computing Systems, CHI EA’12, pp. 2213–2218. ACM, New York (2012). doi:10.1145/2212776.2223778. http://doi.acm.org/10.1145/2212776.2223778

  73. Lucchese, G., Field, M., Ho, J., Gutierrez-Osuna, R., Hammond, T.: Gesturecommander: continuous touch-based gesture prediction. In: CHI’12 Extended Abstracts on Human Factors in Computing Systems, CHI EA’12, pp. 1925–1930. ACM, New York (2012). doi:10.1145/2212776.2223730. http://doi.acm.org/10.1145/2212776.2223730

  74. Lupfer, N., Field, M., Kerne, A., Hammond, T.: sketchy: morphing user sketches for artistic assistance. In: Proceedings of the 2011 ACM International conference on intelligent user interface. ACM, Palo Alto (2011)

    Google Scholar 

  75. MacKenzie, I.S., Zhang, S.X.: The immediate usability of graffiti. In: Graphics Interface, vol. 97, pp. 129–137, 1997

    Google Scholar 

  76. Miller, J., Hammond, T.: Wiiolin: a virtual instrument using the Wii remote. In: Proceedings of the 2010 Conference on New Interfaces for Musical Expression (NIME), pp. 497–500. Sydney (2010)

    Google Scholar 

  77. Nelligan, T., Polsley, S., Ray, J., Helms, M., Linsey, J., Hammond, T.: Mechanix: a sketch-based educational interface. In: Proceedings of the 20th International Conference on Intelligent User Interfaces Companion, IUI Companion’15, pp. 53–56. ACM, New York (2015). doi:10.1145/2732158.2732194. http://doi.acm.org/10.1145/2732158.2732194

  78. Paulson, B., Cummings, D., Hammond, T.: Object interaction detection using hand posture cues in an office setting. Int. J. Hum. Comput. Stud. 69(1–2), 19–29 (2011). doi:10.1016/j.ijhcs.2010.09.003. http://dx.doi.org/10.1016/j.ijhcs.2010.09.003

  79. Paulson, B., Eoff, B., Wolin, A., Johnston, J., Hammond, T.: Sketch-based educational games: drawing kids away from traditional interfaces. In: Proceedings of the 7th International Conference on Interaction Design and Children, IDC’08, pp. 133–136. ACM, New York (2008). doi:10.1145/1463689.1463739. http://doi.acm.org/10.1145/1463689.1463739

  80. Paulson, B., Hammond, T.: A system for recognizing and beautifying low-level sketch shapes using NDDE and DCR. In: ACM Symposium on User Interface Software and Technology (UIST). ACM, Newport (2007)

    Google Scholar 

  81. Paulson, B., Hammond, T.: MARQS: retrieving sketches learned from a single example using a dual-classifier. J. Multimodal User Interface 2(1), 3–11 (2008). doi:10.1007/s12193-008-0006-0. http://dx.doi.org/10.1007/s12193-008-0006-0

  82. Paulson, B., Hammond, T.: Office activity recognition using hand posture cues. In: Proceedings of the 22nd British HCI Group Annual Conference on People and Computers: Culture, Creativity, Interaction, BCS-HCI’08, vol. 2, pp. 75–78. British Computer Society, Swinton (2008). http://dl.acm.org/citation.cfm?id=1531826.1531845

  83. Paulson, B., Hammond, T.: Paleosketch: accurate primitive sketch recognition and beautification. In: Proceedings of the 13th International Conference on Intelligent User Interfaces, IUI’08, pp. 1–10. ACM, New York (2008). doi:10.1145/1378773.1378775. http://doi.acm.org/10.1145/1378773.1378775

  84. Paulson, B., Hammond, T.: Towards a framework for truly natural low-level sketch recognition. In: Proceedings of the Workshop on Sketch Recognition at the 14th International Conference of Intelligent User Interfaces (IUI). ACM, Sanibel (2009)

    Google Scholar 

  85. Paulson, B., Rajan, P., Davalos, P., Gutierrez-Osuna, R., Hammond, T.: What!?! no rubine features? using geometric-based features to produce normalized confidence values for sketch recognition. In: HCC Workshop: Sketch Tools for Diagramming (VL/HCC), pp. 57–63. VL/HCC, Herrsching am Ammersee (2008)

    Google Scholar 

  86. Paulson, B., Wolin, A., Johnston, J., Hammond, T.: SOUSA: sketch-based online user study applet. In: Proceedings of the Fifth Eurographics Conference on Sketch-Based Interfaces and Modeling, SBM’08, pp. 81–88. Eurographics Association, Aire-la-Ville (2008). doi:10.2312/SBM/SBM08/081-088. http://dx.doi.org/10.2312/SBM/SBM08/081-088

  87. Peschel, J.M., Hammond, T.A.: Strat: a sketched-truss recognition and analysis tool. In: 2008 International Workshop on Visual Languages and Computing (VLC) at the 14th International Conference on distributed Multimedia Systems (DMS), pp. 282–287. Knowledge Systems Instistute, Boston (2008)

    Google Scholar 

  88. Peschel, J.M., Paulson, B., Hammond, T.: A surfaceless pen-based interface. In: Proceedings of the Seventh ACM Conference on Creativity and Cognition, C&C’09, pp. 433–434. ACM, New York (2009). doi:10.1145/1640233.1640338. http://doi.acm.org/10.1145/1640233.1640338

  89. Polsley, S., Ray, J., Nelligan, T., Helms, M., Linsey, J., Hammond, T.: Leveraging trends in student interaction to enhance the effectiveness of sketch-based educational software. In: Hammond, T., Valentine, S., Adler, A. (eds.) Revolutionizing Education with Digital Ink: The Impact of Pen and Touch Technology on Education, pp. 103–114. Springer, Cham (2016). doi:10.1007/978-3-319-31193-7_7. http://dx.doi.org/10.1007/978-3-319-31193-7_7

  90. Prasad, M., Hammond, T.: Observational study on teaching artifacts created using tablet pc. In: CHI’12 Extended Abstracts on Human Factors in Computing Systems, CHI EA’12, pp. 301–316. ACM, New York (2012). doi:10.1145/2212776.2212809. http://doi.acm.org/10.1145/2212776.2212809

  91. Prasad, M., Russell, M., Hammond, T.A.: Designing vibrotactile codes to communicate verb phrases. ACM Trans. Multimed. Comput. Commun. Appl. 11(1s), 11:1–11:21 (2014). doi:10.1145/2637289. http://doi.acm.org/10.1145/2637289

  92. Prasad, M., Russell, M.I., Hammond, T.A.: A user centric model to design tactile codes with shapes and waveforms. In: Haptics Symposium (HAPTICS), pp. 597–602. IEEE (2014). doi:10.1109/HAPTICS.2014.6775523

  93. Prasad, M., Taele, P., Goldberg, D., Hammond, T.A.: Haptimoto: turn-by-turn haptic route guidance interface for motorcyclists. In: Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems, CHI’14, pp. 3597–3606. ACM, New York (2014). doi:10.1145/2556288.2557404. http://doi.acm.org/10.1145/2556288.2557404

  94. Prasad, M., Taele, P., Olubeko, A., Hammond, T.: Haptigo: a navigational tap on the shoulder. In: Haptics Symposium (HAPTICS), pp. 339–345. IEEE (2014). doi:10.1109/HAPTICS.2014.6775478

  95. Rajan, P., Hammond, T.: From paper to machine: extracting strokes from images for use in sketch recognition. In: Proceedings of the Fifth Eurographics Conference on Sketch-Based Interfaces and Modeling, SBM’08, pp. 41–48. Eurographics Association, Aire-la-Ville (2008). doi:10.2312/SBM/SBM08/041-048. http://dx.doi.org/10.2312/SBM/SBM08/041-048

  96. Rajan, P., Hammond, T.: Applying online sketch recognition algorithms to a scanned-in sketch. In: Proceedings of the Workshop on Sketch Recognition at the 14th International Conference of Intelligent User Interfaces Posters (IUI). ACM, Sanibel (2009)

    Google Scholar 

  97. Rajan, P., Taele, P., Hammond, T.: Evaluation of paper-pen based sketching interface. In: Proceedings of the 16th International Conference on Distributed Multimedia Systems (DMS), pp. 321–326 (2010)

    Google Scholar 

  98. Rajanna, V., Alamudun, F., Goldberg, D., Hammond, T.: Let me relax: toward automated sedentary state recognition and ubiquitous mental wellness solutions. In: Proceedings of the 5th EAI International Conference on Wireless Mobile Communication and Healthcare, MOBIHEALTH’15, pp. 28–33. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), Brussels (2015). doi:10.4108/eai.14-10-2015.2261900. http://dx.doi.org/10.4108/eai.14-10-2015.2261900

  99. Rajanna, V., Lara-Garduno, R., Behera, D.J., Madanagopal, K., Goldberg, D., Hammond, T.: Step up life: a context aware health assistant. In: Proceedings of the Third ACM SIGSPATIAL International Workshop on the Use of GIS in Public Health, HealthGIS’14, pp. 21–30. ACM, New York (2014). doi:10.1145/2676629.2676636. http://doi.acm.org/10.1145/2676629.2676636

  100. Rajanna, V., Vo, P., Barth, J., Mjelde, M., Grey, T., Oduola, C., Hammond, T.: Kinohaptics: an automated, wearable, haptic assisted, physio-therapeutic system for post-surgery rehabilitation and self-care. J. Med. Syst. 40(3), 1–12 (2015). doi:10.1007/s10916-015-0391-3. http://dx.doi.org/10.1007/s10916-015-0391-3

  101. Raymond, D., Liew, J., Hammond, T.A.: A vision for education: transforming how formal systems are taught within mass lectures by using pen technology to create a personalized learning environment. In: Hammond, T., Valentine, S., Adler, A., Payton, M. (eds.) The Impact of Pen and Touch Technology on Education, pp. 355–363. Springer, Cham (2015). doi:10.1007/978-3-319-15594-4

    Chapter  Google Scholar 

  102. Sezgin, T.M., Stahovich, T., Davis, R.: Sketch based interfaces: early processing for sketch understanding. In: ACM SIGGRAPH 2006 Courses. ACM (2006)

    Google Scholar 

  103. Shahzad, N., Paulson, B., Hammond, T.: Urdu qaeda: recognition system for isolated urdu characters. In: Proceedings of the Workshop on Sketch Recognition at the 14th International Conference of Intelligent User Interfaces (IUI). ACM, Sanibel (2009)

    Google Scholar 

  104. Taele, P., Barreto, L., Hammond, T.: Maestoso: an intelligent educational sketching tool for learning music theory. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, AAAI’15, pp. 3999–4005. AAAI Press (2015). http://dl.acm.org/citation.cfm?id=2888116.2888271

  105. Taele, P., Dixon, D., Hammond, T.: Telling the user, “no”: Sketch recognition for improving sketch technique. In: SkCHI: Designing Sketch Recognition Interfaces, A CHI 2010 Workshop. Atlanta (2010)

    Google Scholar 

  106. Taele, P., Hammond, T.: Chinese characters as sketch diagrams using a geometric-based approach. In: Proceedings of the 2008 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) Workshop on Sketch Tools for Diagramming, pp. 74–82. VL/HCC, Herrsching am Ammersee (2008)

    Google Scholar 

  107. Taele, P., Hammond, T.: Using a geometric-based sketch recognition approach to sketch chinese radicals. In: Proceedings of the 23rd National Conference on Artificial Intelligence, AAAI’08, vol. 3, pp. 1832–1833. AAAI Press (2008). http://dl.acm.org/citation.cfm?id=1620270.1620376

  108. Taele, P., Hammond, T.: Hashigo: a next-generation sketch interactive system for Japanese kanji. In: Proceedings of the Twenty-First Innovative Applications of Artificial Intelligence Conference (IAAI), pp. 153–158. AAAI, Pasadena (2009)

    Google Scholar 

  109. Taele, P., Hammond, T.: Lamps: a sketch recognition-based teaching tool for mandarin phonetic symbols I. J. Vis. Lang. Comput. 21(2), 109–120 (2010). doi:10.1016/j.jvlc.2009.12.004. http://dx.doi.org/10.1016/j.jvlc.2009.12.004

  110. Taele, P., Hammond, T.: Initial approaches for extending sketch recognition to beyond-surface environments. In: CHI’12 Extended Abstracts on Human Factors in Computing Systems, CHI EA’12, pp. 2039–2044. ACM, New York (2012). doi:10.1145/2212776.2223749. http://doi.acm.org/10.1145/2212776.2223749

  111. Taele, P., Hammond, T.: Adapting surface sketch recognition techniques for surfaceless sketches. In: Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI’13, pp. 3243–3244. AAAI Press (2013). http://dl.acm.org/citation.cfm?id=2540128.2540630

  112. Taele, P., Hammond, T.: Developing sketch recognition and interaction techniques for intelligent surfaceless sketching user interfaces. In: Proceedings of the Companion Publication of the 19th International Conference on Intelligent User Interfaces, IUI Companion’14, pp. 53–56. ACM, New York (2014). doi:10.1145/2559184.2559185. http://doi.acm.org/10.1145/2559184.2559185

  113. Taele, P., Hammond, T.: Enhancing instruction of written east asian languages with sketch recognition-based intelligent language workbook interfaces. In: Hammond, T., Valentine, S., Adler, A., Payton, M. (eds.) The Impact of Pen and Touch Technology on Education, 1st edn., pp. 119–126. Springer Publishing Company, Incorporated (2015)

    Chapter  Google Scholar 

  114. Taele, P., Hammond, T.: Invisishapes: a recognition system for sketched 3D primitives in continuous interaction spaces. In: Proceedings of the 2015 International Symposium on Smart Graphics. Chengdu (2015)

    Google Scholar 

  115. Taele, P., Hammond, T.: An intelligent sketch-based educational interface for learning complex written east asian phonetic symbols. In: Hammond, T., Valentine, S., Adler, A. (eds.) Revolutionizing Education with Digital Ink: The Impact of Pen and Touch Technology on Education, pp. 129–140. Springer, Cham (2016). doi:10.1007/978-3-319-31193-7_9. http://dx.doi.org/10.1007/978-3-319-31193-7_9

  116. Taele, P., Hammond, T.A.: A geometric-based sketch recognition approach for handwritten mandarin phonetic symbols I. In: 2008 International Workshop on Visual Languages and Computing (VLC) at the 14th International Conference on distributed Multimedia Systems (DMS). Knowledge Systems Instistute, Boston (2008)

    Google Scholar 

  117. Taele, P., Peschel, J., Hammond, T.: A sketch interactive approach to computer-assisted biology instruction. In: Proceedings of the Workshop on Sketch Recognition at the 14th International Conference of Intelligent User Interfaces Posters (IUI). ACM, Sanibel (2009)

    Google Scholar 

  118. Valentine, S., Field, M., Smith, A., Hammond, T.: A shape comparison technique for use in sketch-based tutoring systems. In: Proceedings of the 2011 Intelligent User Interfaces Workshop on Sketch Recognition. ASEE Conferences, Palo Alto (2011)

    Google Scholar 

  119. Valentine, S., Lara-Garduno, R., Linsey, J., Hammond, T.: Mechanix: A sketch-based tutoring system that automatically corrects hand-sketched statics homework. In: Hammond, T., Valentine, S., Adler, A., Payton, M. (eds.) The Impact of Pen and Touch Technology on Education, 1st edn., pp. 91–105. Springer Publishing Company, Incorporated (2015)

    Chapter  Google Scholar 

  120. Valentine, S., Leyva-McMurtry, A., Borgos-Rodriguez, K., Hammond, T.: The digital sash: a sketch-based badge system in a social network for children. In: Hammond, T., Valentine, S., Adler, A. (eds.) Revolutionizing Education with Digital Ink: The Impact of Pen and Touch Technology on Education, pp. 179–189. Springer, Cham (2016). doi:10.1007/978-3-319-31193-7_12. http://dx.doi.org/10.1007/978-3-319-31193-7_12

  121. Valentine, S., Vides, F., Lucchese, G., Turner, D., Kim, H.h., Li, W., Linsey, J., Hammond, T.: Mechanix: a sketch-based tutoring system for statics courses. In: Proceedings of the Twenty-Fourth Innovative Applications of Artificial Intelligence Conference (IAAI), pp. 2253–2260. AAAI, Toronto (2012)

    Google Scholar 

  122. Valentine, S., Vides, F., Lucchese, G., Turner, D., Kim, H.H., Li, W., Linsey, J., Hammond, T.: Mechanix: a sketch-based tutoring and grading system for free-body diagrams. AI Mag. 34(1), 55–66 (2013)

    Article  Google Scholar 

  123. Vangavolu, S., Wood, H., Newman, J., Polsley, S., Hammond, T.: Frontier: a directed graph system for web navigation. In: Proceedings of the 11th Biannual Conference on Italian SIGCHI Chapter, CHItaly 2015, pp. 82–85. ACM, New York (2015). doi:10.1145/2808435.2808465. http://doi.acm.org/10.1145/2808435.2808465

  124. Veselova, O., Davis, R.: Perceptually based learning of shape descriptions for sketch recognition. ACM, New York (2006)

    Book  Google Scholar 

  125. Vides, F., Taele, P., Kim, H., Ho, J., Hammond, T.: Intelligent feedback for kids using sketch recognition. In: ACM SIGCHI 2012 Conference on Human Factors in Computing Systems Workshop on Educational Interfaces, Software, and Technology. ACM (2012)

    Google Scholar 

  126. Williford, B., Taele, P., Nelligan, T., Li, W., Linsey, J., Hammond, T.: Persketchtivity: an intelligent pen-based educational application for design sketching instruction. In: Hammond, T., Valentine, S., Adler, A. (eds.) Revolutionizing Education with Digital Ink: The Impact of Pen and Touch Technology on Education, pp. 115–127. Springer, Cham (2016). doi:10.1007/978-3-319-31193-7_8. http://dx.doi.org/10.1007/978-3-319-31193-7_8

  127. Wolin, A., Eoff, B., Hammond, T.: Shortstraw: a simple and effective corner finder for polylines. In: Proceedings of the Fifth Eurographics Conference on Sketch-Based Interfaces and Modeling, SBM’08, pp. 33–40. Eurographics Association, Aire-la-Ville (2008). doi:10.2312/SBM/SBM08/033-040. http://dx.doi.org/10.2312/SBM/SBM08/033-040

  128. Wolin, A., Eoff, B., Hammond, T.: Search your mobile sketch: improving the ratio of interaction to information on mobile devices. In: Proceedings of the Workshop on Sketch Recognition at the 14th International Conference of Intelligent User Interfaces (IUI). ACM, Sanibel (2009)

    Google Scholar 

  129. Wolin, A., Field, M., Hammond, T.: Combining corners from multiple segmenters. In: Proceedings of the Eighth Eurographics Symposium on Sketch-Based Interfaces and Modeling, SBIM’11, pp. 117–124. ACM, New York (2011). doi:10.1145/2021164.2021185. http://doi.acm.org/10.1145/2021164.2021185

  130. Wolin, A., Paulson, B., Hammond, T.: Eliminating false positives during corner finding by merging similar segments. In: Proceedings of the 23rd National Conference on Artificial Intelligence, AAAI’08, vol. 3, pp. 1836–1837. AAAI Press (2008). http://dl.acm.org/citation.cfm?id=1620270.1620378

  131. Wolin, A., Paulson, B., Hammond, T.: Sort, merge, repeat: an algorithm for effectively finding corners in hand-sketched strokes. In: Proceedings of the 6th Eurographics Symposium on Sketch-Based Interfaces and Modeling, SBIM’09, pp. 93–99. ACM, New York (2009). doi:10.1145/1572741.1572758. http://doi.acm.org/10.1145/1572741.1572758

  132. Zhu, Y., Johnston, J., Hammond, T.: Ringedit: A control point based editing approach in sketch recognition systems. In: Proceedings of the Workshop on Sketch Recognition at the 14th International Conference of Intelligent User Interfaces Posters (IUI). ACM, Sanibel (2009)

    Google Scholar 

Download references

Acknowledgements

None of this research is possible without the support of funding agencies that have supported this work, including NSF, DARPA, NIH, TDADS, ONR, Google, Microsoft, Rockwell Collins, TEES, and TAMU as well as the fantastic work of the student members of the Sketch Recognition Lab. Special thanks to Dr. Anna Stepanova, and Dr. Jan Hammond for their help in this transcription as well as Lisa Manokofsky for her help in providing the videos for translation.

Author information

Authors and Affiliations

  1. Sketch Recognition Lab, Department of Computer Science & Engineering, Texas A&M University, College Station, TX, 77843, USA

    Tracy Hammond

Authors
  1. Tracy Hammond
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tracy Hammond .

Editor information

Editors and Affiliations

  1. Sketch Recognition Lab, Department of Computer Science & Engineering, Texas A&M University, College Station, Texas, USA

    Tracy Hammond

  2. Information and Knowledge Technologies, Raytheon BBN Technologies, Columbia, Massachusetts, USA

    Aaron Adler

  3. Microsoft, Redmond, Washington, USA

    Manoj Prasad

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Hammond, T. (2017). Learning Through the Lens of Sketch. In: Hammond, T., Adler, A., Prasad, M. (eds) Frontiers in Pen and Touch. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-319-64239-0_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-64239-0_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-64238-3

  • Online ISBN: 978-3-319-64239-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation