Skip to main content

Advertisement

SpringerLink
Log in

Enhancing education through natural interaction with physical paper

  • Long paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

Pervasive computing environments have permeated current research and practice, unobtrusively augmenting existing environments with digital content. The present work, following a pervasive computing approach, proposes a framework to augment an educational environment, being a typical classroom or any studying environment. In this context, the work presented in this paper investigates unobtrusive interaction and support of active educational or studying activities through appropriate context-sensitive information. To this end, passive visual sensing is employed in order to unobtrusively perceive the current context and users’ actions, thus providing novel ways to implement natural interaction. The suitability of the proposed interaction technologies and overall approach has been demonstrated through three interactive applications integrated in the framework, each one supporting different interaction techniques and addressing different educational activities. Finally, a user experience evaluation of the three test-bed applications has been carried out, aiming to assess the applicability of the approach and the suitability of each of the proposed technologies to the educational tasks in hand.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. FAMINE (FORTH’s AMI Network Environment), has been implemented in the context of the ICS—FORTH’s internal RTD Programme ‘Ambient Intelligence and Smart Environments’.

  2. For school books this is a reasonable requirement, as most are available electronically.

  3. http://wiki.services.openoffice.org/wiki/Dictionaries.

  4. http://images.google.com.

  5. http://www.youtube.com.

References

  1. Weiser, M.: The computer for the 21st century. Sci. Am. 265(3), 94–104 (1991)

    Article  Google Scholar 

  2. Abowd, G.D.: What next, ubicomp? Celebrating an intellectual disappearing act. In: Proceedings of the 2012 ACM Conference on Ubiquitous Computing, pp. 31–40. ACM, New York (2012)

  3. O’Hara, K., Abigail, S.: A comparison of reading paper and on-line documents. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York (1997)

  4. Spencer, C.: Research on learners’ preferences for reading from a printed text or from a computer screen. J Distance Educ/Revue de l’Éducation à Distance 21.1, 33–50 (2006)

    Google Scholar 

  5. de Jong, M.T., Bus, A.G.: How well suited are electronic books to supporting literacy? J. Early Child. Lit. 3(2), 147–164 (2003)

    Article  Google Scholar 

  6. Satyanarayanan, M.: Pervasive computing: vision and challenges. IEEE Pers. Commun. 8(4), 10–17 (2001)

    Article  Google Scholar 

  7. Antona, M., Margetis, G., Ntoa, S., Leonidis, A., Korozi, M., Paparoulis, G., Stephanidis, C.: Ambient Intelligence in the classroom: an augmented school desk. In: Proceedings of the 2010 AHFE International Conference (3rd International Conference on Applied Human Factors and Ergonomics), pp. 17–20. Miami, FL, USA (2010)

  8. Cooperstock, J.R.: The classroom of the future: enhancing education through augmented reality. In: Proceedings of HCI International 2001 Conference on Human-Computer Interaction, pp. 688–692 (2001)

  9. Margetis, G., Leonidis, A., Antona, M., Stephanidis, C.: Towards ambient intelligence in the classroom. In: Universal Access in Human-Computer Interaction. Applications and Services. Springer, Berlin, pp. 577–586 (2011)

  10. Schmalstieg, D., Fuhrmann, V., Hesina, G., Szalav, Z., Encarna, L.M., Gervautz, M., Purgathofer, W.: The studierstube augmented reality project. Presence: Teleoper. Virtual Environ 11, 33–54 (2002)

    Google Scholar 

  11. Kaufmann, H.: Collaborative Augmented Reality in Education. Institute of Software Technology and Interactive Systems, Vienna University of Technology, Vienna (2003)

    Google Scholar 

  12. Shelton, B.E., Hedley, N.R.: Using augmented reality for teaching earth-sun relationships to undergraduate geography students. In: Augmented Reality Toolkit, The First IEEE International Workshop. IEEE (2002)

  13. Juan, M.C., Carrizo, M., Giménez, M., Abad, F.: Using an augmented reality game to find matching pairs, In: Proceedings of the 19th International Conference on Computer Graphics, Visualization and Computer Vision, pp. 59–66 (2011)

  14. Liu, T., Tan, T., Chu, Y.: 2D barcode and augmented reality supported English learning system. In: Proceedings of the Computer and Information Science (ICIS’07), pp. 5–10 (2007)

  15. Karpischek, S., Marforio, C., Godenzi, M., Heuel, S., Michahelles, F.: Mobile augmented reality to identify mountains, In: Adjunct Proceedings of the 3rd European Conference on Ambient Intelligence (AmI-09) (2009)

  16. Wellner, P.: Interacting with paper on the DigitalDesk. Commun. ACM 36(7), 87–96 (1993)

    Article  Google Scholar 

  17. Kobayashi, M., Koike, H.: EnhancedDesk: Integrating paper documents and digital documents. Asia Pacific Computer Human Interaction (APCHI’98). IEEE CS, pp. 57–62 (1998)

  18. Billinghurst, M., Kato, H., Poupyrev, I.: The MagicBook: moving seamlessly between reality and virtuality. IEEE Computer Graphics, pp. 6–8 (2001)

  19. Anoto.: Development Guide for Service Enabled by Anoto Functionality (2002). http://www.anoto.com

  20. Liao, C., Guimbreti, F., Hinckley, K., Hollan, J.: Papiercraft: a gesture-based command system for interactive paper. ACM Trans. Comput. Hum. Interact. 14(4), 27 (2008)

    Article  Google Scholar 

  21. Luff, P., Heath, C., Norrie, M., Signer, B., Herdman, P.: Only touching the surface: creating affinities between digital content and paper. In: Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work (CSCW ‘04), pp. 523–532. ACM, New York (2004)

  22. Forsberg, A.S., LaViola, Jr. J.J., Zeleznik, R.C.: ErgoDesk: A Framework for Two- and Three-Dimensional Interaction at the ActiveDesk. Second International Immersive Projection Technology Workshop, pp. 11–12 (1998)

  23. Jeong, H.T., Lee, D.W., Heo, G.S., Lee, C.H.: Live book: A mixed reality book using a projection system. In: 2012 IEEE International Conference on Consumer Electronics (ICCE), pp. 680–681 (2012)

  24. Grasset, R., Billinghurst, M., Dünser, A., Seichter, H.: The Mixed Reality Book: A New Multimedia Reading Experience. San Jose, CA, USA (2007)

  25. Dymetman, M., Copperman, M.: Intelligent Paper. International Conference on Electronic Publishing, Document Manipulation, and Typography, pp 392–406 (1998)

  26. Smith, D.R., Munro, E.: Educational card games. Phys. Educ. 44(5), 479 (2009)

    Article  Google Scholar 

  27. Chen, P., Kuo, R., Chang, M., Heh, J.-S.: Designing a trading card game as educational reward system to improve students’ learning motivations. Transactions on Edutainment III, pp. 116–128 (2009)

  28. Korozi, M., Leonidis, A., Margetis, G., Koutlemanis, P., Zabulis, X., Antona, M., Stephanidis, C.: Ambient educational mini-games. In: Proceedings of the International Working Conference on Advanced Visual Interfaces, pp. 802–803. ACM, New York (2012)

  29. Antona, M., Leonidis, A., Margetis, G., Korozi, M., Ntoa, S., Stephanidis, C.: A student-centric intelligent classroom. In: Keyson, D.V., Maher, M.L., Streitz, N., Cheok, A. Augusto, J.C. (eds.) Proceedings of the Second international conference on Ambient Intelligence (AmI’11), pp. 248–252. Springer, Berlin (2011)

  30. Leonidis, A., Korozi, M., Margetis, G., Ntoa, S., Papagiannakis, H., Antona, M., Stephanidis, C.: A glimpse into the ambient classroom. Bull. IEEE Tech. Comm. Learn. Technol. 14(4), 3–6 (2012)

    Google Scholar 

  31. Rekimoto, J.: Smartskin an infrastructure for freehand manipulation on interactive surfaces. In: CHI, pp. 113–120 (2002)

  32. Streitz, N., Tandler, P., Muller-Tomfelde, C., Konomi, S.: Roomware towards the next generation of human-computer interaction based on an integrated design of real and virtual worlds (2001)

  33. Wilson, A.: Playanywhere: a compact interactive tabletop projection-vision system. In: UIST, pp. 83–92 (2005)

  34. Han, J.: Low-cost multi-touch sensing through frustrated total internal reflection. In: UIST, pp. 115–118 (2005)

  35. Gross, T., Fetter, M., Liebsch, S.: The cuetable: cooperative and competitive multi-touch interaction on a tabletop. In: CHI, pp. 3465–3470 (2008)

  36. Gaver, W., Bowers, J., Boucher, A., Gellerson, H., Pennington, S., Schmidt, A., Steed, A., Villars, N., Walker, B.: The drift table: designing for ludic engagement. In: CHI, pp. 885–900 (2004)

  37. Microsoft (Microsoft surface). http://www.surface.com

  38. Dietz, P., Leigh, D.: Diamondtouch: a multi-user touch technology. In: UIST, pp. 219–226 (2001)

  39. SMART: Smart table (2008). http://www.smarttech.com/

  40. Pinhanez, C.: Using a steerable projector and a camera to transform surfaces into interactive displays. In: CHI, pp. 369–370 (2001)

  41. Kjeldsen, R., Pinhanez, C., Pingali, G., Hartman, J., Levas, T., Podlaseck, M.: Interacting with steerable projected displays. In: FG (2002)

  42. Jones, B., Sodhi, R., Campbell, R., Garnett, G., Bailey, B.: Build your world and play in it: Interacting with surface particles on complex objects. In: ISMAR, pp. 165–174 (2010)

  43. Goldberg, D., Richardson, C.: Touch-typing with a stylus. In: Proceedings of the INTERACT’93 and CHI’93 Conference on Human Factors in Computing Systems, pp. 80–87. ACM, New York (1993)

  44. Harrison, C., Benko, H., Wilson, A.: Omnitouch wearable multitouch interaction everywhere. In: UIST, pp. 441–450 (2011)

  45. Song, P., Winkler, S., Tedjokusumo, J.: A tangible game interface using projector-camera systems. In: HCI, pp. 956–965 (2007)

  46. Grammenos, D., Michel, D., Zabulis, X., Argyros, A.: Paperview augmenting physical surfaces with location-aware digital information. In: TEI, pp. 57–60 (2011)

  47. Reitmayr, G., Eade, E., Drummond, T.: Localisation and interaction for augmented maps. In: ISMAR, pp. 120–129 (2005)

  48. Zabulis, X., Koutlemanis, P., Grammenos, D.: Augmented multitouch interaction upon a 2-DOF rotating disk. International Symposium on Visual Computing, Rethymno (2012)

    Book  Google Scholar 

  49. Cook, D.J., Das, S.K.: How smart are our environments? An updated look at the state of the art. J. Pervasive Mob. Comput. 3(2), 53–73 (2007)

    Article  Google Scholar 

  50. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  51. Margetis, G., Zabulis, X., Koutlemanis, P., Antona, M., Stephanidis, C.: Augmented interaction with physical books in an Ambient Intelligence learning environment. Int. J. Multimed. Tools Appl. 67(2), 473–495 (2013)

  52. Bookstein, F.L.: Morphometric Tools for Landmark Data. Cambridge University Press, Cambridge (1991)

    Google Scholar 

  53. Gelb, A.: Applied Optimal Estimation. MIT Press, New York (1974)

    Google Scholar 

  54. Smith R., Chang S.: VisualSEEk: A Fully Automated Content-Based Image Query System. ADM Multimedia, pp. 87–89 (1996)

  55. Duda, R., Hart, P.: Use of the Hough transformation to detect lines and curves in pictures. Commun. ACM 15, 11–15 (1972)

    Article  Google Scholar 

  56. Zabulis, X., Koutlemanis, P., Baltzakis, H., Grammenos, D.: Multiview 3D Pose Estimation of a Wand for Human-Computer Interaction. International Symposium on Visual Computing, Las Vegas, Nevada, USA, pp. 104–115 (2011)

  57. Wilson, A.: Using a depth camera as a touch sensor. In: ACM International Conference on Interactive Tabletops and Surfaces, pp. 69–72 (2010)

  58. Margetis, G., Ntelidakis, A., Zabulis, X., Ntoa, S., Koutlemanis, P., & Stephanidis, C.: Augmenting physical books towards education enhancement. In: Workshop on User-Centred Computer Vision, in Conjunction with Workshop on the Applications of Computer Vision (2013)

  59. Papadaki, E., Zabulis, X., Ntoa, S., Margetis, G., Koutlemanis, P., Karamaounas, P., Stephanidis, C.: The Book of Ellie: An Interactive Book for Teaching the Alphabet to Children. IEEE International Conference on Multimedia and Expo Workshops, San Jose, California, USA (2013)

  60. Nielsen, J.: Usability Engineering, pp. 165–206. Morgan Kaufmann, San Francisco (1994)

    Google Scholar 

  61. Nielsen, J.: Usability Engineering, pp. 207–226. Morgan Kaufmann, San Francisco (1994)

    Google Scholar 

  62. Lewis, C.: Using the “thinking-aloud” method in cognitive interface design. IBM TJ Watson Research Center (1982)

  63. Reynolds, T.J., Gutman, J.: Laddering theory, method, analysis, and interpretation. J. Advert. Res. 28(1), 11–31 (1988)

    Google Scholar 

  64. Brush, B.: Ubiquitous computing field studies. In: Krumm, J. (ed.) Ubiquitous computing fundamentals, pp. 161–202. CRC Press, New York (2009)

    Chapter  Google Scholar 

Download references

Acknowledgments

This work was supported by the Foundation for Research and Technology—Hellas, Institute of Computer Science (FORTH—ICS) internal RTD Programme “Ambient Intelligence and Smart Environments.” The authors would also like to thank Antonios Ntelidakis for his contribution in the artwork of Fig. 6.

Author information

Authors and Affiliations

  1. Institute of Computer Science, Foundation for Research and Technology – Hellas (FORTH), 70013, Heraklion, Crete, Greece

    George Margetis, Xenophon Zabulis, Stavroula Ntoa, Panagiotis Koutlemanis, Eleni Papadaki, Margherita Antona & Constantine Stephanidis

  2. Department of Computer Science, University of Crete, Heraklion, Crete, Greece

    Constantine Stephanidis

Authors
  1. George Margetis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xenophon Zabulis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stavroula Ntoa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Panagiotis Koutlemanis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Eleni Papadaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Margherita Antona
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Constantine Stephanidis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Constantine Stephanidis.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Margetis, G., Zabulis, X., Ntoa, S. et al. Enhancing education through natural interaction with physical paper. Univ Access Inf Soc 14, 427–447 (2015). https://doi.org/10.1007/s10209-014-0365-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-014-0365-0

Keywords

Search

Navigation