Collaborative Annotation Sharing in Physical and Digital Worlds

  • Jan Grbac
  • Matjaž Kljun
  • Klen Čopič Pucihar
  • Leo Gombač
Conference paper
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 474)


Despite the existence of a plethora of annotating software for digital documents, many users still prefer reading and annotating them physically on paper. While others have proposed the idea of merging these two worlds, none of them fits all the design requirements identified in this paper (working in real-time, use readily available hardware, augment physical annotations with digital content, support annotation sharing and collaborative learning). In this paper we present the implemented prototype and a focus group study aimed at understanding studying habits and how the system would fit in these. The focus group revealed that paper material is often discarded or archived and annotations lost, web resources are not saved and fade with time, and that the prototype proposed fits in their studying habits and does not introduce any privacy concerns – be it ones related to the prototype’s camera (used in public setting) or ones related to annotations sharing.


Document annotation Change detection Computer vision Physical-digital interaction 


  1. 1.
    Sellen, A.J., Harper, R.H.R.: The Myth of the Paperless Office. MIT Press, Cambridge (2001)Google Scholar
  2. 2.
    Mangen, A., Walgermo, B.R., Brønnick, K.: Reading linear texts on paper versus computer screen: effects on reading comprehension. Int. J. Educ. Res. 58, 61–68 (2013)CrossRefGoogle Scholar
  3. 3.
    Marshall, C.C., Bly, S.: Turning the page on navigation. In: Proceedings of the 5th ACM/IEEE-CS Joint Conference Digital Library (JCDL 2005) (2005)Google Scholar
  4. 4.
    Mueller, P.A., Oppenheimer, D.M.: The pen is mightier than the keyboard: advantages of longhand over laptop note taking. Psychol. Sci. (2014)Google Scholar
  5. 5.
    Lawrie, D., Rus, D.: A self-organized file cabinet. In: Proceedings of the Eighth International Conference on Information and Knowledge Management - CIKM 1999, pp. 499–506. ACM Press, New York (1999)Google Scholar
  6. 6.
    Rao, R., Card, S.K., Johnson, W., Klotz, L., Trigg, R.H.: Protofoil: storing and finding the information worker’s paper documents in an electronic file cabinet. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI 1994, pp. 180–185. ACM Press, New York (1994)Google Scholar
  7. 7.
    Newman, W., Wellner, P.: A desk supporting computer-based interaction with paper documents. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI 1992, pp. 587–592. ACM Press, New York (1992)Google Scholar
  8. 8.
    Voida, S., Mynatt, E.D., MacIntyre, B., Corso, G.M.: Integrating virtual and physical context to support knowledge workers. IEEE Pervasive Comput. 1, 73–79 (2002)CrossRefGoogle Scholar
  9. 9.
    Pederson, T.: Magic touch: a simple object location tracking system enabling the development of physical-virtual artefacts in office environments. Pers. Ubiquit. Comput. 5, 54–57 (2001)CrossRefGoogle Scholar
  10. 10.
    Smith, J., Long, J., Lung, T., Anwar, M.M., Subramanian, S.: PaperSpace: a system for managing digital and paper documents. In: Proceedings of the SIGCHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA 2006, p. 1343. ACM Press, New York (2006)Google Scholar
  11. 11.
    Mackay, W.E.: The missing link: integrating paper and electronic documents. In: Proceedings of the 15th French-Speaking Conference Human-Computer Interaction on 15eme Conference Francophone sur l’Interaction Homme-Machine, pp. 1–8 (2003)Google Scholar
  12. 12.
    Fallman, D.: The BubbleFish: digital documents available on hand. In: INTERACT 2001: Proceedings of IFIP TC13 Conference on Human-Computer Interaction, Waseda University Conference Center, Shinjuku, Tokyo, Japan (2001)Google Scholar
  13. 13.
    Liao, C., Liu, Q., Liew, B., Wilcox, L.: Pacer: fine-grained interactive paper via camera-touch hybrid gestures on a cell phone. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2441–2450 (2010)Google Scholar
  14. 14.
    Hardy, J.: Experiences: a year in the life of an interactive desk. In: Proceedings of the Designing Interactive Systems Conference on - DIS 2012, p. 679. ACM Press, New York (2012)Google Scholar
  15. 15.
    Hincapié-Ramos, J., Roscher, S.: tPad: designing transparent-display mobile interactions. In: Proceeding of the DIS 2014, pp. 161–170 (2014)Google Scholar
  16. 16.
    Hincapié-Ramos, J.D., Roscher, S., Büschel, W., Kister, U., Dachselt, R., Irani, P.: cAR: contact augmented reality with transparent-display mobile devices. In: Proceedings of the International Symposium on Pervasive Displays, pp. 80:80–80:85 (2014)Google Scholar
  17. 17.
    Kunze, K., Tanaka, K., Iwamura, M., Kise, K.: Annotate me - supporting active reading using real-time document image retrieval on mobile devices. In: UbiComp 2013 Adjunct - Adjunct Publication of the 2013 ACM Conference on Ubiquitous Computing, pp. 231–234 (2013)Google Scholar
  18. 18.
    Adler, A., Gujar, A., Harrison, B.L., O’Hara, K., Sellen, A.: A diary study of work-related reading: design implications for digital reading devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems – CHI, pp. 241–248 (1998)Google Scholar
  19. 19.
    West, D., Quigley, A., Kay, J.: MEMENTO: a digital-physical scrapbook for memory sharing. Pers. Ubiquit. Comput. 11, 313–328 (2007)CrossRefGoogle Scholar
  20. 20.
    Johnson, W., Jellinek, H., Klotz, L., Rao, R., Card, S.K.: Bridging the paper and electronic worlds: the paper user interface. In: Proceedings of the INTERACT 1993 and CHI 1993 Conference on Human Factors in Computing Systems, pp. 507–512 (1993)Google Scholar
  21. 21.
    Guimbretière, F.: Paper augmented digital documents. In: Proceedings of the 16th Annual ACM Symposium User Interface Software Technology UIST 2004, pp. 51–60 (2003)Google Scholar
  22. 22.
    Pietrzak, T., Malacria, S., Lecolinet, É.: S-Notebook. In: Proceedings of the International Workshop Conference Advanced Visual Interfaces - AVI 2012 (2012)Google Scholar
  23. 23.
    Pearson, J., Buchanan, G., Thimbleby, H.: The reading desk: applying physical interactions to digital documents. In: Jones, M.J., Palanque, P. (eds.) Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems - CHI 2011, p. 3199 (2011)Google Scholar
  24. 24.
    Applegate, R.: The library is for studying. J. Acad. Librariansh. 35, 341–346 (2009)CrossRefGoogle Scholar
  25. 25.
    Harrop, D., Turpin, B.: A study exploring learners’ informal learning space behaviors, attitudes, and preferences. New Rev. Acad. Librariansh. 19, 58–77 (2013)CrossRefGoogle Scholar
  26. 26.
    Roesner, F., Kohno, T., Molnar, D.: Security and privacy for augmented reality systems. Commun. ACM 57, 88–96 (2014)CrossRefGoogle Scholar
  27. 27.
    Venkatesh, V., Morris, M.G., Davis, G.B., Davis, F.D.: User acceptance of information technology: toward a unified view. MIS Q. 27, 425–478 (2003)Google Scholar
  28. 28.
    Sakar, A., Ercetin, G.: Effectiveness of hypermedia annotations for foreign language reading. J. Comput. Assist. Learn. 21, 28–38 (2005)CrossRefGoogle Scholar
  29. 29.
    Marshall, C.C., Brush, A.J.B.: Exploring the relationship between personal and public annotations. In: Proceedings of the 2004 Joint ACM/IEEE Conference Digital Libraries 2004, pp. 349–357 (2004)Google Scholar
  30. 30.
    Marshall, C.C., Brush, A.J.B.: From personal to shared annotations. In: CHI 2002 Extended Abstracts on Human Factors in Computing Systems - CHI 2002, p. 812. ACM Press, New York (2002)Google Scholar
  31. 31.
    Koller, D., Klinker, G., Rose, E., Breen, D., Whitaker, R., Tuceryan, M.: Real-time vision-based camera tracking for augmented reality applications. In: Proceedings of the ACM Symposium Virtual Reality Software Technology - VRST 1997, pp. 87–94 (1997)Google Scholar
  32. 32.
    Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision with the OpenCV Library (2008)Google Scholar
  33. 33.
    Jones, W.: Keeping Found Things Found: The Study and Practice of Personal Information Management. Morgan Kaufman, Burlington (2007)Google Scholar
  34. 34.
    Payne, S.J.: Users’ mental models: the very ideas. In: HCI Models, Theories, and Frameworks: Toward a Multidisciplinary Science, pp. 135–156 (2003)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2016

Authors and Affiliations

  • Jan Grbac
    • 1
  • Matjaž Kljun
    • 1
  • Klen Čopič Pucihar
    • 1
  • Leo Gombač
    • 1
  1. 1.FAMNITUniversity of PrimorskaKoperSlovenia

Personalised recommendations