PaperUI is a human-information interface concept that advocates using paper as displays and using mobile devices, such as camera phones or camera pens, as traditional computer-mice. When emphasizing technical efforts, some researchers like to refer the PaperUI related underlying work as interactive paper system. We prefer the term PaperUI for emphasizing the final goal, narrowing the discussion focus, and avoiding terminology confusion between interactive paper system and interactive paper computer [40]. PaperUI combines the merits of paper and the mobile devices, in that users can comfortably read and flexibly arrange document content on paper, and access digital functions related to the document via the mobile computing devices. This concept aims at novel interface technology to seamlessly bridge the gap between paper and computers for better user experience in handling documents. Compared with traditional laptops and tablet PCs, devices involved in the PaperUI concept are more light-weight, compact, energy efficient, and widely adopted. Therefore, we believe this interface vision can make computation more convenient to access for general public.


paper document camera phone human-computer interface humancomputer interaction document recognition augmented paper vision-based paper interface 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
    IDAUTOMATION, About Barcode Accuracy and Misreads,
  3. 3.
    Maxemchuk, N.F., Low, S.: Marking text documents. In: Proc. IEEE Int. Conf. Image Processing 1997 (ICIP 1997), Santa Barbara, CA, vol. 3, pp. 13–16 (October 1997)Google Scholar
  4. 4.
    Fuji Xerox, Paper Fingerprint Recognition Technology (August 3, 2011),
  5. 5.
    Hartung, F., Kutter, M.: Multimedia watermarking techniques. Proceedings of the IEEE 87, 1079–1107 (1999)CrossRefGoogle Scholar
  6. 6.
    Holley, R.: How Good Can It Get? D-Lib Magazine (March/April 2009),
  7. 7.
    Jacobi, J.L.: ABBYY FineReader 10 Professional Edition review (August 3, 2011),
  8. 8.
    Fox, M.: Alan Haberman, Who Ushered In the Bar Code, Dies at 81. The New York Times (June 15, 2011) Google Scholar
  9. 9.
    Resource Conservation Alliance, Focus On Paper Consumption,
  10. 10.
    Liu, X., Doermann, D.: Mobile Retriever: access to digital documents from their physical source. Int. J. Doc. Anal. Recognit. 11(1), 19–27 (2008)CrossRefGoogle Scholar
  11. 11.
    Henze, N., Boll, S.: Snap and share your photobooks. In: Proceedings of ACM Multimedia 2008, pp. 409–418 (2008)Google Scholar
  12. 12.
    Hare, J., Lewis, P., Gordon, L., Hart, G.: MapSnapper: Engineering an Efficient Algorithm for Matching Images of Maps from Mobile Phones. In: Proceedings of Multimedia Content Access: Algorithms and Systems II (2008)Google Scholar
  13. 13.
    Lowe, D.G.: Distinctive Image Features from Scale-Invariant Keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  14. 14.
  15. 15.
    Asaravala, A.: Camera Phones Help Buyers Beware, Tech Biz: IT,
  16. 16.
    Hanlon, M.: ScanBuy - barcode software on your camera phone creates the Physical World Hyperlink, Mobile Technology,
  17. 17.
    Byron Acohido, New ’2D barcodes’ puts info at the tip of your camera phone, USA TODAY (May 20, 2009)Google Scholar
  18. 18.
    Rohs, M.: Real-World Interaction with Camera Phones. In: Murakami, H., Nakashima, H., Tokuda, H., Yasumura, M. (eds.) UCS 2004. LNCS, vol. 3598, pp. 74–89. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  19. 19.
    Reilly, D., et al.: Marked-Up Maps: Combining Paper Maps and Electronic Information Resources. Personal Ubiquitous Computing 10(4), 215–226 (2006)CrossRefGoogle Scholar
  20. 20.
    Ellison, J.: How To Make The Cell Phone Into A Portable Scanner. News Week Magazine (October 20, 2007)Google Scholar
  21. 21.
    Iwamura, M., Tsuji, T., Kise, K.: Memory-based recognition of camera-captured characters. In: Proceedings of the 9th IAPR International Workshop on Document Analysis Systems (DAS 2010), pp. 89–96. ACM, New York (2010)CrossRefGoogle Scholar
  22. 22.
    DigiMarc, Enhanced, Interactive Experiences from Media Using Smartphones (August 3, 2011),
  23. 23.
    Google Goggles, Google Goggles in action (August 3, 2011),,
  24. 24.
    Kooaba, Paperboy: Deliver digital extras for print (August 3, 2011),
  25. 25.
    Hull, J.J., Erol, B., Graham, J., Ke, Q., Kishi, H., Moraleda, J., Van Olst, D.G.: Paper-based augmented reality. In: Proc. 17th Int. Conf. Artificial Reality and TelexistenceGoogle Scholar
  26. 26.
    Erol, B., Antunez, E., Hull, J.J.: HOTPAPER: multimedia interaction with paper using mobile phones. In: Proceedings of ACM Multimedia 2008, pp. 399–408 (2008)Google Scholar
  27. 27.
    Graham, J., Hull, J.: Icandy: A tangible user interface for itunes. In: Proc. CHI 2008: Extended Abstracts on Human Factors in Computing Systems, Florence, Italy (2008)Google Scholar
  28. 28.
    Toto, S.: SnapTell: Instant Product Lookup From The iPhone. You Want This. Tech Crunch (November 19, 2008),
  29. 29.
    Hecht, D.L.: Embedded Data Glyph Technology for Hardcopy Digital Documents. In: SPIE -Color Hard Copy and Graphics Arts III, vol. 2171, pp. 341–352 (February 1994)Google Scholar
  30. 30.
    Nakai, T., Kise, K., Iwamura, M.: Camera-Based Document Image Retrieval as Voting for Partial Signatures of Projective Invariants. In: Proc. 8th International Conference on Document Analysis and Recognition (ICDAR 2005), pp. 379–383 (September 2005)Google Scholar
  31. 31.
    Nakai, T., Kise, K., Iwamura, M.: Real-Time Retrieval for Images of Documents in Various Languages using a Web Camera. In: Proceedings of the 10th International Conference on Document Analysis and Recognition (ICDAR 2009), pp. 146–150 (July 2009)Google Scholar
  32. 32.
    Liu, Q., Liao, C., Wilcox, L., Dunnigan, A., Liew, B.: Embedded Media Markers: Marks on Paper that Signify Associated Media. In: Proceedings of ACM IUI 2010, pp. 149–158 (2010)Google Scholar
  33. 33.
    Liu, Q., Yano, H., Kimber, D., Liao, C., Wilcox, L.: High Accuracy and Language Independent Document Retrieval With A Fast Invariant Transform. In: Proceedings of IEEE ICME 2009, pp. 386–389 (2009)Google Scholar
  34. 34.
    Liao, C., Liu, Q., Liew, B., Wilcox, L.: Pacer: fine-grained interactive paper via camera-touch hybrid gestures on a cell phone. In: Proceedings of the 28th International Conference on Human Factors in Computing Systems (CHI 2010). ACM, New York (2010)Google Scholar
  35. 35.
    Rohs, M., Oulasvirta, A.: Target Acquisition with Camera Phones when userd as Magic Lenses. In: Proceedings of CHI 2008, pp. 1409–1418 (2008)Google Scholar
  36. 36.
    Liao, C., Tang, H., Liu, Q., Chiu, P., Chen, F.: 2010. FACT: fine-grained cross-media interaction with documents via a portable hybrid paper-laptop interface. In: Proceedings of the International Conference on Multimedia (MM 2010), pp. 361–370. ACM, New York (2010)Google Scholar
  37. 37.
    Liao, C., Liu, Q.: MixPad: Augmenting Interactive Paper with Mice & Keyboards for Cross-media Interaction with Documents. To be published Proc. ACM UBICOMP 2011, Beijing, China (2011)Google Scholar
  38. 38.
    Liu, Q., Liao, C., Wilcox, L., Dunnigan, A.: Embedded media barcode links: optimally blended barcode overlay on paper for linking to associated media. In: International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction (ICMI-MLMI 2010). ACM, New York (2010)Google Scholar
  39. 39.
    The Office of the Future, Business Week (2387): 48–70 (June 30, 1975) Google Scholar
  40. 40.
    Lahey, B., Girouard, A., Burleson, W., Vertegaal, R.: Paperphone: understanding the use of bend gestures in mobile devices with flexible electronic paper displays. In: Proceedings of CHI 2011. ACM Press (2011)Google Scholar
  41. 41.
    Wellner, P.: Interacting with paper on the DigitalDesk. Communications of the ACM 36(7), 87–96 (1993)CrossRefGoogle Scholar
  42. 42.
    Anoto, THE DIGITAL PEN, (August 3, 2011),
  43. 43.
    Mackay, W.E., Fayard, A.-L.: Designing Interactive Paper: Lessons from three Augmented Reality Projects. In: Proc. IWAR 1998, pp. 81–90 (1998)Google Scholar
  44. 44.
    easeMD, Digital Pen for Off-site Use by Physicians (August 3, 2011),
  45. 45.
    Livescribe, Record and Play Back (August 3, 2011),
  46. 46.
    Yeh, R.B., Liao, C., Klemmer, S., Guimbretière, F., Lee, B., Kakaradov, B., Stamberger, J., Paepcke, A.: ButterflyNet: A Mobile Capture and Access System for Field Biology Research. In: Proceedings of CHI 2006, pp. 571–580 (2006)Google Scholar
  47. 47.
    Hu, C., Liu, Q., Liu, X., Liao, C., McEvoy, P.: Poems: A Paper Based Meeting Service Management Tool. In: ICME 2007, February 7 (2007)Google Scholar
  48. 48.
  49. 49.
    Capturx, Automate Any Paperwork,
  50. 50.
    Infomax, Digital Writing Slutions Made Simple,
  51. 51.
    Song, H., Guimbretiere, F., Grossman, T., Fitzmaurice, G.: MouseLight: bimanual interactions on digital paper using a pen and a spatially-aware mobile projector. In: Proceedings of the 28th International Conference on Human Factors in Computing Systems (CHI 2010). ACM, New York (2010)Google Scholar
  52. 52.
    Song, H., Grossman, T., Fitzmaurice, G., Guimbretiere, F., Khan, A., Attar, R., Kurtenbach, G.: PenLight: combining a mobile projector and a digital pen for dynamic visual overlay. In: Proceedings of the 27th International Conference on Human Factors in Computing Systems (CHI 2009), pp. 143–152. ACM, New York (2009)Google Scholar
  53. 53.
    Liao, C., Guimbretière, F., Anderson, R., Linnell, N., Prince, C., Razmov, V.: PaperCP: Exploring the Integration of Physical and Digital Affordances for Active Learning. In: Baranauskas, C., Abascal, J., Barbosa, S.D.J. (eds.) INTERACT 2007, Part II. LNCS, vol. 4663, pp. 15–28. Springer, Heidelberg (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Qiong Liu
    • 1
  • Chunyuan Liao
    • 1
  1. 1.FX Palo Alto LaboratoryPalo AltoU.S.A.

Personalised recommendations