Resolving display shape dependence issues on tabletops

Abstract

Advances in display technologies are transforming the capabilities—and potential applications—of system interfaces. Previously, the overwhelming majority of systems have utilised rectangular displays; this may soon change with digital devices increasingly designed to be ubiquitous and pervasive, to facilitate frictionless human interaction. At present, software is invariably designed assuming it will be used with a display of a specific shape; however, there is an emerging demand for systems built around interacting with tabletop interfaces to be capable of handling a wide range of potential display shapes. In this paper, the design of software for use on a range of differently shaped tabletop displays is considered, proposing a novel but extensible technique that can be used to minimise the influence of the issues of using different display shapes. Furthermore, we present a study that applies the technique to adapt several software applications to several different display shapes.

References

  1. [1]

    Dietz, P.; Raskar, R.; Booth, S.; van Baar, J.; Wittenburg, K.; Knep, B. Multi-projectors and implicit interaction in persuasive public displays. In: Proceedings of the Working Conference on Advanced Visual Interfaces, 209–217, 2004.

    Google Scholar 

  2. [2]

    Boyd, J. Circular LCD debuts. 2007. Available at https://doi.org/spectrum.ieee.org/computing/hardware/circularlcd-debuts.

    Google Scholar 

  3. [3]

    Finney, M. D.; Oliver, M. W.; Pierce, P. M.; Sutherland, T. Communication device. U.S. Patent USD600228. 2009.

    Google Scholar 

  4. [4]

    Battersby, S. J. Non rectangular display device. U.S. Patent 7,253,865. 2007.

    Google Scholar 

  5. [5]

    Hansen, T. E.; Hourcade, J. P.; Virbel, M.; Patali, S.; Serra, T. PyMT: A post-WIMP multi-touch user interface toolkit. In: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, 17–24, 2009.

    Google Scholar 

  6. [6]

    Shen, C.; Vernier, F. D.; Forlines, C.; Ringel, M. DiamondSpin: An extensible toolkit for around-thetable interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 167–174, 2004.

    Google Scholar 

  7. [7]

    Weiser, M. The computer for the 21st century. Mobile Computing and Communications Review Vol. 3, No. 3, 3–11, 1999.

    Article  Google Scholar 

  8. [8]

    Greenfield, A. Everyware: The Dawning Age of Ubiquitous Computing. Peachpit Press, 2006.

    Google Scholar 

  9. [9]

    Mostafa, M.; Crick, T.; Calderon, A. C.; Oatley, G. Incorporating emotion and personality-based analysis in user-centered modelling. In: Research and Development in Intelligent Systems XXXIII. Bramer, M.; Petridis, M. Eds. Springer Cham, 383–389, 2016.

    Google Scholar 

  10. [10]

    Smith, S. P.; Burd, E.; Rick, J. Developing, evaluating and deploying multi-touch systems. International Journal of Human-Computer Studies Vol. 70, No. 10, 653–656, 2012.

    Article  Google Scholar 

  11. [11]

    Vernier, F.; Lesh, N.; Shen, C. Visualization techniques for circular tabletop interfaces. In: Proceedings of the Working Conference on Advanced Visual Interfaces, 257–265, 2002.

    Google Scholar 

  12. [12]

    Kitchenham, B. Procedures for performing systematic reviews. Technical Report TR/SE-0401. Keele University, 2004.

    Google Scholar 

  13. [13]

    Serrano, M.; Roudaut, A.; Irani, P. Investigating text legibility on non-rectangular displays. In: Proceedings of the CHI Conference on Human Factors in Computing Systems, 498–508, 2016.

    Google Scholar 

  14. [14]

    Serrano, M.; Roudaut, A.; Irani, P. Visual composition of graphical elements on non-rectangular displays. In: Proceedings of the CHI Conference on Human Factors in Computing Systems, 4405–4416, 2017.

    Google Scholar 

  15. [15]

    Meskens, J.; Vermeulen, J.; Luyten, K.; Coninx, K. Gummy for multi-platform user interface designs: Shape me, multiply me, fix me, use me. In: Proceedings of the Working Conference on Advanced Visual Interfaces, 233–240, 2008.

    Google Scholar 

  16. [16]

    Gajos, K.; Weld, D. S. SUPPLE: Automatically generating user interfaces. In: Proceedings of the 9th International Conference on Intelligent User Interfaces, 93–100, 2004.

    Google Scholar 

  17. [17]

    Waldner, M.; Grasset, R.; Steinberger, M.; Schmalstieg, D. Display-adaptive window management for irregular surfaces. In: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, 222–231, 2011.

    Google Scholar 

  18. [18]

    Constantine, L. L.; Lockwood, L. A. D. Software for use: A practical guide to the models and methods of usage-centered design. ACM SIGCHI Bulletin Vol. 32, No. 1, 111–114, 1999.

    Google Scholar 

  19. [19]

    McNaughton, J.; Crick, T.; Hatch, A. Determining device position through minimal user input. Humancentric Computing and Information Sciences Vol. 7, No. 1, 37, 2017.

    Article  Google Scholar 

  20. [20]

    Milliron, T.; Jensen, R. J.; Barzel, R.; Finkelstein, A. A framework for geometric warps and deformations. ACM Transactions on Graphics Vol. 21, No. 1, 20–51, 2002.

    Article  Google Scholar 

  21. [21]

    Shen, C.; Ryall, K.; Forlines, C.; Esenther, A.; Vernier, F. D.; Everitt, K.; Wu, M.; Wigdor, D.; Morris, M. R.; Hancock, M.; Tse, E. Informing the design of direct-touch tabletops. IEEE Computer Graphics and Applications Vol. 26, No. 5, 36–46, 2006.

    Article  Google Scholar 

  22. [22]

    Scott, S. D.; Sheelagh, M.; Carpendale, T.; Inkpen, K. M. Territoriality in collaborative tabletop workspaces. In: Proceedings of the ACM Conference on Computer- Supported Cooperative Work, 294–303, 2004.

    Google Scholar 

  23. [23]

    Smith, S. P.; Burd, E. L.; Ma, L.; AlAgha, I.; Hatch, A. Relative and absolute mappings for rotating remote 3D objects on multi-touch tabletops. In: Proceedings of the 25th BCS Conference on Human-Computer Interaction, 299–308, 2011.

    Google Scholar 

  24. [24]

    Schöning, J.; Brandl, P.; Daiber, F.; Echtler, F.; Hilliges, O.; Hook, J.; Löchtefeld, M.; Motamedi, N.; Muller, L.; Olivier, P.; Roth, T.; von Zadow, U. Multitouch surfaces: A technical guide. Technical Report TUM-I0833. University of Munich, 2008.

    Google Scholar 

  25. [25]

    Aggarwal, A.; Suri, S. Fast algorithms for computing the largest empty rectangle. In: Proceedings of the 3rd Annual Symposium on Computational Geometry, 278–290, 1987.

    Google Scholar 

  26. [26]

    Naamad, A.; Lee, D. T. On the maximum empty rectangle problem. Discrete Applied Mathematics Vol. 8, No. 3, 267–277, 1984.

    MathSciNet  MATH  Article  Google Scholar 

  27. [27]

    Toussaint, G. T. Computing largest empty circles with location constraints. International Journal of Computer & Information Sciences Vol. 12, No. 5, 347–358, 1983.

    MathSciNet  MATH  Article  Google Scholar 

  28. [28]

    Cotting, D.; Gross, M. Interactive environment-aware display bubbles. In: Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology, 245–254, 2006.

    Google Scholar 

  29. [29]

    Raskar, R.; van Baar, J.; Beardsley, P.; Willwacher, T.; Rao, S.; Forlines, C. iLamps: Geometrically aware and self-configuring projectors. ACM Transactions on Graphics Vol. 22, No. 3, 809–818, 2003.

    Article  Google Scholar 

  30. [30]

    Van Dam, A. User interfaces: Disappearing, dissolving, and evolving. Communications of the ACM Vol. 44, No. 3, 50–52, 2001.

    Article  Google Scholar 

  31. [31]

    McNaughton, J.; Crick, T.; Joyce-Gibbons, A.; Beauchamp, G.; Young, N.; Tan, E. Facilitating collaborative learning between two primary schools using large multi-touch devices. Journal of Computers in Education Vol. 4, No. 3, 307–320, 2017.

    Article  Google Scholar 

  32. [32]

    AlAgha, I.; Hatch, A.; Ma, L.; Burd, E. Towards a teacher-centric approach for multi-touch surfaces in classrooms. In: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, 187–196, 2010.

    Google Scholar 

  33. [33]

    Kaltenbrunner, M.; Bencina, R. reacTIVision: A computer-vision framework for table-based tangible interaction. In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction, 69–74, 2007.

    Google Scholar 

  34. [34]

    Higgins, S.; Mercier, E.; Burd, L.; Joyce-Gibbons, A. Multi-touch tables and collaborative learning. British Journal of Educational Technology Vol. 43, No. 6, 1041–1054, 2012.

    Article  Google Scholar 

  35. [35]

    Jung, Y.; Kim, S.; Choi, B. Consumer valuation of the wearables: The case of smartwatches. Computers in Human Behavior Vol. 63, 899–905, 2016.

    Article  Google Scholar 

  36. [36]

    Ngo, D. C. L.; Teo, L. S.; Byrne, J. G. Formalising guidelines for the design of screen layouts. Displays Vol. 21, No. 1, 3–15, 2000.

    Article  Google Scholar 

  37. [37]

    Joyce-Gibbons, A.; McNaughton, J.; Tan, E.; Young, N.; Beauchamp, G.; Crick, T. SynergyNet into schools: Facilitating remote inter-group collaborative learning using multi-touch tables. In: Proceedings of the 12th International Conference on Computer Support Collaborative Learning, 2017.

    Google Scholar 

Download references

Acknowledgements

This work was partially funded under the UK’s EPSRC/ERSC Teaching and Learning Research Programme (TLRP) SynergyNet project (RES-139-25-0400). The authors would also like to thank Professor Liz Burd and Dr. Andrew Hatch supervising the primary author’s master degree from which this work originally stems. The authors would also like to thank the members of the Durham University Technology Enhanced Learning Special Interest Group for supporting the redrafting of this manuscript. Source code for the technique’s implementation is available at https://doi.org/github.com/synergynet/synergynet2.1.

Author information

Affiliations

Authors

Corresponding author

Correspondence to James McNaughton.

Additional information

James McNaughton is a researcher at Durham University, UK, whose research interests include HCI, natural user interfaces, and augmented reality. His current work involves investigating the use of emerging interaction technologies in classroom environments.

Tom Crick is a professor of digital education & policy at Swansea University, UK. His research interests are interdisciplinary, including data science, intelligent systems, digital public services, software sustainability, and computer science education.

Shamus Smith is a senior lecturer in computer science at the University of Newcastle, Australia. His current research interests include touch-based technologies, mobile technology for augmented reality, and the reuse of gaming technology.

Rights and permissions

Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (https://doi.org/creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Other papers from this open access journal are available free of charge from https://doi.org/www.springer.com/journal/41095. To submit a manuscript, please go to https://doi.org/www.editorialmanager.com/cvmj.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

McNaughton, J., Crick, T. & Smith, S. Resolving display shape dependence issues on tabletops. Comp. Visual Media 4, 349–365 (2018). https://doi.org/10.1007/s41095-018-0124-x

Download citation

Keywords

  • visual content management
  • irregular displays
  • screen design
  • multi-touch surfaces
  • tabletop displays
  • ubiquitous computing