Skip to main content
Log in

Pull-the-strings

Generic mapping model for digital puppetry

  • 1161: Multimedia Alternate Realities
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Pull-the-Strings presents a mapping model for digital puppetry based on a transparent framework to support generic device controllers and generic tools. Digital puppetry requires a creative interaction design, in particular in the way designers map the puppet to the puppeteer using specific devices. This process depends on a constantly changing interface technology, which limits the reuse of devices and mappings. This paper proposes a methodology and a set of tools that facilitate the mapping process, and promote the recycling of technologies. A flexible and generic environment independent from device specifications. By abstracting the hardware layer, the artist is motivated to think in terms of signal flow, establishing relations through meaningful mappings instead of handling the diverse specifications of each device and application. Pull-the-Strings is a data-flow ecosystem that focus on the functional usage of control signals. It provides a scalable environment for building semantic blocks that connect, transform and generate signals for the manipulation of virtual objects. Its goal is to make technology as transparent as possible, facilitating connections and reducing the obstacles between the performer and the performing object. On the other hand, it proposes an interaction design space that takes into account the manipulation and perception distance, responding to the specifications of the digital puppetry medium. This model was evaluated comparing a set of tools and methods with experienced and non-experienced users.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. Zeroconf web site – http://www.zeroconf.org

  2. Node Box 3: https://www.nodebox.net/node/

  3. Fugio Web site: https://www.bigfug.com/software/fugio/

  4. RPD Web site: https://shamansir.github.io/rpd/

  5. Node–RED web site: https://nodered.org

  6. Virtual Marionette website: http://virtualmarionette.grifu.com

  7. Puppet Tools Github repository: https://github.com/grifu

References

  1. Anson E (1982) The device model of interaction. In: Proceedings of the 9th annual conference on computer graphics and interactive techniques. ACM, New York, pp 107–114

  2. Bleser FD, Smedt TD, Nijs L (2002) NodeBox

  3. Bodenheimer B, Rose C, Rosenthal S, Pella J (1997) The process of motion capture: Dealing with the data. In: Thalmann D, van de Panne M (eds) Computer animation and simulation ’97: Proceedings of the eurographics workshop in budapest, Hungary, September 2-3, 1997. Springer Vienna, Vienna, pp 3–18

  4. Card SK, Mackinlay JD, Robertson GG (1990) The design space of input devices. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, pp 117–124

  5. Coduys T, Ferry G Iannix: Aesthetical/Symbolic visualisations for hypermedia composition. In: International conference sound and music computing, The Hague, Netherlands, pp 194–196

  6. Francis P (2011) Puppetry: a reader in theatre practice. Readers in theatre practices palgrave macmillan

  7. Hinckley K, Wigdor D (2012) Input technologies and techniques Jacko J (ed)

  8. Jacob RJK, Girouard A, Hirshfield LM, Horn MS, Shaer O, Solovey ET, Zigelbaum J (2008) Reality-based interaction: A framework for post-WIMP interfaces. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, pp 201–210

  9. Leite L (2018) Virtual Marionette: Interaction Model for Digital Puppetry. Ph.D. thesis. University of Porto, Porto

    Google Scholar 

  10. Leite L, Amândio A (2020) Solitária - Gestural Interface for Puppetry Performance. In: International conference on live interfaces. trondheim, Norway

  11. Leite L, Orvalho V (2017) Mani-pull-action: Hand-based digital puppetry. Proc ACM Hum-Comput Interact 1(EICS):2:1–2:16

    Article  Google Scholar 

  12. Leite L, Torres R, Aly L (2018) Common spaces: Multi-modal-media ecosystem for live performances. MATLIT Materialities of Literature 6(1):187–198

    Google Scholar 

  13. Leite LM, Lafontana M (2016) Digital theatrograph: Cinematographic puppetry. In: Proceedings of the 1st international workshop on multimedia alternate realities. ACM, New York, pp 3–8

  14. Morrison JP (2013) Flow-Based Programming application developers’ news (1)

  15. Rudraraju V (2011) A Tool for Configuring Mappings for Musical Systems using Wireless Sensor Networks. Chulich School of Music McGill University pp 1–90

  16. Walther-Franks B, Malaka R (2014) An interaction approach to computer animation. Entertainment computing, Elsevier, pp 1–37

  17. Wang Y, Lucey S, Cohn JF, Saragih J (2010) Non-rigid face tracking with local appearance consistency constraint. Image and Vision Computing 28 (5):781–789

    Article  Google Scholar 

  18. Wright M, Freed A, Lee A, Madden T, Momeni A (2001) Managing complexity with explicit mapping of gestures to sound control with OSC. In: ICMC

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Luis Leite.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Leite, L. Pull-the-strings. Multimed Tools Appl 83, 46731–46756 (2024). https://doi.org/10.1007/s11042-021-11876-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11876-y

Keywords

Navigation