Reactive Statistical Mapping: Towards the Sketching of Performative Control with Data

  • Nicolas d’Alessandro
  • Joëlle Tilmanne
  • Maria Astrinaki
  • Thomas Hueber
  • Rasmus Dall
  • Thierry Ravet
  • Alexis Moinet
  • Huseyin Cakmak
  • Onur Babacan
  • Adela Barbulescu
  • Valentin Parfait
  • Victor Huguenin
  • Emine Sümeyye Kalaycı
  • Qiong Hu
Conference paper

DOI: 10.1007/978-3-642-55143-7_2

Volume 425 of the book series IFIP Advances in Information and Communication Technology (IFIPAICT)
Cite this paper as:
d’Alessandro N. et al. (2014) Reactive Statistical Mapping: Towards the Sketching of Performative Control with Data. In: Rybarczyk Y., Cardoso T., Rosas J., Camarinha-Matos L.M. (eds) Innovative and Creative Developments in Multimodal Interaction Systems. eNTERFACE 2013. IFIP Advances in Information and Communication Technology, vol 425. Springer, Berlin, Heidelberg

Abstract

This paper presents the results of our participation to the ninth eNTERFACE workshop on multimodal user interfaces. Our target for this workshop was to bring some technologies currently used in speech recognition and synthesis to a new level, i.e. being the core of a new HMM-based mapping system. The idea of statistical mapping has been investigated, more precisely how to use Gaussian Mixture Models and Hidden Markov Models for realtime and reactive generation of new trajectories from inputted labels and for realtime regression in a continuous-to-continuous use case. As a result, we have developed several proofs of concept, including an incremental speech synthesiser, a software for exploring stylistic spaces for gait and facial motion in realtime, a reactive audiovisual laughter and a prototype demonstrating the realtime reconstruction of lower body gait motion strictly from upper body motion, with conservation of the stylistic properties. This project has been the opportunity to formalise HMM-based mapping, integrate various of these innovations into the Mage library and explore the development of a realtime gesture recognition tool.

Keywords

Statistical Modelling Hidden Markov Models Motion Capture Speech Singing Laughter Realtime Systems Mapping 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© IFIP International Federation for Information Processing 2014

Authors and Affiliations

  • Nicolas d’Alessandro
    • 1
  • Joëlle Tilmanne
    • 1
  • Maria Astrinaki
    • 1
  • Thomas Hueber
    • 2
  • Rasmus Dall
    • 3
  • Thierry Ravet
    • 1
  • Alexis Moinet
    • 1
  • Huseyin Cakmak
    • 1
  • Onur Babacan
    • 1
  • Adela Barbulescu
    • 2
  • Valentin Parfait
    • 1
  • Victor Huguenin
    • 1
  • Emine Sümeyye Kalaycı
    • 1
  • Qiong Hu
    • 3
  1. 1.Numediart Institute for New Media Art TechnologyUniversity of MonsBelgium
  2. 2.GIPSA-labUMR 5216/CNRS/INP/UJF/Stendhal UniversityGrenobleFrance
  3. 3.Centre for Speech Technology ResearchUniversity of EdinburghScotlandUK