, Volume 6, Issue 3-4, pp 95-99,
Open Access This content is freely available online to anyone, anywhere at any time.

Conceptual frameworks for multimodal social signal processing

This is an excerpt from the content

This special issue is about a research area which is developing rapidly. Pentland [4] gave it a name which has become widely used, ‘Social Signal Processing’ (SSP for short), and his phrase provides the title of a European project, SSPnet, which has a brief to consolidate the area.


The challenge that Pentland highlighted was understanding the nonlinguistic signals that serve as the basis for “subconscious discussions between humans about relationships, resources, risks, and rewards”. He identified it as an area where computational research had made interesting progress, and could usefully make more.

If effective progress is to be made, one of the requirements is to develop some consensus on a variety of issues that are basic to the area—obviously the topics to be covered, but also terminology, the literature that people in the field are expected to know, the simplifications that are considered acceptable, and so on. That kind of statement might look routine, but in the