1 Abstract

In this introduction we offer an overview of the 15 articles of the special issue What’s so special about faces? Visages at the crossroad between philosophy, semiotics and cognition, sorting them into five broad topics, i.e. (i) the attempt at overcoming the notion that there is a one-to-one mapping between certain emotions and facial expressions, (ii) the nature of some properties at play in face perception, (iii) the special status of faces (especially our own) in cognition, (iv) facial tools such as masks, (v) the socio-cultural meaning of represented faces.

2 Faces as theoretical objects

From the very beginning of the ERC Project FACETS, which supports the publication of this issue (the editors, Marco Viola and Massimo Leone, are FACETS’ post-doctoral researcher and PI, respectively), it has been immediately clear that the central object of the project’s research — variously named, in different languages, with the quasi-synonyms “face”, “visage”, “countenance”, etc. — was and would continue to be a huge success, both in the project’s main discipline, that is, semiotics, and in those that were intertwined with it in the project’s highly interdisciplinary program of research. This success, which has allowed FACETS to organize conferences, publish special issues of journals, and even launch a new series of books on the subject, was further accentuated with the advent of the pandemic, whose dramatic developments considerably thematized the face, in terms of both the obligation to cover it to defend it against the virus, and the need to digitize it to continue its social existence. To what do we owe the success of the face? An immediate answer might be the following: the face is so central to the lives of both individuals and societies that it is present like a rhizome with very deep and extremely ramified roots not only in individual psychologies but also in societies, so that almost everything, in the mental dimension as well as in the cultural one, can be related to the face. One of the realizations that have emerged from editing this special issue of the journal presented here, however, is that this individual and social centrality of the face alone cannot explain its academic success. Other objects of study are equally present in the lives of individuals and societies, but for this reason they do not attract such intense, multifaceted, prolonged, and fervent disciplinary efforts. Perhaps then the explanation is to be found elsewhere. In the field of art theory, which has also been highly concerned with the face, Hubert Damisch and others have begun to speak of “theoretical objects”, i.e., those objects that, while on the one hand are figures of representation, on the other hand become lumps around which theoretical elaborations coagulate, not only concerning the object in question but the entire field of art theory. Clouds, for instance, so elegantly studied by Damisch (1972), may appear an iconographic subject among the many represented in the history of art, even more so because they often remain in the background, embellishing the skies that surround the real subjects of representation, such as Madonnas, saints, and stories of Christianity. And yet, looking at those clouds with different eyes, one can discern in them a whole development of art history according to new theoretical coordinates, which radically change the general approach of the discipline.

There is no doubt that the face too is a theoretical object. As the FACETS project has shown with increasing evidence, disciplines approach the face not only because of its psychological and social importance, and not only because it is at the center of an emergency, but because, by approaching the face, they feel they can test their own limits, conduct frontier research that not only probes the perimeter of the methodologies adopted, but also breaks through it, often crossing over into other disciplinary fields. There are in fact at least two types of interdisciplinarity, one a priori, the other a posteriori. The first is programmatic, cunning, fictitious, occasional; it is the one that is promised in an attempt to align itself with the logic of the times and funding calls, the one that very often does not produce any meaningful exchange; interdisciplinarity a priori is like an arranged marriage, a marriage of interests; interdisciplinarity a posteriori is, instead, the one that is born as a need for research, when the fire of the questions to which one seeks an answer pushes one to stretch the limits of one’s own field of competence more and more until it leads to a different, still uninhabited place. As editors of this issue of the journal, we can only thank Topoi and its editor-in-chief, Prof. Fabio Paglieri, for having kept faith with the very name of the publication, and for having provided us, thanks to it, with a series of places, with a series of Topoi, to give voice to this interdisciplinary not of interest but of substance, not arranged but the result of attraction for what is different, an attraction that mushrooms all around the articles in this collection, and around the theoretical object they aim at: the face.

3 The Topics of this Issue

To offer an overview of the contents of this issue, we have sorted them into five topical clusters. First, some authors’ efforts are directed at overcoming the deterministic “expression = emotion” approach to face-reading, indicating new ways to interpret facial movements (§ 2.1). Second, a couple of authors frame the topic of facial properties within ongoing debates on the philosophy of perception (§ 2.2). In a third group we find papers that directly address the special status of faces (and especially one’s own face) by surveying experimental literature (§ 2.3). A fourth group of articles deals with facial prosthetics, ranging from relatively traditional ones such as the mask toward more contemporary tools such as the ballgag (§ 2.4). Lastly, four papers investigate facial representations, scrutinizing the techniques (e.g., photography) and values (e.g., beauty) at stake when they are produced and consumed (§ 2.5).

This classification has been operated post-hoc by us guest editors, and by no means exhausts the rich and multi-faceted contents of individual articles. Indeed, these clusters have fuzzy boundaries: some papers included in one group offer valuable insights also about the other topics.

Below we describe the papers in some more detail.

3.1 Beyond the Simplistic “Expression = Emotion” Dogma

The scientific legacy of Silvan Tomkins, Paul Ekman, and Carroll Izard has knotted together the study of emotion with that of facial expression. The starting point was Tomkins’ idea that facial affect programs (i.e., hardwired innate movements of facial muscles) impress the seal of some specific emotion onto undifferentiated states of affective arousal (Tomkins and McCarter 1964). Following his lead, Izard and Ekman collected evidence that similar patterns of facial movements were consistently interpreted as a specific emotion across different cultures, including remote populations that could have not learnt it via cultural means such as cinematic representations (see notably Ekman and Friesen 1971; Izard 1977). By the end of the 20th century, the existence of universal expressions distinctive for each (basic) emotion was regarded as received wisdom and included as ‘canon’ in psychology textbooks. This belief percolated in other fields, such as philosophy and affective computing. Most extant implementations of facial emotion recognition technologies rely on that assumption (see for instance Cohn and De la Torre 2015).

During the last quarter of the century, the expression-emotion dogma underwent harsh criticism within psychology. The evidence that supported it has been challenged both on methodological (e.g., Russell 1994) and on empirical (Barrett et al. 2019) grounds. Yet, compared to the abundant literature aiming at dismissing the “expression = emotion” dogma, very few positive proposals have been advanced that address the following question: “If not deterministically expressing one emotional state, then what else do facial movements communicate?” Three papers included in this collection make some steps to fill this gap.

In his paper Making Faces, Paul Dumouchel (2022) reviews some arguments against the alleged universality of basic emotion displays, in favor of a rival account: the Behavioral Ecology View advocated by Fridlund (1994). According to this view, facial displays are best construed as communicative tools for steering social interactions toward one’s goal. According to Dumouchel, the centrality of the Behavioral Ecology View is paradoxically supported by the fact that facial movements of social robots are designed to reflect alleged universal expression, rather than as social tools, because robots (unlike humans) are not built to pursue their own goals.

The dialectic between the universal emotion expression thesis and the Behavioral Ecology View is also the starting point of Carolina Scotto’s (2022) paper A Pragmatics-First Approach to Faces. Rather than picking one side, however, the author continues Scarantino’s efforts at integrating the insights of both approaches into a comprehensive account. According to Scarantino’s Theory of Affective Pragmatics, allegedly emotional expressions are both expressions of inner emotional states and social tools - which he invites to construe in analogy with Speech Acts. This is where Scotto disagrees, as the notion upon which she lays the foundations for her Pragmatics-First Approach to Human Communication of face-reading is that of affordance (Gibson 1979). By doing so, she claims, we will be in a better position to account for the relational nature of many facial displays; and to account for more facial signals beside those traditionally linked with emotional expressions - among other things.

The notion of affordance is prominent also in Faces and Situational Agency, where Matthew Crippen and Giovanni Rolla (2022) articulate and defend an account of situational mindreading. They report several threads of evidence indicating that the meaning of some ‘emotional’ expressions can be influenced by the context, e.g., the well-known Kuleshov effect (a cinematic effect in which an actor’s facial expression is interpreted in diverging ways, based on which scenes are shown before and after it), or more recent psychological experiments. The article’s proposal, however, goes beyond recognizing that the context modulates the meaning of facial expressions: in their view, the proper and primary object of our ‘mindreading’ is the overall situation (rather than some of its constituents, such as faces). As they embrace a 4E framework, in which minds arise in the interaction between agents and their environment, emotional properties can be rightfully ascribed to ‘inanimate’ objects such as paintings and landscapes.

3.2 Perceiving Facial Properties

A longstanding debate in philosophy is the distinction between perception and cognition (or other higher cognitive faculties). Recently, a popular strategy to frame this topic has been to articulate a distinction between “low-level” properties that can be uncontroversially ascribed to perception (e.g., shape in color in vision, volume in hearing) and other “high-level” properties whose status seem to imply some cognitive articulation, perhaps some conceptual knowledge (e.g., being a dog, being a rap song). Phenomenological and experimental evidence have been both mobilized to investigate whether and in what sense the latter kind of properties can be legitimately called “perceptual”, and in what respects they resemble and differ to the former kind (e.g., Siegel 2010; Fish 2013). Now, despite the prominence of faces both in our everyday social life and in experimental psychology, philosophers have paid surprisingly scant attention to the properties at play in face perception (with few exceptions like Block 2014 and Newen 2017). Is the process of recognizing someone’s identity based on their face perceptual or inferential? And what about their emotional state? And their ethnicity, gender? And their apparent (un)trustworthiness? What about face detection, i.e., the property of recognizing some visual pattern as a face in the first place? Both papers in this section nicely fill the gap by directly addressing the perceptual nature of facial properties.

In their article Perception of Faces and Other Progressively Higher-Order Properties, Fabrizio Calzavarini and Alberto Voltolini (2022) defend the following criterion for ascribing the perceptual nature of a given property: “a property is perceivable if and only if it is not only given immediately and non-volitionally, but also grasped via a holistic form of attention”. Drawing on both phenomenological and empirical evidence, they show how their criterion vindicates the perceptual status for the property ‘being a face’, as well as for other kinds of facial properties that are hierarchically higher-order, i.e., gender and ethnicity.

In Face Perception and Mind(Mis)Reading, Joulia Smortchkova (2022) also holds that faces convey several (high order) perceptual properties, although she mainly focuses on emotional expressions (e.g., “they are angry”) and quick impressions of character traits (e.g., “they are friendly”). The perceptual nature of these properties implies that they may bring about some “perceptual illusions”, as perception tends to be more less amendable than higher cognitive states. Such illusions, which tend to occur more systematically with respect to character traits than to emotional attributions, can be a source of mind misreading.

3.3 The Special Status of (One’s Own) Face

Faces (or visual patterns that resemble them) are salient stimuli for our eyes. A magnet for the eyes, they tend to grab our visual attention more than most other stimuli, and to keep it for a longer time (Palermo and Rhodes 2007). This selective attention for faces seems already in place in newborns (Goren et al. 1975), perhaps even before birth, i.e., in (late stage) fetuses (Reid et al. 2017). While the debate about the origin of our preference for faces is still lively, there is no doubt that faces are special.

The special status of faces is underlined by Alexandra Mouratidou, Jordan Zlatev, and Joost van de Weijer (2022) in their (re-)analysis of an experiment on choice investments. In How Much Do We Really Care What We Pick? Pre-Verbal and Verbal Investment in Choices Concerning Faces and Figures, the authors aim at overturning a popular idea held by many cognitive scientists, namely that most cognitive processes are inaccessible to cognitive awareness. Subjects were asked to choose one out of two images depicting faces or abstract figures; they were later prompted to justify their choice – and, in some cases, to justify why the other option was not chosen. By enrichingthird person methods of classical experimental psychology with a careful examination of the motivations provided by subjects, the article offers a more nuanced grasp of what is at stake in choices.

While all faces are special, some are more special than others. Psychology and neuroscience have highlighted how familiar faces are processed quite differently from unfamiliar ones (Ramon and Gobbini 2018). Among familiar faces, there is one that holds a privileged status: one’s own.

In The Reflected Face as a Mask of the Self: An Appraisal of the Psychological and Neuroscientific Research about Self-Face Recognition, Gabriele Volpara, Andrea Nani, and Franco Cauda (2022) provide a comprehensive overview of the neurocognitive literature on self-face perception mediated by mirrors. The authors also go beyond reviewing the empirical literature, suggesting that the face reflected in the mirror can act as a sort of mask, a scaffold for the Self.

The special status of one’s own face also emerges in Vittorio Gallese’s (2022) paper Embodying the Face: The Intersubjectivity of Portraits and Self-Portraits. Since neuroscientific evidence suggests that our visual system is biased toward the left part of the visual field, the author predicts that self-portraits could be more emotionally expressive than portraits, because painters too could be looking more at the right part of the face while making portraits, whereas in self-portraits the left half of their face would be reflected in their left visual field, in the mirror they would use to paint. Yet further evidence suggests that the left half of the face could be more expressive. A series of experiments reveals that indeed subjects find self-portraits more emotionally intense (and pleasing) than portraits, but this effect is not mediated by the mechanisms hypothesized by the author.

3.4 Masks and Other Facial Tools

Being such a powerful attention-grabbing mechanism, and thanks also to its high motility, the face is a privileged locus of social interactions. Whether to enrich the face’s communicative capacity, or to tame some informational giveaway that is better to keep in check, humans have long garnished their faces with masks and other tools. Indeed, masks and (the cultural representation of) faces have a long history of coevolution (Belting 2017). The articles in this section explore this story a step further.

In Faces in Disguise. Masks, Concealment, and Deceit, after having set the theoretical foundation of a semiotic theory of deceit as resulting from either simulation or dissimulation, Remo Gramigna (2022) discusses how the face can be used to deceive through both its static and dynamical features. Moreover, he enriches the semiotic theory of masks by pointing out a hitherto underappreciated typology of them, aimed at warranting anonymity, i.e., at zeroing-out the wearer’s identity, rather than replacing it with another for the purpose of acting or of deceiving.

In From Mask to Flesh and Back: A Semiotic Analysis of the Actor’s Face between Theatre and Cinema, Massimo Roberto Beato (2022) provides a semiotic analysis of some historical developments of the norms governing facial behaviors in theater — conveyed by acting handbooks inspired by the physiognomic traditions — and in cinema, where the technical possibility of close-ups lead to a new focus on the face with respect to the full body — which, in turn, implied new burdens and challenges on the actors. The author also discusses the neutral mask, employed by the theatrical pedagogue Jacques Lecoq as a tool for training actors thanks to its capacity to ‘zero-out’ its wearers’ subjectivity and expressivity.

Not all facial tools were designed to play a social function. And yet, possibly also due to the prominence of the face in our social life, they are very likely to end up being imbued with social meanings. This is nicely shown by the case of the sanitary facemask (Leone 2021). But other facial artifacts underwent similar processes of re-semantization.

In Artifacting Identity: How Grillz, Ball Gags, and Gas Masks Expand the Face, Cristina Voto and Elsa Soro (2022) describe how the three facial artifacts mentioned in the title, all of them characteristically associated with the mouth region, underwent a process of semiotic subversion within some sub-cultural niches: golden and colored grillz, originally dental prostheses to be placed in oral cavities, have been worn by some rappers to ostensibly display the wealth they have been excluded from; the ball gag, whose roots are to be found in tools for subjugation, is readapted by practitioners of bondage and sado-masochism to perform consensual submission; whereas gas masks, traditionally employed in dangerous breathing conditions, have been redeployed by punks and ravers.

3.5 Representing Faces

Given their prominence in social interactions, it is unsurprising that faces are so widely represented in several media, from painting to photography, from videogames to virtual reality. Like all representations, the facial ones too are shaped by a set of norms that guide the way in which they are codified and interpreted, depending on the specific media and on the context in which they circulate.

In Models, Mannequins, Dolls, and Beautified Faces: A Semiotic and Philosophical.

Approach to the Sense of Beauty, Maria Giulia Dondero (2022) shows how the notion of beauty as pertaining to faces shifts meaning across different corpora and domains. A semiotic analysis of enunciation is adopted to compare the singular beauty of the model Bettina Graziani, the generalized beauty of the doll Barbie, the uncanny beauty of mannequins, and the algorithmic beauty at play in some research applying convolutional neural networks.

In the same group of articles, Dario Martinelli (2022) focuses on a single case study from the history of music, namely, the prominence of the face in the iconography of the Beatles (as opposed to other male rockstars, whose full bodies were emphasized so that they could play the role of sex symbols). The paper “I’ve just Seen a Face”: The Beatles’ Faces as Aesthetic and Cultural Objects, analyzes the iconography of the Beatles’ faces: their aesthetic features, facial parts, expressions, performances, and prostheses; the multiple meanings of faces in the Beatles’ song “I’ve just Seen a Face”.

In the paper Looking into Death: Trauma, Memory and Human Face, Patrizia Violi (2022) rather turns her gaze toward the role played by photographs of faces in memories of death and traumas. Shifting between a variety of sources — from memorial museums such as Auschwitz-Birkenau or the Tuol Sleng Museum of Genocide Crimes in Cambodia, to some artistic creations in Bologna commemorating partisans and victims of the bomb that exploded in the local train station in 1980, to name but a few — she highlights the multifarious expressive potentiality of faces, and especially their capacity to represent, at the same time, both the singularity of victims and their generality as human beings.

Finally, in What is so Special about Contemporary CG Faces? Semiotics of MetaHumans, Gianmarco Thierry Giuliana (2022) puts semiotics in dialogue with cognitive psychology in order to understand why contemporary computer-generated faces feel so realistic. He focuses on the recent software MetaHuman Creator, examining its technological peculiarities and showing how they meet the semiotic and psychological desiderata for make-believe in terms of how they look, what they mean, and how they act. The author also highlights a qualitative difference between MetaHuman faces and previous artificial faces.

4 Opportunities and Challenges of an Interdisciplinary Research on the Face

Interdisciplinarity is easier said than done. Indeed, roburstly engaging with other scientific fields requires a considerable amount of time and effort to get acquainted with different epistemological frameworks. Think of them as Kuhnian paradigms, all of them busy in dealing with their own problems through well-tested research tools and protocols, and eager to sweep anomalies under the rug, i.e., to ignore whatever phenomenon challenges their core assumptions (Kuhn 1962). In some cases, however, the result is worth the pain. We think that this special issue qualifies as one of such cases. Indeed, while all five topics are mainly rooted in one of the three disciplinary fields at stake in the issue (i.e., cognitive science, philosophy, or semiotics), the papers all show the fruitfulness of adopting multiple perspectives on the same topic. Many of the published authors crossed the boundaries of their discipline, trading notions and ideas with other intellectual fields, and exposing themselves to a challenging peer review process, since it often involved experts coming from different disciplines. To both authors and reviewers, the editors of the present special issue wholeheartedly express our gratitude.

Taking a tour outside one’s paradigm allows one to get a better, perspectival understanding of its strengths and weaknesses. It can also reveal interesting and hitherto unexplored research avenues. For instance, psychological studies on faces covered with facemasks would have a hard time accommodating all differences between masked and uncovered faces only in terms of “less facial surface perceived”, ignoring the complex and sometimes paradoxical social stratification of meanings displayed by face masks. And in that respect, psychology has much to learn from semiotics. Similarly, philosophical accounts of perception have much to gain from a careful scrutiny of the rich and growing cognitive literature on face-reading.

A problem when two or more disciplinary traditions cross paths is that people working in different paradigms employ different (and sometimes possibly incommensurable) theories and models to explain phenomena. Our experience as PI (ML) and former member (MV) of the research project that supported this special issue suggests that a possible way out is to anchor the communication in phenomena themselves. We all have a face, see faces, and live in an environment where face representations abound and play several roles. While we were developing our project, we all witnessed the practice of covering our faces with medical face masks shifting from ‘weird’ to ‘normal’ because of the pandemic. Sharing these experiences afforded solid and disciplinary-neutral grounds for framing research questions.