From the study of semiotic paradigms in relation to the face, we focus on the traces, understanding how some flourish from the tangible but ignored signs left by humans daily, while others are totally imbricated in the face from/in which they transpire.
We typologize them in three varieties, from their multidimensional configuration, offering case studies of emanation, imbrication, and cancellation. First, between art and forensic tendencies, Dewey-Hagborg uses hair, cigarettes, and chewing gum off the streets to program and build 3D faces through the DNA found in them. Secondly, we examine the artistic work of Jorit who engraves on his face the sign that symbolizes belonging to a tribe he is working with. Name-face isomorphism emerges in the third case: Janez Janša carries out a performative sociopolitical program to test, destabilize, and reorganize cultural complexity.
All offer a syncretic situation analyzable by means of the semiotic approach and bioanthropological resources. The divergent weights of similar elements make us reflect on the relationship between the innermost meanings of our faces and their tracks in a sort of anticlockwise movement but also on the convergence between macro-cultural and techno-political orientations with intimate and located magnitude.
- Multimedia art
- Body semiotics
- Facial emanations
- Human physiology
Donde hay orejas pero no hay canto
En este mundo que se desvanece
Y el ser se otorga a quien no lo merece
Soy mucho más mis huellas que mis pasos. (Original text: “Where there are ears but no song
In this fading world
And being is bestowed on the undeserving
I am much more my footprints than my footsteps.”)
(Jodorowsky, 2008: 97)
This article is part of FACETS: the project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement No 819649—FACETS, PI Massimo LEONE)
Many thanks to FACETS team NB.
Starting from Peirce’s tripartition of signs, we turn our attention to indexes such as those types of signs causally connected with their object (Peirce in Eco, 2016, p. 190). Indexicality opens its territory by expanding the presential option of the object of reference and allowing the introduction of an indirect but imbricated correlation between expression and content: “The presence of the object is not necessary for the sign to signify, although it is required to verify the use of the sign in an act of reference” (Eco, 2016, p. 297).
The topo-sensitivity inherent in the indexes highlights the spatial relationship that builds the semantic pathway, which propitiates different directional movements and has various dynamic possibilities. Chrono-sensitivity, on the other hand, emphasizes the temporal relation which, as we shall see, can take backward or proximal routes: a trace can therefore be read as a sign that makes it possible to return to the origin or as an indication yielding a new route. Motivation versus arbitrariness seems to be one of the driving forces behind circumstantial research: the trace as the motivated result of an underlying content, the trace as the motivator of a developing content, ex post facto.
The objective of this text is to work with the indicative emanations called traces, the constitutive clues of subjective and collective meaning, which are advanced as a continuous processual configuration in which the hypothetical directions can only be comprised in the forming formation, toward the elaboration of a sociopolitical, bio-metaphysical fabric whose semiotic substrate is necessary as well as pertinent. A certain phenomenon, a sign, a set of signs, can constitute not only a model but rather the basis on which a model is established. Approaching the traces implies a prior work of pertinentization and therefore of observation, delimitation, and argumentation; the outcome is a punctilious work of questioning not only the cause-effect linearity but also the logic underlying the sign, i.e., the constitution of a text and of a space.
We actually enter the semiotics of the body as text and archive, as a space of figural memory resulting from a process of synchronic contact and a diachronic reading of an echo of matter that refracts on itself and dissipates in the world, of a spongy face imbued with penetrating magnitude. We are in a semiotics of art as a special space of passionate, political, and transhistorical daily experimentation. The underlying anthropological paradigm completes the absolution of being as part of a cosmic and natural organization in which culture is involved: the human face is our spatial situation of analysis. Thinking of the face as a place of discourse, as the writing of a tale and a narrative sign, does nothing other than propose to the analytical gaze an opening -scopic inclusive of the stories already experienced, a summing up: “From the perspective of a semiotics of text and discourse, the body is first and foremost a site of meaning, and of a meaning that takes shape from the sensations and impressions that this body experiences in contact with the world.”Footnote 1 (Fontanille, 2003, p. 3).
We will be providing a general panorama, selecting only a few emblematic art pieces as a reduced corpus that can propitiate a kind of coherence with the entire position, showing how porous contemporary society is: recalling the sign as something that stands for something else, we thus see in otherness the cohesive element and in the intervening spatiality the construction of semiosis.
We will travel between different historical epochs, contextualizing contemporaneity within a necessarily historical marker, and transit geographically between extremes in order to be located in a transversal and multi-situational topographical position where the face has the property of constituting, reinforcing, or dissolving intimate and collective subjectivities.
Phenomenological and ontological references are often necessary to bring attention back to the becoming of material signs, to in situ even if not necessarily analogical incorporation, to the deconstruction of the subject through its traces, and to the constitution of the traced personality. Denuded of the complex machinery that composes our face, expression and name, and vibration and trace, there remains the neutral that is intrinsically expressive versatility, collection of immanent memory, and secularized sediment of inadvertent powers.
We take the artistic paradigm as a model through which reality is approached and treated in accordance with criteria of intuition and freedom and by referring to a specific modelling system that is “a structure of elements and rules of their combination, existing in a state of fixed analogy to the whole sphere of the object of perception, cognition, or organization. (…) The content of art as modelling system is the world of reality, translated to the language of our consciousness, translated in turn to the language of the given form of art” (Lotman, 2011, p. 250).
Let us consider the traces following three primary, if not exclusive, lines of investigation:
Traces emanating from the face
Traces on or in the face
Traces erased toward new tracks
On the basis of a semiotics of the face in inclusive assonance with a semiotics of the body, according to the idea of the witness body (Fontanille, 2003) and elaborating it further in line with our criteria and those of other authors of reference, we can try to understand how these three paradigms integrate specific examples and sense vectors built on different semiotic axes. Although we do not intend to reduce the semiotic interpretation to a diagrammatic schematization, we will simplify by pointing out the following characteristics of each in the light of their multidimensional intersection.
The body in movement constitutes itself in a deictically marked path and therefore, at least in part, is recorded, traced; from it derived the first type of emanations, in a sort of enunciation in which the narration of the trace runs along opposite routes in the two cases mentioned. In the artistic case, the trace is contextualized within an aspirational path projected into the future, and the predicative argument underlying it is a discovery rather than a finding. The mode of existence of the new constituted subject is an inventive simulacrum, actualized, and projected.
In forensic cases, however, the trace is contextualized within an aspectual path projected into the past, the basic argument being the backward search aimed at the semantic area of finding again. The body in movement, whose matter degenerates because it is encroached upon by its own wholeness and compactness, blurs into the world: the rediscovery of its parts and above all the reunion with its own origin, as well as the reintegration of the gap previously left, aspires to figurative and formal completeness.
The second model, on the other hand, deals with the face in dialogue first and foremost with its own circumscribed material form, with a well-known syntactical feature: both in the combinations of reading the signs on the face on the basis of physiognomy or other pseudosciences and in the inscription of new signs, the trace is preponderantly an element of memory. Its elements inhabit the organic microcosm which, in the provision of a paradigmatic as well as syntagmatic reading of the figurativeness of the face, is then immersed in the authentication of the evidence of a state, the articulation of the face as witness of contact, enunciated autopoietic resource, and filiation of intimate inscription whose vision and interpretation are crucial.
The act of inscription can be automatic or involuntary but also sought after or propitiated: in the first sphere, we can refer to the signs of time and character (whose reading is attributed to physiognomy and other pseudosciences), while the second sphere comprises those signs caused by accidents or misfortunes. The analyses of physiognomic traces but also of all those linked to the ontologically experienced face are part of the figure of the stratified body, an inhabited archive whose continuous movement becomes a memory in construction: an embodied face whose movement over time sediments, creating forms.
This second group, however, also includes those traces of intentional origin, at the basis of which we usually find aesthetic, political, or cultural motives, as in the case of the street artist involved in our study: “matter is subject to forces, the balance of which gives it form”Footnote 2 (Fontanille, 2003, p. 4). The malleable and elastic face takes on new signs symbolizing membership of a tribe, and the face belonging to the so-called fleshly body is materially reshaped through the process of scarring, a figure of reconciled frontier and indelible recision at the same time. The new syntactical marks, with a specific directionality and form, are strictly connected with the semantical vestige of affiliation to a major collectivity: the trace also becomes a symbol for distinguishing group membership.
This brings us to our third tensive band of actorly displacement, which tries to establish a heuristic dialogue between the reference in the natural world and the intelligible sensory inference. It is not a question of corporal inscription in these cases but of textual reformulation with erasure as a supporting strategy: the story written in the traces wants to change “the relationship between the thread—the thread of the story that helps us to orientate ourselves in the labyrinth of reality—and the trace”‘(Ginzburg, 2020, p. 7)Footnote 3 is broken.
If we think of the following semiotic square from Greimas and Courtes (1982, pp. 308–311. Fig. 21.1), we propose to problematize the relation between permanence and cancellation and underline different strategies of memory fixation through traces.Footnote 4 While in the case of Saraji, we will see a strong trace of resemantization through the erasure of faces represented photographically, in that of Janez Janša, it is a matter of cancellation of face traces as bio-political instances (Fig. 21.1).
The body emphasizes its subjectivity by following an a-rebours path in search of the inscriptions no longer of the ontological and actualized face but of a symbolic and potential face, inscriptions that prove the univocal relationship formed by the face-name isomorphism: all the texts projecting this formation are the basis of the change. The face becomes the spectator of a radical change that sees in the alteration of name a strategy with which the body concurs, in a kind of performance absorbed in the proximal environment.
Perception and action undergo slight techno-aesthetic variations, enabling us “to relate interactively with our environment, from which we grasp potentialities that can be elaborated not only on a cognitive-conceptual level, but also, and in a more original sense, techno-aesthetically elaborated, with a view, for example, to the production and use of artefacts” (Binda, 2017, p. 138).
It is precisely these latter works that become meta-witnesses of the witnessing body: identity documents, credit cards, and all those artifacts in which the name appears, often combined with the face photograph, are moments of identity reformulation but also reformulation of the environment. This is, in fact, the resonance of the art pieces of this collective, in which every day technical objects take on a different function, being shown as evidence of a synergic conversion with the environment.
The ontological face and the semiotic face are mainly interdependent to the point of discovering a disciplinary syncretism that focuses on the same conscious function of the passage and its phases.
Traces Emanating from the Face: [Material Turn/Material Split]
Traces have always been an important tool not only in establishing a contract, albeit a provisional one, with missing texts, but also in setting up possible paths on the basis of conjectural similarities. In some cultures, their reading becomes a kind of science or art. Magli (1995) reminds us of the Argentinean rastreador mentioned by Borges, in charge of finding lost cattle in the fields, but also presents to us a similar practice widespread in Muslim jurisprudence: “the art called Kiyafa (“to follow”, in Arabic), says Fakhr al-Dīn al-Rāzī, is divided into Kiyafat al-athar (“to follow the track”) and Kiyafat al-bashar (“to follow the man along a path according to a direction”). “As regards the first type, it consists in knowing how to follow footprints or hoofprints left on sandy paths, which are likely to receive their imprint. (…) The second was instead an art that made it possible to discover kinship links. This art was so called because the Kai’f carefully examined the appearance of individuals, the epidermis, the shape of the limbs with particular attention to that of the feet. The consideration of the globality of all these clues allowed him to establish if there were blood ties between two individuals” (Magli, 1995, pp. 47–48).
The experience of unidirectionally matching interpretations and traces requires, first and foremost, a meticulous capacity for observation of the details and the whole in a sort of mereological attitude, but other prerequisites include the recognition of the truthfulness of the gesture of inscription, as well as the ability to decrypt the writing and thus the sign. How are we to follow this path if the signifier is apparently completely free or far from its initial meaning? “It is we, our culture, our arbitrariness, who decide on the status of referentiality of a writing. What does this mean? That the signifier is free, sovereign” (Barth, 1999, p. 29).
In this freedom, the pertinence inherent in the written trace suggests a certain relationship with the object-subject through which it is inscribed. Here then is the possible inverse, inventive process: a construction resulting from the assembly of deconstructed fragments which, allowing a glimpse of connective hints because they are inscribed in the often-invisible residual matter, contain indicative and therefore significant information.
Atmospheric aerosols and dropletsFootnote 5 are perhaps particularly resonant terms from 2020 onward, the year in which the COVID-19 pandemic takes shape, and we begin to understand that our saying, speaking, and even breathing take place in contact and exchange with the surrounding atmosphere, subtly penetrated by the invisible traces we impalpably emanate. The face, immediately covered by the mask which has been made compulsory in most of the countries involved, has to deal with what it exhales.
Fluid dynamics researcher Lydia Bourouiba has recently been working on detecting (with high-speed cameras and the support of light) the dynamics of pathogen expulsion from the human body. “Slowed down to 2000 frames per second, the videos and images from her lab show that the fine mist of mucus and saliva expelled from a person’s mouth can reach a speed of 160 km per hour and cover a distance of up to 8 meters.”Footnote 6 Their permanence, however, would depend on specific site conditions, including humidity and temperature.
We become potential transmitters of the virus, occluding ourselves behind our new appearance, often behind the medical masks, creating internal hoods of potentially asphyxiating, stale atmospheres. “The problem of social distancing, a proxemic problem, becomes our magnifying glass with which to observe how our bodies, and above all our faces, change in the days of the pandemic, how they resemantize and transform themselves as the viral emergency spreads” (Gramigna & Voto, 2020, p. 134).
The emission of substances from our faces was already known to us, as was the fact that they contained viruses and bacteria; we understood, too, that there is invisible matter that traces our passage, from a penetrating gaze to the droplets of speech. Its possible permanence and active perseverance over time, however, have perhaps been less considered. Indeed, while in forensic science the practice of identifying and reinvigorating traces has always been fundamental, in common knowledge it has so far been sidelined. Despite the fact that tracing is articulated by placing a certain element in retroactive connection with the subject of origin, the motive paradigm of such a path can be very different.
Heather Dewey-Hagborg,Footnote 7 for example, works with traces of works of art and, using hair, cigarettes, and chewing gum taken off the street, programs and builds 3D faces through the DNA found in them: characterized by the interpenetration between new media and human physiological nature, her proposal questions the awareness of production emanating from materials which, read backward, can venture into the rewriting of new texts.
The frontiers between biomolecular and indifferent modulation, one’s own and others’ spatiality, and the intimate and contextual body, blur but still perpetuate. The physical delimitation, which so clearly distinguishes between ipseity (ownness) and otherness, fades away, giving rise to a continuous echo, a constant diffuse mnemonic trace, which interacts with the environment: it may simply merge with it, or instead be considered writing, and thus be read, interpreted, and made pertinent.
The artist in question, for example, calls herself a biohacker; in her project entitled Stranger Vision, she follows these steps:
Collect hair, chewed gum, fingernails, and cigarette butts from public spaces.
Extract DNA from them.
Analyze it computationally through genomic research.
Parameterize a 3D face based on these traits.
Print it with a 3D printing machine (Fig. 21.2).
“Initially,—she says—the idea behind the project was to research what I could learn about the people who had dropped them. What I found was that by combining an arsenal of published research, bioinformatics and machine learning tools, I could draw statistical inferences or predictions about what these individuals looked and behaved like, what kind of health conditions they had and even what their surnames were.”Footnote 8
In the same article, Dewey-Hagborg criticizes the current use of this practice, appropriated from forensic work that not only constructs typologies but, in some ways, consolidates stereotypes: “The practice of rendering appearance from forensic samples is called ‘Forensic DNA Phenotyping’ (FDP) or ‘molecular photofitting’ (…).” While there are some traits such as eye and hair color that can be predicted from DNA with a high degree of certainty, the bulk of FDP is based on algorithmically derived statistical compositions. We tend to look at technical systems as neutral black boxes, but if you open them up and look at the component parts, you find that they reflect the assumptions and motivations of their designers. (ibidem) “Indeed, we have to question how the algorithmic work made automatic is actually the result of lengthy, extensive teaching based on parameters and indications that are already questionable beforehand “to show in practice the fallacy of the myth of algorithmic neutrality, as well as to examine its various discursive facets. In particular, three key aspects of the myth will be discussed and problematized (…): firstly, the assumption that no systemic distortions have interfered in the design and/or training phase of the algorithm and that, therefore, the output returns an objective point of view on reality; secondly, the idea that the automation and disintermediation of complex processes are in themselves guarantees of neutrality; thirdly, the claim that the results of algorithmic computation, narrated as accurate and infallible, do not alter reality according to socially, historically and politically determined logics” (Airoldi & Gambetta, 2018, pp. 33–34).Footnote 9
For example, Parabon SnapshotFootnote 10 elaborates specific personalities by exclusion and predicts physical characteristics including skin pigmentation, eye and hair color, face morphology, sex, and genomic ancestry and identifies distant familial relationships between samples. They define their work in this way: “Using genomic data from large populations of subjects with known phenotypes, Parabon’s bio informaticists have built statistical models for forensic traits, which can be used to predict the physical appearance of unknown individuals.”Footnote 11 But we are able, now, to understand how spurious this prediction may be.
In Dewey-Hagborg’s intentions, the possibility of being the precursor of a model later adopted by the laboratories of the judicial police was perhaps not contemplated. And this itinerary is typical of the artistic journey in which the avant-garde of art is a visionary predecessor with respect to future actions, sometimes even misleading in relation to its initial ideals and prerogatives.
The trace emanating from the face is an indication of what has been, an organic remnant of a passage that has taken place whose signs, albeit ephemeral, are installed in a material that preserves them. Some artists work with the materiality of residues, with the emanations and secretions of the body and face but also with the creation of ad hoc traces propitiated by physical contact between the body and other matter. Let us think in this regard of the works of Ana Mendieta, whose naked body becomes a matrix of mold, whose generated fluids become themselves pictorial materials for printing: with her blood, she invades her body, which in turn paints her canvases; with her bodyFootnote 12 she leaves traces after contact with earth or with sand as, for example, in the Silueta Series, Mexico, 1976.
“If a body is capable of preserving, by way of figurative memory, the traces and imprints of its sensory interactions with other bodies, then we can hypothesize that a subject of enunciation that would also be a body is likely to testify to its experiences”Footnote 13 (Fontanille, 2003: p. 1). The face that traces, the traced face, involves inscription and possible backward reading: “it is always a question of keeping track of an event, so as to be able, in an act of verification, to return to the origin” (Fontanille, 2003, p. 1).
Traces on or in the Face [Intra-Face: Introspection/Imbrication]
We now move from extrinsic traces that are dislocated with respect to the subject to which they belong and eventually return in a sort of spatial adjustment, to intrinsic ones that are an integral part of the operational framework of their subject. These kinds of traces can be found and understood mainly through their reading, which can be effected in various ways, for example, through physiognomy. “Physiognomy, the practice of reading the face, dates back to the Paleo-Babylonian period in Mesopotamia. Scholarly references can be found as early as the eighth century BC (Homer) and fifth century (Hippocrates), as can social references: Pythagoras reportedly used it as an interviewing technique, claiming to have his potential students physiognomized before agreeing to be their tutor” (Helfand, 2019, p. 153).
It is doubtful that the traces of a delinquent personality are genetically established and therefore transmitted from birth and scrutinized in the face from an early age, but it is probable that cultural, habitual, and event-related traces are decanted into the face. While in the former cases we speak of anthropometry and biometrics, in the latter we are closer to physiognomy and traces in the face as a residual indication.
In addition to distinguishing between types and temperaments physiognomy, as the art of interpretation, is concerned with giving voice to the traces present in the face not only so as to study their origin and thus give them a diagnostic role but also to possibly redeem their predictive role. Among the leading physiognomists, Della Porta certainly stands out, his juxtapositions of man-nature and, above all, man-animal enjoying great success between the middle and the end of the sixteenth century: his studies in the footsteps of predecessors Polemon and Adamantius compared the entire human figure and, above all, the face with animals that were similar in figure and character. In his second book, for instance, we find explanatory tables of the details of the mouth in comparison and assimilation to certain animals according to position and form; the author also goes into detail about the lips, mouth, teeth, and tongue; about yearning, sighing, and laughing; about the beard, neck, Adam’s apple, and cervical bones; and up to the interpretation of the voice and speaking. Only in book number three is the subject of the eyes the exclusive theme of the entire manuscript, which is characterized by numerous references to the previous studies of Aristotle and Galen: the constitution, size, and shape of the eyes, the consistency of the angles, the upper and lower eyelids, the pupils and their degree of hydration, the color and their location in the facial set, and the attitude to the surrounding space (open, trembling, pleading, oblique, twisted, threatening) are analyzed.
“Laughing eyes, of firm and menacing aspect. But if with a threatening countenance, and with steady eyes they regard, they counsel iniquitous deeds. Adamantius. But Albert. Mainly by these signs the eyelids sometimes draw close, and some separate, there is for sure the sign that they have iniquitous thoughts in their hearts”Footnote 14 (Della Porta, 1995, p. 218). He was followed by Petrus Camper, a Dutch naturalist and biologist, and Johann Casper Lavater, who in eighteenth century Switzerland studied and represented human qualities in the form of a map.
The tendency to measure has always been inherent in the perfect geometry of the human being: from the very beginning, exactitude has constituted an ideal to be aspired to (think of Leonardo’s Vitruvian man and the harmonious inscription of the details of the body), with indications being proposed concerning supposed normality and certain quantitative parameters within which to fall in order not to be deviant.
Above all, this normalizing determinism, a vocation established mainly in medical and police circles, was based on an arbitrary deliberation of averageness: “Inspired by the writings of Adolphe Quetelet, a Belgian statistician who had introduced in 1835 the concept of what he called l’homme moyen (the average man)—a standard-bearer against whom the general population could be effectively compared—the young Bertillon chose to devote himself to the study of a more precise visual observation” (Helfand, 2019, pp. 17–19).
And if in some cases the identification of anomalies could suggest illness, discomfort, or disease, in many cases moving away from established normality meant approaching the suspicious and different otherness often attributed to criminality. Indeed, Bertillon became an expert in the judicial area by coining what was to become known as the Portrait parlé, the spoken portrait. Looking at a copy, we can see that it is a file in which, in addition to the photographs (front and profile) of the accused, there is a series of items of information divided into:
Anthropometric observations (body and head measurements)
Color information (eyes, beard, hair)
Descriptive information of profile analysis (contour, forehead, nose, right ear, lips, and chin)
Descriptive information of face analysis (contour, eyelashes, eyelids, mouth, corpulence, clothing, others)
Each of these categories was in turn subdivided into the articulation of detailed volumetric formulations. A few years later, Sir Francis Galton created an anthropometric laboratory, the purpose of which was “For the measurement in various ways of Human Form and Faculty.” Other experts in anthropometry specialized precisely in criminals: Cesare Lombroso and Earnest A. Hooton, for example, shared the idea that one could be a criminal from birth and that this was recognizable from biometric features.
Such pseudoscientific foundations, then as now, incur a number of anthropological, ethical, and semiotic problems. The informativeness of the reading codes established in such practices potentially nurtures racial tendencies and stereotypical extremism. The information conveyed is pre-established by the positional authority of questionable and hierarchical knowledge; in the face would therefore be written what is right and what is wrong in a delicate and imputable axiological line.
With the coming of the world wars and the misuse of information from phrenology and physiognomy for racist speculation, these pseudosciences degenerated almost completely. Both, in fact, initially developed as sciences of facial observation and as integrations of the qualitative aspects aimed at specific reading and interpretation and then became inflections of a limiting tendency more linked to the biology of the face than to the traces of its becoming.
In recent years, however, we seem to be seeing a revival and rapprochement with physiognomy thanks to new technological supports. Going back to the examples given in the first part, we have seen how, to the naked eye, a hair or the residue of lipstick left on a cigarette butt tell us nothing, while the use of technology, hybridizing with human design skills, has made it possible to interpret and decode what in the past could be tackled only by highly specialized observers. There are currently a number of innovative projects concerned with reading what the human eye is not, or is no longer, capable of doing.
Numerous apps for smartphones fall into this category, such as Face2Gene which, generated by Boston’s FDNA, now seems to be able to recognize genetic diseases such as Cornelia de Lange, Angelman, and Noonan syndromes with relative accuracy (around 65% of correct diagnoses): after uploading a photograph of the patients into the program, phenotypes are extracted that serve to reduce the possibilities of interpretation and offer support to the diagnosis, which will then certainly have to be supplemented and possibly confirmed by targeted DNA tests. So-called deep learning consists of wide-ranging algorithmic teaching: enormous quantities of data are entered into the machines, and consequently the variants resulting from their crossing are also far-reaching (Airoldi & Gambetta, 2018).
Human observation and tactile and presential precision, even in medicine, are rapidly being supplemented and partly replaced by artificial intelligence. Some holistic sciences still use them today to deepen the psychophysical situation and thus facilitate its improvement and rebalancing. Perhaps this inversion is part of the so-called material turn, in which the return to the signs caused by the passage of an energy, or in material connection – albeit indirectly – with other original signs, takes shape and meaning again.
Guglielmini proposes a distinction between fixed and “temporary” traits. The former are “part of the physical constitution and show the genetic heritage inherited from one’s family of origin. (…) These permanent traits, such as the shape of the face or the size of the ears, reveal our basic nature; they cannot be changed, they can only be fully developed or repressed” (Guglielmini, 2012, p. 8). The latter “are temporary changes that appear on our face and represent the psychophysical state at a given time in our lives. Once that time has passed, the face resumes its usual appearance. (…) Finally, the ‘mobile’ features are the most obvious, changing and expressive part of our face. The face contracts and moves incessantly and with its mimicry, conscious or unconscious, manifests the emotions we are feeling” (Guglielmini, 2012, p. 9).
Thinking about the inscription of these signs, but above all their legacy on the face, means considering the presence of absence, a trace that takes on character and expressive features, an attitude made up of sedimented gestures whose material transformation can be examined. It also implies reopening the connective possibilities of epistemological spaces that are distant but ontologically coexistent: semiosis, a kind of bridge between the two foundations, connects existence with knowledge and makes sense of it.
Finally, it is important to remark that the face’s organic exteriority, normally contemplated as the skin tissue formed by the outer epiderma and the underlying dermis, is a complex organ and part of the peripheral sensory mechanisms: “The haptic system uses sensory information derived from mechanoreceptors and thermoreceptors embedded in the skin (“cutaneous” inputs) together with mechanoreceptors embedded in muscles, tendons, and joints (“kinesthetic” inputs)” (Lederman & Klatzky, 2009, p. 1439).
The traces belong to a haptic influence of two or more bodies that meet and leave the echo of their contact in the form of an aesthetic, narrative, epidermal seal. The physiological role of this layer, as an account of all that surrounds it and the counterpart of what it excludes and that from which it is excluded, is that of a frontier: its elasticity allows the malleability needed in acting as a transition. The cutaneous receptors which densely inhabit the face regulate its sensitivity, temperature, excretion, and absorption. “The body—world open to the world—just as it continually gives off substances of different significance and consistency, also continually incorporates other substances: air, dust, sound, light or shadow. Subjected to the siege of what comes upon it, the body needs the senses so that the absorption is gradual and the world, therefore, is not only bearable but as far as possible physical or at least always significant. Thus, the senses, at the same time as they open the body to the absorption of the world, protect it from it, organize its defense”Footnote 15 (Dorra, 2005, p. 107).
The marks on the face, provoked or received without choice, mark a contact with otherness that is solidified not only in action but also in permanence. Jorit is the pseudonym of an Italian street artist known for his monumental graffiti scattered mainly around the city of Naples, where he lives. After travelling to Africa and approaching African culture, he decided to include a sign of recognition of African origin in his hyperrealistic paintings. A trace of pigment in his portraits, the two lines that cross the cheeks of his characters “refer to the African practice of fleshing out, to symbolize the unity of the tribe as opposed to the singularity of the individual.”Footnote 16 In 2020, the artist decided to make a gesture that was not representative but embodied, having the stripes imprinted on his cheek, in a sort of personified ritualization and subjective identification of the nucleus conveyed by his works: the tribe. This permanent performance resulting in a face engraved forever opens its body to a form of empathizing with its creatures where the trace also becomes a symbol.
Traces Erased Toward New Tracks [Bio-societal Propagation/Recognition]
Assuming the examples shown until now to be part of a polyphonic semiosphere, we can observe how some of them flourish from the tangible but often ignored traces left everywhere by humans during daily life, while others are instances completely imbricated in the face itself. “To move oneself or have self-consciousness is in effect to refer oneself to oneself, to be an origin. Then a subject-origin which is also a subject of flesh and blood becomes problematic. The effort is made to understand it on the basis of an incarnation as an avatar of the representation of oneself, as a deficiency of this representation, the occultation of a translucid and spontaneous consciousness into receptivity and finitude” (Levinas, 2006, p. 78).
Photography, an index by definition, is a trace of the subject filtered and transmitted in the form of light. The portrait, a genre par excellence in which the face is the protagonist, becomes a trace of absence if the face is dulled or even erased. Dondero proposes the analysis of a visual apparatus in term of forces, where it is possible to distinguish intensive and extensive weights: “Here, it is a question of intensive magnitudes, and the forces which are at play consist in rhetorical operations of subtraction or inhibition. In what concerns extensive forces, it is necessary to be able to conceive of negative space: The negation of a portion of space consists in the suspension of a sensible quality which is normally associated with a certain presence in such space. In other words, negation can manifest itself through a rupture of the dependency between portions of space and sensible qualities” (Dondero, 2020, pp. 40–41).
Numerous artists use the medium of photography and post-portraiture to address, argue, and represent this posture. In Saraí’s case, the face made memory through photography is colonized by an invasive, clear-cut, tabula rasa erasure of faces. The contextual traces, which are often still visually present, constitute the foundations for the gaze to still exist despite its absence: the severed faces conceal an intimate conversation within themselves, as if the gaze, previously turned outward, has performed a horizontal rotation of 160 degrees, moving the forces inside and showing us the back. A back that is not the rear portion of the head but a new face: in the white there is absence and infinity, in the negation of the gaze a new imagined density (Figs. 21.3 and 21.4).
In the emerging texture, we cannot avoid the simulacral reference, the cathartic cancellation, and the plasticity of a gesture substituted in a form exemplifying reality. Like a kind of magic, the stroke replaces the action and allows, at least visually, a complete semantic restructuring: “both simulacrum and copying are related to magic, with the only difference being that, in the first case, it is a magic of substitution and in the second, a magic of resemblance” (Stoichita, 1999, p. 33).
Taking as our reference the compactness of the present body but partially emptied of its receptive and productive formal and politico-social accumulation and stratification, in this last part, we deal with the erasure of traces, the backward path whereby from a trace of absence, one can eventually trace a new presence, a significant constitution of the same blood in a new imbricated and shared name.
The Janez Janša artistic collective is surely integrated in a kind of action where traces are emerging as symbolical and pragmatical signs in order to test, destabilize, and reorganize, in an innovative way, parts of the cultural complexity. Composed of three people, the group worked for years on an artistic piece that has gone down in history as emblematic evidence of contemporaneity and has been profoundly explained in a film directed by one of them. “My Name is Janez Janša is a documentary film about names and name changes, focusing on one particular and rather unique name change that took place in 2007, when three artists officially changed their names into the name of the Prime Minister of Slovenia, Janez Janša” (Figs. 21.5 and 21.6).Footnote 17
“Jaz sem Janez Janša,” the reverberation of the sentence as a mnemonic stabilization of the self, is associated with ancestral practices, in some cultures called mantra: the continuous repetition of certain sounds allows them to be internalized, whether they are morphemic or phonemic sets. A new identity needs to be impersonated by the three protagonists, and the incorporation of the new name is one of the fundamental steps to be carried out. “The name is probably the most important marker of an essential human being, that we have in language. (…) There would be other markers, that are not linguistic, in visual language, but in written language and in spoken language, the name is the most important thing” (Catherine M. Soussl in My Name is Janez Janša).
Other artists have resorted to executing a similar performance: for example, on 5 October 2007, the artist Kristin Sue Lucas decided to go into what she called “the utmost conversion of myself.” The honorable Frank Roesch confirmed that she changed her name in the spirit of “refreshing herself”: in effect, the artist waded through a long bureaucratic, political, intimate mechanism to foster a symbolic change and reiterate her own name, thus engaging in subjective linguistic isomorphism and identifying traces.
The Janez Janša go through their autobiographies to rewrite themselves in a collective and constant political act; at the end of the documentary that tells their story, an infinity of people repeats Jaz sem Janez Janša in a sort of emphatic process. A biography of them has also been written despite the fact that, during the documentary, Caronia tells us that “Biography is a destabilizing colonization operation because it constructs a referent that does not exist. The life of Janez Janša, if I read and try to interpret literally what is written in this book, I will never be able to find the real referent who is a totally imaginary character, whose life derives from the intersection of the three lives of three real characters” (Fig. 21.7).
The writing of the biography can be considered a partial achievement, because it implies that the subject about whom it narrates exists, albeit in symbolic terms: in fact, this subject has been constituted after overcoming an articulated pragmatic narrative program. Interpreted according to generative semiotics, we might think that the constitution of the new hybrid identity can be considered the object of value accomplished. The initially virtual modalities, in which duties and wishes coincided (wanting to change name and therefore having to face a whole series of sociopolitical but also intimate/mathematical vicissitudes), immediately intersected with actualizing modalities. The latter category includes both knowledge (whether intrinsic or acquired along the way) and power (that set of endogenous or exogenous possibilities).
The virtual and actualizing modes can give rise to a third mode, in which the conjunction of the subject with its object of value, the new shared name, is achieved.
“The naive interpreter reads every projection as an imprint, i.e. as a direct transformation from the properties of a real thing; whereas projection is always the result of transformative conventions whereby certain traces on a surface are stimuli that prompt one to transform backward and postulate a type of content where in fact there is only one occurrence of expression. Thus it is possible to project from nothing or from contents to which no referents correspond”Footnote 18 (Eco, 2016, p. 370).
It is worth noting that the analysis of single signs is always part of a broader reading in which the crucial factor is the discursive whole, which also includes the ideological and cultural imbrication of the interpretations involved. There thus seems to be a kind of abductive logic in the genesis of the trace: a sign, as index and trace, acts as a mediation between two or more dimensions which, as envisaged by the abductive logic, come together by finding or creating the semiotic thread that sustains them: “at first sight abduction seems more like a free movement of the imagination nourished by emotions (like a vague ‘intuition’) than a normal process of decoding. (…) But the abductive movement is accomplished when a new sense (a new combinatorial quality) is assigned to each sound (or every kind of sign, note of author) as a component of the contextual meaning of the whole piece” (Eco, 2016, p. 211).
The proposed argumentations are thus also products of a combinatorial positioning that does not appeal to precise codifying rules but rather leverages the continuous performative possibility of form and track: “Underlying and beneath every semiotic impulse is the aesthetic pleasure of discovery” (Leone, 2020, p. 31).
Original text: “Dans la perspective d’une sémiotique du texte et du discours, le corp est d’abord un lieu de signification, et d’une signification qui prend forme à partir de sensation et impressions que ce corps éprouve au contact du monde.”
Original text: “la matière est soumise à des forces, dont l’équilibre lui procure une forme.”
Original text: “il rapporto tra il filo—il filo del racconto che ci aiuta a orientarci nel labirinto della realtà—e le tracce.”
Semiotic square presented during FACETS program “Chronotopes of the face n. 9”: https://www.youtube.com/watch?v=MANCsmJrdz0
Original text: “a mostrare in pratica la fallacia del mito della neutralità algoritmica, cosi come a esaminarne le varie sfaccettature discorsive. In particolare, tre aspetti cardine del mito verranno discussi e problematizzati (…): primo, l’assunto che nella fase di progettazione e/o addestramento dell’algoritmo non abbiano interferito distorsioni sistemiche e che, pertanto, l’output restituisca un punto di vista oggettivo sulla realtà; secondo, l’idea che l’automatizzazione e la disintermediazione di processi complessi siano di per se stesse garanzie di neutralità; terzo, la pretesa che i risultati del calcolo algoritmico, narrati come accurati e infallibili, non alterino la realtà secondo logiche socialmente, storicamente e politicamente determinate” (Airoldi, Gambetta, 2018: 33–34).
To emphasize this possibility of error, we again recommend delving into the text by Airoldi and Gambetta (2018: 37–38) who, in a passage devoted to application fallibilities, give the example of the COMPAS software (Correctional Offender Management Profiling for Alternative Sanctions) and the dataset used by Wu and Zhang.
To get acquainted with the work of Mendieta, we recommend visiting the page www.guggenheim.org/artwork/5221 or https://www.malba.org.ar/ana-mendieta-alma-silueta-en-fuego-1975/.
Original text: “Si un corps est susceptible de conserver, au titre de la mémoire figurative, les traces et empreintes de ses interactions sensorielles avec d’autres corps, alors on peut faire l’hypothèse qu’un sujet d’enonciation qui serai aussi un corps est susceptible de témoigner de ses expériences.”
Literary translation from old Italian. Original text: “Occhi ridenti, di fermo, e minaccevole aspetto. Ma se con volto minaccevole, e con fermi occhi riguardino, consigliano iniqui opere. Adamantino. Ma Alberto. Principalmente sì con quelli segni le palpebre alcune volte li giongono, & alcuni si separano, s’hà per certo segno che hanno nel cuore iniqui pensieri.” (Adamantius and Albert where two philosophers with different opinions about laughing eyes.)
Our translation from the original text: “El cuerpo—mundo abierto al mundo—así como continuamente desprende sustancias de diversa significación y consistencia también continuamente incorpora otras sustancias: aire, polvo, sonido, luz o sombra. Sometido al asedio de lo que viene sobre él, el cuerpo necesita de los sentidos para que la absorción sea gradual y el mundo, por lo tanto, sea no sólo soportable sino en lo posible fórico o al menos siempre significante. Así, los sentidos, al mismo tiempo que abren el cuerpo a la absorción del mundo lo protegen de él, organizan su defensa.”
https://www.napolitoday.it/cultura/jorit-segni-faccia-tatuaggio.html Last consultation on 26th of April 2021.
Original text: “The naive interpreter reads every projection as an imprint, i.e. as a direct transformation from the properties of a real thing; whereas projection is always the result of transformative conventions whereby certain traces on a surface are stimuli that prompt one to transform backward and postulate a type of content where in fact there is only one occurrence of expression. Thus, it is possible to project from nothing or from contents to which no referents correspond.”
Airoldi, M., & Gambetta, D. (2018). Sul mito della neutralità algoritmica. In A. Martella, E. Campo and L. Ciccarese (Eds.) Gli algoritmi come costruzione sociale (pp. 25–45). LQ The Lab’s Quarterly a.XX.n.4.
Barth, R. (1999). Variazioni sulla scrittura. Seguite da Il piacere del testo. Einaudi. (Original work pubblished 1973, Ed. Seuil).
Binda, E. (2017). Interagire con l’ambiente. In S. Capezzuto, D. Ciccone, & A. Mileto (Eds.), Dentro/Fuori. Il lavoro dell’immaginazione e le forme del montaggio. Ebook: Il lavoro culturale.
Della Porta, G. (1995). Della fisionomia dell’ huomo. Bibliotèque numèrique.
Dondero, M. G. (2020). The language of images. The forms and the forces. In A. Sarti (Ed.), Serie lecture notes in morphogenesis. Springer.
Dorra, R. (2005). La casa y el caracol. Plaza y Valdes Eds.
Eco, U. (2016). Trattato di semiotica generale. La nave di Teseo.
Fontanille, J. (2003). Quand le corps témoigne : approche sémiotique du reportage. Documents de Travail 329. Università degli Studi ‘Carlo Bo’.
Ginzburg, C. (2020). Il filo e le tracce. Vero falso finto. Feltrinelli. (Original work pubblished 2006).
Gramigna, R., & Voto, C. (2020). Semiotica, prossemica e contagio. Il senso delle distanze ai tempi del Covid-19. In M. Leone (Ed.), Volti virali. FACETS Digital Press, Open Access.
Greimas, A. J., & Courtés, J. (1982). Semiotics and language. An analytical dictionary. Indiana University Press. (Original work pubblished 1979).
Guglielmini, A. (2012). Il linguaggio segreto del volto. Ebook: Piemme.
Helfand, J. (2019). FACE. A visual Odissey. The MIT Press.
Jodorowsky, A. (2008). La danza della realtà (M. Finassi Parolo Trans.). Feltrinelli.
Lederman, S. J., & Klatzky, R. L. (2009). Haptic perception: A tutorial. In attention. Perception & Psychophysics, 71(7), 1439–1459.
Leone, M. (2020) Colpire nel segno. La semiotica dell’irragionevole. I Saggi di Lexia 38: Aracne.
Levinas, E. (2006 ). Otherwise than being or beyond essence (A. Lingis Trans.). Duquesne University Press. (Original work published 1981).
Lotman, J. (2011). The place of art among other modelling systems. (in 3rd volume of Trudy po znakovym sistemam, 1967). Sign Systems Studies, 39(2/4), 249–270.
Magli, P. (1995). Il volto e l’anima. Fisiognomica e passioni. CDE.
Stoichita, V. I. (1999). Breve historia de la sombra. (A.M. Coderch Trans.) Madrid: Siruela. (Original work published 1997).
Editors and Affiliations
© 2022 The Author(s)
About this chapter
Cite this chapter
Barbotto Forzano, S. (2022). Trace and Traceability in/of the Face: A Semiotic Reading through Art. In: Comunello, F., Martire, F., Sabetta, L. (eds) What People Leave Behind. Frontiers in Sociology and Social Research, vol 7. Springer, Cham. https://doi.org/10.1007/978-3-031-11756-5_21
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-11755-8
Online ISBN: 978-3-031-11756-5