Keywords

1 Introduction

We live in a world permeated with signs that belong to multiple areas of knowledge. Signs which have been filtered and stored by our natural and artificial interfaces can determine our perceptions and conceptions about the complexities of the world. The proposal of this study is to present poetic experiments that sustain a consistency between what is real and what can be defined as semiotically real [1, 2], and that sustain an efficient behavior but, in being poetic, exert a degree of tension on this close relationship. The exploration of the limits of the conformed thought - understood here as a composition of codes, standards, patterns and cultural representations – expands and reveals a variety of settings that are not readily perceived in the world. It is important to note that we are referring to thought that is “conformed” and not “shaped”. The two words can be synonymous in some cases but here we mean that we are not restricted to forms, appearances, expressions of patterns, and we include the sense of model, where form and content are indissoluble and conceptual characteristics are inseparable from the materials. Furthermore, conformed thought is the thought or action that is determined by the prevailing standards, attitudes, practices, etc., of society or a group.Footnote 1

There must be consistency between the things of the world and “things” as we conceive of them (known as signs) that we define as reality; otherwise, we could not act in the world. Nothing would work if signs and things did not maintain a close relationship. It would not even be possible to cross a street if things and signs were not closely related.

Based on this assumption, we can presume that we will always reach a fragmentation of the real, as we are always accessing the things of the world through signs. In addition, signs become habits, converting themselves into states of affordable things – which we define as objects.

Our understanding of the world becomes conformed through a combination of irreducible relations between all that contributes to the physical environment where we live and the organisms that share the same environment, and the ways in which this physical and interpreted environment incorporates the same significant sphere of existence shared by all that therein has any sense [3]. We mean that signs are in themselves multiple layers of meanings shared within this sphere.

What we can say is that the ability of “human beings” to produce signs is fed back to the environment, which in turn forces the evolution of man to create increasingly complex systems of interpretation. Therefore, new signs are generated through feedback, making us look at and recognize the objects in the world in many different ways (i.e. targeted things).

According to John Deely, based on Jakob von Uexkull’s Umwelt concept [3, 4], there is also

a distinction between sensation (as the action of the environment upon the animal body objectifying certain aspects only of the surroundings) and the higher-level perceptual response to that stimulus (wherein the data of sensation, never atomic but already a complex and multiple network of naturally determined sign-relations, wherein differentiations of light reveal also shapes, positions, and movements, etc., are further structured into objects of experience) [ 5 ].

This study will identify some aspects of these reconstructed objects of our conformed thought into things that can engender new sensations and responses in perceptual terms, in the form of images, sounds, and haptic (tactile) feedback.

This can be easily illustrated by the classic effect generated by the figure below:

The Stroop test was developed by J. Ridley Stroop [6] to evaluate the reading process automation. The task is to name the color of the printed words, and not actually read them, as the actual words do not denote the name of the color of the printed word. In general, the use of words strongly interferes with our ability to identify the actual colors. Thus, the interference generated in the processing of the divergent information (the word itself and the color of the word) by the brain creates a conflicting message. Neuroscience attempts to explain the reasons behind this phenomenon, but for us the important thing is to focus on the interference and the effort evoked here. The interference and the effort required demonstrate that we are invariably being guided by habits and beliefs, despite not always being aware of it. This entails that every abstract thought should create an interference of some sort in the way we perceive and act in the world (See Figs. 1, 2, 3 and 4)

Fig. 1.
figure 1

Image test to Stroop effect

Full size image
Fig. 2.
figure 2

Still of artwork installation “(-1) x (-1) = + 1 - An enigma for Flusser”

Full size image
Fig. 3.
figure 3

Still of artwork installation“⨏(Δ t) – An enigma for Bergson”

Full size image
Fig. 4.
figure 4

Still of artwork installation “Φ - An enigma for Gibson”

Full size image

The work by Janet Cardiff and George Miller entitled Alter Bahnhof Video Walk, presents another important example related to this matter:

The Alter Bahnhof Video Walk was designed for the old train station in Kassel, Germany as part of dOCUMENTA (13). Participants are able to borrow an iPod and headphones from a check-out booth. They are then directed by Cardiff and Miller through the station. An alternate world opens up where reality and fiction meld in a disturbing and uncanny way that has been referred to as “physical cinema”. The participants watch things unfold on the small screen but feel the presence of those events deeply because of being situated in the exact location where the footage was shot. As they follow the moving images (and try to frame them as if they were the camera operator) a strange confusion of realities occurs. In this confusion, the past and present conflate and Cardiff and Miller guide us through a meditation on memory and reveal the poignant moments of being alive and present. [ 7 ]

In this way, the habit of using a camera guides the experience and the way we interpret a place. Even though we are looking at pre-recorded images, we are able to navigate through different dimensions even as we stand in the same place, which results in a different sense of reality. The practice of “selfies” also conform to our present thought-process. Moreover, and this is precisely our point, the mediums conform our thinking, and the habit of this conformed thought becomes a unity of feeling for new signs and new thoughts that are formed in a continuous semiosis. Therefore, there evidently is a close relationship between feeling and thinking.

In order to achieve our goal we will base ourselves on the artwork developed by the Realities Research Group (www2.eca.usp.br/realidades), at the School of Communication and Arts of the University of São Paulo - SP.

This group explores concepts arising from the relationship between art and science. From virtual to augmented and mixed reality environments; from cybrid modalities to ubiquitous computing; from representations to emulations and simulations, we can find a broad spectrum that contemplates games, websites, interactive art and installations, and that challenges and redefines the concept of reality.

In an era characterized by increasing complexity, we question how to coherently deal with the systems that grant us access to the ‘semiotically real’ [1, 2, 8, 9] and which, for now, will be referred to as ‘reality’. This research group periodically aims to:

  1. a)

    Compile a significant number of documents and create a collection on this subject, with the intent of contributing to and supporting the development of several Master dissertations and Doctorate theses;

  2. b)

    Produce primary research sources (interviews and testimonies of artists and those who are part of the artistic and cultural milieu, whose trajectories are relevant to art and technology studies), which will be released on the Internet or made available for the academic community under any other format that grants free access to these researchers;

  3. c)

    Produce poetic experiments as a practical laboratory for research.

The exploration of this aspect of the sign has been the research goal of the Realities Group and we would hereby like to share our results and experiences. The members of the Realities Research Group have developed poetic experiments since 2012, questioning this conformed thought - such as representation by reflection, familiar recognition of faces in the mirror, the inversion generated by video cameras, webcams and by face recognition software programs, as well as in “selfies” used in social networks. One particular artwork also examines the habit acquired by the camera and by film and video. When using a webcam, we are guided by a distinct point of view, which nonetheless bears familiarity to the reflection in the mirror, acting upon us in ways that we may not be aware of.

2 Poetic Experiments of the Realities Research Group: Three Enigmas

2.1 “(-1) X (-1) = + 1 - an Enigma for Flusser” (2012)

The artwork “(-1) x (-1) = + 1 - An enigma for Flusser” is an interactive installation with three different images overlaid in the same two-dimensional space: the image of a mirror, the image captured by a camera, and the image calculated by the computer. The dialogue between camera and mirror makes the visualization of the three layers of images and their different contexts possible. A camera positioned above the monitor aims to simulate a mirror on the screen, but in this case, the mirror image is not reversed, and its colors have been altered (via software generation) in a negative way (-1).

When this reverse image is returned as a reflection in the mirror, it remains negative and mirrored (-1). This way, the camera acts as a false mirror; on the other hand, the mirror itself acts as a disguised camera.

In the third layer of the image, we can recognize ourselves due to the resemblance of the simulation of a mirror (+1). Added to this, the software is equipped with a system of facial recognition which also recognizes us, or rather, recognizes the human face.

Since childhood, we are taught that we are the image that we see reflected in the mirror, and that it is the representation of ourselves. That is, the representation of ourselves has always been reversed. When standing in front of a mirror, we suppose that as we raise our left hand, the right hand of the image that is reflected will be raised. This is exactly what happens, it has always been this way, and it is natural. Proof of this is that we find it strange when instead we see an image where the left hand is raised - because we expect the right to be raised.

In the feedback process, the image becomes positive and mirrored. At this point we recognize the “real” when we perceive the “(i) real” text which is now also mirrored. This is the point where the image acts like a mirror, in a way that is so familiar to us. These layers of images represent the process (-1) x (-1) = + 1.

Flusser [10] stated that the technical image is the result of a text. Here the text triggers the (i) reality of the image, at the time that (-1) x (-1) can be (+) and/or (-) 1. Therefore, the principle of reality is questionable, and the relationship between image and text demonstrates this conflict. The image with multiple layers challenges the precepts of exact truths. Thus, the system operates as a “meta-Flussean” apparatus.

We could ask: at present, is the mirror image our most familiar image? Webcams and cell phones are rendering pictures of ourselves that are being distributed by networks and stored in clouds, reaching more people and at greater distances than any mirror could ever achieve. Furthermore, “selfies” are superseding our pictures as representations of events and locations, in ways that no mirror could have done before. Therefore, today, a mirror image may not be the most familiar image of myself that I can actually have. This, in turn, generates a series of changes, including sensory changes.

2.2 “⨏(Δ T) – An Enigma for Bergson” (2013)

The artwork “⨏(Δ t) – An enigma for Bergson” is an Interactive installation with a projection that is a feedback image from a camera, constituted by merging images with a transparency percentage according to the specific time they were captured. Every time an individual image is captured by the camera and exhibited in real time, the system saves it to a file, where it is immediately overlaid by the forthcoming image, and so on, in eternal circularity. Every captured image is displayed on the screen over the camera image, which is still capturing the screen - with a specific percentage of transparency. The overall effect is an apparent movement that also moves between volumes and transparencies generated by the overlays. That is a motion that occurs among the various layers of overlapping images captured and recorded. As a result of these marks of time, greater stability is achieved, and in turn, the greater the durability of the images in the system. Greater instability, on the other hand, inevitably leads to the disappearance of the image.

Delta-T is an abstract concept for calculating the time difference obtained by subtracting Universal Time (UT) from Terrestrial Time (TT). Universal Time is a timescale based on the rotation of the Earth, which is somehow irregular because there are forces that can change the rate of rotation of the Earth. This means that environmental factors should be taken into consideration, revealing a relationship between dependent systems, even though they are autonomous. The systemic dependence was used poetically in this artwork, because the transparency percentage applied to each overlaid image depends on the specific moment when it is exhibited. As a result, this creates an effect where the flows of duration and permanence of the image will appear to have different speeds during each passing hour. Feedback to the system with the location in the solar system creates a dependency between systems - artwork and its environment, which also generates perceptual changes from this relationship. On the other hand, this is another way to visualize changes in time we are not used to doing and this representation model suggests new experiments for the space-time dimension.

2.3 “Φ - An Enigma for Gibson” (2014),

The artwork “Φ - An enigma for Gibson [11] ” is an interactive installation that sets two video cameras in the same place, generating images that are captured by a computer in real-time and manipulated through Processing Software. The captured light is decoded in straight and curved monochromatic lines by the computer, creating an abstract representation of the physical space. The important point here is not the interpretation of shapes and colors but the image constructed by the variance and invariance of light in space. The luminous flux (Φ), measured in lumen (lm), is responsible for the variations of the image, causing the effect of another space, not the one we are used to, but the one that comes to us in a synthetic and rational way. One that uses different kinds of lines to build itself, immediately referring to the drawing, the design. By applying filters to the image captured by the cameras in real-time, the difference in brightness between each segment determines the changes that may occur in the screen area and the system draws contours with lines of minor thickness, mapping spatial structures in time. The tracing creates maps that are intertwined, generating new representations and visualizations of three-space data.

We explore the representation of space through cameras, charts and graphs. They are traces that create mappings that interweave, bringing out new representations, rendering maps and scanning, and data visualization of these spaces. This is also a real-time data calculation experiment, and that generates a combination of drawings, diagrams and records of physical space lights, creating an effect of augmented reality, although not actually use apparatuses for this kind of representation as is understood today by the technological area.

3 Considerations

In conclusion, we can say that there are sensory elements in the conformed thought; all abstract thought has the power to generate an interference of some sort in the way we perceive the world; and that there is a close relationship between the things of the world and signs. Consequently, our relationship with the world depends on our relationship with our surroundings, an expanded Umwelt formed by a complex network of interlaced interpretations of things, objects and conformed thought. Finally, as the Umwelt acts as an interface that selects and filters information of the environment and internalizes it in encoded form, any material used by a living system to build knowledge has a representational condition, i.e., is formed by a myriad of ‘somethings’ that represent ‘external somethings’, which are rendered into ‘particular somethings’ of their cognitive system [12–14]. This feature interferes with our perception of the world, and transforms our senses, body and mind in an indissoluble way.