Robots and Art pp 229-254 | Cite as

Still and Useless: The Ultimate Automaton

Part of the Cognitive Science and Technology book series (CSAT)


Robots descend from the long genealogy of automata, machines with no practical purposes essentially meant to simulate objects embedded with an anima. Our hypothesis is that the thrust for the creation of every robot is rooted in the primordial myth of infusing inanimate matter with the breath of life: the aim of any automaton is to become a living thing. The ultimate automaton does not need to move or to do anything: the essence of any robot lies in the desire to simulate life to the point where it actually becomes alive. This chapter presents the Aerostabile research-creation program, which progressively evolved from an architectural origin to a research platform for exploring the nature of the elements that maximizes this deliberately created illusion. It goes through the origins and main methodologies of the program, then describes several artworks that were created along its evolution, focusing on the notion of behaviour and observed interactivity.

Automata and the Art of Life-Simulation

The proliferation of robots in all spheres of current life tends to obliviate the fact that they were, up to the beginning of the 60s, a tiny subset of the huge family of automata, so called because they are animated by an internal source of energy, from which they descend through a long and complex genealogy. It is all the most interesting to realize that the English word “automaton”, dated from the beginning of the XVIIth century, and the French word “automate”, dated one century earlier, do not only refer to movement: they have been coined from the same latin word automatus, itself derived from the Greek word automatos; auto refers to self, and matos has the triple meaning of moving, thinking and willing. Diving a little deeper in the past, it appears that matos itself comes from the much older Proto-Indo-European root *mn-to, from *men, “to think”—the same word that gave “mind” and “mental”.
Fig. 1

Two Tryphons aerostabiles in a state called “Paradoxical Sleep”, Chalet du Mont Royal (Montreal 2014). The aerostabiles are flying cubic automata with no anthropomorphic or biomorphic features. In this state, they stand  almost completely still in the air. To reach that result however, information and signal transfers in their electronic circuitry is frantic (Photo by Nicolas Reeves)

Etymologically speaking, “automaton” thus describes a machine that can not only move or work, but also think and will, three notions that are usually associated with beings infused with a mind: conscious living beings. The oldest known automata were made for purposes that were often quite far from what we expect from contemporary robots: during Egyptian, Roman and Greek Antiquity, as well as in the Japanese Edo era, they were created in order to simulate animated or living beings, in order to infuse a sense of awe or mysticism, or simply for amusement. In most of the cases, their designers, or the people presenting them, declared that they were moved by some kind of spirit of deity.

Robots appeared in the XXth century as automata of a specific kind. As it is well known, the word “robot” appeared for the first time in the 1920 R.U.R theatre play by Czech writer Karel Čapek (though the word itself was coined by his brother Josef). It comes from the Czech word “robota”, or “worker”, itself derived from a Slavic root that means “slave”. It conveys the status of robots as machines specifically designed to compensate for humans’ limited abilities in the execution of tedious, precise, dangerous, costly or heavy tasks. Such working automata began to develop at a large scale very late in history, at the beginning of the 60s. Before that, from the Renaissance on and all along the XVIIth century, automata were created mainly to simulate complex human or animal behaviours: playing music, writing letters, playing chess, eating, and even digesting and defecating, which resulted in some of the finest mechanical pieces of all times. The idea behind such attempts was to simulate life through its most complex manifestations: the precision of the simulation would reinforce the interpretation of the machine as a living organism. Smaller automata created for pleasure or amusement became very popular during the XIXth century.

If we except water clocks, whose origin is lost in the depth of times, the first automata specifically designed for practical purposes were most likely the 13th century early timepieces. All along their history, mechanical clocks remained intimately connected with the world of automata. Some of them, like elaborated Swiss cuckoos or James Cox’s extraordinary Peacock Clock (now at the Ermitage museum in Saint-Petersburg), were associated with animated characters whose sophistication reveal their belonging to the realm of automata. Even today, complex clockworks, such as the Supercomplication watch by Henry Graves, reach tag prices of several million dollars, an amount completely disproportionate for a device whose sole function is to indicate time, but begins to make (some) sense for an automaton artwork—a device that seems to be animated by a living process.

The first mentions of robots specifically made for the execution of tasks date from the 20s. Jacquard’s looms in the XVIIIth century had several characteristics of automata, but they were powered by human beings. This was also the case for the first computing devices such as Pascal’s Pascaline or Babbage’s machines, whose mechanism was directly inspired by Jacquard’s looms. Apart from the first computing machines such as the Zuse (1941), the ENIAC or the Colossus, the first practical device that fully deserves the name “robot” seems to be General Motor’s “Unimate”, put to work in 1961. Computing machines also belong to the category of automata, but they are unable to implement any physical task; moreover, they have a unique feature that distinguishes them from all others automata: their ability to simulate themselves, and to simulate automata that replicate themselves. They can contain all the information required to produce a copy of themselves, as well as the information to produce the devices required to implement these copies. This property of self-representation/self-replication is unique and important enough to provide a precise definition for a computer; here again, it is usually associated with living organisms.

The long history of automata, joined to our fascination for self-animated machines, gives to all of them a powerful mythical stance, which can be seen as the essential cause for their very existence and proliferation. Trying to communicate with objects made from inert materials can be seen as one of the manifestations of human’s primordial will to relate with every element of the world, even the non-living ones; and to convince themselves that they are not strangers in this universe that surrounds them. The obsession for the imitation of living beings does not only appear through robots, but was for long the object of many forms of art, from sculpture and painting to architecture. Automata is the realm where our impulse for animation, which fundamentally means the process by which a soul (anima) can appear spontaneously in an artefact, expands to include movement and behaviour. It is directly related to a wealth of ancient legends in which animated beings are created from inert materials, from Prometheus to Adam, from Frankenstein to Pinocchio.

Some of these attempts may seem utterly naïve to us, but their role in the development of science and technology cannot be neglected.1 Most of our contemporary technologies are related to mythological obsessions that can be traced to the oldest Antiquity: skyscrapers (a building like a mountain—the Babel tower), planes (flying—Daedalus and Icarus), rapid prototyping machines (fairy magic wands), internet (ubiquity)… The myth of a machine that simulates a living being to a point where it can be infused with life, thus transforming its creator into a demiurge, is at the root of the genealogy of about all robots and automata. It is thus not surprising that humanoids robots remain so popular and remain the object of so much research and experiments, despite the non-adequation of the human morphology and abilities for most of the tasks we try to delegate to robots: rationally speaking, the design of a robot should be optimized in order to implement tasks that they can do better, or more efficiently, than us. Humanoids shape are seldom optimal in that respect. This might be the best demonstration of the non-rationality of all attempts at creating artificial humanoids: such experiments reveal to which extent the field of automata, despite its new techno-scientific clothes, escapes rational logics by several aspects, and remains deeply rooted in the fields of mythology, poetics, and arts.

“Robotic arts” is the most common expression to designate artistic practices in which robots are designed implemented, or even hacked for the sole purpose of producing emotions, impressions and feelings, and for creating sense and signification through events that are originally senseless. In the vast majority of the cases, such practices should be more appropriately named “arts of automata”, since very few pieces of robotic arts are actually made to implement any kind of practical task. Arts of automata can be conceptually described as the process of eliminating all pragmatic or practical functionalities from a robot, in order to create a machine whose sole purpose is to trigger empathy, fear, amusement, compassion—the whole range of human feelings and emotions, including awe, just like the first religious automata. More than any robotic technology, robotic arts are rooted in the most distant past through their similarities with such very early attempts.

Emerging Emotions, Induced Feelings

The question to know what are the features of a robot that actually triggers these feelings is mandatory for our research programs. It is a vast and important topic which is the object of an increasing number of studies.2 Obviously, robots with humanoid features have an advantage: the interpretation of their movements and facial expressions is facilitated by our own acquired knowledge and culture. Even very approximate simulations provide enough clues for an observer to find out the meaning or message they try to convey. In the medical field, several experiments use human-face robots for therapeutic purposes, e.g. for helping people with mental disorders such as autism or Asperger3: the exact repetition and predictability of their reactions creates a safety perimeter within which these patients will take the risk of attempting a relation with them. It may however be easily observed that such features are not necessary for generating emotions. For instance, most of Bill Vorn’s machines have no face,4 and adopt a heavily industrial aspect. Every element of their morphology is inspired by working robots; their appearance is often more hostile than welcoming. Their expressive power is nonetheless undeniable.

In every respect, the expressive power of an automaton depends not only on its morphology, but also on the number of configurations it can take. Each configuration (“state”), as well as every transition between these states, has the potential to trigger specific emotions in the observer. At first glance, this seemingly reductive statement can be seen as limiting their expressivity: being mechanical devices, automata cannot compete with biological organisms at the level of the number, variety, precision or subtlety of their movements. Strangely enough though, several robots with a very limited number of states reach an astonishingly high level of expressivity, despite the fact that the observer is fully aware of their artificial nature.

Such observations naturally raise the question to know which features or reactions of an artificial device are at the origin of the emotions and feelings felt by the people that interact with them. This topic has been the object of an exponentially growing number of studies in the last years, especially in the HRI community5; most of them point out to the vast number of disciplines that are involved. One of the most famous and most quoted attempt at understanding the link between human emotions and robotic morphology is already old: Mori’s model called the “Uncanny Valley” links the nature of the feelings we experiment while looking at a robot to the level of resemblance between that robot and a human being.6 Mori’s hypothesis is unfortunately plagued by blurred definitions and a strong level of empiricism, which prevents it to be really useful even for planning an experimental protocol. The feelings that are listed (“negative feelings”, “revulsion”) are too vaguely defined to even allow the possibility of a metrics, mainly because the level and nature of feelings in front of any stimuli are not observer-independent: they are inextricably linked to the cultural origin of people, and to their personal history. Numerous observations and examples show frequent cases where automata with no biological or anthropomorphic features whatsoever do trigger feelings of empathy that can be stronger than the ones triggered by human-shaped or animal-inspired artefacts. Our own observations confirmed that the behaviour and reactions of an artificial system, especially during interactive processes, are much more important than any particular morphological feature. It is from our own researches that we came to this conclusion, along the development of a research-creation program called Aerostabiles, derived from a former research program on Self-Assembling Intelligent Lighter-than-Air Structures (SAILS). Before elaborating on this point though, some information should be given on the nature, history and evolution of one of our first robotic art projects, called Paradoxical Sleep.

An Architectural Origin

The purpose of the Aerostabile program is to design and implement automata that hover in mid-air and that are able to generate flying architectures by self-assembling themselves in flight. It was born from the desire to materialize another age-old myth, this time originating from the field of architecture: the myth of a heavy mass freed from the law of gravity.7 This idea can be found in several countries all along the history of architecture. Even today, to make a building like a castle or a palace fly in mid-air with its thousands of tons of stone or concrete is everywhere seen as the manifestation of a supernatural power. Some of the oldest mythological examples are the flying vimanas (Chariots or Palaces) mentioned in Ancient India; though their mention in literature is not rigorously attested, they are still the object of a lasting fascination, and some representations show them as seven-storey high flying buildings.

Besides such imaginary structures, immense efforts were made across history to give stone, concrete or steel the appearance of weightless materials. Seen from inside, at certain times of the day, the dome of Hagia Sofia in Istanbul seems to hover on a layer of light. The contrast between this phenomenon and the bulky, massive appearance of the church is striking. It reveals the amount of energy and efforts that was deployed to create this effect at a specific, privileged place of the building. Six to seven centuries later, the architects of the Gothic period, especially those of the late Gothic, have developed an expertise in the use of stone that allowed them to use it to its very limits. Solid walls almost disappeared in order to maximize the penetration of sunlight, so as to best elevate the soul towards the weightless Heavenly City. The most striking example is the upper nave of the Holy Chapel in Paris (XIIIth century), in which the walls almost dissolve into huge glassworks separated by incredibly thin columns. Boullée’s XVIIth century imaginary monuments, like his cenotaph for Newton, were so tall and huge that they could not be realized with the materials and techniques of the time. They implicitly supposed the use of materials with an unseen relation to gravity. Right after the invention of flight, numerous projects or experiments were made on the theme of weightlessness by artists such as Malevich or El Lissitsky, or architects such as Krutikov,8 whose project for constructivist flying cities used flocks of helium blimps to suspend whole communities between the clouds:

Malevich believed that weightlessness constituted the highest aim of technology, and hoped that scientific advances would make free unpowered flight feasible, allowing cities to be placed as satellites floating in the cosmos.9

Without reaching such extremes, more familiar structures such as cantilever bridges or skyscrapers represent challenges to the limitations imposed by the physics of gravity and materials. Recent examples include buildings inspired from aeronautics, such as Jean Nouvel’s Guthrie Theater with its cantilever awning, or Calatrava’s opera in Valencia, where a huge, leaf-like structure seems to defy all laws of gravity and resistance of materials by seemingly hovering over an egg-shaped structure.

At its own scale, the Paradoxical Sleep project, derived from the Aerostabiles research program, inscribes itself in these mythological attempts to simulate weightless masses. At the origin of it is a very simple vision: to make a cube hover in the air as if it was levitating. The paradox between the cubic shape, whose all characteristics and features contradict the very idea of flying, and its ability to fly, was by itself an architectural manifesto (Fig. 1) . This vision proved powerful enough to encourage several teams of scientists and engineers to contribute to the project: in the following years, the program has involved up to four science and/or technology labs (France, Switzerland, UK and Canada),10 several individual researchers, and two major art centres. Developments in applied physics were undertaken specifically for it. The structure of the first cubes were made with basswood, which was easy to work, but proved too fragile on the long run. Current trusses are made with carbon fibre. After several iterations and studies, we were able to produce a 225 cm-edge rigid structure whose bare weight is just under one kilogram (Fig. 2). Each element of the cube (structure, mechatronics, software) was the object of successive design steps and kept evolving during the years, following the availability on the market of components, materials of increased performance, as well as the arrival on our team of engineers and students of greater expertise.
Fig. 2

A Tryphon aerostabile being assembled, showing details of the carbon fibre trusses and the polycarbonate tubes that lead the flux of air coming from the ducted fans (at the centre of the trusses) towards the corners of the cube. From 2007 on, all mechanical connectors and components were realized by 3D prototyping (Photo by Nicolas Reeves).

As a robot, the flying cube had originally no expected or intended practical use: it takes some efforts to even imagine a possible application for it. It was just meant to float still, like in a deep artificial meditation. This first suspended shape was christened “Aerostabile”, in reference to Calder’s “mobiles”, which gave the name to the whole research program. In such a work, technology becomes its own poetics. The flying automaton is only there as a being: no doing or making is involved, no action or role justifies its existence, like it would be the case for a conventional robot. No arm, clamp, leg or protrusion is even there to suggest possible uses. The planned immobility further increases this impression of uselessness: building an automata (etymologically: he who moves, will and think autonomously) that does not even move contradicts the very idea of a robot in the same way as a cubic shape contradicts the very idea of flying.

From Architecture to Artificial Beings

Conceptually speaking, the flying cubes of the Aerostabile program are automatic machines from which everything that could contribute to identify them as robots has been removed, to focus on what constitutes the symbolical essence of the automaton. No one builds a robot for the sole reason of leaving it still: stillness is a trivial and uninteresting task for a ground robot. It is not considered as its most desirable behaviour, and it is very easy to achieve: when unplugged and discharged, most robots will end up still and remain still forever. It is however quite a challenge for a hovering automata.11,12 Even when it reaches aerostatic equilibrium, several forces and influences, such as micro-atmospheric movements, convection streams, ventilation, pressure variations, concur to make it drift from its original position, to which it may never come back. To make it still requires a complex combination of physics, mechatronics and software. In order to better manage and coordinate it, we had to develop a workflow made from several parallel threads corresponding to the different expertises required, which, considering the scope of disciplines that were involved, became by itself a specificity of the project,13 and led to the development of an international cooperation. Each aerostabile is equipped with up to fourteen distance sensors, as many light sensors, a compass, an inclinometer, eight or twelve ducted fans, a series of controllers and an onboard computer. In the simplest version, the distance to the nearest walls is measured at very short time intervals. Each departure from the prescribed position is immediately rectified by a thrust from the ducted fans, the strength, duration and acceleration curve of which being precisely determined by the computer. Such repositioning processes may occur up to one hundred times a second: for a hovering object, stillness is not a state, but a dynamic process.

The counterpart of the immobility of the automaton is thus a frantic agitation of electrons in all of its circuitry, making it extremely active in an invisible way. It is from this state that the name of the installation was decided: for humans, “paradoxical sleep” is the last sleeping phase of the night, during which the brain dreams. Though the body is totally relaxed, the brain is more active than during wake time, in a direct analogy with the state of a hovering aerostabile.

Like the vast majority of technological art projects, ours did not completely work quite as expected or planned. After one year of tests and experiments, we managed to reach a quasi-still state, but the constant repositioning of the cube created small, smooth oscillations that could easily be interpreted as a form of hesitations, translating the mood of an uncertain or undecided mind rather than the appearance of pure levitation. The intermittent noise of the motors began to be interpreted as a kind of breathing. Despite all our intentions, and despite a morphology that is all but biomorphic, the flying cube was explicitly seen by many visitors as a big, clumsy animal, immersed in a deep dream or meditation. It revealed in a rather radical way that no automata can escape its interpretation as a living organism, and that any animated objects can readily trigger such interpretations and meanings. It crystallized the essential symbolical ambition of any automata, the mythological impulse without which no robot would ever have existed: to be assimilated to a living being.

The flying cubes, as well as people’s reaction to them, refute the basic claim of the Uncanny Valley hypothesis: though their morphology presents no similarity with humans or animals, even remotely, they usually elicit very positive feelings—more than several animal-like or human-like artefacts.14 The range of feelings mentioned by the visitors includes empathy, tenderness, sympathy, amusement… but fear, uneasiness or weirdness are seldom heard. Curiously enough, toddlers are strongly attracted by the cubes, and demonstrate by their movements or facial expressions a strong desire to interact with them. After their first performances,  the flying cubes project evolved from their architectural origin to give birth to a complex and dense art piece about relations and artificial emotions.

Towards Hybrid Choreographies

From these observations, the idea to explore the potential of planned interactions with people quickly emerged. Several projects and works in that direction were developed during the last years, including performances in which dancers or actors developed hybrid, interactive choreographies with the cubes. Among them, some were specifically conceived to maximize the expressive ability of the automata. They encouraged our team to undertake a detailed study in order to identify the elements of their behaviour that could best convey expressions or emotions.

We first thought that these elements would be very limited in number. The cubes have no limbs or moving protrusions that can generate emotions or feelings through movement: they can only communicate through displacements of their whole body, or through the sounds they produce. In terms of movements, they have, in their first version, only four degrees of freedom: three translations (back and forth, up and down, left and right), and one rotation (around the vertical axis). This seems very few at a first glance: a human head whose expressivity would be limited to its three degrees of freedom relatively to the body would only be able to say “yes” (rotation around the left-right axis), “no” (rotation around the top-bottom axis) or “maybe” (rotation around the back-front axis). But every movement needs more than three parameters to be fully defined, and it appears more fruitful to characterize it through an analytic description which physically corresponds to the position and to its two time derivatives, speed and acceleration. To this, we add for our purposes another feature that corresponds to the acceleration curve: the oscillating rotation of a human head around a vertical axis will convey very different meanings if the oscillation rhythm is fast, slow, or if its stops after one half-rotation.15

Each of the four initial degrees of freedom is thus replaced by four new parameters, each of which requiring three sub-parameters for position (three scalars), three for orientation (three scalars), six for displacement (three vectors for speed and three for acceleration) and six for rotation (three vectors for speed and three for acceleration). This gives a total of eighteen parameters to find out where the cube is and where it is heading to. If we consider that each vector needs three components to be fully defined, and if we add that the acceleration curves for each of the acceleration vectors can itself be controlled by an arbitrary number of parameters, it is easy to see that the number of expressions that can be conveyed by a single floating cube becomes much greater than what the minimalism of its shape seems to imply.

Several research-creations experiments, as well as experimental protocols, were designed in order to identify more precisely some of the mechanisms and displacements through which the cube’s expressive potential could be expanded. They were mainly implemented on our largest cubes, 225 cm-edge aerostabiles christened the “Tryphons”.16 They all call for sequences of movements whose dynamics (amplitude, speed and acceleration) brings a key role for the visitor’s interpretation of their inner mood. For instance, a soft 2-m X translation (back-front axis, towards the visitor) does not carry the same meaning than a brisk 4-m one: the first may look like a manifestation of interest or curiosity, whereas the second can translate a threatening behaviour. A single, slow 45° oscillation around the left-right axis (horizontal and perpendicular to the visitor) may look like a greeting movement, whereas a series of short 30° oscillations around the same axis may translate a clear approbation, like the movement of head saying “yes”. A cube lying on the ground and slowly rising to about 1 m when a visitor approaches may look friendly, interested, and ready for interaction; if it rises quickly to 3 or 4 m, it may look feared. The slow movements of a cubes adjusting its position in the Paradoxical Sleep installation gives the image of a big, sleepy animal, lost in a contemplative dream; when shorter and faster, the same movements looks like a feverish tremor, translating a very nervous attitude.

The Geometry of Expressions

By carefully studying all the parameters of these movements and sequences, the development of a full vocabulary of intended feelings and expressions becomes possible. Each of them is associated with a sequence of displacements and rotations, and with the precise dynamics of this sequence. A fully equipped Tryphon is a 6-degrees of freedom robot. If we except the three coordinates associated with positioning, this gives a total of eighteen basic parameters that become the basic elements of a vocabulary of expressions (see Table 1): they can be associated in a huge number of ways to create expressive sequences. It is easy to see that the number of such sequences, which defines the expressive potential of a single automaton floating in the air, is theoretically almost unlimited.
Table 1

Basic elements of the expressive vocabulary of a flying cube


X (long.; back-to-front)

Y (transv.; left-to-right)

Z (vertical)


X-translation (m)

Y-translation (m)

Z-translation (m)

X-axis speed (m/s)

Y-axis speed (m/s)

Z-axis speed (m/s)

X-axis acceleration (m/s2)

Y-axis acceleration (m/s2)

Z-axis acceleration (m/s2)


X-axis rotation (deg)

Y-axis rotation (deg)

Z-axis rotation (deg)

X-axis rotational speed (deg/s)

Y-axis rotational speed (deg/s)

Z-axis rotational speed (deg/s)

X-axis rotational acceleration (deg/s2)

Y-axis rotational acceleration (deg/s2)

Z-axis rotational acceleration (deg/s2)

Each association of these elements becomes a sequence of instructions that can allow for the evocation of a specific feeling or emotion through an appropriate coding. Back-to-front and left-to-right are defined in relation to the observer’s location.

Practical considerations however limit this potential. First, the geometrical precision of this vocabulary can only convey the desired meaning if the cube is able to precisely follow a prescribed sequences of instructions. But a large flying cube, with its inefficient aerodynamics and its large inertia, cannot be controlled as easily as a ground object, or as a flying object with a flight-adapted geometry; its ranges of acceleration and speed are limited. Certain sequences of opposite displacements or rotations are forbidden, because of their negative impact on the stabilization and equilibrium of the automaton. Full rotations around arbitrary axis are difficult to control, since all references to external objects vary continuously. Then, the expressional or emotional interpretation of displacements and rotations is everything but an exact science. First, it strongly depends on the cultural background of the visitor 17: the rotation of the head around the back-to-front axis is interpreted as “not too sure” in the Western world, and as “yes” in the Indian subcontinent. Second, like for all interaction processes, the attitude of a visitor or performer interacting with the cube can deeply influence the interpretation of the cube’s moods by other visitors or by an audience. One of the ways we choose to explore the impact of this “cultural dialogue effect” is the implementation a software module that allows the control of the cubes by human voice, through short 3-notes melodies sung by a perfomer, or by anyone with minimal singing skills. The expressive potential of the human voice, combined to the general mood of each of these melodies (major, minor, 7th…) installs an initial atmosphere in which the reactions of the cubes take different meanings than in full silence.

Evolving Performances

Because of the number of relevant variables, the conditions of a given performance are not repeatable. To reach valid and useful conclusions, the exploration of the expressive potential of artificial beings requires a methodology that differs from what is commonly encountered in applied or fundamental sciences. From the beginning of the project, we decided to develop our research around intensive work periods called “research-creation residencies”, lasting from a few days to a few weeks, during which engineers, scientists and artists from several disciplines would work together towards the elaboration and implementation of public human-automata interactive performances. The results and conclusions of such events oriented the technological and artistic developments for the following months, up to the next residency where they could be evaluated and finalized.

After the first presentations of the Paradoxical Sleep installation, we worked on an event called “ROM<evo>—the Evolution of a Dead Memory”, based on a script jointly written with Quebec media artist Luc Courchesne, which took place in 2006 at the Quebec Museum of Civilization.18 It was based on the results of a collaboration with two European research groups, and with our recent collaboration with a Montreal lab that specializes in the field of artificial vision. During this event, three cubes were hovering in a wide space in which a short footbridge was installed, so as to allow the visitors to get very close to them. At the arrival of a visitor, the closest cubes would light up; a pair of eyes would appear on its faces, and the cube would trigger a spoken conversation with the visitor. The voice belonged to an actress19 that was hidden on a mezzanine. She could hear and see the visitors through a monitor and a pair of headphones; her eyes were filmed real-time by a pair of small cameras (Fig. 3). She was instructed to speak as if she was a computer-being trying to relate with human beings, which means that her way to communicate with people should simulate that of a computer program: every word and sentence was taken at its face value; the cube could not understand metaphors, analogies or second degree. It had no other knowledge than what it could learn form the visitors: its global knowledge was supposed to accumulate during the performance. This scenario resulted in the emergence of a schizophrenic personality for which every element of human vocabulary was considered as a mathematical variable with a unique significance.
Fig. 3

A Mascarillon aerostabile (170 cm edge) during the ROM<evo> Performance at the Québec Museum of Civilization in Québec City. Actress Maryse Lapierre’s eyes, filmed through a pair of small cameras, are projected real-time on the faces of the cube through an adaptive projection system developed by Sebastien Roy and the Vision3D lab at University of Montreal. The Mascarillons were wade with basswood; the material was aesthetically compelling, but the trusses were way too fragile. The following models were made with carbon fiber (Photo by Nicolas Reeves).

The reactions of the audience were extremely diverse, ranging from amusement to anger. To our surprise however, several people tried for a rather long time to interact and speak with the cubes. Among the most intriguing moments, we saw a man who tried to teach a poem to a cube, like if he was hoping to counteract—or maybe heal—its dry, algorithmic and monosemic language through poetry; as opposed to computer code, poetry is the form of language that is opened to the largest number of potential interpretations. An old woman came several times during the three weeks of the exhibition and began to confide in the cubes, complaining for instance that she felt very alone because her children never visited her. It is hard to explain why the Rom<evo>installation, with is high-tech aesthetics, triggered behaviours usually associated with confidence or intimacy. We made the hypothesis that the artificial nature of the automata, associated with the almost complete predictability of its answers and its obvious inability to interpret, judge or criticize, created an atmosphere where some people could feel secure enough to enter into a more intimate mode of discussion.

On the technological point of view, this performance, along with a few other ones, made us realize that the most critical problem associated with flying cubes was the question of precise positioning and displacement. We addressed this problem directly during a major installation at the Grand Palais in Paris (Fig. 4), during which the cubes were supposed to fly in a gigantic space—more than 200 m long by 45 m high—during an important Paris art event (The 2008 Summer of Dance). Due to the size of the space, the cubes could not rely anymore on the distance to the walls and floors to locate themselves. Positioning was more critical than ever, since adaptive video projections were planned on their faces for several hours. The problem was solved by using robotic video projectors equipped with cameras which detected the orientation and distance of the cubes as revealed by their 2D projections on the vision plane. This in turn allowed the computers to precisely track their position and orientation, which theoretically allowed the control of their displacements as well as the proper adjustments for the adaptive video projections. For reasons principally linked with the very turbulent and agitated atmosphere of the Grand Palais, in which sudden drafts created very unstable conditions, we had to back up the automatic control procedures with remote control systems, thus transforming the cubes performance in a kind of high-tech puppetry.
Fig. 4

A Tryphon aerostabile (225 cm edge) hovering mid-air in the huge nave of the Grand Palais (Paris, France) during the Summers of Dance 2008 event. Three cubes were flying for this event. During night performances, adaptive video projections real time by a team of Montreal DJ’s (from Elektra), as well as text messages from the audience, were projected on their faces, transforming them in flying video lanterns (Photo by Nicolas Reeves).

The adaptive video projections nonetheless worked fairly well. They showed sequences from the previous evening dance shows as transformed live by Montreal VJs during after-hour performances. We however realized that the expressive potential of the cubes themselves was strongly diminished by these projections: the content of the projected sequences overwhelmed the artistic impact of the cubes, which almost disappeared as automata to become mere floating screens. Instead of being artwork by themselves, they became supports for a non-related artwork.

We then decided to orient our development axis towards human-to-automata interaction. A few months before the Grand Palais event, we had presented our first hybrid performance in Montreal. Called Nestor & Veronique, it involved our smallest floating cube (the “Nestor”, 160 cm-edge) and an actress in a 10-min narrative performance. The actress was instructed to try to interact with the cube as if it was a real, living organism—like a wild animal she was trying to tame. She piloted it by her movements and displacements and through small LEDs attached to the palms of her hands (Fig. 5). Though implemented in a very controlled environment, this simple event revealed that while performing, the actress actually adapted the rhythm and speed of her movements to those of the automaton, resulting in a very fluid and smooth kind of dance: she reacted to behaviours of the cube that were triggered by her own behaviours. For the first time in our research, we could attend the emergence of a full, complex 2-ways interaction between an actress and our artificial beings.
Fig. 5

Actress Véronique Daudelin taming the Nestor flying cube (160 cm edge) in the first aerostabile hybrid performance during the Robofolies festival, Montreal 2007. The actress was controlling the cube through her displacements and movements, and with small LED lamps hidden in the palms of her hands (Photo by Nicolas Reeves).

Another installation that we presented later, during the FILE 2012 festival in Sao Paulo, was precisely based on this observation. It was a variation of the Paradoxical Sleep installation in which a performer tried to interact with the cube through her own movements and different vocal sequences, thus modifying the general ambiance in which both evolve, enlarging the cube’s expressive potential and generating new possibilities for human-automata relationships. The scenario was written in collaboration with the performer, who was actually a dancer and choreographer from the Montreal dance scene.20 The different states of the cube were changing according to short vocal melodies (three notes only); in order to facilitate the learning for the performer, the cube was programmed to detect the intervals between the notes, instead of the notes themselves. One melody triggered the taking off of the cube, another one a particular rotation, and so on. Other melodies were mapped on a variety of states such as “oscillate”, “get nervous”, “fall asleep”. The combination of the melodies and of the combined movements of the cube and of the performer generated a very fluid, semi-improvised “pas-de-deux” between the dancer and the automaton, in a hybrid choreography which involved no pre-programming whatsoever at the level of the cubes movements and dance (Fig. 6).
Fig. 6

Dancer and choreographer Ghislaine Doté performing with a Tryphon flying cube during the Sao Paulo FILE festival (2012). The dancer was controlling the cube with her movements and displacements, as well as with very short sung melodies. The voice and frequency analysis software was developed specifically for the project by Belgian engineer François Séverin (Photo by Nicolas Reeves).

Geometric Butterflies

Several experiments and performances occurred between Nestor & Veronique and the Sao Paulo festival. Each one was triggered by a new development on the software or hardware, or by the emergence of new technological components or devices. The Geometric Butterflies performance took place in Moscow in 2009. For this event, three cubes were instructed to fly freely in a large indoor space. Their flight area was surrounded with dark blue robotized spotlight that were slowly oscillating from left to right. Their behaviour did not consist in state transitions, like for the previous example, but was based on two very simple rules, in the manner of boids: they were instructed to avoid light (“be afraid of light”) and obstacles. At some point during their flight, they were coming close to the blue spotlights, which sent them back towards the center of the flying space. By doing so, they unavoidably got close to the other cubes, which sent them back towards the spotlights, and so on. Through these elementary reactions, they flew autonomously during more than three weeks, following never-ending and never-repeating orbits (Fig. 7).
Fig. 7

Three Tryphons cubes in the Winzavod Centre for Contemporary Arts in Moscow, during the Science as Suspense festival (2009). The cubes were flying in an area surrounded by intense blue spotlights. They were instructed to fly away from light and to avoid obstacles. These two simple instructions led to complex, never-repeated and unpredictable orbits for more than three weeks (Photo by Asya Ablogina).

Like for any technological arts installation, unpredictable events occurred during these weeks. At some point, the three cubes found themselves in the same corner of the flight area. They tried desperately to avoid each other, but they were so close from the lights that no one could manage to do so: each of their displacements was sending them towards the other cubes, or towards the spotlights. The collisions that resulted, joined with the roaring and the grunting of the motors that were frantically reversing their rotation direction every few seconds, gave the impression of a fight. The cubes managed to solve the situation by themselves when one of them, through a particular interaction, was abruptly ejected from the group. It went so fast that it managed to overcome the spotlight virtual barrier and to fly over the audience towards the exit of the exhibition hall, like if he was fed up with the situation and wanted to go out.

Here again, obviously, the interpretation of the cubes’physical behaviour as resulting from intentions or emotions results from our interpretation of strictly physical, meaningless events. What deserves to be noted is the wide difference between the simplicity of the programmed behaviour and the complexity of the interpreted one: getting involved into a fight, being fed up, running away because of exasperation, are by no way simple behaviours. The experiment revealed to which extent our brain tries to make sense with everything that surrounds us and to project onto inanimate objects sets of interpretations that actually correspond to a part of ourselves.21 It shows how promptly we believe in the self-autonomy of animated artefacts, and how enthusiastically we surrender ourselves to this voluntary deception. Another anecdote is revealing in that respect: a psychiatry student came twice to see the Geometric Butterfly installation, and shared with us at length her “analysis” of the personality of the cubes: one was more extroverted, and acted as a leader; the second one had a more reserved and quiet personality, and tended to remain in the backstage; the third one was acting as a mediator who tried in its own way to reconcile the two others. What makes this analysis all the most interesting is that the three cubes were perfectly identical and identically programmed: they behaved essentially the same way.

The Floating Head Experiment

As mentioned above, from their artistic origin, the flying cubes were quickly seen by engineers and scientists as a rich research and development platform, opened to a wide variety of experiments in robotics, swarm intelligence, mechatronics, emerging behaviours, and so on. Several artists and art labs also collaborated to the project. In 2010, we had the pleasure of working with Stelarc and his team in the design of a performance that was presented in the Montreal Elektra festival.22 This event gave us the opportunity to merge the flying cube project with an art installation from this team, namely the Prosthetic Head, a virtual model of Stelarc’s head that is able to talk, to answer questions and to show facial expressions through the simulated musculature of his face (Fig. 8). Such apparatus have a long history: written accounts of mechanical devices meant to simulate the human speech have been found as soon as the XIIIth century (they have apparently been destroyed, since the church considered them as heretic devices). In the middle of the XIXth century, German astronomer Joseph Faber’s Talking Euphonia managed to produce a mechanic talking machine by animating an artificial face through levers and the breath of a bellow. Though the voice was described as ghostly and sepulchral, and though the machine itself was eerie to look at, the speech itself was perfectly understandable, and the machine is seen as the first “disembodied head” of the history of automata.
Fig. 8

The real Stelarc and David St-Onge, from the NXI Gestatio Design Lab, in front of Stelarc’s Floating Head at the Elektra festival (Montreal 2010). Thanks to the “attention model” developed by Stelarc’s team (Christian Kroos and Damith Herath), Stelarc’s Prosthetic Head and a Tryphon aerostabile from NXI Gestatio could merge to create a hovering oracle who was able to maintain a cyberdialogue with members of the audience (Photo by Elektra).

Stelarc’s talking head was not only disembodied: it was also dematerialized, and the idea to project it onto a flying cube was partially triggered by the idea of reconnecting it to a physical body.23 As a matter of fact, after decades of progressive dematerialization, the current state of automata evolution seems to imply that any machine meant to learn and evolve in the real world should be aware of the state of his environment at any moment, and to learn not only from its internal processes, but also from this environment. In order to do this, it cannot limit itself to a virtual being, communicating only with the material world through fluxes of information. Physical information coming from a perceptive body appears a primordial component of learning processes, and of the adaptation to a changing physical world.

By projecting the Talking Head onto an aerostabile, it became possible to increase its expressivity through the movements of the cube itself. An “attention model”, a clever piece of software developed by scientist Christian Kroos and engineer Damith Herath, allowed the cube to rotate towards a specific visitor while Stelarc’s face was orienting its eyes towards him, so as to increase its interaction level with him. Though rather elementary, this synergy between the two projects resulted in a haunting, strange installation, where the cube and its projected face, hovering in a dark space, looked like a levitating oracle, pronouncing prophetic sentences and answering questions about the future of intelligence, awareness and consciousness in a world were the distinction between artefacts and biological organisms is becoming more and more blurred.

Balades: A Major Art-Science-Technology Event

The spring of 2014 saw the most ambitious event ever realized with the flying cubes. A team of twenty-five people, including artists, scientists, engineers, students, scenographs, choreographs, musicians, dancers and capoeirists gathered around two flying cubes in a beautiful 19th-century hall located at the top of Mount Royal, a tall hill located in the heart of Montreal, in the middle of a forest. During three weeks, the team developed from scratch a performance in which two dancers and two capoeirists interacted with the cube in a twenty minutes choreography (Figs. 9, 10 and 11), on a musical sequence that was composed specifically for the event. The music was composed from sounds recorded on the hill and on the Mount-Royal park—birds, insects, frogs, rains. Set designers also worked at bringing the mountain forest inside the building, by using animal-inspired make-ups, a fence-wall of trees and a dancing carpet of water-like material. All elements of the performance had to take in account the specific sensibility of the aerostabiles to their surroundings. One of the engineer teams worked at developing new control devices, using ground-based infrared laser beams; another one worked at improving a docking system so as to allow the cubes to autonomously bond to each other like giant atoms, and to fly together like huge molecules. A third one worked on a program that allowed a flying cube to detect, through an on-board camera, elements on a scene that were considered “interesting”—elements which differs from the rest of the captured image through their contrast, texture, density, movement, or other distinct parameters. At the end of this residency, other artists came in the same space to present their work; the cubes were regularly asked to participate in other performances. For instance, during the days that preceded the final event, the images captured by the on-board cameras were projected real time on a large screen behind a scene where musical shows were going on. The result was presented in front of a large crowd at the opening night of the 5th edition of the Chromatic digital arts festival (Fig. 12); it was the first large-scale public event using the flying cubes in their full potential as real actors of a hybrid human-automata performance.
Fig. 9

The team of reseachers and graduate students working on software and technological development at the Chalet du Mont-Royal during the May 2014 research-creation residency. Twenty-five people, including a scenographer, a light designer, a choreographer, four performers, two musicians, worked together during this residency to develop and implement a hybrid performance for a contemporary art event called "Chromatic”. On the forefront is professor Philippe Giguère, from the department of robotics at University Laval in Québec City (Photo by Nicolas Reeves)

Fig. 10

Dancer Aychele Szot discusses with choreographer Eli Toussaint during the development of the Chromatic hybrid performance (Photo by Nicolas Reeves).

Fig. 11

The four performers rehearsing for the final performance of the 2014 Chromatic event. Development and rehearsals of the performance were open to the audience. Lighting and set design were developed by Montreal designers Josée Bergeron-Proulx and Audrey-Anne Bouchard (Photo by Nicolas Reeves).

Fig. 12

Dancers Ghislaine Doté and Aychele Szot with caopeirists Eric Prido and Michel Zambrano dancing with two Tryphon aerostabiles during the opening night of the Chromatic contemporary arts festival (Montreal 2014) (Photo by MtlBlog)


From the simple architectural image of a heavy mass freed from the law of gravity, the aerostabiles haves developed into a full research program that generated a series of art and technological projects, some of them being now on the verge of producing transferable applications for theatrical scene, museology, education, space studies, robotics.24 But none of them may be more surprising that his passage from an art piece, an automata that does not move and whose only skill is immobility, to a mechatronic being able and willing to interact with humans through the definition of a series of artificial emotions. The first Paradoxical Sleep installation puts technology to work; but technology here does not do anything practical, and does not create anything material. It tries to eliminate everything it is expected to do for the sake of generating a representation of itself—or rather, of its own mythical or symbolical load. From a deep, lonely meditation to an active relation with humans in which the development of hybrid choreographies becomes possible, the flying cubes exemplify these situations where technology not only enriches the potential poetics of a project, but becomes itself a poetics and an imaginary, not through what it is or what it can do, but through what it represents. Combining a rigorously calculated morphology and a radically technological geometry with the hesitations and errances of a wandering being, the aerostabiles translate the implicit fact that any automaton wishes, more than anything else, to become a living being. Indeed, a lucid attitude would make us tell that no automaton ever wished anything, and that the wish actually comes out of our own minds—we project it on inanimate artefacts. But this wish transfer is precisely at the core of every attempt at creating automata, as well as an example where lucidity may not be the most fertile attitude. For artists as well as engineers and scientists, the deliberately accepted illusion of the automaton as a living being opens exploration territories that are infinitely wider than a too strict, objective interpretation of the machine as a sheer assemblage of inanimate components.

The authors want to thank:

  • The School of Design at University of Quebec in Montreal

  • The Hexagram Institute for Research and Creation in Media Arts and Technology

  • The Canadian Council for the Arts

  • The Quebec Council for Arts and Letters

  • The Quebec Research Fund for Nature and Technologies

  • The Quebec Research Fund for Society and Culture

  • The Natural Science and Engineering Research Council of Canada


  1. 1.

    See Bedini [1] for an historical account of the intersection between automata, life simulation and technology.

  2. 2.

    See for instance Bruce et al. [2], or Imai et al. [3].

  3. 3.

    See for instance Pioggia et al. [4].

  4. 4.

    See for instance the Mega Hysterical Machine at

  5. 5.

    Destephe et al. [5].

  6. 6.

    Mori, M.—The Uncanny Valley, Energy 7(4), pp. 33–35, 1970.

  7. 7.

    Reeves et al. [6].

  8. 8.

    Cooke [7].

  9. 9.

    Bunge [8].

  10. 10.

    Science/Technology: Laboratoire d’éthologie animale (G. Théraulaz, U. Paul Sabatier, Toulouse, France); Intelligent Autonomous System Lab (A. Winfield, U. of the West of England, UK); Collective Robotics Lab (now DISAL, A. Martinoli, EPFL, Lausanne, Switzerland; 3DVision (S. Roy, U. of Montreal, Canada). Arts: Society for Arts and Technology (SAT, L. Courchesne, Montreal, Canada); Hexagram (N. Reeves, Montreal, Canada). Researchers: P. Giguere, Laval University, Quebec, Canada; I. Sharf, G. Dudek, I. Rekleitis, U. McGill, Montreal, Canada.

  11. 11.

    Van der Zwaan et al. [9].

  12. 12.

    Lozano [10].

  13. 13.

    St-Onge et al. [11, 12].

  14. 14.

    We could compare the reaction of the audience to several kinds of automata with various morphologies, including ours, in specific robotic arts events, such as the Moscow “Science as Suspense” event [13].

  15. 15.

    St-Onge et al. [11, 12].

  16. 16.

    “Tryphon” comes from the first name of the famous absent-minded scientist Tryphon Tournesol, in Herge’s Adventures of Tintin. He is known as Cuthbert Calculus in the English translation.

  17. 17.

    See for instance Joose et al. [14].

  18. 18.

    A more detailed description of this work appears in Reeves [13].

  19. 19.

    Quebec city actresses Véronique Daudelin, Maryse Lapierre and Klervi Thienpont were alternatively the cube’s eyes and voice.

  20. 20.

    Ghislaine Doté from Montreal Sinha Dance company.

  21. 21.

    This concept of self-extension on inanimate things is explored through an interesting experiment by Kiesler et al. [15].

  22. 22.

    St-Onge et al. [11, 12].

  23. 23.

    Kroos et al. [16].

  24. 24.

    Some applications are described in St-Onge et al. [17].


  1. 1.
    Bedini SA (1964) The role of automata in the history of technology in technology and culture, vol 5, no 1, pp 24–42Google Scholar
  2. 2.
    Bruce A, Nourbakhsh I, Simmons R (2002) The role of expressiveness and attention in human-robot interaction. In: Proceedings from the ieee international conference on robotics and automation, pp 4138–4142Google Scholar
  3. 3.
    Imai M, Ono T, Ishiguro H (2003) Physical relation and expression: joint attention for human-robot interaction. IEEE Trans Industr Electron 50(4):636–643CrossRefGoogle Scholar
  4. 4.
    Pioggia G, Igliozzi R, Ferro M, Ahluwalia A, Muratori F, De Rossi D (2005) An android for enhancing social skills and emotion recognition in people with autism, in neural systems and rehabilitation engineering, vol 13, 4, pp 507–515Google Scholar
  5. 5.
    Destephe M, Maruyama T, Zecca M, Hashimoto K, Takanishi A (2013) Improving the human-robot interaction through emotive movements, a special case: walking. HRI 2013:115–116Google Scholar
  6. 6.
    Reeves N, Nembrini J, Poncet E et al (2005) Mascarillons—flying swarm intelligence for architectural research. IEEE Swarm Intell Symp 2005:225–232Google Scholar
  7. 7.
    Cooke C et al (1990) Architectural drawings of the russian avant-garde (see in particular Krutikov’s flying cities). Editions of The Museum of Modern Art, New YorkGoogle Scholar
  8. 8.
    Bunge E (2003) Jealousy: modern architecture and flight, in cabinet, Issue 11, Flight Summer 2003, New YorkGoogle Scholar
  9. 9.
    Van der Zwaan S, Bernardino A, José Santos-Victor J (2000) Vision based station keeping and docking for an aerial blimp. IROS 2000:614–619Google Scholar
  10. 10.
    Lozano R (2007) Objets volants miniatures: modélisation et commande embarquée (ch. 2), Lavoisier, Cachan (France)Google Scholar
  11. 11.
    St-Onge D, Gosselin C, Reeves N (Voiles|Sails) (2011) A modular architecture for a fast parallel development in an international multidisciplinary project. In: Proceedings of IEEE ICAR 2011, Tallin, Estonia, pp 482–488Google Scholar
  12. 12.
    St-Onge D, Reeves N, Herald D, Kroos C, Hanafi M, Stelarc S (2011) The floating head experiment. Proceedings of HRI 2011. Lausanne, Switzerland, pp 395–396Google Scholar
  13. 13.
    Reeves N (2009) Rom<evo>: Evolution of a dead memory. New realities: being syncretic, Angewandte edn. Springer, Vienna, pp 236–239CrossRefGoogle Scholar
  14. 14.
    Joosse M, Lohse M, Evers V (2014) Lost in proxemics: spatial behavior for cross-cultural HRI. HRI 2014:184–185Google Scholar
  15. 15.
    Kiesler T, Kiesler S (2004) My pet rock and me: an experimental exploration of the self-extension concept. Adv Consum Res 32:365–370Google Scholar
  16. 16.
    Kroos C, Herath D, Stelarc S (2012) Evoking agency: attention model and behavior control in a robotic art installation. Leonardo 45(5):401–407CrossRefGoogle Scholar
  17. 17.
    St-Onge D, Reeves N, Persson M, Sharf I (2014) Development of aerobots for satellite emulation, architecture and art. In: Proceedings of 13th international symposium on experimental robotics. Quebec Canada, pp 167–181Google Scholar

Copyright information

© Springer Science+Business Media Singapore 2016

Authors and Affiliations

  1. 1.NXI GESTATIO Design Lab (Arts Design Architecture Informatique) École de DesignUniversité du Québec à MontréalMontrealCanada

Personalised recommendations