Introduction: “Fully Nonlinear”

Abstract

The introduction to Animating Unpredictable Effects establishes a specific category of digital animation that is based on making unpredictable simulations. The book situates this form of animation in the context of key discussions about the relationship between animation and automation, and about the agency digital animators have when working with increasingly opaque software tools. The introduction further situates this type of animation in the history of cinema, which has long been preoccupied with natural motion and topics of contingency.

The computer science department at Stanford University offers a course called Computer Graphics: Animation and Simulation. Many departments at other universities have offered a similar course, including the University of North Carolina, University of California, California Polytechnic, and Carnegie Mellon. A title like Animation and Simulation will sound to some like a betrayal of the principles of animation. To many fans, students, and scholars, animation represents an anarchic, unpredictable, representationally unrestricted form of moving image, while simulation represents the rationalizing numerical authority of objectivity and control. This is not just a course where computer science students learn to make tools for animators to use either. Courses such as these are as much about making moving images as they are about making software, and graduates with experience in this domain are as likely to work for visual effects (VFX), animation, or game studios as they are software companies. Indeed, Stanford’s computer science department has a strong connection with special effects studio Industrial Light and Magic. Though this type of animation is easy to dismiss because of its relationship to engineering, this is exactly why we should pay close attention to it. It represents a particular conceptualization of the relationship between engineering and animation production that has been taking shape since the late 1970s. It also provides a window into a paradigm of control shared between contemporary animation and numerous other facets of society that employ simulations of nonlinear systems, which have shaped everything from finance to the way we understand climate change since the 1940s.

The weekly classes of Stanford’s Animation and Simulation course include titles like procedural modeling, collision processing, character FX, particle-based fluids, and character animation FX. These types of animations create motion not through the manual control of sequential images but through algorithms that are designed to produce automated unpredictable outputs. Studios often use these methods to animate natural phenomena: the flow of hair, the splash of water, vortices in smoke, or the behavior of groups of animals. Textbooks on computer graphics and animation create a similar grouping of topics.¹ Any map of contemporary animation, VFX, or large-budget video game production workflows also includes such a category as its own branch of production. This grouping of techniques and tools goes by several names like, “procedural,” “dynamic,” “simulated,” or “technical” animation. Sometimes they are simply referred to as “FX.”

Many observers and critics group these types of animation under the term “physical simulation.” But more fundamental to this group than the imitation of physics is their programmed unpredictability, or simulated nonlinearity. In this sense, they are akin to a genre of computer art Frieder Nake terms “generative art,” practiced by artist like Georg Nees, Ernest Edmonds, and Nake himself since the 1960s. Yet these forms of animation are not experimental art, they are the product of industrial forces and discourses. Indeed, they are the paradigmatic products of a trend that has seen research and development (R&D) become a substantial part of media industries production and economics. Whereas conferences held by the Association for Computing Machinery (ACM) or Institute of Electrical and Electronics Engineers (IEEE) on computer graphics and simulation were once dominated by federal funding and the military-industrial complex, since the 1980s media industries like Hollywood have become major sponsors of research. This combination of military and media industries R&D produced a very particular way of seeing contingency and seeking to control it.

This book will refer to these tools and production practices as “nonlinear animation,” emphasizing that they are in fact a form of animation. They seek to bring images to life, to animate them, as all animated media do. Just as early film and animation embodied the animating energies of the nineteenth century like clockwork, spiritualism, and electricity, nonlinear animation demonstrates the élan vital of the late twentieth and early twenty-first centuries.² When computer graphics researcher and Pixar co-founder Alvy Ray Smith once infuriated Steve Jobs so severely that Jobs stormed out of a company meeting, Smith described Jobs as having gone “fully nonlinear.”³ He had become wild, unpredictable, and undeniably animated. This is how someone like Smith makes sense of such chaotic unruliness.

Chaotic motion has always been a key element of cinema’s animate vitality. When Georges Sadoul and Georges Méliès first saw the Lumière brother’s 1895 actuality RepasDe Bébé, their attention was drawn not to the middleclass domestic scene in the foreground of the film but instead to the background, where the leaves of the trees were being rustled by the wind.⁴ When our attention is adequately directed, we can still marvel at cinema’s ability to capture chaotic, unpredictable motion and events. In his film Grizzly Man (2005), Werner Herzog includes a long shot of grass blowing in the wind, narrating “sometimes images themselves develop their own life.” Cinema’s ability to capture the unpredictable has always been one of its fundamental properties, even in otherwise artificial circumstances. This is an important component of Mary Ann Doane’s influential analysis of cinematic time as it relates to industrial modernity.⁵ In her work, she finds that contingency had an almost irresistible appeal. The camera was unique for its ability to capture unexpected occurrences like a building toppling over, delivering uncanny effects. Cinema contained contingency in a “representational system while maintaining both its threat and its allure.”⁶

Animators have long been interested in the unique, complex quality of natural movement as well. Disney animators Sandy Strother and Ugo D’Orsi were dedicated to this subject on projects such as Fantasia (1940) and Pinocchio (1940), where they specialized in animating the complex movement and splashes of water. These animators were seeking to bring water to life. Hand-drawn animation may seem like the antithesis of the captured contingency in RepasDe Bébé, but their preoccupations are alike. As many film and animation scholars argue, we need to think beyond a simple dichotomous view of animation and live-action cinema.⁷Vivian Sobchack notes that the dynamic between the effortless vitality of animation and the regulated mechanical control of automation has been central to all animated media, including live-action cinema.⁸ All of these moving images are brought to life, animated, by seemingly unpredictable movement, yet they also entail a different apparatus for shaping that movement and making meaning from it. For the Lumières it was the capture of the camera, and for Strother and D’Orsi it was the manual manipulation of drawn frames by the artist. For nonlinear animation, this animating force is something else, something that sits in-between capture and manual manipulation.

Nonlinear animation sees vitality extending from the unpredictability of dynamic complexity or randomness. Control of this vitality comes in the form of manipulating data parameters. In an example like Disney’s digital animated feature Moana (2016), the ocean moves in uncannily lifelike ways, yet it also so extensively manipulated it has its own personality, as though it were a character. Artists are able to make the water behave a certain way, sculpting a living, moving thing. Making an animation like this entails not just a different kind of artistic work, but a different relationship between engineering and production. It requires making tools, writing code and scripts, and combining different software and plug-ins. A former Vice President at VFX studio Digital Domain says these jobs require, “a combination of computer scientist and fine artist… the eye of an animator but the brain of a hardcore technologist.”⁹ A Stanford computer science professor and frequent contributor to special effect studio Industrial Light and Magic similarly claims, “a little chaos goes a long way… we’ve found that less control, better algorithms, and a different breed of artist is the key.”¹⁰ Clearly there are some industrial promotional discourses working through statements like these, but getting to the bottom of these discourses is key to understanding this form of animation as a product of an industrial-institutional machine that is constantly manufacturing this animated novelty. Embedded in these statements is a particular conception of control and a particular way of thinking about the relationship between image making and technology development.

This discourse of nonlinear control negotiates the already fraught territory of digital animation work. Formerly common representations of all-powerful animators, able to fully control the most minute detail of the worlds they create, have given way to anxieties about being “ousted by technology, made obsolete, or – worse yet – turned into mechanical slaves to digital software.”¹¹Aylish Wood finds that fears about animators being disempowered by black-boxed technology have shaped the design of animation software interfaces.¹² This is why Autodesk’s Maya software offers a 3D preview that puts the user in touch with the images they are making, giving a sense of creative control, while its features like the “Channel Box” give the sense of access to deeper software functions.¹³ Nonlinearity represents a total rethink of this question of artistic control. While its unpredictable autonomy would seem to pose new threats to the artist’s agency, it also extends control into new domains. This different understanding of control is linked to a different conceptualization of creativity that puts greater emphasis on making technical apparatuses.

At the heart of nonlinear animation is a way of thinking that seeks to make use of unpredictable nonlinear complexity by shaping it toward specific applications. This applies both to the way animation and VFX studios build tools to direct the look of simulated images and also to the way they use hands-off management techniques that seek to direct unpredictable labor tasks involving R&D and creativity. This way of seeing and managing the world is imbricated with the development of similar nonlinear simulation approaches in a number of other industries and research disciplines, such as climate science, sociology, geology, management science, and financial mathematics.¹⁴ Understanding nonlinear animation thus entails understanding a broader archeological layer of knowledge that includes various institutions and forms of organization and management. This epistemic horizon, this episteme, applies not just to our supposedly “post-cinematic” digital lives, but also to the way society sees materiality and material phenomena.

The following chapters will investigate this subject by charting the circulation of ideas, technologies, moving images, and people through contact zones such as the ACM’s Special Interest Group on Computer Graphics and Interactive Techniques (SIGGRAPH), using archival research of trade communications, scholarly publications, and conference proceedings, as well as interviews with industry workers. This book is structured into five parts that draw an arc from the historical and philosophical roots of nonlinear simulation through to representations of these ideas in themes and VFX aesthetics on cinema screens.

Chapter 2 traces the roots of simulation, nonlinearity, and R&D in the nineteenth century and observes their growth in the institutional context of World War II, the Cold War, and after, revealing how nonlinear animation is a product of this history and proposing some theoretical frameworks for understanding how simulated images make meaning. Chapter 3 studies the shifting role of R&D in the film industry since 1980, explaining the economic and strategic value technology ownership has gained in contemporary VFX and animation. Chapter 4 shifts focus to the more recent past, studying the way technological and software development principles have informed animation and VFX “workflows” and “pipelines,” transforming the organization of production and blurring the line between technical and creative work since the 1990s. Chapter 5 keeps this more recent historical focus and studies how the management principles of Pixar have been influenced by nonlinear animation paradigms, offering an updated, nuanced understanding of post-Fordist control. Finally, Chap. 6 studies films since 1982 that both feature nonlinear animation and thematically engage topics in nonlinearity such as chaos theory, catastrophe theory, and perfect storms, finding a complex interaction between the fear nonlinear unpredictability can inspire and the reassuring mastery simulation promises. These chapters represent a range of different conceptual frameworks and methodological approaches to this one subject. Through this, the book offers a more holistic view of how a particular set of animation practices and technologies are interlinked with symbolic, economic, institutional, and discursive historical factors.

No existing term satisfactorily describes what I will continue to refer to as nonlinear animation. But each of the names commonly used to describe some part of nonlinear animation, such as “physical simulation,” “technical animation,” “procedural animation,” and “FX,” provide some insight into the particularity of these tools and production practices, as well as the way they are constructed within the industry. Taking a moment to address each of these terms in turn will help us grapple with the complex ontology of nonlinear animation and will highlight what we can learn by studying it.

Physical Simulation

A phrase often used by observers to describe nonlinear animation is “physical simulation.” This is a term that carries some heavy connotations in media studies. For many, simulation is a marker of false artifice. For example, the term “visual simulation” is frequently applied to many kinds of digital animation to refer to their “perceptual realism.” In other words, simulation describes the way these images are cleverly made to look real through perceptual cues like reflection and shadow, when they are in fact mere fakery.¹⁵ Simulation is also a key term of postmodern critique, used to describe the artificiality of late capitalism.¹⁶ Yet simulation has gradually become an important way to understand the world and to confront the limits of understanding, yielding an everyday utility that we cannot quickly dismiss as mere fakery. Parsing the meaning of simulation is vital to getting to the bottom of how nonlinear animations make meaning.

Some nonlinear animations are based on modified physics equations. Some imitate other kinds of processes in nature, such as the patterns of branch and leaf growth or the behavior of groups of animals. Many seek to represent natural phenomena without being grounded in real-world science, and others are completely abstract. Simulations can range in their fidelity to the mechanism they imitate. The field of ludology, an early subset of game studies, has already demonstrated this.¹⁷ Games produce meaning through programmed rules and structure. Those game mechanics may refer to real-world mechanics, like the way Monopoly is about real estate economics, but they do not necessarily try to imitate that real-world mechanism they are representing exactly. Games can use simulation to be expressive or imaginative, rather than realistic. Simulations thus require us to think through a different epistemic frame that understands the world through making models rather than through sensation or recording. Historians of technology and science such as Walter Vincenti, Mario Bunge, and Herbert Simon have theorized this form of engineering epistemology, which creates knowledge through “knowing how” instead of “knowing that.” Chapter 2 will engage these ideas to develop a framework for how nonlinear animations make meaning as simulations.

Technical and Procedural Animation

“Technical animation” is a screen credit sometimes given to workers in VFX and animation studios that work on “character FX” like hair, fur, and cloth. The use of this term points to the special technical skills the workers have, and it identifies a type of animation where every job requires customization and R&D. In contrast to other character work like creating models or manipulating those models directly, nonlinear animation entails configuring software and programming simulations with algorithmic rules or procedures: creating a technological apparatus that will in turn create movement. This emphasis on tool building as a component of image making modifies the labor division between creative work and technical work studied by scholars like Vicki Mayer.¹⁸ As Chap. 4 will discuss, there is a very blurry line between nonlinear animation artists and technical directors. This is a discursive shift within the industry, but it also betrays an epistemic shift from the aforementioned “knowing that” to “knowing how,” in other words, from creating images of the world to creating models of how it might function.

Nonlinear animation might seem to be furthering the trend of obfuscating the work of animation workers (especially international ones) behind seemingly automated technologies, much the way depictions of performance capture elide the work of animators.¹⁹ Focusing on the technical work integrated into production helps to reveal this labor though. Indeed, it may even help reveal some of the labor obfuscated by performance capture. Because, while the myth of an automated capture system obscures the work of the animators who modify and sometimes replace capture data, beyond these workers there are also numerous technical staff creating pipelines, modify data, and upgrading and maintaining equipment.

Beyond its creative value, technical work has economic value both because of the images it produces and because of the technological intellectual properties it can lead to. Chapter 3 will explain how economically and strategically important the development of new technology has become for large VFX and animation studios. These studios profit from developing and owning technologies on every step of their journey from novel emergence to standardization and dominance, proceeding on what Tom Gunning calls, “the cycle from wonder to habit.”²⁰ Technological change and R&D have been a part of film industries since the days of Thomas Edison, and special effects have always played a key role in negotiating technological change.²¹ As have animation studios like Disney.²² Yet terms like technical animation and procedural animation point to a historically specific shift in the way technology and R&D are constructed within these industries.

Contemporary animation and VFX studios like Industrial Light and Magic and Pixar frequently promote their cutting-edge technology and the way they integrate creativity with technical innovation. This Silicon Valley-informed discourse sees both creative and technological advances as the product of entrepreneurial innovation that disrupts the ossified structure of large existing businesses and institutions.²³ Large studios do invest a great deal of money and effort into technology development, yet the realities of their R&D contradict these Silicon Valley values in many ways. For one, supporting R&D has meant creating strong connections with public and non-profit research institutions. These connections are quite apparent at the ACM’s annual graphics conference SIGGRAPH. Indeed, media industries’ voracious appetite for nonlinear simulation researchers has largely replaced the once central role Cold War military funding played in supporting research. This new R&D complex between media industries and research institutions affirms the value of the government’s role is supporting research, undermining the idea that nimble start-ups are the prime source of innovation. These findings echo economist Mariana Mazzucato’s work on Apple’s reliance on government-funded research.²⁴ Furthermore, the development and ownership of technologies is something only the largest VFX and animation studios can do, and they use their ownership strategically to maintain control of the market, undermining competition. This strategy works in congress with other market-controlling tactics. Thus, the realities of R&D do not align with the myths of Silicon Valley.

Studios in fact use nonlinear animation paradigms to manage these contradictions. As Chap. 5 argues, Pixar Studio’s approach to management is modeled around the concept of nonlinear control. They create the conditions for the unpredictable and unexpected, but they also contain this chaos within carefully engineered parameters. They style the animating force of nonlinearity as a source of creativity and innovation. Through this they are able to construct an image of themselves as an innovative Silicon Valley business while also being a gigantic, controlling force in their industry.

The Silicon Valley ideology that fuels so many animation and VFX studios favors a more technological determinist view, as Richard Barbrook and Andy Cameron describe.²⁵ Yet the concept of technical animation demonstrates the interactive relationship between society and media technology development. As Raymond Williams notes in his work on television, R&D is a key site where we can observe society’s influence on the shape of media technologies. Media like television were “looked for and developed with certain purposes and practices already in mind,” and R&D is one place where those social desires were turned into reality.²⁶ Thus, film production is not simply being transformed by the introduction of new external technologies; these tools are being shaped by the demands of studios.

Media R&D also does not operate in a vacuum. It works in concert with other technological and scientific research fields. Nonlinear animation has a close relationship with similar tools used everywhere, from sociology to geology. Chapter 2 explores these connections in detail. While these connections sometimes fuel industry promotional rhetoric that positions a studio’s technology as cutting-edge, what this really shows us is connections across an archeological layer of history. As Thomas Elsaesser notes, thinking archeologically about cinema leads us to pay closer attention to the “S/M (science and military) perversions” of cinema: different conceptualizations and uses of the moving image in science, medicine, surveillance, and military applications.²⁷ We have historically neglected these “parallel histories” of cinema. The relationship between media industries and the institutions and businesses that sponsor nonlinear simulation R&D form a kind of epistemic feedback loop, with neither technology nor practices nor discourses nor institutions being the sole source of the conditions of knowledge, but instead with each feeding into the other. This book seeks to uncover the “archive” of this place in history, in the Foucauldian sense of the word, the “system of enunciability,” the totality of both knowledge and power.²⁸

FX

FX is increasingly becoming the most popular industrial term to describe nonlinear animation. On the one hand, it is probably the least descriptive of all possible terms. It does not tell us anything about how nonlinear animation works. But it does tell us a great deal about the role it plays in contemporary production. One might assume that FX, a seemingly unnecessary short form of “effects,” would refer to any visual or special effects, but the term has gained its own particular meaning, distinguishing nonlinear animation as a special form of production within already special modes of production.

The term FX points to a key question haunting the study of contemporary special and visual effects. At the 2013 Magic of Special Effects conference held in Montreal, a preponderance of scholars addressed the question of whether special effects is the correct term to use to describe their object of study. How special are special effects anymore? Were they ever special or exceptional? There is little difference between digital post-production work like color correction and VFX work like keying-in backgrounds, and digital post-production is quickly replacing many filmmaking jobs that we once considered standard. This lead scholars such as John Belton to conclude that special effects are no longer special but standard practice.²⁹ This difficult distinction applies not just to special effects versus standard filmmaking practices but also to the distinction between animation and VFX. In his influential book Digital Visual Effects in CinemaStephen Prince argues that animation and VFX now belong to one single, large, undifferentiated group of motion graphics.³⁰ In other words, digital tools having flattened former conventional differences. Other scholars have contested this assumption, however. Julie Turnock contends that there are still important conventional differences between VFX and animation production practices.³¹ So which is it? Do old categories still matter or have all images become the same and thus eliminated any special categories of production?

There is a third possibility we could consider, that, as Wendy Chun argues, digital technology is producing divergence and variety rather than convergence.³² New forms and differences are taking shape with their own specificity and do not necessarily conform to old categories. FX (nonlinear animation) cuts across categories like animation, VFX, and video games, but it represents a discrete category of image making within these different fields. Every large VFX, animation, or game studio will have an FX department. As one artist who worked in both VFX and animation explained to me, FX is the extra flair, “the icing on the cake” of a VFX or animation project.³³ FX demarcates a special kind of image making within industries that we would already consider special.

The terms used to describe nonlinear animation give us clues as to what exactly makes it special. The work of producing these animations entails making unpredictable simulations rather than directly controlling the image. This work is technical in nature, putting particular emphasis on making and customizing software. Nonlinear animation is to animation and VFX what special effects used to be to cinema, an exceptional practice that puts particular emphasis on unconventional technical work and custom solutions for a particular effect.

Media industries have been seeing a broad shift in production labor over the past few decades. More and more film production work is being done by ranks of technicians sitting behind computers, and the ascendance of media forms like video games, which require extensive technical work, has further fuelled this trend. It has also become a commonplace for observers and critics to bemoan the lack of risk-taking in industries like Hollywood, especially in their most VFX and animation-laden features. It is easy to see the minute control and techno-centric nature of VFX and animation as an extension of this.³⁴ These highly technical VFX-laden productions do not abhor risk though; they conceive of risk differently. They know the value of novelty, of surprise, and chaos, and they have developed strategies for occasioning that contingency in such a way that they can control it. The risk we can see on screen takes the shape of nonlinear animation, of explosions, smoke, and water that look just chaotic enough to be uncanny, yet which can also be shaped by artists. Films such as these support extensive R&D work that has uncertain outcomes but pays potentially great economic and competitive dividends. This way of thinking about risk suffuses numerous facets of economics and management beyond media industries. In the same way Mary Ann Doane analyzes cinema’s relationship to contingency in the context of industrial modernity, nonlinear animation can be studied as the product of broad historical epistemic change.³⁵ Nonlinear animation represents an approach to risk present across society that we can see at work in movies of the past four decades as an animating force. Rather than being a betrayal of the cinematic tradition, it is a new chapter that responds to its historical context in the same way cinema always has, and it represents a repetition of the vitalizing, enlivening force of animation.