Digital Communication as a Creative Tool

Abstract

The progress of computing and digital media is a relevant and useful case study in tools for creativity. Which visions of the future in the early days of computing have stood the test of time, and which have vanished without trace? Can this be used as guide for current and future areas of research and development? If one Internet year is equivalent to seven calendar years, are virtual worlds being utilized as an effective accelerator for these new creative ideas and their implementation and evaluation? The nature of digital media and its constituent parts which integrate technology and various forms of creative media provides a diverse environment which can be viewed as a testbed for current and future ideas. Individual disciplines utilize virtual worlds in different ways. As collaboration is often involved in such research environments, does the technology make these collaborations more creative and effective? Three Case Studies (European Projects) are included to illustrate how creativity can be enabled and stimulated in three kinds of collaborative virtual environments. Two further Case Studies are also included to illustrate how multimedia assets may be harnessed and made available to viewers.

6.1 Introduction

Humans have had an inherent need to create and since the dawn of civilization they have been visualizing ideas using drawings and models to research designs that communicate concepts that transform and even change traditions. Creative expression can advance ideas and contribute something new, but at the same time this is inevitably linked to the social and cultural experiences of individuals and communities within which these artifacts were created. The quality of the art produced by a nation is a by-product of the extent to which its culture, traditions and economy is developed. Artworks can define these cultures, secure their place in history, and provide a legacy for future generations. Communication is an important element in creativity with openness and integrity required on the part of the communicator and the receiver(s). Such openness shows a willingness to engage with new ideas and new cultural contexts, and also to reciprocate through sharing ideas and facts in a quest for new understandings of different concepts.

Thus, the development of human creativity has been transformed by communication sources that transfer information such as speech, writing, images, reading, observing, and printing. Tools have been developed to assist in this, with the printing press being a significant example of how the sharing of information has facilitated the process of learning. The increased capacity to communicate means that many channels are now available allowing communities to be more open and innovative. News ways of thinking and working are able to support the generation of new concepts and ideas more readily than closed environments.

6.2 Computer-Assisted Creativity

New tools, frameworks, and scenarios for the expression of new ideas and new designs, are now available, as well as an increase in the use of digital environments that delivers new types of interactions within which ideas can be generated, revealed, and discovered. The degree to which new technologies advance the creative process is discussed along with the central factors that govern it and how it is augmented and analyzed. The ever-increasing volume of information being generated and circulated by digital systems is equivalent to that brought about by the printing press and could thus constitute a new technological paradigm. The extent to which this may constitute a paradigm shift on the status quo is analyzed. Extracting meaning and knowledge from very large sets of information is a significant challenge often requiring multidisciplinary expertise from a range of discipline backgrounds.

The Internet and the digital world represent a substantial change to the way much of global society operated in earlier eras. It is relevant to consider such a change in the context of Kuhn’s concept of paradigm shift evidenced in the scientific and industrial revolutions [1]. There are substantial differences between the medieval era and the modern era brought about by a greater understanding and application of scientific analysis. However, the extent of this change normally only becomes apparent sometime after it has taken place, when there has been opportunity to reflect on the significance of the developments and advancements that have taken place consequent upon the change.

The evidence to date in favor of the Internet and the digital world being a paradigm shift is that they were completely new. The complicating factor is that many of the uses and applications that developed out of this digital framework operated in a more basic form prior to this. Thus, the key question for future historians to answer is whether the Internet and the digital world can demonstrate that major new developments and transformations followed directly from this digital foundation, and that they did not exist before, and would not have followed on naturally from changes in science or society. They were major and transformational and, as for the Scientific Revolution, it is unlikely we would wish to a return (at least in a practical sense) to the state before the revolution happened. This supports the argument for a paradigm shift associated with the digital world.

What are the key aspects of this paradigm shift in the digital world that can most affect society? In this section it is proposed that connectivity and hybridity enable and facilitate the digital world’s transformative effects.

6.3 Interaction and Collaboration

An important component of creativity is communication. This presupposes a degree of openness on the part of the communicator and the recipients(s). Such openness indicates a readiness to receive new ideas and new cultural contexts, and also reciprocate by sharing information. Thus, the development of human creativity has depended to a large extent on speech, writing, images, reading, observing, and printing. The latter took a major step forward with the development of the printing press. This has greatly facilitated the process of learning and the sharing of information.

There are various theories as to what purpose historic cave paintings served but one aspect of their production is clear which is that they all involve an interaction of a human with some kind of tool. The human has used the tool and pigments to create a painting on the wall of a cave to represent in visual form aspects of their live that they have intimate knowledge of such as the hunting of animals.

The relationship of the human to the animal could have been preserved in some form of verbal tradition but through the cave drawing something additional to this was also practiced and has been preserved. This form of creative expression arose out of experiences with the external world. A human’s ability to interact with their environment in a conscious and creative way is what distinguishes them from the animals that don’t interact through conscious reasoning but through instinct and habit. Humans used tools from the earliest eras, formed from various kinds of materials.

6.4 Virtual Environments and Creativity

Tools are utilized in order to interact with a virtual environment where a visual representation of an object can be designed or manipulated (Fig. 6.1). Immersive walk-in environments (such as a Cave—Fig. 6.2) enables the user to be surrounded by a virtual representation as if it were the real world, allowing human potential for creativity to be facilitated by technology. Conversely, if the technology is not fully oriented to the user’s way of working then it can act as an inhibitor and a constraint. This underlines the importance of having well-designed user interfaces that enable the users to interact in creative ways with the virtual worlds via the technology.

Fig. 6.1

Virtual environment—a researcher using VR headset to investigate ideas for controlling rovers on a planet (a team of researchers at ESA’s mission control center in Darmstadt, Germany, are investigating new concepts for controlling rovers on a planet and satellites in orbit). This image is licensed under the https://creativecommons.org/licenses/by-sa/3.0/ license

Fig. 6.2

Engineering teams can use the CAVE at INL’s center for advanced energy studies to tour a virtual nuclear reactor, train staff, orient subcontractors and consider new designs. This file is licensed under the Creative Commons Attribution 2.0 Generic license

The name CAVE is thought to be a reference to the allegory of the Cave in Plato’s Republic. Virtual environments are akin to the allegory in which philosophers chained to the wall of their cave contemplate perception, reality and illusion though watching the shadows projected on the wall from objects passing in front of a fire lit behind them. The user’s immersion in the Cave as depicted in Fig. 6.2 is thought to provide an additional sense of realism over and above that which would be experienced by observing a 2D image or even a 3D stereoscopic image with depth cues.

Immersion of a human in an environment can provoke a kind of “suspension of disbelief” where the unreality is able to be ignored. Even though the world displayed is artificial, it is made to feel more real because the observer feels they are a participant within it. Virtual environments can offer advantages in the design process and have been successfully used for the presentation and simulation of objects and spaces. Virtual environments are more efficient at helping designers perceive volumes, spaces, and spatial relationships than in 2D environments [2].

Computer technology offers the user the capacity to hypothesize from the current situation and project into the future with millions of alternatives calculated in a fraction of a second. Artificial intelligence techniques can help in the selection of the ones likely to be the most interesting to perform interactive analysis on, that cannot be explored on a human time scale.

Case Study 1—European Project SynchroniCity

“SynchroniCity” is an Internet of Things (IoT) project funded by the European Commission (2017–19) [3]. Its aim was to open up the market for IoT-enabled services for cities and communities where barriers prevented local economic and sustainable development. It identified where that the lack of standardized application programming interfaces (APIs) made it difficult to integrate new technologies into city services. The risk of ‘lock-in’ to systems deters large investments in smart city infrastructures and poses real challenges for local authorities and other city organizations.

In response SynchroniCity developed a framework in the form of a set of technical mechanism that enabled local authorities and technology providers to easily exchange digital products, services and data [4]. The “Synchronicity Framework” is built around the Open and Agile Smart Cities (OASC) and Minimal Interoperability Mechanisms’ (MIMs), that together provide the technical foundation for deployment of IoT- and AI-enabled services for cities and communities. The MIMs are vendor-neutral and technology-agnostic and can be integrated with existing systems. This means that local authorities and technology providers to easily exchange digital products, services and data.

The project involved 50 developments of products and services across 21 cities across Europe where pilots were SME-led groups. The foci of the pilot studies were as follows: citizen engagement, environment wellbeing, and sustainability.

Sustainable mobility was the focus of one of the pilots providing active travel insights in Antwerp, Helsinki and Manchester. Detailed understandings of cyclists, pedestrians and vehicle movements were made across the cities to promote green travel. The IoT solution combines data feeds from 3 different sensors, with comparative open-air quality data provided by local authorities. Data from all sources was centralized in a dashboard and the following impact was achieved:

• 42 IoT devices were deployed.

• 98 existing IoT devices were connected.

• 111 datasets were consumed.

• 613 new datasets were generated.

• 158 open datasets were produced.

• The Technology Readiness Level of the team’s IoT solution improved from 6 to 8.

The pilots demonstrated that through using the framework:

It is possible to transfer social and environmental impact to local economic activity, as well as opening new market opportunities for both local authorities and technology service providers [5].

What makes data most useful is not having the most advanced technical capabilities but lowering institutional barriers and identifying the problems that data can address [6].

Technology is evolving at an ever-increasing rate but all too often institutions and organizations and communities do not have easy access to cutting-edge products and services. Therefore, it is essential that technology providers develop hardware and software that does not have barriers to its use. Future-proofing their IoT services is essential, and this can only be done by engaging with communities who use it and incorporate it into their business models.

Case Study 2—i-MareCulture—Advanced VR, iMmersive Serious Games and Augmented Reality as Tools to Raise Awareness and Access to European Underwater Cultural Heritage

i-MareCulture is a European project for digital heritage and virtual museums to raise public awareness of European identity by focusing on maritime cultural heritage. i-MareCulture aims to bring inherently inaccessible underwater cultural heritage to the public by implementing virtual visits, serious games with immersive technologies, and underwater augmented reality. Through the concept of the virtual museum, collaborative research from a range of different scientists, researchers, archeologists, experts and museums ancient sites, many that remain submerged or threatened are brought to life to bring public awareness to important archeological relics. Ten partners from central Europe are developing visualization approaches, and methods including virtual and augmented reality, to improve visibility and help to protect and preserve important archeological heritage.

The project website has a number of tools and games that have been developed, and are open access to the public, including an image enhancement tool and a navigation algorithm. The enhancement tool implements five algorithms aimed at enhancing the quality of underwater images that can automatically process a set of images within a directory. The navigation algorithm Seafaring Webservice, offers the user a visual simulation of Kyrenia’s ancient maritime paths based on real archeological data [7, 8]. By raising awareness of important sites, the regional economic development will improve through tourism and an increased sense of local identity.

Case Study 3—European Project VISTA AR

VISTA AR (Visitor experience Innovation through Systematic Text Analytics & Augmented Reality) is a 4-year European Interreg (European Regional Development Fund) project which started in July 2017 and was extended to December 2021 due to the coronavirus pandemic. It aims to explore and improve visitor experiences for tourist attractions in the South of England and the North of France through creating new VR and AR digital tools.

One site is Exeter Cathedral where virtual reality helmets, tablet and smartphone devices, and immersive room experiences were used to retell history through allowing visitors meet characters from the past and providing access to inaccessible heritage artifacts. The digital tools developed will create new digital experiences in further locations including the National Trust Tin Coast, East Pool mine and the South West Coast Path in the UK and in France at Fougères Castle, the Lorient Submarine Museum and the Gardens of Valloires [9].

The adoption of new technologies by artists and the increasing use of VR and AR as a medium for the production of artworks has stimulated a more detailed reflection on the processes of human creativity. Computer-assisted learning has prompted the study of pedagogy and human learning processes and models of artificial intelligence have initiated a more detailed examination of human intelligence [10,11,12,13].

With the increasing ubiquity of the digital environment there are new possibilities for iterating ideas through different visual representations at a greater rate than was possible using previous manual methods. When data is held in digital form it is possible to run simulations and have virtual ‘walk-throughs’ which has proved particularly important in architectural design and also the design of 3D objects.

With digital technology it is also possible to optimize the designs for situations where there are internal or external constraints, for example, in the total budget for the materials for a building, or construction costs, or in the implementation of an artwork. These include tools for design, mock-ups, real-time simulations, analysis of final representations, and usability testing of final configurations of objects, architectural spaces, and the environments which contain them. For example, designer Thomas Heatherwick’s 3d models and drawings of his architectural project “Vessel” the controversial architectural centerpiece for Hudsons Yard, New York, can be seen on his website [14]. The Heatherwick Studio recently replaced its CAD data management system with Dassault Systèmes’, a Product Lifecycle Management (PLM) solution. This software is used to increase efficiency and reduce the amount of time spent on repetitive design tasks. This can enable more time can be spent by designers on the creative aspects of the projects [15].

Although physical 3D scale models gave previous generations of builders a good understanding of the structure of the building and how it would sit in its environment, it was difficult to fully understand how it would feel at full scale, particularly to a person moving from room to room inside the building. Therefore, it may be difficult to fully envisage how far computers and associated technology will continue to transform the way information is processed, and the way we think about creating in the future.

6.5 New Media in Cyberworlds

With the rise of new media and digital art forms there has been increasing scope for a wider variety of art objects, particularly those that are able to interact with audiences. Such interactions may change the artwork, or the perspective of the viewer, or both. In addition, time-based media such as video may be used in exhibitions either in free-standing mode, or be able to receive input from viewers to change the content of the video being displayed. A virtual exhibition can also be open to global audiences which can result in a sharing of cultural and ethical experiences across national and international boundaries.

In 2018, Nottingham Contemporary hosted a virtual exhibition titled “House of Fame”, that was a novelty at the time allowing the viewer to move, rotate and zoom in the exhibits as if they were there in the gallery space [16]. Today, with the acceleration of the virtual experience as a result of the Covid-19 pandemic many galleries offer the experience of virtual exhibitions on their websites (e.g., https://v21artspace.com/the-house-of-fame).

This introduces the concept of exhibition spaces that are borderless with respect to time, space, and audiences [17, 18]. An artist who wishes to exhibit their work in this rapidly changing environment faces two principal challenges. The first is to demonstrate the relevance of the art to today’s audiences and the rapidly changing environment. The second is to articulate forms of art which are capable of transcending the boundaries of past traditions and demonstrate new horizons and new opportunities. We are ever-increasingly connected to one another and ever more accustomed to swiping our phones to access digital content. Digital technology can offer a greater level of interaction and increase knowledge and allow for richer experiences of culture if developed intelligently.

Just before the pandemic in 2019 Innovate UK funded a research project at MOSTYN Gallery in Llandudno, North Wales to investigate how audiences want to interact with a public art gallery in the digital age [19]. Principal and co-investigators on the project Clare Harding and Mark Lochrie reflect on the impact of lockdowns and social distancing on the development of their engagement tools. They can use MOSTYN build tools to engage and connect audiences who may not be able to engage physically. These digital experiences can be even more attractive in an age of ‘track and trace’, zoom parties, and with the use of QR codes to check in to venues. However, the sharing of the head mounted VR displays or wearables still present challenges for technologists, curators and audiences in galleries [20].

Case Study 4—EMAP European Media Art Platform

The European Media Art Platform (EMAP) was created in 2018 to provide an international platform to promote and disseminate interdisciplinary art forms. It was believed that if Europe is to take a different technological path from China or Silicon Valley, it should concentrate on the creative potential of the arts.. Through the European Media Artist in Residence Exchange (EMARE) it enables partners to select artists and artwork for festivals and exhibitions to provide an international platform to disseminate their work.

Rhone (2021) was an artwork by Gil Delindro stemming from a 6-month EMAP residency undertaken in La Becque, Switzerland. A video sound installation on the theme of the environment, global warming and geological sustainability, Rhone, explores the symbiotic relation between Glacier erosion and sedimentary geologic formation. The artist used a series of infrasound field recordings and video stills captured in the Ice tunnels of the Rhone glacier over a period of three months.

The Rhone was once the biggest Glacier of the Alps, being the main responsible for the geologic formation of Lac Leman and the surrounding alpine landscape. This project follows the connection between the retreating “Rhone Glacier” and “Genetti” - an industrial site that sits 100 kilometers after on the Rhone River [21].

Case Study 5—Cop26 Artists and Climate Change

The UK hosted the 26th UN Climate Change Conference of the Parties (COP26) in Glasgow in November 2021. Artists took the opportunity to participate in the conference with musician Brian Eno hosting a panel on how artists can respond to the climate emergency. Significant works included a projected text piece by Jenny Holzer that quotes activist Greta Thunberg. Artist collective Still/Moving installed the artwork alongside the COP26 conference site. The work comprised a 70-m-long and six-meter-high artwork stating that there are ‘No New Worlds’ lit up in neon.

Los Angeles based artist Beatie Wolfe projected her artwork From Green to Red onto the contemporary SEC Armadillo auditorium on the banks of the Clyde in Glasgow with the projection covering the entire facade of the building for the UN Climate Change Conference (COP26). The work visualizes rising levels of CO₂ in the atmosphere and is based on 800,000 years of NASA data that informs the interactive video installation by Wolfe. Technology allows artists to take impenetrable and intangible data by scientists and translate it into something they can engage with and absorb.

Wolfe’s song accompanies the artwork acting as a giant music video to tell the story of the 50% rise in carbon in the atmosphere due to the activities of mankind. From Green To Red was produced in collaboration with world-renowned specialists in visual effects, creative production and immersive experiences The Mill and was premiered at the 2020 London Design Biennale [22].

6.6 Conclusions

Virtual and augmented reality offer spaces which can enable participants to visit places as they were in the past and which are able to impart a real sense of actually being there. Where a historical site is a virtual reconstruction from archeological data, then this experience is facilitating a visit which would be impossible in the real world. This augmentation to real-world experience is both valuable and transformative, as well as being normative for the future.

The affordances provided by the digital world generate new dimensions of connectivity and communication that are particularly relevant in the context of this research. The digital environment can be an enabler in ways not initially envisaged. One example of digital affordances is the many to many paradigm established by the Internet. Networked communications override the one-to-many model used by broadcasters and enable every communicator to be regarded as significant and a contributor to the whole.