Keywords

1 Introduction

Virtual Reality (VR) art-making tools have become increasingly popular amongst artists. However, VR art is still far from ubiquitous, mainly because of the high cost of purchasing a VR system and the space required to run it. At the same time, VR artworks are fundamentally difficult to view or exhibit outside of VR, as they have a functionality that makes them impossible to reproduce in any other medium without losing some of their essential features. For example, 3D printing, due to its particular constraints, only allows for a relatively narrow range of models to be produced [1]. Online galleries which allow the viewer to explore VR projects using a standard monitor, flatten the three-dimensionality of VR artworks, as is the case with any 2D representation of a volume [2].

Digital holography, despite its own limitations, can constitute good support for VR imagery, offering a high fidelity of representation and preserving the most important properties of such imagery.

The advantage of bringing together these relatively new media is that it allows an artwork produced in VR to be displayed and viewed easily in either a public gallery or a private space, by a much larger number of people simultaneously, without the constraint of requiring viewers to wear a cumbersome headset in order to preserve the 3D appearance.

2 Overview of Holography as an Artistic Medium

In simple terms, a hologram is a photographic recording of a light field, on either film or glass, resulting in a seemingly 3D image which can be seen with the naked eye. Holography can be either analog, when it captures something which exists in reality, or digital, when it displays computer-generated objects or scenes. Both types of holograms require a special, dedicated illumination system in order to be viewed, in the absence of which they appear as dark, indistinct, flat surfaces. There are a number of techniques and optical illusions which are commonly and mistakenly referred to as ‘holography’, such as Pepper’s Ghost or various stereoscopic and lenticular displays. Although holography practitioners will reject these as mere tricks, the general public is perhaps more familiar with such techniques than with holography itself, due mainly to the fact that they are often employed in funfairs, in theatre and in other on-stage acts. Arguably the most notorious use of Pepper’s Ghost was Tupac Shakur’s virtual appearance at Coachella Valley Music and Arts Festival, in 2012, alongside Dr. Dre and Snoop Dogg [3].

Although primarily a scientific field, holography has a significant potential for being used as an art medium and, as a consequence, the domain appeals equally to artists and scientists. Since its invention by Denis Gabor in the 1940s, holography has drawn the attention of a number of artists, amongst whom Salvador Dali is perhaps the best known [4]. Many of these artists have dedicated their career to the medium of holography, for example, Margaret Benyon [5] or August Muth [6], whilst others have tried to incorporate it in, or add it to, their existing practice, as in the case of Moysés Baumstein.

Catalogues, photographic documentation and generally any type of literature on holography don’t really do justice to a medium that doesn’t lend itself easily to 2D reproduction. However, articles on holography written by practicing artists, curators or critics are useful for understanding how holography was, and is, perceived and experienced first-hand.

Two volumes of the Leonardo art and science magazine dedicated to holography art [7] comprise numerous interviews and articles where holography art practitioners amongst whom Margaret Benyon, Andrew Pepper and Paula Dawson express their dissatisfaction with how the art world receives their work.

Perhaps the most striking concept that comes from these texts, and others written later, is that holography and the art establishment still don’t feel comfortable with one another. The medium of holography has encountered constant criticism and dismissal from the art establishment. The main points made against it being vulgarity of colours, awkwardness of display and lack of a proper critical and conceptual vocabulary. This is revealed by D. Tulla Lightfoot in a study titled ‘Contemporary Art World Bias in Regard to Display Holography’ [8], where the author interviews several curators and directors of art galleries in New York, seeking to understand their views on holography art and how they justify the constant rejection of this medium by the establishment. The main conclusion of the study is that major museums and galleries don’t necessarily dismiss the medium itself—they base their rejection on the status of the artist, claiming that, if an artist who has previously gained recognition by working with other media decided to take up holography, they, the art venues, would not hesitate to show their work. If this is the case, one might infer that holographers simply face the same problems as any other artist approaching a venue seeking representation; with the added difficulty of having chosen a medium that hasn’t really managed to prove its aesthetic or conceptual value, as yet.

Nevertheless, there seems to be an ongoing struggle amongst holography practitioners to be accepted and valued or sometimes even considered by major art venues, in spite of long years of sustained efforts and obvious developments, both in the field of holography as a whole and within the individual practices of different artists. During a presentation at the International Symposium for Display Holography 2018, August Muth, one of the most successful artists working with holography today, confessed that when approaching a gallery to propose a collaboration, he simply steered away from using the word ‘hologram’ in reference to his artworks. This was to avoid the immediate negative reaction and consequent rejection which are usually triggered by the mere mention of this word.

It must be acknowledged that the medium still has glaring deficiencies and limitations that make holographic artworks unconvincing and could be the reason for recurring negative responses. For example, common limitations mentioned are brightness and colour balance, the complexity of arranging good replay illumination, blurring in the rear of a scene, vertical non-uniformity of colour, digital pixelization and the lack of a good archival quality photographic recording material.

Perhaps one of the reasons why many attempts at making holography art have failed, is that the practice of holography bears no clear line of demarcation for where the science ends and where the art begins or vice versa. In order for a holographic product, and any new media artwork to be successful from both points of view—technical/scientific and artistic—this distinction needs to be acknowledged and embraced.

Some of the most successful examples of holographic art come from practitioners who have understood the importance of limiting the role of the artist and that of the scientist in order to allow each of them to contribute optimally in this equation.

A collaborative approach between artists and scientists is needed to enable the two fields to intersect in a successful way that leads to further progress. New media often come with the temptation to use their novelty and specificity as an objective in itself. As a result, practitioners with a background in art are unable to produce artworks of satisfactory technical quality, whilst technicians or scientists with no artistic background, seduced by the visually enticing qualities of the media, produce trivial objects of no real artistic value. When working with a difficult medium such as holography, it is important that both the artists and the scientists understand the limits of their knowledge, and by using each other’s expertise and providing constant feedback, work together towards taking the technology to a level that allows it to be acknowledged by the establishment.

3 Virtual Reality Art

To clarify the terminology, VR art is a type of artistic content created directly inside the VR environment using an application designed specifically for the purpose of art-making, and not computer-generated content presented in VR.

VR art-making applications such as Tilt Brush or Gravity Sketch are impressive tools for creating digital 3D imagery, offering great new possibilities for art and the creative industries. They stand out amongst other VR applications due to the fact that they allow users to generate virtual reality content, rather than simply consuming it. This unprecedented combination of painting and sculpture, not bound to physical laws, with its lack of spatial limits and highly intuitive interface, offers a very satisfying immediacy that materializes gestures instantly.

With its impressive variety of brushes often accompanied by special visual and sound effects, Tilt Brush was designed with the intention of offering the user not just art tool but a fully immersive VR experience, entertaining in and by itself. On the other hand, Gravity Sketch, with its focus on creating geometry in the VR environment, has proved a real game changer by providing a level of editability and precision much superior to Tilt Brush, features which appeal greatly to design-oriented creatives. More recently, the team behind the VR animation tool Tvori aim to create a powerful, immersive and intuitive tool for crafting visual stories, which anyone can use, regardless of their previous experience with either 2D or 3D animation [9].

Currently an increasing number of artists have expressed their interest in this medium and have started adding VR art-making to their ‘tool box’. However, because VR art is still in its infancy, and also because of the difficulties associated with showcasing these productions outside of VR space, the works realized in this medium are only accessible to the wider public via online 2D galleries [2].

4 The Creation of a New Artistic Medium

The most important achievement of holographic imagery and what truly differentiates it from other media is its capacity to visually expand space without resorting to the traditionally accepted convention of treating a 2D surface as an interface for 3D content. In seeking to redefine maximalism as a type of art which uses a minimum amount of space to deliver a maximum amount of information or content, holography becomes a good example of an efficient use of the ‘space of art’, in the sense that it employs a flat surface to display a volume, rather than the actual 3D space usually taken up by a physical object or scene.

4.1 ‘Van Gogh by Pioaru’

By using this unique quality of holography in conjunction with VR art, there is an opportunity to not only bridge the gap between the virtual and the real, but also to develop a new form of art that could be unique in itself.

Therefore, a digital hologram was created from a VR artwork. The project was implemented using Google Tilt Brush and an HTC Vive headset and, since it was a reinterpretation of a Van Gogh self-portrait, it was entitled ‘Van Gogh by Pioaru’.

In order to transfer a VR project onto a holographic format (and, to my knowledge, the first time anyone had attempted such a transfer), attention was focused principally on the technological aspects of the process, and the choice of subject was a secondary consideration.

Part of the process of transferring the VR project to holography consisted of using the FBX file format for exporting the scene. However, due to various software incompatibilities, only the volumes were imported from TB into 3ds Max, whilst the colour and texture information was lost. This meant that the scene had to be recolored in 3ds Max. In addition, when imported, the brush strokes were reduced to plain, ribbon-like meshes. All this resulted in significant differences between the original VR scene and the final hologram, which was printed using a commercial digital holographic printer from the data set (Fig. 17.1).

Fig. 17.1
figure 1

Copyright © I. Pioaru 2019

A snapshot taken in Tilt Brush (left) and a photograph of the final hologram (right).

Full size image

4.2 ‘Spectral Figures’

In spite of all technical difficulties, Van Gogh by Pioaru was a promising first step. The artwork was well received at a public demonstration. However, there were some constraints and these were addressed in a follow-on project, ‘Spectral Figures’ which made adjustments to the limitations of the transfer process. This consisted of a series of black and white portraits of artists and philosophers (Fig. 17.2). These were drawn in black and white in order to reduce the risks associated with recoloring the scene (described in the previous section). In order to maintain a degree of similarity to the models, several photographic portraits of the subjects were imported into Tilt Brush. These were sourced online and taken from different angles, and used as reference images for the artworks, with the objective of building into the models the correct volumetric structure. Thin black strokes were used set against a white background in order to simulate the characteristics of drawing on paper. As a consequence, possibly the most striking aspect of the resulting images is that, although effectively they are sculptures, every viewing angle gives the illusion of a 2D line drawing [10].

Fig. 17.2
figure 2

Copyright © I. Pioaru 2019

‘Spectral Figures’ series—a collage of snapshots taken in Tilt Brush.

Full size image

This decision brought the project a step closer to a preferred artistic media, i.e. drawing, and particularly to ‘sculptural drawing’. This is a concept which has been approached in several recent projects and it refers to expanding the characteristics of drawing to create sculptural objects, as part of an exploration of the boundaries between 2D and 3D imagery. Figure 17.3 is a visualization of a project which best illustrates this concept: Insectarium, 2012—a sculptural drawing installation—followed by two close-up photographs (Figs. 17.4 and 17.5) of one of the boxes which make up the installation.

Fig. 17.3
figure 3

Copyright © I. Pioaru 2019

‘Insectarium’ by Ioana Pioaru—sculptural drawing installation, 2016.

Full size image
Figs. 17.4 and 17.5
figure 4

Copyright © I. Pioaru 2019

‘Insectarium’—sculptural drawing installation, close-up.

Full size image

The pipeline for creating a hologram from a VR project is relatively straightforward. In order to generate the image data required by the digital holographic printer for a single-parallax hologram, the VR project needs to be exported as a Filmbox (FBX) file—a feature initially implemented into Tilt Brush to allow artists to share their creations online. This is a format that makes it possible for digital content to be manipulated across a variety of digital creation software. The incompatibilities between Tilt Brush and 3ds Max were addressed by simplifying the scene and using a minimum of colours—effectively just black and white. This reduced the time of post-processing significantly, the main remaining task being the setup of the camera for image rendering. This is achieved by programming an automated camera in 3ds Max, a process described by David Brotherton-Ratcliffe and Hans Bjelkhagen in their book ‘Ultra-realistic Imaging’ [11]. This method produces the required perspective views that can be pixel swapped to generate the data necessary for the digital holographic printer.

The data was sent to Geola for printing on their digital holographic printer and a series of 30 cm × 40 cm colour reflection holograms was produced using a silver halide emulsion. The holograms, when properly illuminated, produce a good impression of the original VR subject (Fig. 17.6).

Fig. 17.6
figure 5

Copyright © I. Pioaru 2019

‘Spectral figures #4’, single-parallax hologram (three photographic views).

Full size image

Digital holograms may generally either be ‘Full-Parallax’ or ‘Single-Parallax’. ‘Single-Parallax’ holograms are also sometimes referred to as ‘Horizontal-Parallax-Only’ holograms. In general, the ‘Full-Parallax’ hologram is to be preferred as it presents the most general and most faithful 3D experience of the original light field to the viewer. The viewer will be able to see both to the left and to the right of an image, as well as properly perceive the image from above and below. The viewer will also be able to approach the image and observe it from close-up; in brief, a full-parallax hologram will relatively faithfully reproduce the original 3D light field from the active side of the hologram.

Single-Parallax or Horizontal-Only Parallax holograms remove the vertical parallax from the light field. As the viewer moves their head up and down, the 3D image will appear to simply tilt up and down. Since human eyes are horizontally separated, this type of hologram still looks very realistic to most people. The tilting of the image is fairly well accepted by the viewer and interpreted as a natural movement of the image itself. The largest drawback with single-parallax holograms is that the viewer cannot approach too closely to the hologram without image distortion becoming excessive.

There are several advantages of the Single-Parallax hologram. The first is the digital image data is usually some hundreds of times smaller than the data required for full parallax holograms. This means that whilst single-parallax holograms can be calculated on a normal laptop or PC, full-parallax holograms often need either a reasonably large network of PCs or a small supercomputer. The second principal advantage of the single-parallax hologram is that it can be illuminated much more easily than the full-parallax hologram leading to brighter images and deeper in-focus scenes.

Nevertheless, full-parallax holograms stand out as being far superior in that they convey the 3D reality of the drawings in a much more efficient and successful manner. The missing vertical parallax is in the single-parallax holograms acts to destroy the illusion of the image integrity that these sculptural drawings seek to embody. The single-parallax holograms are brighter and they can be illuminated by arrays of lights, making them potentially much brighter than the double parallax holograms. However, this does not really compensate for the critical loss of the vertical parallax.

Fully un-apertured double-parallax holograms, giving the maximum field of view, is useful if the hologram is to give the best illusion of reality, printed on Silver Halide material. However, this may inevitably lead to a relatively poor brightness at replay, even when powerful LED lights are used for illumination. Improving this brightness is an important ‘next step’. Potentially, photopolymer could be used instead of Silver Halide, but a few companies are currently offering this choice. Other solutions would be to use more powerful laser diodes for the illumination. The best possible result would be expected if dichromated gelatin glass plates were to be used in the printer.

4.3 Tilgate Forest

The starting point of this project was a homonymous ink drawing first created in 2018. This was an opportunity to explore a theme that was different from previous VRH artworks, which had been exclusively portraits. In addition to this, since the landscape is rather underrepresented in holography this choice of theme seemed likely to produce an interesting result.

A photograph of the original was imported into Tilt Brush to use as a reference image (Fig. 17.7).

Fig. 17.7
figure 6

Copyright © I. Pioaru 2019

Tilgate Forest by Ioana Pioaru, original ink on paper drawing, 2018.

Full size image

Several factors identified in the initial drawing pointed to the possibility of structuring a 3d model of this scene as a diorama: the way in which the elements of the composition are clearly individualized, grouped and layered at varying depths; the relative difference in scale of these elements; and the amount of graphic detail used for rendering each layer. Thus, whilst drawing the scene in VR, the composition was distorted by making objects in the distance much smaller and closer to the main viewpoint than they would be in an accurate scale model of the real landscape. Figure 17.8 shows the scene from the main viewpoint and Fig. 17.9 shows what it is as seen from above.

Fig. 17.8
figure 7

Copyright © I. Pioaru 2019

‘Tilgate Forest’: snapshot taken in Tilt Brush.

Full size image
Fig. 17.9
figure 8

Copyright © I. Pioaru 2019

‘Tilgate Forest’: scene viewed from above.

Full size image

The compressed depth also offered several advantages holographically, primarily that of limiting the angular resolution required to resolve the rearmost parts of the scene. This is a particularly important consideration in light of the fine brush strokes employed. Another advantage is geometric: due to the ‘window’ nature of a hologram, scene components nearer the glass are viewable from a wider angle of view before being obscured by the edge of the hologram. Thus, compressing the scene makes the compositional layout more robust to viewer movement.

The use of the brushes roughly followed the same method employed for previous projects, in that the sense that the brush used for the white areas had to be different from that used for the black areas.

The resulting ‘sculptural drawing’ was close enough to the original ink drawing and quite convincing as a 3D depiction of a landscape, in terms of the parallax effect. However, pure black and white images such as ‘Spectral Figures’ and ‘Tilgate Forest’ represent particularly challenging subjects for holography, straining its capabilities with respect to brightness and colour balance. Additionally, whilst the maximum angular resolution of digital holography itself is state of the art (thousands of views), current replay illumination techniques reduce this by an order of magnitude. As a result, although the depth compression used in ‘Tilgate Forest’ was an effective illusion in its own right, it was insufficient to fully mitigate the blurring imposed on the background of the scene. This somewhat compromised the pen-drawing appearance, the very fine brush strokes seemingly blending together to create a charcoal-like effect (Fig. 17.10).

Fig. 17.10
figure 9

Copyright © I. Pioaru 2019

‘Tilgate Forest’: photograph of the hologram.

Full size image

4.4 Heterotopia

The project titled ‘Heterotopia’ is also based on an earlier work, a two-colour reduction linocut print, part of a larger series (Fig. 17.11). This combined architectural structures and pseudo-realistic machinery, in the ‘techno pop’ style characteristic of recent art practice, where the stark, rigid appearance of the machines and buildings contrasts with the rawness and vividness of the colours and the cartoon style. Gravity Sketch seemed much more appropriate a tool, as it offers the features necessary for achieving this type of imagery: the possibility to generate and edit basic shapes and volumes, a certain degree of precision and perhaps most importantly, a grid and world axes/coordinates system that the objects can be aligned to.

Fig. 17.11
figure 10

Copyright © I. Pioaru 2019

‘Heterotopia’ series: first print (left) and colour permutations digital mock-up (right).

Full size image

As the original artwork was a flat, frontal representation of the architectural/‘machinic’ structure, adding depth was the first obvious step necessary in the creation of the 3D model of the subject. Consequently, a roof-type surface and a lateral wall were constructed and filled with graphic detail, so that in the hologram, the roof—an element that is not conceptually essential to the composition—would be revealed and concealed by the viewer moving their head up and down. The added lateral wall was intended as a ‘transition area’ combining architectural features with ‘machinic’ details (Fig. 17.12).

Fig. 17.12
figure 11

Copyright © I. Pioaru 2019

‘Heterotopia’: collage of three views of the Gravity Sketch model.

Full size image

The post-processing necessary to prepare the data for the holographic printer was different and much more complex than those required by previous projects, and was therefore done in collaboration with Tal Stokes from Geola Technologies, whose knowledge and expertise were essential (Fig. 17.13).

Fig. 17.13
figure 12

Copyright © I. Pioaru 2019

‘Heterotopia’: photograph of the hologram.

Full size image

5 Presenting Virtual Reality Holograms to the Public

The ‘Van Gogh by Pioaru’ hologram was first presented in 2017 at the Cyberworlds International Conference in Chester. During the 2018 Focus Wales Festival, it was on display in one of the exhibitions organized at the Ty Pawb Gallery in Wrexham.

The ‘Spectral Figures’ VRH series was presented for the first time at the 11th International Symposium for Display Holography which took place in Portugal in 2018. On the same occasion, it was part of the holography exhibition held at the Museum of the City of Aveiro, titled ‘Art in Holography: Light, Space and Time’ [12]. In 2019, during the Artists’ Open House event held within the Dulwich Festival, ‘Spectral Figures #1’ was exhibited together with some recent artworks made in other media.

‘Tilgate Forest’ and ‘Heterotopia’ were presented at the SPIE Photonics West conference which took place in San Francisco in February 2019, together with one of the pieces from the ‘Spectral Figures’ series.

On each occasion, the work was well received both by the general public and by artists and scientists attending the events and was deemed highly innovative, particularly the concept and method of combining traditional drawing with the medium of digital holography.

In August 2019, a selection of the VR holograms will be included in a group exhibition which will look at the artistic drive being immersed by the image, organized at the Centre for the Holographic Arts in New York.

6 Discussion

For artists and researchers interested in ultra-realistic imagery, it may be important to understand how these relatively new art-making media, holography and virtual reality sculptural drawing, which share the ephemeral nature of their content, can be employed together and how they can highlight and enhance each other’s potential.

Major companies developing VR technology are striving to make their devices and products more affordable, lighter, wireless, more intuitive and to offer the possibility to swap between AR and VR. This will inevitably impact on how artists make, experience, and understand art. On the other hand, it may not be necessary to find ways of ‘exporting’ productions of VR to other platforms, simply because the best way to experience them will be inside VR. Nevertheless, due to the challenges and difficulties associated with such ambitious projects, VR technology is still far from ubiquitous.

In this transition period, it is therefore relevant to examine how the intrinsic gap between the virtual and the real can be bridged. As a stage in developing Tilt Brush, Google has recently initiated an Artist in Residence programme [13], emphasizing the importance of an active collaboration between artists and scientists in understanding what steps are necessary to take VR art-making to the next level. Because of this, many artists are expected to use VR as an instrument to develop their practice, and this highlights the need to find ways to exhibit their creations.

VR holography is currently dependent on commercial digital holographic printing services. These services take the processed VR data set and generate the physical hologram. Perhaps the main problem for the practicing VRH artist today is that commercial holographic printers usually print on plastic film. To achieve the flatness required for the display of the artwork, this film must be laminated onto either a plastic or glass sheet using an optical glue. Unfortunately, even if the visual result is usually excellent using this technique, the process itself is associated with poor archival properties. This is a significant problem for the artist and indeed for the wider art collecting industry. The situation is slowly changing in this respect and digital glass hologram printers are starting to appear. In the EU, Geola, in addition to offering large holographic prints (up to 1.5 m × 1 m) on film now offers a service for printing some types of smaller glass holograms (up to 15 cm × 15 cm) on photoresist. The Centre for Ultrarealistic Imaging at Glyndwr University in Wales is also planning on recommissioning its large glass printer (capable of printing up to 1.2 m × 1.2 m) later this year.

Another problem with commercial holographic printing is the availability of small hogel sizes. Currently, Geola offers a 0.8 mm hogel with its film printers although its new glass service can in principle goes down to 100 microns. ‘Hogelization’ or the visual pixelation caused by larger hogel sizes is a distracting feature in the context of VR holographic art. It is a particular problem with full-parallax holograms where the nature of the image invites the viewer to view the artwork from all distances including from very close proximities. Reducing the hogel size by a factor of two does however increase print time by a factor of 4. And with a 30 × 40 cm ×  0.8 mm hogel hologram usually printing in around 3–3 h, commercial printers are not keen to reduce the hogel size dramatically. Nevertheless, technology is again coming to the rescue here with the latest generation of digital printers now forecast to offer printing speeds between 120 and 180 hogels per second—a significant improvement over typical current rates of 30 hogels per second.

A further problem encountered with VR full-parallax holography is image brightness. This is usually related to the preferred use of Silver Halide emulsions in the hologram printing industry. Sculptural VR drawings can be particularly susceptible to this brightness problem if there is too much white (or coloured) background and too few black drawing lines.

A way around the brightness issue is to use the technique of drawing with a white pen and a black background. This effectively routes the light energy available from the hologram into the drawing lines. Because the drawing lines usually occupy a much smaller solid angle in total than the background, the relative brightness of the lines can be much higher. Another effect of this technique is that when the hologram is switched on, the only thing which changes is the image lines appearing in front of a black background. This is in contrast to the case of drawing with a ‘black’ pen. In this case, when the hologram is switched on, the background changes from black to white. A further advantage of this technique is that hogelization should be less perceptible through the bright pen lines when compared to their perceptibility through a white background. ‘White’ is also the hardest colour to achieve in holography and large areas of white tend to often show up discoloration. Discoloration should be minimized by both the geometry and brightness of the lines.

7 Conclusions

The primary conclusion of this paper is that digital full-parallax full-colour holography provides an appropriate and useful means of visualizing 3D art created in VR. Whilst digital holography is not without its problems, it is currently the only medium that affords simultaneous group viewing of 3D virtual works without equipment such as glasses or head-mounted devices. This is an important consideration in the context of an art gallery. The artist should therefore be encouraged to consider its flaws as simply an inherent constraint of the medium and understand how to work with the medium and how to use it to its best effect. In this regard, VR holography is no different from any other medium. Perhaps the principal difference is the highly technical nature of holography and the burden that this places on the artist.

The next step is to present these VRH artworks in the context of an art gallery. Here they could be brought to the attention of a wider audience and also art experts who could assess their relevance within the landscape of contemporary art. Hopefully, the dissemination of VRH artworks would then result in more art practitioners deciding to work with it and finding it stimulating, useful and inspiring.