Keywords

1 Introduction

Digital methodologies can be used to complement more traditional approaches to art history. Yet, mainly due to the difficulties of analyzing visual material with computer-assisted methods, digital art history and visual analysis are areas that have arguably developed slower than other branches of digital humanities (see Drucker 2013, 5; Klinke 2016, 16; Lozano 2017, 2). With regard to training in digital methods, the efforts in the humanities are rather scattered and digital training in art history is especially lacking (Zorich 2013, 16). However, the field of digital humanities could benefit immensely from the traditionally strong expertise of art historians in curating and creating cultural data (Schelbert 2017, 5). Indeed, digital art history is not quite as new a field as is often thought. Some even argue that art history is not behind other disciplines in the development of the digital (Zweig 2015, 40–41). Nonetheless, despite the recent advances in image recognition, digital methods have not been as widely adopted in the analysis of the visual as in some other fields of humanities, making it easy to overlook the efforts that do exist.

Though Soviet and Russian visual materials have been studied quite extensively, they have not been analyzed much within the framework of digital art history. This is partly because many of the widely used digital art repositories contain only Western European and Renaissance art, leaving other art in a marginalized role (see, e.g., Münster et al. 2018, 380). However, scholars are now debating the advantages of creating and making accessible digitized Russian visual material in collections and archives in- and outside of Russia (see Kizhner et al. 2018; Bridgers and Blood 2010; Kain 2018). For example, Biryukova et al. (2017) discuss how virtual cultural storages and virtual museums can be used to preserve the Russian cultural heritage. Other researchers have analyzed the possibilities and problems associated with making 3D models of cultural heritage objects and Russian architectural monuments, such as churches and monasteries, and presenting them online (see e.g. Borodkin et al. 2015; Zherebyatiev and Ionova 2014). Indeed, as Biryukova et al. (2017, 157) show, many of Russia’s most popular virtual museums contain churches and monasteries reconstructed in a virtual space. Some researchers, like Anna Sanina (2019), Olha Korniienko (2014), and S. Polovinets and E. Baranova (2018), have even applied digital methods to the study of Russian and Soviet satirical visual material. However, the majority of research using digital methods is based on content analyses, which in turn have relied on the coding of the images by the researcher or research assistants. To the best of my knowledge, no machine learning or computer vision methods have been used in visual studies of Russian and Soviet material.

In this chapter, I chalk out some options and possibilities to expand and apply new digital research methods and visual analyses, in order to complement the more traditional approaches to Russian and Soviet art history. As an example of a field of art historical research that may benefit from such digital methods, I use my own research on Soviet political cartoons published during the “Great Patriotic War,” as the years of war between the Soviet Union and Nazi Germany, 1941–1945, of the Second World War are known in Russian. During these years, the official party newspaper Pravda published 185 political cartoons, bearing nine different artists’ signatures. In the course of my research, I manually collected these political cartoons and assembled an Excel spreadsheet with detailed annotations that essentially functioned as a database and allowed me to conduct a qualitative analysis on them.

To interpret a political cartoon, it is necessary to understand the contextual, textual, and visual features and information contained in them. This requires the researcher to have a certain amount of background knowledge. I employ Ernst Gombrich’s (2002, 142–154) ideas, according to which a communicating image consists of three components—context (the environment within which the cartoon exists), caption (the verbal elements of the image), and code (the visual language the artist uses). This chapter thus ultimately discusses how digital methods could facilitate a Gombrichian analysis of a communicative image, such as a Soviet political cartoon.

Before getting into the use of digital methods to enhance the research of visual material, it is first necessary to give a brief overview of the situation in Russia regarding the digitization of material, copyright laws, and open access. Next, I look at recent developments in digital methods for art history and their potential application to Russian and Soviet art, especially with regard to visualizing data and the use of machine-learning algorithms to help analyze databases of relevant textual and visual material. One of the benefits of using such algorithms is that they could facilitate piecing together the cultural context for an art historical research project. However, as I discuss in the following section, the vast amount of cultural knowledge that is required for a machine to properly understand the representations within an image is where machine-learning algorithms reach their limits. As is laid out in the final section, these limits could be overcome by combining the new digital methods with traditional arthistorical research methods. Ideally, larger research projects, featuring both Information Technologies (IT) professionals and trained art historians, would enable us to create more useful art historical databases that would allow for a more effective use of the new digital methods while also combining the strengths of both digital and traditional arthistorical research.

2 Digitization, Copyright Laws, and Open Access

It is a common trend that archival material and cultural artifacts are being digitized at an increasing rate. However, the level of digitization is not universal, and its organizational forms differ. Certain cultures, mainly Western European and North American ones, are making bigger investments in their digitization projects and are thus overrepresented, while others remain in a marginalized position (Rodríguez Ortega 2013, 131). Some countries, like Russia, have government involvement driving the digitization, while in others it remains the task of individual organizations.

It is often difficult for art historians to find relevant, available, open-access, and good quality visual material in digital repositories (Münster et al. 2018, 369–371; ibid., 380). And with digital archives that are not exclusively devoted to Russian data, it is occasionally difficult to search for Russian material, because not all archives attach keywords such as “Russia” or “Slavic” to their documents and objects (cf. Bridgers and Blood 2010, 78; for more, see Chap. 20). These problems with accessing digital databases often lead to researchers creating their own personal collections (Münster et al. 2018, 371). Thus, Russian art history remains very much a question of the researcher knowing where to look for accessible and relevant material, and in many cases still traveling to the destination to retrieve it.

Online resources of Russian digitized art are rather limited. However, some resources do exist. For instance, some Russian art museums have now made parts of their collections available online, and some have even created virtual tours of their museums (see, e.g., Virtual Visit, the State Hermitage Museum). A number of museums and galleries, including the State Russian Museum, are also collaborating with the Google Arts and Culture project to digitize and put parts of their collections online (see Virtual Russian Museum). In addition to such classic art resources, there are also newspaper, journal, and photography repositories that may be of interest to art historians. For instance, a collaboration of the Russian search engine Yandex with museums and private collectors has resulted in a large online photo archive (see Istoriâ Rossii v fotografiâh).

The National Library of Finland has also recently subscribed to East View’s digital collections (http://www.eastview.com), sidelining their microfilm collections. However, the East View interface offers only a text-based search option, which makes looking for images in the newspaper difficult. Furthermore, compared to the library’s microfilms, the digital archive’s image quality is worse and some issues of Pravda that were available on microfilm are missing from the digital archive. Nonetheless, these digital copies of Pravda provide an easier option for studying the textual environment—the Gombrichian caption—within which the image exists. But while digital repositories like East View make accessing digitized material easier for those who have online access, they do not provide the services for free (for more, see Chap. 20). They offer researchers material that would otherwise require them travelling to archives to retrieve, but they do not make the information openly available to everyone. Furthermore, the digitization of textual sources is generally much more common than that of, for example, art objects.

An ongoing Ministry of Culture led program aims to have all the Russian Federation Museum Collections cataloged, digitized, and available online at https://goskatalog.ru by 2026 (see Gosudarstvennyj katalog Muzejnogo fonda Rossijskoj Federacii). That is, it aims to make available metadata and pictures of all the items in the public museums. The participation of private museums in this project is voluntary (Kizhner et al. 2018, 351–352). In 2018—eight years before the project was supposed to be completed—only 14% of the objects were digitized and 9%, that is 7,034,904 objects, were included in the database (ibid., 352–354). By May 2019, the number of objects in the online catalogue was 11,017,513, which means that the digitization and cataloging process advanced by about four million objects within the past year or so.

This digitization project by the Russian Ministry of Culture, however, does not currently grant complete open online access to the cultural heritage objects. For example, in St. Petersburg the number of digitized items is higher than on average in Russia, and even higher than digitization on average in European museums, but the number of items available online is lower than the average in Russia (Kizhner et al. 2018, 356–358). Furthermore, the quality of the photographs is not necessarily an aspect to which much attention has been paid. This becomes evident when scrolling through the various images of the catalog. A more thorough “digital museumification,” that is, a proper transformation of the object into digital form with full metadata, would be needed to make the objects in the catalog more useable to the researcher (cf. Biryukova et al. 2017, 153). This could be achieved by using crawlers or appropriate scripts. If the Russian Ministry of Culture’s project’s aim is achieved by 2026, and if the Russian policies allow for open licenses on cultural heritage objects, this would provide substantially easier access to the Russian cultural heritage to a wider audience, including international researchers (Kizhner et al. 2018, 363). It remains to be seen where the digitization project will lead and what it will in the end provide to the researchers of Russia.

Copyright laws largely limit what digitized material remains closed and what becomes public, which influences the research and other projects connected to cultural objects (Arditi 2018, 54; Roued-Cunliffe 2018, 288). Internationally, copyright laws largely accept the so-called fair use policy of images, which means that they can be used without obtaining permission from the copyright holder in certain cases, such as in research papers where the image is the direct subject of the analysis rather than merely an illustration. However, the Russian laws are more restrictive. Here, the state legislation supports the so-called “permissions culture,” which works counter to the fair use policy. Accordingly, museums can retain the rights to all their material even in a case that would be regarded as fair use in a research publication. In practice, this varies from museum to museum and the researcher needs to figure out the museum’s practices. For instance, the State Hermitage allows their material to be used, for example, for educational purposes, in conference presentations, and in PhD theses. But permission is required to use images for commercial purposes or in research publications, or to publish conference slides online (Kizhner et al. 2018, 359). The fact that visual material is by nature copyright heavy, when compared, for example, to text sources, hinders the work of individual researchers as well as the building of digital repositories that would benefit the field more broadly.

The complexities of the copyright laws and the “permission culture” that prevail in Russia make it unfeasible for an individual researcher to make their personal databases of primary material open to other researchers. While nothing prevents me from publishing my metadata, the fear of litigation or of being denied further access to the archival material has kept me from making my collection of Pravda political cartoons accessible to the public. Instead, I have the database stored on personal devices and the images saved in accessible formats, such as PNG and TIFF. Indeed, when thinking about the storage of data, it is necessary to consider whether the data can be made open and who could benefit from it. For data to be openly accessible, it is necessary to use file formats that are possible to use with a variety of non-commercial programs and are likely to stay in use for a long period of time (Roued-Cunliffe 2018, 292). Such formats include, but are not limited to, JSON, XML, and IIIF. With regard to Russian images, a more widespread use of the annotation ready IIIF format by the heritage institutions and in the Russian museum catalogs would provide researchers better access and more possibilities to present the images with stable Uniform Resource Identifiers (URIs).

3 New Digital Approaches to Visual Analysis and Art History

Digital methods provide new approaches to art history, such as the visualization and display of data and research results, the digitization and digital rendering of art, and most recently the use of convolutional neural networks (CNNs) for simple and even more complex recognition tasks. The increasing computational power we have at our disposal now enables rather more complex visualizations than the traditional bar chart or pie chart. For instance, one can create complex visualizations that consist of a large quantity of individual images that, when combined, provide an overall picture (Schelbert 2017, 4). New visualization techniques also allow for more dynamic “moving” charts in electronic form. The use of such visualizations in digital visual research has been criticized for its lack of accompanying interpretation (see Bishop 2017, paragraph 9). However, when approached with care, new modes of visualization can be very powerful at revealing tendencies that might otherwise be missed. Thus, with the Pravda political cartoons, one could create visualizations to exemplify their structure, their connections to historical events, their intertextuality, and other aspects, in the spirit of Gombrich’s notion of contextualizing an image. One could, for example, place the cartoons on a map of Europe, showing where each one takes place, or do a cross-referencing of countries and animal characters to see the significance of animal symbolism in the cartoons.

In addition to such visualizations of data, digital methods also offer other options for representing research findings. Digitized art and the digital rendering of art artifacts enable the researchers to bring the art to a wider audience. For example, the University of Nottingham’s project Windows on War: Soviet Posters 1943–1945 (see Windows on War), which was conducted by a multidisciplinary team, allows the visitor to look at the images while at the same time reading about culturally specific information and the historical contextualization of the images. In a sense, this makes the communication of art history independent of both location and time, allowing people to immediately access art from around the world and even to view a virtual restructuration of an already destroyed artwork, such as an old building (Kellaway 2013, 95–96; Borodkin et al. 2015, 5–7). Furthermore, contemporary digital online spaces offer us the possibility to reconstruct old exhibitions, of which we have photographic evidence, such as the “godless corners” of the early twentieth-century Soviet Union (Kain 2018, 219). Thus, the use of digital methods is not limited to the actual process of conducting the research or disseminating the results within the academic environment; they can be employed in researchers’ popular outreach efforts as well.

One of the difficulties of digital humanities is to turn the primary material into useful data: to quantify a body of material that is not traditionally handled in such a way and to combine this quantification with humanities methods and theories of enquiry (Otty and Thomson 2016, 135). Manually annotating images is perhaps still the most common way of approaching the problem of turning an image into a format that is possible to analyze with the use of a computer. Here, researchers annotate the images with appropriate keywords, that is, metadata, which are then used as a basis to build a database and conduct a computer-assisted analysis to find underlying tendencies of the material (see, e.g., Korniienko 2014; Sanina 2019). I followed this procedure when I made my database of the 185 Pravda political cartoons. My metadata included, when relevant, information on the cartoon’s date of publication, page, position on the page, artist, title, captions, text in image, quote, poet, poem, characters, countries represented, symbols, combinations of a symbol and a country/person, combinations of attributes and a person/country, and the roles of the characters. This allowed me to analyze the cartoons on the basis of the assigned keywords and to make cross-references between them. Such studies rely on the human to do the coding, instead of employing machine vision, which to this date is not yet at a level where most researchers would completely rely on it or know how to use it for the principal annotation of the primary visual material.

The possibility that a machine could take over such basic art historical analyses would help immensely with metadata extraction and other rather mechanical work. The extraction of this metadata, in turn, would directly facilitate the analysis of Gombrich’s caption and code—text, quotes, and title being part of the caption and characters, symbols, and attributes part of the code. Naturally, using machines to do this would enable researchers to process much larger datasets. And furthermore, a machine would assign keywords more consistently than a group of coders, who are each assigning keywords based on their varying interpretations (cf. Rose 2007, 60–61; Bell 2001, 22). By combining this computer analysis of a vast body of imagery with an art historian’s analysis of specific images from that same body, one could also create a two-sided database. The first side would comprise simple computer-assigned characteristics of large amounts of images, while the second would consist of the art historian’s keywords and would address the more semantic notions of the image (Dressen 2017, 8). This would allow the researcher to conduct a qualitative analysis with specific images as examples, while the bulk of the images serve as a contextualizing device.

The computer vision technology that would facilitate such analyses is in a process of constant development. For some time now, computers have been able to reliably detect the colors and textures of an artwork, which does not help us to make any semantic interpretations but does facilitate more precise quantitative analyses of colors and shadings used by various artists as well as to identify artworks and attribute them to artists (Manovich 2015, 22; Schelbert 2017, 5). According to Emily L. Spratt, the image analysis capabilities of machines is now approaching the second of Erwin Panofsky’s three levels of art historical analysis. That is, they can not only identify basic elements within the image, such as animals or people, but also detect conventional cultural representations, such as religious motifs (Spratt 2017, 12). In the case of the Union of Soviet Socialist Republics (USSR), these could include ideological motifs, such as depictions of revolutionary events, or certain types of characters, such as the archetypal worker or peasant. Applying such a tried and true art historical theory as Panofsky’s to these new developments is, of course, not straightforward. But the fact that the capabilities of computer image analysis are now being directly compared to the image analysis skills of people is, on its own, very telling.

At present, a number of research groups are working to push the limits of what computer vision can do with comics (cf. Laubrock and Dubray 2019; Laubrock and Dunst 2019; Young-Min 2019). Any such research is, of course, heavily dependent on the availability of appropriate training sets. For instance, the Digital Comic Museum (DCM) hosts a set of nearly 200,000 pages of American comics published before 1959, segmented into panels and text bubbles by machine, and transcribed using optical character recognition (OCR), which can be downloaded at https://github.com/miyyer/comics (see Digital Comic Museum). Due to the imperfections of having the segmentation done by a machine, Nguyen et al. (2018) have also produced a subset of 772 pages from the DCM that has been fully annotated by humans. With the help of this training set, among others, researchers have achieved good results in identifying various elements of a comic, such as speech bubbles, panels, and captions. And they are now moving on to more advanced recognition tasks, such as getting machines to recognize recurrent characters, image-text relations, and simple narrative structures (Laubrock and Dunst 2019, 11–20; ibid. 28). It is conceivable, that similar datasets could be created of recent Russian comics. But unfortunately, many other areas of art history do not yet benefit from such vast and high-quality datasets, and as discussed in the following section, Russian art history is no exception to this rule.

4 The Current Limits of Machine Learning

Perhaps the biggest challenge of using machine learning for analyzing visual imagery is that it requires very large datasets to train the algorithms. For large corpora of visual material with repetitive elements, such as medieval manuscripts, it has already proven to be especially feasible to use computer vision to annotate the images, saving the researchers countless working hours (Bell et al. 2013, 27). As with the medieval manuscripts, if it were possible to construct a sufficiently large training set, it now seems entirely possible that a machine could be trained to help analyze political cartoons. For instance, a machine could learn to identify certain characters by distinguishing the exaggerated physical attributes that make a caricature look like its target, such as the moustache and hair of Hitler or the big mouth and short stature of Goebbels. However, care would have to be taken to include enough features so that the computer would not, for example, mistake Chaplin for Hitler on the basis of his moustache. Furthermore, conventional facial recognition currently works by identifying the dimensions of the face. So, for this to work, either the facial recognition software would have to be expanded or a separate recognition algorithm would have to be developed to identify exaggerated and satirized physical features (for more on machine learning, see Chap. 26).

The question in this specific case is whether the Soviet political cartoons, or visual propaganda more generally, are repetitive enough that training computers to do the annotation would be feasible. While features like Hitler’s moustache certainly repeat, each cartoonist has their own style and the themes and topics vary. The only way to find out for sure would be to gather a sufficiently large dataset and try it out. Saurav Jha et al. (2018) point out that the training sets that are currently available are too small to train a neural network to recognize cartoon characters. Hence, the training sets need to be supplemented with the inclusion of photographs of the people who appear in the cartoons. However, the inclusion of large amounts of photographic material decreases the feature recognition of the cartoons. Additionally, the more exaggerated the features of the character, the more the machine has trouble identifying the face. Going even further, one wonders whether a machine could learn to detect satire and ridicule. Or make the connection and find the similarities and differences between a caricatured and a photographed Hitler. If a computer could effectively learn to examine the Gombrichian code of an image, it would enable the faster analysis of large visual corpora of propaganda imagery and, possibly, provide us with a more complete picture of the ways in which visual propaganda functions.

In addition to the size of the training set, the quality of its images and their similarity to the material that is to be investigated are also crucial, lest the neural network end up interpreting images in different ways than the trainer’s intention. As the training set of convolutional neural networks influences the way the network interprets other images shown to it, the training set needs to be carefully planned so that it will not cause erratic results (Spratt 2017, 4). When an image is different from the images of the training set, the machine may end up facing difficulties. For example, in one image-to-image translation project, the machine was taught to transform images so that a winter scenery became a summer scenery, a photograph into a painting by a famous artist, or a horse into a zebra. However, the training set did not contain images of horses, which resulted in the machine transforming not only the horse’s coat but also the skin of the bare-chested President of Russia riding the horse into a zebra-patterned being (Zhu et al. 2017). This exemplifies how the computer interprets visual material only on the basis of the training set that has been used. Thus, the machine does not have the contextual information and interpretation capabilities that a human in a similar situation would have.

Images have a high level of cultural coding. So even if a computer can extract large amounts of data from an image, it cannot understand the semantic side of an image as well as a human does. Current developments in computer vision aim to bridge this “semantic gap,” to allow a machine to detect basic semantic meanings based on the information it can obtain from an image (Manovich 2015, 22). However, in the same way that computer vision needs to be trained to recognize images, humans have been trained by culture and society to recognize and interpret them correctly (Spratt 2017, 7). In other words, for a machine to be able to correctly analyze the significant elements of an image, it essentially needs the same training and cultural knowledge that a human has. The almost incomprehensibly vast amount of information that forms this cultural context of an image is where machine learning and computer vision reach their limits and where the guidance and supervision by trained art historians will for the foreseeable future remain essential for any research project.

Our interpretations are always dependent on our spatial, temporal, and cultural contexts. Any interpretation by an art historian—or anyone else for that matter—is dependent on their background (Gaehtgens 2013, 23–24). For example, given an image of a miserable tiger, a contemporary of the wartime political cartoons in the Soviet Union would have understood its significance as a play on the German heavy tank Tiger getting stuck in the muddy spring of the Eastern Front, as would someone familiar with the fate of Tiger tanks. However, without the contextual knowledge, the symbolism of the animal could end up signifying something else, such as the characteristics of the Germans as defeated wild animals. That is, the interpretation I make might differ largely from the interpretation someone else makes—can a computer make such semantic interpretations?

In the same way that the interpretation of data is dependent on the background of the researcher, the way that the data are organized depends on the interpretations of the researcher. Thus, the way I organize data might differ largely from how someone else does it (see Otty and Thomson 2016, 115). In other words, when making interpretations or organizing data, one needs to remember one’s own contextual situation and not blindly trust digital methods and believe that they will provide completely replicable and authoritative results (for more, see Chap. 21). And until machine-learning algorithms can be trained to take into account a reasonable proportion of the cultural context of its target material, we must bear in mind that any interpretations made by such algorithms will be based on a considerably narrower background than that of any human researcher.

5 How Humans and Machines Can Work Together

The advantages of the new digital methods and of traditional arthistorical research are conveniently complementary. Indeed, by combining the strengths of a trained researcher with the capabilities of machine-learning algorithms, it should be possible to cancel out any limitations of either. The digitization of visual imagery enables researchers to conduct contextual analyses of images that would not be feasible without access to digital resources. Thus, it facilitates an even wider contextual analysis than Gombrich (2002, 142–154) could have had in mind when writing about the context of the image. As has been discussed, developments in the digital methods are currently on the cusp of making this possible. Well-designed databases with easy accessibility and properly annotated images would help researchers to examine the intertextuality and connections between different works of art and other cultural, social, and historical phenomena (Brandhorst 2013, 72–73). For instance, it would help the researchers if computers could do a search and comparison within a database for art works that are similar to the one that they are examining. Of course, for this to be practical and feasible, it will first be necessary for the machines to be able to reliably identify and catalog certain elements within the visual artifacts. In this way, a computer could go through large corpora and their metadata considerably faster than a human (Klinke 2016, 16). Furthermore, the comparison of many images with each other when it comes to composition and aesthetics could provide new insights into how different artists composed their images (Pfisterer 2018, 138). After all, it is impossible to compare as many images in person as it would be with a computer.

The high level of intertextuality of political cartoons would also become more evident with such comparative computational methods. Their connections to various areas of culture, including the Soviet visual propaganda imagery, which had the tendency to repeat and borrow ideas from previous images, illustrate propaganda’s dependency on the cultural context within which it operates (see Kangas 2017, 46–47). For a contextual analysis of the political cartoons, the pages on which they were published—or even whole issues of Pravda—could be processed with OCR for a cross-referencing of the news text with cartoon. The comparison of the text surrounding the image with the actual image could provide additional contextualization, complementing the researcher’s efforts to place the image within the context of the war events. The computer could also assign a value to the similarity between specific features of the cartoons and other images and cultural artifacts, war events, or their geographical location. These values could then be mapped onto a graph in which all the variables would be presented together in a dynamic visualization. However, due to the complexity and the wide variety of cultural representations, this is currently still beyond the capabilities of machine-learning.

More generally, with the help of computer programs that could search for such open access information—this requires open access as well—many projects could benefit from the information as a part of the contextualization of art (Dressen 2017, 4–5). And the emergence of large text databases of art historical material will enable a more thorough and accessible contextual analysis of the visual, which has traditionally been slow and cumbersome due to the large amount of background material needed (Drucker 2013, 10). Thus, with digital methods, it is possible for human researchers to take into account ever larger amounts of background information when conducting their research.

With regard to training a machine-learning algorithm to detect conventional cultural representations and thus establish connections to the broader cultural context, here too it is a question of having sufficient information available. Thus, apart from computers with the necessary processing power to do the analysis, the digitization and accessibility of cultural artifacts is essential for a full analysis that takes into account both the cultural context and the formal properties of the object (Schelbert 2017, 5). But is it possible to train a machine to see what is not presented? Indeed, in images, and texts too, what has been left out conveys semantic information. That is, in an image what you cannot see is often as important—or even more important—than what you can see (Rose 2007, 72). Even if it is possible to train a machine-learning algorithm to “see” what is missing in an image, it is difficult for it to do an analysis of the meaning of the omissions. For example, in Soviet political cartoons, the Soviet Union or its allies are rarely shown. However, their omission does not mean that their presence is not implied. Once again, here, the interpretative skills and supervision by a trained art historian is necessary to fully understand what is going on.

The increasing use of digital methods in the study of the visual does not necessarily mean the overthrow of art history’s more traditional methodologies. In combining digital and traditional quantitative methods, the researcher can draw a range of conclusions from their datasets that would be difficult to manage without the machine’s computational power. But at the same time, through qualitative analyses, the researcher can make interpretations and evaluations that a computer cannot (see Klinke 2016, 28; Lozano 2017, 6; Rose 2007, 70–71). To employ such a wide methodological oeuvre calls for interdisciplinarity and/or collaboration between specialists from varying backgrounds. There have been several calls for such collaborations (e.g., Glinka et al. 2016, 209; Klinke 2016, 31; Mercuriali 2018, 149). Having teams that employ people with expertise from different fields and with different skill sets would further the goal of creating large, accessible databases, as well as the planning of new complex methods of analysis.

6 Conclusion

In this chapter I looked at some of the advantages and challenges that the digital study of visual material will encounter. My starting point was to consider these issues in the light of a previous research project which was conducted mainly with more “traditional” methods, such as archival work on microfilms, digitization of material, and conducting a “manual” qualitative analysis of the primary material. I used my earlier analysis of Soviet visual data as an example and discussed the possibilities of digital methods that I could have used in the project.

In some ways, researchers of Russian and Soviet visual material face many of the same challenges that any other academics face when using machine-learning methods to enhance their research. For instance, it is important to train the machine to be “intelligent” and to learn to “see” appropriately and ethically—we would not want the machine to learn to tamper with the data to make the researcher happy. Additionally, the training process is still too slow and complex to be used in a small-scale research project, but the development of machine and deep learning might change the situation and make these methods more approachable for a wider base of researchers.

In other ways, researchers of Russian art history face their own unique set of challenges in adopting the new digital methods. For instance, the problem of the “semantic gap,” that is, that computers are not able to handle the semantic side of the objects they are analyzing, is especially pertinent when analyzing visual imagery, which is heavily reliant on a large amount of contextualizing information. And the collection of such contextualizing information in digital databases, so as to make it useful for machine-learning algorithms, is further confounded in Russia by their restrictive copyright laws and permission culture.

These restrictive copyright laws are especially detrimental, as the lack of openly accessible, large-scale databases of visual material is the primary bottleneck preventing the use of new digital methods for conducting art historical research in Russia. As a result, these digital methods have not yet found a secure foothold within Russian visual studies. Some research has been conducted, but it has mainly relied on rather traditional computational methods, such as content analysis. Considering the breadth of visual material that Russia—which is generally considered to be a very visual culture—and the Soviet Union have produced, it would be extremely advantageous to employ some of the more recent digital methodologies to that material.

Nonetheless, larger projects featuring interdisciplinary teams and collaborations within art history and other fields that employ digital methods for visual analysis could yield considerable results. In many cases, a digital project would benefit from the participation of people with varying backgrounds and skills, such as IT, quantitative, and qualitative methods. Through the co-operation of people with all of these different skill sets, it would be possible to employ the digital methods more fully and find new creative solutions to, for instance, create suitable databases that better serve the researchers or more dynamic visualizations of the research results. So, despite the challenges some of them still face, the new digital methods provide many new possibilities for the study of the visual, facilitating an easier examination of the images’ context, caption, and code, in the spirit of Ernst Gombrich.