Towards Embodied Paradata. A Diffractive Art/Archaeology Approach

Abstract

For archaeologists, artists, and cultural heritage workers, paradata are generally viewed as explicitly selected and documented attributes, or defined sets of circumstances, authoritatively considered to have a material outcome on the provenance, collection, and manipulation of both recorded data and metadata and their subsequent interpretation or analysis of artefacts and other (contextual) remains. Being chosen, their own provenance is questionable: why were the selected data, metadata, and paradata more relevant than other options? We (re)consider embodied practice as a form of paradata-making normally airbrushed out of the hegemonic accounts of how works of art and archaeological excavations are presented and analysed. Decisions to not include the embodied worker, their apparatus, and their practices of making, or uncovering, haunt images purporting to be historical accounts in the art and archaeology literature by their absence. Adopting a diffractive art/archaeology approach, and subversively applying several well-known cultural heritage recording and presentation techniques, recursively and unconventionally, we throw light on embodied paradata and (re)present them as potentially very valuable pedagogical boundary objects. We also dislocate paradata away from a purely epistemological dimension into an entangled onto-epistemological nexus.

1 Introduction and Background

Artefacts and artworks are widely supposed to stand up and speak for themselves. Opinions appear to be ancillary. We believe this is misleading. Artefacts and assemblages are deliberately articulated in ways to present specific points of view. Transdisciplinarity or cross-boundary knowledge sharing—that is reading through one another’s disciplinary filters—offers the promise of revealing informative diffraction patterns in which these invisible decision-making processes, choices, and opinions become visible and where productive knowledge accidents might appear. Promising, but no one said it would be easy. Transdisciplinary knowledge sharing, management, and communication are without doubt challenging. The accounts we offer for the decisions we make, in other words the paradata we feature, in practice-based disciplines such as field archaeology and fine art, where much knowledge is tacit and unspoken, are not necessarily readily apparent, especially for newcomers or students to the field. Even where we have developed explicit disciplinary conceptual frameworks, miscommunication across disciplinary or domain borders is a real possibility. For instance, the words provenance, provenience, paradata, context, and assemblage are examples of shared terminology for a sophisticated group of related concepts that, allegedly, account for the circumstances of discovery, interpretation, and subsequent life history of an artefact or assemblage and have subtle but consequential differences in meaning in the disciplines of art, archaeology, and cultural heritage more generally (Huvila & Sköld, 2021, chapter ‘A Leap of Faith: Revisiting Paradata in 3D Scholarship’ in this volume; Reilly et al., 2021; Sköld et al., 2022). In this chapter, we focus on embodied paradata.

Our approach to embodied decision-making is from an Art /Archaeology perspective, which is not a simple combination of art in archaeology or archaeology in art. We adopt a far more disruptive and unsettling emerging discipline which constitutes a kind of diffraction zone where embodied, and material, archaeological, and artist perspectives and practices interlace one another (Bailey, 2017, 2018). The resulting interference patterns that emerge in this transdisciplinary diffraction zone show important, perhaps subversive, but revealing aspects of difference.

Both authors were trained in distinct practice-based disciplines. Our knowledge continues to build through embodied experiences of interacting with materials in trenches and studios, where embodied practices intersect and mingle in the realms of (im)materiality, temporality, movement, gesture, and mark-making. Dawson is a sculptor. Reilly is an archaeologist. We are also collaborators entangled with digital technology, particularly technologies of imaging and 3D printing. We explore diffractively the affordances of the digital by applying our originally separate archaeology and art practices through one another’s fields. Beyond this, we have experimented with some novel, combined, and metabolic art/archaeology practices. We employ the word metabolic to suggest that the way these practices combine is more than a simple intersection of practices. Metabolism involves the transfer or conversion of material, quite literally a stuff exchange. The conversion of matter, or Stoffwechsel, was identified as early as the 1800s by Gottfried Semper (1803–1879) in his architectural writing:

When an artistic motif undergoes any kind of material treatment, its original type will be modified; it will receive, so to speak, a specific colouring. The type is no longer in its primary stage of development but has undergone a more or less pronounced metamorphosis. If the motif undergoes a new change of material as a result of this secondary or even multiple transformation, the resulting new form will be a composite, one that expresses the primeval type and all the stages preceding the latest form. (Semper, 2004, p.250)

Thinking about the metabolism of vibrant matter (Bennett, 2010) in this way makes intangible things and tacit knowledge perceptible. Haraway also thinks through ideas about the organic transfer of matter when describing the ethics of collaboration as a form of composting. She argues that ‘staying with the trouble requires odd-kin; that is, we require each other in unexpected collaborations and combinations, in hot compost piles. We become-with each other or not at all.’ (Haraway, 2016, p.4)

One phenomenon that we have been exploring is the disturbing way in which archaeological and artistic artefacts metamorphose into unique stand-alone objects through that fallacious space described by Agamben (2019, p.IV) as ‘the mythical fixity of images[s]’, where artefact and art works are presented (de)contextualised within illusionary timeless, motionless, and apparently empty spaces. In these uncanny spaces, traces of both the object makers, their processes, and embodied practices of revealing and recording the object and the image makers—in short crucial paradata—are deliberately airbrushed away, masked, or cropped out. Such conventions have been common practice in archaeology since photography was first introduced into the discipline more than a century ago (e.g. Baird, 2019; Knight & McFadyen, 2019; McFadyen & Hicks, 2019; Thomas, 2019; Witmore, 2007). Derrida (1994) argued that the act of selection forecloses other alternative futures, but paradoxically these lost potential futures can haunt current and historical discourse and have real effects. The decision to not include the embodied worker and their deep skilful practices haunts countless images with their absence in the art and archaeology literature. Besides being ethically questionable, these absences create pedagogical chasms which deny students fuller, multimodal apprehension of the embodied practices required to produce the incomplete assemblage, as would-be students are deprived of much valuable tacit, or implicit, knowledge about how tasks of recording can themselves be usefully (re)presented. We argue that these diffractive images surface embodied knowledge pertaining to how practitioners perceive and communicate kinaesthetically. They emerge as potential pedagogic boundary objects that bring non-conscious and tacit knowledge into the explicit domain, hopefully to provide further pedagogical articulations to a broader set of learners/sharers for whom written or spoken accounts do not convey the full intent of the embodied practitioner (e.g. Derudas & Berggren, 2021). We want to make perceptible the image makers and some of the constraints and choices that affect the decisions made when (re)presenting artefacts or assemblages in images. In other words, we want to expose practitioner paradata that are normally excluded from, or deliberately disguised in, images. To this end, we adopt a diffractive transdisciplinary approach that also interlaces and entangles several standard cultural heritage recording methods to reveal some of these ‘silent processes’ of documentation (Huggett, 2020), the ‘fingerprints’ of researchers (Jones & Bunn, chapter ‘Mapping Accessions to Repositories Data: A Case Study in Paradata’ in this volume), and the subtly different ‘paradata dustings’ that different art and heritage practitioners produce (Buchanan & Huntsman, chapter ‘Dustings of Paradata as Pedagogical Support at Four Archaeological Field-School Sites’ in this volume).

Taken literally, diffraction describes the interference of light waves when they encounter an obstruction, and much has been talked of about the patterns that this interference creates by physicists and philosophers. Haraway introduced the concept of diffraction to contest the paradigm of reflexivity which ‘mirrors the geometrical optics of reflection’ (Barad, 2007, p.72). If the two-way reflective approach nurtures sameness, diffraction is remarkable for revealing the patterns of difference. Haraway harnessed this diffractive metaphor to discuss how better to account for the effects of the researcher on the experiment. Following Haraway, Barad, a feminist physicist, developed diffraction to much more than a metaphor. She argues that diffraction, actually and not just metaphorically, causes patterns that make a difference. ‘Diffraction not only brings the reality of entanglements to light, it is itself an entangled phenomenon’ (ibid., p.73).

Here, we apply Art/Archaeology diffraction filters to the concept of paradata and to their further layers of recursive introspection, supplements that we call peridata. We intend to reveal hidden paradata and peridata haunting the pixels of our so-called diffractive images (Dawson et al., 2022). After describing the object of study, we will define in more detail what we mean and intend by the terms ‘Art/Archaeology approach’, ‘paradata’, and ‘peridata’. Then we will demonstrate how these para/peridata come about, non-consciously as practitioners’ ecologies of attention constantly shift within a dynamically adjustable cognitive assemblage consisting of the artefact under investigation, the instruments of analysis, the changing position and settings of both instruments and practitioners, and the constant visual recalibrations of our knowledge that are intended to help ‘share’ our knowledge about the artefact. We will proceed, adopting a diffractive art/archaeology approach, by subversively applying several well-known cultural heritage recording and presentation techniques, recursively and unconventionally, to throw fresh light on embodied paradata and (re)present them as potentially very valuable pedagogical boundary objects. Rather than allowing artefacts and their paradata to subsist in inert, isolated, and sanitised vacuums, we will attempt to dislocate paradata away from a purely epistemological dimension and reposition them in an energetic, entangled, onto-epistemological nexus.

2 Introducing the Nessglyph

Our project centres on a remarkable carving, made on a block of red sandstone, found during the 2021 excavations of an Iron Age hillfort called Nesscliffe Hill Camp, Shropshire, UK (Hume & Jones, 1959; Lock & Reilly, 2019, 2020, 2021). This pluritemporal petroglyphic assemblage known as the Nessglyph is a unique carved sub-triangular block of red sandstone (400 mm × 390 mm × 250 mm at its widest points). The Nessglyph is composite. Nevertheless, it feels as if it was composed, and it is certainly made by a sequence of overlapping marks, and therein lies its remarkability—an image in the making (Fig. 1).

Fig. 1

The Nesscliffe Petroglyph, dubbed the Nessglyph (frame from an interactive RTI)

The Nessglyph consists of a ‘cup mark’ that was later inscribed by a series of linear grooves. We examine its pluritemporal development, (re)composition, and interpretative possibilities in detail elsewhere (Reilly et al., 2023; Reilly & Lock, 2023). In this art/archaeology study, we are more concerned with looking at our own approach, our different ways of looking and knowing, such images in the making, and the creation of the paradata that we invest in, and attach to, the object. This image in the making has been defined by at least seven interventions affecting the biography and itinerary of the Nessglyph: initially the cup mark was ground out; then linear grooves were overscored; the stone (in the entrance) was buried; the stone was then disturbed unnoticed in a poorly controlled excavation; and once again reburied in backfill; in 2021 the stone was recovered, logged, and identified; the stone/Nessglyph is recorded using modern 3D photogrammetric and scanning devices. The last four archaeological interventions are likely to bear evidence of non-verbal paradata dustings and fingerprints. In the next section, we will begin to develop what we mean by non-verbal paradata and how it is fundamentally different from ‘traditional’ paradata. We start by laying out the case for apportioning greater weight and value to this previously overlooked source of embodied ‘investigative documentation’.

3 Towards Alternative Embodied Perspectives on Paradata

The meaning, modes, and values of paradata are not homogeneous across disciplines (Andersson, Sköld, & Huvila, chapter ‘An Introduction to Paradata’ in this volume; Börjesson et al., 2022; Buchanan & Huntsman, chapter ‘Dustings of Paradata as Pedagogical Support at Four Archaeological Field-School Sites’ in this volume; Papadopoulos, chapter ‘A Leap of Faith: Revisiting Paradata in 3D Scholarship’ in this volume). In archaeology, the latent value of the concept was captured, but not recognised as such, in the rich alternative possibilities that 1980s ‘data exploration loops’ offered (Burridge et al., 1989; Reilly, 1989) within ‘virtual archaeology’ excavation sites (Reilly, 1989, 1991, 1992), in which practitioners of field archaeology could ask ‘what might happen if we did things differently; what might we find otherwise?’ Oddly, the term ‘virtual archaeology’, and subsequently ‘virtual heritage’, became synonymous with interpretive visualisations of reconstructed buildings. A growing number of these imaginative projects were delivered by expert modellers who were not trained archaeologists nor architects. Questions began to arise about the authority of the model makers. For example, how sure were they that this or that detail of the model was securely established? (e.g. Messemer, 2016; Miller & Richards, 1995; Opgenhaffen, 2021; Wittur, 2013). Over time, the need to augment these models with paradata pertaining to the veracity, degree of confidence, and possible alternative interpretations was formalised in the London Charter and Seville Principles (Bendicho, 2013; Bentkowska-Kafel et al., 2012; Börjesson et al., 2020; Opgenhaffen et al., 2021; Papadopoulos, chapter ‘A Leap of Faith: Revisiting Paradata in 3D Scholarship’ in this volume). The original focus on alternative perspectives and approaches in archaeological field and laboratory work was renamed ‘digital creativity in archaeology’ (Beale & Reilly, 2017a, 2017b). It is to fieldwork, laboratory, and studio work, that we now turn, to begin looking at embodied modes of decision-making (i.e. paradata) for recording and presenting artefacts and assemblages.

4 Techne, Poiesis, and Empeiria: ‘Am I Doing This Right?’

Artefacts and assemblages do not exist in vacuums. They are not found or magically appear. They are produced. Removing soil and other materials by troweling, mattocking, or shoveling, exposing buried deposits, are central skills of the field archaeologist. Volunteer newcomers and novice students on training excavations frequently, and quite rightly, ask ‘am I doing this right?’ This is also a common refrain in art school studios as students navigate the entangled and shifting histories of techne (technique), poiesis (bringing something into being), and empeiria (practice without knowledge). Techne—the knowing how to make—has become a purely technical and technique-bound question associated more readily with craft and not art. Art is often more easily conveyed like empeiria as something that doesn’t depend on rules and cannot be taught but is to be absorbed instead through non-verbal learning. Problems arise however from making assumptions that technique is somehow separate from art and that it has a lesser role in the development of an artwork. There is now an opportunity to rethink the role of techne in art, to consider technique as woven again into art making and to reconsider the distinctions between techne and empeiria (Elkins, 2001, p.103).

This apparently innocuous question—‘am I doing this right?’—opens pedagogical issues across both disciplines, from the art student who is confused with how to think through making to the archaeology student when performing a seemingly simple mechanical action which is fraught with many imponderables (Pijpers, 2021). Can art students be taught? (Elkins, 2001). Similarly, how do you teach the archaeology student how to navigate through opaque deposits? Best practice in field archaeology is very relational. The soil matrix, weather, light conditions, and accessibility to the deposits all must be negotiated on a case-by-case basis. Certainly, an experienced excavator can usually determine if the novice is making a mess of matters, but it is not at all straightforward explaining how you adapt to the material circumstances immediately in front of you. Some students just seem to grasp how to explore deposits using handheld tools almost instinctively. Others struggle. How to articulate themselves and their tools does not come naturally to many students, who may then decide that fieldwork is not for them. Some, however, will persevere and gradually develop their technique and skill levels to learn to know through the tools they wield with their hands. We do not believe that troweling or mark-making more generally is necessarily an innate skill. It seems to us that the issue is really a pedagogical shortcoming and that perhaps embodied paradata could give practitioners another avenue to share their craft skills via another medium.

We are increasingly aware that movements, gestures, and rhythms of troweling, something that Tringham (2016) describes as ‘hand-ballets’ and Pijpers (2021) characterises as ‘worlding with trowels’, are fundamental to archaeological practice (see also Edgeworth, 2012; Wendrich, 2012) and can form rich sources of meaning-making and non-verbal decision-laden paradata (Gant & Reilly, 2018; Reilly et al., 2021). For instance, artist Stefan Gant’s extended drawing studies of moving trowels in the hands of archaeologists at work showed skilful ‘gamuts’ of motion, with distinctive phrasing unique to individual diggers being deployed to probe and detect buried archaeological features. Initially, looking over the diggers’ shoulder he traced their unique, deft, movements of the trowel’s blade in pencil upon a sketchpad as their excavation proceeded. Complex ‘meshworks’ of normally unarticulated decision-making emerged and became available for study from the perspective of drawing theorists. These troweled gamuts form distinctive signatures (Gant & Reilly, 2018) of the otherwise anonymous diggers (Everill, 2009; Huvila, 2017). The start and end of each inscribed transect were initially read as a form of nonconscious paradata recording ‘moments of tension’ (Ingold, 2007, p.79) and ‘moments of completion’ (ibid., 81) at the ‘trowel’s edge’ (Berggren & Hodder, 2003). However, while the overall gamut of movements can be discerned in these drawings, the sequence of marks made by both the digger and the mimetic transcriber were not always obvious. Digital video recordings were enlisted to address this issue. Gamuts of movement could now be (re)analysed frame by frame and (re)materialised as 3D extended drawings. Each trowel mark in an extended engagement with arbitrarily selected archaeological features was reverse engineered by cutting card strips to the appropriate length and orientation of the trowel’s indents and stacking them in reverse order. After applying the well-known Cultural Heritage Imaging technique known as Reflectance Transformation Imaging, or RTI (CHI, n.d.), to this 3D extended drawing of troweling marks, the morphing temporal diffraction shadows (Callery et al., 2022) we observed under several RTI filters impressed upon us that these truncated, interweaving gamuts did not constitute meshworks. Rather, these troweling gamuts might be better conceived of as sinuous ‘knots’ (Ingold, 2015) of meaningful motion in which the decision about where and how to apply the trowel occurs, non-consciously, during the excavator’s expressive world-making looping of the tool before and after the trowel’s blade scored the ground. Some of this meaning-making action is also detectable in the accompanying acoustic registers. Recording the archaeologists’ mattocking, shoveling, and troweling different materials and displaying this activity as sonography gave new ‘voice’ to these workers, and their skilful use of their tools, as the contexts and artefacts they encountered emerged. Distinctive ‘sonic stratigraphies’ could be detected in the soundtrack of these unchoreographed ‘hand ballets’ of discovery when they are presented as acoustic and visual paradata (Gant & Reilly, 2018; Reilly et al., 2021). We have exposed significant underlying patterns of expression through these initial observations, but the precise articulations and subtle gestures of the hands of the working archaeologists in these dense loops of meaning-making are still partially withdrawn from these extended drawings made in the field. Perhaps we can get more traction on subsequent gestures of meaning-making and authorship (i.e. visual, haptic, and acoustic paradata) made in the better controlled environment of the finds hut, post-excavation laboratories, and artist studios (e.g. Min et al., 2020).

Jones and Smith (2017) foregrounded the performative nature of producing Reflective Transformation Images (RTIs). Dawson and Reilly (2019) then realised that RTIs contained ‘inadvertent images’ (Geimer, 2018) recording autographic traces of the RTIs in their making. All RTIs naturally operate as ‘metapictures’ (Mitchell, 1994, 2004) embodying a self-referential quality that triggers a meta-level discursive opportunity to consider what, when, where, and how, this form of technical image operates. Painted in light on the crucial RTI sphere is a significant amount of the set, the choreography, and the prompts surrounding these RTI ‘performances’ captured in a time-lapse sequence as each image is produced.

The artefact or assemblage intended to be recorded, the DSLR device, the strobe, the reflective sphere, and the image maker, already entangled, are further entangled by residual traces of light. The reflections caught on the surface of the RTI sphere can be thought of as the spontaneous and co-authored signature of the total assemblage (Fig. 2). They are also another form of auto-archived visual paradata, recording the circumstances, environment, relative position, poses, gestures, and the condition of all the actants and their intra-actions in this emerging ‘assemblage of practice’ (Antczak & Beaudry, 2019) or ‘cognitive assemblage’ (Hayles, 2017) as it unfolds from image to image. RTI can, quite literally, shine a light on what the image maker is most concerned to bring forth from the artefact or assemblage (Callery et al., 2022).

Fig. 2

Auto archiving paradata via a RTI metapicture

5 Shining a Light on the Image Makers

We attempt to extend these insights concerning non-verbal paradata through four techniques. Three are familiar to archaeology and cultural heritage: Highlight-Reflectance Transformation Imaging (H-RTI), Structure from Motion photogrammetry (SfM), and 3D Structured Light scanning (SL). The fourth is Fused Filament Deposition (FFD), the technology of 3D printing, an emerging area in the cultural heritage sector as a supplementary tool for tactile engagement (e.g. Reilly, 2015; Reilly & Dawson 2021; Reilly et al., 2016). All these processes are additive. In RTI the subject under documentation and the digital SLR (DSLR) are held static and a series of images of the subject are taken as strong directional lights are moved around the subject, eventually forming a dome of lighting positions, to bring out details. The outcome is a synthetic model of the recorded object that can be interactively relit, and its surface properties redefined to allow, for example, specular enhancement (Fig. 1). In the case of SL scanning, bands of projected light haptically stroke the object of study in order to capture its surface geometry. In SfM, digital photographs taken from multiple different but overlapping viewpoints are also processed and assembled into synthetic images or three-dimensional models that can be rotated, panned, and zoomed, interactively. With FFD printing the digital model is cut into tiny slices, turned into a sequence of silhouettes to be printed in material layers.

We apply these technologies in (un)familiar ways to test the documentation processes that we hope will further the decipherment of the Nessglyph. We apply standard, and experiment with ‘dirty’, versions of H-RTI, SfM, and 3D SL scanning. Each of these three techniques in their pure form has associated best practices, all involving movement to take multiple overlapping impressions of the subject under study (Historic England, 2018a, 2018b). Images conforming to the best practices of each of these techniques produce a genre that tend to look very similar to one another, which is hardly surprising as they are composed in conventionalised poses and executed under constrained parameters. This means that, in practice, many of the decisions needed to record an artefact are taken away from the operator ‘in charge’ and are instead prescribed or delegated to the nonconscious cognition of algorithms and sensors concealed within technological or cognitive assemblages (Hayles, 2017; Huggett, 2017) that the operators handle. Flusser (2011) argues that the role of the operator in the making of ‘technical images’ has been reduced to that of a mere ‘functionary’, someone who just must point the instrument in the appropriate way and press a button. Cubitt (2014, 270) goes further and claims that operators of these instruments are ‘enslaved’ to the technology. Crucially, in the context of this discussion, the paradata (i.e. the decision-making criteria and constraining factors that had to be overcome) are now rendered silent and invisible in their black boxes. Ironically, their disappearance draws attention to their whereabouts and prompts further questions regarding their epistemic status in our knowledge frameworks. In other words, the provenance of the selected paradata to record is also called into question. Why were these paradata chosen as being more important than other options in the first place? These emerging layers of introspection—that is paradata about the selection and use of the paradata that are articulated—are called ‘peridata’ (Gant & Reilly, 2018).

Significant knowledge is certainly buried within these technical assemblages. However, we will argue that the performative images remain images in the making and therefore retain opportunities to break free of the shackles of the functionary and expose some of the ‘silences’ in the data, metadata, paradata, and peridata (Huggett, 2020; Ortolja-Baird & Nyhan, 2021)—those nonconscious and tacit aspects in our digital imaging knowledge making—through visual forms of diffractive analysis. Instead of attempting to consciously document the decision processes that culminated in our models, we create digital skeuomorphs of the Nessglyph in order to identify and explore, diffractively, non-conscious paradata, and non-verbal gestural peridata, embedded in the image datasets that were derived from the artefact. Skeuomorphs are objects or features produced in one medium that mimic, or cite, the processes of making, or the inherent properties, of a similar-appearing prototype made using another medium with different properties (e.g. a ceramic pot moulded to look like a woven wicker basket). Our approach is to record one documentation process through the lens of an ontologically different technological assemblage, including the embodied movements of the operators. So, for example, we will trace the making of a SL 3D scan through the uber-cyborgic gaze of a dirty-RTI (DiRTI¹) session (Dawson, 2020). Our focus is not the Nessglyph per se, nor its making, nor its interpretation. Here, we (re)consider our documentation processes diffractively through a form of visual introspection that enables us to critique our own embodied and mediated interactions with the artefact. Through this subversive art/archaeology manoeuvre, we endeavour to record how we document the Nessglyph, and extract visual traces of both the embodied and cyborgic decision-making processes (i.e. paradata) in their making. Crucially, we deliberately put ourselves in the frame to haunt the pixels we render (Fig. 3).

Fig. 3

Remote DiRTI scene of Ian Dawson’s Plastic Studio (2022)

In short, we offer a non-standard, transdisciplinary, and diffractive analysis of embodied art/archaeology gestural approaches to recording this carving and, specifically, what we call auto-archived non-verbal paradata and potential convolved peridata (i.e. paradata about the paradata and other related metainformation scenarios (Huggett, 2023; Martin-Rodilla & Gonzalez-Perez, 2019)).

The ‘technical images’ we develop, using these (non)standard documentary techniques, expose the effects of tell-tale gestures and actions, and allow us to recognise and reconsider significant non-conscious decisions (i.e. spontaneous paradata) made by practitioners within the overall cognitive assemblages through which these digital datasets are collected, at what density, and from what specific viewpoints, and what is lost in this blending. Each of the three standard recording methods employed produces interactive images, which means that part of the process of analysing these data (and, consequently, the paradata) can be outsourced by the maker to diffractive viewers when—most significantly here—the paradata about decisions on how to interact with these (para)data can become untethered. Nevertheless, the collected data (i.e. images) and assembly files may still be used to reveal careful, but perhaps unconscious or non-conscious, decisions about what matters to the practitioner in the making of the model. For example, in both RTI and SfM imaging every viewpoint is embedded and (re)displayable in the finished model. We can therefore begin to reverse engineer the practitioner’s focus of attention and, to a degree, their ‘intent’ by showing, for instance, which parts of the object under investigation bear more scrutiny than others. Equally, we can determine some of the constraints imposed by the process of data collection. For example, we experiment with a subversive art/archaeology version of Reflectance Transformation Imaging (RTI), called dirty-RTI (DiRTI), to document, recursively, both a structured light 3D scanning and RTI-recording session. During the DiRTI sessions we capture non-conscious decision-making gestures and movements of the practitioners made in the mise en abyme occupying the space between a panopticon-like digital camera and the petroglyph.

Later, we extract geometric and other surface data, such as colour and surface normal from our image datasets and reuse them to create 3D prints of the artefact. During this metamorphic process much of the previously auto-archived gestural paradata outlined above is severed, and a whole new layer of physical and digital (phygital) paradata physically congeals in these 3D material (re)renderings of earlier image datasets. Decisions in the workflow—which include, for instance, digital coring and slicing—leading to printing components in a particular orientation and then their physical assembly all become evident in the object, giving these latest skeuomorphs a set of interactive parameters of their own. In some instances, in an odd inversion of properties, such as specular enhancement, these new objects become analogous to RTI when held and rotated in one’s hand.

6 Paradata in Motion

The Nessglyph has been subject to a series of documentation processes familiar to archaeologists and art historians. Figure 4, for example, is a rough sketch made by Reilly in his scuffed and blotted day journal. These reflexive notes illustrate the usual sorts of metadata, such as dimensions, annotating sketches of new finds, and developing interpretation of features as they emerge. Field notebooks are paradata par excellence. Unfortunately, in terms of being knowledge containers for knowledge management they are problematic. They are highly idiosyncratic, unstructured, and, as apparent in Fig. 4, often badly treated by wear and weather. As Huggett (2020) observes, the journals of the fieldwork team do not feature in the final hegemonic reports that get published under the names of the authoritative principal investigators. They may not even reach the physical archive. With some exceptions, these thoughtful notes will go mostly unread. Although internalised by the individual practitioner, this knowledge is not widely shared and so the valuable written and drawn insights and learning points become lost opportunities, that is orphaned paradata.

Fig. 4

Typical archaeological paradata: annotated, weathered, and blotted sketch of the Nessglyph in a day journal

Far more likely to find their way into a knowledge base and, perhaps, the hegemonic site report, are the carefully constructed, and processed, images (Morgan & Wright, 2018). Compare how the carved Nessglyph is presented in Fig. 1 compared to Fig. 5. The incised sandstone block is showcased conventionally in Fig. 5. In Fig. 1 the Nessglyph is presented in an apparently static, empty, featureless, and distraction-free space (except for the bottle holding the reflective sphere, which also operates as a scale). However, this illustration disguises many important aspects of the Nessglyph, not least the circumstances and environments surrounding its discovery and its recording, and, crucially, it forecloses other potential interpretations (Derrida, 1994; McFadyen & Hicks, 2019; Thomas, 2019). To begin with, despite appearances, the exhibit in Fig. 1 is not a photograph. It is a frame from an interactive polynomial texture map (.PTM) produced using the technique called Highlight-Reflectance Transformation Imaging (H-RTI) (Historic England, 2018b). Similarly, Figs. 2, 3, and 5 are synthetic views derived from a series of digitally skeuomorphic photographs, using skeuomorphic photographic technology and in the genealogy of analogue photographic tropes (Jones & Díaz-Guardamino, 2019; Taylor & Dell’Unto, 2021), but updated to create something Flusser (2011) characterises as a ‘technical image’. Like all technical images, they have more in common with spreadsheets than pictures (May, 2019). As Galloway (2011, p.88) emphasises, data in their purest form exist as numbers and therefore their ‘primary mode of existence is not a visual one’. To create this technical image, data were ‘assigned visual characteristics and converted, or rather translated, into what we humans recognise as photographs’ (Zylinska, 2017, p.26) However, as Rubenstein (2020, p.4) points out, the same data could equally be ‘output not as an image file but as a sound file, a text file, as a string of numbers, or it could be left unprocessed.’ So, whereas the material artefact is a heavy carved red sandstone block that is difficult to handle, the digital skeuomorph (re)presented in Fig. 1 is a dimensionally elastic, weightless, surface apparition. The rendered surface of this simulacrum has no thickness and envelopes nothing, but is susceptible to panning, zooming, and relighting. The seemingly empty, inert space surrounding the digital simulacrum is equally skeuomorphic and just as deceptive. This apparent void is defined in the same code and data definitions as the featured artefact, and potentially contains the memory of light, shot through with movements that are traces of decisions and choices in motion, which flooded the recording session as it progressed. We will reveal some of these energetic, non-verbal, paradata haunting the pixels, and their underlying data that were intended to (re)present the Nessglyph.

Fig. 5

Frame from an H-RTI session recording the Nessglyph

We have been on the trail of non-verbal paradata for a while now. During the pandemic, forced to collaborate at a distance, we began to consider how the widely used heritage recording technique of RTI could throw light on itself as both a technical assemblage and a process. After seeing ourselves so often within Teams and Zoom sessions it was perhaps inevitable that we would try to find ways of breaking away from the tyranny of technology (pace Flusser (2011) and Cubitt (2014)) and subvert, bend, or break some rules. We decided that instead of hiding ourselves behind the image-making process we would deliberately situate ourselves within the frame of the camera in order that we could, later on, observe our embodied decisions about where and how to position the all-important and meaning-making light source in each successive frame of the RTI (Dawson, 2020; Reilly et al., 2021).

The interactive images that began to emerge from these sessions were startling (e.g. Figs. 1 and 5). The ‘empty’ studio space was now filled with metabolic energy and movement (Fig. 5). The studio space also became plastic and metamorphic. The end user experience of this dirty form of RTI file through the RTIViewer is transformed as ghostly apparitions come in and out of view as the user interacts with the RTI. This is possible because RTIs are miniature quantum-like universes where the linear arrow of time does not apply. What was previously ‘before-and-after’ now stumble around one another through quantum superpositions of pixel properties (Callery et al., 2022). We realised that through these images we could watch our decision-making (un)folding, that paradata could now haunt the pixels of the interactive image, and, perhaps most interesting of all, that we had stumbled upon a back door for the functionary—a way into, and back out of, the black box of RTI.

We wondered if we could extend this insight into investigating the other two standard archaeological computational photography methods that we had already deployed on the Nessglyph. Could we diffract similar kinds of embodied paradata that are interstitial to Structured Light (SL) scanning through a DiRTI process to reconsider our unthought embodied decisions—the indirect visual haptic strokes of the scanner at a distance—as diffracted peridata?

As we have already mentioned, in RTI the ostensible subject of study—accompanied by a reflective sphere and the imaging device (DSLR)—remains static in the recording session. All the action takes place between each frame being taken when the strobe or other light source is moved around the set. Highlights on the reflective sphere enable the RTI software to simulate all the various lighting angles required to interactively relight the model. By contrast, working with the Artec hand scanners available to us, the image maker has to constantly negotiate with the object. The lasers record thousands of readings a second in waves of cold flickering white light. It occurred to us that we could use those flickers as the directional light source for a DiRTI session on the Nessglyph (i.e. a hybrid or interlaced SL and DiRTI recording session run concurrently). Experimentally, we discovered that by adjusting the exposure for each RTI image to about 3 s could produce stunning hybrid forms of extended drawing to help us reconsider the gestures made by the operator.

Figure 7 is a single frame from one such interactive SL/DIRTI hybrid recording session. Here the graceful knots of movement that eluded us in the RTI of the stacked cardboard trowel marks produced with Stefan Gant (Gant & Reilly, 2018) snake and twist in the form of mottled ribbons of light around the Nessglyph when the compiled composite dirty-RTI is inspected through the RTIViewer. These snakeskin-like ribbons of light are the traces of remarkable unsighted hand ballets.

Fig. 6

SL Nessglyph scanning session note operator looks at laptop

Fig. 7

Diffracting structured light scanning through dirty-RTI

While the scanner operator must look away from the Nessglyph and concentrate on the screen informing the functionary about which parts of the stone are being rendered by the flickering lasers (Fig. 6), the slower eyes of the RTI trace subtle and delicate twists, turns, and rolls of the hand-operated scanner. Unseen by the operator, the stone is being stroked by strobing light at a constantly adjusted hovering distance from the surface of the stone to expose every mark incised on the red sandstone block in interactive detail. Decisions about where to point and wave the lasers to bring out important details are made haptically through fine adjustments of proprioception in the hand, wrist, and forearm of the operator. These gliding, handheld, scanning gestures bear an uncanny resemblance to the actions of the troweling archaeologist trying to envisage the form of buried artefacts and features hidden from direct view. These SL paradata now figure in the RTI paradata (Fig. 7).

7 Phygitally Dislocated Paradata

As Mitchell (2003, p.3) declared two decades ago in Me++, the separation of bits (the elementary unit of information) and atoms (the elementary unit of matter) is over. With increasing frequency, events in physical domains reflect events in virtual domains. Phygital information can, for example, direct the movement of the printer nozzle of a 3D printer to produce another skeuomorphic iteration of the Nessglyph. Now the movements and gestures (i.e. embodied and gestural paradata) of the original glyph maker(s) and the art/archaeological investigators are interlaced with the post-human cyborgic gestures of an additive manufacturing fabrication process. For example, the physical (re)presentations of the Nessglyph shown in Figs. 8 and 9, produced by slicing the digital prototype into a sequence of silhouettes, allow for a new phygital skeuomorph of the original artefact to be 3D printed in layers.

Fig. 8

3D print derived from a structured light scan responding to the idea that the Nessglyph might have been carved into the stone with a bronze tool

Fig. 9

Performative mattering: Nessglyph (re)presented using the wholly different set of gestures made by the plastic artist and a 3D printer

To accomplish this, the SL scans were converted into three packages of data: an OBJ file, which defines its geometry as a sequence of code that lists the xyz coordinates of the vertices of the object, plus a Material Template Library (MTL) file providing the lighting information, with a texture map (often saved in a JPG or TIFF format) wrapped over the OBJ code. These were converted into an STL file in order to 3D print the model. The STL file, developed by 3D Systems as part of their development work with the 3D Stereolithographic process, is another sequence of code which enables a model built from interconnecting triangles to be easily sliced horizontally in preparation for the 3D print process. On the 3D printer these horizontal layers are printed in a continuous sequence. A PRUSA i3 MK3 printer was used to print the Nessglyph firstly on a reduced scale with a plastic/bronze amalgamated filament (Fig. 8). This object changed colour like a conventional bronze; its green patina referencing its metabolism as it slowly oxidised. The action of the 3D printer is captivating, like watching a stylus on a vinyl record. There are multiple enchantments that the machine gestures towards. The 3D print we hold in our hands is a souvenir of the myth of the pure potentiality of the formless, and the capacity of the phygital to morph, recompose, and reformat. The 3D print as a souvenir performs the role of catalyst for a new narrative. Once extracted from the 3D print bed, the 3D printed Nessglyph substitutes the context of origin of the glyph with the second-hand experience of the handler. We are now in possession of a metonymic object. Such objects are but samples of a now past experience, and by their very own impoverished translation, the partiality of the souvenir is laid bare. These copies are allusions, not models. They function best alongside a loosely attached supplementary narrative discourse that creates a new myth with regard to their origin (see Stewart, 1984, p.135). In other words, these souvenirs foster the need for further (mythical) paradata schema.

During this phygital metabolic exchange, the concepts of techne, poiesis, and empeiria have all been infused into a material amalgam that replaces the paradata associated with the initial digital model in a dazzling rendering of completely new, cyborgically expressed, gestures implemented in colourful plastics. In this metabolic transformation, links to any pre-existing paradata are severed. Paradoxically, these new mythical plastic instantiations of the Nessglyph amplify the initial inquiry: ‘what am I recording or (re)presenting?’ The outcome of our experiments it seems is that epistemological and ontological concerns are now entangled (Fig. 9).

8 Summary, Discussion, and Conclusions

We have argued that attempts by galleries, libraries, archives, and museums to present artefacts within empty spaces are misleading and divisive. These exhibits and accessions hide the considered, embodied, work and decisions—in other words the layers of paradata (peridata)—made by many largely invisible people, applying their considerable skill sets, knowledge, and experience, by framing and presenting the artefact in a disembodied featureless vacuum.

Fig. 10

Peridata: layers of (data)introversion. By choosing specific data, metadata, paradata to be recorded, we invite further layers of introspection: why were they chosen? (ad infinitum)

In contrast, by diffracting several imaging processes through one another, we have tried to help anonymous field archaeologists and artist assistants, and other enslaved technological functionaries, escape the shackles of automated image-making apparatus and the requirement for them to produce images that correspond to certain general conventions and configurations. We have also attempted to break away from conventional strategies of teaching embodied practices that rely on verbal instruction for learning. Our unconventional DiRTI approach has enabled many layers of recursion through Structured Light scanning and Structure from Motion photogrammetry information infrastructures and, admittedly, the images we create take considerable effort to unpick. At a top level, we have compiled ourselves and our meaning-making processes, recursively, into these DiRTIs of the Nessglyph (Fig. 3). The blended RTI sphere (Fig. 10, top right) highlights the diffractive shadows we cast on the data underpinning this RTI. Each flash of the strobe creates a metapicture which, when assembled, reveals the many different positions that we and our apparatus adopted during the shoots (Dawson & Reilly, 2019). What becomes clear is that the apparently inert, empty, space surrounding the object under study is an intersubjective space full of energy, light, movements, gestures, and equipment (Figs. 2, 5 and 7). These interactive intersubjective spaces are laden with meaning-mattering decisions and adjustments. We can think of DiRTIs as a form of ‘autographic’ image (Offenhuber, 2020). That is to say that it contains ‘a trace of the process itself: it retains some interpretive authority, and it is taken as a product of the phenomenon at its face value’ (Likavčan & Heinicker, 2021, p.212).

Autographic images, which include timelapses, live streams from space, and the RTI images that we have practised here, become phantom operational images (Farocki, 2004) as human labour is joined with the labour of computational algorithms. Autographic images are sensitive to the socio-material context (Fenwick et al., 2012; Pelizza, 2021) of their contrasting ‘information infrastructures’ (Huvila, 2019) and, just as in our hybrid DiRTIs, they reveal material phenomena as visible traces which draw the viewer’s attention back to the intimate ways in which the previously opaque reality of the process of imaging can now unveil itself in the productive alignments between its human and non-human elements. These ‘images in the making’ contain things in motion, involving conscious and non-conscious processes of assembling and reassembling, and reimaginations of the world, in other words paradata. As Back Danielsson and Jones (2020, p.4, original emphasis) point out, ‘if we understand imaging as a process of assemblage making, subsequent processes of viewing and intra-action are also components of the continuous process of imaging.’ Put another way, these images can be understood as assemblages of ongoing processes and ongoing paradata in the making.

Remembering that all RTIs naturally operate as ‘metapictures’, the DiRTIs presented in this chapter are not simply epistemological models. The unconventional cognitive assemblages we have created also allow us to observe the observed and the observers. The images are not intended merely to serve as illustrations to a commentary on decision-making in practice-based disciplines like archaeology and fine art. They directly picture layers of paradata in the making (i.e. peridata). Figure 10 summarises this recursive process: the data of interest are selected and then assigned attributes (metadata); since both data and metadata are chosen there is a need in some intellectual quarters to justify how these decisions were arrived at (paradata). The blended RTI highlights are a form of visual paradata recording how and in what order the data were imaged. This blended DiRTI highlight image is a hauntology in which the black areas represent lost futures of analysis in this model. In these dirty sessions the highlight detection algorithm was often unable to disambiguate the true highlight of the source. They had to be manually edited. Unfortunately, the finished blended highlight file is not updated by the software with these manual corrections and so this summary image appears to give a much lower density and coverage of light source positions. However, the decision to record those paradata and their attributes remains just another layer of introspection. Put simply, one person’s metadata or paradata may be another’s data, requiring further introspection. These layers of introspection—we have called peridata—could, in principle, be never-ending.

The case for non-verbal paradata in art/archaeology as business-as-usual is therefore problematic since the reasons for elevating certain decisions to paradata status also need to be analysed and explicated (recursively via peridata). Problematic but not necessarily futile (contra Reilly et al., 2021). In this study, we have diffracted embodied paradata haunting several different forms of ‘undigital images’ (Zylinska, 2021) that were created and modified by archaeologists, artists, and cultural heritage workers more generally, to reveal spectral pedagogical boundary objects that enable us to share and exchange non-verbal, non-conscious, embodied, multimodal, decision-making processes across disciplinary boundaries and domains, and across different learning levels (apprentice, journeyman, master) and times. In their physical translations earlier paradata became untethered as new paradata of performative mattering reinstated the Nessglyph in novel dazzling plastic forms. In this admittedly subversive art/archaeology context, diffracting embodied paradata allows us to interlace different registers of techne, poiesis, and empeiria which then enables us to expose important points of difference and start reconsidering some of those elusive layers of tacit learning and teaching that underpin the development of skilful meaning-making embodied practices in field archaeology and art. In the process we have shifted the role of paradata from the reflexive epistemic considerations contained in, for example, traditional field notebooks to a diffractive position in which a relational ontology has emerged that can no longer be categorically separated from epistemological processes. Our entangled computational photographic and 3D printing methods of engagement are simultaneously generating radically immanent but relational new worldings. In these art/archaeology material-discursive experimental and metabolic entanglements, ontology and epistemology have become intra-laced, implying that both paradata and peridata have opened a new onto-epistemological dimension requiring (re)theorising.