Introduction: Observing Educational Relations

Digital technology produces environmental effects of which sound is an important component. The term ‘postdigital' can usefully point to these  effects — particularly the increased contingency and uncertainty in social relations and the blurring of the distinction between message and medium, as predicted by media theorists since McLuhan (1994) and Kittler (2006). It represents a Gestalt-switch from seeing artefacts, technologies and intended purposes as separate from an observer, towards seeing relations between ‘observing systems’ (Von Foerster 1981). If there is a precursor to this Gestalt-switch, it lies in the shift from ‘first-order’ to ‘second-order’ cybernetics in the early 1970s (Von Foerster 1981; Maturana and Varela 1980). To analyse sound is to examine the dynamics between observing systems, and this can shed light on meaningful communication in education.

Schütz (1951) pointed out that music was a meaningful communication without having any conceptual reference. It worked, according to Schütz, because musicians observed and ‘tuned-in’ to the inner-worlds of each other. The postdigital environment appears to be characterized by similar flows of meaningful communication where conceptual schemes are increasingly fluid. In the light of this, it is perhaps unsurprising to see artists — and particularly musicians — embracing the postdigital term where awareness of ‘edge boundaries’ (Cascone and Jandrić 2021) resonate with earlier interest by cybernetically-inspired musicians like Brian Eno and Pauline Oliveros. More recently, Bridle's (2022) work on intelligence in the interface between humans and machines points both to the liminality of human reason and the cybernetic history of what he calls ‘non-binary machines’. To study the sonic environment of education in the context of the postdigital is to study how this mutual tuning-in process might work.

But how can this be analysed? How can an observing system observe the relations between itself and other observing systems to shed light onto the nature of the system of which it is part? It is in the context of addressing this question that we have embarked on the present analysis of the sonic environment in education. In his consideration of mother–child relations, Bruner (2006) draws attention to the prosody of the mother’s voice in the management of joint attention. He points out that the intonation of her voice in uttering ‘see the pretty dolly’ is coordinated with ‘moving the object into the child’s line of regard and shaking or otherwise “forefronting” it’ (Bruner 2006: 71). In the light of Bruner’s insight, we might ask how we manage joint attention in a postdigital educational environment. What role does sound play in these relations and how might its variation be seen alongside other (often competing) stimuli in the digital realm?

Relations must be manifestations of some underlying process, and that process must involve (a) internal processes of biological and psychological organization within individuals and (b) the external productions of those processes. In Bruner’s example, internal physiological processes in mother and child unfold where the mother’s utterance and the child’s apprehension produce coordination between them. Following Harré et al. (2015), we can distinguish these dimensions as ‘engrams’ (internal) and ‘exterograms’ (external). Then we can ask, what turns an exterogram into an engram and vice-versa? Furthermore, we can compare the different contexts within which this transformation unfolds — so what is the difference between a face-to-face sonic environment and an online sonic environment? One field of intellectual inquiry which has focused on these questions is that of ‘communicative musicality’ (Trevarthen 2015).

Within communicative musicality, sound can be considered as an exterogram which — as an energetic phenomenon arising from the oscillation of matter — must have a correlate in the biological energy in people making sound. The effects of the transformation of the biological energy in people into external utterances are apparent in educational studies of the social interactions of young children. Nome (2020), for example, noted the significance of non-verbal communication in establishing group cohesion among toddlers, while Erikson (2009) identified patterns of rhythm, pitch and voice quality in classroom interactions. It is obvious that learning is enveloped by sound energy - whether the sound of the classroom or café, the Zoom lecture, or the Youtube channel. If the energy of the exterogram can be analysed, can that tell us something about the physiological energy of the engram? How might these two forms of energy be related?

Our empirical approach here is to examine and redescribe sonic phenomena to shed light on the dynamics that must relate external sound energy to physiological processes. In doing so, important sociological perspectives on the sonic environment can acquire new empirically investigable dimensions. For example, Lefebvre’s rhythmanalysis (2004) concerns different orders of patterning of phenomena (e.g., rhythms in social life), and this has invited postdigital studies invoking Lefebvre’s theories to exploring the rhythmic impact of digital tools (Ford 2022). In this paper, our redescription involves turning what Lefebvre sees as different patterns into measurements of information entropy (Shannon and Weaver 1949), following similar approaches in the analysis of ecosystems (Ulanowicz 2011) and analysis of music (Johnson and Leydesdorff 2020). These redescriptions of the sonic environment may provide an empirical foundation for both understanding and intervening in the sonic environment for learning, such as those recently suggested by Ahern (2022), or in revisiting digital approaches to Oliveros’s cybernetically-inspired concept of ‘deep listening’.

As Bateson highlights (2002), all mental processes require energy. Energy comes from many different sources — from the ion transfers of cellular chemiosmosis, to food and sunlight. The role of sound in the energy equation of biological development is less well-understood than chemical or photosynthetic processes — although recent work on the effects of sound on plant growth are indicating that it has a causal bearing on growth (Hassanien et al. 2014; Jung and Pauli 2014; Jung et al. 2018), and that music impacts non-auditory human cells in similar ways to auditory cells (Lestard and Capella 2016). In other words, the sonic exterogram has causal effects on the physiological mechanisms of the engram. Energy is the underlying common factor mediating these processes. This suggests to us that examining the sonic environment in relation to learning renders the dynamics of educational development as a process of energy flow inspectable.

Sound is a manifestation of kinetic energy which can be measured in various ways. The decibel, for example, is a logarithmic measurement of power deriving from the amplitude of vibration. Perception of sound, like other forms of energy, requires a physiological process to transduce the energy of vibration into physiological changes which result in a range of phenomena that we associate with ‘hearing’ and ‘listening’, while through further processes of reflexivity, the effects of sound produce expectations, meaning and understanding. Other forms of transduction of sound energy are possible, including the visualization of waveforms on oscilloscopes through to the beautiful cymatic patterns of vibrating fluid on a flat surface. The meaningfulness of sound is dependent on this energetic transduction where there is ‘a difference that makes a difference’ (Bateson 1987): kinetic energy makes a difference to physiology.

Part of the empirical aspect of our study is the consideration and comparison of the sounds of the classroom and the sounds of the online environment. In the former, we have analysed the class group activity in a school exercise of children engaged in an inquiry-driven process in a school in Denmark. In the latter, we have explored some examples of online streaming of gameplay and the sonic excitement associated with this.

To examine these case-studies entails the need for an analytical approach to ‘observable’ sound which can then be related to speculations as to what happens to individuals in the sonic environment. To do so, we present a theory that the essential pattern of sound is mirrored by patterns in physiological changes. This is significant because regarding the totality of sound and physiology together, sound is a flow of energy which is transduced into different forms, from kinetic energy in the air to physiological transformation in cells (e.g. cochlea, neural structures, and endocrine dynamics). This means questions about the dynamics and effects of sound become questions about the conservation of energy through these different transformations. This situates processes like learning within a cosmological context of physics and biology, as opposed to more conventional views of learning as a local phenomenon of transfer of information.

Central to our approach is the consideration of time, where we can divide sound events into those whose properties are synchronic (i.e., occurring at the same time) and those which are diachronic (i.e. those occurring one after another). The harmonic spectrum, harmony itself, timbre, amplitude and frequency can be considered synchronic (albeit overlooking the oscillating — and so diachronic — interference patterns that affect timbre). Synchronic patterns can be studied purely from the domain of physics, and indeed, it is no accident that these synchronic aspects of sound have fascinated scientists since antiquity. Diachronic patterns meanwhile, concern identification of rhythm, motif and structure.

Vectors of Variation: Sonic Comparisons in Educational Settings

Before embarking on a deeper investigation, as a way of setting the scene for our inquiry, we can use computer tools to provide a means of visualizing what we are talking about. Where Bruner (2006) draws the intonation of speech by hand, technology now allows us to graph the pitch of speech automatically, and this allows for more sophisticated analysis on its qualities and variation. Figures 1, 2 and 3 show visualizations of the dominant pitch of sound in three different situations with which we concern ourselves. Figure 1 shows the pitch of sounds from a classroom where children (aged about 14) are discussing an educational computer game intended to encourage them to learn the physics of visiting Mars. A group of 5 children are talking together, and the graph picks out the differences between the pitches of their voices. Figure 1 shows a distribution of pitches representing the contributions of different members of the team and their relative dominance in the conversation.

Fig. 1
figure 1

Classroom conversation between small groups of children

Full size image

Another classroom situation is shown in Fig. 2. This is in relation to the same exercise, but with a different teacher. This teacher delivers instructions about what to do with the game to the whole class. Most of the audio in this section is the teacher’s voice. The sonic graph therefore picks out the varying pitch of the teacher’s voice. This representation illustrates the contrast between the children’s interaction in Fig. 1 and the teacher’s monologue in Fig. 2 and provides an analytical foundation for asking deeper questions.

Fig. 2
figure 2

A teacher-led session in the classroom

Full size image

Figure 3 shows a snapshot of a video game tutorial taken from YouTube. This sonic landscape is in many ways similar to Fig. 2, because there is an ongoing commentary by a single person, but there are specific differences in the delivery. While this is a monologue conducted in the context of demonstrating the particular features of a game, the pitch of the voice is more varied than the classroom teacher’s pitch. It is this kind of variation in all three examples which interests us.

Fig. 3
figure 3

Online instruction about gameplay

Full size image

One way of considering the variation in pitch of the sound is to consider its information entropy (Shannon and Weaver 1949), as a way of establishing the degree of difference and variety there is in the sonic engagement. This can be easily calculated (see below), but at a first visual inspection, Fig. 1 has the highest entropy; Fig. 2 has the lowest; and Fig. 3 is somewhere between.

The variation in the pitch will have a corresponding variation in the vibration of molecules. Unlike established quantitative approaches to the measurement of sound energy (e.g. decibels), this measurement of variation in the vibration of the air is not a scalar measurement but rather a vector over time. Moreover, while our simple visualization has included just the pitch of sound, we should also ask: what about the rhythm, or the tone of speech, or the variation in loudness, or pattern of actual words used, or even the semantics and significance of what is said in its context? All of these factors must feature in the sonic environment: a fire alarm, for example, looked at in isolation will be uninteresting in terms of pitch, but when looked at in context, the very shift in entropy of sound which it creates is precisely the difference that indicates something to which attention must be drawn. What is interesting about the fire alarm is that while it inspires people to act (expending energy) it does not encourage people to imitate or continue the production of the sound. Yet, in Fig. 1, this is precisely what happens: utterances by one person instil the potential to produce utterances by others. The vectors of variation of sound have an effect on physiology which results in the continued generation of sound. Our question is whether and how vectors of variation in sound vibration affects the physiological processes which give rise to the ongoing production of sound and to the phenomena of learning which might be associated with this.

Kinetic Energy and Physiology

Biological organisms produce sound in response to a physiological disequilibrium: a cry of pain is an outward sign of inner disequilibrium, whether that disequilibrium has an external (an inflicted injury) or internal (illness) cause. Whatever the cause, the result is the creation of kinetic energy which physics would regard as the release of ‘potential’ energy. But what (and where) is potential energy stored in a physiological body and what are the mechanisms for both its creation and its release?

Simondon (2020) pointed out that potential energy is an abstract concept which, whilst inviting measurements through (for example) calculation of the heat stored in a thermal body or the height of a weight to fall, he writes: ‘The capacity for an energy to be potential is strictly linked to the presence of a heterogeneity; i.e. of dissymmetry relative to another energetic support.’ (Simondon 2020)

In other words, energy is only potential if there is a difference in which it can make a difference. In information theoretical terms, as Wiener (2013) pointed out at the beginning of cybernetics, this implies a kind of Maxwell Demon whose action is to separate energetic states:conversion of potential to kinetic energy requires some kind of agency to act in such a way as to start the process of releasing kinetic energy. Wiener’s point is to suggest a deep connection between Boltzmann entropy and the concept of information. However, if Wiener’s intuition is correct, then how does this translate to the relationship between the kinetic energy of molecules in the air and the information which we humans construct from those vibrating molecules?

The circular connection between sound energy and physiology suggests that the sound energy itself is not something transferred from a source to a receiver, but rather something that is continually transformed within different substrates: from vibrations in the air to signals in the body and signals in the body to vibrations in the air. While living systems counter the entropy of the universe through producing negative entropy (Schrödinger 2012), they increase the net entropy through the production of energy through heat and decay. Throughout the natural world, animals create sound where the potential for doing so appears to be a response to sound. Communicative musicality is the study of the patterns of a continual process of energy transformation between biological systems and their environment. Since interpretation of sound as a physiological reaction to the kinetic energy of molecules is critical in sound’s ongoing production, the physiological construction of meaning must be considered as an integral part of the transduction process.

Online educational environments have produced a novel intervention in the physics and physiology of sound where the importance of noise is overlooked. Like all digital systems, there is a distinction between signal and noise, where signals are amplified and noise is attenuated. The attenuation of noise produces a particular kind of absence: the dynamics of teaching online involve teachers in making sound (talking) which is not met with the noise of class chatter, and this can produce a physiological imbalance. The teacher releases potential energy in utterances but has no means of registering the energetic effects of sound on the audience. The physiological imbalance might compel the teacher to make further utterances (because utterances result from a physiological imbalance), but these too are met with an absence of noise. In the natural environment, the utterances might entail shouting or surprising the audience in some way (is there anyone there?) — but many of these utterances are prohibited in the environment because the teacher knows it is recorded. More than that, other expenses of energy to deal with physiological imbalance which would be otherwise available in the natural environment such as moving around the class, are not available. They must remain fixed in front of a screen which is recording them.

Having said this, the absence of sound is not necessarily an indication that the energetic flow of the communicative sonic environment is absent, and neither does the abundance of sound indicate the flow of energy. Silence, as understood by Cage (2011), is part of a dynamic process: ‘What we require is silence, but what silence requires is that I go on talking.’ Silence therefore indicates thought — and thus, the transfer of energy in constructing new physiological structures. However, if the sonic environment produces, through the attenuation of noise, a conflict in expectations as to what sound production is expected, and what should be received, then this can be a sign that psychological and physiological processes become detached from the natural environment, blocking the energetic flow and producing anxiety or alienation. Any teacher who finds their students in Zoom with their cameras off and no sound will understand the depleting effects of this lack of energy, Meanwhile, the media and social media, bombard us with sound whose energetic effects amount to little more than what Beer (1994) calls an ‘entropy pump’ whose purpose is to exercise social control through confusion.

Energy Conservation and Learning

The primal role of physiology becomes clearer when considering both a synchronic dimension and a diachronic dimension in sound. Anything diachronic depends on physiology: biological and psychological adaptive processes are essential for establishment of expectation and habit, identification of regularity and disruption, through to the identification of rhythm, melody and deeper structures of speech, form and tonality. The processing of expectations can only be obtained reflexively — that is through organization processes which operate retrospectively against the arrow of time (Leydesdorff and Dubois 2009). These diachronic organization processes are what we typically associate with learning.

To put this in Schrödinger’s terms, the entropic (disorder-producing) nature of kinetic energy is met by negentropic (order-producing) processes of physiological reorganization. The kinetic energy of sound must be transformed by physiology into structures of expectation which produce order within cognitive systems. Metabolic changes involving different physiological systems, including the endocrine, nervous, respiratory, musculoskeletal and cardiovascular systems, dispose the biological system into responding with the production of sound. But given that the different forms of measurable phenomena of sound relate both to synchronic and diachronic aspects, and given that for the synchronic aspects, these have a diachronic aspect too (for example, the changes to timbre), what theoretical angle can be brought to bear to see these processes of expectation formation and physical properties of sound together?

Typically, education is (implicitly) associated with the process of negentropy: the increase in order in the physiological (psychological) system, where increasing order is seen as ‘knowing more stuff’. However, if learning is seen in Schrödinger’s terms, then the decrease in entropy within physiology (which produces order) will be related to the ongoing increase in entropy in the environment. Seen in this way, learning is an energetic balancing process with the environment — one that, we might hope, serves to ensure that the energy within one generation flows to the next generation. An important question (beyond the scope of this paper) is whether our current education system facilitates the intergenerational flow of energy or whether it obstructs it.

A Holographic Analytical Method: Entropy, Learning and Sound

Now we can return to Figs. 1, 2 and 3. We suggested that pitch was only one factor: rhythm, intonation, volume, the pattern of words and indeed the meaning of words are all parameters in an energetic process that implicates physics and physiology. In considering the synchronic and diachronic aspects of sound, we can consider that patterns of change in one dimension are related to patterns of change in another dimension. Furthermore, we suggest that the synchronic dimension and diachronic dimensions interfere with one another, and that this pattern of interference can be examined. Like the interference pattern of light frequency produced in a hologram, this produces a fractal-like pattern which is a 2-dimensional representation of a 3-dimensional situation: a metaphor which has been used within quantum mechanics to explain the structure of the universe (Bohm 2002).

We can use the Shannon and Weaver (1949) entropy equation, \(H=-\Sigma { }_{i=1}^{n}{p}_{i}{\mathrm{log}}_{2}\left({p}_{i}\right)\), as a way of directly measuring the variation in Figs. 1, 2 and 3. This can be done in a number of ways. For example, we can measure the entropy over the entire sound period or we can measure the entropy of a ‘window’ which moves as the sound progresses. The advantage of the sliding window technique is that it reveals the variation in entropy of sound over time: the ways in which the ‘vector’ of variation in pitch over time shifts. However, as we noted, this variation in pitch is not the only element in the sound to vary. Rhythm, intonation and volume all have associated entropy metrics. Using the three sound sources in Figs. 1, 2 and 3, we can also measure these values and plot the different entropy measures against one another.

Figure 4 shows the entropy of the timbre (richness) of the sound (ent1), the loudness (ent2) and the pitch (ent3) from a short excerpt of human speech. The result is a kind of ‘counterpoint’ between the different entropy values. This counterpoint has a direct correlate in music: is it quite common to see a distinction between ‘melody’ which has a variety of gestures in its movement, and an ‘accompaniment’ which can be typified by simpler repeated patterns, for example, in arpeggios, chords or the ‘Alberti bass’ of classical music. In such cases, one part displays low entropy (the accompaniment), while the other part displays higher entropy (Johnson and Leydesdorff 2020).

Fig. 4
figure 4

Flow of entropy of richness (ent1), loudness (ent2) and pitch (ent3) in human speech

Full size image

The relative movement of entropy between different synchronic factors provides a set of relations between those factors which, if taken in pairs, mean that beyond measured variables A, B and C, there are relations of AB, AC, BC and ABC. As more variables are added, so the number of combinations rises. Equally, if the relatative entropy of AB is considered as a new variable existing alongside the original factors, then that new variable also has a relative entropy with every other variable. In this way, a fractal pattern between variables can be established. The movement of relative entropies can be represented as a fractal of the sound in a way that we describe in the following section.

When a sound stops, all entropy values become 0. In music, prior to stopping, it is not uncommon to find multiple dimensions converge in the transformation of entropy. For example, musical endings are typified by repetition on the final chord (low entropy), alongside a collective change in volume (either louder or quieter) alongside a collective change in tempo (for example, everything slowing down or speeding up). Where there might have been high divergence of entropy in different dimensions and an associated high relative entropy between dimensions, there becomes convergence, and relative entropy tending to zero.

In examining a graph of relative entropies between different dimensions, we can ask whether there is a pattern in the fluctuating counterpoint of entropy values of parameters and in the relative entropy between those parameters. The identification of pattern between these dynamics would suggest that while specific phenomena at a particular point in time might be different in their content, in their relative entropic structure, they are similar to other different phenomena at a different scale. In other words, the structures of entropy flow between synchronic and diachronic dimensions can be fractal or holographic.

Segmenting Sound

Fractality is important in understanding the ways humans anticipate their environment. In learning, Kelly (2013) has argued that anticipation is a central feature in the formation of ‘personal constructs’. This idea was later developed into the conversational learning theory of Harri-Augstein and Thomas (2013). Anticipation is a process of recognition of the emerging pattern of present events based on identification of patterns in previous events. This requires self-similarity between the patterns of the past and the patterns of the future, and since anticipation is not mere repetition, it assumes the ability of such patterns to ‘scale’ from patterns established from one set of experiences to a different set of experiences. Scale invariance is a key attribute of fractality.

It has long been established in biology that even the simplest biological systems (for example, cyanobacteria) anticipate their environment. Theoretical work both in systems research (for example, Conant and Ashby 1970), biology (Rosen 2012), and computer science. Dubois (1998) has explored the mechanisms of anticipation, each concluding that anticipation must result from a fractal structure.

Leydesdorff and Dubois (2009) have presented a technical explanation of this mechanism. Since anticipation is about having an idea of what is about to happen next (at time t + 1), this expectation must depend on the simultaneous mutual inter-relationships between what has happened before (t − 1), what is happening now (t) and whatever one expects to happen (t + 1). He calls these mutually interfering dimensions recursion, incursion and hyperincursion. The relationship and interaction between recursion, incursion and hyperincursion can be explored mathematically using Lotka-Volterra equations for predicting the dynamics of predator/prey populations. These interactions produce fractal patterns.

The technique can be compared to other forms of interference pattern known from physics. For example, holograms are fractals resulting from an interference pattern produced on a photographic plate, which reflects the interaction between a laser shining on an object, and a reference beam. This means that 3D physical dimensions of an object can be inferred from 2D photographic data. Similarly, in modern approaches to machine learning such as convolutional neural networks, interference patterns are encoded in a machine learning model as the result of performing the same mathematical operations at different orders of scale of a collection of images of an object to be recognized. This enables the machine learning model to anticipate (predict) the likely categories of data it has not seen before.

One advantage of both these fractal techniques is that the units of analysis emerge naturally as a repeating pattern. This addresses a critical problem in the analysis of natural phenomena like sound of how to segment data for analysis: fractals define their segments by repetition. For our analysis of entropies, the fact that sound is wrapped by silence means that repeated patterns are wrapped by moments of zero entropy.

Silence has zero entropy because nothing changes. Zero entropy can also be produced through repetition, or what Shannon and Weaver (1949) calls redundancy, because in Shannon’s equation, if the probability of something is 1, and \(\mathrm{log}\left(1\right)=0\), then \(H=0\). While redundancy can be produced through the repetition of a signal over time, it can also be produced through the addition of alternative representations of a feature which exists in sympathy with an original feature but adds no essential new information (for example, the same lines spoken by a different actor, or a melody played by a different instrument). In music, redundancy can result from the harmonization of one melody with another, or the addition of backing chords to a melody.

The generation of redundancy in sound keeps the dynamics of entropy in different dimensions moving. While one dimension may have high Shannon entropy where other dimensions may have low or zero entropy, the relative entropy remains high. Once all dimensions and all relative dimensions become zero, there is silence and things stop. Therefore, we might suggest that it is the generation of pattern (redundancy) which is the driving force behind sound production. Since communicating agents may have initially little understanding of each other, the production of pattern by one (through sound) is a way of signalling the mechanisms of their physiology, which can be read and responded to by others. If communication is successful, then our understanding of the constraints of communication in one another arises through the production of mutual redundancy: each individual understands the patterns of the other.

To recap, we are suggesting that the physical kinetic energy of sound (vibrating molecules) exists in a circular relationship with physiological processes which are generated through the transduction of those physical processes, producing expectations which can be considered as potential energy to produce new sound. If energy is conserved in the universe, the totality of these processes must be zero. If this is the case, then processes of learning can be seen as processes which convert physical energy into physiological potential to make sound and generate new patterns. Each individual person, however, is not a totality, and so the tendency towards a totality of zero may be seen as a driving force which continually pushes the generation of new sound in learning processes. The hallmark of learning and of increasing knowledge is increasing discrimination in the way that sounds are expressed, with deeper anticipation of their likely effects. A methodological question, however, concerns whether it is possible, through analysing sound, to identify evidence for how sound dynamics relate to the deeper workings of the physiological system in learning.

Information, Meaning, and Sound

If sound is seen as a form of information entropy, physiological dynamics create a second-order anticipatory mechanism whose actions in making further sound are reflected in subsequent measurable events. As Leydesdorff (2021) has suggested, a second-order mechanism can be inferred as a meaning-construction process that steers the ongoing production of information. Information can be measured, suggesting that approaches to identifying fractals in sound can be used to speculate on what might be happening within bodies.

In this inquiry, we are concerned with the relationship between the patterning of sound and human experience. Phenomenological experiences within a sonic context such as excitement, boredom, curiosity, depression, joy and transcendence are — according to our hypothesis — the result of a physiological response which participates in the conservation of the totality of energy. The physiological energy of cell communication and organization, endocrine, vascular, nervous, respiratory and musculoskeletal systems are, we suggest, in a seesaw relationship with the physics of sound in the local environment, such that we might sketch their relationship as a product equal to zero:

$$\left(physiology\right)\left(sonic\;environment\right)\approx 0$$

There are two things to say about this relation. Firstly, it is a relation grounded in energy: both physiology and the sonic environment are manifestations of energy. Secondly, as the product of two entities, a simple mathematical objection would suggest that products cannot equal zero. However, within the matrix multiplication techniques used extensively in quantum mechanics, it is possible to have products that make zero. In recent years, there have been many suggestions for simplifying this matrix algebra using Hamilton’s quaternions which are 3-dimensional complex numbers, alongside what is known as ‘geometric algebra’, or Clifford Algebra. One of the principal exponents of this ‘geometric algebraic’ approach to quantum mechanics is Rowlands (2015).

Rowlands’s physics expresses the relationship between what quantum physicists call ‘local’ and ‘nonlocal’ phenomena as producing a zero product, or a nilpotent. He defends this as not only being an extension of Newton’s third law (‘every action has an equal and opposite reaction’), but also highlighting how this approach is consistent with Einstein’s energy equation. It is this relationship to Einstein which is particularly relevant to our inquiry into sound energy.

Rowlands takes Einstein’s mass-energy equation in the form derived by Dirac:

$${E}^{2}-{m}_{0}{\rho }^{2}-{c}^{2}=0^{1}$$

explaining how, since everything in the universe must obey the same laws about energy, this must underpin both local and non-local phenomena. Therefore, if

$$\left(local\right)\left(non-local\right)=0$$

then

$$\left({E}^{2}-{m}_{0}{\rho }^{2}-{c}^{2}\right)\left({E}^{2}-{m}_{0}{\rho }^{2}-{c}^{2}\right)=0$$

Since we are suggesting that physiology and the sonic environment are also in a relation that produces a zero product, it is possible to envisage a relationship between the energy equations used by Rowlands and the measurable sound phenomena of the sonic environment with physiological processes.

In expanding the quaternion algebra from his equations, Rowlands presents a practical way in which the connection between them can be realized as a fractal process. This produces what he calls a ‘rewrite system’ where a number of different parameters are combined in all their combinations and compliments (i.e. A and not-A) to produce zero. For example, given parameters A, B and C, all the combinations are:

$$A,\cdot\neg A,\cdot B,\cdot\neg B,\cdot C\cdot,\neg AB\cdot, \neg AB\cdot,\cdot AC\cdot,\cdot\neg AC\cdot,\cdot BC\cdot,\cdot\neg BC\cdot,\cdot\neg ABC$$

where the combination of any element with its complement (A \(\times \neg\) A) is zero.

This combination of parameters can be compared to the shifts in entropy in Figs. 1–3, where a rise in entropy can be considered as A, and a fall is \(\neg\) A. Moreover, just as the sonic environment produces silence eventually, so too does the combination of parameters in Rowlands’s equation.

The analysis of the sonic environment through encoded patterns of the entropies of frequencies, rhythms, textures, etc. produces a fractal-like pattern of events to which, we suggest, physiology responds. This technique was pursued by one the present authors in the analysis of music, demonstrating how patterns of notes can also produce fractal like patterns (Johnson and Leydesdorff 2020). In the following section, we use a similar technique using frequencies as the fundamental data — which are measures of physical energy — similarly inquiring into the extent to which physiological and the emotional reactions to which physiology gives rise to, complement these physical patterns.

Mapping an Algebra to Sound and Physiology

In doing our analysis, we consider the dynamics of three different parameters: frequency, volume, and timbre. By implication, we can also see rhythm in terms of the regularities of fluctuations in these values. We have arranged these to interact following the combinatorial logic of Rowlands (2015) ‘rewrite system’. Information entropy (Shannon) equations are useful in mapping how the different parameters overlap to eventually produce silence. In addressing our question about the relationship between exterograms and engrams, if the observable parameters represent the exterogram, then the non-observable parameters representing the engram must result from adaptive processes such that the product of these is responsible both for the ongoing dynamics of sound and the ongoing processes of learning.

It is possible to visualize patterns of relative entropy in sound in a variety of different situations. The graph in Fig. 4 can be re-represented as a codification of values and relative values over time, where a 1 is used to indicate a rise in entropy and a 0 is used to indicate a fall in entropy. In the graphs shown in Figs. 5–8, columns are treated in pairs, so for each variable the first item is black if the entropy rises, and the second is black if it falls. Beyond the initial pairs of variables, A, B and C and their complements (\(\neg\) A, \(\neg\) B, \(\neg\) C), there are relative variables for AB, AC and BC, with their complements, together with the collective value of ABC and \(\neg\) ABC. This produces a set of 28 columns. There are a range of different patterns which are producible through the method we adopt, and each of these patterns has a different level of fractality.

Fig. 5
figure 5

Entropy of sound in an instructional YouTube channel for a game (from Fig. 3)

Full size image

Visualizing Sound Interventions

Returning to the sound examples that we began with (Figs. 1, 2 and 3), we consider that each of the sound extracts can be analysed for the entropy in the sound spectrum, and the results of these analyses can be compared to each situation. Examining the sonic environment produces a picture of the sonic exterogram. We see the kinetic energy produced by vibrating molecules, but we only see remotely the effects of those vibrations on physiological systems. While physiological effects can be determined through ongoing production of sound, these physiological processes are the result of the effects of the visible physical phenomena of sound.

We have suggested that there is a balance between the energy in the physical context and the physiological energy which transforms that energy into expectations and further utterances. We have further suggested that the formation of expectations can be imagined as a fractal process where the diachronic and synchronic aspects of the sound interfere with each other to produce patterns which are predictive of ongoing events.

Using the technique shown in Fig. 4, plotting the rise and fall of entropy invites deeper inquiry about the nature of the differences between contrasting educational situations. It is an approach which, whilst open to much refinement, focuses on the betweenness of phenomena: entropy is fundamentally a measure of surprisingness, and surprisingness depends on an observer who may express their surprise by making more sound. The graphs below (Figs. 5, 6 and 7) show the same 3 lines as in Fig. 4: ent1 is the entropy of richness of sound; ent2 is the entropy of loudness; and ent3 is the entropy of pitch.

Fig. 6
figure 6

Entropy of online lesson — teacher talking with little student response

Full size image
Fig. 7
figure 7

Entropy of children working in groups face-to-face (from Fig. 1)

Full size image

We introduce these results in terms of the most striking differences between the different contexts they represent. These are entropy graphs of the audio samples whose pitches are shown in Figs. 1, 2 and 3. Figures 5 and 6 relate to Figs. 1 and 2, respectively. Both samples are very short (lasting about 30 s) but are strikingly different in a number of respects. In Fig. 5, the entropy of the pitch of the voice is relatively static. This is because the flow of sound in the YouTube example (as shown in Fig. 3) is pretty constant: there is little extended silence in the delivery. Contrast this with the range of pitches, volume and richness of the sound in Fig. 6. This shows that the entropy of the richness of the sound (ent1) is less than that of the pitch (ent3), although it should be noted that the maximum entropy value in Fig. 6 is 3, as opposed to the maximum entropy value of 6 in Fig. 7. This is probably due to the poor sound quality of the zoom session, which introduces quite a lot of interference in the sound. As a result, there is a chaotic range of different values for the pitches, which contributes to the general lack of structure in this diagram. This lack of structure, we suggest, entails more physiological work in interpreting the meaning of what is said.

In the second set of comparisons, we see smaller differences between the two graphs. These show the sound of a group of children working together in a face-to-face environment on a project ( Fig. 7 from Fig. 1) and the sound of a teacher’s monologue explaining those activities ( Fig. 8 from Fig. 2). Despite the larger range of pitches in Fig. 1, the entropy of these pitches shown in Fig. 7 is relatively static. This is because entropy measures the change in variety, not variety itself. There are fluctuations in the richness and volume of the sound which generally move in similar directions — which are to be expected if occasionally a number of children are talking at once.

Fig. 8
figure 8

Entropy of teacher talking to face-to-face class (from Fig. 2)

Full size image

In the fractal images shown in Figs. 7 and 8, some of the underlying differences between these two cases can be shown. In Fig. 8, the most striking feature of the fractal is the ‘checkerboard’ patterns on the right of the diagram. This indicates that at successive moments, there is an oscillation in the relative entropy between the different parameters (for example, the entropy of pitch goes up, while the entropy of volume goes down, and then vice-versa). The corresponding graph shows that these are relatively small fluctuations however, and perhaps these fluctuations are a result of the acoustic of the classroom (the voice raises entropy, while the resonance of the space will subsequently lower it).

The opposite of the checkerboard pattern is the vertical bar which is a feature of Fig. 7 — this is where the entropy of one or many of the parameters increases or decreases together over successive time periods. This might occur because something is getting progressively louder or quieter, for example — such as when a key point is being emphasized, or there is an increasing level of excitement. These moments where entropy moves together in different dimensions are where the meaningfulness of the interactions is most explicit. This might be compared to situations in music where a long crescendo will lead to a climax, and then a falling-back to silence, or the kind of situations in the animal world where a commotion causes a flurry of activity and calling. Furthermore, in Fig. 7, the richness of the sound varies according to the number of people talking partly because this reflects the number of pitches which are heard. Comparing the graph of entropies against the graph of pitches (in Fig. 1), it is noticeable that as the range of pitches decreases (at around 6 s) so does the entropy of the richness of the sound and the entropy of the volume. In other words, the social dynamics has a direct impact on the sonic dynamics which in turn informs the patterning from which meaning is inferred.

Conclusion: A Sharper Lens

If we are to measure relations in learning, we require new models of the dynamics that underpin relational phenomena and new techniques to analyse them together. The dynamics of the sonic landscape present a compelling example of the kind of relational modelling that might be possible. Not only can the dynamics of sound reveal the manifestations of individual behaviour in making sound, but it reveals the impact of the context within which any sound-making occurs. Further to this is the question as to what connects the external manifestation of sound with the internal expectation and potential for further sound. We have suggested an ecological energy-based perspective and used Shannon entropy equations as a way of considering information as variety which reflects the physiological organization of participants. At the heart of this approach is the idea that whatever the sonic environment presents to physiology in terms of differences, physiology reacts in ways where expectations, constructed through the reflexivity on preceding patterns, creates the potential for making new sound.

The focus on energy which we have introduced, alongside the information-theoretical analysis, has potential for further application. The advantage of entropy as a measure is that it can situate any variation in phenomena along commensurable scales. As we have suggested, it is not just entropy of the three variables of sound we have considered here, but additionally we could include the entropy of any words spoken, alongside any visual phenomena which are experienced in the communication situation. For example, in the process of Bruner's (2006) ‘management of joint attention’ between mother and child, we suggest that not only the prosody has entropy, but so do the physical movements, the facial expressions, eye movements and other gestures. It is not only the vibration of molecules in the air which contributes to the overall energy equation, but the frequencies of light, the physical conditions of the environment and the epigenetic effects of chemicals and hormones in physical interaction with one another. Our presentation of the sonic environment of learning presents one way of pursuing these inquiries.

We have suggested that if the Gestalt-switch of the postdigital is to carry scientific benefit that contributes to the improvement of education, then this must relate to the study of relations between observing systems. This means that we must somehow understand the dynamics of the in-between-ness of learning. Energy, we suggest, is a way of talking about this in-between-ness. Looking to physics and its import on information in human observer-relations provides a way in which deeper insights might be gained into the different settings within which energetic processes lead to learning. Such an approach is both analytical and generative. A sharper lens facilitates the gaining of new insights and the construction of new interventions.