Coming to See Action as Symbol: the Computer as Collaborator

Our work with young children began as a project and a place that we called The Laboratory for Making Things. Our hypothesis was that deep learning could accrue in an environment where projects were designed that used differing kinds of objects/materials, that utilized differing sensory modalities, that held the potential for differing modes of description, but that shared conceptual underpinnings. This article focuses on the work of one eight-year-child, whom I call Laf, whose most notable quality was integrity – he needed to understand for himself. I trace Laf’s work as an example of a response to the question I had put to myself: Could the computer be a collaborator in helping children effectively make moves between their own body actions in clapping and the necessary numerical–symbolic instructions to make the computer drums play what they had clapped?

and create a demand for fresh knowledge. […] noting the bearing and function of things acquired [...] has the advantage of being of the kind followed in study and learning outside of school walls where data and principles do not offer themselves in isolated segments with labels already affixed. (1911/1974, p. 423) Learning from the children, our goal became to provide an environment in which making occurred in a variety of media (LEGOs, geometric blocks, huge cardboard gears, pulleys, foam core houses, drums for playing rhythms), along with using Apple computers as a platform for construction (graphics, music, quiz programs, puzzles). We would design projects that differed in the kinds of objects/materials used, that utilized differing sensory modalities, that held the potential for differing modes of description, but that shared conceptual underpinnings (Bamberger 1991).
In the Lab, the computer played a role of a collaborator. Like the hands-on materials, the computer was used as another medium for designing and building working systems. But there were striking differences in use and function between these two media. For example, in the hands-on situation, children begin with action and rarely, if at all, made written descriptions. In the computer situation, the maker must begin with a description and the description becomes what the maker has describedsymbol becomes digitized object/action! We urged the children to pay attention to differences in the kinds of things that inhabited the two worlds. How did the differences between these design worlds influence what they thought of to think about; what was different in the kinds of problems, confusions, and puzzles they encountered as they moved from familiar hands-on action in real time/space to the virtual, symbolic world of the computer?
For example, the computer experiments had a special quality. Because descriptions actually become (virtual) actions, the move from symbol to action could be tested. Indeed, chasing surprises, tracing the paths that led from symbol to action, turned out to be a productive way for the children to explore their own understandings and confusions. Interrogating their confusions was often the critical and exciting step towards insight: Strange encounters of a special kind (Fig. 1).
Gears played an important role in the Lab as a means for helping children both see and feel shared principles across media and sensory modalitiese.g. kinds of fast and slow, a counting unit, periodicity. As Papert has pointed out: As well as connecting with the formal knowledge of mathematics, it [the gear] also connects with the "body knowledge", the sensorimotor schemata of a child. You can be the gear, you can understand how it turns by projecting yourself into its place and turning with it. It is this double relationshipboth abstract and sensorythat gives the gear the power to carry powerful mathematics into the mind.
[…] the gear acts here as a transitional object. (1980, p. viii; italics in original) My hunch was that moving between clapping rhythms and playing with gears could be a particularly lively playground for making this 'double relationship' manifest. While I have explored this hunch already (Bamberger 2016), this article is the result of a new focus and an emergent re-thinking of one particular student's actions and collaborations with the computer.

A Day in the Life of the Lab
On this particular Wednesday afternoon, six fourth-grade children moved through several eventsfrom drumming, to playing with the very large cardboard gears, to clapping, and eventually to 'telling' the computer how to 'play' drum patterns using MusicLogo. 1 At one moment during the afternoon activity, while playing with the gears, Rachel had noticed that the small gear went around four times when the big gear went around once (see Fig. 2). Arthur Ganson, who was also working with the children that day, saw an opportunity 2 : 1 MusicLogo is a version of Logo that includes music primitives. 2 Arthur Ganson, a kinetic sculptor, designed both the materials and the tools with which children had built the cardboard gears. Leah's two hands in clapping, acted out the relative rotations of the two gears. Each tap of her left hand was the bigger gear going around once; each group of four taps of her right hand was the smaller gear going around four times (see Fig. 3). The actions of the gears were informing the actions of clapping (Fig. 4).
My question to myself at this moment was: Could the computer be a collaborator in helping the children effectively make moves between the gears, their own body actions in clapping and the necessary numerical-symbolic instructions to make the computer drums play what Leah had clapped? This would be a prime example of learning by confronting the tension between continuous action in clapping and discrete static symbols. I asked the children: Jeanne: Can you get the computer synthesizer drums to play what Leah clapped? Except, to begin with, we'll make it a little easier.
Using two hands, as Leah had, I tapped out a simpler 2 to 1 rhythm (see Fig. 5). The children all clapped back the two-layered rhythm: The children were already somewhat familiar with using the computer language Logo, along with various meanings for numbers in doing Logo graphics. Now the children would need to give new meaning to numbers in this MusicLogo context. What links would emerge between the actions and sounds the children made in clapping, the numbers used in doing ordinary arithmetic, the numbers used in doing Logo graphics,  and now numbers used in MusicLogo as instructions to the computer to make temporal relations of drumbeats?
While the children were used to having conversations with one another or with an adult, when they would try to explain what they were trying to do, they got stuck. However, descriptions of such real-time building did not usually include symbolic/ numerical expressions. Thus, what the children were familiar with would now be reversed: Instead of reflecting back to make descriptions after the fact and after the act, they would need to describe, as instructions to the computer, what they wanted to happen before the act. And instructions would have to be in the symbolic form of MusicLogo. These were some of the issues as we moved to the computers and to the new task.

Laf Invents an Experiment
MusicLogo could make the computer sound-synthesizer play two different percussion sounds called BOOM and PING. To help the children get started, I typed the following instructions and we listened (PM is the command for PlayMusic):

BOOM [8 8 8 8 8 8 8] PM
We heard seven BOOM sounds each with a duration of '8'. However, at this point, the children still had to discover what '8' meant. I gave another example, saying, "This one will go faster":  I also showed the children how MusicLogo could make the computer play BOOM and PING together, each in a separate 'voice' (see Fig. 6).

From Action to Symbol
As the children went to work on their computers, I went around to work with Laf, who was a quintessential example of a child for whom to 'grasp an idea' was literally a physical experience. All of us seek ways of holding on to a new idea. But, for most children, especially those growing up living in an unstable, unpredictable world, grasping, holding still, is a persistent need. Laf's explorations made that quite clear. And, like probably so many times before, I almost missed it.
Laf talked very little; it was in the Lab that we discovered his most notable quality: integrity. Unlike more school-smart children, he would just turn off rather than going through the motions to get a right answer. He needed to understand for himself. However, on this occasion, as on so many others in watching a child work, I learned that we adults often need to slow down in order to catch up with another's thinking. We are so often fixated on instilling our conceptual goals that we tend to miss the significant knowledge that the student is already bringing to the situation. Instead of BOOM PING

Fig. 6 PING and BOOM together
Digital Experiences in Mathematics Education (2020) 6:233-243 rushing on to get the child to our teaching goals, we need to 'slow down' to search for and value the child's own sense-making.
Sitting down next to Laf, I saw that he was slowly, determinedly, filling up the whole screen typing BOOM or PING followed by rows of 1 s or 2 s (see Fig. 7). Despite my best intentions I found it difficult to find reason in what Laf was doing. Only later, looking at the video of the whole session, did I realize how wrong I was. Because I was focusing on my taskto make two levels of beats in a 2 to 1 (2:1) relationship, I missed the significance of Laf's work.
True to his integrity, Laf had designed an organized experiment to answer questions he had silently put to himself. What is the difference between 1 and 2 in this context and between BOOM and PING? What exactly do these numbers do? And how can I find out? Listening, participating in his own experiment, he confirmed his previous understanding. Laf's numbers were turning into sounding actionsthe very fast 1 s and the slower 2 s were eminently hearable. Listening again to his typed numbers, this time he followed the numbers with his finger. Starting with what he knew already ("The lower you get the faster it gets. […] If you put all 1s it'll go fast"), he tested his hunch.
Sweeping through the faster 1 s as they (virtually) went by, Laf said, "Where am I?" And, as if finding his way, he carefully 'finger-drummed' in time along with the slower 2 s (see Fig. 8). The numbers literally stand still in screen space, as Laf marked the drum beats passing by and sounding out. A perfect example of action meeting symbol. Like the scientist who must gather enough data to make sense of their meaning, Laf collected multiple data and, together with his available resources, did the work of making meaning. MusicLogo became a collaborator in Laf's actions in real-time/space and the virtual world of symbols. In retrospect, Laf also taught me how important it is to be able to slow down and take the time to repeat so as literally to practice grasping meaning.

Going on
But there was more. Building on his experiment, Laf now used what he had learned, to make a whole piece of music. Not surprisingly, he named his MusicLogo procedure TO LAF. He would teach the computer TO LAF.
Laf's procedure filled the Lab as the children listened attentively all the way to the end. The usually quiet Laf looked triumphant and the children clapped in appreciation. Notice that using just his 1's and 2's (see Fig. 9), Laf had created organized and repeated rhythm patterns. What initially seemed random experimenting turned out to be successfully making meaninga joining of symbol, action, time and coherent sound (Fig. 10).

Moving Farther Towards Symbolic Representation
The biggest surprises came when the children returned to the Lab the next week. Clearly feeling comfortable enough with the meaning of 1 s and 2 s, and perhaps learning from what he had seen and heard the other children doing, Laf went on to make the two-voiced experiment that I had proposed originally. Beginning again with the familiar 1 s and 2 s, (BOOM 1 in Voice 1 and PING 2 in Voice 2), he surprisingly moved on to make a 4:8 relationship as well (see Fig. 11).
And most remarkably, by the end of the session, Laf made more of what he had learned to build a whole table of 2:1 drumming examples (see Fig. 12).
Laf's first experiment, which had seemed at the time a rather mindless activity, turned out to have been a playground for gathering, grasping, mulling over and, finally, developing and giving meaning to a whole symbolic table of shared 2:1 drumming ratios. Importantly, as evidence for the importance, Laf gave to time passing, notice that the PINGS, which go twice as fast as the BOOMS, also have twice as many iterations as the BOOMS. Thus, each pair of BOOMS and PINGS will come out togetherequal in total time. The computer has literally become a collaborator helping to give meaning to symbol, sound and action.
But wait! On the basis of this table, it is easy to assume that in building on and generalizing upon his experiments with 1 s and 2 s, Laf had developed the powerful idea of ratio. But notice the difference between my use of the conventional form of ratio description, 8:4, 6:3 ... and Laf's depiction of ratio by using the number of numbers. Put most simply, I have imposed my representation and attributed it to Laf. And in that process, I have taken away time and action. The ratio representation, e.g. 6:3 or 10:5, obliterates motion and time. Collapsed into a single symbolic representationmany events become one (see Fig. 13). And, coming full circle, Laf's finger drumming following the computer's 'performance' brought the static, discrete symbols back into action.
The table turns out to be a snapshot in an evolutionary process of learning, an onthe-way 'abstracting process'. Continuous actions and time are extracted, made discrete, held still, gradually transforming into the symbolic expressions that we teach in school.
Abstraction can be defined as a mental activity by means of which parts of a unit are detached from the whole and separate qualitiescolor, form, etc.are experienced in isolation. (Werner 1926(Werner /1957

Summing up
To recapitulate, the example of Laf at work illustrates how one child, with the computer as collaborator, mobilized the potentially generative tension between action and symbol to develop new and powerful insight. The example is perhaps most relevant for children whose personal, powerful know-how is failing them in school, largely because it has no way of coming in off the street into the classroom. These are the children who often feel that school is irrelevant and, reciprocally, are made to perceive themselves as irrelevant, as peripheral in school settings.
In sum, I argue that the computer may play a special role as a resource for inquiry and invention when children can work at a pace and within a conceptual space that they can grasp and that, thus, feels secure. In this environment, instead of being consumers of other people's ideas and products, children can potentially become makers of their own new knowledge of which they can feel proud. And, through this empowerment, they may also discover strategies for learning how to learn within the school world and beyond.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.