From turtles to Tangible Programming Bricks: explorations in physical language design
- First Online:
- Cite this article as:
- McNerney, T.S. Pers Ubiquit Comput (2004) 8: 326. doi:10.1007/s00779-004-0295-6
- 840 Downloads
This article provides a historical overview of educational computing research at MIT from the mid-1960s to the present day, focusing on physical interfaces. It discusses some of the results of this research: electronic toys that help children develop advanced modes of thinking through free-form play. In this historical context, the article then describes and discusses the author’s own research into tangible programming, culminating in the development of the Tangible Programming Bricks system—a platform for creating microworlds for children to explore computation and scientific thinking.
KeywordsProgramming languages Microworlds Tangible user interfaces Education Children History of computing Construction toys Hands-on learning
Over 35 years have passed since Seymour Papert co-founded the MIT Artificial Intelligence Laboratory with Marvin Minsky. However, the results of Papert’s research continue to provide inspiration for researchers in educational technology even today, at the dawn of the third millennium. Papert’s groundbreaking insights into how computers can help children to learn actively and to create knowledge have firmly withstood the test of time.
This article provides a historical overview of educational computing research at MIT, from Papert’s pioneering work in the mid-1960s to the present day, focusing on physical human–computer interfaces. It discusses some of the laboratory prototypes as well as commercial fruits of this research: electronic toys that help children develop advanced modes of thinking through free-form play. In the context of this historical perspective, the article then describes and discusses the author’s own research into tangible programming. His work culminated in the development of the Tangible Programming Bricks system—a platform that was used to build the Digital Construction Set. This is a physical microworld that children can use to understand the logic of digital devices and explore scientific thinking. Issues of language design that make such digital manipulatives  easy to learn and use as powerful and concise digital building blocks are also discussed.
Particular attention is paid to the continuation of Papert-style educational computing research through the efforts of his students at the MIT Media Lab. Their work is bridging the gap between abstract computation and the learning abilities of children by bringing computer programming into the physical world as a creative activity. This research on programmable bricks and tangible user interfaces has led directly to innovative commercial products. Many of these products were manufactured and marketed by the LEGO Group—an ongoing supporter, partner, and beneficiary of MIT’s educational computing research for more than two decades. During his tenure in the Learning and Epistemology group at the MIT Media Lab, Papert was appointed the LEGO Professor of Media Arts and Sciences. As of this writing, Papert’s student, Mitchel Resnick, now heads the “Lifelong Kindergarten” group at the Media Lab, and currently holds the LEGO Chair.
2 Historical background
2.1 Logo: a programming language for children
Logo is a variant of the LISP programming language, popular among artificial intelligence researchers. It is a powerful yet simple language for exploring linguistic and mathematical problem-solving. Logo is relatively free of the syntactic rules that make so many programming languages difficult for children to learn. At the same time, Logo shares much of LISP’s power to create new, domain-specific languages. These properties of Logo led to the slogan: “low threshold and no ceiling.”
The Logo floor turtle, so-called because it lived on the floor of the lab, could be fitted with a pen and be used to draw pictures on a large sheet of paper placed on the floor. Children learned to “teach” the turtle to draw basic shapes, such as triangles, squares, circles, and even letters. The turtle could also be instructed to draw complex Spirograph1-like shapes by repeatedly drawing simpler shapes, rotating the turtle slightly before each repetition.
2.2 The Logo turtle makes geometry physical
2.3 Programming by example
Over years of teaching Logo, it became clear to researchers that most children were not ready to start programming computers in the traditional manner (i.e., typing Logo code into a computer using a keyboard) until at least 10–14 years of age. Radia Perlman, a graduate student at the MIT Logo Lab in the mid-1970s, believed that the major impediments to children’s access to computer programming were not only the language syntax, but also the user interface. Perlman  proceeded to design interfaces that would allow even preschoolers to learn to program a turtle. She came up with two novel input devices, informally called the Button Box and the Slot Machine.
The Button Box was her first creation. One prototype was a 50×50×10 cm transparent, plastic box with four groups of buttons:
The Number Box: 1–10
The Action Box: FORWARD, BACKWARD, RIGHT (turn clockwise), LEFT (turn counter-clockwise), PENUP, PENDOWN, plus “turn your light on,” “turn your light off,” and “toot your horn” (the clear favorite among the three- and four-year-old test subjects)
The Memory Box: “Start remembering,” “Do it,” and “Forget it”
The Four Procedure Box: YELLOW, RED, GREEN, and BLUE
The Memory Box innovation made the Button Box perhaps one of the earliest illustrations of programming by example . Rather than creating a list of commands for a computer to perform, which requires a child to translate concrete actions into abstract directives, the child could physically show the computer what to do and have it memorize the sequence for later playback.
2.4 Playing the Slot Machine
Perlman soon discovered two major problems with her Button Box system. First, the concept of procedure calling, as implemented by the Button Box’s “Four Procedures Box,” was too abstract for younger children. Second, the system provided no way for a child to modify a program once it was recorded. If the child found a mistake, he had to re-record the entire sequence from the beginning. This last issue did, however, mean that children were not presented with the somewhat complex task of editing “buggy” programs.
2.5 The turtle goes to school
The Logo programming language acquired unprecedented popularity in the early 1980s when personal computers (particularly the Apple II) became common in schools. However, few robotic turtles made it to school—although fine for pioneering laboratory research, the robots proved expensive, unreliable, and unsuitable for exploring complex geometric patterns. Educational computing researchers at MIT temporarily moved away from this approach, and instead developed the screen turtle, a representation of the floor turtle on the computer’s video display. The screen turtle became the standard for Logo-style computational geometry. The notion of physically rehearsing geometric ideas was certainly not abandoned, but the practice of using the floor turtle as a physical thing to think with lay dormant for a number of years.
Though the screen turtle was obviously more practical, it added a layer of abstraction that may have caused difficulty for some children, because the screen turtle was removed from the floor, and thus removed from the child’s everyday environment. Also, the distinction between turning left and turning right was less intuitive, since the turtle was on a vertical display screen, whereas floors are horizontal. Nevertheless, the virtual nature of the screen turtle offered new opportunities for learning and exploration that were hard to resist.
2.6 Connecting with the physical world: LEGO and Logo
It was not until the 1990s that educational computing researchers at MIT returned in earnest to studying interaction with the physical world. The LEGO TC Logo2 interface developed by Steve Ocko allowed young inventors to write Logo programs on a personal computer to control devices built out of LEGO building blocks. Even though these contraptions were tied by an “umbilical cord” to a computer, they could perform simple yet amazing real-world tasks, for example, sorting LEGO pieces by size.
2.7 Toys with “personality”
While code bricks like “gas pump” definitely suggest specific interaction, the main “personality” bricks in the MyBot system do not impose any explicit scripting on the child’s play. These bricks simply make non-imposing sound effects, variations on the timeless “vroom” (car) and “neeaw” (airplane) sounds.
2.8 P-Bricks “go mini”: the cricket and its progeny
2.9 Tangible Mozart
The musical phrases in the Music Blocks toy were carefully chosen so that the music sounds “right,” regardless of the order in which the phrase blocks are arranged. The child can create new, entirely plausible variations, or try to re-create the Mozart original. Adding to the fun, the different faces of the cubical blocks play different arrangements of the music, from “doo wop” a capella to classical orchestral.
2.10 Tangible programming with trains
In informal experiments, the author noticed that the concept of preparing train tracks with instructions to be executed later by the Intelli-train was initially not intuitive to four-year-olds. Shown how the code bricks affect the train’s behavior, their first instinct was to place the code bricks directly in front of the approaching train to achieve immediate results. It is too bad that this approach mostly results in derailments, because providing a mechanism for “interactive” program execution would make a good transitional learning activity: immediacy first, delayed “programming” later.
2.11 The AlgoBlock system: a tangible video game
2.12 Tangible programming bricks
Easy to understand
Conducive to free-form play
Small vocabulary–lexicon of building blocks
Large number of legal programs
Debuggable without external tools
Power and ground
I2C serial bus
Buzzer or capacitive touch sensor
Serial Cricket bus
Analog sensor input.
A brick can communicate with its neighbors. It can send messages containing events or data to its neighboring brick above, and it can receive (and answer) messages from its neighbor below. This asymmetry leads to a natural upward data flow. Because each brick can only communicate with its immediate neighbors, passing data from the bottom brick to the top brick in a stack requires the cooperation of all the bricks in between to form a “bucket brigade.”
2.12.2 Early experiments
Early experiments with the Tangible Programming Bricks focused on programming external devices, like toy cars and kitchen appliances, using a sequential, procedural language. The building blocks shown in Figs. 11 and 12 were from an early experimental language aimed at the end-user programming of microwave ovens.
2.13 Functional programming is well suited to physical interfaces
In the spring of 2000, a more successful tangible programming language emerged. Previously, the author had been working in much the same direction as his predecessors: researching physical media for imperative programming. The author began exploring the physical expression of another programming style, functional programming.
An imperative program describes an algorithm, or “how to” do something. When a child programs a Logo turtle or Intelli-train, he sets up a sequence of commands (a.k.a. imperatives) that the device executes one-at-a-time, in the order that it encounters them. A functional program denotes “what is” being computed. The most familiar example of a functional programming system (although unbeknownst to many of its users) is an electronic spreadsheet program. Each cell of a spreadsheet describes a piece of the computation. The order of evaluation is not explicitly specified; rather, it is derived from the relationships between the cells. For instance, if cell B2 depends on the value of A4, then A4 will be computed before B2.
Digital Construction Set for Bricks and Cards (refers to Fig. 13)
This provides a computationally passive foundation for building a stack of bricks (the unlabeled brick in top right of figure)
Generates a continuous stream of samples from analog SENSOR, KNOB, THUMBWHEEL, or CONSTANT cards
Generates a stream of discrete events triggered when the analog sensor data rises above a threshold (no event is generated when sensor data falls)
Multiplies input data stream by a value supplied by card
Subtracts from input data stream a value supplied by card
Tallies events (ignores data associated with each event)
Counts the number of events in a specific period of time (e.g., 10 s)
Displays, in succession, each element of input stream
Clicks whenever it receives a new data element (a.k.a. event)
Used either as a push-button, or as a symbolic or numeric parameter
Used to enter a variable input parameter
Gives the CLICK brick its “voice,” or can be used for other sound, music, or noise-making
Allows the user to download Logo code into a brick to give it its own distinct behavior; can be used to communicate with Crickets, other stacks of bricks, or peripherals (e.g., display)
Used for entering numbers that are not available in the limited set of CONSTANT cards (seen showing “0259” in figure)
Used to sample analog data from sensors (e.g., to measure light, temperature, magnetism, etc.)
The message-passing architecture of the Digital Construction Set uses a variation of the “bucket brigade” concept described above. Here, a brick is allowed to modify each element of the data stream as it passes it up the chain. For example, a MULTIPLY brick with a “2” card takes each number received from its lower neighbor, multiplies it by two and passes the result up to its upper neighbor. So, if the brick receives a 21 from below, a 42 is passed up the chain.
2.14 Building physical microworlds
The Digital Construction Set was designed for children to explore the logic of everyday electronic devices through hands-on learning. In his seminal book, Mindstorms , Papert introduces the concept of a microworld, a deliberately simplified computational environment that allows students to explore ideas that are not normally demonstrable in an average classroom setting. Papert describes how microworlds can be used to learn aspects of Newtonian mechanics that students never experience in their everyday lives, such as the motion of objects in a vacuum. Children, programming a Dynaturtle (dynamic turtle, a.k.a. sprite) in Logo, can design experiments not only to model traditional Newtonian mechanics, but also alternate laws of motion (e.g., Aristotelian), or even fictional universes whose natural laws are defined entirely by the student. Papert’s theory is that, when a child creates a simulation by herself, she has “made it her own,” it becomes personal, and thus, more powerful.
2.14.1 “Getting under the hood”
Fifty years ago, home appliances and scientific instruments were made from individual components and invited amateur disassembly, experimentation, and learning—but the microprocessor changed all that. Even if you did open the case of a modern gadget, there is almost nothing to see that can be used to understand its inner workings. The National Science Foundation funded the Beyond Black Boxes initiative  to explore the theory that children more readily learn scientific concepts by building their own scientific equipment.
2.15 Bricks at work: a bicycle trip computer
The Digital Construction Set provides all the building blocks required to make a simple bicycle trip computer. In the simplified example shown here, the Hall-effect sensor on the toy bicycle is wired to the TRIGGER brick. Every time the bicycle wheel goes around, the magnet on the wheel rim moves past the sensor. When TRIGGER detects this, it sends a message representing an event up to CLICK, which beeps once per message, and forwards the message to COUNTS/T. This brick tallies messages within a 10-s interval and passes the total count to DISPLAY. In the full example, MULTIPLY would be used to compute velocity (e.g., MPH or KPH).
A second DISPLAY (shown in the lower right of Fig. 14) is connected directly to COUNTS/T. It is used to view the state of its internal counter. This was added because, in early versions of the Digital Construction Set, the counter’s internal state was invisible during the 10 s between clock ticks. This made it difficult to explain the operation of COUNTS/T to children. The lesson here: “Avoid hidden states. Make everything visible.”
2.16 Features of the Digital Construction Set
A valuable aspect of the Digital Construction Set is that it naturally encourages the user to start simple and to add complexity on top of already working structures. As the system becomes more complex, users spend more and more time debugging—figuring out why things do not work as expected. For this, it is important to be able to see what is going on inside the computation .
The Digital Construction Set offers the ability to debug itself. The CLICK and DISPLAY bricks can be used to examine intermediate results in a stack of bricks, as well as to understand the internal message-passing architecture. These special bricks do not have any computational effect on the values passed up the stack. For example, the CLICK brick can be used to learn the difference between TRIGGER and MEASURE. When CLICK is placed above MEASURE, CLICK beeps rapidly. Put it above TRIGGER, and it beeps only after high–low transitions of the input data. Before an external sensor is connected, TRIGGER can be tested in isolation by using the KNOB card. Each time the knob is turned clockwise, CLICK beeps once. When the knob is turned counter-clockwise, no beep is heard.
This self-debugging feature turned out to be very useful when the author was first programming the bricks that make up the Digital Construction Set. Later, when the KNOB and THUMBWHEEL cards were added, the CLICK brick was used to listen for bugs in the internal message-passing system. Children can debug their creations in the same manner.
2.16.2 Consistency and simplicity
A fundamental design tenet of the Digital Construction Set is simplicity through consistency. In particular, the CONSTANT, KNOB, SENSOR, and THUMBWHEEL cards can be used interchangeably. For example, if a child wants to understand how the THUMBWHEEL works, she can simply insert it in the MEASURE brick and attach a DISPLAY brick above it. It becomes a game to manipulate the digits of the THUMBWHEEL and watch the DISPLAY change. The design of the Digital Construction Set makes it easy for users to understand the behavior of individual building blocks and small assemblies before moving to bigger projects.
Perhaps the most powerful feature of the Digital Construction Set system is its ability to allow researchers and even young programmers to “get under the hood” of the bricks themselves and reprogram the behavior of any brick. In Papert’s spirit of technology that children can “make their own,” children are offered the opportunity to design their very own physical microworlds by inventing new, domain-specific vocabulary—a new set of building blocks suited to a particular task.
2.17 Why not allow different topologies?
Many constructive tangible user interfaces (e.g., [21, 22]) use computational building blocks to specify a wide range of geometric and topological structures. However, the Tangible Programming Bricks only stack in one direction—up. One cannot build 2-D or branching structures using the Digital Construction Set. The author deliberately chose this approach to avoid adding layers of complexity. Two of the main goals of this research were ease of explanation and ease of learning. Although the linear stacking constraint does limit expression, the overall simplicity of the Digital Construction Set is a clear benefit for the novice. Simplicity is an important feature of a Papert-style microworld.
2.18 Evaluating the Digital Construction Set
The Tangible Programming Bricks were demonstrated to hundreds of adults in the course of testing the suitability of various types of physical programming languages. Compared to the prototype languages based on an imperative programming model, the author found the streams-based programming model of the Digital Construction Set to be dramatically easier to explain to adults with both technical and non-technical backgrounds.
A few users were confused by the physical similarity of the connector systems used for stacking bricks and inserting cards. This was because the same gold “smart card” pattern appeared on both the bricks and the cards. Also, there was nothing to prevent cards from being inserted upside down. This would be easy to fix in a commercial implementation.
Only limited testing was conducted with children. To evaluate the Digital Construction Set, the author arranged informal user sessions with four children ranging from 6 to 13 years of age. A session consisted of a 10-min demonstration of the Digital Construction Set, followed by 10–20 min of free-form play and experimentation. An 11-year-old girl and a 13-year-old boy were successful at solving two problems posed to them with little prompting. The girl also launched into an impromptu activity using the bricks and the bicycle to measure distance along a yardstick. The two younger children did not really “take” to the activity. The author speculates that a less scientifically advanced microworld would have been more appropriate for younger children.
In fact, subsequent research studies by Wyeth and Purchase into the usability of tangible programming systems found that preschoolers can create simple programs and benefit from the physical immediacy of stackable, electronic blocks . The blocks of Wyeth and Purchase were much simpler than the Digital Construction Set. They employed on/off sensors, lights, and Boolean logic functions similar to the commercial toy, LogiBlocs.
2.19 Other related research
The research described in this paper is within an area situated squarely between visual programming languages [3, 24, 25], direct manipulation , tangible user interfaces [14, 27, 28, 29, 30, 31], and end-user programming . It focuses specifically on children as programmers. In addition to the programming systems described in this document, a number of other systems have been designed for children: derivatives of Logo (e.g., StarLogo ), ToonTalk , Cocoa (a.k.a. KidSim) , and Agentsheets [35, 36], just to name a few. Martin’s Braitenberg Bricks  demonstrated a physical embodiment of Braitenberg’s behavior language described in .
The monumental work of Seymour Papert and his students both at MIT and elsewhere have provided a rich body of research into the field of educational programming and tangible user interfaces. The author has used this work as a foundation for the development of the Digital Construction Set, exploring the feasibility of expressing functional programming concepts in the physical world. The Digital Construction Set allows teachers and students alike to build physical microworlds that children can use to model the behavior of digital devices and to conduct scientific experiments involving sensing and computation directly in the physical world. Key properties of successful physical and language design make this set of digital manipulatives easy to learn and use. The author’s research suggests that, compared to screen-based user interfaces, tangible user interfaces make computation immediate and, thus, more accessible, and that they are appropriate for children learning about computation and scientific exploration. Compared to languages based on an imperative programming model, the streams programming model is significantly easier to explain to both children and adults.
There is now a growing body of research in tangible user interfaces that focuses on traditional usability, as well as a mature body of research in elegant programming language design. The author hopes to inspire a new breed of interdisciplinary researchers to combine tangible user interfaces with language design, so that today’s tangible toys lead to tangible tools for tomorrow’s professionals.
Spirograph is a trademark of Hasbro, Inc.
LEGO is a trademark of the LEGO Group.
Fred Martin co-created the now famous 6.270 MIT LEGO robot competitions with Randy Sargent, author of the “Interactive C” robot programming language.
Brian Silverman was one of Papert’s early students at MIT, who went on to become a founder of Logo Computer Systems, Inc. (LCSI).
Mr. Potatohead is a trademark of Hasbro, Inc.
PIC is a trademark of Microchip, Inc.
Fred Martin’s commercial version, called the Handy Cricket, is about the size of 4×AA batteries.
Their paper was accepted for publication concurrently with the commercial release of the Music Blocks product, which was developed independently by Neurosmith.
The author’s research was funded by the MIT Media Lab’s Things That Think consortium and the LEGO Company (which had no editorial influence over this article). Many thanks go to my thesis committee, Fred Martin, Mitchel Resnick, Hiroshi Ishii, and Hal Abelson for their guidance and encouragement; to Bakhtiar Mikhak and Rick Borovoy for their enthusiastic collaboration; and to Bonnie Friedman, Colin Ferguson, and Shari Goldin for their careful editing and help in preparing this article for publication.