Analyzing teaching quality in Botswana and South Africa

Abstract

This study focuses on quantifying the quality of mathematics teaching in 183 randomly selected sixth grade classrooms: 100 from the North West province of South Africa and 83 from South East Botswana. The teaching quality is measured by coding videotaped lessons for three different components: mathematical proficiency, level of cognitive demand, and observed teacher knowledge. Results suggest that the overall teaching quality is about the same in both regions. Some variation was observed at the level of each component. For example, in the South Africa sample the students engage more in tasks that just involve “memorization” and less in tasks that involve “procedures without connections” in comparison with the Botswana students. Teachers in Botswana implement the official curriculum more faithfully than do those in North West. In both countries most of the learners engaged only in low-level tasks (very little activity involved “procedures with connections”) and teachers demonstrated a lack of knowledge about how to integrate mathematical content with effective pedagogical techniques.

Appendices

Appendix 1

Coding Instrument

Part I:

Content area and curriculum reference for the topic covered in the lesson

Parts II–IV:

A code of “0” or “1” indicating “not-present” or “present” for the development of each component during the lesson. Note: A code of “1” was entered even when the development of the component was “present” in the slightest form

Part V:

An overall teaching quality rating of “1”, “2” or “3” indicating “low”, “better” or “best” respectively

I. Content	Mathematical topic covered in the lesson.
II. Mathematical proficiency	a. Conceptual understanding (comprehension of mathematical concepts, operations and relations)
	b. Procedural fluency (skill in carrying out procedures flexibly, accurately, efficiently, and appropriately)
	c. Strategic competence (ability to formulate, represent, and solve mathematical problems)
	d. Adaptive reasoning (capacity for logical thought, reflection, explanation, and justification)
	e. Productive disposition (habitual inclination to see mathematics as sensible, useful, and worthwhile, coupled with a belief in diligence and one’s own efficacy)
III. Level of cognitive demand	a. Memorization (recollecting facts, formulae, or definitions)
	b. Procedures without connections (performing algorithmic type problems that have no connection to the underlying concept or meaning)
	c. Procedures with connections (using procedures with the purpose of developing deeper understanding of concepts or ideas)
	d. Doing mathematics (using complex and non-algorithmic thinking to explore and investigate the nature of the concepts and relationships)
IV. Teacher’s observed mathematical knowledge	a. Core knowledge (grade 6 mathematics)
	b. General pedagogical knowledge (use of instructional techniques clearly learned from teacher preparation or training)
	c. Pedagogical content knowledge (effective use of instructional techniques and mathematical knowledge that help children learn)
V. Overall quality:	Lower (1); better (2); best (3)

Appendix 2

Sample of a “low” quality lesson

I. Content	Algorithm for dividing whole numbers (up to five digits) by two-digit whole numbers
	Component	Code
II. Mathematical proficiency	a. Conceptual understanding	0	The lesson focused on performing procedures of long division. The teacher wrote five problems on the board, and did three of them. Her explanations focused on the steps of the algorithms. She made no mention of place value, offered no conceptual link to what it means to divide, and used no models for division. She did not explain why the algorithm works the way it does. When the students were working on the final two problems, the teacher supervised their work, but this primarily involved making notes in the students’ books: ticking or crossing to indicate correct work or errors, and dating and signing the page. The supervision did not take advantage of the many different division strategies the students used: some of them were doing multiplication as repeated addition (not very effective for large numbers). They did not share or discuss strategies and ways of doing the algorithms.
	b. Procedural fluency	1
	c. Strategic competence	0
	d. Adaptive reasoning	0
	e. Productive disposition	0
III. Level of cognitive demand	a. Memorization	1	The five division questions written on the board were: 125 ÷ 25 1,632 ÷ 16 5,192 ÷ 44 13,981 ÷ 31 10,864 ÷ 14 The level of demand in the questions was slightly higher in the questions left for the learners to complete on their own, since those two were 5-digit numbers divided by 2-digit numbers (though they followed on from 3- and 4-digit numbers divided by 2-digit numbers). We characterized this lesson as procedurally focused, offering a low level of cognitive demand. Much of the talk focused on recall of multiplication tables and number bonds as used in the division algorithm
	b. Procedures without connections	1
	c. Procedures with connections	0
	d. Doing Mathematics	0
IV. Teacher’s observed mathematical knowledge	a. Core knowledge	1	Whole class teaching: teacher wrote up a sample algorithm for working on long division but without any interaction with learners. She demonstrated knowledge of the algorithm but we saw no evidence of pedagogical knowledge. She could have written it up that same way if the students were not even sitting there. Nor was there evidence of pedagogical content knowledge since she made no attempt to teach or explain the mathematics involved in dividing
	b. General pedagogical knowledge	0
	c. Pedagogical content knowledge	0
V. Overall quality:	Low: 1

Appendix 3

Sample of a “better” quality lesson

I. Content	Classification and measurement of angles
	Component	Code
II. Mathematical proficiency	a. Conceptual understanding	1	The focus of the lesson was to classify and measure angles. The teacher started by reviewing the definitions of angles according to their sizes. Students then engaged in the activities as written on the board (see below). During the discussion, we noticed that only a few students were responding to the classification of the angle; then when the teacher asked for the estimated size of the angle, the students based their estimate on the definition of the angle and not on its actual size. Hence some of their estimates were wildly inaccurate. The teacher seemed to think the students were not estimating correctly; however, the reason for this seems to be the order of the activities. First, they were to estimate, then measure, and then classify. Mostly for this reason, the lesson was not as effective as it could have been. But learners discussed their reasoning for making the estimates and they developed conceptual understanding alongside the basic procedural skill of measuring and naming angles
	b. Procedural fluency	1
	c. Strategic competence	0
	d. Adaptive reasoning	1
	e. Productive disposition	0
III. Level of cognitive demand	a. Memorization	1	Notes written on board: Classification of angles according to size 1. Classify angles. 2. Estimate the size. 3. Use the protractor to measure the size She drew four angles on the board. She told the students to open their books and proceeded in a question-and-answer mode to define and write the definitions of different types of angles on the board: straight angle, right angle, revolution, reflex angle, acute angle, and obtuse angle (memorization). She gave them instructions not to draw but to classify the angles. Some drew and classified them nonetheless. She checked sporadically. One had written: Obtuse angle. She said that was incorrect and he redid it. She then proceeded in another question-and-answer session to ask the students to classify and give an estimation of the different types of angles, and then moved on to asking them to measure the angles using a protractor. One student went up to the board, but the teacher told her the answer as she held the protractor. Some students had their cases on their laps. They were quiet, but ready to leave, no longer paying attention. The teacher moved on to ask, “How many 180°s are there in 360°?” But then she stopped the lesson
	b. Procedures without connections	1
	c. Procedures with connections	1
	d. Doing Mathematics	0
IV. Teacher’s observed mathematical knowledge	a. Core knowledge	1	Whole-class, teacher-centred lesson with much writing on blackboard and use of questions and answers. The teacher used good questioning skills. She began by writing some notes on the board to structure the lesson, and also wrote notes for the learners to copy and refer to. After allowing the learners some time to work on the activity, she ended the lesson abruptly. The mathematical content of the lesson was sound though not necessarily well sequenced. The students had an opportunity for learning, though it was not optimal
	b. Pedagogical knowledge	1
	c. General pedagogical content knowledge	0
V. Overall quality:	Medium: 2

Appendix 4

Sample of “best” quality lesson

I. Content	Fractions—finding parts of a discontinuous whole which is not necessarily divisible by the denominator.
	Component	Code
II. Mathematical proficiency	a. Conceptual understanding	1	The lesson started with a problem posed: “Five sausages are divided among 3 people. How much sausage will each person get?” Concrete models were used to illustrate the problem; students worked on different strategies, shared their reasoning, and discussed different methods that led to the same solution. The mathematics focused on the concept of fractions, and we observed some procedural connection to the concept. Students were solving problems, making conjectures, and sharing their reasoning. The teacher demonstrated excellent questioning and guiding skills. Overall a very good lesson. We only observed one incident where the teacher did not key into the students’ responses correctly. This happened when the teacher asked, “What fraction of the 5 sausages did each child get?” She was looking for $\frac{1}{3}$ as the “right answer”; however the students had found that the solution to the original problem was “1 and $\frac{2}{3}$ sausages”, so their response to the second question was either “1 whole and $\frac{2}{3}$” or “$\frac{5}{3}$”. The teacher led them to believe that those were not acceptable answers, when in fact, 1 and $\frac{2}{3}$ sausages is equivalent to $\frac{1}{3}$of the full set of five sausages. She seemed not to realize that the referent unit for the students was still sausages and her referent unit was the set of 5 sausages. If she had realised this she would have explained it and the conceptual web for fractions would have been expanded rather than left open to question. However, the richness of the discussion provided many opportunities for learning
	b. Procedural fluency	1
	c. Strategic competence	1
	d. Adaptive reasoning	1
	e. Productive disposition	1
III. Level of cognitive demand	a. Memorization	0	She asked the students to solve several problems, which they did in pairs and then using the overhead projector (OHP) or through Q&A sessions, while the teacher circulated, looked at their work, or stood at the front. The questions were: How will the five sausages be divided equally between 3 girls? What if we divide the sausages into thirds? What would each child get? Show me the different ways in which you got to the answer Divide each sausage into three and how many pieces does each child get? What fraction does each child get? I have 16 apples. How many apples do I pack into 2 trays? Now I’ve got 4 trays, how many do I pack in them? And now I have 8 trays? How many? Now it’s a little more difficult. I want you to find $\frac{2}{4}$ of the 16 apples. You may do drawings Now I want $\frac{4}{8}$of the 16 apples—what do I do? How many apples are in 3 of the 8 groups? What will $\frac{5}{8}$ be? What will $\frac{3}{4}$ be?
	b. Procedures without connections	0
	c. Procedures with connections	1
	d. Doing mathematics	1
IV. Teacher’s observed mathematical knowledge	a. Core knowledge	1	The lesson was about fractions. This was whole-class, teacher-centred teaching in which the teacher gave the students problems which they solved on the OHP. The teacher first used paper representations of sausages and then apples to show students how division can lead to different fractions. She ended the lesson with a discussion of denominators. The quality of the activity and linked discussion in this lesson was conceptually rich and it would have extended their understanding of the fraction concept
	b. General pedagogical knowledge	1
	c. Pedagogical content knowledge	1
V. Overall quality:	High: 3