Keywords

Introduction

Historically, German kindergarten has been characterized by a holistic approach rather than a focus on content specific teaching and learning processes. Now, even after recent changes in curricula and educational policy concerning early mathematics education, in Germany there is still a need for further development in in-service education. Kindergarten professionals’ training routes can differ widely and for many of the professionals currently working in kindergarten or pre-school, early mathematics education has not been part of their own pre-service education. The majority of kindergarten professionals in Germany do not have a university background but graduated from vocational schools.

The long-term in-service project “Children and Adults Explore Mathematics together” acts as one answer to this demand. One major part of the long-term project is a visit of kindergarten professionals together with a group of children to a join-in-studio at university twice a year. The visit is embedded in different project components like a workshop for the professionals before they visit the join-in-studio and a reflection meeting afterwards (Benz, 2016).

Due to the pandemic situation, in 2020 and 2021 many of the activities could not be conducted as usual. However, it was not only in-service training that could not be implemented as it was before the pandemic: mathematics education in kindergarten was also affected over a long period of time, because kindergartens were closed. Even after they re-opened, mathematics was not necessarily in the focus of the daily kindergarten routine.

Therefore, we planned a short in-service offer from May to July 2021 with the focus on enabling kindergarten professionals to support especially those children who would enter primary school in September 2021 and who had not had many chances to acquire mathematical basic skills. In order to enable mathematical individual-adaptive learning for these children, we offered the professionals a short in-service training. The focus of the in-service offer was on the mathematical content of counting and using structures for seeing the quantity of sets (Björklund et al., 2019; Sprenger & Benz, 2020). The short-term in-service project started with the component of a virtual training, providing content knowledge about children’s mathematical basic skills in the area of numbers and operations. The main focus in the workshop was on presenting different possibilities and tools for observing and diagnosing these skills as well as materials for supporting counting and using structures. In this paper, we focus on the professionals’ use and assessment of the different diagnostic tools.

Theoretical and Empirical Background – Importance of Observing and Diagnosing Children’s Competencies

Observing children’s mathematical skills is seen as a cornerstone for supporting children’s mathematical learning (e.g. Bruns et al., 2020; Fosse et al., 2018) not only in early mathematics education. Teacher adaptability as a widely accepted important aspect of effective instruction and support (Parsons et al., 2018) is based on observing and diagnosing children’s mathematical competencies. Wullschlegel (2017, p. 141) proposes that diagnosing is the first step for an individual-adaptive support. It is important to note that in the educational context, the term diagnosing is increasingly used in a non-pathologizing way, signifying “various practices of continuously gathering and evaluating knowledge about students” (Hoppe et al., 2020, p. 3). In this paper, we use the term diagnosing in the sense of one aspect of teachers’ competencies: “These diagnostic processes and activities include, for example, teachers choosing an appropriate question to learn more about a student’s conceptions or teachers evaluating the information given by a student in order to gain an understanding of this student’s conceptions. Growing research interest in those diagnostic processes and activities is one reason the term ‘diagnosing’ has become increasingly prevalent in the educational field” (Hoppe et al., 2020, p. 3).

Diagnostic activities help to get an idea of what mathematical skills a specific child still needs to acquire. Therefore, observing and diagnosing children’s mathematical skills can be found as a facet or aspect in many different models of professional knowledge (Gasteiger & Benz, 2018), and also in studies investigating professional competencies (Bruns et al., 2014; Yang et al., 2019). In the competence model of professional knowledge for early mathematics education (Gasteiger & Benz, 2018, p. 85) it is part of the facet situative observing and perceiving (see Fig. 1) and also part of the perspective individual-related diagnosis and fostering when assessing the learning levels.

Fig. 1
A model framework. Explicit knowledge leads to situative observing and perceiving, then to pedagogical and didactical action, evaluation, and lastly implicit knowledge.

Competence model of professional knowledge for early mathematics education. (Adapted from Gasteiger & Benz, 2018, p. 85)

Full size image

Perceiving mathematically relevant aspects in children’s actions as well as in their statements is necessary in order to ask adequate questions and to support children’s mathematical learning. This perspective shows the importance of observing for acting and responding adaptively in a situation of concrete interaction. The situational observing and perceiving of individual abilities (diagnostic aspect SOP) represents the prerequisite for knowing how to foster individual children. The ability to diagnose and foster as part of the pedagogical didactical action can manifest itself on the one hand in spontaneous purposeful interventional-diagnostic questions and stimulations (Steinweg, 2009) and on the other hand in the deliberate choice of learning stimulations, games or materials that adequately foster the mathematical learning process.

Diagnosing as one main facet of teachers’ professional competencies is described as generating information about children’s understanding, eliciting students’ cognition (Kron et al., 2021) or as the goal-directed accumulation and integration of information to reduce uncertainty when making educational decisions (Heitzmann et al., 2019). Diagnosing implies obtaining a diagnosis.

Different Ways of Diagnosing – Generating and Recording Information About Children’s Mathematical Competencies

Diagnostic competencies are required in very different situations with different aims. In each diagnostic situation the professional aims to gain information about children’s skills either for future learning (initial) or for adapting instructional choices during the learning process (formative) or for getting information about learning results (summative). Moreover, the way of generating information can also be differentiated: information can be generated by a diagnostic interview or by conducting a test with individual children about the chosen mathematical content (e.g. Clarke et al., 2006). Both are artificial, planned one-to-one-situations and a child could easily realize that someone aims to interrogate them. According to the level of structuring and standardizing, this is the most structured way to generate information.

An everyday observation during times of free play in kindergarten is the least structured way and demands a high level of competence from early childhood professionals (Lembrer et al., 2018; Fosse et al., 2018; Bruns et al., 2020). Also, research on teacher noticing underlines the importance of situative observing. Mason points out that “[e]very act of teaching depends on noticing: noticing what children are doing, how they respond, evaluating what is being said or done against expectations and criteria, and considering what might be said or done next” (Mason, 2002, p. 7). Schoenfeld (2011, p. 228) also describes the importance of noticing for consequential actions: “What you see and don’t see shapes what you do and don’t do.” For the broad spectrum of perceiving children’s mathematical skills, different terms with slightly different notions are already used in this paper. Situative observing and perceiving as well as noticing have a strong situative aspect and focus on the interaction between professionals and children. These terms cover a broader spectrum where not only the (mathematical) skills of children are observed, seen or noticed but also other aspects of a teaching situation are identified. In the current discussion about the construct of noticing, not only the identification of significant interactions (concerning children’s thinking) or the attention to students’ ideas but also the interpretation and the decision how to respond are included (Sherin et al., 2011).

Between these two poles, different situations can be designed with direct tasks, board or card games, and guided play, in order to diagnose children’s mathematical competencies.

Sometimes, the way of recording the information is linked to the way of generating it. In a standardized test and in some diagnostic interviews the answers are scored or labeled as wrong or right (Peter Koop & Grüßing, 2011; Wollring, 2004). A semi structured way of collecting data for example can be found in the Documentation of Learning (Lerndokumentation) introduced by Steinweg (2009). Different mathematical competencies are presented in tables and the professionals can fill in their observations. For each given competence there are different columns labeled “with assistance”, “sometimes independently”, “often independently”, and “absolutely independently”. Thus, development in each competence can be documented as well. The least structured way of recording would be an essay or free notes. Between these different levels many nuances of pre-structured recording/collecting are possible.

Concerning adaptive learning support, the evaluation of the generated and recorded information does not focus on summative scores or on labels based on statistic data. For adaptive learning support, detailed information about the child’s specific mathematical skills and understanding are important.

Design of the Study

Different diagnostic and observation tools were offered to get detailed information about the children’s mathematical competencies and moreover, to evaluate the use of different diagnostic tools in terms of professional development.

Diagnostic Possibilities in an In-Service Offer

  1. 1.

    Direct observation

    A diagnostic one-to-one interview with a detailed description of mathematical tasks as well as a recording sheet on which the child’s approach could be recorded.

  2. 2.

    Situative observation in everyday or planned learning situations

    (a) A recording sheet with a table presenting concrete actions for the diagnosis of different mathematical competencies, e.g. “identifying quantity of fingers of two hands without counting”.

    (b) A recording sheet without structured suggestions (open questions).

Figure 2 shows an example of one of the recording tools, a table for situative observation.

Fig. 2
A form is titled counting and seeing daily observation table, page 2. The child's name is Emma. Below is a table of 5 columns and 4 rows. The column headers are what quantity, with help, partly with help, and autonomously. There is a logo at the top right of the form.

Recording situative observation with a table

Full size image

After introducing different diagnostic tools in the workshop, games, materials, picture books and tasks supporting mathematical number skills were illustrated. Every participant got the above mentioned different diagnostic tools and a box with games, materials and picture books for their institution. For every game, material, or picture book, the game idea and the learning possibilities were explained in a booklet, as well as possible impulses and questions to support and diagnose children’s competencies.

Altogether, 25 professionals from 20 different kindergartens participated in the workshop. Thus, 20 boxes were needed. In order to introduce the materials in the box, a student teacher came to each kindergarten, presented the box and played with the children at least one or two times. Thus, the kindergarten professionals were not only introduced to the games, but also had the opportunity to observe their children while they were interacting with the student teacher.

Two-Part Evaluation

In order to evaluate the in-service offer, we asked the professionals to take part in an online-survey. Here, they were asked to assess the practicability of the different tools. 11 kindergarten professionals took part in the online-survey. Three professionals recorded different agreements to the different diagnostic tools, showing highest agreement on the practicability of the direct observation, followed by the situative observation with the given table and lastly by the situative observation without the possibility of structured recording. The answers to the open questions concerning the practicability of the different tools led to more insights: “For every colleague, especially the 1:1 interview was interesting, because the guidelines were clear and within a short time we got meaningful information.” “…I experienced that the children know ‘more’, the knowledge is wider, because the children could concentrate better.”

The agreement and the answers to the open question led us to the following research question for a subsequent qualitative evaluation:

  • What do professionals report about the use of different diagnostic tools for initial and formative diagnosis of basic mathematical skills in the area of number and operation?

To better understand professionals’ assessment of the different tools, a second, qualitative research approach was conducted. Two synchronous virtual focus-group interviews (Stewart & Shamdesani, 2015) with respectively two participants were conducted. The two structured guideline interviews were conducted by the same interviewer and lasted 72 and 60 min. The group interviews were transcribed and categorized by qualitative content analysis in accordance to Kuckartz (2018). Parts of the data were analyzed by three different researchers in order to validate the categories via intercoder reliability. As a result, five main categories were defined:

  • Organisation

  • Perceived Quality

  • Arrangement

  • Professional Development

  • Suggested Improvements

For each category, subcategories were defined and for both diagnosing situations it was recorded whether the statements given had positive or negative connotations.

Results

  1. 1.

    Organisation

Organisation of the observation: As in the case of situative observation the situation itself is not pre-organized by the professionals, only statements for the 1:1 interview were assigned to this category. The statements mentioned place and time as relevant factors that are specifically required for the 1:1 interview. Solutions were named though, like conducting the interviews in the morning when children are arriving, and using “a table in the hall.”

Organisation of the documentation situation: Organisation of documentation in the 1:1 interview is reported to be easy, whereas for the situative observation it often seems to be complicated and challenging because the “sheet is not always at hand” and the professionals “forget” the situation easily or “cannot remember it at the end of the day”.

  1. 2.

    Perceived Quality

Accuracy: The 1:1 interview is perceived as being more accurate than the situative observation. As an explanation, the professionals name on the one hand the explicitly different tasks that approach different aspects of basic mathematical skills, and on the other hand the suggestions given for inquiring about the children’s thinking, or for requesting further explanations from them about how they are solving the problems: “My observations of the children were more focused [...] with the subitizing and I realized that really many children counted very much”. Also, the professionals reported that the 1:1 situation helped them to alter as well as to complete their evaluation of children’s competencies they had generated from everyday situations.

Mathematical focus: As everyday situations are very complex, the professionals reported difficulties in focusing on mathematical aspects here: „in the free observation, it is still sometimes the case that I lay the focus on other things, but then, one still perceives how the children interact with each other and then I am not focused anymore on what I wanted to observe… “Another aspect of the mentioned complexity is that professionals often do not have sufficient time and/or space to follow through with their observation because there are demands of other children as well. The professionals describe it as an advantage that in the 1:1 interview they have the possibility to focus on different “important” mathematical aspects at once. They also perceive their observation in the 1:1 interview as more detailed: „Still, with your interview guideline one gets – of course – a much more differentiated picture of the mathematical competencies of the child.“

Usability of documentation: The documentation of the 1:1 interview is also reported to be very helpful in conversations with parents or teachers about the specific mathematical development of each child. This is due to the detailed description of tasks and the feeling of having a diagnosis “in black and white” at hand. A similar use of the documentation of the situative observation is not mentioned.

  1. 3.

    Arrangement

Learning situation: The 1:1 interview was described as an artificial situation compared to natural situations. Besides reduction of complexity and a need for extra place and time (see categories above), different impacts of this artificial situation on the children were perceived.

Motivation of children: Most of the professionals reported that the children were very eager to join the 1:1 interview and to get undivided attention as well as to show the professionals their capabilities: “I was at the table, the other children also walked by – children who usually were not easy to capture also wanted to take part in the interview”.

Emotional stress: One professional reported that some children perceived the artificial 1:1 interview as a “test-situation”: “they immediately have the feeling of a test-situation” (…) “I also have the reporting sheet next to me, filling it in and then they ask ‘what are you writing there?’.” The professional reported that mainly insecure children felt uncomfortable in the situation. However, it should be mentioned that this professional called the 1:1 interview a test herself, whereas the other professionals rather described it as “a game”, indicating an awareness of possible emotional stress. This aspect was never linked to the situative observation.

Interaction-feedback: In situative observations, the professionals did not perceive a need to give feedback to children’s actions, but one reported about the challenge of appropriate reactions in the 1:1 situation if children could not solve the tasks. Here they also perceived that children felt sometimes discouraged.

  1. 4.

    Professional Development

Especially the 1:1 interview was mentioned concerning perceived professional development for themselves as well as for their colleagues.

“Door-opener” for mathematics education: The 1:1 interview was reported to represent an access to mathematics education for professionals who are not affiliated with it: „It helped my colleagues to gain access to the subject at all. Because, I think for many it is just really difficult, somehow. Mathematics – where do I begin? It’s counting, somewhat... And I think it was helpful for many to conduct a direct observation in order to get a sense of important aspects.”

The last part of the statement also refers to the professional competence facet explicit knowledge (Gasteiger & Benz, 2018).

The data also provides insights into the professionals’ self-perceived development.

Perceived acquisition of explicit knowledge: The participants reported that by using the detailed guidelines of the 1:1 interview and the pre-structured table – they acquired explicit knowledge. Especially knowledge about perceiving and using structures was mentioned.

Perceived acquisition of situative observing and perceiving skills: All professionals reported that conducting the 1:1 interview led to an enhanced view for children’s competencies as well as for mathematical potential in learning situations: “because it also simply trains the view on the child’s competencies and because then, it is also easier to observe in everyday situations”.

Perceived development of pedagogical and didactical actions: The participants also reported a higher awareness of mathematical content in observations and the deliberate initiation of situations to promote mathematical competencies.

Conducting the 1:1 interviews promoted the professionals’ process oriented view on children’s mathematical competencies, especially concerning the determination of quantities by counting or using structure. Adaptive learning support was reportedly implemented as a result of insights gained from the interviews.

  1. 5.

    Suggested Changes for designing observation situations and documentation

Despite the 1:1 interview being their preferred diagnosing tool, the participants suggested some modulation concerning the role of the interviewer. As stated above, a perceived “test situation” may induce emotional stress in children. Thus the professionals expressed a need for proposed wording of feedback that encourages a child’s participation in the interview whilst remaining neutral towards the accuracy of their answer.

A suggestion relating to the documentation of the situative observation was to integrate it into their existing documentation tools.

Discussion

The professionals reported about many positive aspects of the 1:1 interview, not only concerning the interview situation itself in terms of accuracy, mathematical focus, organization and usability of the documentation, and the children’s motivation, but also concerning self-perceived positive effects on their implementation of adaptive learning support. They felt they could train their diagnosing competencies in a prescribed situation reduced of complexity, and thus had the possibility to acquire knowledge not only about relevant mathematical contents but also about what (mathematical) thought processes are important and how they can be assessed. They reported about transferring acquired competencies from the artificial diagnosing situation to everyday situations in which they were able to discover mathematical aspects and act and react adaptively.

They also mentioned negative aspects of the artificial, strongly focused situation which may be perceived as a test situation. Those professionals who called the interview a “test” also reported about children’s emotional stress and problems with getting feedback. This shows the danger of using direct diagnostic tools as assessment in an artificial “test-situation” in pre-school settings. 1:1 interviews can create negative experiences for children if they get negative feedback like “this is an incorrect answer” or “you are not able to solve the task”. Also, one consequence of 1:1 interviews could be that due to summative scores, children are labelled as “weak” or as “children at risk” instead of using the differentiated results for further adaptive support (Gasteiger, 2010; Meisels & Atkins-Burnett, 2000; Meisels, 2007).

Therefore, materials which can be used as tests should not be accessible without training for the professionals.

Some professionals stated that they started to integrate parts of the 1:1 interview into suitable everyday situations – here it can be interpreted that their aim is to apply those parts in natural situations and thus reduce the artificial, test-like atmosphere. The previous use of the 1:1 interview could be interpreted as an exercise for observing and a possibility for getting to know “what and how” mathematical competencies can be observed in natural situations.

The situative observation was reported to be quite demanding because of the complexity of everyday situations, especially the organization of documentation. Still, regarding the above mentioned perceived acquisition of facets of professional competencies, the professionals reported transferring knowledge from the 1:1 interview to natural situations, so it can be assumed that they do see a need and utility of situative observation for supporting mathematics education.

Therefore, the following conclusion can be drawn with regard to the design and development of diagnosing and observation tools for early mathematical education: In order to compensate for the negative aspects of the 1:1 interview but also to keep the accuracy, mathematical focus, organization, and usability of the documentation, a prescribed diagnostic guided play and corresponding documentation tool could be developed. Thus, professionals would be able to train their diagnostic competencies in a situation reduced of complexity with time for documentation scheduled in advance, but without any negative effects on the children. Such structured diagnosis situation would also aim to sharpen the professionals’ view of children’s mathematical competencies and situational mathematical potential, which is needed for adaptive learning support.