Keywords

1 Introduction

Participation in international large-scale assessments enables education systems to compare and learn from each other; the results are used as a means to improve educational quality. However, to interpret the outcomes, it is important to consider the various, and likely unique combination of antecedent contextual and explanatory factors that also play a role in the assessment results. Education systems serve social and cultural goals (Komatsu & Rappleye, 2017) and reflect the overall economic situation that interconnects governmental decisions related to specific educational goals with politics and resources (see, e.g., Hanushek & Woessmann, 2019). Looking into educational policies and practices can provide essential context for understanding student performance results from large-scale assessments, such as IEA’s Trends in International Mathematics and Science Study (TIMSS), and can eventually contribute to improving results over time. Establishing the context for teaching and learning is therefore the basis for analyzing data and making comparisons between education systems.

Seven participants from the Dinaric region took part in TIMSS 2019, namely Albania, Bosnia and Herzegovina, Croatia, Kosovo,Footnote 1 Montenegro, North Macedonia, and Serbia. We provide a comparative overview of the structures of the participating education systems and a brief introduction to some of the topics that are covered in depth in this book. We first explore the paths of schooling that children in the region typically go through during their school career, including opportunities for early childhood learning. We then provide a brief synopsis of the mathematics and science curricula at the system level. A particular focus of this chapter is how quality assurance functions in each of the respective education systems and what role student assessments play in that context. We conclude by examining the most important features of TIMSS 2019 and the implementation of the study at grade four across the region.

2 Schooling Paths in the Region

2.1 Early Childhood Education and Care

Generally, the term early childhood education and care (ECEC) refers to the “provision for children before the start of compulsory primary education” (European Commission/EACEA [Education, Audiovisual and Culture Executive Agency]/Eurydice, 2019c, p. 12) and includes services that ensure the child’s safety and care as well as services that support the child’s educational development. The ECEC sector can be further categorized, and the transition from a childcare-type setting (e.g., nurseries) to an education-type setting (e.g., kindergartens), or the age of transition (under and over three years), are commonly used to divide the sector (European Commission/EACEA/Eurydice, 2019c).

ECEC provision is generally available in form of nurseries, kindergartens, and pre-school preparatory programs in the Dinaric region, and attendance at this level is not mandatory, except in parts of Bosnia and Herzegovina, Croatia, and Serbia. Bosnia and Herzegovina has structured their provision into three different levels, which include nurseries for children between six months and three years, kindergartens for children from three to six years, and pre-school preparatory programs, which start a year before the first grade of primary school. The last level is compulsory for the majority of the population (Popić & Džumhur, 2020). Serbia provides pre-school education in form of nurseries for children between six months and three years, and kindergartens for children aged three to seven. Since the school year 2006, one year of preparatory pre-school education before the beginning of primary school has become mandatory (Đerić et al., 2020). In Croatia, attending the last year of ECEC in part-time format was made compulsory in 2014. This implies that children in Croatia, Serbia, and parts of Bosnia and Herzegovina are legally entitled to have a guaranteed place for the last year of ECEC (European Commission/EACEA/Eurydice, 2019c).

Participation rates for the ECEC sector are available for the region from different sources (e.g., both Eurostat, the European Commission’s statistics hub, and UNICEF, the United Nations Children's Fund, publish such statistics). However, depending on the methodology of national statistical institutes, the definition of an ECEC institution, or the classification of age groups, numbers can vary considerably between education systems and thus, comparisons may be difficult. For instance, participation rates could include only children in early childhood educational development programs (coded as International Standard Classification of Education [ISCED] 010; see UNESCO [United Nations Educational, Scientific and Cultural Organization] Institute of Statistics, 2012 for an explanation of the ISCED classifications) and exclude children in childcare provisions that fall outside of the ISCED classification or vice versa (European Commission/EACEA/Eurydice, 2019c). Another factor that needs to be taken into account is the structure of the sector and whether the offer is mainly public or private. This can have an effect on official participation rates (UNICEF, 2017).

According to Eurostat data, participation rates for children under the age of three in ECEC provisions are only available for some education systems in the region; for 2017, these included Croatia (15.9%), North Macedonia (10.3%), and Serbia (14.5%). In 2017, participation rates of children between the age of four and the starting age of compulsory primary education were 82.8% in Croatia, 70.4% in Montenegro, 68.6% in Serbia, and 39.5% in North Macedonia (European Commission/EACEA/Eurydice, 2019c). Even though enrollment rates in the ECEC sector are rather low throughout the region, it is notable that considerable efforts have been made in recent years to increase the participation of children in ECEC provision one year prior to the beginning of primary education, notably in Kosovo (Aliu, 2019; UNICEF, 2017) and Bosnia and Herzegovina (UNICEF, 2020).

2.2 Compulsory Education

Children in the Dinaric region generally enter primary school in the calendar year of their sixth or seventh birthday, depending on the various regionally established cut-off dates. In Croatia, only the children who reach their sixth birthday before 1 April can start school in September of that same calendar year, while, in Serbia, the equivalent cut-off date is 1 March. In addition to official policies, some education systems allow a degree of parental discretion and choice, or include a medical and psychological examination procedure in the enrollment process (Kelly et al., 2020). Considering compulsory education as “a period of full-time education/training that is compulsory for all students” (European Commission/EACEA/Eurydice, 2019a, p. 5), the duration of compulsory education in the region ranges from eight years in Croatia to 11–13 years in North Macedonia (see Table 1). The duration of compulsory education in North Macedonia depends on the chosen secondary school track, which can be general or vocational, and takes two to four years to complete (Lameva, 2020).

Table 1 Students’ starting and leaving age for compulsory education/training and duration
Full size table

Primary and lower secondary education is organized as a single-structure model across the region, with no transition between primary and lower secondary education (see Fig. 1) and includes general and compulsory education for all students. In addition to some differences with regards to the entry age mentioned above, the duration of the primary level differs. Grade four may have a different position within the single structure. While grade four is the final grade of the first educational cycle in Croatia and Serbia, it is grade five in Albania, Bosnia and Herzegovina, and Kosovo. In Montenegro, and North Macedonia the primary and lower secondary education is organized in three-year cycles, grade four being the lowest grade of the second cycle (European Commission/EACEA/Eurydice, 2018).

Fig. 1
figure 1

Main models of primary and lower secondary education (ISCED 1–2) in Europe 2018/2019

Source Based on information provided by European Commission/EACEA/Eurydice (2018)

Full size image

2.3 Language of Instruction

Across the region, several different languages are used for teaching in schools. The TIMSS 2019 assessment was administered in the main languages of instruction in the region (see Sect. 5, Table 2).

Table 2 TIMSS 2019 administration and sampling information
Full size table

In Croatia, Croatian is the only language of instruction in the great majority of the schools, however, in certain regions, minority languages are recognized as a second official language. Minorities are guaranteed education in their native language using three different approaches; namely, schools where all classes are taught in the minority language, schools where both Croatian and the minority language are taught, and schools where at least some additional language classes in the minority language are available. The recognized minority languages in Croatia are Albanian, Czech, Hungarian, Italian, German, Macedonian, Polish, Serbian, Russian, Rusyn, Slovak, Slovene, and Ukrainian (Elezović & Muraja, 2020).

Serbia uses the Serbian language as their main language of instruction, but ethnic minorities may receive instruction in their mother tongue. So far, instruction in primary schools has been organized in Albanian, Bosnian, Bulgarian, Croatian, Hungarian, Romanian, Rusyn, and Slovak (Đerić et al., 2020).

In Bosnia and Herzegovina, education is delivered in the official languages Bosnian, Croatian, and Serbian, depending on the curricula. In addition, some private schools offer instruction in English, French, Turkish, and German (Popić & Džumhur, 2020).

In North Macedonia, the official language of instruction is Macedonian, however, national minority groups are entitled to receive compulsory education in their mother tongue. Instruction in primary schools is available in Macedonian, Albanian, Bosnian, Serbian, and Turkish, whereas secondary schools only provide instruction in Macedonian, Albanian, and Turkish (Lameva, 2020).

The official language of instruction in Montenegro is Montenegrin. However, teaching is also performed in other languages in official use, which include Serbian, Bosnian, Croatian, and Albanian (Radović, 2020).

Kosovo has two official languages that are used for instruction in schools, namely Albanian and Serbian. In addition, national minorities have a right to education in their native language, such as Bosnian and Turkish (DQASAL/MEST, 2020).

The majority of students in Albania are taught in Albanian. Recognized national minorities may receive schooling in their native language, which is currently available for the Greek and North Macedonian ethnic minorities (Council of Europe, 2017).

3 Mathematics and Science Curricula in the Region

In general terms, a curriculum can be defined as “the major organizing concept in considering how educational opportunities are provided to students and the factors that influence how students use these opportunities” (Mullis & Martin, 2017, p. 4). More precisely, the curriculum can be presented at three levels: (1) the intended; (2) implemented; and (3) attained curriculum. These levels, arranged in a top-down order, represent: (1) educational policies and national and social contexts; (2) actual teaching in the classroom, which includes home and school context; and (3) learning outcomes. To make improvements in student attainment, authorities in the education systems need to ensure that any intended changes are being implemented in schools and classrooms. Usually, it is a time-consuming effort to change routine and habit, which includes learning infrastructure and teaching practices. Quality assurance activities in the form of external evaluation, particularly if all stakeholders are involved, can aid with the implementation of the intended plans and improvements.

3.1 Teaching and Instruction

A national curriculum that covers mathematics and science instruction at grade four is commonly used across the region, except in Bosnia and Herzegovina, where mathematics and science curricula vary among its constituents (however, in 2018, a new common core curriculum based on learning outcomes was established as a common foundation from which to create the individual curricula; Bosnia and Herzegovina Council of Ministers, 2018). For the remainder of the region, the mathematics curriculum that was in place during the TIMSS 2019 administration was introduced in 2000 in Kosovo, in 2006 in Croatia, in 2007 in Serbia, in 2014 in Albania, in 2015 in North Macedonia, and in 2017 in Montenegro. Across the region, the science curriculum was generally introduced at the same time as the mathematics curriculum, except in Serbia, where the science curriculum was introduced in 2006, a year earlier than the mathematics curriculum (Kelly et al., 2020). During 2014–2016, North Macedonia adopted an adapted version of the Cambridge International Assessment curriculum (see Cambridge Assessment, 2021) for mathematics and science, which provides a comprehensive set of progressive learning objectives and a structure for teaching across grades one to nine (Lameva, 2020). In Croatia, Kosovo, and Serbia, the mathematics and science curricula have undergone continuous reform since the establishment of the curricula that were in place during the TIMSS 2019 administration. During the 2018/2019 academic year, a new curricular reform was introduced on an experimental basis to a limited number of schools in Croatia, and this implementation was subsequently extended to all schools in the following school year. The gradual introduction of the new curriculum, starting at grade one, means that reform of the grade four curriculum is scheduled for completion by school year 2022/2023 (Elezović & Muraja, 2020). In Kosovo, a new competency-based curriculum was initiated in the school year 2017/2018 at the school entry grades and its further implementation is still ongoing (DQASAL/MEST, 2020).

For all TIMSS participants in the region, both mathematics and science are taught by the classroom teacher (see chapter “Teachers, Teaching and Student Achievement” for more information on teachers’ instructional practice and measures of teacher quality). Across the Dinaric region, 17–22% of the total instruction time in grade four is devoted to teaching and learning mathematics, and nine to 17% of the time is allocated to teaching and learning science (see chapter “Opportunity to Learn Mathematics and Science” for more information on opportunities to learn). To evaluate the implementation of their mathematics and science curricula, all education systems make use of school inspector visits and school self-evaluation. Albania, Croatia, Kosovo, and Montenegro use national or regional examinations to evaluate the (achieved) mathematics curriculum, and Kosovo also undertakes a research program for this purpose. Albania, Bosnia and Herzegovina, Croatia, Kosovo, and Montenegro use national or regional examinations to evaluate the science curriculum (Kelly et al., 2020).

3.2 Digital Competence and Use of Information and Communications Technology

From the perspective of national mathematics curricula in the region, Albania, Kosovo, Montenegro, and North Macedonia include statements or policies about students’ use of digital devices in grade four mathematics instruction. For instance, in Montenegro, information and communications technology (ICT) use is frequently recommended for undertaking calculations and researching new ideas. Calculators are used in grade four mathematics instruction to check the correctness of calculations. Acquiring ICT skills, such as finding, processing, and saving information, is also a general goal of the state’s education program, although none of the region’s education systems include specific statements about students’ use of digital devices during mathematics tests or exams in their curricula. However, the recently implemented curriculum in Croatia (which was initiated for grade one in the school year 2018/2019) emphasizes using digital technologies in teaching and learning from grade one onwards. For science instruction in grade four, the national curricula in Albania, Kosovo, Montenegro, and North Macedonia contain statements about the use of digital devices, such as computers, tablets, or calculators. For example, in Montenegro, students are encouraged to become familiar with using digital devices in all subjects. Teachers may decide to use computers in the classroom to demonstrate experiments or to get students to practice their skills, although this is not mandatory (Kelly et al., 2020).

There are some special initiatives related to ICT in the region. In Croatia, the development of digital competencies is seen as a cross-curricular endeavor and, as part of the reforms that started on an experimental basis in the school year 2018/2019, school equipment is being upgraded and ICT slowly introduced into teaching practices. Before this reform, computers were rarely used outside of informatics lessons in primary education, and the infrastructure and conditions for ICT use in teaching varied considerably between schools and counties. ICT use is considered central to improving and modernizing teaching and learning in Croatia (Elezović & Muraja, 2020). Across the wider region, there are several other ICT initiatives, including the twenty-first century schools program managed by the British Council; this provides support to around 4500 primary schools in the region, to strengthen digital education and digital literacy (British Council, 2020).

According to a European Council recommendation (European Union, 2018), digital competence can be defined as “the confident, critical and responsible use of, and engagement with, digital technologies for learning, for work, and for participation in society.” This definition, accompanied by a comprehensive framework known as DigComp 2.1 (Carretero et al., 2017), has become a popular reference tool for many European countries in incorporating digital competence in their national curricula, and has also been adopted in the Dinaric region. Bosnia and Herzegovina refers to the European definition of digital competence for school education in its curricula and related strategy documents, whereas Albania and Serbia refer to the European definition in addition to their own national definition. In contrast, Croatia has its individual national definition of digital competence, and Montenegro and North Macedonia do not have a common definition. In principle, digital competence in primary and secondary education can be implemented as a cross-curricular theme, as a separate subject, or integrated into other subjects. In the school year of 2018/2019, the national curriculum for primary education (ISCED 1) in Serbia included digital competence as a cross-curricular component, while in Montenegro and North Macedonia it was included as a compulsory separate subject (European Commission/EACEA/Eurydice, 2019b).

4 The Role of Assessments for Quality Assurance

4.1 Decision-Making Authorities and Distribution of Responsibilities

Throughout the Dinaric region, the decision-making authority and responsibilities related to different factors within the education system are generally structured in a similar way, reflecting the similar circumstances across the region when the end of communism around 1990 resulted in the process of transition from centralized planning to a market-based economy. The general trend of recent years has been to delegate more responsibility from central governmental levels downwards through the hierarchies within the systems. For example, North Macedonia has transferred greater responsibility for educational matters to municipalities and, since 2005, the municipalities have been responsible for funding and running pre-school institutions, primary schools, and secondary schools. They were also empowered to open new establishments, distribute central funding, maintain and audit schools, and appoint principals, teachers, and school board members. These reforms have led to relatively high levels of school autonomy in terms of resource management and student assessment policies. Principals have increased responsibility when it comes to human resource activities related to teachers, and teachers have more control over the development of classroom assessments. The North Macedonian government considered this step as an important move toward improving the quality of education, equality of opportunities, and the overall efficiency of the system (Kitchen et al., 2019; Lameva, 2020).

Albania has gone through a similar series of educational reforms, deemed necessary to improve educational outcomes such as student performance. In 2003, regional education directorates and offices were set up to support the Ministry of Education, Sport, and Youth with implementing the national education policies in schools. Further decentralization of school services in 2019 made regional directorates responsible for school evaluation. Reforms led to increased school autonomy, which should enable schools to make effective use of resources and reflect on their policies and practices. As one example, after approval from the local educational institutions, schools are now entitled to draft individual curricula based on the ministry-approved curriculum framework (Maghnouj et al., 2020).

In Kosovo, municipalities are in charge of the construction and maintenance of educational facilities and for ensuring a healthy environment for staff and students. They are also responsible for enrolling students and hiring teachers and other school staff (DQASAL/MEST, 2020).

In Serbia, the Ministry of Education, Science, and Technological Development is responsible for educational research, planning, development, supervision of pre-school, elementary, secondary, and higher education, and the organization, evaluation, and supervision of employees in the education sector. Public educational institutions are financed mainly from national and local budgets. It has been estimated that, during the last few years, almost 90% of the total funds have been provided from the national budget, with the remainder derived from local budgets. Recently, municipalities have become more involved in the funding of education; however, the government still provides the essential materials and technical resources that are a necessary minimum for school operations (Đerić et al., 2020). The process of democratization, decentralization, and depoliticization of education in Serbia started in 2000 at all levels of education, targeted at educational policy and practice and quality and equity in education (Spasenović et al., 2007).

4.2 Quality Assurance Structure in Education Systems

Quality assurance is an integral part of education systems, impacting educational outcomes. Exactly how quality assurance is implemented and where it is located within the education system depends on the structure of a system. It can form part of the role of the ministry of education or be organized as a completely independent external institution. Across the Dinaric region, multiple relevant institutions deal with quality assurance in a pre-university context. These include internal quality assurance processes, which can be defined as a continuous process of rethinking and reevaluating current practice, guided by various procedures, including statistics, indicators, and other information collected from and serving different stakeholders. Conversely, external quality assurance processes may include national and international assessments and evaluations by school inspectorates.

The education systems in the Dinaric region usually subdivide quality assurance into the following sections: early childhood education and school education, higher education, and adult education and training. Quality assurance in the school sector is structured in different ways and focuses on varying aspects throughout the region.

In Albania, the bodies responsible for quality assurance in the school sector are the National Agency for Education, Training, and Qualifications and the State Education Inspectorate. Internal quality assurance mechanisms in Albania include school self-evaluation, which is conducted via the Albanian school performance charter (Albanian Ministry of Education and Sports, 2014), a document that incorporates information on achievement, support for the school in performing their duties, and standard indicators, and functions as a self-evaluation instrument for schools. National student assessments have been an established external quality assurance instrument since 2016, conducted at the end of grades five, nine, and 12, and managed by the Centre for Educational Services (European Commission/EACEA/Eurydice, 2020).

Quality assurance in Bosnia and Herzegovina is integrated into all institutions related to education, and the Agency for Pre-primary, Primary, and Secondary Education (APOSO) holds the authority to establish learning standards, evaluation of achievements, and development of the common core curriculum (Bosnia and Herzegovina Council of Ministers, 2018). At school level, internal evaluation is part of the governance structure and external evaluation is carried out by expert advisors, who may review school operations or the work of teachers (European Commission/EACEA/Eurydice, 2020). The first external evaluation at state level concerning grade four was carried out as a national assessment in 2002 by a predecessor of APOSO. National assessments in Bosnia and Herzegovina do not occur regularly due to a lack of financial resources (Popić & Džumhur, 2020).

Quality assurance in Croatia is managed by several governmental agencies with a focus on the specific levels of education. The Teacher Training and Education Agency is mainly responsible for quality assurance of the pre-primary, primary, and secondary education sector (European Commission/EACEA/Eurydice, 2020). External national examinations are usually conducted by the National Center for External Evaluation of Education (NCEEE); since its establishment in 2006, national exams have been conducted at rather irregular intervals and are seen as diagnostic tools. In 2011, IEA studies were introduced in the form of a combined TIMSS and PIRLS (Progress in International Reading Literacy Study; see IEA, 2021) assessment at grade four. Since then, these IEA studies have become an independent international tool for deepening the understanding of student achievement and student attitudes in primary education (Elezović & Muraja, 2020).

In Kosovo, the Ministry of Education Science, and Technology (MEST) is primarily responsible for ensuring the quality of pre-university education. Quality assurance activities are carried out by various subdivisions and subordinate agencies of the ministry, such as the Education Inspectorate or the Division for Quality Assurance, Standards, Assessment, and Licensing (DQASAL). The latter was established in 2001 to develop education policies and to conduct national and international assessments. Among other tasks, they also evaluate the quality of service of educational institutions or advise executives of education institutions. TIMSS 2019 represents the first international assessment in Kosovo dedicated to grade four studentsFootnote 2 (DQASAL/MEST, 2020).

The quality assurance system in Montenegro stipulates that all educational institutions carry out annual quality assurance and improvement activities in the form of self-evaluation. In addition, the Bureau for Education Services is responsible for external quality assurance in the sector of pre-school, primary, and secondary education. All quality assurance activities in Montenegro are based on the Rulebook on the content, form, and manner of quality assessment of educational work at institutions (Government of Montenegro, 2020). This covers areas such as students’ achievement, quality of teaching and learning, the governance and management of the institution, or cooperation with parents (European Commission/EACEA/Eurydice, 2020).

Quality assurance in the pre-school, primary, and secondary education sector in North Macedonia is carried out by the State Education Inspection (SEI), the Bureau for Development of Education (BDE), and the State Examination Center (SEC). The last body is responsible for the external evaluation of students’ achievement and teaching (European Commission/EACEA/Eurydice, 2020). The first national assessment of mathematics in North Macedonia was organized at the end of grade four in 2000 and followed by an assessment for natural sciences in 2006, which aimed to provide the relevant education institutions with valid data about student attainment that could be used to create educational policy and provide feedback for schools and teachers. Over the period from 2014 to 2016, electronic external testing was carried out for students in grade four. A new law on primary education demanding national assessment is currently being developed, with the first tests planned for 2021 (Lameva, 2020).

The Serbian Ministry of Education, Science, and Technological Development supervises external quality assurance activities that are conducted by the National Education Council, the Institute for the Improvement of Education, and the Institute for Education Quality and Evaluation. These institutions are responsible for determining educational standards, national testing, suggesting improvements, or adapting educational policies. The main aim of the internal quality assurance lies in strengthening school self-governance and autonomy and in ensuring the professional development of teachers by identifying ways for improvement of their practices and management (European Commission/EACEA/Eurydice, 2020). The first national testing of grade four students was undertaken in 2006 by the Institute for Educational Quality and Evaluation. In 2015, Serbia introduced annual tests in mathematics for students in grades four and six.

4.3 TIMSS 2019 Implementing Institutions

International large-scale assessments have become an integral part of external quality assurance across the Dinaric region in recent years, as internationally standardized testing provides high-quality data and potentially beneficial comparable insights into the strengths and weaknesses of education systems. Although TIMSS had been previously implemented in many parts of the region, 2019 was the first time that all education systems in the region chose to use the assessment to monitor the mathematics and science achievement of their grade four students. The national institutions involved in implementing TIMSS 2019 across the region have been vital for the success of the study in the participating education systems.

The Educational Services Center (QSHA) in Albania was established in 2017 as an evolution from multiple older institutions. Currently, QSHA is a special institution for organizing, monitoring, and supervising educational reforms and quality assurance in the field of evaluation of achievement and exams. It exercises the technical, professional, supportive, and recommendatory function of the assessment and organization policies of state exams (e.g., the school-leaving examinationFootnote 3). The institution is also involved in all international large-scale assessments in which Albania participates (European Commission/EACEA/Eurydice, 2020).

In Bosnia and Herzegovina, APOSO was established to serve as an independent administrative organization promoting quality of education at pre-primary, primary, and secondary levels. The agency was also appointed to conduct the TIMSS administration in Bosnia and Herzegovina (Popić & Džumhur, 2020). The agency has its seat in Mostar and two regional units located in Sarajevo and Banja Luka. Planning and implementing international comparative studies form part of the work of the regional unit in Sarajevo, together with other tasks related to learning standards, student achievements, and evaluating the results in primary and secondary education. APOSO is further involved in developmental and research programs and projects, designing and maintaining relevant databases, performing psychometric measurements, statistical analysis, publishing activities, and performing translation tasks (APOSO, 2020).

The NCEEE in Croatia originated with the mission to provide valid and objective monitoring and external evaluation of the national education system, in collaboration with its stakeholders, to define and improve the quality of education. The NCEEE, located in Zagreb, is responsible for comprehensive external evaluation of the pre-tertiary education system in Croatia. In addition to the objective and transparent external evaluation of learning outcomes and the development, implementation of the system of external evaluation of education, and conduct of international research in education, the NCEEE licenses, certifies and educates key stakeholders in education on the quality of education at all levels of the system, and research and development in educational measurement, and continually promotes and develops the quality of the education system to foster positive change (NCEEE, 2020).

In Kosovo, DQASAL was first established in 2001 as part of the Ministry of Education, Science and Technology of the Republic of Kosovo, and is responsible for the development of educational policies and national and international student assessments. The competencies of the division include the organization and implementation of international assessments, the dissemination of study results, and the implementation of the national school-leaving examination. The institution further consults and cooperates with municipal education institutions to support them with self-assessment and development based on test results, and organizes teacher seminars about assessing student achievement (DQASAL/MEST, 2020).

The government of Montenegro established the Examination Center in Podgorica in 2005. The Examination Center conducts the external evaluation of achieved standards of knowledge and competences of students, organizes counseling for teachers to provide professional assistance for a better implementation of external knowledge assessment, prepares and organizes regional and national competitions for primary and secondary school students, organizes and conducts state examinations (e.g., the school-leaving examination), investigates and develops external assessments, and assesses the international comparability of the quality of the education system, including implementing relevant international assessments (Examination Center, 2005).

The North Macedonian State Examinations Center (SEC) was established in 2009 with headquarters in Skopje as an independent successor of the Assessment Unit. Its role is to evaluate the quality of education through external assessments. SEC’s competencies include the organization, implementation, and support of external assessments, and evaluating student achievement in primary and secondary education in North Macedonia. SEC also prepares and implements national external assessments, such as the school-leaving examination at the end of secondary school. The responsibilities of the center additionally include preparing and monitoring examination materials for high school and secondary vocational education. As an institution, SEC oversees the training and licensing of principals of primary and secondary schools and also issues certificates (licenses) to teachers and professional associates from primary and secondary education. SEC is responsible for the preparation and implementation of international assessments (State Examinations Center, 2020).

The Institute for Educational Research (IER) is an independent research institute in Serbia, founded in 1959. Granted the status of a scientific institution in 1961, the institute is currently a leading research institution in the field of education and contributes to the development and improvement of the quality of education in Serbia. Its main objectives are researching excellence and innovation, and promoting science and the awareness of the role of educational research. It integrates basic, applied, and developmental multidisciplinary research to contribute to scientific, educational, and social development in Serbia. IER was responsible for implementing TIMSS from 2003 to 2019, focusing on grade four since 2011 (IER, 2012).

5 TIMSS Survey Design and Its Implementation in the Region

The TIMSS 2019 assessment was organized around a content dimension and a cognitive dimension, both of which consisted of several sub-domains. The assessment framework describes the content domains (e.g., measurement and geometry in mathematics or life science in science) and topics within these domains include specific abilities that grade four students should be able to demonstrate. The cognitive domains of knowing, applying, and reasoning describe the thinking skills students are expected to use to solve the mathematics and science problems. Each item in the assessment was assigned to a specific content and cognitive domain (Centurino & Jones, 2017; Lindquist et al., 2017) and, similarly, the aspects of the learning context covered by TIMSS 2019 were addressed in the context questionnaires framework (Hooper et al., 2017).

In recognition of the increasing use of information technology for learning and assessment in recent years, the TIMSS pen-and-paper assessment transitioned to a digital data collection mode in 2019 (“eTIMSS”; see Mullis & Martin, 2017), however the pen-and-paper option was retained as an option.

Among the Dinaric region TIMSS 2019 participants, only Croatia opted for eTIMSS. The eTIMSS design was more extensive because it also included four blocks of problem-solving and inquiry (PSI) tasks and items. However, for this research we only considered the item blocks in eTIMSS 2019 that had a matching counterpart in the pen-and-paper format, although they were adapted to make appropriate use of digital components (e.g., slightly modified so students could make use of additional “drag and drop,” or “sorting” features).

The individual grade four student response burden for the TIMSS 2019 assessment was the same as it has been since TIMSS 2007, allowing 72 min for the assessment and 30 min for the student questionnaire, with a short break before undertaking the second part of the assigned student achievement booklet and again before completing the student questionnaire. TIMSS’ ambitious reporting goals require many more questions in the assessment than could be answered by a single student in the amount of testing time available. TIMSS thus uses a matrix sampling approach that involves packaging the entire assessment pool of mathematics and science items at each grade level into a set of 14 student achievement booklets, with each student completing just one booklet. Item response theory scaling methods are then used to assemble a comprehensive picture of the achievement of the entire student population of a country from the combined responses of individual students to the assessment booklets that they are assigned (Martin et al., 2017).

Participants of TIMSS 2019 administering the assessment at grade four could choose to use booklets with some less difficult blocks in mathematics than the regular TIMSS grade four assessment. Out of the seven TIMSS 2019 participants in the Dinaric region, all but Croatia and Serbia opted for the less difficult mathematics assessment. The less difficult mathematics assessment was designed for students that are still developing fundamental mathematics skills. The results of these tests are reported on the same achievement scale as other TIMSS participants, but the less difficult items extend the TIMSS mathematics achievement scale to provide better measurement at the lower end of the scale. Experiences with TIMSS Numeracy in 2015 and PIRLS Literacy in 2016 (the less difficult version of IEA’s PIRLS reading assessment) indicate that lower-performing students are more strongly motivated by less difficult items, and better demonstrate what they know and can do, resulting in fewer omitted items and higher completion rates. Consequently, the results from the less difficult TIMSS and regular TIMSS are comparable, regardless of the version of the assessment the students have taken (Mullis & Martin, 2017).

5.1 Sampling and Some Key Characteristics of the Target Grade

The international TIMSS sample design calls for a minimum of 150 schools with one or more intact classes in grade four, resulting in a student sample of approximately 4000 students per participating entity. The target grade for this publication is grade four and the population is defined internationally as “the grade that represents four years of schooling, counting from the first year of ISCED Level 1” (Martin et al., 2017, p. 81). ISCED Level 1 corresponds to primary education or the first stage of basic education. In addition to the grade, TIMSS attempts to avoid assessing very young students and sets the minimum average age at the time of testing of 9.5 years. If this condition is not met, participants need to assess the next higher grade. For the participants in this publication, TIMSS 2019 was administered at grade four. The average student’s age on the day of testing in the region ranged from 9.8 in Montenegro and North Macedonia to 10.6 in Serbia. Data from 3270 to 5628 students per participating education system were collected between 11 March and 15 May 2019 across the region. The student assessment was administered in the main languages of instruction within the education system. North Macedonia administered the test in one minority language, namely Albanian, and Croatia prepared the home and school questionnaire in Serbian and Italian as well as Croatian (see Table 2).

TIMSS employs a two-stage random sampling design. A sample of schools is drawn first, then one or more intact classes of students are selected from each of the sampled schools. Intact classes of students are sampled rather than individuals from across the grade level or of a certain age because TIMSS pays particular attention to students’ curricular and instructional experiences in the classrooms. Sampling complete classes also has the operational advantage of less disruption to the school day compared to individual student sampling (Joncas & Foy, 2012). Each TIMSS participant needed to define the national target population and apply the TIMSS sampling methods to achieve a nationally representative sample of schools and students. The development and implementation of the national sampling plan is a collaborative exercise involving the country’s national research coordinator (NRC) and the TIMSS 2019 sampling experts. This procedure ensures that the school sampling frame (the school population list from which the school sample is drawn) provided by the NRC is complete, checking that categories of excluded students are clearly defined, justified, and kept to a minimum. The objective is to draw a nationally representative sample of students, while making sure both international and national requirements regarding sampling precision are met. National requirements are often addressed by applying a specific stratification approach (Meinck, 2020).

School location within the participating education systems and whether this location was urban or rural were the major explicit stratification variables used across the region. Albania also stratified their schools by school type (whether public or private), Kosovo by school shifts, and North Macedonia and Bosnia and Herzegovina by language of instruction. TIMSS aims to cover the whole grade four student population in all countries, but allows the exclusion of specific populations for practical reasons. At the within-school level, across the Dinaric region, classes or students with functional or intellectual disabilities were excluded, as were non-native language speakers, however, there were some differences at the school level. Most of the exclusions could be attributed to a small number of grade four students in schools (less than three), and the exclusion of schools that only taught students with special needs or provided instruction in a minority language, or those studying a different curriculum (e.g., international schools). In Albania, Montenegro, and Serbia, all students who were not taught in the majority language of instruction were specified as language-based exclusions. In TIMSS 2019, Kosovo and Serbia exceeded the five percent maximum exclusion rate that is usually set for studies designed for cross-national comparisons. In both cases, this happened because of their linguistically diverse populations; TIMSS 2019 could not always be administered in all recognized languages of instruction. In Kosovo, schools with the Bosnian or Serbian language as the primary language of instruction accounted for 8.56% of the overall exclusion rate, and the schools serving the minority populations in Serbia accounted for 8.21%. The analyses in this study only used the data from assessments delivered in the prevailing Albanian language in Kosovo and the Serbian language in Serbia. It is important to underline that the TIMSS 2019 test was administered in the language of instruction. The vast majority of the sampled students responded to the test in a language that they were using at home (Table 3). However, the data collected by TIMSS also indicates that classrooms are not linguistically homogenous.

Table 3 How often do students speak the language of the test at home?
Full size table

After sampling had been completed and all data collected, the TIMSS sampling experts documented population coverage, and school and student participation rates, and constructed appropriate sampling weights for data analysis. The target for TIMSS is a sampling participation of 100% for all sampled schools, classrooms, and students, and the achievement data are reported according to this target. TIMSS participants were assigned to one of three categories on the basis of their sampling participation: category 1 (considered to have met all TIMSS 2019 sampling requirements and to have acceptable participation rates), category 2 (meeting the participation requirements only after including replacement schools), and category 3 (failed to meet the participation requirements even with the use of replacement schools) (LaRoche & Foy, 2016). Across the region, all participants fell in category 1, indicating that the quality and validity of the results can be trusted.

5.2 Quality Assurance for the TIMSS 2019 Test Administration Across the Dinaric Region

Strict quality assurance procedures ensure large-scale assessments produce high quality, internationally comparable data. In TIMSS 2019, quality assurance was an integral part of the study and was implemented at both the international and national levels. This encompassed all main activities from the assessment framework, including assessment and questionnaire development, sampling, instrument preparation, data collection, scaling, and data analysis. The TIMSS quality assurance components that were implemented during the data collection period were the international and national quality control programs, and the TIMSS Survey activities questionnaire (SAQ), completed by all NRCs.

The SAQ is used by NRCs to document their experiences of the TIMSS 2019 main data collection processes. The questions cover all activities from school sampling, preparing the national study instruments including their translations, reviews, printing, checking and distribution, selecting and training the school coordinators and test administrators, maintaining the security of the assessment materials and the confidentiality of the responses, observing testing sessions, scoring the obtained responses, and performing data entry and checks. All these activities are additionally described in the TIMSS 2019 Survey operations procedures units, which are documents designed to provide guidance to NRCs. The NRCs reported that these guidance documents were considered clear and useful, although some participating entities encountered various challenges during the preparation of the assessment instruments (e.g., difficulties in using the Adobe InDesign® package to prepare the national versions of the international instruments).

Three interlinked aspects were related to data availability and quality: (1) school coordinator and test administration appointment and training; (2) the implementation of the national quality control program within the participating schools; and (3) independent observations undertaken as part of the international quality control program, which was overseen by the TIMSS & PIRLS International Study Center.

All school coordinators from the Dinaric region were appointed by the participating schools, and all undertook formal training. In most cases, these school coordinators trained the test administrators in the participating schools. However, in Bosnia and Herzegovina, most of the test administrators were trained by staff from the national study center. In Kosovo and North Macedonia, the test administrators were external to the participating schools. They brought the assessment materials for the testing session to the school, conducted the session, and collected all materials after the session was finished. Based on the completed TIMSS test administration forms, the most frequent problem for students was encountering unknown words and/or tasks that they had not yet covered in school (e.g., fractions in Croatia).

As part of the national quality control programs, the data collection was observed in the participating schools; these observations took place in all participating entities in TIMSS 2019 from the Dinaric region, except Kosovo and Montenegro, who had insufficient budget for this activity. Testing sessions in the recommended 10% of participating schools were observed in Albania, Croatia, and Serbia, 12% of the sampled schools were visited in Bosnia and Herzegovina, and 100% of schools were visited in North Macedonia. The national quality control monitors (NQCMs) did not document any major problems or special circumstances that occurred repeatedly during the administration of the TIMSS assessment.

In addition to the information from national quality control and the SAQ, international quality control monitoring was also part of TIMSS 2019. International quality control depended on reported observations from selected experts appointed as international quality control monitors (IQCMs). These individuals were employed independent of the TIMSS 2019 national centers and personally trained for monitoring activities by the TIMSS & PIRLS International Study Center (see Johansone & Flicop, 2020 for more detailed information on these aspects).