Portugal: The PISA Effects on Education
- 4.9k Downloads
From the bottom of the league table for PISA 2000 Portugal has raised to the OCDE average being the only OECD member that showed, up to PISA 2018, consistent growth in reading, mathematics, and science. This chapter gives a brief description of the Portuguese Education system and how PISA outcomes have shaped Portuguese education policies. It identifies the policies that probably explain the improvement in PISA and pinpoints weakness of the Portuguese education system through the lenses of PISA.
Accountability is a key feature of modern education systems since countries invest a significant portion of their resources educating their young. In the early 1960s, the International Association for the Evaluation of Educational Achievement (IEA) demonstrated the feasibility of large-scale studies and cross-country comparisons of student achievement in key school subjects (mathematics, reading, and science). During the last decades of the twentieth century the Organization for Economic Cooperation and Development (OECD), following in IEA’s footsteps, identified the need to regularly collect reliable and valid educational indicators that could be used to compare its member countries’ educational systems and inform policymakers on the outcomes of education policies. The Programme for International Student Assessment (PISA) was the OECD’s response to address some limitations of the IEA’s studies, namely insufficiencies in education quality measurement and limited international coverage (Breakspear 2014). PISA would fill this perceived gap by providing a measure of comparative educational systems outcomes not focused on curricular knowledge, like the IEA studies, but in terms of what students can do with the knowledge, skills, and competencies learned in school to solve everyday problems and be active citizens. On OECD’s own words “PISA represents a new commitment by the government of OECD countries to monitor the outcomes of education systems in terms of student achievement regularly and within a common framework that is internationally agreed upon. PISA aims at providing a new basis for policy dialogue and for collaboration in defining and operationalizing educational goals. (…) PISA can assist countries in seeking to bring about improvements in schooling and better preparation for young people as they enter an adult life of rapid change and deepening global interdependence” (OECD 2001, p. 3).
The first relevant international assessment of the Portuguese education system took place in 1995 with the IEA’s Trends in International Mathematics and Science Study (TIMSS). The Portuguese results were so poor that policymakers at the time argued that the study failed to assess the country’s students’ knowledge and skills. As a result, Portugal withdrew from TIMSS until 2011 (Barroso 2010; Marôco 2020). However, TIMSS 1995 revealed that the poor performance of Portuguese students went much deeper into the Portuguese social structure, with the education system echoing structural problems that went far beyond schools, teachers, or policymakers (Justino 2010). The aftermath of the TIMSS 1995 shock set the seed for the external evaluation of the Portuguese education system and evidence-based policy reforms.
Portugal, as a founding member of the OECD, participated in the first edition of the OECD’s PISA study in 2000. Like in TIMSS 1995, the Portuguese students were ranked at the bottom of the table of the ordered OECD participants in mathematics, science, and reading. These PISA results finally set the stage for the much-needed education reforms that took place in the following years. In 2018 Portuguese students stabilized their position on the PISA OECD average, being the only OECD country with a positive trend in all three domains during the 2000 to 2018 PISA life frame (OECD 2019c). According to Andreas Schleicher, the OECD Director of Education and kills, “Portugal is Europe's biggest success story at PISA” (Tavares 2017). This chapter briefly reviews both the Portuguese education system, the Portuguese educational reforms driven by PISA, and its effects on the PISA national results.
2 The Portuguese Education System
The Education pace in Portugal, from the beginning, was set by the Catholic Church with the predominant action of the Jesuits Order. The 1826 Constitutional Bill of Rights was the first official document setting the trend for the gratuity of a primary education focusing on reading, writing, and mathematics (MEC-OEI 2003; Ramos 2004). The implementation of a republican regime in 1910 led to the expulsion of religious orders from Portugal and brought the first republican education reform. This reform was especially aimed at improving the very low literacy rates of the population, namely that from the rural areas, with emphasis on the importance of reading at an early age (Candeias et al. 2007; MEC-OEI 2003).
Post-WWII reforms in Europe found the Portuguese education systems still lagging behind its European neighbors (Justino 2010). Salazar’s fascist regime promoted an inward-looking education system that focused on the Portuguese mainland and its African colonies and very little exposure to external influences. It was not until 1960 that the 3 or 4-year schooling mandatory for all children (girls and boys, respectively) was imposed. By that time, the OECD’s sponsored Mediterranean Regional Project set the first attempt at aligning the Portuguese education with the international education frameworks as part of an effort to meet economic dynamic growth needs (Alves 2012; Mendonça 2011). The Mediterranean project found a country largely deprived of education. In 1970, 18% of the Portuguese population was illiterate, 66% of 15-year olds had not completed any level of formal education, and only 0.9% of the total population had a higher education degree (Crato 2020). A large set of reforms were therefore proposed, in the early 1970s, to all levels of basic, secondary and university studies. Amongst the introduced measures was the mandatory attendance of school for at least eight years. However, the 1974 military coup that ended the almost 50 years of fascist regime aborted these reforms and set the country into revolutionary fervor. A major reorganization of schooling cycles and curricula reforms occurred at a furious pace according to social constructivist views of education. In 1986, a new “Basic Law of the Educational System” ensured the right of education and culture to all children promoting the training required for active citizenship, equality of opportunities, and freedom of learning and teaching. This reform also expanded mandatory schooling to nine years. Vocational and professional tracks aimed at accessing a profession or higher education were introduced in parallel with the regular sciences and humanities tracks during the late 1990s, early 2000s. The mandatory exams that were abolished during the years following the 1974 Portuguese revolution were slowly reintroduced after 1996 for grade 12 certifying the terminus of secondary education and, from 1998 on, ranking the students’ access to higher education. National high-stake Portuguese language and Mathematics exams were introduced for grade 9 in 2005, and in 2009 the Parliament decided to extend mandatory schooling to 12 years of basic (grades 1–9) and secondary (grades 10–12) education. This was put in place gradually in the 2012–2015 period. Universal access to preschool for 5-year old children, but not mandatory enrollment, was also introduced in 2009. In 2015, the preschool age group was extended to the 4-year olds.
3 The Portuguese PISA Trends
4 PISA’s Education-Driven Policies
Portugal’s policymakers have looked at international assessments of educational systems since the first OECD Mediterranean Regional Project diagnosis in the late 1960s. Before PISA, Portugal participated in the Second International Assessment of Education Progress (IAEP II 1991) and IEA’s Third International Mathematics and Science Study (TIMSS 1995). While IAEP II went relatively unnoticed, TIMSS 1995 was the first large scale comparative assessment that showed Portuguese students considerably lagging behind similar age peers from the 26 countries who sampled 4th and 8th-grade students. At that time these very poor results were dismissed because policymakers felt that TIMSS was not a valid measure of Portugal’s students’ specific knowledge and skills that were not aligned with the TIMSS curricula framework (Barroso 2010; Carvalho et al. 2017). Despite the TIMSS 1995 insights being unfavorably received, the 1995 large scale assessment set the seed to assessment policy changes and mathematics and science curricular reforms. It was also a turning point in the acknowledgement of the need to not only further assess Portugal’s educational system according to international standards but also to pay more attention to results in basic subjects (Crato 2020; Marôco 2020).
As an OECD member, Portugal participated in the first edition of PISA (2000) all the others that followed. In the words of the minister of Education Nuno Crato, in office from 2012 to 2015, PISA in Portugal is mainly “seen as a mirror” reflecting where the country stands in comparison to other countries in the PISA picture (Carvalho et al. 2017) but has nevertheless allowed education policymakers to propose evidence-based policies changes as follows.
The PISA 2000 debacle and the publication of its results in 2001 set the stage for the endorsement of a series of ongoing measures aimed at out of class students support (accompanied studies) and the reformulation of upper secondary curricula by the minister of education Júlio Pedrosa (Carvalho 2009; Carvalho et al. 2017). The next minister, in office from 2003 to 2004, David Justino, implicitly recognized the poor PISA and TIMSS results to promote the re-emergence of national assessments in Portugal, first as a low-stakes, in 2003, and, in 2005, as high-stakes exams for mathematics and Portuguese language at the end of grade nine (see Justino and Almeida 2017 for further details). The next explicit mention of PISA was done by Minister Carmo Seabra in an address to the national parliament based on “the last OCDE clear results” (Afonso and Costa 2009). Carmo Seabra brought the PISA 2003 results to the political agenda to promote curricular changes and the need to prioritize the learning of the Portuguese language, mathematics, and science.
The next education minister, Maria de Lurdes Rodrigues, in office from 2005 to 2009, identified PISA as a major source of statistical data on Portuguese students’ literacy and its importance to support evidence-driven policies. The evidence provided by PISA 2000 and 2003 that Portuguese students were performing poorly in terms of reading, mathematics and science literacies drove the introduction of a series of programs and strategic plans consolidating educational policies started in the early 1990s after the publication of the 1986’s Basic Law of the Educational System (Fernandes et al. 2019; Fernandes and Gonçalves 2018). These included the Training Program in Experimental Science Teaching (2006), the National Program for Portuguese Language Teaching (2007), aimed at primary education, the Mathematics Action Plan (2006), and the National Reading Plan (2007) (Afonso and Costa 2009; Carvalho et al. 2017). During her tenure, Lurdes Rodrigues quoted the association between poor results in PISA and families’ cultural and socioeconomic status to enlarge the economic support for students from low-income families; to facilitate the access to internet and computers for primary education (the 2007 Technological Education Plan); and to reorganize the Priority Intervention Educational Territories (TEIP) Program for schools located in economically depressed areas (Afonso and Costa 2009).
5 What May Explain the Portuguese Evolution in PISA
The concurrent validity of the PISA 2015 mathematics results with the Portuguese national exams and its trends have been demonstrated elsewhere (Marôco 2018; Marôco and Lourenço 2017) and thus it is possible to safely use PISA as a reasonable proxy for the Portuguese education system. However, it must be acknowledged that PISA is a correlational study. PISA uses a complex test design and statistical methods to impute students’ missing by design responses (see e.g., OECD 2009). In each PISA cycle, a different cohort of students is sampled, and trends are estimated from items that are common to two or more editions of the test. Henceforth, although causal inferences may be suggested by PISA data, there is no way to ensure that the data support causal effects since correlation does not imply causation. Furthermore, the PISA test has also drawn criticism from several sources both on the importance of the limited subjects covered by the test for students’ and economies’ development (see e.g., Schult and Sparfeldt 2016), to the lack of transcultural invariance when comparing countries’ results (Rutkowski and Svetina 2014). Despite not being free from criticism, PISA is generally accepted by policymakers and the public in general as a valid and reliable instrument to benchmark the performance of education systems and facilitate education reforms both abroad (Breakspear 2012; Phillips and Jiang 2015) and locally (Carvalho et al. 2017; Fernandes et al. 2019; Justino 2010). A lag between policy changes and results observed in the PISA test as well as cumulative effects must also be considered when linking policies with PISA results. And again, inferring causality from correlation may just be a form of statistical fantasy.
Portuguese students’ performance has improved significantly in the PISA test and PISA outcomes have supported Portuguese education policy changes. In every edition and following the release of the PISA results, commentators, from journalists to academics to policymakers, profusely give their accounts of what causes the evolution of Portuguese results. Carvalho et al (2017), regarding PISA 2015, reviewed all the opinion articles published after the public release of the PISA 2015 report. As far as education policies are concerned, a consensus emerged about the causal effects of the extension of pre-school education, differentiation of pedagogical practices, improvement of schools infrastructures, a culture of ‘exigency’ supported on the reinforcement of curricula aligned with international frameworks, high-stake exams, and increasing offer of vocational/professional courses targeted at students with a lesser interest in the regular track. Although, for some (see e.g., Fernandes et al. 2019, p. 42) these PISA effects on policy were just a reflection and a continuation of the education policies set almost 15 years before PISA by the 1986’s Basic Law of the Educational System. Ferreira et al. (2017) looking at PISA results from to 2000 to 2015, with a major emphasis on PISA 2012, have identified, as follows, the principal features that explain Portugal’s evolution: (1) the overall expenditure in education per capita in line with other OECD member states, in spite of Portugal being a relatively poor country in OECD terms [the 2018 GDP per capita for Portugal was 32.4 k USD versus 43.5 k USD for the OECD average (OECD Stat 2019.)]; (2) Pre-school coverage close to 100%; (3) Teachers appropriate specific and pedagogical training, competence and motivation towards teaching; (4) Students’ support by parents and teachers, motivation and persistence; (5) Schools in less favored economic regions performing above the expectation and schools’ educational projects aligned with the community; and (6) Improvement in Parents’ education.
However, PISA also shows that there is still much need for improvement in the Portuguese education system. At system level, there is an urgent need to promote measures aimed at the reduction of grade retention, increase parental education, renew aging teachers, and improve schools’ autonomy, especially as far as teacher recruitment is concerned (Ferreira et al. 2017). It is noticeable that despite the economic crisis of 2008–2013, when national GDP was reduced by 8% (Perez and Matsaganis 2018) and the overall negative evolution of the education expenditure (−0.07% of GDP per year from 2000 to 2018), Portugal was still able to increase its overall PISA scores. Indeed, the correlation between the Portuguese expenditure in education (as % of GDP, see Fig. 5) and the PISA results from 2000 to 2018 is r = −0.72. The same correlation for the OECD is r = 0.50. At student level, secondary analysis of data from PISA 2105 (Marôco 2017) as well as PISA 2018 (Gomes et al. 2019; Marôco 2019) reveals that students expectations on their future occupation and the families’ socioeconomic and cultural status are still major determinants of the Portuguese students’ performance.
6 Concluding Remarks
Like in any other country and economies that take part in PISA, the Portuguese media, the public, educators, and policymakers accept PISA as a robust and legitimate proxy for the Portuguese education system. Although the bottom-of-the-table PISA 2000 results were not received in Portugal with as much “shock” as they were in other poor performing countries (e.g. Germany), PISA has nevertheless produced data and evidence that has been used by the Portuguese education policymakers to justify and promote reforms at different levels of the system. First, and foremost, the poor PISA results were the evidence required to promote the always controversial curricula restructuration in key disciplinary areas, namely Portuguese language, mathematics, and natural sciences, both at the basic and secondary education levels. National programs were aimed at the promotion of reading habits and the increase of time for teaching Portuguese language and mathematics. Teachers’ requirements and training, as well as increased teaching times, were also promoted based on comparisons with other PISA participants. Although no causal effects of policies motivated by PISA can be undoubtedly defended due to the correlational nature of the study, Portuguese students’ improvements in PISA were aligned with some key education policies changes. The biggest jump in the Portuguese PISA results was observed from 2006 to 2009 and the temporal coincidence with the introduction of the 9th-grade mathematics and Portuguese language exams in 2005 is undeniable. Also, regarding the evolution of PISA from 2015 to 2018, Science was the only subject with a statistically significant drop, coincidently this is the PISA domain that does not have a high stake national assessment. The effect of high-stake assessments on PISA scores has also been observed in several other countries (Bergbauer et al. 2018). The introduction of high-stake assessments in the Portuguese system is probably the policy with a larger effect on the Portuguese PISA story. Strengthen of curricula, learning targets, and structural teaching and class changes in response to PISA may also play an important role, and those have been consistently pursued by both socialist and social democrat ministers up to 2015 (see Fig. 4). This has, however, changed with the last cycle of governance which brought the extinction of grade four and six national exams and the teacher screening exams. The high-stake exams for grades four and six were exchanged by low-stake diagnostics tests for grades two, six and eight in Portuguese language, mathematics and several other rotating subjects. The new minister also introduced curricular flexibility and curricula trimming to “essential learning targets” at public schools, as well as the continuity of measures aimed at the reduction of grade retention to the OECD levels. A short-term effect of these policies may well just surface in the PISA 2018 results. The devaluation of external high-stake assessments and the suggestion for trimming of learning targets may reduce the effort and engagement of students with low-stake tests like PISA. Indeed, three out of four Portuguese students reported expending less effort on the PISA test than if the test counted towards their marks, the same figure for the OECD was 68% (OECD 2019c). Also, and for the first time, the participation rate of the Portuguese students (76%) was below the PISA standard of 80% (OECD 2019c).
One recurrent criticism of PISA effects on education is the funneling of school’s subjects to the PISA domains—see, e.g. the open letter from academics from all over the world to Dr. Andreas Schleicher published by The Guardian in May 2014 (Various 2014). The new education policies in place since 2016 acknowledge this and other criticisms. According to minister Tiago Brandão Rodrigues, in office since late 2015, “PISA recommendations are embodied in the current Government’s program” (Bourbon 2016). Its major effects will, however, only be seen in the next PISA cycles, once the policy-lag effect is overcome.
Despite the praised evolution of the Portuguese students in PISA, a trend with no companion in the OECD, PISA reveals that student performance is strongly asymmetric within the country. An equivalent of two school years separates the highest and the lowest-achieving regions of the country. PISA also shows that schools have failed consistently to serve as social elevators. Students’ expectations and families’ socioeconomic social status are the major determinants of Portuguese students’ results. These effects have been present in all PISA editions, including the last.
From an epistemic lag in the last century, the Portuguese education system has raised to the level of its OECD counterparts as measured by PISA. Education requirements are changing at a pace faster than ever before, and education policies are changing in accordance to meet the need for the so-called XXI century skills. In the coming waves, PISA will tell us whether Portugal is still moving in the right direction.
- Afonso, N., & Costa, E. (2009). The influence of the Programme for International Student Assessment (PISA) on policy decision in Portugal: The education policies of the 17th Portuguese Constitutional Government. Sísifo. Educational Sciences Journal, 10(set/dez), 53–64. https://sisifo.fpce.ul.pt.
- Alves, L. A. M. (2012). História da Educação - Uma Introdução. https://ler.letras.up.pt/uploads/ficheiros/10021.pdf.
- Barroso, C. F. de A. (2010). Os estudos PISA e o ensino das ciências físico-naturais em Portugal: a comparabilidade dos resultados nacionais e as implicações para a política educacional. [Universidade Nova de Lisboa]. https://run.unl.pt/bitstream/10362/5279/1/carlosbarroso.pdf.
- Bergbauer, A. B., Hanushek, E. A., & Woessmann, L. (2018). Testing. In NBER working paper (Vol. 24836). https://doi.org/10.1021/ja01163a101.
- Bourbon, M. J. (2016). Recomendações do PISA estão plasmadas no programa do atual Governo. Expresso. https://expresso.pt/sociedade/2016-12-06-Recomendacoes-do-PISA-estao-plasmadas-no-programa-do-atual-Governo.
- Breakspear, S. (2012). The Policy Impact of PISA: An exploratation of the normative effects of international benchmarking in school system performance. OECD Education Working Papers, 71, 1–31. https://doi.org/10.1787/5k9fdfqffr28-en.
- Breakspear, S. (2014). How does PISA shape education policy making? Why how we measure learning determines what counts in education. Centre for Strategic Education: Seminar Series, 240, 16. https://simonbreakspear.com/wp-content/uploads/2015/09/Breakspear-PISA-Paper.pdf.
- Candeias, A., Paz, A., & Rocha, M. (2007). Alfabetização e escola em Portugal nos séculos XIX e XX: os censos e as estatísticas. Fundação Calouste Gulbenkian.Google Scholar
- Carvalho, L. M. (2009). Pisa and educational public policies: studies in six European countries. Sisifo Educational Sciences Journal, 10, 3–84. https://sisifo.fpce.ul.pt/pdfs/Journal10ENG.pdf.
- Crato, N. (2020). Curriculum and educational reforms in Portugal: An analysis on why and how students’ knowledge and skills improved. In F. M. Reimers (Ed.), Audacious Education Purposes: How governments transform the goals of education systems. Spring.Google Scholar
- Cristo, A. H. (2017). Os três problemas dos exames nacionais. Observador. https://observador.pt/especiais/os-tres-problemas-dos-exames-nacionais/.
- Eurydice. (2019). Portugal. Retrieved December 18, 2019, from https://eacea.ec.europa.eu/national-policies/eurydice/content/portugal_en.
- Fernandes, D., & Gonçalves, C. (2018). Para Compreender O Desempenho Dos Alunos Portugueses No PISA (2000–2015). In Maria Isabel Ramalho Ortigão (Ed.), Políticas de Avaliação, Currículo e Qualidade: Diálogos sobre o PISA – Volume 3 (pp. 39–68). EDITORA CRV. https://doi.org/10.24824/978854442369.1.
- Fernandes, D., Neves, C., Tinoca, L., Viseu, S., & Henriques, S. (2019). Políticas educativas e desempenho de Portugal no PISA (2000–2015). https://afc.dge.mec.pt/sites/default/files/docs/RelatorioFinal_PolitEdu_PISA2000_2015.pdf.
- Ferreira, A. S., Flores, I., & Casas-Novas, T. (2017). Porque Melhoraram os Resultados PISA em Portugal. Estudo Longitudinal e comparado (2000–2015). Fundação Francisco Manuel dos Santos. https://www.ffms.pt/FileDownload/9857244f-4dfb-48ad-b196-0448dc444865/porque-melhoraram-os-resultados-pisa-em-portugal.
- Gomes, J. F., Hormigo, I., & Marôco, J. (2019). Primeiras lições do PISA: dois países a seguir e Portugal no mundo. Teresa e Alexandre Soares Dos Santos Iniciativa Educação. https://www.iniciativaeducacao.org/pt/ed-on/ed-on-estatisticas/primeiras-licoes-do-pisa-dois-paises-a-seguir-e-portugal-no-mundo.
- Justino, D. (2010). Difícil é Educá-los. Fundação Francisco Manuel dos Santos.Google Scholar
- Justino, D., & Almeida, S. (2017). International assessment, curriculum policy induction and instruction time management. European Journal of Curriculum Studies, 4(2), 671–691. https://pages.ie.uminho.pt/ejcs/index.php/ejcs/article/view/162/94.
- Marôco, J. (2017). Assimetrias Educacionais em Portugal: Através das Lentes do PISA. In Conselho Nacional de Educação (CNE) (Ed.), Estado da Educação 2016 (pp. 254–274). Conselho Nacional de Educação (CNE). https://www.cnedu.pt/content/noticias/CNE/CNE-EE2016_web.pdf.
- Marôco, J. (2018). E pur si muove! – Observador. https://observador.pt/opiniao/e-pur-si-muove/.
- Marôco, J. (2019). PISA 2018: Portugal, é ainda a maior história de sucesso na Europa? Teresa e Alexandre Soares Dos Santos Iniciativa Educação. https://www.iniciativaeducacao.org/pt/ed-on/ed-on-estatisticas/pisa-2018.
- Marôco, J. (2020). International large-scale assessments: Trends and effects on the portuguese public education system. In H. Harju-Luukkainen, N. McElvany, & J. Stang (Eds.), Monitoring student achievement in the 21st century european policy perspectives and assessment strategies. Springer. https://www.springer.com/gp/book/9783030389680#aboutBook.
- Marôco, J., Gonçaves, C., Lourenço, V., & Mendes, R. (2016). PISA 2015 - Portugal. Vol. I: Literacia Científica, Literacia de Leitura & Literacia Matemática. Lisboa: IAVE.Google Scholar
- Marôco, J., & Lourenço, V. (2017). Search of concurrent validity of large-scale international students assessments (LSISA) and high-stakes national exams: the portuguese case study. Paper presented at the 2017 European Conference on Educational Reaserch: Network 9 Assessment, Evaluation, Testing and Measurement, 126. https://www.iave.pt/images/FicheirosPDF/Estudos_Internacionais/ECER2017_InSearchOfConcurrentValidity.pdf.
- MEC-OEI. (2003). Breve Evolução Histórica Do Sistema Educativo. https://www.oei.es/historico/quipu/portugal/historia.pdf.
- Mendonça, A. (2011). Evolução da Politica Educativa em Portugal. Retrieved September 20, 2018, from https://www3.uma.pt/alicemendonca/conteudo/investigacao/evolucaodapoliticaeducativaemPortugal.pdf.
- OECD.Stat. (2019). Level of GDP per capita and productivity. https://stats.oecd.org/index.aspx?DataSetCode=PDB_LV.
- OECD. (2001). Knowledge and skills for life: First results from the OECD Programme for International Student Assessment (PISA) 2000. OECD Publications. http://www.oecd.org/edu/school/programmeforinternationalstudentassessmentpisa/33691596.pdf.
- OECD. (2009). PISA data analysis manual: SPSS. In Analysis (2nd edn). OECD. https://doi.org/10.1787/9789264056275-en.
- OECD. (2016). Pisa 2015 results: Excellence and equity in education: Vol. I. OECD Publishing. https://doi.org/10.1787/9789264266490-en.
- OECD. (2019a). 2018 Database—PISA. https://www.oecd.org/pisa/data/2018database/.
- OECD. (2019b). Education at a Glance 2019 (OECD Indic). OECD Publishing. https://doi.org/10.1787/f6dc8198-es.
- OECD. (2019c). PISA 2018 results (Volume I): What students know and can do: Vol. I (O. Publishing (ed.)). https://doi.org/10.1787/5f07c754-en.
- Phillips, G. W., & Jiang, T. (2015). Using PISA as an international benchmark in standard setting. Journal of Applied Measurement, 16(2), 161–170. https://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2015-26021-004&lang=pt-br&site=ehost-live&scope=site.
- PORDATA. (2019a). Government expenditure on education as a share of GDP. https://doi.org/10.1787/9789264301900-graph65-en.
- PORDATA. (2019b). PORDATA—Actual schooling rate. https://www.pordata.pt/en/Portugal/Actual+schooling+rate-987-7864.
- PORDATA. (2019c). PORDATA—Education. PORDATA. Base de Dados de Portugal Contemporâneo. https://www.pordata.pt/en/Theme/Portugal/Education-17.
- Schult, J., & Sparfeldt, J. R. (2016). Reliability and Validity of PIRLS and TIMSS. European Journal of Psychological Assessment, 1–12. https://doi.org/10.1027/1015-5759/a000338.
- Tavares, P. S. (2017). Andreas Schleicher: Portugal é a maior história de sucesso da Europa no PISA. Diário de Notícias. https://www.dn.pt/portugal/interior/andreas-schleicher-portugal-e-a-maior-historia-de-sucesso-da-europa-no-pisa-5659076.html.
- Various. (2014). OECD and Pisa tests are damaging education worldwide—academics. The Guardian, 06.05.14. https://www.theguardian.com/education/2014/may/06/oecd-pisa-tests-damaging-education-academics.
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.