Introduction

This research paper outlines the development of a Theory of Change (ToC) to shape child-focussed Science, Technology, Engineering, and Mathematics (STEM) interventions of an Outreach Project in the North East of England. The aim of the project is to increase the diversity and number of young people choosing further study and a career in STEM.

The ToC developed identifies how children, young people and their teachers and families can be engaged to increase the diversity and number of young people choosing STEM careers. The development process for the ToC, and the ToC itself, is presented, together with brief case studies to illustrate the creation of different interventions using the ToC. Finally, the implications for policy and practice for other organisations active in STEM education are discussed.

Background

Governments and industry across the globe have been considering diversity issues in STEM around gender, race and socio-economic status for over 40 years, often expressed in terms of the productivity and economic cost to each country’s economy (see, for example, Noonan (2017)). Numerous other reports have focused on the uptake of STEM by women (Greenfield 2002; DfES 2004; Masanja 2010). There are also issues of under-representation in STEM around race (National Science Foundation 2017) and socio-economic status (Chetty et al. 2017; HESA 2018).

There have been many attempts to rectify this lack of diversity in STEM in the UK, focussed on altering school curricula, making STEM more attractive to women and other under-represented groups, and improving career advice. Whilst the number of young people studying some STEM subjects at university has increased, limited progress has been made on changing the diversity of the young people interested in a career in STEM, particularly outside of the biological sciences.

The continuing lack of diversity of young people engaging in physical sciences, technology and engineering study suggests that the standard narratives and solutions for increasing uptake and diversity of STEM careers are not working. Archer et al. (2015) introduced the term “science capital” to describe a number of factors correlated with expressions of interest towards science careers in young people, including science-related attitudes, values and dispositions, knowledge about the transferability of science, talking about science in everyday life, and knowing people in science-related jobs. Young people with high levels of science capital are more likely to express a desire for a future science career (DeWitt et al. 2016).

A “wicked problem” (Rittel and Webber 1973) is one that has many causes which are interlinked and consequently does not have one single, simple solution. Increasing the number and diversity of young people choosing a STEM career is one such “wicked problem”. For example, whilst the gendered subject choices of young children have their roots in the individual’s experience of learning in school, subject choice also depends on the gender socialisation they have experienced through their families, the media they consume and their role models. As a consequence, any solutions which are posited will also need to be wide-ranging and address the breadth of identified causes.

The aim of this paper is to explore the research question: What are the affordances of a ToC for increasing the diversity and number of young people choosing a career in STEM post-18? Additionally, the following sub-question will also be addressed: What are the affordances of a ToC for shaping the design of activities aimed at supporting young people to choose a career in STEM?

Theoretical Underpinnings

In order to increase the number of young people choosing a career in STEM, those young people will need to exhibit specific behaviours at certain ages: at age 16+ they need to choose study options which will allow them to continue in STEM (A-levels and/or vocational qualifications) and at 18+ they need to choose career options (either further study, apprenticeships or work) within a STEM sector. Changing behaviour and attitudes in people is challenging (Institute for Government 2010). Ajzen (1985) proposed that the intention to perform a behaviour can be used as an accurate predictor of whether a person will actually perform the behaviour. These intentions are informed by a person’s beliefs about success and failure and the subjective norms (attitudes towards the behaviour) of people that s/he considers significant. This Theory of Planned Behaviour can therefore be used as a theoretical basis to explore the connection between a young person’s intention to exhibit a particular career behaviour at the ages of 16 and 18 and the attitudes, subjective norms and perceived behavioural control that influence the behaviour.

Stakeholders in STEM Career Choices

Children’s career choices are influenced by family members, teachers and careers advisors (Wellcome Trust 2013a). These key influencersFootnote 1 help shape the subjective norms related to particular career-related behaviours.

Children and Young People

Using “science capital” as a lens to understand young people’s choice of science, and by extension STEM careers, DeWitt et al. (2016) suggest that there is a need to move the narrative away from inspiration and towards showing the application of science to increase science participation.

For young children, career interests are relatively fluid (Helwig 2003); however, from about ages 5–6, they start to make career-limiting decisions about what they do not want to do (Gottfredson 2005; Bian et al. 2017). The factors that influence these decisions include perceived gender-appropriateness of careers, social level of careers and accessibility (Chambers et al. 2018), as well as the young person’s concept of their own ability (Nagengast and Marsh 2012).

Teacher Influence at Primary and Secondary SchoolFootnote 2

As young people become more sophisticated in thinking about their careers they begin to seek information about future possibilities from a range of different sources (Howard and Walsh 2010). Teachers are considered one of the most used, and useful, source of careers information by young people (Wellcome Trust 2013a).

The majority of children in primary schools in England are taught science and maths by teachers who do not have an advanced science or maths qualification (Wellcome Trust 2013b) and may also have limited/stereotypical views of STEM and people who work in STEM (Breiner et al. 2012).

Science teachers in secondary school may have a subject specialism but are also likely to be teaching all three sciences (biology, chemistry, and physics) to pupils aged 16 and below. Thus, even specialist teachers, when teaching “out of field” may have areas of the curriculum they are less confident to teach (Hobbs 2013), and this in turn can affect how young people in their classes view those subject areas (Salleh and Darmawan 2013).

Teachers can also affect children’s and young people’s career choices through stereotyping and unconscious bias. These can have detrimental effects in the classroom, particularly through teachers’ interactions with children (Van den Bergh, et al. 2010; Lavy and Sand 2015) and their expectations of student achievement (Tan et al. 2013).

The Importance of Parents

Family members are seen by young people as their most important source of careers information (Wellcome Trust 2013a), and so, parents and carersFootnote 3 are the third group of stakeholders in young people’s career choices. The association between parental involvement and a child’s educational achievement is well established (Goodall et al. 2011) with activities that promote conversations about school experiences in the home being directly correlated with children’s achievement in school (Desforges and Abouchaar 2003).

Alongside the effects of previous negative science-related educational experiences or lack of confidence, bias (conscious or unconscious) can also affect the level of educational support that parents provide to their children, and the career aspirations that they may consider appropriate. Gender is one of the strongest factors to affect a child’s development in any society (Bem 1993), and the social and career roles considered appropriate for a child often depend on the perceived gender of the child (Miller and Hayward 2006). Parents of daughters are less likely to believe that their child is interested in science and that science is more difficult for their child than parents of sons (Tenenbaum and Leaper 2003), and parents can share gender-stereotyped views of occupations with their children (Ikonen et al. 2017).

Addressing the “Wicked Problem” of Diversity in STEM

Any effective solution must be able to address the complexity and long timescales involved and treat the “wicked problem” of diversity in STEM, and its solution, holistically. The ToC described in this paper is robust enough to accommodate and explicate how different stakeholders can be engaged to increase the number and diversity of young people choosing a STEM career. The strategy outlined encompasses children from the ages of 2 to 18 years alongside their key influencers, parents and teachers, and provides a strategy for solving the “wicked problem” in the long term.

Methodology

Outreach Project Background

The aim of the Outreach Project was to increase the number of young people choosing to study physics, and physics-related degrees (including those in engineering and technology), with particular focus on increasing diversity through greater participation by females and people from lower socio-economic backgrounds. Previous efforts to achieve this aim have been focussed on secondary school pupils and have met with limited success. The project therefore chose to work with children and young people in primary schools, as well as secondary schools. The Outreach Project worked with around 30 partner schools covering the age range from 2 years to 19 years old and provided ongoing interactions with children and young people, as well as their teachers and families, initially for three years. The project was a partnership of 10 organisations, including local authorities, visitor attractions, STEM organisations and the university where the outreach team was situated. The majority of the schools engaged in the project were in areas of deprivation.Footnote 4

The broad age range of children and young people involved in the Outreach Project means that evaluating the impact of the project in relation to its stated aim is not possible for all participants because after three years, children in primary school will not be at a point of career decision making. Dyson and Todd (2009, p. 124) note that ToC evaluations “rely on predicting what outcomes might emerge as much as identifying outcomes that are already apparent. Outcomes in ToC evaluations are conceptualized as materializing at the end point of a change of intermediate changes which the evaluation seeks to track.”

This feature means that a ToC approach is particularly suitable for evaluating the outcomes of interventions in complex contexts, such as education, or in situations where the outcomes emerge after the completion of the intervention (Dyson and Todd 2009) as in the case for the Outreach Project.

Theory of Change

ToC approaches were initially developed in the US as a way of evaluating complex community initiatives (Sullivan and Stewart 2006) but have been used in the UK as a way to evaluate policy initiatives, such as Full Service Extended Schools (Dyson and Todd 2009).

Developing a ToC involves “a systematic and cumulative study of the links between activities, outcomes and context” (Connell and Kubisch 1998, p. 16) and provides “an overarching framework for understanding, systematically testing and refining the assumed connections (i.e., the theory) between an intervention and the anticipated impacts” (HM Treasury 2011, p. 57). They can be particularly useful in the evaluation of complex interventions where it is difficult to identify or track the endpoint outcomes of the intervention (Connell and Kubisch 1998; Dyson and Todd 2009). Once the first step of identifying the final impact or change that the programme or intervention is intended to bring about is completed, a process of backward mapping is undertaken and intermediate outcomes that are required to achieve this goal are articulated. This mapping process helps to surface the explicit or implicit theories that are held by those involved in developing the intervention. Intermediate outcomes may be short-, medium- or long-term. Together, the outcomes will create a causal pathway which supports the final goal of the programme (Taplin et al. 2013). A ToC is both a process and a product (Vogel 2012); therefore, an iterative approach can be helpful.

Process Used to Develop the ToC

The Outreach Project was a multi-year intervention, with an intended long-term evaluation of children’s qualification choices planned using the National Pupil DatabaseFootnote 5 which will take place ten years after the start of the project in primary schools. However, evaluation of the project on a short-to-medium-term timescale was also required. Rather than relying on short-term evaluation of individual activities, a ToC approach was chosen to allow the evaluation to be clearly linked to the project’s long-term aim through a chain of intermediate outcomes which are more amenable to tracking and evaluation (Dyson and Todd 2009).

At the beginning of the Outreach Project, a simple model was produced outlining the journey of a child through different activities, ages and stages, along with complementary activities for key influencers (Fig. 1). This was a first step in development of a ToC and enabled the development of a narrative understanding of the expanse of the STEM ecosystem in which children make decisions about careers. However, it did not allow for a clear elucidation of the behaviour changes and linked subjective norms required to lead to an increase in young people choosing a STEM career. Creating and using a detailed ToC allowed the development of a layered series of outcomes encompassing short-, medium- and long-term time-scales and guided the level and nature of evaluation of the intermediate stages.

Fig. 1
figure 1

Simple diagram showing the educational journey of a child and their key influencers, and the changing nature of activities during that time

Full size image

The ToC was developed through an iterative series of workshops with members of the Outreach Project team and other academic staff within the university. Backward mapping was used to clarify the steps required to attain the overall aim of the project: increasing the number and diversity of students choosing a career in STEM post-18 (see bottom of Fig. 2).

Fig. 2
figure 2

ToC showing short-, medium-, and long-term outcomes linked to increasing diversity in STEM

Full size image

The backward mapping process began with a workshop which involved the core delivery team of the outreach project and two academic staff from the university. Firstly, key stakeholders with an interest in the project aim were identified: these included children and young people, teachers and schools, parents and families, companies in different STEM sectors (both locally and nationally), further and higher education institutions, and government. However, the project team realised that aiming to work directly with all of these stakeholders was unrealistic. It was therefore important to narrow the range of stakeholders targeted. The choice of key stakeholders is supported by the Theory of Planned Behaviour (Ajzen 1985) and the importance of subjective norms on the intention to try a particular behaviour. Children and young people are the ones whose behaviour we are aiming to change, but parents and teachers are the referents (significant others) that strongly influence the subjective norms, and whose views children and young people are (usually) motivated to comply with. The more a child believes that parents and teachers think they should exhibit a behaviour, then the stronger the subjective norm towards that behaviour will be. Therefore, it was important that the key stakeholders described by the ToC would allow influence and change to be effected on those subjective norms. For this reason, in addition to children and young people, the project team identified teachers and parents/carers as the stakeholders that were key to the impact of the project in the North East. This narrowing of stakeholder focus also allowed the team to identify the level of evidence at which impact could be measured (Kazimirski and Pritchard 2014).

Having identified the key stakeholders, the group then started with the aim and worked backwards in time to consider what attitudinal, behavioural and structural changes for those stakeholders would need to occur to achieve the aim—the backward mapping .

This was an iterative process. The group first worked individually to identify the interim changes required for each group of stakeholders, writing each potential change on a post-it note. The choice of changes was informed by previous examination of the research literature, and also professional expertise. This identification process resulted in a large number of possible changes written on post-it notes. Two members of the group then worked together to group the notes by theme for each stakeholder group and to categorise them into long-, medium- and short-term changes. The validity of the groupings and categorisation were then discussed by the group as a whole, and changes were made to the organisation until consensus was reached.

The discussions identified long-term outcomes for the project as shown in Fig. 2.

For children and young people, two long-term outcomes were identified: “Increased confidence in ability to study STEM post-16”Footnote 6 and “Increased number choose to study A-level or vocational qualification in STEM subjects”. These two outcomes link to the changes in behaviour required at age 16 to enable young people to progress into STEM careers at age 18 (project aim). Improving a young person’s attitude towards success through increased confidence will increase the likelihood of them choosing to exhibit the desired behaviour (Ajzen 1985).

For schools, the long-term outcome “School environment mitigates effects of bias and stereotypes” was identified as important, and for parents and carers, the long-term outcome identified was “Parents and carers support and encourage STEM career choices for their children”.

To achieve these longer term aims, other medium-term changes were required. These changes were categorised to create a number of medium-term outcomes for each stakeholder group which were those that were expected to develop as a consequence of repeated interactions or which were time-critical to the school year or maturity of the children and young people.

A final round of mapping took place to identify short-term outcomes which fed into the medium-term outcomes, and the overall ToC diagram was created (Fig. 2). There are often many-to-many relationships between the different outcomes. This is due to the complex nature of the system within which the project is situated. The project team used the categorisation of short-, medium- and long-term outcomes to identify causal chains which linked the short-term outcomes with the long-term outcomes. Figure 3 shows an example of one such causal chain taken from the ToC.

Fig. 3
figure 3

Example of a causal chain in the ToC linking together the short-, medium, and long-term outcomes

Full size image

Having created a first draft of the ToC, the outreach team then audited a number of the interventions that had already been developed and delivered in schools against the draft ToC. This audit was focussed on the following two questions: (a) does the ToC accommodate the intervention and (b) does the ToC have something to say about the value of that intervention. The audit identified a number of causal links that had not been included in the initial mapping (for example, including an explicit link between medium-term outcomes for the family and child stakeholder groups), which were then added. The audit also caused the outreach team to change the focus of some of the interventions, with the activities becoming more explicitly careers-centred as a result.

After the draft was produced, each outcome was cross-referenced to relevant research literature to ensure that the ToC was supported by prior research (see the Supplementary Material S1 for an overview of supporting research literature).

To further increase the confidence in, and trustworthiness of, the draft ToC, it was shared and discussed with the advisory and management bodies of the Outreach project. These groups included representatives from formal and informal education, industrial and charitable groups. The comments from these discussions were then used to finesse the ToC.

Developing Interventions Using the Short- and Medium-Term Outcomes

A number of case studies are presented in Table 1 which illustrate interventions that have been developed using the ToC. These case studies were chosen to represent the breadth of academic STEM subjects included in the Outreach Project overall and provide examples of collaboration between the outreach team and other research-active academics.

Table 1 Case studies of different interventions linked to short- and medium-term outcomes from the ToC
Full size table

The ToC applies to all parts of a child’s educational journey, but the nature of the interventions developed to meet the outcomes change as the child gets older. For children aged between 2 and 5 years old, workshops focus on encouraging children to ask scientific questions and provide opportunities for role play based around different employment sectors. Between the ages of 6 and 11 years, children are introduced to a range of different STEM careers through exploratory workshops with titles, such as the Solar Physicist (solar system and light) and the Mechanical Engineer (gears and simple mechanisms).

At secondary school (age 11–16 years), support is focussed on careers ideas and information through workshops and assemblies, and on sustaining young people’s identification as a “STEM person”, together with the provision of careers-linked subject resources for use in the classroom. Once a young person has chosen to study physics and maths at 15–16 years old, the focus moves to activities aimed at supporting both attainment and self-concept as a “STEM person”, such as research experience weeks, networking events and after-school lecture series. These are designed to reduce the drop-off in STEM aspirations often seen at this age.

To support the short-term outcomes for teachers, CPD sessions about science topics, careers support and unconscious bias are provided to schools. In addition, science coordinators in primary schools are invited to attend a Primary Science Coordinators Forum six times a year where they are supported to strengthen the teaching of science and STEM in their schools.

Families are engaged through after-school workshops, holiday pop-up STEM shops and online materials. These activities challenge the gendered expectations of different careers, through careful activity design and delivery which removed gendered language, images and role models, and normalise scientific enquiry and science conversations for all participants: adults and children.

Discussion and Conclusion

This paper has aimed to answer the following research questions: “What are the affordances of a ToC for increasing the diversity and number of young people choosing a career in STEM post-18” and “What are the affordances of a ToC for shaping the design of activities aimed at supporting young people to choose a career in STEM.”

Using a ToC enabled the identification of causal chains that could realistically lead to a long-term increase in the diversity and number of young people choosing a STEM career and provided the theoretical underpinnings and context for those causal chains. Given the extended time-scale required to achieve this increase, using a ToC provides a way of evaluating the interim steps towards the aim of the project. The case studies provide examples of how the ToC is used in practice to underpin the development of each intervention and outlines the impacts on the young people and/or their key influencers.

One of the affordances of using a ToC process in STEM education and engagement is that it allows the development of solutions to complex problems in a structured way. The ToC developed here provides a mechanism to connect together a number of causal chains which, taken together, provide a plausible solution to increasing the diversity and number of young people choosing a STEM career. Two key aspects of the solution are the identification of the key stakeholders in the ToC, namely, children and young people and their key influencers (parents and carers, and their teachers). It can be seen that the short- and medium-term outcomes are strongly interlinked with each other, highlighting the need for coordination and planning across the STEM engagement sector as a whole.

Developing a ToC for an outreach project provides clarity when developing individual activities, and offers a mechanism through which the desired outcomes can be made explicit for all the stakeholders in the project. The authors have found the development of the ToC presented here to be beneficial in their own practice, in developing strategies towards that practice and in conversation with others. The ToC offers both predictive and diagnostic utility, of individual activities and of an overall programme of work, assisting the authors in recognising where their practice has strengths and limitations. The ToC also encourages a cycle of review and reflection and has provided a route for the Outreach Project team to reflect on their own practice and clearly articulate their vision and aim to others.

As well as identifying causal chains, a ToC takes into account the pre-requisites and assumptions that underlie them, along with the barriers which may work against them (Supplementary Material S2). The Outreach Project aims to increase the number and diversity of people in STEM careers through repeated and ongoing interactions with children and their key influencers. In achieving this aim, a ceiling of accountability can be identified which is the “Level at which you stop using indicators to measure whether the outcomes have been achieved and therefore stop accepting responsibility for achieving those outcomes” (De Silva et al. 2014, p. 5). Ultimately, the longer-term outcomes are beyond the control of the Outreach Project. For example, the long-term outcome Increased confidence to study STEM post-16 is a personal psychological attitude in individuals. This therefore sets the ceiling of accountability for the ToC. Similarly, the ToC does not include industry bodies or companies. For a long-term solution, these organisations will also need to improve their recruitment and retention of staff (Airbus 2018).

However, even with the limitations identified, this approach and ToC have wide applicability across STEM, education and public engagement fields. For example, the ToC has utility across the wider STEM engagement community where activities may be of a shorter timescale or more limited scope than those of this Outreach Project. Using this broad ToC allows a community to identify individual activities that can be developed and aligned within the structure of short- and medium-term goals, yet still contribute to long-term goals. This should help increase their likely effectiveness rather than taking a purely short term view. It also allows identification of potential measures of success for evaluation of activities.

The ToC could also be applied by those responsible for career guidance within school or college settings to ensure they are supporting the long-term goals for the career aspirations and perceptions of those young people.

Ultimately, solving the STEM skills shortage requires a number of nested Theories of Change, each developed by the key actors within the STEM space. These actors include companies, learned societies, governments and charities. The authors recommend that each organisation looking to improve the uptake of STEM careers articulates their expected impact through a ToC, utilising the ToC described in this paper as a guide, and that they openly share their ToC with others. This would have two affordances: it would identify clearly how change is to be achieved, and it would allow a more holistic approach to encouraging young people to enter, and remain, in STEM careers. Working in partnership in this way is essential if the “wicked problem” of diversity in STEM is to be tackled effectively in a realistic fashion.