Introduction

Assessing a students’ knowledge and skills is one of the most challenging tasks that teachers face (Eriksson 2012; Klapp Ekholm 2010; Lindström 2013; Törnvall 2001; Stiggins and Conklin 1992; Black and Wiliam 1998). Nevertheless, being able to assess is an essential skill of the teaching profession as it necessitates devising questions for evaluating students’ learning process and how likely they are to meet the curriculum’s goals of education (Håkansson 2011; Wahlström and Sundberg 2015; Kimbell 1997). However, studies about assessment and teachers’ practice of it are quite limited (Black and Wiliam 1998; Kimbell 2007).

Though most teachers can find support for their assessment practice through a school subject’s identity and teaching tradition (Kimbell 1997), technology teachers lack this historical backing due to the late emergence of the subject of technology (Blomdahl 2007; Hartell 2012; Kimbell 2007). In comparison with more traditional subjects in the school curriculum, technology education can therefore be seen as especially troublesome for the technology teachers’ development of assessment practice. Previous studies have shown that the lack of a tradition complicates technology teachers’ ability to design and evaluate their formative and summative teaching approaches to accommodate the students’ specific levels of understanding (Kimbell 2007; Hartell 2015).

Technology education has been mandatory in Sweden for several decades, but the lack of tradition and identity can be explained also through the subject’s several interchanging contexts as it has been transformed from an industrial and sloyd context to a more integrated part of the natural sciences in the 1980s (Hallström 2011; Hallström et al. 2013; Riis 2013). The commentary material to the national curriculum of 1994 (Lpo94) provided by the Swedish National Agency for Education in the early 1990s showed that several technology teachers felt great uncertainty about the subject, what it represented, and its relevance in teaching (Skolverket 2000). Alleviating this uncertainty was one of the main reasons why technology emerged as a subject of its own in the curriculum of Lpo94. As a consequence of this long period of transformation, technology education still contains several vague and imprecise elements in the most recent curriculum (Norström 2011).

Technological systems as subject matter within the technology subject

The subject matter of technological systems was introduced for the first time in the curriculum of Lgr80 (Skolöverstyrelsen 1980) with curriculum items such as “orientation of different communications systems” and “the computer, its development and consequences for humans and society”. In the curriculum of Lpo94 in 1994 (Skolverket 2006), technological systems had a more prominent position and therefore also included an understanding of a system’s inner connections and processes. However, the curriculum formulated the assessment criteria so that it was optional to grade students’ understanding about technological systems (Skolverket 2008).

Turning to the present curriculum of Lgr11, introduced in 2011, the artefact-driven viewpoint on technology has expanded to include a broader more holistic understanding of technology (Bjurulf 2013; Klasander 2010; Svensson 2011). There are three categories of core content for the technology subject in years 7–9; (a) technological solutions, (b) working methods for developing technological solutions, and (c) technology, man, society and the environment. Within these categories, systems are mentioned in the contexts of “control and regulatory systems”, “industrial processes”, “technological solutions in communication and information technologies”, “the Internet” and “other global technical systems”. The knowledge requirement for students to receive the passing grade at the end of year 9 states that students should be able to “apply simple and to some extent informed reasoning about how some objects and technical systems in society change over time, and show the driving forces behind technological development”. Furthermore, they should also be able to “apply simple and to some extent informed reasoning about how different technical solutions can have different consequences on the individual, society and the environment” (Skolverket 2011a). These aspects of system contexts illustrate technological systems as man-made constructions of complex networking components, rather than being defined as just technological artefacts (Klasander 2010; Svensson 2011). The curriculum also highlights the need to understand how technological systems are linked to the individual, society and the environment (Klasander 2010). Although the curriculum presents some system contexts, the lack of guidelines for systems thinking as well as provision of some exemplifying systems in Lgr11 confirm that there is still some vagueness and imprecision that linger within the technology subject (Norström 2011).

Overall, systems consist of two primary entities; components and the relationships between them. The components are also known (by description and in quantity) and ordered within a framework so that a system boundary is observable (Ingelstam 2002). A simple common representation for technological systems borrowed from the field of engineering focuses on the principle areas of input, process, output and feedback (de Vries and Tamir 1997). An example of another way to describe systems is Soft System Methodology (SSM), which illuminates the human and social contributions to a technological system (Checkland et al. 2010). Another example is System Dynamics (SD), which focuses on complex systems over time where feedback loops and time delays alter the behaviour of a technological system (Karnopp et al. 2012). A fourth way to study technological systems is from a socio-technological systems (STS) standpoint, which focuses on the system’s (a) technical-material components, (b) social components and relationships, and (c) the relationship between technical-material and social components (Ingelstam 2002).

Although most systems can be viewed from these perspectives, systems such as the Internet, global mobile phone networks, and nationwide logistics systems are examples of Large Technological Systems (LTS), which cannot easily be studied as a whole, but are better understood by analysing and isolating the subsystems (Bijker et al. 2012; Hughes 1983). This latter practice is also used in business economics when business analysists try to map and understand the different processes and essential working parts of commercial and non-commercial enterprises (Lind 2001). Yet, a common denominator in these illustrated examples on viewing systems is that they can be rather complex, and not always directly suitable for teaching about systems.

However, recent research has shown that students are able to identify technical systems of different structures and types, despite the technology teachers’ concerns with finding a suitable educational model for teaching about technical systems. Students’ descriptions of the components of a system are often made with different degrees of accuracy. Students’ ability to discuss generic aspects of technological systems, i.e. their purpose, structure and processes, is greatly improved with the use of context and mediating illustrations of technological systems such as mobile phone networks, e-shopping and dishwashers (Örtnäs 2007; Schooner 2012; Svensson 2011). Yet, the majority of students still express difficulty in explaining the components’ relationships within the system, as well as the different internal processes, according to previous studies with younger students. Other areas of difficulty for the students’ comprehension of systems is identifying the human role in a system, and determining the system’s boundaries with its surroundings (Svensson 2011; Koski and de Vries 2013; Mioduser et al. 1996).

Research interest and aim

The complexity of understanding technological systems and the lack of a teaching tradition concerning technological systems as educational content are great challenges for technology teachers when carrying out assessment; there is basically nothing for them to lean on in terms of support. Therefore, there is a need for a qualitative study of technology teachers’ self-reported assessment of student understandings of technological systems, in order to map possible ways of carrying out assessment. The following explorative research questions were posed in a Swedish context to provide insight into technology teachers’ views on assessing student understandings of technological systems:

  • What views do technology teachers have about the types of knowledge they focus on in their assessment of technological systems?

  • How do these technology teachers view essential knowledge about technological systems in relation to their defined levels of basic, intermediate and advanced understanding?

Method

Qualitative interviews were chosen as the method of data collection, and the interview guide was given a semi-structured design (Kvale 1994), because we wanted a selection of informants to contribute with larger amounts of qualitative data (Bryman 2002; Denscombe 2000; Robson 2011). Assessment of technological systems is complex and we intended to extract as rich a qualitative material from each informant as possible.

Six technology teachers from six schools in different regions of Sweden were interviewed in depth regarding their views on assessing students’ knowledge about technological systems within the later years of compulsory school technology education (students aged 13–16 years old). All of the interviewed teachers had completed formal teacher training, but had varying backgrounds as regards technology education. Due to ethical considerations, the teachers’ identities were anonymised and they were given the following fictitious names:

  • Anna, female technology teacher.

  • Benny, male technology teacher.

  • Carl, male technology teacher.

  • Daniel, male technology teacher.

  • Edgar, male technology teacher.

  • Felix, male technology teacher.

In order to validate the categories obtained during the first six interviews, we conducted five additional interviews with technology teachers. They also taught the later years of compulsory school, and had formal teacher training;

  • Gabriel, male technology teacher.

  • Hubert, male technology teacher.

  • Irene, female technology teacher.

  • Julia, female technology teacher.

  • Kelly, female technology teacher.

In accordance with the ethical guidelines presented by the Swedish Research Council (Hermerén 2011) the respondents were presented with the purpose of the study and told that their participation would be fully voluntary. They were also told that the interviews would be de-identified in regard to names and geographical origin and that the collected data would be stored safely and would not be used outside the research context.

Analytical process

The eleven in-depth interviews were transcribed in their entirety. In all, the collected data from the interviews formed ca. 11.5 h of digital audio data and the transcripts reached approximately 88,000 words. The data was then analysed through the utilization of the systematic and interpretative model inspired by the hermeneutical spiral (Alvesson and Sköldberg 2008) and a process of thematic analysis (Braun and Clarke 2006). Through the use of CAQDAS (computer-assisted qualitative data analysis software) software called MAXQDA, the dataset was later structured and the data was coded. The analysis followed an interpretive process to derive themes from the dataset.

The first step of the analysis was to transcribe the digitally recorded interviews. The verbatim transcriptions were then read through several times to correct any errors in the transcripts. This activity of data immersion establishes the practice of “repeated reading” of the empirical material, which is also an essential part of the hermeneutical spiral (Alvesson and Sköldberg 2008). Through this practice, the researchers became very familiar with the material.

The second step of the process involved an initial coding of interview transcripts using the software MAXQDA. The transcribed texts were imported and processed within MAXQDA with meta-descriptive codes. Excerpts of texts were coded using a hermeneutical interpretive approach. Whenever the informants mentioned their view of what, how and why they assess and grade student understandings of technological systems, the excerpts were coded with a descriptive code label. In order to pinpoint exactly when the informants talked about technological systems, we relied on a basic working definition mentioned above, taken from Ingelstam (2002): a system consists of components, relations or connections between these components, and a system boundary. The code system finally consisted of 30 codes and 228 selected text excerpts.

The third step continued with a multitude of derived codes that underwent a sorting process to order them into a tree-structured hierarchy. From the inductively generated codes, three themes emerged that could describe the areas of assessment of knowledge about technological systems from the teachers’ point-of-view. The first identified theme was the “system’s structure”, containing six sub-codes; the second theme was the “system’s relations”, containing three sub-codes. The third theme that was identified was the “system’s development”, containing four sub-codes.

The fourth step required the themes to be reviewed, revised and refined to minimize the overlap between the themes. The highlighted themes for the technology teachers’ narratives were later discussed and confirmed amongst other peers within technological education research.

The fifth step commenced with the definition and naming of the three key themes and what aspects of the data they cover. From the interpretive analysis thus emerged the following three themes of teachers’ assessment: (a) the system’s structure within the system boundary, (b) the system’s relations outside the system boundary, (c) the system’s historical context and technological change. In addition, the interviews also provided several underlying subcategories of interest areas when the informants discussed their assessment. The following table clarifies and provides an overview of what the teachers saw as essential knowledge areas when assessing the students’ understanding of technological systems:

The sixth step involved presenting exemplary data to each theme as part of this study’s results from the thematic analysis. Particularly illustrative quotes were also translated into English and abridged by the authors in order to increase readability.

The additional five interviews did not yield any new or revised categories, and thus strengthened the validity of the above themes. Analysis of the data was also peer-reviewed at a research seminar in order to further check the validity of the categories. The eleven interviews constitute a limited data set, but, as we have shown, it has been enough to achieve data saturation and validate our categories. Our results can, in a strict sense, only be seen as representative of the eleven interviewed teachers, but we hope to generate more general inter-subjective understanding of technology teachers’ assessment through them. That is to say, the results of this study point to possible ways that teachers do and can assess technological systems, even beyond the Swedish context (Cutcliffe and McKenna 1999).

Results and findings

In an effort to derive explicit information on how the teachers evaluated different aspects of knowledge about technological systems, the interview focused directly on what they perceived to be appropriate knowledge for grades E, C and A in the Swedish grade system (Skolverket 2014). Because the informants related relatively freely to the grading criteria, we here refer to their statements about grading levels as demonstrating basic (grade E), intermediate (grade C), and advanced (grade A) levels of understanding about technological systems. The informants’ statements are presented with exemplifying quotations for each analysed theme and underlying systems elements.

The system’s structure within the system boundary

The system’s behaviour and operations

Edgar explained that the students should understand on a basic (grade E) level that the overall functionality of a system depends on the operative relationship of the system’s different components:

The components must be able to cooperate with each other so that the functionality of the system can be achieved. (Edgar)

Explaining in more detail (but not perfectly) how a system’s behaviour operated was viewed as appropriate knowledge on an intermediate level according to several teachers. Felix also expressed some uncertainty when drawing the line between an intermediate and an advanced level of understanding:

It is no longer about giving superficial examples, the student must know more in depth about the system. For example, the student should be able to say “I have an understanding of how the whole system works, but I don’t know all the details”. (Felix)

The advanced understanding of a system’s functionality involved knowing how the system functioned as a whole. Both Benny and Gabriel elaborated that the students had to have a clear idea how the system operated.

The system’s management of inputs and outputs

Benny also saw that the system’s functionality incorporated the interplay of all the internal structure’s different elements, and so he pointed out that the students should be able to identify simple causality between the system’s input and its output in a constructed technological artefact with a system structure (using a student-designed “electrical motor” as part of an inter-collaboration with the subject of physics):

The students should have the ability to see how the input affects the system’s output and why it does so. (Benny)

However, Hubert as well as several other teachers had issues with the concept of input and output as the teachers, on the one hand, saw it as essential that the students learned how a system operates with its surroundings through its interfacing components, but also, on the other hand, that this might be too cumbersome for the teachers themselves to comprehend. Benny added to his interview that he felt uncertain about requiring this understanding from the students:

Now I’m getting very uncertain, I might be saying things that I do not really know if I can stand behind. This is the feeling that I have right now, however. (Benny)

Edgar saw it as essential that knowing about the system’s input and output on a basic level had to do with the students’ fundamental thinking about systems, realising that the technological system did not operate through the use of “magic”:

The student should have evaluated the observed system and seen that the functionality was not derived from “magic”. (Edgar)

In general, all the teachers saw the system element of input and output as an essential level of understanding.

The system’s structure of components

The understanding of the system’s components, on the other hand, had a knowledge span ranging from an E- level to a higher A-level understanding. Irene and Julia, as well as other teachers, elaborated from their own perceived teaching that when studying a technological system a great number of tasks focused on the student being able to map the structure of components within a system. A fairly rudimentary overview with few central components was seen as acceptable on a basic level of understanding, but in the case of an advanced level of understanding, the student had to know about nearly all of the system’s components. Edgar said that his students had to be able to identify some crucial elements of the system without necessarily knowing in detail how the specific components interacted in the system:

A student that reaches an E-level understanding should be able to exemplify with some important components within the system. He or she may not have a full comprehension of how these components interact with each other, but the student should understand the importance of the identified components. (Edgar)

For a higher level of understanding, Felix explained that the student should know the whole system. He was unable to give details of what he meant by “the whole system”. He continued clarifying, in a contradictive way, that it was not necessary for the same student to be able to explain the entire system and the components’ interactions with each other perfectly:

The student should know the whole system and be able to name the different parts within it. However, the student is not required to explain every aspect of it in depth. The description of the system can be incomplete. (Felix)

To know the system’s components is to be able to identify them and to explain their role in the system, but also be able to order the components after significance for the system. Knowing which components affect the concrete and overall functionality of the system was also significant in the interviews. On an advanced level, Benny and Kelly focused on the student’s knowledge of how the components are physically constructed regarding material. Benny explained a reason for this, namely the student’s ability to interact with the system directly by replacing components:

Students know which components in the system are most important, and how they can be replaced. (Benny)

The system’s processes and flows

System processes and flows were something that all informants had some trouble defining adequate knowledge about. Hubert mentioned that understanding a system’s processes is essential for further discussions about the internal and external consequences of the system. In addition, most teachers also saw a simple understanding of a system’s different processes and flows as characteristic for students’ E-level understanding. When Anna used the local water treatment plant to exemplify technological systems in her teaching, she recollected that the students were not able to talk about the process within the system, although they recognised it:

By not mentioning the treatment process, the student may remain on an E-level of understanding. (Anna)

Felix also used the local water supply system to exemplify how the students perceive and understand the system’s processes and flows, and what type of understanding he assesses on a basic level:

If we take the water faucet, for example, the water is brought into the household as the tap is turned. The student understands this, and also knows that the water running from the faucet is not brought directly from a lake but first undergoes treatment. The water treatment plant has done things to the water, and the student understands that the water then has to be transported up to the water tower. (Felix)

Benny and others mentioned that for a student to have an A-level understanding of a system, he or she would have to explain the processes and flows within the system. This can be seen as great leap in progression from the knowledge requirements tied to the basic level of understanding where the students had merely to identify the processes (without fully understanding them). However, being on an advanced level means that the student also has to explain the system’s identifiable processes and flows:

Students should understand the entire journey of the electric current through the electric motor and what happens along the way. Why the motor has or does not have coils, and that the coils can be arranged in different ways for the functionality of the motor. (Benny)

Overall, the teachers had difficulty defining what they saw as knowledge progression regarding the system’s processes and flows.

The system’s human agent

The human as an element within the system was something that Anna saw as essential knowledge about systems, although most of her students had difficulties in seeing humans as a part of a technological system. Hubert gave a possible reason for this as some systems can operate without the visual presence of a human, even though technological systems are made by humans. In Anna’s teaching, the students on an E-level were unable to clearly identify the human’s role as an agent within the system. The student on an A-level however had some understanding about the human’s role as an agent. Anna summarized her assessment experience in evaluating the students’ understanding of the human agent in the following manner, exemplifying from an earlier fieldtrip to the local water treatment plant:

My students saw all those people working there, and the guide described the functions that this man or woman had at the water treatment plant. Still, this was not something that the students picked up, especially not the students having a basic understanding of technological systems. (Anna)

The system’s boundary

Kelly, as well as the other interviewed teachers, explained that in their teaching it was hard to define and properly evaluate the concept of the system’s boundary. As the teachers explained, to understand a system’s boundary is to know how the technological system is delimited from the rest of the world. As such, classifying the notion of the system boundary to higher levels of system understanding was something that both Daniel and Hubert had a shared view on:

I’d say that it is probably a bit too advanced for the student to actually understand where the system boundary is. I would say that ability is on a C or higher level of understanding. (Daniel)

Anna had an analogical viewpoint on the system boundary, likening it to a country’s borders. As water treatment was a central theme in her teaching about technological systems, she explained in the interview that the students on an A-level should be able to relate the local water treatment plant with other treatment plants around the world (or at least outside Sweden).

The system’s relations outside the system boundary

Implications for individuals, society and the environment

Knowing what influences a technological system had on the individual, society, environment and other systems was something that all teachers deemed necessary on a basic (grade E) level. Anna explained in her interview that both she and her colleagues based their teaching on technological systems from a local context: the municipal local sewage treatment plant. She firmly believed that students should be able to understand the consequences that can occur when unfiltered wastewater returns to the natural water cycle:

They do grasp the fact that nature gets polluted, or that sewage water is discharged into the fresh water that should be clean. They can see that fish lie floating on the water’s surface, but it is hard to know how they reason about this because they seldom express anything more than that when describing the effects of the system. (Anna)

Going beyond basic understanding and simple mentioning of the system’s implications on its surroundings, Edgar explained that on a C-level the student should be able to describe the effects with the aid of natural science so that the student’s discussion of the system becomes more developed. Hubert mentioned a similar view on higher understanding about technological systems when he saw it evident for the students to be able to identify the consequences of a technological system. Especially, as Kelly explained in her interview, the environmental consequences of a technological system. Benny took it further and deemed it necessary for the student on an advanced level to have a clear understanding of the system’s relations to individuals, society and the environment:

They have a clear idea of how the system works, how it works in other contexts and the different ways the system affects things like the environment, society and all that. (Benny)

Felix tried to be more precise in clarifying what could be seen as clear or full understanding of the system’s external effects. By being able to evaluate the benefits and the disadvantages the system has on individuals, society and the environment, the student presents his or her deeper understanding of the system.

Comparison with other technological systems

Nonetheless, being able to compare two or more systems, or being able to regard a system as a subsystem within a larger system context, was deemed as too complex for students on an E- or a C-level. Hubert focused more, in his teaching, on comparing how two systems cooperated with each other rather than highlighting the shared traits of the systems. From Anna’s experience, the students that had system understanding on a basic E-level, were not able to comprehend or identify the concept of sub-systems, nor were they able to compare different systems. What all teachers agreed on was that the ability to discuss and compare different systems with each other defined a higher level of understanding (that of an A-level). When asked if the students on an A-level had to be able to describe the inner structure and outer relations of a system, Benny said the following:

The student might be able to say that this system could work together with this system. (Benny)

Daniel added in his interview that the students with a more developed understanding should be able to understand what context the system was situated in and that a system could be part of a different and larger context. This is additional to what he requires of the students showing basic or intermediate understanding:

Above all, to be able to put things in context more, to understand that this technical system is a part of a larger context. (Daniel)

Edgar emphasised that students on an A-level, compared with those on an E- level, would be better able to discuss a system in a wider sense:

A student on an E-level would probably shake his or her head if I were to ask that question about comparing different systems. But, a student who is on an A-level would be able to compare between systems. It feels almost a bit justifiable that the student on that level should be able to do such a comparison. (Edgar)

The ethics of the technological system

In Kelly’s and Julia’s teaching, the issue of values and ethics is present when discussing the environmental aspects of technological systems, especially power consumption. However, as they and others added, the ethics of a system was important when the teachers assessed students’ understanding about technological systems on a C-level. Felix explained that intermediate level students, in contrast to basic level students, should be able to evaluate the consequences of a system. Students reaching the A-level of understanding would also have discussed the ethical implications of a system in a more qualitative manner than the C-level students. Felix added in his interview that students should derive their conclusions about the advantages and disadvantages of a system:

When viewing the A-level student, he or she would be able to discuss both the advantages and disadvantages of a system. He or she would come up with a balanced conclusion and thus present a different reasoning than that of the E- or C-level student. (Felix)

The system’s historical context and technological change

Historical perspective

Knowledge about a system’s historical context, its purpose and emergence and later its development to its present form—was mostly seen as knowledge beyond basic understanding about a technological system. The question of “why does the system exist” was something that Hubert especially found relevant when teaching about technological systems. Students with E-level understanding had to grasp at least the background situation from which the system had originated. Regarding the historical context, Felix mentioned that students had to have an understanding of the system from a historical perspective:

To some extent, I think, the student must have a historical perspective on a technological system. The student would not be required to have a future perspective, but they must have a historical understanding. (Felix)

He continued to exemplify the importance of having a historical perspective, saying that students should know that systems that are very common and familiar to the students have not always existed, for example, the global technological system of Internet.

The process of technological change

The teachers were in agreement that knowing how systems have developed and undergone a process of technological change required a higher level of understanding. Knowing about a system’s technological change on a C-level requires students to know how the system has changed, from the start to its present form. Both Edgar and Hubert mentioned that students having C-level understanding would have to know how and why a system has emerged. Carl also added that the student should know something about the people behind the system:

When mentioning the history of the technological system, a student may be able to see that history tells us what has taken us where we are today. They formulate questions like “what is that (the system), who were the people that made the development (of the system) go forward?”. (Carl)

Holistic overview of the technological system

Anna and Daniel both emphasized that the students should also be able to view the technological system from a more holistic perspective. Kelly stated that her students would only go beyond the basic level of understanding when they were able to grasp the wider perspective of a system. Although several other teachers also mentioned the wider perspective, Anna explained that, when assessing students’ knowledge on an A-level, the student had to look beyond the narrow context that the system was confined to. This would be in contrast to basic and intermediate students that had only to observe the nearest (often locally) external relations:

The student needs to able to analyze and to regard the system from a global perspective. I think that the student should be able to look outside the box, beyond the national borders of Sweden, and see how the system operates from a wider perspective. (Anna)

Daniel claimed that he used the same perspective in his assessment, pointing out that a system could be a part of a larger system context.

Understanding how to manipulate and change a system

For an A-level understanding of the technological changes affecting systems, Daniel saw it as essential knowing how to manipulate a system:

If I change a part of this system, for example, what will be the consequence? If one understands those consequences, then one has caught the drift in some way. This is a typical feature of an A student, I think. (Daniel)

Teachers’ assessment views in relation to the themes

The following Table 1 assembles the analysed knowledge progression that the teachers claimed they utilized when assessing students’ understanding of technological systems:

On a general level, the teachers also added that they had great difficulty in grading students beyond the basic level. Only a few students ever reached the advanced level, whereas the great majority of the students passed the technology subject on a basic level.

Table 1 Themes and corresponding sub-categories (system elements) of what the students should be able to understand about technological systems of various structures and types according to the interviewed teachers
Full size table

Discussion

Results from the hermeneutical and thematic analysis provided insights into what types of system understanding the interviewed technology teachers claimed they assessed. The questions from the interview guide allowed the teachers to express their perspectives about their process of assessing students’ knowledge of technical systems. Previous research has revealed that most technology teachers lack sufficient systematic training in assessing knowledge about technology, especially the subject matter of technological systems. Although the curriculum of Lgr11 does provide some hints on what to assess, it still leaves great areas of the central content and the grading scale to be interpreted by the teachers (Bjurulf 2013; Hartell 2015; Kimbell 1997; Norström 2011; Klasander 2010; Svensson 2011). This study provides insight into assessment views of technology teachers. The most revealing finding from the gathered data is that amongst the interviewed teachers there is no unified way of assessing students’ knowledge about technological systems. The findings also show that the teachers interpret the present technology curriculum differently, in such a way that it is possible to portray a diversity of interest areas. When assembling these areas together, however, the teachers form a matrix (see Table 2) for assessing understanding of different crucial system elements.

Table 2 The knowledge progression of understanding technological systems
Full size table

In this discussion, we will address the research questions and the analysed empirical data related to each question.

What views do technology teachers have about the types of knowledge they focus on in their assessment of technological systems?

The collected excerpts and the analytical categorization revealed three significant aspects of assessment regarding technological systems; (1) the system’s structure within the system boundary, (2), the system’s relations outside the system boundary, and (3), the system’s historical context and technological change.

The system’s structure within the system boundary

The teachers agreed that understanding the system’s internal structure within its boundary was essential knowledge. This result is in line with other studies done on young students’ understanding of technological systems, which is closely tied to the system’s structure. Students can with different levels of detail identify system components. However, the internal flows and component relationships require a more profound understanding of the system (Svensson 2011). In a study by Koski and de Vries (2013) students understood the concept of input better than the system’s processes (flows) and its output. The study also showed that they found it difficult to identify system boundaries (Koski and de Vries 2013), something which the teachers in our study considered to be at least at an intermediate level. Anna also pointed out that it was desirable that students were aware of the human role as an agent within the system, although her students did not easily grasp that a human operator could be a part of a technological system. While it is important to point out the human and man-made attributions to a system’s operations, teaching about the social aspects of a technological system should not be reduced only to recognizing the human agent as a “machine-part” in the overall machinery (Ingelstam 2002). The framing of the discussed technological systems indicated that the interviewed teachers expressed views that were closely related to the general description of a technological system in regards of components and their internal relationships (Ingelstam 2002; Tamir and de Vries 1997).

The system’s relations outside the system boundary

The assessment of students’ abilities to identify impacts and consequences on individuals, society and the environment follows a more socio-technological perspective (de Vries 2005; Feenberg 2006; Ropohl 1997), and actualizes social and conceptual knowledge besides procedural and technical understanding of technological systems (Jones 1997). Anna demonstrated this clearly in her interview when she said that she and her co-teachers made the students study a socio-technological system (water plant) in the local municipality. Concerning implications and consequences, Daniel exemplified the clear connection between technology and natural science when he claimed to assess the students’ understanding of a technological system’s biological and chemical implications for the environment. The ability to identify and compare a system with other systems was something that Carl highlighted in his interview, an ability that both Svensson (2011) and Ingelstam (2002) see as a key factor in developing an understanding of technological systems. Also considering the ethics in relation to the usage and consequences of a technological system was something that stood out in the analysis. Ethics as a curriculum item has not been addressed in recent curricula (Skolöverstyrelsen 1980; Skolverket 2006, 2011b), although previous research (Svensson 2011) has also shown that students are able to discuss the problem-solving aspects of a technological system. However, adding ethics provides better understanding of systems as a social construct and not only as a series of physical entities (Bijker et al. 2012; Bijker 1995).

The system’s historical context and technological change

While the two first themes focus on the existing properties of a technological system, the third theme focuses on understanding the reasons and driving forces behind a system and its technological change. Felix addressed the need to have a historical perspective, which connects with what Edgar also pointed out regarding knowing why the system has emerged. Understanding the development of technological systems illuminates the social construction of technology (Bijker et al. 2012). Interestingly enough, this was viewed as a difficult task to present in the teaching about technological systems, according to Anna in her efforts to make it visible to students that humans can be part of a technological system. Still, the students should understand that a system emerges and changes over time due to different needs and that people make these things happen. The teachers also addressed the necessity for students to understand that the need-driven purpose behind a system has a social origin while the implementation of the needs is processed through engineering (Feenberg 1999; Hallström and Gyberg 2011). Moreover, the interviews show the closeness to the socio-technological perspective, where knowing about the historical context and technological change are important features when establishing a holistic overview of the system (Bijker et al. 2012). As such, the students are required to move past a system-centred understanding with a closed system framing on a higher level of understanding (Karnopp et al. 2012).

How do these technology teachers view essential knowledge about technological systems in relation to their defined levels of basic, intermediate and advanced understanding?

Seeing that the informants had wide opinions on what to assess concerning system understanding, a few things were quite apparent—the progression from a microscopic to a macroscopic understanding of technological systems.

Basic systems understanding (grade E) revolves around superficial and concrete knowledge about a system. According to the teachers, students that can tell what historical context a system is derived from, together with its purpose, possess a basic understanding of systems. In addition, the teachers require the students to be able to recognize the central workings and components of a system through simple linearity, which is consistent with previous research showing that most students (and adults) have difficulty understanding systems beyond simple one-way connections (Sweeney and Sterman 2007). The system’s (observable) relations with its direct surrounding are also required on a basic level of understanding. Given that the inner workings of the system are central according to the interviewed teachers, basic systems understanding can also be described as the student having a topological perspective on systems.

Students having intermediate systems understanding (grade C) are required to explain the system’s inner workings and structure, as well as discuss the system’s ethical effects on individuals, society and nature. Ethical considerations are also important on this level of understanding as the students discuss the implications on a wider scale than a basic level. It is also necessary for the students to know how the system has emerged and the reasons for its technological change, through a historical perspective. This socio-technological aspect contributes democratic emancipation through a wider understanding of technology and its temporal place and a need-driven change in relation to society and nature (Keirl 2006; Mumford 2000; Feenberg 1999). Interestingly, when comparing the defined levels of basic and intermediate understanding, the intermediate level includes more elements of technological literacy than the basic level of understanding, such as being able to identify the human agent and discussing the ethics of a system (Kahn and Kellner 2006; Waks 2006; Compton 2013). From here, it becomes evident that an intermediate systems understanding can be described as having a socio-technological perspective of technological systems.

Transcending the socio-technological understanding is the advanced level of understanding (grade A). Students are more or less required to have full comprehension of a system and its neighbouring systems, as well as its effects on individuals, society and the environment. Key concepts that the interviewed teachers search for when assessing students on this level are the abilities to identify, explain and discuss the whole system within a given system boundary. As ethics is also an assessable quality on this level, an advanced level of understanding closely coincides with ideas of technological literacy, especially knowing the system’s intention, design, realisation, use, background and stakeholder benefits (Keirl 2006, 2009; Feenberg 1999; de Vries 2006; Compton 2013).

Conclusion

The research findings suggest that the interviewed technology teachers assess different aspects of technological systems but that at least some of them evaluate students’ understanding of systems in a progressive manner as they add more assessment items for each level of understanding. Given that the curriculum of Lgr11 is open to a great deal of interpretation, it is understandable that the teachers both diverge and converge in their views when interviewed about how they carry out their assessment in technology education. It should also be noted that the teachers had great difficulty finding appropriate aspects of understanding on an intermediate level, beyond basic understanding. This could be explained by the fact that the teachers had more experience grading students on a basic level, and less experience of grading those with the higher levels of understanding, especially the intermediate level and above.

What was surprising was that the teachers excluded most of the socio-technological aspects from the basic level of understanding, as well as the multifaceted technological literacy aspect. Given that most students only pass the technology subject with grade E, i.e. basic level of understanding (Hartell 2012), more can be done to incorporate and balance the assessment expectations of intermediate and advanced levels of understanding. On the other hand, previous research has shown that most students have difficulty identifying social elements such as the human role in systems, which could explain why the interviewed teachers have moved the socio-technological and technological literacy aspects to intermediate and advanced levels of understanding. Still, the analysed themes can provide a basis for further discussion about defining an assessment matrix regarding technological systems as subject matter within technology education. From a methodological standpoint this study also indicates that interviews with individual teachers can provide an in-depth opportunity to explore teachers’ views about assessment which is beneficial when gathering and evaluating items for an assessment matrix (which Table 2 illustrates).

However, the assessment views of the interviewed teachers need to be further discussed as they show an imbalance between the levels of understanding and the assessment items (see Table 2); few students were said to reach beyond the basic level, but at the same time most assessment items lay on the intermediate and advanced levels.

This study has been an exploration into the discourse on how technology teachers assess students’ generic understanding of technological systems of various structure and type. Further studies should investigate technology teachers’ own epistemological views of technological systems, as well as their self-efficacy concerning assessing students’ knowledge about technological systems.