Introduction

Student’s attitudes toward technology have been a topic in technology education research since the 1980s. Measuring students’ attitudes was first done through the Pupils’ Attitudes Toward Technology (PATT) questionnaire (developed by Raat and de Vries 1986). Bame et al. (1993) updated this questionnaire for the USA. Ardies and De Maeyer further developed the survey along with Gijbels (2013), which in fact was the same questionnaire, with a reduced number of items, called PATT short questionnaire (PATT-SQ). The PATT-SQ was problematized and further developed by Svenningsson, Hultén and Hallström, PATT-SQ-SE (2018). Throughout the history of the PATT questionnaire, the main focus has been on measuring and discussing the affective component of attitudes. The PATT-SQ (and SE) survey is specifically intended to measure the affective component of students’ attitudes toward technology.

A central issue when measuring attitudes toward an object, whether it is the affective, cognitive or behavioral component, is the students’ understanding of the object of interest. Attitudes toward technology are based on a person’s beliefs about it, and those beliefs have an influence on their behavior (Fishbein and Ajzen 1975). A person’s beliefs can, however, be weakened or strengthened and even replaced, which can lead to a change in attitude. In some cases, attitudes toward technology positively correlate with students’ knowledge (Garmire and Pearson 2006), meaning that a clearer concept of technology among students could also improve positive attitudes toward it (Rohaan et al. 2010). Students generally have a positive attitude toward technology, but their concept of technology has been found to be limited (Ankiewicz 2018). Despite this, we still know very little about how students’ attitudes and their conceptions of technology correlate. This may be an issue that needs to be addressed, as in the affective part of the PATT survey, 17 of 24 statements include the word technology (PATT-SQ and PATT-SQ-SE, see Ardies et al. 2013, p. 18). The US version of the PATT questionnaire includes the word technology in 48 of the 58 statements measuring the affective component of students’ attitudes (Bame et al. 1993, pp. 51–52). One may raise the question of how much you can actually tell about students’ attitudes toward technology when it is unknown what they consider technology to be. In this paper, I suggest that, in order to measure students’ attitudes toward technology, we need a method to analyze their descriptions and to include that analysis as part of a student’s attitudinal profile.

The aim of this study is twofold. First, to develop a method to analyze and classify students’ descriptions of technology. Second, to be able to use this developed method in quantitative studies, such as PATT studies, in order to better analyze students’ attitudes toward technology. The research questions for this paper are: What aspects of technology are covered in students’ descriptions of technology? and How can the aspects found be implemented in attitude studies?

Background

In order to reach an understanding of students’ concepts of technology, their descriptions have been a part of PATT studies since the 1980s. However, student descriptions of technology have not been researched in relation to attitudes. As a background, I will first describe the results of previous studies of student descriptions of technology, both those originating from PATT studies and those solely focusing on students’ descriptions. Then a proposition for how these types of studies can be quantified and analyzed is presented.

Students in PATT studies have traditionally been asked to write a short essay to describe what technology is (see e.g. Bame et al. 1993). The main results from studies in which students’ descriptions have been analyzed are that students often understand technology strictly as technological objects, and more specifically as modern electrical objects (de Vries 2006; Kőycű and de Vries 2015; Garmire and Pearson 2006; Riis and Augustsson 1991). Similar results are found when asking American adults; two-thirds of the adults asked see technology as modern electrical objects and one third describe a somewhat broader view of technology; for example, manipulating the natural world according to our needs (Rose and Dugger Jr. 2002).

Even though the most common description of technology involves different technological objects, many people do describe it in a broader way. Some studies attempt to analyze the breadth of understandings of technology by sorting students’ descriptions into themes. When doing so, it is possible to gather knowledge about what else students consider to be technology. In research, there have been two approaches to analyzing students’ descriptions of technology, one in which themes are created based on the students’ descriptions (inductive approach), and one in which the themes are created beforehand and the descriptions are placed within them (deductive approach).

In Australia, for instance, where the first approach was used, the PATT essay analysis ended up with three main dimensions of technology: Product, Process, Social and a fourth dimension called Incorrect (Rennie 1987). Burns (1992) analyzed student descriptions of the meaning of technology by including four main concepts—Products, Activities, Harms/benefits, and technology as something Universal. These four main concepts broke down into nine sub-concepts. The most commonly mentioned concepts in this study were modern products, but many students also included the benefits of technology and technology as a problem-solving process in their descriptions. Rennie and Jarvis (1995) ended up with five concepts when studying children’s writings and drawings about technology: Product ideas, Design ideas, Temporal aspects, Affective aspects, and Other aspects. They also presented the breadth of students’ descriptions by the number of ideas or themes present in their writings and drawings, which led to a score ranging from no understanding of technology (score = 0) to a good understanding (score = 3).

In Poland, where the second approach was used, students’ answers were placed within the themes Research and creative work, Objects, As a science and Others (Dudziak and Szydlowski 1987). A well-defined description of the nature of technology within the educational context is the conceptual framework developed by DiGironimo (2011). She presents a three-sided prism, where the sides represent technology as: Artifacts, a Creation Process, and a Human Practice. The bottom of the prism represents The History of Technology and the top, The Current Role of Technology. Students were asked to write a response to the question “What, in your opinion, is technology?” (DiGironimo 2011, p. 1344). When analyzing the sample of students (20 US students) in that study, using a deductive approach to fit into the conceptual framework, half of the student descriptions contained artifacts and a quarter contained the current role of technology. The three other sides of the prism were rare in the student descriptions (DiGironimo 2011). Liou (2015) used that same question to analyze 455 (Taiwanese) high-school students’ answers. Based on this content analysis, a questionnaire was created to survey high-school students’ views of technology. Liou reached similar results within six pre-defined constructs (technology as Innovation change, Artifacts, Science-based, Double-edged sword, History of technology and the Current role of technology). The most common construct in the student descriptions was technology as an innovative change. A small proportion of the students described the history of technology or technology as a double-edged sword.

There are of course some pros and cons to using predefined themes or exploring themes based on the written answers. Creating themes based on students’ own writing will include most student descriptions since the concepts are based on these descriptions. This approach will be more suitable if the goal is to research what students do not see as technology or to explore potential misconceptions. The predefined approach might miss many students whose answers do not fit the framework. On the other hand, the results will be easier to compare with other studies if the same framework is applied. It is important to note that there are some recurring themes in the research presented, where technological objects and activities are the most commonly found (and described by students) and some themes include technology as a part of society and human beings (see Table 1).

Table 1 Themes in previous studies with the percentage of student descriptions within the themes, placed within Mitcham’s (1994) four aspects of technology. Themes in bold only cover one of Mitcham’s aspects, themes in parentheses are partly included within Mitcham’s aspect. Studies presented in chronological order
Full size table

To reach some understanding of what technology can be, in order to create themes or use predefined themes, we can turn to the philosophy of technology. There are traditionally two branches: engineering philosophy of technology and humanities philosophy of technology (Mitcham 1985). Mitcham (1994) has created a framework that bridges the gap between them, by combining the human aspect with the engineering aspect and thus the view from both engineering and humanities philosophy. Mitcham’s framework forms a typology in which technology is manifested as technological Knowledge, Volition, Activities, and Objects. His typology includes the most common concepts in the findings of student descriptions above; namely, the human being as one part, and technological objects and activities as another part. An aspect of the human being that appears to be missing in the studies above is the concept of technological knowledge, which Mitcham (1994) includes as part of the human being (Fig. 1).

Fig. 1
figure 1

The four manifestations of technology (from Mitcham 1994, p. 160)

Full size image

Mitcham’s framework presents a broad description of technology as knowledge about technology along with the human drive to create and improve, which leads to developing, manufacturing and using (new) technological objects. This broad manifestation of technology has the advantage of being time-independent, it includes ancient to modern technology, simple to advanced technology, and the model is not limited to any specific type of technology. Technology and technology education are subjects undergoing constant change and development, and since the model does not obstruct with this development, the framework is suitable for technology education.

Mitcham’s typology has been used by Nia and de Vries (2016a) to analyze ITEEA Standards for Technological Literacy and the New Zealand syllabus in technology (Nia and de Vries 2016b). Their analysis demonstrates the usefulness of this broad framework. They also argue that technological knowledge is scarcely considered, and that technological activities and objects are more adequately considered in technology syllabuses. Ankiewicz (2019) uses Mitcham’s manifestations of technology in combination with the attitudinal framework. He discusses how volition is, in a way, similar to the affective component of attitudes. Furthermore, the cognitive component is represented by knowledge and the behavioral component by activities. Mitcham’s framework should also be suitable for use when analyzing student descriptions of technology, since it includes concepts found in student descriptions in previous research, and technological knowledge as a part of the typology. It is also interesting to use Mitcham’s framework since it has similarities with the attitudinal framework, often used in PATT studies. To get an impression of what the themes in previous studies have in common with the philosophical typology developed by Mitcham (1994), previous studies are placed within the typology in Table 1.

When comparing the different studies with Mitcham’s (1994) typology (Table 1), it is clear that the focus has been on technological objects and technological activities, either in student descriptions or in researchers’ interpretations. DiGironimo (2011) includes a specific theme that could be considered as technological volition (human practice). The themes in parentheses partly fit into Mitcham’s typology; for example, Rennie and Jarvis (1995) include knowledge and learning as a part of their “other aspects” theme. The interpretation of harms/benefit (Burns 1992) is not very clear and could be argued to be part of the Knowledge as well as the Volition aspect. DiGironimo’s (2011) theme “creation process” contains both different kind of activities and/or the knowledge needed to perform technological activities. Technological knowledge does not appear as a specific theme in any of the studies above, while Nia and de Vries’ (2016a, b) research reports an inadequate presence of technological knowledge in ITEEA Standards for Technological Literacy and the New Zealand syllabus in technology. Technological knowledge needs to be addressed and highlighted in students’ descriptions of technology.

Method

Taking inspiration from Rennie and Jarvis’s (1995) study, where students’ descriptions receive a score that represents how broad the description of technology is, this study uses Mitcham’s (1994) framework of technology to analyze students’ descriptions to produce a score. By choosing a deductive approach, we can achieve a more suitable analysis for comparing studies from different contexts. Mitcham’s framework covers the most common themes presented in both types of approach (predefined themes and description-based themes) in previous studies but adds a specific theme; namely, technological knowledge.

Mitcham (1994) describes technical objects in a broad way so as to include technical systems as a series of integrated objects. Technological objects can be either structures, such as buildings or roads (static objects), or machines, such as cars or computers (dynamic objects). Technological activities involve making or using technological objects. The activities defined as making can be the action of crafting, inventing or designing technological objects. It can also be the process of manufacturing, working, operating, troubleshooting, and maintaining technological objects. Using technological objects can be either passive or active. Passive use of a technological object can be the usage of a house, simply by being within its walls. An active way of using objects can be when using technological objects (or tools) to make new objects. Mitcham exemplifies several levels of technological knowledge. It can be the trial-and-error process or the imitative process of a master-apprentice type, referred to as “know-how”. Another level of knowledge is being able to articulate strategies to solve a problem by successfully making or using a technological object. The fourth aspect, volition, is the human drive to survive, control our surroundings and make our lives more efficient. Volition is an aspect that often acts together with our knowledge. The human will can, for instance, override our knowledge of the negative effects of a new product, and of course the other way around.

The method section for this study contains three parts. In the first part, the questions that were formulated to investigate students’ descriptions of technology are presented. The second part presents how these descriptions are classified within the Mitcham typology. In the final part, the process used to control reliability is presented.

Stimulating questions

Student concepts of technology could, of course, be limited and narrowed by the questions asked in the studies (see Table 1). The question given for students to answer in those studies was different versions of: “What in your opinion, is technology?” That one question does not seem to trigger students to actually describe technology within the intended themes used in the studies (Table 1). In DiGironimo’s study, the students’ descriptions came to focus on just two of the five sides of the prism. That question does not seem to stimulate students to describe technological knowledge in the more inductive analyzes either (e.g. Rennie and Jarvis 1995), even though knowledge is present. For this study, two questions were developed in order to stimulate students into writing more extended descriptions of technology. These two open-ended questions were added at the beginning of a PATT survey to let the students describe their views on technology before getting possible inspiration from the rest of the survey.

The questions were:

a: Describe what you consider to be technology (not the school subject of technology)? and

b: If you were to describe the school subject of technology, for anyone who has not studied it in school themselves, how would you describe it?

Together, these two questions elicit data to classify students’ descriptions of technology in a broader way than only the one question could. Most students have at least some experience of technology education. Therefore, the second question potentially includes students who do not feel that they know what technology is but have some experience of technology education.

Classification analysis

In order to classify the answers concerning students’ descriptions of technology, a thematic coding approach (Robson 2011) is used. Mitcham’s (1994) four ways of describing technology (Objects–Activities–Knowledge–Volition) is used as the framework for the classification. The respondents’ descriptions were coded as 0 or 1 point for each of the four ways of describing technology. The aspects in the classification are seen as dichotomies: objects are considered either to have been mentioned in the description or not. The sum of the four aspects results in a total score of 0–4 points, which is named “The Mitcham Score”. The intention of the Mitcham Score is to create a scale similar to that in Rennie and Jarvis’s (1995) study, where student descriptions can be considered in terms of their breadth. To score in the Object aspect, the respondents have to write that technology has to do with man-made objects. In the Activities aspect, the student mentions the process of making or using technology. To score in the Knowledge aspect, the respondent needs to use technology in an example that requires knowledge, “how to/know how/know that”. The final aspect, Volition, includes respondents who consciously express technology as the human drive to improve or control technology (or society by using technology). The above description of technology is used to classify students’ descriptions of technology and technology education. The words that trigger the score are printed in bold and followed by a superscript letter to visualize the aspect classified (objectO, knowledgeK, activitiesA and volitionV). Typical examples of scoring in the Mitcham Score scale are:

1.

a “Nothing important”

b “boring and unnecessary”

Does not fit into any of the predefined categories. This student rather expresses his/her feelings about technology (affective component of attitudes). It could be argued that this may be an expression of the volitional aspect, but we do not take it as such since we still do not know what the student considers to be technology. These types of answers are consistently classified as not being technological volition (or any of the other aspects). Mitcham score = 0.

2.

a “Technology is computersO, phonesO and stuffO”

b “Don’t really know”

This student describes different types of objects. “Stuff” in this description is considered to be an object, since (s)he uses “stuff” together with computers, phones and technology. Fits into the category Technological objectsO. Mitcham score = 1.

3.

a “InventionsO”

b “To buildA and inventA thingsO”

In this description of technology, the response about technology education includes a wider description. Fits into the categories Technological objectsO and ActivitiesA when including answers to both a and b. because simply using the word invent is classified as a technological activity. “Things” in this example is considered to refer to objects, since it is written together with “build” and “invent”, then “things” are distinguished from, for example, a biological thing. If the student had developed the answer into: “invent things to make life easier”, it would have been considered to include the volitional aspect as well (including the purpose of the invention). Mitcham score = 2.

4.

a “It’s the skillsK for constructingA thingsO”

b “You learn howK different thingsO functionK”

This description of technology is another example in which the description of technology education includes a wider description. Fits into the categories of Technological objectsO, ActivitiesA and KnowledgeK. The word “things” is considered to refer to technological objects since the word “constructing” refers to a technological activity. Learning how things function implies increased knowledge about the technological world. Mitcham score = 3.

5.

a “InventionsO and howK to designA thingsO to fulfillV a smart functionK.”

b “How you learnK to buildA thingsO in the best wayK and how different principlesK and modelsO fulfillV different functionsK.”

Fits into all four categories: Technological objectsO, KnowledgeK, ActivitiesA and VolitionV. This description is very thorough, with an emphasis on different levels of knowledge. The described intentions (“to fulfill”) when creating technological objects in this description is considered to imply volition. Mitcham score = 4.

The four aspects of technology in the classification are measured and valued on equal terms. In theory, this means that a student could express, for example, only the volitional aspect in their description of technology. In practice, though, it becomes difficult to express volition without any other aspect from the typology, as mentioned in the first example. This also means that a 1-point Mitcham Score is 1 point no matter which aspect is described. By completing the classification, two results are achieved, the first being the Mitcham Score (sum of aspects, breadth of description) and the second being to tell us which different aspects are described (see Table 2).

Table 2 Quantification of students’ descriptions. Example students 1–5, same as above. 1 = present 0 = absent
Full size table

Intra-rater reliability

If the Mitcham score classification method is to be used by other researchers, it is important that the score is consistent when used on different occasions and between different researchers. Hence, one researcher completed the classification on a random sample (15 students) of descriptions using the instructions in “Appendix”. The researcher is a teacher and technology education researcher but was not trained further within the typology than the instructions in “Appendix”. This researcher completed the same classification on two occasions, 3 weeks apart. Cohen’s Kappa was used to measure intra-rater reliability in this study. The agreement is measured from − 1 to 1, where − 1 is total disagreement, 0 is agreement that could be achieved by chance and 1 is perfect agreement (Viera and Garrett 2005). To interpret the Kappa result, Viera and Garret’s (2005, p. 362) recommendations are used, where a Kappa between 0.41–0.60 indicates moderate agreement, 0.61–0.80 substantial agreement and 0.81–0.99 an almost perfect agreement.

Inter-rater reliability

To explore this method for classification, three researchers (referred to as researchers 1, 2 and 3) in technology education were given the same random sample (as above) of 15 (9% of the total) student descriptions of technology and technology education. They were asked to follow the instructions in “Appendix” to judge which of the different aspects of Mitcham’s typology appeared in each student’s description. To measure inter-rater reliability, Krippendorff’s alpha was used, between the three researchers’ ratings, on every part of the typology. Krippendorff’s’ alpha was calculated using IBM Statistical Package for the Social Sciences (SPSS) using Hayes and Krippendorff’s macro (2007). This statistical method simultaneously measures several raters to determine the classification method’s reproducibility. The aim in this type of study is to achieve an alpha value above .667 (Krippendorff 2004). Inter-rater agreement calculations are sensitive to rare findings, meaning that a low alpha does not have to mean low agreement (Viera and Garrett 2005). All three participating researchers are well acquainted with the philosophical framework used here: why percentage agreement can serve as a good benchmark (McHugh 2012). Therefore, the percentage agreement between the three researchers is calculated as well.

Data collection

Four schools were visited to gather data to explore the method of classifying student descriptions of technology. The schools were selected based on differences in location and background, with the intention of getting a variety of school types. The sample schools include classes from two public schools (one suburban school and one small-town school) and two private schools (one urban and one countryside suburban). The total number of participating students was 173 (ages 12–15). The participants completed an attitude questionnaire in which their descriptions of technology form the first part of the survey. An important factor affecting students’ performances (and thus their descriptions of technology) is socioeconomic background (Sirin 2005). The descriptions of technology may be more similar than if we were to study 173 students from a greater variety of schools. The intention is to study the method used to analyze student descriptions, rather than to provide a representative distribution of the breadth of technology descriptions.

The ethical principles for research in the humanities and social sciences set out by the Swedish Research Council regarding “information, consent, and confidentiality, and stipulating how research data may be used” were employed (Gustafsson et al. 2006, p. 83). All the participating students were informed about the study, its use for research purposes and their right to choose to participate or not. All respondents participated anonymously and completed the whole survey. Since some of the participating students were below 15 years old, information about the study was given to the teacher responsible for the class and the school’s principal, and they agreed for the students to take the survey during class.

Results

Firstly, the intra-rater agreement for the one participating researcher’s classifications and the inter-rater agreement between the three researchers’ classifications are presented. This is followed by a presentation of the results of the classification within the four aspects of technology, both as a distribution across the different categories and the distribution of the students’ Mitcham scores (0–4). Nine of the 173 students were excluded because they made no written response at all to any of the open questions. Therefore, in total, 164 student descriptions were analyzed.

Classification reliability

The classifications of the 15 randomly selected student descriptions were examined in order to determine how well the researcher who completed the classification on two separate occasions agreed with herself. In the Cross-tabulation (Table 3), it is shown that the researcher made different interpretations of the student descriptions four times out of the possible 75 (percentage agreement .947). This results in a Kappa value of .893 (Cohen’s Kappa, p < .001), where a Kappa above .8 is considered as an almost perfect agreement (Viera and Garrett 2005).

Table 3 Researcher 1st * Researcher 2nd. Crosstabulation of the rater between 1st and 2nd classification
Full size table

The next step is to compare the three raters when using the template in “Appendix”. Two of the categories are considered as rare findings, Volition, and Objects (see results of descriptions in Fig. 2). Volition is a rare finding since it is rarely included in student descriptions. On the other hand, most students describe technological objects in their descriptions, therefore not finding technological objects in the descriptions is rare. These rare findings can lead to a high alpha reliability being harder to achieve in these two categories. Therefore, the percentage agreement is presented as well (Table 4).

Fig. 2
figure 2

(Adapted model from Mitcham 1994 p. 160)

Percentage distribution of the different aspects of the Mitcham score.

Full size image
Table 4 Krippendorff’s alpha inter-rater reliability for researchers 1 (R1), 2 (R2) and 3 (R3) and percentage agreement between all three researchers
Full size table

The Krippendorff’s alpha, in this case, is below the recommended .667 (Krippendorff 2004), for Objects and Volition in the typology. In the two aspects of Knowledge and Activities, Krippendorff’s alpha is above the recommended value and the observed percentage agreement is relatively high. Student descriptions where all three researchers totally agree on the classification varies from 60 to 87% (or 9/15, to 13/15). The effect on Krippendorff’s alpha due to rare findings sensitivity is clear when comparing the alpha between “No definition”, “Activities” and “Knowledge”. All three have the same percentage agreement but receive different alpha values. The aspects of Objects and Volition are low in both alpha and percentage agreement.

Descriptions of technology

The classification was made on the 164 valid respondents’ descriptions. This gave two different types of results. The first result consists of which aspects of technology are found in the descriptions (presented in Fig. 2). The second result is the students’ Mitcham Score when counting how many aspects of technology are present in a description (presented in Table 5). As explained in the introduction, the four aspects of technology could be expressed as knowledge about technology along with the human drive to create and improve leading to developing, manufacturing and using of new technological objects, where technological knowledge and volition lie within the human being and technological activities and objects are seen as outcomes of the first two aspects, In Fig. 2, the distribution of the four aspects is placed within this model. In theory, a student can express any one or more of the aspects in any combination (for example, technological activities and knowledge). The participating students in this study rarely describe volition as an aspect of technology. The most common types of description contain technological objects and/or activities.

Table 5 Distribution of the students’ Mitcham Score, expressing the breadth of student descriptions
Full size table

Of the 164 students, 14.6% describe technology as something that does not fit the model of Mitcham’s four aspects of describing technology (Table 5). Examples of such answers are: “Don’t know”, “Technology is technology” and “Not fun. Love soccer”. The most common description is technology as different kinds of technological objects. However, 65.9% of the students state that there is more to technology than objects (Mitcham Score 2–4). Of the 164 participating students, 44.5% of them describe technology as something that has to do with or requires knowledge.

Limitations of results

The most difficult classification judgments occur in the aspects of technological objects and volition. All three researchers in the inter-rater results classify the 15 students’ descriptions consistently. This means that if one researcher considers one type of written response to be an expression of technological objects and another researcher does not, the differences become obvious. Objects are difficult to agree on since students can use vague concepts when describing objects, such as things, gadgets or stuff. In general, the disagreements are recurrent based on the individual researchers’ interpretations of these aspects. One of the researchers consistently considered the word “inventions” to be an indication of volition, while the other two did not. The interpretation of inventions as volition explains 3 of the 5 differences in the classification of the descriptions. In the technological objects aspect, the three researchers disagreed in 6 of the 15 classifications. Four of the differences in object classification is explained by different interpretations of things or stuff. The student description that caused most disagreement in the classification was:

a “Technology is everything technical. It is everything that works in a technical way. I consider that it is everything technical.”

b “A very practical subject where you learn about how everything in society works.”

  • Researcher 1 classified this description as not fitting into any of the four aspects.

  • Researcher 2 classified this description as technological objects, activities, and knowledge.

  • Researcher 3 classified this as technological knowledge.

One can argue that all three researchers are correct in some way. Researcher 1 considers the student to be explaining technology by using the word technology and that the description is so broad that it could explain almost any topic. Researcher 2 considers the student to be describing how things work (knowledge and object) and that it is practical (activities). Researcher 3 is focusing on learning about how society works (knowledge).

It is also questionable how much information can be gathered about what students perceive as technology from a short description in a survey. Even though the two questions for students in this study seem to capture a broader way of describing technology than in previous studies (presented in Table 1), the volitional aspect does not seem to come naturally to their descriptions (less than 5%).

Discussion

The aim of this study was, firstly, to develop a suitable method to analyze and quantify students’ descriptions of technology. This was intended to enable such quantification for an analysis of the correlation between attitudes and descriptions of technology. The classified descriptions can be used in two different ways. Firstly, there is the classification of the different aspects that are present in the descriptions. Secondly, the breadth of students’ descriptions can be used, through The Mitcham Score. The method for classification has been developed, but is in need of some consideration for future users.

The results of the intra-rater agreement calculations show that this method is sufficiently sustainable, with Kappa .893. This result must be considered very good, especially since every student description has 16 different possible ratings (by chance). A more difficult judgment occurs when there are several raters (in this case three). Krippendorff’s alpha was used to measure the level of agreement between all three raters simultaneously, with consideration of the results due to chance. Problems occurred in the technology as Object and Volition categories, which can both be considered as rare findings (Viera and Garrett 2005), while the aspects of Activities and Knowledge are present in half of the student descriptions and receive high alpha values of .819 and .778, respectively. The effect of rare findings can be exemplified by comparing the alpha values in Activities and Knowledge to the No definition category, with an alpha of .560. It is notable that the percentage agreement is the same for all these categories (87%) with the only difference being that the No definition category is more rarely expected. Taking both the Krippendorff’s alpha and the percentage agreement into account, No definition, Knowledge, and Activities are reliable aspects. A conclusion is that it is more difficult to classify and agree in the categories of Volition and Objects. Therefore, it is important for a research group to agree on some recurring issues, which can be context dependent, such as technological objects in this study (described by students as “things”, “stuff” etc.). For future use, the guide used in this study (“Appendix”) is in need of further clarification regarding examples of what can be classified as Objects and Volition. By using a small sample of the descriptions gathered, one can find general differences in the classification and visualize some of them to reach a joint understanding. This can lead to a joint view of what should be considered as technological objects and what should not. The volitional aspect is rarely described in students’ descriptions (and is difficult for raters to agree on), thus the Mitcham Score could advantageously be reduced to a 0–3 scale, with the omission of Volition. Alternatively, researchers could find another way to extract students’ thoughts on the volitional aspect, if it is possible. Another approach is to use a broader scale than the dichotomies used in this study. For example, using a scale from 0 to 1 (e.g. 0, 0.5, and 1), where 0.5 applies to the cases that can be interpreted differently, and uncertainties. The classification is not intended to be used on single individuals; therefore, minor differences between raters could be acceptable.

In comparison with other studies researching students’ descriptions of technology, there are some clear similarities. To enable the comparison of studies, this research is placed in the same table as previous studies (Table 6). In the distinct aspects of Objects and Activities (cf. Burns 1992; Dudziak and Szydlowski 1987; Rennie and Jarvis 1995) the percentage distribution is quite similar to this study. The aspect of Volition is only distinct in DiGironimo’s (2011) study and rarely occurs in student descriptions in either DiGironimo’s or in this study (3% and 5% of students’ descriptions, respectively). As mentioned earlier, technological knowledge has not been a distinct aspect in previous studies, therefore a comparison becomes difficult.

Table 6 Comparison of Mitcham’s aspects of technology with this and previous studies. Numbers in bold are distinct aspects
Full size table

With the development of this method, it becomes possible to study the different aspects of technology, according to Mitcham’s framework, that students cover in their descriptions. The method also allows us to analyze how broad students’ descriptions of technology are, by using the Mitcham Score. It is important to acknowledge the fact that there are students, at ages 12–15, who can describe technology using all four of Mitcham’s aspects of technology. The results tell us that technology, among these Swedish students, is commonly described as technological objects. More than 65% of the students also stated that technology is more than just objects; it has to do with knowledge and/or activities as well (Mitcham Score ≥ 2). Yes, technological objects is the most common way to describe technology in this study as well as being mentioned in previous research (Burns 1992; DiGironimo 2011; Garmire and Pearson 2006; de Vries 2006; Kőycű and de Vries 2015; Riis and Augustsson 1991). However, we must emphasize the fact that a majority of the students in this study describe technology in a broader way than as technological objects.

By using this method to classify students’ descriptions of technology, qualitative data becomes quantifiable. This makes it possible to perform correlation studies between different factors and to compare schools, or to use the data when performing attitude surveys. The method can also give us insights into what kind of technology their attitude is connected to. For instance, a student can be surveyed as having a positive attitude toward technology, while also describing it in a narrow way (Mitcham Score 0–1); is this a desirable attitude?

To be able to use the different aspects appearing in student descriptions in attitude studies, Ankiewicz’s (2019) approach using Mitcham’s framework as an explanatory model for attitudes can be used (Fig. 3). The affective part of Attitudes, measured in PATT studies, is very similar to Volition. At the same time, the volitional aspect is rarely described in students’ descriptions of technology and is difficult to for raters to agree upon. The PATT-SQ (see Ardies et al. 2013, p. 14; Svenningsson et al. 2018, p. 71) surveys precisely this: students’ perceived difficulties in technology, their interest in gathering more knowledge about technology and so on. By inserting one of the attitudinal categories (e.g. difficulties, interest, gender), one can study the different aspects described together with students’ attitudes. For instance, one could compare the different aspects that are highlighted by students with a high interest in technology education, compared to those with low interest. Thus, we would be able to visualize that, for instance, high-interest students might be more likely to see technological activities as an aspect of technology.

Fig. 3
figure 3

Model used to combine PATT with the Mitcham Score (cf. Ankiewicz 2019, p. 9; Mitcham 1994, p. 160)

Full size image

The Mitcham Score, on the other hand, which investigates the breadth of students’ descriptions of technology, can be used to conduct correlation studies with students’ attitudes. The Mitcham Score adds another factor that affects attitudes, apart from gender, age, socioeconomic background and so on. This new factor can be included as part of a student’s attitudinal profile. Furthermore, it can be used as a factor in performing Structural Equation Modelling (SEM) studies to investigate the relations between different factors, including the Mitcham Score, and attitudes.

Another possibility is to create a questionnaire, similar to Liou’s (2015), within the Mitcham framework. Even though a questionnaire eases the researcher’s work, it would lead to a longer survey, which contradicts the intention of creating the PATT-SQ. One would also miss the opportunity to extract and identify potential changes in students’ own written descriptions of technology.

To conclude, this method, ease to highlight the aspects present in students’ descriptions of technology and which aspects that are not, within Mitcham’s (1994) philosophical framework. In the framework, the human being is central and based on our knowledge and volition. The aspect of technological knowledge among student descriptions was prior to this study not a distinct aspect in analyses. In this study, 44.5% of the participating students exemplify technology and/or technology education as something that requires knowledge. This also means that 55.5% of the students do not express technological knowledge in their descriptions. Only 4.9% of the students include a volitional aspect when they describe technology. A majority of the students do not describe technology as something that requires knowledge or is driven by human will.

As further research, apart from using the Mitcham Score in attitude research, it would be interesting to create sub-concepts within these four aspects, similar to the research by Burns (1992), and Rennie and Jarvis (1995). What types of Objects, Knowledge, Activities, and Volition are present in students’ descriptions of technology? This can also open up the possibilities to use the Mitcham Score in a non-attitudinal context and use the scoring procedure as a tool to evaluate in-class interventions as done by Svenningsson (2019). If we find it important that students possess a broad view of technology, then, what can we do to increase their awareness of different aspects of technology? More specifically how to increase their awareness of human interaction with technology, technological knowledge and volition? If any specific aspects are generally missing, for instance, in a school class, the specific aspect can be illuminated in future education.