Technological problem solving: an investigation of differences associated with levels of task success

Research into technological problem solving has shown it to exist in a range of forms and draw upon different processes and knowledge types. This paper adds to this understanding by identifying procedural and epistemic differences in relation to task performance for pupils solving a well-defined technological problem. The study is theoretically grounded in a transformative epistemology of technology education. 50 pupils in small groups worked through a cantilever problem, the most and least successful solutions to which were identified using a Delphi technique. Time-interval photography, verbal interactions, observations and supplementary data formed a composite representation of activity which was analysed with successively less contrasting groups to isolate sustained differences. Analyses revealed key differences in three areas. First, more successful groups used better and more proactive process-management strategies including use of planning, role and task allocation with lower levels of group tension. Second, they made greater use of reflection in which knowledge associated with the technological solution was explicitly verblised. This was defined as ‘analytical reflection’ and reveals aspects of pupils’ qualitative technical knowledge. Third, higher-performing groups exhibited greater levels of tacit-procedural knowledge within their solutions. There was also evidence that less successful groups were less affected by competition and not as comprehensive in translating prior conceptual learning into their tangible technological solutions. Overall findings suggest that proactive management, and making contextual and technical connections, are important for pupils solving well-defined technological problems. This understanding can be used to support classroom pedagogies that help pupils learn to problem solve more effectively.


Introduction
Problem solving is an activity, a context and a dominant pedagogical frame for Technology Education. It constitutes a central method and a critical skill through which school pupils learn about and become proficient in technology (Custer et al., 2001). Research has, among other things, been able to identify and investigate sets of intellectual and cognitive processes (Buckley et al., 2019;Haupt, 2018;Johnson, 1997;Sung & Kelly, 2019) and shown there to be conceptual, procedural, relational and harderto-get-to forms of 'technological knowledge' involved when pupils develop technological solutions (de Vries, 2005;McCormick, 1997McCormick, , 2004Rauscher, 2011). Some authors argue that technological problem solving (and design) is a situated activity (Jackson & Strimel, 2018;Murphy & McCormick, 1997;Liddament, 1996), but with social and context-independent processes also playing an important role (e.g. Jones, 1997;Winkelmann & Hacker, 2011). Within and across this vista, there has been strong interest in more open-ended, creative and design-based problem-solving (Lewis, 2005(Lewis, , 2009, which Xu et al. (2019) notes became particularly prominent after 2006. These studies have helped to understand some of the challenges and pedagogies of design (Gómez Puente et al., 2013;Lavonen et al., 2002;Mioduser & Dagan, 2007;Mawson, 2003) including those that mitigate effects such as cognitive fixation (e.g. McLellan & Nicholl, 2011). Problem solving, it seems, is a pervasive idea in technology education research and policy. Middleton (2009) notes that problem solving is found in almost all international technology education curricula.
The pace, nature and complexity of contemporary societal challenges make it more critical than ever that technology classrooms prepare people who can think through and respond to technological problems effectively. It requires that we strengthen our understanding in ways that will ultimately be powerful for shaping classroom learning. One way of contributing to this is to learn more about the differences between learners who are more and less successful at technological problem solving. Studies that share a comparative perspective and/or a focus upon task success are relatively few. Doornekamp (2001) compared pupils (circa 13 years old) who solved technological problems using weakly structured instructional materials with those using strongly structured materials. It was shown that the latter led to statistically significant improvements in the quality of the technical solutions. More recently, Bartholomew & Strimel (2018) were able to show that, for open-ended problem solving, there was no significant relationship between prior experience and folio creation, but that more in-school school experience of open-ended problem solving corresponded to higher ranked solutions.
This paper contributes to this work by reporting on a study that compares groups of pupils during technological problem solving in order to identify areas of difference and the factors associated with more successful outcomes. Specifically, it addresses the question: 'In terms of intellectual processes and knowledge, what are the differences in the modi operandi between groups of pupils that produced more and less successful technological solutions to a well-defined problem?' Theoretically grounded in a transformative epistemology of technology education (Morrison-Love, 2017), the study identifies prominent procedural and epistemic differences in pupils' thinking and technical solutions. Groups of pupils engaged with a structures problem requiring them to develop a cantilever bridge system which would facilitate safe travel across a body of water.
The paper begins by setting out the theoretical basis and conceptual framework for investigation before describing the comparative methodological and analytical approaches that were adopted. Following an analysis and presentation of key findings, conclusion and implications are discussed.

A theoretical basis for the study of technological problem solving
Despite there being no single comprehensive paradigm for technological problem solving, a theoretical grounding and conceptual framework necessary for this study are presented. At the theoretical level, this study is based upon a 'transformative epistemology' for technology education (Morrison-Love, 2017). From this, a ternary conceptual framework based upon mode, epistemology and process is developed to support study design and initiate data analysis.
A transformative epistemology for technology education (Morrison-Love, ibid) proposes that pupils' technological knowledge and capability arises from the ontological transformation of their technical solution from 'perdurant' (more conceptual, mutable, less well-defined, partial) in the early stages, to 'endurant' (comprehensive, tangible, stable over time) upon completion. It proposes that technical outcomes exist in material and tangible forms and that to be technological (rather than, for example, social, cultural or aesthetic) these must somehow enhance human capabilities in their intended systems of use. For this study, the ideas of transformative epistemology support problem solving in which pupils build technological knowledge by iteratively moving from concept to tangible, material solution. Moreover, it means pupils are successful in this when their solutions or prototypes: (1) enhance existing human capabilities in some way, and (2) are sufficiently developed to be stable over time, beyond the problem-solving activity that created it.

A conceptual framework for technological problem solving
A ternary conceptual framework ( Fig. 1) of mode, process and epistemology was developed from the literature in which the knowledge and cognitive/intellectual processes used by pupils are enacted in the 'process application block'. This is like the 'problem space' described in a model proposed by Mioduser (1998). Collectively, the goal of creating a physical artefact, the solution itself, the epistemic and procedural dimensions reflect the four dimensions of technology identified by Custer (1995).

Mode and forms dimension
Although problem solving may be 'technological', several classifications of both problem type and problem solving are found in the literature. Ill-defined and well-defined problems build upon the earlier work of information processing and cognitive psychology (see Jonassen, 1997). Typically, these two forms reflect different extents to which the outcome is specified to the solver at the outset. Ill-defined problems are strongly associated with design and creativity, and Twyford and Järvinen (2000) suggest that these more open briefs promote greater social interaction and use by pupils of prior knowledge and experience. Additionally, two forms of troubleshooting were identified in the literature: emergent troubleshooting and discrete troubleshooting. MacPherson (1998) argues that 'troubleshooting' constitutes a particular subset of technological problem solving-something earlier recognised by McCade (1990), who views it as the identification and overcoming of problems encountered during the production or use of a technical solution. In this study, emergent troubleshooting occurs in the process of developing solutions in response to emergent problems (McCormick, 1994). Discrete troubleshooting is a process in which significant technical understanding is applied in a structured way (Schaafstal et al., 2000) to resolve something about an existing artefact.

Intellectual and cognitive process dimension
Studies often conceptualise cognitive processes discretely rather than hierarchically, and different studies employ different process sets. Williams (2000), identifies evaluation, communication, modelling, generating ideas, research and investigation, producing and documenting as important to technological problem solving, while DeLuca (1991) identifies troubleshooting, the scientific process, the design process, research and development, and project management. There are also studies that employ specific, or more established, coding schemes for sets of intellectual and cognitive processes. A detailed analysis of these is given Grubbs et al. (2018), although the extent to which a particular process remains discrete or could form a sub-process of another remains problematic. In DeLuca's (1991) break down for example, to what extent are research and investigation part of design and does this depend on the scale at which we conceptualise different processes?
Regardless of the processes a study defines, it is typically understood that pupils apply them in iterative or cyclic fashion. This is reflected across several models from Argyle's (1972) 'Motor Skill Process Model' (perception-translation-motor response) through to those of Miodusre and Kipperman (2002) and Scrivener et al. (2002) (evaluation-modification cycles) which pertain specifically to technology education. All these models bridge pupils' conceptual-internal representations with their practical-external representations as they move towards an ontologically endurant solution and this is captured by the 'Re-Application/Transformation Loop' of the conceptual framework. Given that little is known about where differences might lie, the process set identified by Halfin (1973) was adopted due to its rigour and the breadth of thinking it encompasses. This set was validated for technology classrooms by Hill and Wicklein (1999) and used successfully by other studies of pupils technological thinking including Hill (1997), Kelley (2008) and Strimel (2014).

Epistemology dimension
The nature and sources of knowledge play a critical role for pupils when solving technological problems, but these remain far from straightforward to conceptualise. McCormick (1997) observes that the activity of technology education, and its body of content, can be thought of as 'procedural knowledge' and 'conceptual knowledge' respectively. Vincenti (1990), in the context of Engineering, makes the case for descriptive knowledge (things as they currently are) and prescriptive knowledge (of that with is required to meet a desired state) but also recognises knowledge can take on implicit, or tacit forms relating to an individual's skill, judgement, and practice (Polanyi, 1967;Schön, 1992;Sternberg, 1999;Welch, 1998). Arguably, moving from concept to physical solution will demand from pupils a certain level of practical skill and judgement, and Morgan (2008) observes that procedural knowledge which is explicit in the early stages becomes increasingly implicit with mastery. Notably, in addition to conceptual, procedural and tacit forms of knowledge, there is also evidence that knowledge of principles plays a role. Distinct from impoverished notions of technology as 'applied science ', Rophol (1997) shows that it is often technological principles, laws and maxims that are applied during problem solving rather than scientific ones. Frey (1989) makes similar observations and sees this form of knowledge arising largely from practice. In this study, knowledge of principles involves knowledge of a relationship between things. It is not constrained to those that are represented scientifically.
The conceptual framework finally accounts for pupils' sources of knowledge during problem solving, building principally on a design knowledge framework of media, community and domain presented by Erkip et al. (1997). In this study, media includes task information, representations and materials; community includes teachers and peers, and domain relates to prior technological knowledge from within technology lessons and prior personal knowledge from out with technology lessons. Finally, the developing solution is itself recognised a source of knowledge that pupils iteratively engage with and reflect upon, even when it appears that limited progress in being made (Hamel & Elshout, 2000).

Methodology
The research question in this study is concerned with differences in the knowledge and intellectual processes used by pupils in moving from a perdurant to an endurant technical solution. From an exploratory stance, this elicits a dualistic activity system involving pupils' subjective constructions of reality as well as the resultant tangible and more objective material solution. The study does not aim to investigate pupils' own subjective constructions from an emic perspective, but rather seeks to determine the nature and occurrences of any differences during observable real-time problem-solving activity. As such, content rather than thematic analysis was used (Elo & Kyngäs, 2008;Vaismoradi et al., 2013) with concurrent data collection to build a composite representation of reality (Gall et al., 2003, p.14). Complementary data provided insights into study effects, the classrooms and contexts within which problem-solving took place.
This study assumes that should differences exist, these will be discernible in the inferred cognitive processes, external material transformations, interactions and verbalisation (even though this tends to diminish as activity becomes more practical). Absolute and objective observation is not possible. This study also accepts that data gathering and analysis are influenced by theory, researcher fallibility and bias which will be explicitly accounted for as far as possible. Finally, while the conceptual framework provides an analytical starting point, it should not preclude the capture of differences that may lie elsewhere in the data including, for example, process that lie out with those identified by Halfin (1973).

Participants, selection and grouping
To support transferability, a representative spread of pupils from low, medium and high socio-economic backgrounds took part in this study. Purposeful, four-stage criterion sampling was used (Gall et al., 2003, p.179). Stage one identified six schools at each socioeconomic level from all Scottish secondary schools that presented pupils for one or more technology subjects with the Scottish Qualifications Authority. This was done using socioeconomic data from the Scottish Area Deprivation Index, the Carstairs Index and pupil eligibility for subsidised meals. Secondary school catchment areas were used to account for pupil demographics as accurately as possible. All eighteen schools were subsequently ranked with one median drawn from low, medium and high bands of socio-economic deprivation (School 1: Low, School 2: Medium, School 3: High).
One class in each school was then selected from the second year of study prior to pupils making specific subject choices to minimise variations in curricular experience. In total, 3 class teachers and 50 pupils (20 female, 30 male) aged between 12 and 13 years old took part in the study. The group rather than the individual was defined as unit of study to centralise verbal interaction.
None of the pupils participating in this study had experience of group approaches such as co-operative learning and it was likely that groups might experience participation effects including inter-group conflict and interaction effects (Harkins, 1987;Sherif et al., 1961), social loafing (Salomon & Globerson, 1989), free-rider (Strong & Anderson, 1990) and status differential effects (Rowell, 2002). Relevant also to this study is evidence suggesting that gender effects can take place in untrained groups undertaking practical/material manipulation activities. To maximise interaction between group members and the material solution, thirteen single sex groups averaging four pupils were formed in order to: (1) minimise the marginalisation of girls with boys' tendency to monopolise materials and apparatus in groups (Conwell et al., 1993;Whyte, 1984); (2) recognise boys' tendency to respond more readily to other boys (Webb, 1984) and, (3) maximise girls' opportunities to interact which is seen to erode in mixed groups (Parker & Rennie, 2002;Rennie & Parker, 1987). Hong et al. (2012) examines such gender differences in detail specifically within the context of technological problem solving. Teacher participation in group allocation minimised inter-group conflict and interaction effects although groups still experienced naturally fluctuating attrition from pupil absences (School 1 = 17.6%; School 2 = 2.5% and School 3 = 8.8%).

Identification of most and least successful solutions
The research question requires differences to be identified in terms of levels of success. The overall trustworthiness of any differences therefore depends upon the credible identification of the most and least successful solutions from the thirteen produced. Wholly objective assessment of the pupils' solutions is not possible, and material imperfections in different solutions negated reliable physical testing across the three classes. Moreover, because the researcher earlier observed pupils while problem solving, neutrality of researcher judgement in establishing a rank order of group solutions was equally problematic. Everton and Green (1986) identify this biasing risk between and early and later stages of research as a form of contamination.
To address these limitations, a Delphi technique was design using the work of Gordon (1994), Rowe and Wright (1999) and Yousuf (2007). This was undertaken anonymously prior to any analysis and, in conjunction with the results of physical testing, enabled the four most successful and four least successful solutions to be confidently identified independently of the researcher. A panel of eight secondary school teachers was convened from schools out with the study. All panel members had expertise in teaching structures with no dependent relationships or conflicts of interest. Following Delphi training, and a threshold level of 75%, the four most and four least successful solutions on outcome alone were identified after two rounds. Qualitative content validity checks confirmed that panel judgements fell within the scope of the accessible information. 37/43 reasons given were 'High', with six considered 'Medium' because the reasoning was partially speculative. When triangulated with additional evidence from physical testing, two cohorts of four groups were identified and paired to form four dyads (Table 1).

Study design
As noted, 'Structures' was chosen as a topic area and was new to all participants. It was appropriate for setting well-defined problems and allowed pupils to draw upon a sufficiently wide range of processes and knowledge types in developing a tangible, endurant solution. In discussion with the researcher, teachers did not alter their teaching style and adopted pedagogy and formative interactions that would support independent thinking, reasoning and problem solving. This study involved a learning phase, followed by a problem-solving phase.
In the learning phase, groups engaged over three fifty-minute periods with a unit of work on structures which was developed collaboratively with, and delivered by, the three classroom teachers. This allowed pupils to interact with materials and develop a qualitative understanding of key structural concepts including strength, tension and compression, triangulation, and turning moments. During this time, pupils also acclimatised to the presence of the researcher and recording equipment which helped to reduce any potential Hawthorne effect (Gall et al., 2003). Structured observations, teacher de-briefs and questionnaires were used in this phase to capture study effects, unit content coverage and environmental consistency between the three classrooms. Content coverage and environmental consistency were shown to be extremely high. Scores from the unit activity sheets that pupils completed were used to gauge group understanding of key concepts. The problem-solving phase took place over two circa 50-minute periods (range: 40-52 m) in which pupils responded to a well-defined problem brief. This required them to develop a cantilever bridge enabling travel across a body of water. This bridge would enhance people's ability to traverse terrain (conditions for being 'technological') with maximal span rigidity and minimal deflection (conditions for an ontologically 'endurant' solution). All groups had access to the same range and number of materials and resources and were issued with a base board showing water and land on which to develop their solutions.
While video capture was explored in depth (Lomax & Casey, 1998), challenges in reliably capturing solution detail resulted in group verbalisation being recorded as audio. This was synchronised with time interval photography and supplemented with structured observer-participant observation that focused on a sub-set of observable processes from the conceptual framework (Halfin, 1973). The developing technical solutions were viewed as manifestations of the knowledge and intellectual processes used by pupils at different points in time through their cognitive and material interactions. Photographs captured the results of these interactions in group solutions every 3-4 min but did not capture interactions between pupils. The structured observational approach adopted continuous coding similar to that found in the Flanders System of Interaction analysis (Amatari, 2015) and was refined through two pilot studies. During each problem-solving session, groups were observed at least twice between photographs and, following each session, pupil questionnaires, teacher de-briefs and solution videos (360° panoramic pivot about the solution) were completed to support future analysis. Reflexive accounts by the researcher also captured critical events, observer and study effects.

Analytical approach
All data were prepared, time-synchronised and analysed in three stages. Core verbal data (apx. 12h) and photographic data (n = 206) were triangulated with observational and other data against time. The problem-solving phase for each class was broken into a series of 3-4 min samples labelled S = 1, S = 2, S = 3…with durations in each recorded in minutes and seconds. Verbal data were analysed using NVivo software using digital waveforms rather than transcribed files to preserve immediacy, accuracy and minimise levels of interpretation (Wainwright & Russell, 2010;Zamawe, 2015). Photographic data were coded for the physical developments of the solutions (e.g. adding/removing materials in particular places) allowing solution development to be mapped for different groups over time. Triangulation of data also allowed coding to capture whether individual developments enhanced or detracted from the overall function efficacy of the solution.
The first stage of analysis was immersive, beginning with an initial codebook derived from the conceptual framework. In response to the data this iteratively shifted to a more inductive mode. To sensitise the analysis to differences, the most successful and the least successful groups were compared first as is discussed by Strauss 1987 (Miles & Huberman, 1994, p.58). Three frameworks of differences emerged from this: (1) epistemic differences, (2) process differences, and (3) social and extrinsic differences. These were then applied to dyads of decreasing contrast and successively refined in response to how these differences were reflected in the wider data set. Seven complete passes allowed non-profitable codes to be omitted and frameworks to be finalised. A final stage summarised differences across all groups.

Analysis and findings
The analysis and findings are presented in two main parts: (1) findings from the learning phase, and (2) findings from the problem-solving phase. Verbal data forms a core data source throughout and coding includes both counts and durations (in minutes and seconds). Direct quotations are used from verbal data, although the pupils involved in the study were from regions of Scotland with differing and often very strong local dialects. Quotations are therefore presented with dialect effects removed: Example data excerpt reflecting dialect: "See instead-e all-e-us watchin', we could all be doin' su-hum instead-o watchin' Leanne.." Example data excerpt with dialect removed: "See instead of all of us watching, we could all be doing something instead of watching Leanne.."

Part 1: Findings from the Learning Phase
Both teacher and researcher observation confirmed that pupils in all three classes engaged well with the unit of work (50 pupils across 13 groups) with all 40 content indicators covered by each class. Teachers of classes 1 and 3 commented that the lesson pace was slightly faster than pupils were used to. As expected, different teaching styles and examples were between classes, but all pupils completed the same unit activity sheets. The teacher of class 2, for example, used man-made structures and insect wings to explore triangulation; and the teacher in class 3 talked about the improved stability pupils get by standing with their feet apart. The understanding reflected in activity sheets was very good overall and Table 2 illustrates the percentage of correct responses for each class in relation to each of the three core concept areas.
Though unit activity sheets are not standardised tests, the conditions of administration, scoring, standards for interpretation, fairness and concept validity discussed by Gall et al. (2003, p.xx) were maintained as far as possible. Evidence did not show that representational/stylistic variations by teachers had any discernible effect on pupil understanding and was seen to maintain normality from the pupils' perspective. Class 3 scored consistently highly across all conceptual areas, although the qualitative understanding of turning moments was least secure for all three classes. Non-completion of selected questions in the task sheets partially explains lower numerical attainment for this concept in class 1 and 2, however, it is unknown if omissions resulted from a lack of understanding. The figures in Table 2 are absence corrected to account for fluctuating pupil attendance at sessions: (17.6% pupil absence across sessions for class 1, compared with 8.8% and 2.5% for classes 3 and 3 respectively). Table 3 illustrates the percentage scores for activity sheets completed by groups in the more and less successful cohorts. Observational and reflexive data highlighted evidence of some researcher and recorder effects. These were typically associated with pupils' interest in understanding the roles of the researcher and class teacher, and discussion around what they could say while being recorded. These subsided over time for all but two groups in Class 1, but with no substantive effect on pupils' technological thinking.
In summary, findings from the learning phase show that: (1) Pupils engagement was high, and all classes covered the core structural concepts in the unit; (2) pupil knowledge and understanding, as measured by activity sheet responses, was very good overall but scores for turning moments were comparatively lower, and (3) study effect subsided quite quickly for all but two groups and there was no evidence showing these to be detrimental to technological thinking. These differences are considered epistemic and are captured in the framework of difference in Fig. 5.

Part 2: findings from the problem-solving phase
Part 2 begins by describing the differences from comparing the material solutions produced by the most and least successful groups (dyad 1). Subsequent sections report upon the three areas in which difference were found: epistemic differences, process differences and social and extrinsic differences. Each of these sections lead with the analysis from the most contrasting groups (dyad 1) before presenting the resultant framework of difference. They conclude by reporting on how the differences in these frameworks are reflected across the wider data set. As with findings across all sections, findings only account for areas of the conceptual framework in which differences were identified. For processes such as measuring and testing, no difference was found and other processes, such as computing, did not feature for any of the groups.

Differences in the solutions produced by the most & least successful groups (dyad 1)
Group 5′s solution was identified as the most successful and Group7′s solution was identified as the least successful. Overall, both of these groups engaged well with the task and produced cantilevers that are shown in Figs. 2 and 3. The order in which different parts of the solutions were developed is indicated by colour with the lighter parts appearing earlier in problem solving than the darker parts. Figure 4 shows this cumulative physical development of each solution over time. Both groups shared a similar conceptual basis and employed triangulation above and below the road surface. Figure 4 shows that Group 5′s solution evolved through 36 developments, while Group 7 undertook 23 developments and chose to strip down and restart their solution at the beginning of the second problem solving session. Similarly, groups 6, 11 and 13 removed or rebuilt significant sections of their solution. Neither group 5 or 7 undertook any developments under the rear of the road surface, and the greatest percentage of developments applied to the road surface itself (Group 7: 30.6%; Group 5: 47.8%). For Group 5, it was only developments 5 and 6 ( Fig. 2) which offered little to no functional structural advantage. All other developments contributed to either triangulation, rigidity or strength through configuration and material choice with no evidence of misconception, which was also noted by the Delphi panel. The orientation, configuration and choice of materials by Group 7 share similarities with Group 5 insofar as each reflected knowledge of a cognate concept or principle (e.g. triangulation). Delphi Panel Member 8 described Group 7′s solution as having a good conceptual basis. Key differences, however, lay in the overall structural integrity of the solution and the underdevelopment of the road surface (Fig. 3, Dev.1 and Dev.5) which mean that Group 5 achieved a more ontologically endurant solution than Group 7 did. Evidence from Group 7′s discussion (S = 3, 3.34-3.37; S = 3, 3.38-3.39; S = 16, 3.26-3.30) suggests this is partly because of misconception and deficits in knowledge about materials and the task/cantilever requirements. This was also reflected in the group's activity responses during structures unit in the learning phase. Alongside the photographic evidence and reflexive notes of the researcher, this suggest that there was some difficultly in translating concepts and ideas into a practical form. This constitutes a difference in tacit-procedural knowledge between Group 5 and Group 7.

Epistemic differences during problem solving
As well as the knowledge differences in the learning phase and the physical solutions, analysis of the most and least successful groups revealed epistemic differences in problem solving activity related to 'task knowledge' and 'knowledge of concepts and principles'. The extent to which 'knowledge' can be reliably coded for in this context is limited because it rapidly becomes inseparable from process. Skills are processes which, in turn, are forms of enacted knowledge. Consequently, although Halfin (1973) defines idea generation as a knowledge generating process using all the senses, attempts to code for this were unsuccessful because it was not possible to ascertain with any confidence where one idea ended, and another began. Coding was possible, however, for 'prior personal knowledge', 'task knowledge' and 'prior technological knowledge'. The analysis of these is present along with the resulting framework of epistemic difference with prior personal knowledge omitted on the basis that no differences between groups was found. The final section looks at how epistemic differences are reflected in the activity of the remaining groups.

Epistemic differences between the most & least successful groups (dyad 1)
Task knowledge is the knowledge pupils have of the problem statement and includes relatively objective parameters, conditions, and constraints. One key difference was the extent to which groups explicitly used this to support decision making. Group 5 spent considerably more time than Group 7 discussing what they knew and understood of the task prior to construction (1m10s vs. 8 s) but during construction, had more instances where their knowledge of the task appeared uncertain or was questioned (n = 6 for Group 5 vs. n = 2 for Group 7). Differences were also found in the prior technological knowledge used by groups. This knowledge includes core structural concepts and principles explored in the learning phase. As with task knowledge, Group 5 verbalised this category of knowledge to a far greater extent than Group 7, both apart from, and as part of, formative discussions with the class teacher (18:59 s vs. 14:43 s). In only one instance was the prior technological knowledge of Group 5 incorrect or uncertain compared with four instances for Group 7. These included misconceptions about triangulation and strength despite performing well with these in the learning phase. Furthermore, some instances of erroneous knowledge impacted directly upon solution development. In response to a discussion about rigidity and the physical performance of the road surface, one pupil stated: "Yes, but it is supposed to be able to bend in the middle.." (Group 7, S = 3, 3.34-3.37) meaning that the group did not sufficient attend to this point of structural weakness which resulted in a less endurant solution. No such occurrences took place with Group 5. More prominent and accurate use of this type of knowledge supports stronger application of learning into the problem-solving context and appeared to accompany greater solution integrity. From these findings, and those from the learning phase, the framework of difference shown in Fig. 5 was developed:

Epistemic differences across all groups (dyads 1-4)
As with dyad 1, the more successful groups in dyads 3 and 4 scored higher (+ 14% and + 20.7%, respectively) in the learning phase compared with their less successful partner groups. This, however, was not seen with dyad 2. The less successful group achieved a higher average score of 86.3% compared with 71% and, despite greater fluctuations in pupil attendance, scored 100% for turning moments compared with 58% for the more successful group. Although comparatively minimal across all groups, more successful groups in each dyad tended to explicitly verbalise technological and task knowledge more than less successful groups. Furthermore, it was more often correct or certain for more successful groups. This was particularly true for dyad 2, although there was some uncertainty about the strongest shapes for given materials in, for example, Group 12 which was the more successful group of dyad 3. The greatest similarity in verbalised task knowledge was observed with the least contrasting dyad, although evidence from concept sketching (Figs. 6, 7) illustrated a shared misunderstanding between both groups of the cantilever and task requirements.
The differences in tacit-procedural knowledge between Group 5 and 7 were reflected quite consistently across other dyads, with more successful groups showing greater accuracy, skill and judgement in solution construction. The more successful groups in dyads 2 and 3 undertook three material developments that offered little to no functional advantage, and each of the developments these groups undertook correctly embodied knowledge of cognate structural concepts and principles. Notably, Group 8 of dyad 4 was able to achieved this with no structural redundancy at all. Less successful groups, however, were not as secure in their grasp of the functional dependencies and interrelationships between different parts of their structural systems. The starkest example of this was with Group 4 of dyad 3, who explicitly used triangulation but their failure to physically connect it with other parts of the structure rendered the triangulation redundant. Group 2 of dyad 4 were the only group not to triangulate the underside of the road surface. Less successful groups tended to focus slightly more of their material developments in areas of the bridge other than the road surface, whereas the opposite tended to be true for the other groups. Significantly, while all groups in the study included developments that offered little to no functional advantage, it was only in the case of less successful groups that these impaired the overall functional performance of solutions in some way. Table 4 summarises the sustained epistemic difference across all four dyads.

Process differences
Analysis of the most contrasting dyad yielded process differences in: (1) managing (Halfin, 1973), (2) planning, and (3) reflection. Groups managed role and task allocation differently, as well as engaging in different approaches to planning aspects of solution development. Reflection, as a process of drawing meaning or conclusions from past events, is not explicitly identified by Halfin or the conceptual framework. Two new forms of reflection for well-defined technological problem solving (declarative reflection and analytical reflection) were therefor developed to account for the differences found. The analysis of the process differences is presented with the resulting framework for this dyad. The final section presents sustained process differences across all groups.

Difference in managing-role & task allocation & adoption (dyad 1)
The autonomous creation of roles and allocation of tasks featured heavily in the activity of Group 5. These typically clustered around agreed tasks such as sketching (S = 2, 1.46), and points where group members were not directly engaged in construction. In total, Group 5 allocated or adopted roles or task on 31 occasions during problem solving compared with only 7 for Group 7. Both groups did so to assist other members (Group 5, S = 16, 3.33-3.38; Group 7, S = 3, 0.37-0.41), to take advantage of certain skills that group members were perceived to possess (Group 5, S = 2, 1.47-1.49; Group 7, S = 2, 2.03-2.06) and, for one instance in Group 7, to prevent one group member from executing something incorrectly (S = 16, 2.11-2.13). There was evidence, however, that Group 5 moved beyond these quite pragmatic drivers. Member often had more of a choice and, as shown in Excerpt 5, allocation and adoption is mediated by sense of ownership and fairness.  The effective use of role and task allocation appeared to play an important role in realising an effective technical solution, however, negative managerial traits were perhaps more significant.

Difference in managing-negative managerial traits (dyad 1)
Evidence of differences between Group 5 and 7 were found in relation to: (1) group involvement, and (2) fragmentation of group vision, which were found to be highly interrelated. Negative group involvement accounted for traits of dominance and dismissiveness. For Group 7, this was more prevalent earlier in the problem-solving activity where one group member tended to dominate the process. This pupil tabled 9 out of 11 proposals prior to working with physical materials and, at times, readily dismissed suggestions by other group members (See Excerpt 1). Moreover, ideas and proposals within the group were sometimes poorly communicated (Excerpt 2), which led to a growing level of disenfranchisement for some group members and a fragmented group vision for solution development.

Excerpt 1
Pupil 1:"We could do it that way…" (Pupils continue discussion without acknowledgement) Pupil 1:"You could do that.." Pupil 2:"Shut up, how are we going to do that?" Pupil 1:"Well you're allowed glue, and you're allowed scissors.." Several group members: "Shut-up!" (Group 7, S = 1, 2.07-2.28) Excerpt 2 "(Loud inhalation) Watch my brilliant idea… I need scissors.. Are you allowed scissors?" (Group 7, S = 1, 1.36-1.41) The was some evidence of dismissiveness present with Group 5 also (e.g. S = 9, 1.32-1.46), however, group members were able to voice their ideas which appeared to support a better shared understanding among group members. Notably, Group 5 reached a degree of consensus about what they would do prior to constructing anything, whilst Group 7 did not. Even in these early stages, two of the four members of Group 7 made it very clear that they did not know what was happening (Excerpt 3).

Excerpt 3
Pupil 1: "What are you all up to?" Pupil 2: "Move you" Pupil 4: "No idea" Pupil 2: "You're allowed to say hell are you not?" Pupil ?: "Helli-yeh" Pupil 2: "Hellilouya" (slight laughter) Pupil 3: "Right so were going to..(unclear) and do that.." Pupil 1: "What are you all up to?" Pupil 2: "Just… I know what he's thinking of.." Pupil 4: "I don't have a clue what you're thinking of.." Pupil 3: "Neither do I.." (Group 7,S = 2, Occurrence like these contributed to a growing sense of fragmentation in the group. Verbal and observational data show this to have been picked up by the class teacher who tried to encourage and support the group to share and discussed ideas more fully. Despite this, the group lost their sense of shared vision about how to approach a solution and, part way through the first session, two group members attempted to begin developing a separate solution of their own (S-3, 2.52).
The final managerial difference between Group 5 and 7 was the way in which efforts were made to increase the efficiency of solution development. Seen as a positive managerial trait, both groups did this, but it was more frequent and more developed with Group 5. There were four examples of this with Group 7 in the form of simple prompts to speed the process up (E.g. S = 5:3.02-3.04; S = 6:2.22-2.23; S = 11: 1.34-1.35) and 25 examples with Group 5 involving prompts and orchestrating parallel rather than successive activity.

Differences in planning (dyad 1)
Differences emerged in how Group 5 and 7 thought about and prepared for future problemsolving activity. While the complexity of the pupils' problem-solving prevented cause and effect from being attributed to planning decisions, four areas of difference were identified: (1) determining use of/amount of materials/resources, (2) sequencing, ordering or prioritising, (3) identification of global solution requirements, and (4) working through how an idea should be practically executed. Across both problem-solving sessions, Group 5 spent over three times as long as Group 7 did, engaging in these forms of planning (8m17s vs. 2.23 s), but Group 7 planned on almost twice as many occasions (n = 98 vs. n = 56). Both groups considered the availability of materials for, and matching of materials to, given ideas (e.g. Group 5, S = 5:3.38-3.48; Group 7, S = 4:2.20-2.34; S = 12:1.53-2.00) and both identified global solution requirements. At the start, Group 5 engaged in 12 min of planning in which they read task instructions (S = 1, 0.49-1.49), explored, tested, and compared the available materials (S = 1, 1.49-2.10), and agreed on a starting point. As shown in Excerpt 4, these discussions attempted to integrate thinking on materials, joining methods, placement. As the class teacher observed, Group 7 were eager to begin construction after 4m45s and did so without an agreed starting point. Pupils in this group explored materials in a more reactive way in response to construction. Despite similar types of planning processes, the planning discourse of Group 5 was more proactive, and this may have minimised inefficiencies and avoidable errors. For Group 7, two group members unintentionally drew the same idea (S = 2, 3.19-3.26), parts were taped in the wrong place (S = 17, 1.26-1.40) and others glued in the wrong order (S = 5, 1.28-1.30 and 1.48-1.56). Such occurrences, however, notably reduced after the group re-started their solution in the second session which also mirrored a 73% drop in poor group involvement. Communication played an important role in planning and there was no evidence of avoidable errors with Group 5.

Differences in reflection (dyad 1)
The most prevent differences in this study were found in how Group 5 and Group 7 reflected upon their developing solutions. Analysis revealed two main forms of reflection that were used differently by groups. 'Declarative reflection' lies close to observation and is defined by this study as reflection that does not explicitly reveal anything of a pupil's knowledge of technical relationships within their solution, e.g.: "that's not going to be strong…" (Group 7, S = 2, 0.49-0.51). This form of reflection was critical for both groups who used it heuristically to quality assure material developments, but it was used slightly more often by Group 7 (n = 164:4m30s vs. n = 145:4m07s). By contrast, 'analytical reflection' is defined as that which does reveal something of a pupil's knowledge of technical relationships between two or more parts of a solution. Examples of this are shown in Excerpts 5 and 6 where pupils are reflecting upon an attempt made to support the underside of the road surface.

Excerpt 5:
"It's not going to work because it's in compression and straws bend.." (Group 5, S = 9, 2.3-2.35) Excerpt 6: "no, that'll be… oh, aye, because that would weight it down and it would go into the water." (Group 5,S = 14, Looking across verbal and observational data, there was no consistent pattern to the use of declarative reflection but analytical reflection for both groups was almost exclusively anchored around, and promoted by, the practical enactment of an idea and could be associated with predictions about the future performance of their solution. Overall, both Group 5 and 7 reflected a similar number of times (n = 216 and n = 209, respectively) although the total amount of time spent reflecting was 17% longer for Group 5. This difference in time was accounted for by comparatively more analytical reflection in Group 5 (n = 75:3m47s vs. n = 45:2m10s for Group 7), particularly during the first half of problem solving. It was also interesting that Group 7 engaged with no analytical reflection at all prior to construction.
Findings from process management, planning and reflection led to the framework of difference in Fig. 8. This also accounts for differences in the amount of time each group reflected upon the task detail, but this was extremely limited (Group 5: n = 7, 26 s; Group 7: n = 5, 10 s).

Process differences across all groups (dyads 1-4)
Task reflection, attempts at increasing efficiency and differences of fragmented vision found with the most contrasting dyad were not sustained across remaining groups. The only sufficiently consistent difference in patterns of solution development was that more successful groups, on average, spent 18% longer in planning and discussion before beginning to construct anything.
Overall, the nature and patterns of good and poor group involvement from dyad 1 were reflected more widely, with some instances of deviation. The more successful group in dyad 4 had more significant and numerous examples of poor group involvement than did the less successful group (n = 16 vs. n = 10), although they made more effective use of roles and task allocation and spent longer engaged in planning processes. Dyad 2 deviated also insofar as the less successful group (13) actually had fewer avoidable errors than Group 6 who accidentally cut the incorrect parts (e.g. S = 15, 2.44-2.47), undertook developments that were not required (e.g. S = 6, 2.11-2.16) and integrated the wrong parts into their solution (e.g. S = 7, 1.10-1.13).
Differences in the nature and use of reflection was one of the most consistently sustained findings between the most and least successful cohorts. All four of the more successful groups engaged more heavily in reflective processes and more of this reflection was analytical in nature. This shows that reflection which explicitly integrates knowledge of technical relationships between different aspects of a solution plays an important role in more successful technical outcomes. Whilst declarative reflection remained important for all groups, it was also less prominent for groups in the less successful cohort. Table 5 summarises the sustained process difference across dyads 1, 2, 3 and 4.

Social & extrinsic differences (dyad 1)
Differences reported in this section lie out with the formal conceptual framework of the study but, nonetheless, were shown to play a role in the technological problem-solving activity of dyad 1. Differences between Group 5 and 7 emerged in three areas: (1) group tension, (2) effects of the classroom competitive dynamic, and (3) study effects. Group tension, which relates to aspects of interaction such as argumentative discourse, raised voices and exasperation, were negligible for Group 5 (n = 4, 0m24s) when compared with Group 7 (n = 38, 2m38s) and related exclusively to pupils having their voiced heard. For group 7, tension was evident during both sessions, but was more significant in the first session before re-starting the solution in session 2 and purposeful attempts to work more collaboratively with the support of the teacher (Group 7, S = 10, 0.36-1.29). Observations revealed that tension was typically caused by pupils failing to carry out practical processes to the standard of other group members, or breaking parts such as the thread supporting the road surface in the 36 th minute of Session 2.
Despite collaborative efforts within groups, there was a sense of competitive dynamic which appeared either to positively bias, negatively bias, or to not affect group activity. This competitive dynamic was present in groups comparing themselves to other groups in the class. Group 7 had 3.7 times as many instances of this as Group 5 with 73% of these negatively affecting the group. These included interference from and with other groups (S = 7, 0.07-0.12), attempting to copy other groups (S = 7, 1.14-1.22) and comparing the solutions of other groups to their own (S = 8, 2.55-2.59). In contrast, Group 5 appeared to be far less affected by perceptions of competition. Around a third of instances were coded as neutral, however, Group 7 experienced more instances of positive competitive effects than Group 5 did (n = 5 vs. n = 1).
Study effects were present for both groups often triggered by the arrival of the researcher at their table to observe or take photographs. The biggest difference in study effects was associated with the audio recorder. Recorder effects for Group 7 were three and half times that of Group 5 involving discussion about how it worked (Group 7, S = 10, 3.04-3.17), or about what was caught or not caught on tape (Group 7, S = 14, 1.01-1.45). Although questionnaire data showed that pupils in Group 5 felt that they talked less in the presence of the recorder, this was not supported by observations, verbal data, or the class teacher. From these findings, the framework of social and extrinsic difference in Fig. 9 was developed.

Social & extrinsic differences across all groups (dyads 1-4)
Most of the social and extrinsic differences identified with Groups 5 and 7 were reflected to greater or lesser extents in other dyads. In addition to less successful groups being more susceptible to researcher and recorder effects, two specific points of interest emerged. Firstly, group tension was considerably more prominent for less successful groups than it was for more successful groups. Although no evidence of a direct relationship was established, tension appeared to accompany poor managerial traits and the changing of group composition (e.g. Group 8, Group 13). The most significant differences in tension were found with dyad 3. No occurrences were found for the most successful group and 29 were seen with the least successful group including aggressive and abrupt communication between pupils involving blame for substandard construction (S = 10, 2.28-2.38), through to name calling (S = 12, 0.20-0.22), arguing (S = 6, 1.46-2.10) and threats of physical violence (S = 11, 3.25-3.29). Secondly, the more successful groups were influenced by the competitive class dynamic more than the less successful groups were. This is the only sustained finding that directly opposes what was found with dyad 1. These took the form of neutral or negative inter-group effects involving comparing and judging other groups (e.g. Group 6), espionage, copying or suspicion thereof (e.g. Group 6, 8 and 12). Table 6 summarises the sustained social and extrinsic differences across the more and less successful cohorts.

Discussion and Conclusions
This study established and applied three frameworks to capture the epistemic, procedural, and social and extrinsic differences between groups of pupils as they developed solutions to a well-defined technological problem. Social & extrinsic findings revealed higher levels of group tension for the less successful cohort, but that more successful groups elicited more negative responses to the competitive class dynamic created by different groups solving the same problem. Major findings about differences in knowledge and process are discussed. Thereafter, a three-part characterisation of thinking for well-defined technological problem solving is presented in support of pedagogy for Design & Technology classrooms.
The most important of those knowledge differences uncovered were found in: (1) the material development of the solution itself, and (2) the reflective processes used by groups during problem solving. The conceptual framework characterises 'tacit-procedural knowledge' as the implicit procedural knowledge embodied in technical skill, accuracy and judgement, and this was further refined in the solutions of more successful groups. Linked to this was the fact that several of the material developments for triangulation and strength were improperly realised by less successful groups which negatively impacted on the functional performance of their solutions. Often, this was despite evidence of a good conceptual understanding of triangulation, tension, and compression in the learning phase. An ontologically endurant solution requires stability over time and lesser developed aspects of tacit-procedural knowledge and knowledge application meant that this was not realised as fully as possible for some groups.
This can be partly explained by the challenge of learning transfer, or more accurately, learning application. Several notable studies have explored these difficulties in technology education (Brown, 2001;Dixon & Brown, 2012;Kelly & Kellam, 2009;Wicklein & Schell, 1995), but typically at a subject or interdisciplinary level. The findings of this study suggest that, even when the concepts in a learning unit are tightly aligned with a welldefined problem brief, some pupils find difficulty in applying them within a tangible, material context. It could be argued that more successful groups were better at connecting learning between different contexts associated with the problem-solving task and could apply this with more developed skill and judgement.
The second important knowledge difference arose in the various forms of reflection that groups engaged with. Reflection in this study supports pupils in cycling through the reapplication/transformation loop in a similar way to the perception/translation/evaluation blocks of the iterative models of problem solving (Argyle, 1972;Miodusre & Kipperman, 2002;Scrivener et al., 2002). Surprisingly few studies explore 'reflection' as a process in technological thinking (Kavousi et al., 2020;Luppicini, 2003;Lousberg et al., 2020), and fewer still in the context of school-level technological problem solving. This study found that more successful groups reflected more frequently, and that more of this reflection was analytical insofar as it explicitly revealed knowledge of technical relationships between different variables or parts of their solution. Such instances are likely to have been powerful in shaping the shared understanding of the group. This type of reflection is significant because it takes place at a deeper level than declarative reflection and is amalgamated with pupils' subject knowledge and qualitative understanding of their technical solution. This Group tension n = 116, 12m23s n = 11, 1m04s n = 105, 11m19s Competitive dynamic (negative) n = 108, 15m58s n = 69, 11m35s n = 39, 4m23s Competitive dynamic (positive) n = 37, 5m19s n = 28, 4m15s n = 9, 1m03s Study effects (recording) n = 67, 9m53s n = 23, 2m32s n = 44, 7m21s Study effects (researcher) n = 50, 5m52s n = 20, 1m37s n = 30, 4m14s allowed pupils to look back and to predict by explicitly making connections between technical aspects of their solution. The final area in which important differences were found was management of the problem-solving process which is accounted for by Halfin (1973) in his mental process set. When analysed, the more successful cohort exploited more positive managerial strategies, and fewer negative traits. They made more extensive and effective use of role and task allocation, spent more time planning ahead and longer in the earlier conceptual phase prior to construction. Other studies have also captured aspects of these for technology education. Hennessy and Murphy (1999) discuss peer interaction, planning, co-operation and conflict, and changing roles and responsibilities as features of collaboration with significant potential for problem solving in technology. Rowell (2002), in a study of a single pair of technology pupils, demonstrated the significance of roles and participative decisions as enablers and inhibiters of what pupils take away from learning situations. What was interesting about the groups involved in this study, was that the managerial approaches were collectively more proactive in nature for more successful groups. Less successful groups were generally more reactive to emergent successes or problems during solution development.
The problem-solving activity of pupils in this study was exceptionally complex and a fuller understanding of how these complexities interacted would have to be further explored. Yet, key differences in knowledge and process collectively suggest that effectively solving well-defined technological problems involves a combination of proactive rather than reactive process management, and an ability to make two different types of technology-specific connections: contextual connections and technical connections. Proactively managing is generic and involves planning, sequencing, and resourcing developments beyond those that are immediately in play to minimise avoidable errors with reference to problem parameters. It involves group members through agreed roles and task allocation that, where possible, capitalise on their strengths. Contextual connections involve effectively linking and applying technological knowledge, concepts, and principles to the material context that have been learnt form other contexts out with solution development. This is supported by skill and judgement in the material developments that embody this knowledge. Finally, technical connections appear to be important for better functioning solutions. These are links in understanding that pupils make between different parts of the developing solution that reveal and build knowledge of interrelationships, dependencies and how their solution works. In addition to helping pupils developing effective managerial approaches in group work, this suggests that pedagogical approaches should not assume pupils are simply able to make contextual and technical connections during technological problem solving. Rather, pedagogy should actively seek to help pupils make both forms of connection explicit in their thinking.
This study has determined that proactive management, contextual and technical connections are important characteristics of the modus operandi of pupils who successfully solve well-defined technological problems. This study does not make any claim about the learning that pupils might have taken from the problem-solving experience. It does, however, provide key findings that teachers can use to support questioning, formative assessment and pedagogies that help pupils in solving well-structured technological problems more effectively.