Introduction

We view the purpose of educational research as relating to two main goals: to produce new knowledge; and to improve educational practice. The history of educational research shows that these two goals have mostly been viewed as mutually exclusive. For decades if not centuries, the goal of knowledge production has been pursued largely through basic research (in nomological, and more recently, interpretivist traditions); whereas the improvement of practice has been pursued through applied work, which is only just beginning to overcome its second-class stature. This traditional orientation has done much to account for the long-lamented research–practice gap, as has the notion that knowledge flows uni-directionally from research to practice. In contrast, we started from the mindset that working on these two goals can, and in many cases should, be synergistic and simultaneous. This perspective was already put forward a century ago by Münsterberg and Dewey, successive presidents of the American Psychological Association who called upon the social science research community to embrace a ‘linking science’ connecting research and practice; by Glaser (1976) who proposed a science of design in education; and Stokes (1997) who pointed to Pasteur as an example for linking basic and applied sciences. In the last decade, researchers, practitioners and policymakers have argued, lobbied and legislated for new forms of research that bring research and practice closer together through mutually-beneficial interactions (Burkhard and Schoenfeld 2003).

Among the new forms of research emerging is design research, which addresses the need to speak directly to problems of practice, together with practitioners, that lead to the development of relevant and usable knowledge (Design-Based Research Collective 2003; Reeves 2006). Several motives for undertaking design research relate to bringing research and practice closer together. First, because design research takes place in authentic settings that are rich with complexity, the findings from these studies stand to have strong ecological validity (Brown 1992), rendering findings from such research potentially more usable (e.g. Lagemann 2002). Second, because practitioner voices—to varying degrees—shape design research projects, the findings from these studies stand to be more relevant and therefore more usable (cf. McKenney and Reeves 2012). Third, engaging practitioners in the co-creation of new knowledge, which is common in design research projects, is a powerful process for promoting the uptake and use of new insights among participants (Vanderlinde and van Braak 2010). Increasingly, design research is also being introduced to increase the robustness of professional educational designers’ practices (Van den Akker 1999) and contribute to much-needed theory building in the arena of educational design.

We observe that much of the design research literature to date has concerned its description and/or potential for closing the research–practice gap. Examples of design research are cropping up in literature; and the educational design research community is beginning to learn from each other’s work by way of example. While impact on practice is a common theme (e.g. Anderson and Shattuck 2012), we know of no effort to specifically examine the complex interaction between research and practice within specific empirical studies, or in discussions of design research examples (e.g. Orrill et al. 2003). The purpose of this paper is to begin to describe the nature of the research–practice interactions within selected design research endeavors. We do not attempt to provide a comprehensive and representative overview. Instead, we look for important features relating to one, highly prominent focus, present in many design studies and dear to this journal’s readership: the design of instructional solutions. But first, attention is given to the nature of design research and the framework used to study the research–practice interactions.

Theoretical underpinnings

Characterizing educational design research

Educational design research uses quantitative, qualitative and mixed methods to answer research questions, thereby contributing to theoretical understanding, through the process of developing and testing solutions to educational problems (Barab and Squire 2004; Van den Akker et al. 2006; Reeves et al. 2010). There is healthy variation in interpretations and uses of the term, but consensus appears to be present in characterizing design research (cf. Kelly 2003; Reinking and Bradley 2008; Van den Akker et al. 2006; Wang and Hannafin 2005) as: interventionist (undertaken to improve practice); iterative (consisting of multiple cycles of research, intervention development, testing and revision); and collaborative (involving researchers and practitioners, and sometimes other groups). In addition, educational design research uses existing knowledge to construct instructional solutions to complex educational problems, and contributes to new, theoretical understandings by studying what happens when those solutions come to life in real classrooms. Working systematically and simultaneously toward the dual goals of solution development and theoretical understanding may be considered the most defining feature of educational design research (McKenney and Reeves, in press).

Variety in design studies can be found with respect to the grain size of study (e.g. individual learning activity, full year of inquiry science activities); subject areas addressed (e.g. mathematics, science, art, language); kinds of research questions being asked (e.g. about characteristics of the intervention, insights engendered by use of the intervention, or both); scope of implementation (e.g. one classroom, three schools, 50 states), and the methodological traditions of research teams (influencing both researcher values and expertise available). Not only do studies vary, so do views on paradigmatic issues. Discussions relate for example to the relation between evidence and findings, the generalizability of findings, and the form the theoretical understandings take (e.g. Kelly 2004; Dede 2004).

Design research projects tend to be long term (Burkhardt 2006), evolving through multiple iterations of (re)design and field investigation, together with practitioners, to develop the dual outcomes of solutions and theoretical understanding. During the process, it is common not only for researchers and practitioners to collaborate, but to take on multiple roles. Design research participants often become researcher/designer/teacher/facilitators, thus finding themselves playing the conflicting roles of advocate and critic (Design-Based Research Collective 2003). As has been discussed in literature, the multiple roles present both challenges to rigorous inquiry that values objectivity; and opportunities to incorporate insider perspectives efficiently and effectively in interpreting findings and revising solutions (McKenney 2001).

In terms of outputs, the kinds of solutions developed through design research vary widely. What can be expected from the solutions created through design research is that they are derived from a systematic process, initially grounded in existing knowledge and evolving further through empirical testing (cf. Sandoval 2004). Similarly, the theoretical understanding produced by design research can take myriad forms. Such insights may be more closely tied to a particular type of solution. For example, Kim and Hannafin’s (2008) principles for grounding the design of web-enhanced case-based activity offer guidelines to facilitate the transition of novices towards experts’ reasoning and applications. Alternatively, theoretical understanding may contribute more toward specific theory building, like Thomas et al. (2009) theoretical notions regarding the implementation and acceptance of technology-rich innovations in education. Finally, the scientific understanding yielded by educational design research can describe educational realities as they are. This was done by, for example, Raval et al. (2012), who systematically analyzed Indian para-teacher learning needs. While this information also informed the design of a professional development program, it was especially valuable in its own right—demonstrating the focus and urgency of attention for capacity building of para educators, who make up a substantial proportion of the (developing) world’s educational workforce. While design research often also contributes to the knowledge production among those participating, a key feature of this approach is that it generates theoretical understandings that can be used by others as well.

A framework for understanding research–practice interactions within EDR

We view design research as extremely well-suited to the dual goals of enhancing theoretical understanding and improving educational practice. Central to mutually-beneficial research–practice interaction are two elements: the understanding generated by research; and the people who produce and use that understanding to improve educational practice. We also acknowledge that the context and rationale for interaction influences the research–practice interaction, since the rationale pertains to designing solutions to educational problems for use in particular contexts.

The ‘social design’ of educational research in general (cf. Wagner 1997) and design research in particular (cf. Barab et al. 2007; Reeves et al. 2005) plays an important, if not determining role in shaping the research activities and—though we might not like to admit it—sometimes even the findings. Researchers and practitioners take on multiple roles during design studies, which is one way to facilitate the flow and uptake of new knowledge. For example, by taking on the role of designer, researchers may become more sensitized as they gather parameters within which solutions will have to function. Similarly, by taking on the role of researcher, teachers may take a fresh look at phenomena in their classroom, enabled by a new lens or perspective. After pointing out that design research is a ‘highly-interactional’ mode of inquiry (as opposed, for example, to unidirectional), Bauer and Fischer (2007) suggest the need to study researcher-designer interaction using distinctions commonly seen in research on group learning, between cooperation (distributing tasks among partners to make use of complementary expertise) and collaboration (implying that team members equally share rights, duties and abilities).

Design research features the well-informed design of solutions to educational problems (cf. Oh and Reeves 2010). Here, we distinguish three sources of knowledge that we see are commonly used to inform the design of solutions, often in combination with one another: literature, project data, practical knowledge. Literature-based design builds upon theory and/or evidence found in literature. Data-driven design is steered by empirical findings from field investigation during project development; e.g., generated through a form of evaluation. Building on the notion of teacher practical knowledge (Verloop et al. 2001), practical knowledge-informed design incorporates the wide range of insights and knowledge about research, design, teaching, learning and schooling. This includes, but is not limited to tacit knowledge, experiential knowledge, professional knowledge and propositional knowledge. While the first two forms of knowledge are widely accepted as useful inputs for design, appreciation for practical knowledge, especially that of teachers, to inform the design of educational solutions has only recently begun to gain momentum and has been under-represented in design research literature to date (McKenney and Reeves, in press).

In addition to the solution designed, a major output of design research is theoretical understanding. As a scientific endeavor, new insights generated from design research is at least public (that is, accessible and usable by others) and also often local (for example, contributing to the practical knowledge of researchers and practitioners participating in a particular project). Building on previous work (Edelson 2002; Van den Akker 1999), McKenney and Reeves (in press) indicate that the nature of knowledge produced by design research may be one or more of the following: declarative (describing products, concepts or theories); procedural (informing how to take action); or observable (empirical findings or experiences). Each of these has the potential to contribute to theory building. In addition, knowledge produced by design research may be characterized by its focus, for example on teaching, learning and/or resources. This study aims to identify and describe immediate interactions between research and practice within a carefully-selected set of studies, and does not explicitly aim to portray broader dissemination and use of the research findings.

The aim of this study was to explore the research–practice interaction in design studies as reported in literature. We used the considerations above and the following research questions to focus our inquiry:

  • What characterizes the projects in general, the contexts in which they take place, and the designed instructional solutions in particular?

  • Which participants are involved in design research projects focusing on the design of instructional solutions (e.g., teachers, researchers, facilitators, teacher educators, etc.), what are their main roles, and what is said about the nature of their interaction?

  • What sources of knowledge are used to inform design research teams while engaging in the design of instructional solutions (literature, project data, practical knowledge)?

  • What do the reports say about production of public and/or local knowledge and what initiatives or perspectives are described to disseminate public knowledge? What do the reports demonstrate about the nature (declarative, procedural, observable); and focus (e.g. teaching, learning, resources) of the knowledge created, and what specific contributions to theory building are evident?

Methods

Project selection

The general goals and characteristics of design research projects (see above) were used to derive relevant search terms and inclusion criteria. Three groups of descriptors distilled from relevant literature were used to search for projects: design research and its synonyms (e.g. “design-based research”, “developmental research”, “design experiment”); descriptors of the knowledge claim (e.g. “design guideline”, “local theory”) and terms reflecting the approach (e.g. “evolutionary”, “iterative”) in three scientific databases: ERIC, Scopus and Web of Science. To allow in-depth analysis of recent design research, the search was limited to articles published in 2008 and 2009; this resulted in 375 articles. Next, abstracts were screened by two independent researchers in a number of test runs of 20 abstracts each and differences were discussed until agreement was reached. This was done until the inter-rater reliability using Cohen’s κ indicated an appropriate level of agreement (κ > .8). To be included in this set of recent, educational design research projects exemplifying research–practice interaction, articles had to meet the following criteria:

  • Educational orientation: The project described was developed either within a formal educational setting (i.e., primary, secondary or tertiary education) and/or as part of a teacher professional development program.

  • Researcher-practitioner participants: Besides the researchers, the project involved (student) teachers, and/or intermediaries (e.g., teacher educators, content experts, etc.).

  • Research contributes to a practical output: The article explicitly discusses the ways in which (design) research informed the design of instructional solutions (i.e., lesson plans, pedagogical strategies, etc.).

  • Empiricism: The article is based on the collection and analysis of empirical data.

The design-based research characteristics were not used to exclude any project if it met these criteria. After initial screening, 172 articles remained for full-text screening, many having been labeled “possibly relevant” due to limited descriptions in the abstracts. During full-text screening, the same criteria were used. Of the 154 articles excluded at this point, most lacked explicit discussion of how research contributed to the design of instructional solutions. An instrument was developed to analyze the 18 remaining projects, as described in the following section.

Project analysis

The analysis instrument took the form of a semi-structured template, used to capture project information extracted from each article. Open questions were used to identify specific project characteristics (e.g. location, subject area, duration) and specific instructional solutions being designed. In addition to open comments, participant involvement was analyzed through four items relating to: profession (researcher, practitioner, facilitator, designer and other), the number of each involved, the roles they had, and activities they conducted. Five items were related to the knowledge used to support design (e.g. nature of the design process, type of knowledge used). The theoretical understanding produced and disseminated through each study was coded according to type of knowledge (observable, declarative and procedural), and explicit contributions to theory-building were also studied. In addition, the common themes within the project set were also sought. Activities undertaken to disseminate research products and findings were noted, as were references to other publications concerning the same project. In addition, characteristics of the studies (e.g., research approach, methodology, key findings, etc.) were extracted to round out understanding of the 18 design research projects.

We adopted both a deductive and an inductive approach, starting from pre-determined categories that defined each major theme while also remaining open to the emergence of unique and particular instances across projects. Common patterns and themes were identified across studies and projects through constant comparisons (cf. Denzin and Lincoln 2000). For two articles, two researchers filled in the instrument independently and discussed differences as to ensure validity in using the instrument. After analyzing all 18 articles, findings were discussed first within the research team and then in a working conference involving scholars with expertise related to (narrowing) the research–practice gap.

Findings

Characterization of the projects

As illustrated in Table 1, the 18 projects reflect substantial variation across location, educational level, subject area, designed solution, duration and phase. Ten projects took place in the USA, two in China and six come from different countries. Seven projects aimed at primary education, three at secondary and six at tertiary level. Two focused on teacher professional development. The science (n = 7) and math (n = 3) domains account for the three secondary and for four primary education projects. Technology is featured in eight designs. 13 projects reported were in an initial phase of development, of which five have explicit links with earlier experiences or evolved from earlier research and four report changes to the design. Five projects were beyond the stage of the initial designs, of which one explicitly addressed the process of scaling up.

Table 1 Overview of the 18 projects studied (complete citations are noted in the reference list with*)
Full size table

Participants and their roles

The 18 projects descriptions were analyzed for the roles played by different participant groups. Participation was defined as any form of direct involvement with the design research process and/or products.

Typically, up to five teachers participated in a project, though this number is—naturally—much higher for when professional development programs were the solution type being developed (Gu et al. 2009; Swain and Swan 2009) or when projects go to scale (Tiberghien et al. 2009). As expected, teachers all contribute by using the newly developed products. In three primary school cases (and in one professional development program, see below), their role is limited to this (Hickey et al. 2009; Lamberg and Middleton 2009; Tatar et al. 2008): after instructions, these primary level school teachers are asked to implement what has been designed by the researchers/designers such as a game-based ecology curriculum or a series of math lessons. These projects share their focus on testing the ‘robustness’ and generalizability in producing an artifact (instructional solution) and/or of theoretical understandings, requiring teachers to implement a new design or teach a control group. However, in these projects other teachers had been involved in an earlier stage more substantially (e.g. Tatar et al. 2008; also Tiberghien et al. 2009).

Work reported by Gu et al. (2009) and Swain and Swan (2009) both concern teachers’ professional development programs: aimed at teachers in other educational levels, but the authors themselves working as researchers and/or teacher educators at university level. As such, they were involved in designing, researching and using the programs. In one of both professional development programs, the program consisted of online support for teachers involved in distance learning (Gu et al. 2009). The teachers’ role here was also limited to the use of the support program since the overall goal was to compare the effectiveness of the support with earlier face-to-face experiences. In the other project, teachers in adult education participated in a professional development program which required them to design and test new math activities in their own practice, based on a framework provided in the program (Swain and Swan 2009).

It is insightful to differentiate between higher education projects and the others, since the teachers in primary and secondary level education have different roles compared to participants from tertiary level. 1st and 2nd level teachers often became involved in design activities. Teachers had a say in the topic to be dealt with in education if it would not conflict with the research interests. For example, the teacher participating with Barton and Tan (2009) introduced food and nutrition as a proper theme for exploring the topic the researchers were interested in: the use of pupils’ knowledge in science lessons. Together with students from his class, this teacher and both researcher designed the activities to be conducted. Similarly, teachers in Singapore planned and designed a learning trail through Chinatown while the researchers’ interest was in mobile learning (So et al. 2009). Birchfield and Megowan-Romanowicz (2009) describe a teacher’s idea to use the researchers’ multimedia laboratory for teaching about geologic evolution. In contrast, Tiberghien et al. (2009) were specifically aiming for innovative mechanics education. Teachers involved in the learning community of Van Rens et al. (2009) collaborated to design activities for teaching inquiry in chemistry education. With either the subject set in advance or open for discussion, in all these cases teachers were involved as co-designers, fleshing out teaching activities and providing ideas for improvement.

Less often, teachers were involved in research activities. Four teachers observed each other when the lesson was taught which they had designed collaboratively (Cheung 2009), teachers within the university network tested mechanics teaching activities to see whether these were robust enough to be done independent from the design team (Tiberghien et al. 2009), and one teacher co-authored the article (Zhang et al. 2009).

Six projects were conducted at tertiary level. In four of these, teachers in tertiary education published about the design and/or improvements made to courses for computer science, science teaching, physics and service learning. They conducted design research in their own organizations, being responsible for all design, research and teaching activities (Barnes et al. 2008; Flannery and Pragman 2008; Hadjerrouit 2008; Schwarz 2009). In two other projects tasks were more spread. Casotti, Rieser-Danner, and Knabb describe university teaching staff who were involved in developing a new physiology curriculum which they implemented themselves, but they were also facilitators for other colleagues who taught the new curriculum as well, and they collected student data. Choi and Lee (2009) report about an online learning environment for student teachers, which was developed together with help of others (not further specified) and used in a course by one of the authors. This teacher/co-author also participated in designing the evaluation instruments. The other author was involved in the design process and in research activities, by reviewing the literature, using it to design the online environment, gathering and analyzing data.

Based on author information and on further project information in the project descriptions, typically up to three researchers participated in a project. However, numbers are not always clear, especially when the study reported was part of a larger project. Researchers were responsible for all research activities in projects in primary and secondary education and they were the principal or co-designer in the projects as well.

Apart from teachers and researchers, only a few other participants are mentioned. Choi and Lee (2009) discuss the help of doctoral students for evaluating data and of some individuals who assisted with the development of the learning environment; they also had a developer on their team. Also Hickey et al. (2009) involved a doctoral student as a content expert for making improvements to the feedback system of the game they were developing. Tatar et al. (2008) reported on the development of a math replacement unit, which was created by a designer with prior experience writing school mathematics curriculum. In two studies, students were involved in design activities. Barton and Tan (2009) describe how five students together with their teacher and both researchers thought of classroom activities around food and nutrition and reflected upon these; while Zhang et al. (2009) invited all students in class to determine which aspects of light were to be studied. So et al. (2009) had cultural experts involved in designing and implementing tours for pupils to discover the cultural heritage of the town.

To summarize: multiple roles are often taken on by design research project participants. Academic staff is sometimes involved in design research related to their own curriculum and courses, combining designers’, teachers’ and researchers’ roles. Teachers and researchers working together in primary and secondary education projects have distinct fields of expertise. Especially in the small-scale projects, the interaction between them is more collaborative than cooperative: when designing together, both contributing from their own expertise and sharing responsibility for the results (Barton and Tan 2009; Birchfield and Megowan-Romanowicz 2009; Cheung 2009; Van Rens et al. 2009). In contrast, the studies on larger projects (like Hickey et al. 2009; Tatar et al. 2008) suggest that interaction is more cooperative (researchers are accountable for research tasks, teachers are accountable for teaching tasks) and with less collaborative or shared responsibilities. It must be noted, however, that often very few details are provided about the interaction.

Knowledge sources informing design of instructional solutions

Three sources of knowledge commonly used to inform the design of solutions are literature, project data and practical knowledge; these are often used in combination with one another. Figure 1 shows a Venn diagram mapping all 18 projects according to the sources used to inform the development of interventions: literature (literature-based); data gathered during the project (data-driven); and/or the practical knowledge of participants (practical knowledge-informed). The sources in Fig. 1 do not represent all the data gathered in the studies but only those sources which were said to have informed the (re)designed product. For example, Birchfield and Megowan-Romanowicz (2009) collected data to study interaction patterns between teachers and students, in order to answer their research question about collaborative learning. Since their study does not show any modifications made to the design based on these data, this was not included in the diagram as project data.

Fig. 1
figure 1

Sources of knowledge used to design interventions according to project descriptions (referred to by the first author’s name, complete citations are noted in the reference list with*)

Full size image

Only two projects clearly indicated that they used all three sources of knowledge to inform their work. Flannery and Pragman (2008) aligned their design approach with the PDCA cycle for total quality management. Improvements to the design were based on systematic data gathering (student surveys and qualitative assessments) as well as the feedback and ideas from the participating teaching staff. For their online learning support system, Gu et al. (2009) adopted learning support elements from literature. Opinions (beforehand) and earlier experiences from the researchers informed the design, as well as systematically gathered survey data.

Seven studies report to have used evidence from literature and project data as input for their instructional design. Typical project data are users’ feedback (either students or teachers), collected via surveys and interviews. When discussing the literature that is used in a design, it is usually stated that modifications are made (e.g. “a modified version of …”, “draws on the work of…”, “was adapted from…”), but there is very little information on how. Some only refer to other work (Hadjerrouit (2008) based his work ‘on literature’, Hickey et al. (2009) make references to others’ work and earlier project work they draw upon). Some explain shortly what they used. When designing a new physiology course, Casotti et al. (2008) adopted a ‘modified version of an inquiry-based curriculum’. Swain and Swan (2009) incorporated research-based principles for teaching mathematical concepts and supportive recourses which were developed in earlier projects. Schwarz (2009) used a framework which was adapted from ‘BSCS five E’s inquiry model’. Only in two cases it was shown how literature informed the design. Choi and Lee (2009) explained the adaptations made to a model for ill-structured problem solving which they used to design an instructional framework. In their math module, Lamberg and Middleton (2009) included a series of problem types drawn from literature and explain how their teaching sequence aligns with suggestions from literature.

Three studies relied on practical knowledge in combination with either literature (Van Rens et al. 2009) or project data (Barton and Tan 2009; Tiberghien et al. 2009). Van Rens et al. state explicitly to involve “teachers in the design process and all benefitted from their teaching expertise. From working together we expected an educational design that would be feasible in practice and would lead to an increase in student conceptual knowledge” (p. 1437). Tiberghien et al. (2009) brought their design to scale after several rounds of pilot testing the teaching activities for mechanics designed by physics teachers.

Six projects describe only one source of knowledge to inform the design. Cheung (2009) mentions only the use of literature to inform the design. Birchfield and Megowan-Romanowicz (2009) seem to use only practical knowledge to develop new lessons about geological evolution using multimedia. These lessons were designed in a professional learning community set up especially to design learning scenarios using the multimedia laboratory. Data is gathered to research interaction patterns, but these are not related to improvements. For the changes made to their designs, Barnes et al. (2008), So et al. (2009), Tatar et al. (2008) and Zhang et al. (2009) refer to project data only but do not specify in much detail how. In one case (Tatar et al. 2008), the phase of the project is to establish the effectiveness of a math replacement unit implemented at quite a large scale and compared to a control group. Where no reference is made to literature to inform the design, this is most likely due to the phase of the project (beyond the initial design, as in Tatar et al. 2008) or to the originality of the topic (e.g. a learning trail, a students’ research community).

To summarize: most design research projects found in this review use literature and/or project data to inform the design of instructional solutions (although for all these projects many more data are presented related to the research questions). Less than half of the reports discuss practical knowledge informing the design of instructional solutions. However, we should consider a number of cases where (3rd-level) teacher-researchers are involved in design research about their own curricula, where we may presume their own practical knowledge is involved but which is different from a researcher drawing on collaborating teachers’ practitioner knowledge. Only very few make an effort to specify how the knowledge input is related to (changes in) the design.

Theoretical understanding produced and disseminated through design research

The new knowledge design research produces may be: observable (empirical findings or experiences); declarative (describing products, concepts or theories); and/or procedural (informing how to take action). Each of these has the potential to contribute to theory building. Table 2 offers an overview of the public knowledge produced by the projects studied.

Table 2 Research aims and new (public) knowledge produced in the projects studied(referred to by the first author’s name, complete citations are noted in the reference list with*)
Full size table

The empirical findings and experiences reported in the studies are broader than the project data used to inform the design, as we discussed earlier. The types of data gathered fall into three main categories. First, user experiences are collected to get an idea of motivation, experienced relevance, difficulties, etc. (Barnes et al. 2008; Casotti et al. 2008; Hadjerrouit 2008; Swain and Swan 2009; Schwarz 2009; Gu et al. 2009). Second, learning gains like test scores provide information about the effectiveness of the design for learning (Barnes et al. 2008; Casotti et al. 2008; Cheung 2009; Choi and Lee 2009; Hickey et al. 2009; Tatar et al. 2008; Zhang et al. 2009). Third, data are gathered to study the teaching/learning practices, e.g. to examine interaction patterns (Birchfield and Megowan-Romanowicz 2009; Zhang et al. 2009), the use of a pedagogical framework (Barton and Tan 2009; Cheung 2009; Schwarz 2009; Swain and Swan 2009) or the learning process itself (Lamberg and Middleton 2009; So et al. 2009; Tiberghien et al. 2009; Van Rens et al. 2009).

In half of all studies, procedural/declarative knowledge is presented to explain changes made to an initial design, although the information is not very detailed. Also half of all studies offer considerations for designs outside of the study context, partly as results of research, partly as reflections on the ‘lessons learned’. Such implications for theory building are most likely found in those projects which ran for a longer time (allowing for multiple rounds of refinements and evaluation) and/or evolved from earlier research projects.

While it is likely that in multiple projects participants have gained new insights, only one project description addressed it explicitly: Cheung (2009) indicated that the lesson study illuminated teachers’ awareness of teaching Chinese writing creatively. Teachers found that being involved in collaborative lesson planning meetings, peer lesson observations and post-lesson conferences helped them facilitate creativity in classroom. The report indicates that what they learnt from the learning study was transferred to their everyday teaching.

Dissemination of the public knowledge, as summarized in Table 2, obviously took place through the articles found in this search. Reports were studied for additional knowledge dissemination strategies for each project. 12 articles referred to additional scientific publications (articles, doctoral theses); three mentioned project websites, of which two were explicitly targeted at teachers; and three described meetings held with schools. Though not surprising given the medium studied, we still find it noteworthy that—even in these studies featuring interaction between research and practice—references to practitioner publications were not found. However, since reporting on a project dissemination strategies is rare in scientific journal articles, these findings are indicative at best, and cannot be considered complete.

Discussion and conclusion

The aim of this study was to explore the nature of research and practice connections across recent design research literature concerning projects focused on the design of instructional solutions. 18 projects were included, which altogether provide a rich variety within the domain of design research with respect to the contexts, the research questions, the nature of the designs, the stage of development and goals.

Practitioners have prominent roles in the projects, although the amount of practitioner control over what is being created differs. Especially the smaller projects describe considerable overlaps in roles of teachers, researchers and developers. The practitioner roles seem to be at least partly dependent on: the stage of project development (e.g. initial design vs. scaling up), and the research focus for the paper (determining how much is said about the participant roles). For example, is the project initially undertaken to produce theoretical understanding, with the design of solutions as a secondary concern; vice versa; or are they viewed as equally important? A project’s stance on this is known to be influenced by many factors, not the least of which is the source of funding behind the work. Although several studies acknowledge the importance of these multiple roles for the success of design research, little is said about the specific tasks and activities undertaken and how these influenced the work and outcomes of the design research project.

Most design research projects found in this review use literature and/or project data to inform the design of instructional solutions. Less than half of the reports discuss practical knowledge informing the design of instructional solutions. While this suggests that practical knowledge does not necessarily play an important role in these projects, it is also possible that discussion of practical knowledge was strategically omitted, perhaps in an attempt to render articles more attractive to publication in journals that do not value this orientation. Furthermore, within those projects where 3rd-level teacher-researchers conduct design research in their own teaching practice, it is not likely to expect a discussion of different knowledge sources that are present within an individual.

Naturally, observable knowledge was found in all 18 studies. Procedural and declarative knowledge contributions were identified that: informed the intervention at hand; could be used for theory building (e.g. on how to solve a particular class of problems); or both. With the exception of one study, the construction of new public knowledge is most prominently visible. In so doing, practitioners are collaborative partners in the design, but research, teaching and implementation is most divided among participants. Their involvement is generally not discussed with relation to the production of knowledge or theories (although we assume that university teacher-researchers certainly were involved in knowledge production). The knowledge disseminated by university researchers in the reports studied provides more understanding of the effectiveness of designed solutions than on how those solutions were designed or what, specifically, renders them effective. Beyond what is disseminated through the journal articles studied, limited discussion is given to dissemination of knowledge and/or solutions outside the local design research context, although we acknowledge that the norms of scientific publication likely account for under-representation of dissemination work in the reports studied. We consider that additional research is needed to explore in more detail not only the ideal but the actual nature of the knowledge generated through design research, as well as the role of practitioners in the generation of both local and public knowledge, and the strategies and conditions that could encourage broader dissemination and utilization of knowledge generated through design research.

Limitations of the study

In this study, we chose to focus on the interaction of research and practice in design projects as evidenced through study of: the instructional solutions created through researcher-practitioner collaboration; participants and their multiple roles, varied sources informing solution development; and multiple forms of knowledge produced. While we are confident that the 18 stringently-selected recent projects studied offer useful insights, here we point out several limitations resulting from our methodological choices. First, by valuing empirical scientific articles as data sources, we were unsurprisingly confronted with manuscripts that privileged description of the research over description of the design work; many articles had to be excluded from our review because they did not explicitly describe the activities undertaken by design research teams and/or how research informed the (re-)design of instructional solutions. Second, while the choice to restrict our findings to papers published in 2008 and in 2009 allowed for in-depth study of recent, projects that met our requirements relating to quality and focus, it limits the possibility of identifying trends that could be evident over a longer period of time. Finally, it is possible that relevant design research projects were omitted not on the basis of content, relevance or quality, but for the simple reason that they did not explicitly characterize their approach as design research or one of the alternate terms used to search the three databases.

Educational design research literature advocates sharing empirical warrants for both design decisions (Barab et al. 2009; Sandoval 2004) and theoretical contributions (Cobb et al. 2003; Shavelson et al. 2003). Though not extensively, we found examples of both design decisions and theoretical contributions in the descriptions. While it is possible that research did not inform solution design or contribute to theory-building, we suspect that the former is likely a function of publication bias, and the latter is caused by our focus on research–practice connections. Relatively little attention is devoted to elaborating how designs are informed by literature, project data or practitioner knowledge. We find this disappointing, and view such work as essential to advancing design (research) methodology, as well as allowing research consumers to assess the usefulness of others’ work for their own situations. We recognize that this may call for alternate publication formats and outlets, which can bring along additional challenges. We suspect that these same projects may tell very different elements of their same stories in other media. We express our concern at the lack of scientific publication outlets in the field of education that value explication of design processes, and applaud Educational Technology Research & Development and Educational Designer for providing rare forums to make this important work accessible to others. Scientific journal articles that carefully articulate how empirical findings contribute to theory-building may not focus on, or have the space available to report on the research–practice connection as well. We know of several projects, including our own, whose reporting is divided into several smaller chunks in order to share specific lessons learned or meet journal word count requirements. Additional research, driven by different methodological choices, is needed to explore the connections between empirical warrants and both design decisions and theory-building. In addition, if the potential of design research to inform a broader community of researchers and practitioners is to be realized, then dissemination of knowledge and/or solutions outside the local design project context and/or beyond researcher–researcher media (i.e. scientific journal articles) also warrants attention.

Despite the limitations, our findings do provide in-depth descriptions of 18 carefully selected recent examples of design research that explicitly address research and practice connections during design of instructional solutions. A reciprocal relationship between educational research and practice can be seen, since in most cases practitioners have a substantial role in the creation and implementation of products, though few are involved in the dissemination. An interesting orientation found within many design research projects is held by teacher-researchers who systematically design, research and publish about their courses.

Based on the project reports studied, we see the potential of knowledge production in design research being met: frequently in terms of ecologically valid findings; inconclusively when it comes to relevance and use; and sporadically when it comes to collaboration with practitioners. (Practitioner involvement was high in design and implementation, less so in knowledge production.) We join others in the call for sharing research that features practitioner co-creation of knowledge as a vehicle for use and uptake (cf. Vanderlinde and van Braak 2010); and advances (educational design) theory, e.g. by articulating and analyzing the reasoning and influences shaping intervention development (Edelson 2002; McKenney and Reeves 2012). This study may serve as a first attempt to provide researchers, practitioners and policy makers with portraits of design studies which—through their focus on the design of specific instructional solutions, forms of participant engagement, use of existing knowledge, and production of new theoretical understanding—speak directly to reducing the research–practice gap. Toward more fruitful design research in the future, new studies are needed, using a broader range of research instruments such as interviews and project documents analysis, to portray the interactions within design research that promote the actual development of theoretical understanding.