Introduction

There is an increasingly urgent need for research that can help address complex real-world problems such as the global burden of non-communicable diseases (NCDs), the climate and biodiversity crises and widening inequalities. The challenge, however, requires the rapid mobilisation of new research groups that: (a) are large in scale, (b) involve many diverse and disconnected disciplines, (c) are based on co-production with a wide range of external stakeholders, and (d) prioritise ‘real-world’ impact (Bammer 2013; British Academy 2016; Greenhalgh et al 2021; Hall et al 2012; Lang et al. 2012; Mazzucato 2018; Plummer et al. 2022; Roux et al. 2010; UKPRP 2017; Whitmee et al. 2015). Funders are calling increasingly for research like this, that is both interdisciplinary and transdisciplinary (ID/TD), and that can demonstrate not just academic, but ‘societal’ impact (The Foundation for Science and Technology 2022; UKRI 2021). It requires ‘new and bolder approaches’ to research (UKPRP 2017), which are “not constrained by an unduly limited set of perspectives and approaches”, and which include “methods and perspectives where experience is still quite limited” (Skivington et al. 2021, p. 8).

Explicit within these calls are requirements for approaches to be co-produced with people and organisations external to academia, and through partnerships of researchers, practitioners, and lay public (Moore et al 2019; UKPRP 2018). This recognises the critical need to bring together knowledge and perspectives from a wide range of stakeholders to develop new insights to understand complex and systemic problems, and the identification of solutions (Lang et al. 2012; van Breda and Swilling 2019). The formation of such new large groups may itself be considered a ‘wicked’ problem (simplistically, a problem that is resistant to resolution) (Norris et al 2016).

Even describing this type of research is cumbersome. Based on those four characteristics above (a–d), as a shorthand, we have started to use the terms ‘LMITs’ (pronounced ‘limits’) and ‘New LMITs’ to denote similarly characterised projects and teams that are: ‘Newly forming’, ‘Large-scale’, ‘Mission-orientated’, ‘Inter- and Trans-disciplinary’. The terms ‘newly forming’ and ‘large-scale’ are self-explanatory, indicating the creation of a large group of researchers from multiple academic institutions and departments, the majority of which have not worked together before. We use the term ‘mission-oriented’ as a generic descriptor to denote those research projects that are looking to address the root cause and systemic societal challenge areas.

There is a range of characterisations of ID and TD research (e.g. AMS 2016; OECD; 2020; Plummer et al. 2022; Pohl et al 2021), which creates challenges for consistency of definition. Whilst definitions overlap, TD is often suggested to extend the ID concept of integrating academic disciplines and methodological approaches, by also drawing from non-academic practitioners to develop innovative research approaches and solutions to complex real-world problems (Stokols et al. 2013). This is frequently conceptualised as ‘co-production’, but there are a range of understandings of what co-production does or should entail and the extent to which, and how, societal actors should be included (Allan et al 2019; Beresford 2002; Durose et al 2012; Hawkins et al. 2017). Methods of engagement also vary considerably, for example, via the proliferation of ‘Urban Living Labs’ (e.g. Marvin et al. 2018; Menny et al. 2018; Nesti 2018; Steen and van Bueren 2017). Pohl et al (2021, p. 1) use TD to “stress diverse disciplinary perspectives that have to be reorganised for societal problem solving because they are not always a prominent focus in the discourse on co-production”.

There is a long and fast-growing body of literature on ID/TD research, with plentiful guidance including principles informing research collaborations, project phases, tools and processes to use (Thompson-Klein 1990; Bammer 2013; British Academy 2016; Hall et al 2012; OECD 2020; Pohl and Wuelser 2019; Stokols et al 2005; Swiss Academy of Sciences 2019). A decade ago, Lang et al. (2012) stated that this literature was dispersed and difficult to use, and Bammer (2013, p. 4) expressed openly her frustration that the learnings within it “have lacked the necessary scale and traction”. While the demand for this knowledge has clearly increased recently, and the literature has continued to develop, knowledge and—more to the point—experience appears to remain in the margins. For example, Brown et al (2019, p. 1) compare experiences across six recent and large-scale ID research collaborations working to address global challenges and conclude that, despite the increasing evidence-base, “so far there is only limited understanding of the enabling conditions, challenges, lessons, and tools for inter-disciplinary sustainable development research”, and that “increasing our understanding of how to effectively design and deliver interdisciplinary research is crucial”. Freeth and Caniglia (2020) suggest that research teams are often under-prepared when undertaking ID/TD research, and there appears to be limited understanding of fundamental operational challenges (Bammer 2013; Brown et al. 2019; von Wehrden et al. 2019). Van Breda and Swilling (2019, p. 4) suggest that “the existing literature on TD research has not yet generated an adequate set of context-relevant guiding logics and principles”. Unsurprisingly perhaps, it does not yet appear to feature in mainstream research management literature, which focuses on the finer details of established research delivery mechanisms (Andersen et al 2017; Langley 2012).

A fundamental issue appears to be that undertaking this type of research requires ‘cumulative exposures’ (Hall et al. 2012); in other words, it is unlikely to be understood fully if only intellectually, or in the abstract. It is arguably through critically reflecting on the experiences of collaborating across disciplines that learning can be enhanced and better applied in future (von Wehrden et al. 2019). It is in the writing and reflecting that value and knowledge are created and shared. There are multiple recent examples within the sustainability literature of research groups grappling with this same challenge: offering reflections on the process of undertaking transdisciplinary research to support the ‘learning to collaborate while collaborating’ (Mann and Schafer 2018; van Breda and Swilling 2019; Freeth and Caniglia 2020).

This article aims to contribute to this body of reported experience and knowledge, drawing on our own experience in operationalising a newly forming LMIT. We extend this evidence-base by offering a framework and method for critical reflection. We seek to enable capacity in this area in three ways: firstly, by setting our own understanding of the salient literature; secondly, by offering a framework for critical reflection and an overview of how we developed it; and finally, by setting out an account of our experience and the learnings derived thereof. We, therefore, address the following research questions: (1) How might a team go about developing a bespoke framework for themselves? (2) What are the critical areas for reflection during programme operationalisation? (3) What are the key learnings from this process of critical reflection that can support future ID/TD research into complex problems?

Specifically, we reflect on the challenges we experienced and the processes we implemented to overcome these during the first phase of the programme. This covered a 3-year period during which our focus was on understanding the United Kingdom’s (UK) urban development system and defining problem areas to address during the project’s second phase. The mission of the programme is to reduce non-communicable diseases (NCDs) linked to poor quality urban environments, which have a significant and unequally distributed impact on population health, and associated links to the climate and biodiversity crises. Example NCDs include respiratory illness, obesity, mental ill-health, cancers and diabetes caused by, respectively, air pollution, unhealthy food and drinks, lack of access to nature, and car-dominated streets. The project was built on a smaller-scale pilot study (Black et al. 2018), however, most researchers in the new, greatly expanded team had not previously worked together. Establishing a new team was made more challenging as the start of the project coincided with the onset of the COVID-19 pandemic. Recruitment freezes were instigated leading to a staggered start over the first year, and the first phase was also characterised by remote working with limited opportunity for face-to-face contact.

The article is structured as follows. In “Methods”, we describe our methodological approach including three key sources that informed our thinking, the development of our framework for reflection, and the processes we followed to critically reflect on our experiences. In ‘Results’, we share these reflections within our framework, supported with illustrative examples from our programme and the TD/ID literature. In “Discussion” we present recommendations to support the operationalisation of future programmes and discuss the implications of the overarching issue we identified: the structural and institutional barriers within research to cross-disciplinary collaboration.

Methods

Approaches in this kind of experiential learning have been described variously as ‘reflections’, ‘meta-reflection’, ‘learning’, ‘epistemic living’, and ‘transformative knowledge co-production’ (Mann and Schafer 2018; van Breda and Swilling 2019; Freeth and Caniglia 2020). We have primarily been using the term ‘critical reflection’ for two main reasons: (i) it builds on the use of this terminology in a previous pilot study (Black et al. 2018), (ii) to distinguish between the reflexive activities that led to this article with the ‘research-on-research' work we had designed into the programme (Wellcome 2023). This process of critical reflection was independent from, but closely linked to, two other main areas of research activity: management discussions and the research-on-research.

Figure 1 illustrates the three sources that informed our learning, including our perceived strengths and limitations of this combined critical reflection approach. The research leads included the research directors, the programme director and manager, and the leads of each of the six work packages that the programme’s first phase was built around. This group was the principal decision-making body and held monthly meetings and ad-hoc away days, with considerable additional informal discussions taking place to grapple with the operationalisation challenges. Their primary role was to set the programme agenda, resolve any emerging issues, and maximise impact. The group included one of the lead authors and its observations and decisions were documented in extensive note-taking and internal reports.

Fig. 1
figure 1

The strengths and limitations of the sources that informed our critical reflections

Full size image

The research-on-research team was established formally at the start of the project following learnings from the pilot project. Premised on the understanding of the inevitable challenges of operationalising ID/TD projects, the aim was to study how the collaboration was working and how to improve the various processes that operationalising a project of this nature involved. This work ran in parallel to the main research operationalisation, with the ambition of the research-on-research evidence providing intermittent guidance on course-correction. Early engagement with the research-on-research team helped the group to make mental models explicit, boundary setting and actor mapping, mission-orientation and identifying interdependencies. The research-on-research team collected data with project members in several ways throughout operationalisation including qualitative interviewing, researcher observation and workshops. Findings and reflections were fed back to the management group informally in meetings and ad-hoc discussions, as well as in written reports, and were drawn upon throughout the process of critical reflection. The research-on-research lead co-authored this article and helped to identify synergies in the analysis between the three different sources that informed our critical reflections.

Thirdly, to deepen our understanding of the operationalisation challenges we encountered, the lead authors (the programme director and a senior research associate) engaged in a comprehensive investigation of salient literature with a focus on how research might address complex global challenges, including: societal impact-orientation, ID/TD collaboration, co-production, research management, and reflexivity. Initial critical reflections were informal as part of the lead authors’ work co-ordinating the development of a project-wide approach for intervention design. They focussed on the challenges of research operationalisation, and over time it became apparent that these reflexive discussions were an essential exercise in our learning journey. As the reflection (and the associated examination of the literature) evolved, we expanded deeply our understanding of the likely root causes of the challenge areas, and how to respond to these. This prompted the development of a framework to guide our reflection and this paper. We used Hall et al.’s (2012) ‘Four-Phase Model of Transdisciplinary Research’ to structure our thinking and reporting in the pilot study. For this expanded undertaking, we found considerable insight within Bammer’s (2013) ‘Disciplining Interdisciplinarity: Integration and Implementation Sciences for Researching Complex Real World Problems’ and a related online repository of resources supporting research into complex problems (Integration and Implementation Insights 2023). These covered additional concepts and challenge areas that were highly relevant to our experiences and common across the wider literature we drew on (e.g. Pohl et al. 2021; Stokols et al. 2009).

We used the concepts within these two main texts and integrated additional areas that emerged from observations by our research leads group and research-on-research team, to develop a framework of 10 areas for critical reflections on programme operationalisation that were most salient to us and the challenges we faced (Table 1). It became clear through this process that these areas grouped naturally into two overarching categories: (a) ‘foundational’ and (b) ‘operational’ understandings. We provide a more comprehensive table that demonstrates how we developed these areas in the online supplementary materials (Supplementary Material 1).

Table 1 Framework for reflecting on the establishment of our new LMIT
Full size table

The reflections presented here were developed by the lead authors in regular meetings (two hours per month over a year-long period). They drew on the research leads group documentation, the informal and written reports of the research-on-research team, and our exploration of the ID/TD literature. Our reflections were subsequently discussed with the larger authorship team and further developed. Co-authors also contributed to these reflections through their inputs to the research leads group, ad-hoc discussions with the lead authors throughout the first phase of the programme, and the co-writing of this paper. We integrate illustrative data that informed our thinking into these reflections, using quotations drawn from the research-on-research team reports based on a series of interviews and four workshops with members of the team, and extracts from documentation of a research leads group away day (provided in full in Supplementary Material 2).

Results: critical reflections

Foundational challenges

Systems, unknowns and imperfection

While Bammer (2013, p. 35) advocates taking a systems-based view in understanding problems, she also asserted that ‘it is impossible to focus on the whole problem at once’. As with the wider issue of ID/TD working, we would concur that the challenge is one of practical application in a new and inherently unique context and across a large group with a wide range of understandings (Bammer 2013; Mann and Schafer 2018; van Breda and Swilling 2019).

A central tenet of complexity and taking a systems view is the need to deal with uncertainty, imperfection, and unknowns, which are inevitable components in this kind of co-produced research (Bammer 2013; Lawrence and Gatzweiler 2017). This represents a new way of working for many, which is not always comfortable (Kappel 2019; Freeth and Caniglia 2020). This was a substantial issue for us with the team seeking certainty early on, which could only come over time. The research-on-research team highlighted how participants in one researcher workshop “felt there have been many discussions exploring the project mission that have felt inconclusive … This uncertainty is perceived as a conceptual barrier to progress: it is experienced as changing focus and prioritisation and lack of connectivity and clarity of tasks.” Notes from research leads show how that same uncertainty percolated up to management given inevitable hesitancy in decision-making (Table 2). In many instances, decisions required much longer consideration and discussion than would be expected in more traditional, siloed research approaches. The central issues we experienced were not one of complexity per se, as much as: (a) the lack of time and the resulting pressure to make decisions more quickly, and (b) knowing exactly when to stop discussions and compromise. In LMITs, the direction and methods of the project need to be co-produced and negotiated across the team and research leads may have less free reign to self-direct and follow personal or disciplinary preferences than in other projects. Taking a 'systems approach' is not simply about the breadth of scope, but also about directing an explicit focus on relationships between parts of the system or between viewpoints on the system (Meadows 2015), which might otherwise be viewed separately and often along traditional disciplinary lines. There was clear discomfort in various academic debates, especially due to perceptions of authority (or lack thereof) (Kappel 2019; Freeth and Caniglia 2020).

Table 2 Impact of uncertainty
Full size table

We managed to overcome these challenges sufficiently to make progress by simply putting in additional time and effort, which is fine in short doses, but ultimately unsustainable. In future projects, we would consider in recruitment the essential characteristics required for this kind of research activity. We would plan a dedicated training module for those new to this kind of research approach, establishing clearer mechanisms of co-working and decision-making, and spending more time initially to manage expectations and help the team to cope with working with greater uncertainty than they might be used to. We would seek to expand our in-house communications capacity to support this.

As researchers from different disciplines engaged on these issues, several common characteristics emerged, which we presented in a simplified dichotomy to help juxtapose some of the tensions. Figure 2 illustrates some of the relevant dimensions observed in the project and the differing positions of researchers. This is not to suggest that any one researcher is either one or other of the example researchers presented, but rather that there is a diverse and wide range of characterisations across various spectrums. It recognises that researchers may work within different social theory paradigms (Burrell and Morgan 1979; Ferris 2009) which, along with other mental models and worldviews, shape their approaches to research and TD working.

Fig. 2
figure 2

Illustrative, simplistic characterisation of different positions across the group, with some being more open to learning by doing and others more in favour of building on the evidence

Full size image

ID/TD understandings

On the surface, there appeared to be widespread and considerable willingness and enthusiasm up front across the team to embrace ID/TD working. This was supported by early interviews with project members:

one of the things that we are set up to do is to engage in interdisciplinary work, whether with colleagues at City 'A' [city name removed when anonymising data] or in the wider consortium, so this project appealed in that respect, that it was a genuinely interdisciplinary project with significant policy dimensions which is a good fit for us.

However, there were wide variations in terms of foundational understandings of collaborating in a more deeply integrated research venture. This included what ID/TD means in practice and an appreciation of its value, experience of working in that way and its implications for research operationalisation. The same round of research-on-research interviewing revealed unfamiliarity with what 'transdisciplinary' means in practice; its value, experience of working in that way and its implications for research operationalisation:

never worked as a transdisciplinary team, so I don’t even have a definition of it. I know what ‘multidisciplinary’ means; I know what ‘interdisciplinary’ means; I’m not sure I know what ‘transdisciplinary’ means.

Though this plurality of understandings can be positive in terms of encouraging diversity and innovation, it also presents challenges in terms of operationalisation. The extent of the challenge of collaboration is illustrated by mapping work by the research-on-research team as we sought to understand the variation of disciplinary perspectives across the project (Fig. 3).

Fig. 3
figure 3

Mapping of disciplinary expertise across the project by the research-on-research team

Full size image

We undertook a wide range of activities to develop ID/TD understandings, for example: developing discussion papers seeking to make clear our underlying theoretical foundations and the operational implications of that underlying foundation, development of multiple language glossaries, early framing and re-framing of programme theory through multiple research team webinars and presentations. In hindsight a greater focus on reassuring those new to this kind of work, two-way relationship and trust building, understanding researcher personalities (Bammer 2022) and increasing in-house communications capacity would have been beneficial.

Context and stakeholder knowledge

Stakeholder engagement is critical to gain a more rounded understanding of context. While the meaning of what constitutes meaningful engagement and ‘good’ co-production is open to interpretation (Vanleene et al. 2015; Durose et al. 2022), it is commonly suggested that interventions introduced in complex environments need to be designed with, but not necessarily within, academia and in full partnership with decision-makers as well as those affected by those decisions (Moore et al. 2019; Skivington et al. 2021). To do so requires a broad and deep understanding of multiple ‘systems of systems’ (Gardner 2016) as well as comprehensive engagement with those affecting and those affected by the system. The challenge of bringing together all these voices is significant when considering the scale of challenges such as global sustainability, and impossible to do perfectly (Kislov et al. 2017; Oliver et al. 2019).

We experienced strong differences of opinion in the team on (a) who engagement must include, with different perspectives on the role of stakeholder groups including lay public, those external to academia, academics outside our consortium; (b) whether data gathering exercises such as interviews and workshops can be viewed as part of co-production; and (c) what stage co-production could and should be initiated. Frustration at not being able to engage in co-production practices that individuals felt were optimal for the project emerged. For example, the research-on-research team described how some of the team “felt that researchers should have engaged policy partners earlier on … The root cause to this issue was largely around (researchers) not being given agency to engage with policy partners”.

Yet, given the differing conceptualisations of the problem space(s), identification of these stakeholders (and meaningful representation) is a significant issue (Mann and Schafer 2018). This presented strategic implications for the research development, including a possible lack of intellectual incoherence if not carefully managed. This need for coherence and management presents challenges for researcher autonomy, especially when compounded by a lack of time (Bammer 2013).

Key processes underlined by both Hall et al. (2012) and Bammer (2013) of scoping and boundary setting were critical to developing a coherent engagement strategy. We undertook a range of team boundary mapping and stakeholder analysis exercises to help us develop our shared understanding. As an example, through early discussions and researcher workshops using online canvases, we developed a three-dimensional conceptualisation (Fig. 4) that helped us pull together a wide range of different variables into a single conceptual understanding. Specifically, it enabled us to agree which stakeholders were most relevant to which context (e.g. transport, property, land) and themes (e.g. power, incentives, valuation). It also helped us to link researchers to research participants and co-production partners and to identify any missing areas of representation. Though only partially developed it helped cohere the different perspectives and advance the stakeholder analysis. One outcome, for example, was a greater focus on the disaggregation of private sector groups.

Fig. 4
figure 4

3-dimensional concept seeking to enable better understanding of stakeholder coverage based on context, stakeholder and thematic area

Full size image

Identifying and responding to values

Bammer (2013) suggests various approaches to dealing with values and recommends working with applied philosophers, who may be able to alert the team to different ethical issues. Identifying and making values explicit is not straightforward in our experience and there can be uncertainty on which values to consider, how to express them, and potential sensitivities. In developing early discussion papers, the applied philosopher in our group—a specialist in public health law/ethics and the philosophy of public health—was able to query the inevitable place, and also influence, of (potentially hidden) values (Coggon 2012). For example, how might deeply held, but differing views on state intervention in the health of the public help inform our collective positioning on urban planning? With a project focussed on addressing health inequalities, how do we agree on which inequalities are unfair and require to be addressed, and why? What role should evidence play in helping us to resolve these questions? Engaging in such questions takes time and is often not seen as core to delivery, so requires championing from group leaders.

Nevertheless, the salience of such questions is notable because they are directed towards the realisation of particular (and irreducible) socio-political goals made explicit by the primary research aims: in the case of our funders (UKPRP 2017), the generation of fairer, healthier environments, including the prioritisation of planetary health. Through ID/TD research, we also see more acutely how intradisciplinary value commitments—for example, as understood by reference to singular epistemologies—may become more evidently in tension with understandings from other disciplines. Equally, we observe an arguably more complex need for reflexivity on the part of scholars, for example through heightened recognition of the political economy (and wider, structural drivers) in which their research practices are situated. This requires the identification of substantive values and the development of measures that bring in procedural values too (Coggon and Gostin 2020). As a challenge within a research agenda, this may require compromise as well as consensus (Verweij and Thompson 2006) and ‘creative conflict’, derived through a clear sense of the project mission and the mechanisms for achieving it (Mazzucato 2018).

Societal impact

Central to achieving societal impact is the identification of the problem to be addressed. However, in projects that seek to address complex real-world problems through co-production, the problem itself can only be clearly identified following an exhaustive co-production process (Bammer 2013). Additionally, how intervention in such projects may lead to societal change is inherently unclear during their development due to the unknowns and non-linearity inherent in the complex systems in which they are working (van Breda and Swilling 2019). This means there are multiple moving targets that shift and change as the data emerges.

Pinning down a clear, shared understanding of the problem space and target impacts in our case has not been straight-forward despite being aware of the need up front, multiple attempts, and the development of numerous ‘cognitive artefacts’ (information displays that inform thought processes and increase knowledge) (Hall et al 2012; McLane et al. 2010). This challenge was compounded by the wide range of highly specialist knowledge domains and world views in the group. In our experience overcoming this can only come with time and ‘cumulative exposures’ (Hall et al 2012), though the process could be made much easier with better understanding of this way of working and linked expectations, both at senior academic and funder level given the respective need for resourcing (Bammer 2013).

Alongside the identification of the problem, evaluating impact and demonstrating causality are further core challenges for LMITs. For example, our programme is seeking to intervene high ‘upstream’ in decisions that are far removed from the intended societal impacts of health outcomes ‘downstream’ (e.g. changes in local government procurement policy today impacting on the health of citizens 30 years hence). Different perspectives and expectations within the team, partners, and funders on what impacts are desirable and what can realistically be demonstrated require discussion and reflection. We identified a sense within the team of different expectations of what impact was expected and valued. For example, there was a feeling amongst participants in one workshop that some disciplines “are more theoretically driven and others more practically driven, which impacts the timeframes and direction of the project. This results in a differing sense of purpose”. In our experience, dedicating time to discuss interpretations and expectations of impact is critical to gaining a shared sense of direction.

Operational challenges

Project understanding and direction

The research group expanded substantially after funding was secured, by which time the mission and goals were set. The challenge for us, therefore, was not to co-create new goals per se, but to clearly communicate both the underlying programme theory and mission over a staggered recruitment process, as well as the evolving nature problem identification inherent in co-production. Despite multiple attempts, understanding remained variable across the team and for some time due to the range of world views, experiences and expectations. We anticipated that language would be a challenge, but were surprised nonetheless at just how problematic, even for seemingly well-known terms such as: ‘upstream’, ‘health’ and ‘transdisciplinary’. For example, with regards ‘upstream’, some interpreted it to mean that which is immediately upstream from the health outcome (e.g. the quality of a building or street), while for others it was understood as key points of power and decision-making (e.g. the investment and political decisions that influence the quality of buildings and streets). These debates continued throughout the first phase. Feedback from one researcher workshop indicated a perception amongst the team that:

there is a lot of pressure in the project to focus upstream, yet (researchers) felt the project needed to understand downstream in order to address upstream. Participants also felt that health was not present enough in this new intervention phase.” In addition, there was a need to “make the mission clearer and sharpen the focus from the high-level statement of addressing which NCDs and intervention areas are the focus.

We used a variety of methods to address shared project understandings with varying success, including: (i) generation of a wide range of graphic illustrations of the problem space; (ii) presentations of pilot study findings; (iii) co-development of the study protocol; (iv) research team webinars, full consortium meetings, and numerous work package meetings. A consistent challenge was to provide information in ways that would reach and engage the whole team given highly varying researcher preferences, time commitments and foundational understandings of concepts.

We also drew on Mazzucato’s work (2018) ‘mission-orientation’, disaggregating statements and ‘grand missions’ into ‘sub-missions’. This enabled some additional clarity, though the majority felt unable to engage fully in this exercise due to time pressures, and it remained unclear how the missions’ approach was significantly different from the existing aims and objectives. We eventually overcame this challenge through the development and implementation of a process for identifying interventions, which communicated and reinforced clear criteria to guide the identification and design of interventions based on our foundational principles, resources, and goals.

Team cohesion

Establishing team roles and responsibilities may seem relatively straight-forward in a traditional or well-established research project, but in our case, a more explicit exercise in this space may have been useful. For example: (i) the range of newly interacting disciplines was sizeable, so being more explicit about roles could have helped develop shared understandings; (ii) there was also a lack of clarity as to what value practitioner/non-academic experience presented; (iii) a better understanding of researcher knowledge domains, values and biases may have helped us to reflect more deeply on the nature and quality of research findings. A key issue that emerged from one research leads away day was the need for greater inclusion of a variety of team members in key discussions, and the challenges of this in a large team. For example, the research-on-research team identified a common perception that “Established norms in academia and hierarchy do not always support interdisciplinary working”. As illustrated in Table 3, difficulties included understanding which individuals to involve in different processes.

Table 3 Team cohesion
Full size table

The need for ‘psychological safety’ (Hall et al. 2012) was raised as a potential issue at the start of the programme, but we did not discuss this core principle in any depth, nor did we put in place any formal mechanisms for assessing this. In hindsight, this was because spirits were high at the project's inception and it therefore felt heavy-handed and unnecessary. As Hall et al. (2012) underlines, some level of tension is essential when developing new approaches and innovating in research, and sometimes this tension can feel uncomfortable. The implication is that healthy tension is where change becomes desirable and progress is possible; unhealthy tension is where change is needed, but issues get stuck and can fester. Consensus is not always possible, so the challenge is to ensure, if not resolution, at least some form of ‘clumsy solution’ (Verweij and Thompson 2006) alongside efficient decision-making.

We found patience and flexibility in communication styles were essential personal characteristics in enabling the holding and resolving of tension. For example, participants in one research-on-research workshop “felt personal characteristics such as being flexible and open to new approaches, and listening, were crucial for interdisciplinary working. Some root causes around interpersonal issues were around lack of awareness of interdisciplinary working, difficult/differing personalities … and different institutional and disciplinary cultures”. Bammer (2013) and Soskice (2022) highlight personality, character, and social skills as key area of focus for this kind of work. They provide methods for managing differences including personality assessments, principled negotiation and exercises to tease out unfamiliar personal and cultural norms. We reflect that engaging more with such processes might have helped develop the team during its formation. However, in larger teams working across disciplines, significant variations in individual characteristics are inevitable. It is essential for team leaders and funders to be aware of the need to allocate time and resources to building team cohesion during programme operationalisation given they have the power to create space for this essential work (or prevent that from happening).

Communication

Effective communication is essential for successful ID/TD collaboration (Kushnir 2021), but inherently challenging due to the complexity of communicating across large, multi-organisational teams. In a large team there are inevitably a very large number of meetings and discussions where decisions are made, thinking progresses, and updates are provided. Therefore, communication is inevitably imperfect. We never expected everyone to be involved in every discussion or engaged at all points and knew that information would be disseminated along various communication pathways at different speeds. However, for some in the project, this was seen as problematic and a barrier to effective working, with concern expressed in a researcher workshop about “parallel conversations where decisions or information seldomly reaches everyone”. While communication difficulties may be inevitable in LMITs, researchers may be nonetheless frustrated with how this differs from the effective communication they are used to.

There are also problems of being divided by language, which we have found both with how technical meanings of some terms differ across different disciplines, and with foundational concepts that can be hard to pin down because they seem so familiar. In some cases, such language challenges may be anticipated, but in others may be surprising. In our case, it was striking that even such familiar concepts of ‘health’ and ‘intervention’ were interpreted very differently by the team. For example, as had been the case in the pilot, health was perceived by some to be implicit in all discussions but others in the team expressed in a workshop that they “felt that health was not present enough in this new intervention phase”.

Ineffective communications highlight issues of in-house resource and capacity including time, expertise and experience, all of which are reliant on funding. To enable us to address this area we re-recruited a Programme Manager at full-time, up from part-time, invested in an external review of our communications capacity, and subsequently recruited a dedicated Senior Communications Officer as an advisor (1 day per week) and a part-time Communications Officer. This was not straight-forward as there was a substantial difference of opinion as to how capable academic institutions’ existing core services are at providing effective communication to a programme such as ours.

Decision-making

These challenges inevitably led to discussions around the management rationale and decision structure. At the start of the programme the leadership structure was based around two central groups: one including senior management and a separate group consisting of team members who were leading the programme’s six work packages. The rationale for this was based primarily on the experience in the pilot, where the team had experimented with a dispersed leadership structure with multiple project ‘heads’ across faculties, as well as those internal and external to the lead university. This worked well in many ways as it appeared to promote high levels of ID working and external stakeholder emphasis. However, it also became problematic at key points of decision-making with confusion as to where responsibility and the final decision on issues sat. For example, two researcher workshops identified common perceptions that “the loosely formed leadership structure made decision-making unclear” due to a “lack of clarity of who has ownership of decisions”. As such, it was felt that having a clear single point of direction would avoid confusion at key points of decision-making and help to provide certainty for the team.

While therefore a single point of direction, in theory, resided with senior management, decisions were in practice devolved across the research leads, which led to a lack of clarity during programme establishment. The specific tension was in determining the appropriate balance between research lead autonomy and overall project coherence and alignment to the foundational concepts. However, this initial structure did appear to serve a crucial purpose in the first phase of the programme as the group was establishing: it may have prevented a reversion to type, with the resulting uncertainty enabling the group to form organically over time around the original vision and concept. Furthermore, the management resolution was relatively straight-forward once the project became established. At the transition to the delivery stage of the programme, the two leadership groups merged and were expanded to include other senior researchers in the programme. This change to the decision-making model was intended to provide greater clarity to the team and is illustrated in Table 4.

Table 4 Changing governance arrangements at different stages of a research programmes life cycle, with example strengths and weaknesses
Full size table

Methods development

A range of core research and supporting activities were identified for the programme via the bid design and study protocol (Black et al. 2022). Most are largely well-established, but we were aware that researchers from different disciplines may have different epistemological perspectives and methodological preferences (O’Rourke 2019) and will have varied experience in the research methods training they have received (Tobi and Kampen 2018). Striking the right balance between enabling individual approaches, supporting intellectual freedom, whilst also ensuring intellectual coherence and vision alignment within the time and resource constraints of the programme was a substantial challenge.

Early work by the research-on-research team included identifying the different methods that the team identified with, and how these might be integrated. This activity identified 34 methods that team members brought to the project, which the research-on-research grouped into five main clusters: theoretical, qualitative, engineering/systems, quantitative, and reviews. Given this diversity and the different disciplinary backgrounds of the team, we sought, therefore, to prioritise identifying and supporting shared understanding about methodological expectations and preferences. We introduced a webinar series for members of the team to discuss research and methods they had expertise in. We prioritised time for discussion of proposed methods in meetings within work packages and programme-wide to support the whole team to be included and represented in decision-making. However, despite this issue being recognised from the start, towards the end of the first phase of the project the research-on-research identified a need for action “Embracing positive plurality in the project whereby there is acceptance of a level of epistemological difference across research methods, practices, motivations and language, also extended to practitioners”. In hindsight, additional time was needed for the whole team to understand expectations and differences in epistemological and methodological preferences and norms.

Discussion

Applying our framework for critical reflection to the first phase of our collaborative research programme was challenging, but ultimately very rewarding and highly recommended as an exercise for other similar. In developing and reflecting on these ten challenge areas, we have substantially developed our understandings of the challenges and solutions, both realised and potential, from engaging in this area of work. From this, it is possible to draw recommendations that we believe will support other newly forming LMITs to overcome some of the challenges they are likely to experience (Table 5).

Table 5 Six recommendations for new large-scale research collaborations
Full size table

While these recommendations are intended to help address the common challenges facing those embarking on ID/TD programmes, they point to an overarching issue: specifically, that cross-disciplinary collaborations tend not to be easily operationalised within traditional academic ways of working. We recognise, therefore, that the challenges we highlight may stem from significant overarching structural issues that are likely to be beyond any one research programme to address. In particular, our recommendations call for additional time and resources to be allocated to many aspects of LMITs compared to traditional research projects, including project coordination, communication, recruitment, team-building, and developing shared understandings. To assist with this, we recommend working with funders to enable ways of demonstrating alternative progress and a wide range of outputs given much of this work does not result in traditional metrics. Supporting effective working across disciplines and motivating researchers to collaborate in this way requires new ways of thinking for team leaders, academic institutions, and funders (Bammer 2013; The Foundation for Science and Technology 2022).

Commentators have acknowledged institutional barriers to working across disciplines and how well-established academic structures and norms incentivise individualism over ID/TD collaboration (Bammer 2013; Fritz et al. 2022). For example, publication and funder requirements may not necessarily motivate researchers to work outside their own discipline. This kind of research requires far more time and resources for team-building and management, however, institutionalised expectations and metrics of productivity and value may not prioritise these important activities. To compound the issue, a Wellcome (2020) report highlighted how unhealthy academic work environments can be, which may limit the capacity of researchers to push boundaries and create conditions conducive to working in large teams across disciplines and organisations.

A key challenge in this context is the development of the next generation of ID/TD researchers who specialise in working across disciplines and leading research collaborations. As Casadevall and Feng (2014, p. 1) state: “The advantages of specialisation of science include efficiency, the establishment of normative standards, and the potential for greater rigour in experimental research, (but) specialisation also carries risks of monopoly, monotony, and isolation”. Bammer (2013) sets out the case for her new discipline Integration and Implementation Science (I2S), alongside the need for leaders and specialists within this discipline. This builds on the work of pioneers, for example, Klein’s (1990) work on interdisciplinarity, the ‘post-normal’ science of Funtowicz and Ravetz (1993), and Stokols et al.’s (2005) work on the ‘science of team science’. Similarly, Brown et al (2019) sets out as a core principle the need to develop ‘T-shaped researchers’ to suggest the breadth of knowledge rather than solely deep specialisation. There is considerable parallel literature on the need for roles to bridge between research and academic administration, and academia and real-world environments (e.g. Packman et al. 2017; Robson et al. 2002; Rycroft-Smith 2022). However, whilst such institutional barriers to working across disciplines remain, it seems reasonable to assume that researchers are more likely to be de-motivated from pursuing ID/TD roles.

Although these challenges are clearly stated within the ID/TD literature, one area that appears notably absent is that of research governance and management, which is surprising given that it can clearly have profound implications for research design (Andersen et al 2017; Green and Langley 2009; Hollaender et al. 2009; Langley 2012). New approaches to addressing complex real-world problems require development either wholly or substantially in collaboration with stakeholders through co-production (Moore et al. 2019; Skivington 2021; UKPRP 2017). There are not only important associated issues of capacity but also fundamental issues regarding direction and decision-making, in addition to foundational shared understandings of institutional challenges, as set out above. The ID/TD literature along with the now well-established Science of Team Science has been attempting to address these areas for a long time (International Network for the Science of Team Science 2022; Stokols et al. 2009) yet in our experience and from our reading of the literature there appears a general lack of awareness of these initiatives especially at key resource decision-points within research ecosystems (Bammer 2013).

Conclusions

Our framework for reflecting on the establishment of our new LMIT has helped us to gain significant new understanding of what has worked and what else is needed. It has also helped us to gain a much better understanding of the existing expertise across numerous literatures. The reflections and recommendations provided here we hope will support other teams to collaborate more successfully and to respond more effectively to complex real-world research challenges, albeit within the restraints of a research environment that appears resistant to cross-disciplinarity. Critically, teams need to adopt strategies to support researchers with different personalities and preferences to better work with and embrace uncertainties, unknowns and tensions as they develop these shared understandings.

The challenge of operationalising ID/TD required rapidly synthesising learning from research findings across a broad range of reported literature, as well as building on a broad range of past knowledge gained from previous projects. The learning required filtering knowledge reported in the literature, then interpreting and applying this to the specific challenges. Researchers invest considerable time creating transferable and generalisable knowledge, but relatively less energy sharing research knowledge peer to peer. ID/TD co-production is inherently complex and time-consuming (Simon 2018) and researchers, programme leaders, participating organisations and funders need to be open to adapting traditional ways of working to facilitate effective collaboration across disciplines and co-production (UKPRP 2018; Skivington et al. 2021).

We accept that deeply embedded structural barriers will take time to overcome and that these are so deeply engrained institutionally as to present ongoing and considerable barriers to progress. There is an increasing drive in the UK and elsewhere from funders and others for this kind of research to address complex global problems, with considerable changes already in evidence. However, we agree with Skivington et al. (2021) that these research funding initiatives should not be “constrained by an unduly limited set of perspectives and approaches” if they are to effectively create the conditions in which cross-disciplinary research can flourish. Teams who support and commit to rigorous critical reflection while developing these new approaches should benefit substantially, and it should be of benefit too to the wider research community and, ultimately, society.