Techniques for overcoming difficult interdisciplinary dialogue in expert panels: lessons for interactional expertise

Abstract

Which techniques and skills can be used to overcome the obstacle of dialogue between scientists in different disciplines? Drawing on Gorman’s book on trading zones and Collins and Evans’ thinking on interactional expertise, this article analyses the work by individuals to manage five interdisciplinary panels commissioned by French ministries. It observes that these panel managers have different techniques to open, construct and close the debate. These techniques, which condition the submission of the final report, call for skills that managers acquire over the course of their experiences in trading zones. Implications of findings for the formation and management of interdisciplinary expert groups and for the concept of interactional expertise are discussed.

Introduction

Interdisciplinary expert panels, like other interdisciplinary scientific expertise production processes, are exposed to the Kuhnian problem of incommensurability between paradigms (Kuhn, 1962). Given that scientific disciplines each have their own practices, language and representations, bringing them together is not self-evident (Galison, 1997).

To solve this problem, and explain how and when interdisciplinary dialogue becomes possible, Gorman (2002) borrowed the concept of the trading zone (TZ) from Galison and then refined it with the help of Collins and Evans (Collins et al., 2007; Gorman, 2010). A trading zone consists of an interaction between different scientific communities each with their own language. The concept is sub-divided into different ideal-types depending on whether dialogue is collaborative or coerced and whether the outcome is a heterogeneous or homogeneous culture (a new language shared by both communities). For example, a fractionated trading zone corresponds to the combination of voluntary dialogue and a heterogeneous outcome. In this case, dialogue between communities can be mediated by a rudimentary common language: interactional expertise (Collins et al., 2007). The interactional expert is the individual with the necessary expertise to interact with the contributory expert—the individual who contributes to advancing scientific knowledge in their community (Collins & Evans, 2002).

This work on conceptualising trading zones by Gorman, Collins and Evans (Gorman, 2002; Collins et al., 2007; Gorman, 2010; Collins et al., 2019) was combined with the development of a threefold research agenda. In the conclusion to his 2010 book, Gorman advocated: (1) refining the taxonomy of trading zones and their trajectories and limits (293–294); (2) detailing the process of acquiring interactional expertise (294–295); and (3) identifying best collaborative practices in a trading zone¹ (295).

Many researchers have focused on the first and second tracks of Gorman’s agenda (for the first, see, for example: Jenkins, 2010; for the second, see, for example: McFadden et al., 2011; Seager et al., 2012; Brown et al., 2015; Conley et al., 2017; Henderson et al., 2016). The literature on the third track is thinner and more fragmented. There remain a number of untested hypotheses and first observations that need to be further explored and integrated through empirical investigations. Thus, the purpose of this article is to identify which techniques interactional experts can use to overcome the obstacle of dialogue between scientists in different disciplines—in the context of interdisciplinary expert panels.

To this end, the focus here is on the fractionated trading zone which, as mentioned, combines voluntary dialogue with a heterogeneous outcome. This ideal-type corresponds to the situation frequently found whereby scientists from different disciplines come together to submit a bid for a research grant or in a joint response to specifications for a systematic expert review commissioned by a public authority (which falls into the category of policy advice, as defined by Craft and Halligan: “a broader suite of techniques and activities, at various points in the policy process, including the provision of recommendations, guidance, and the articulation of preferences in support of policy work” (2017: p. 49)). This article explores the second case in point. Policy advice to address a wicked—ergo interdisciplinary—problem² is a relevant subject for the study of techniques that interactional experts can use in a trading zone. In this situation, difficult interdisciplinary dialogue combines with some scientists’ reluctance to take a public position on a public problem (Bijker et al., 2009: pp. 94–95).

The subject of study here is the expertise collective³: a systematic expert review method developed in France by the French National Institute of Health and Medical Research (INSERM) and then taken up and adapted by the French National Institute for Agricultural Research (INRA). This policy advice method consists of a systematic review of the scientific literature on a public issue in keeping with a particular procedure. The review is carried out at the request of a public authority by an interdisciplinary ad hoc expert panel under the supervision of INSERM or INRA expert panel managers. This article takes for its case studies the management of five interdisciplinary panels for five INSERM and INRA systematic expert reviews. The choice of cases was based on comparability, diversity and accessibility criteria. The comparison uses a range of sources and methods of collection and analysis: semi-structured face-to-face interviews with expert panel managers; an online questionnaire to panel members; and an analysis of the records on the five systematic expert reviews.

The article first examines the dedicated literature (section “Review of the literature on interactional expertise in trading zones”) and discusses sources and methods (“Sources and methods”). It then observes that the chosen case studies of fractionated trading zones formed to produce policy advice encounter the two obstacles identified in the literature: difficult interdisciplinary dialogue and some scientists’ reluctance to take a public position on a public problem (“Two obstacles to fractionated trading zones: difficult interdisciplinary dialogue and reluctance to take a public position on a policy problem”). It shows that the panel managers have a range of techniques to overcome these obstacles: expert selection criteria (“Individual and group selection criteria”), techniques to open and construct the debate (“Techniques to open the debate”), and methods to close the debate and secure the panel’s approval for the report (“Techniques to close the debate”). In addition, the article points up that these techniques call for specific knowledge and skills acquired by the managers over the course of their experiences in trading zones (“Use of techniques conditional on knowledge and know-how”). Implications of findings for the formation and management of interdisciplinary expert groups and for the concept of interactional expertise are discussed (“Conclusion and discussion”).

Review of the literature on interactional expertise in trading zones

To perform a review of peer-reviewed literature on interactional expertise in trading zones, I preferred using the Scopus bibliographic database over the Web of Science (WOS) and the Google Scholar search engine. First, I dismissed Google Scholar for a number of reasons. To be sure, it has more comprehensive coverage in social sciences than Scopus and the WOS (Martin-Martin et al., 2018). However, most references that are only identified by Google Scholar are not peer-reviewed and have few citations (Ibid.). In addition, the Google search engine has difficulty handling queries with Boolean operators and fails to deliver replicable results (Gusenbauer, Haddaway, 2020). Thus, while Google Scholar may be useful for exploratory research, it is not suitable for rigorous literature reviews (Ibid.). Second, I chose Scopus over the WOS, as Scopus has more comprehensive coverage when it comes to social sciences (Martin-Martin et al., 2018).

Many researchers have taken up the trading zones research agenda tabled by Gorman (2010). A search on the Scopus database returned nearly 90 journal articles and book chapters containing the term trading zone* in their title, abstract or keywords and the name of Gorman in their references. Given the purpose of this article—to pursue the third track of Gorman’s agenda and to identify which techniques can be used to overcome the obstacle of dialogue between scientists in different disciplines—I first focused on the nearly 25 articles and chapters also containing the term interactional expert*. These results included studies by researchers who concentrated on the second track of Gorman’s agenda—the processes of acquiring interactional expertise. The analysis of this body of literature brought to light concepts linked to that of interactional expertise and which could provide answers to previous questions (T-shaped expertise; expertise in research integration; the adaptive, participatory, and transdisciplinary approach). Thus, I performed complementary searches for articles and chapters containing these terms on the Scopus database.

Interactional expertise and how is it acquired

For Gorman, Collins and Evans, a fractionated trading zone can be mediated by interactional experts (Collins et al., 2007). As mentioned above, the interactional expert is the individual with the necessary expertise to interact with the contributory expert—the individual who contributes to advancing scientific knowledge in their community (Collins & Evans, 2002). In other words, one has interactional expertise when one has no contributory expertise or practice in a technical field but is able to speak fluently the language of the field and to express technical judgements that are indistinguishable from those of contributory experts (Collins & Evans, 2015). As Collins put it:

“Interactional expertise enables one to carry on conversations with scientists about their science-conversations that will hold the scientist’s interest and not be too much of a chore for either party. In such a conversation, the party having interactional expertise may supply a lead, anticipate a response in order to shorten and speed the interchange, supply a word or an idea when the expert pauses (…). Interactional expertise can even allow one to take a devil’s advocate position if one is brave enough,” (2004: pp. 773–774).

In an interdisciplinary panel, the interactional expert can understand the nature of the disciplines, their contribution to solving the problem at hand, and justify their participation in the process (Collins & Sanders, 2007: p. 639). These studies suggest that interactional expertise consists of a mixture of knowledge and interpersonal skills.

Collins & Evans argue that acquiring interactional expertise requires a full and sustained immersion in a technical field and speaking the language of the field with contributory experts—as Collins did when he spent a decade in the gravity wave physics community (2015). Accordingly, they categorise as interactional experts: sociologists of science, managers of large science projects, reviewers of grant applications (Ibid.). Other researchers suggest that interactional expertise could also be taught in short interdisciplinary university programmes (Seager et al., 2012). They test out different interactive course formats such as: role playing exercises and problem-solving case studies (McFadden et al., 2011; Stone, 2013; Henderson et al., 2016; Conley et al., 2017). Such experiments allow them to discuss the nature of interactional expertise. For Conley et al., 2017, T-shaped expertise⁴ can be broken down into: knowledge (of another technical field), attitudes and interpersonal skills (openness and awareness to other expertise, to show interest, to listen, to ask dumb questions), and know-how (to learn and to apply new knowledge, to facilitate collaboration). This line of thinking—unravelling the skills at the root of interactional expertise—needs to be further explored through empirical investigations.

The idea of a fractionated trading zone mediated by interactional experts echoes the adaptive, participatory, and transdisciplinary (APT) approach to solving wicked problems (Xiang, 2013; Head & Xiang, 2016). When applying the APT approach, Norris et al. suggest forming a team that includes researchers with prior positive experience in interdisciplinary collaboration, and who can provide solutions to resolve conflicts and deadlock situations (2016). In the same vein, Bammer et al. explore expertise in research integration⁵ and suggest that it covers knowledge and know-how that can be acquired through experience (2020). Drawing on a case study, Huang & London find that collaboration between researchers from different disciplines benefited from hiring a skilled facilitator with expertise in group dynamics, conflict resolution, cross-cultural communication, and leading a learning and deliberation process (2016). This finding is supported by case studies of other interdisciplinary collaborations (Shrestha et al., 2018; Gilligan, 2019). These studies suggest that it may be possible for an individual to acquire interactional expertise by participating in a real-life trading zone.

Which techniques interactional experts can use to overcome the obstacle of dialogue between scientists in different disciplines

Building on a series of case studies, Gorman et al., show that for a trading zone to function, it needs to combine: an individual with interactional expertise; a pressing and compelling superordinate goal that force researchers from different disciplines to work together; and moral imagination—the ability to see the point of view and perspectives of others (2009; 2012; 2013). This combination allows researchers to see their collaborative work as a collective trial-and-error learning process, in which existing solutions are hypotheses to be tested (Gorman et al., 2013). Gorman also finds that interactional experts can rely on boundary objects in the management of a fractionated trading zone. Boundary objects, which are “both plastic enough to adapt to local needs and the constraints of the several parties employing them, yet robust enough to maintain a common identity across sites” (Star & Griesemer, 1989: p. 393), may take the form of planning documents (Gorman et al., 2012) or indicators that interactional experts can use to open and construct the debate (Gorman et al., 2014). Note that metaphors can serve the same function (Gorman et al., 2009).

Outside the trading zones research community, a number of researchers have taken up the question of how to manage an interdisciplinary panel. Building on the quantitative analysis of eight interdisciplinary groups, Love et al. show that practicing even turn-taking is a key factor in collaborative work (2022) (for a description of turn-taking in an interdisciplinary group, see: Bijker et al., 2009: pp. 71–106). Conversely, situations in which one panel member monopolises time and turns are correlated with lower scientific outcomes. Bijker et al. identify five expert panel management techniques in their study of the Dutch Gezondheidsraad: asking an expert to present their work, eliciting another’s reaction, arguing that the debate is not an end in itself, pointing out the time constraint, and referring the subject to another forum (2009: pp. 71–106). Moosavi & Browne find that scenario-building and their visual representation can serve the same function as boundary objects and help interactional experts open the debate and resolve conflicts and deadlock situations (2021). Brown et al. review a stalled interdisciplinary collaboration and made a few recommendations: “to interact in plain English (disciplinary jargon is frowned on); to foster empathy and respect for different disciplinary norms; and to reflect on what is working in collaborative interactions” (2015: p. 317).

A blind spot in the literature on trading zones seems to be the selection of disciplines and researchers that should be included in the trading zone. A number of recent journal articles discuss this particular issue. On the basis of her critical analysis of the management of the COVID crisis in the UK, Mormina argues that the production and utilisation of scientific knowledge for policymaking requires going beyond Fricker’s epistemic injustice⁶ and epistemic narrowing⁷, and forming more diverse groups in terms of epistemic and cognitive perspectives and members identity (2022). According to her, such groups would avoid groupthink (shared reality bias) and orthodoxies (traditional epistemic hierarchies) that ultimately result in path dependency and blind policy spots. More diverse groups would allow for a more shared understanding of the wicked problem at hand. Moosavi and Browne reach the same conclusion based on their analysis of the case of a failed implementation of the APT approach (2021). Beyond the search for diversity, Norris et al. suggest that one should seek balance across disciplines within the group (and should be open to adding new disciplines) to encourage the exploration of all avenues for understanding and solving the problem (2016).

In summary, this literature review reveals that, on the formation and management of an interdisciplinary panel, there remain a number of untested hypotheses that need to be further explored and integrated through empirical investigations. I also note that the internal and external validity of previous observations is assessed differently by authors. For some researchers, their findings are robust and the best practices they identified can be successfully replicated in any context (e.g.: McFadden et al., 2011). For some others, their observations must be corroborated by further empirical investigations and alternative methods (e.g.: Gorman et al., 2013). For another group, there is no such thing as universal best practices that can be successfully replicated anyplace and anytime. The local context, the capabilities, perceptions, and expectations of best practices users, have an influence on what they can achieve (Gorman et al., 2012; Xiang & Head, 2016; Huang & London, 2016; Bammer et al., 2020).

Thus, the question of which techniques interactional experts can use to overcome the obstacle of dialogue between scientists in different disciplines remain unanswered. This observation is shared by others (Bammer et al., 2020: p. 8). When answering this question, two points need particular attention. As illustrated above, the internal and external validity of observations of any empirical investigation needs to be carefully reviewed. Implications of findings for the concept of interactional expertise—its constituent elements and its modes of acquisition—must be discussed.

Sources and methods

The subject of study here is the systematic expert review method developed in France by INSERM between 1982 and 1994. This innovation is the result of a trial-and-error process of implementing INSERM leadership’s belief in the value of biomedical and public health research in health policymaking (Caby, 2021). This decade-long effort began in the wake of the 1982 reform of the French scientific research system. It was later fuelled by the French Ministry of Health’s need for a more holistic, interdisciplinary, and independent source of expertise. In this development process, INSERM researchers sought to distinguish their innovative method from existing practices—namely NIH Consensus Conferences and Cochrane Reviews—by addressing health-related public problems on a macro-policy scale, rather than medical problems on a micro-individual scale. The result is a policy advice method, which consists of an interdisciplinary systematic review of the scientific literature on a public problem. Since 1994, INSERM has published nearly eighty systematic expert reviews on a wide range of health-related public problems, including biological rhythms and school rhythms, gambling addiction, or social inequalities in health. In the early 2000s, the method was taken up by the French National Institute for Agricultural Research (INRA) with minor adaptations.

The systematic expert review method follows a step-by-step procedure that specifies who does what, and when (see INSERM, 2011: pp. 475–479 for an example). The request for a systematic expert review must come from a public authority involved in the solving of a public problem. Representatives of the public authority and INSERM panel managers⁸ must first discuss and agree on specifications and a list of research topics and questions. On this basis, panel managers, together with documentalists, perform a bibliographic search to build up a body of all relevant scientific publications on the problem. This allows them to identify relevant disciplines and researchers, and to put together an interdisciplinary ad hoc expert panel. Each expert panel member must then critically analyse the body of publications in their discipline, present their disciplinary-anchored findings to the group (including panel managers), and compile them in one of the chapters of the final report. In the next step, panel managers, together with panel members, integrate findings into an executive summary for the public authority, thus moving from a multidisciplinary to an interdisciplinary perspective (Rosenfield, 1992). Finally, panel managers organise a public presentation during which expert panel members present their findings. In the case of a particularly controversial, wicked problem, panel managers may collect stakeholders’ views on the problem through hearings or written contributions (these momentary and codified exchanges may be documented and/or transcribed in the annexes of the final report (INSERM, 2010: pp. 489–562)).

Thus, INSERM panel managers must play the role of interactional experts who have no contributory expertise in the disciplines relevant to the public problem at hand but who are able to speak fluently the language of the disciplines. They must be able to carry on smooth, meaningful conversations with panel members about the relevance of the bibliographic search, the validity of a given publication, or the writing of the executive summary. Given this brief account of the historical development and the procedure and principles of INSERM systematic expert review method, I argue that it constitutes a relevant case for the study of techniques that interactional experts can use in a trading zone.

This article compares interdisciplinary panel management in five cases of systematic expert reviews: Stress in the Workplace and Health (INSERM, 2011), Dietary Behaviours and Practices (INRA, 2010), Harm Reduction among Drug Users (INSERM, 2010), Animal Pain (INRA, 2009) and Fruits and Vegetables in Eating Behaviours (INRA, 2007) (see Table 1 for the list of case studies and their characteristics)⁹. These case studies were chosen for criteria of comparability (an identical time-space frame and the same systematic expert review procedure), diversity (in terms of wicked problems, disciplines, contractor and operator), and accessibility (to actors and records). The choice of relatively old cases (data collected between 2013 and 2016) was made to encourage interviewees to speak freely.

Table 1 List of case studies and their characteristics.

A range of sources and methods are used here. I used a bottom-up approach with a key focus on the practices and representations of INSERM and INRA panel managers. In each of the five case studies, I conducted a semi-structured interview with them (N = 7) (panels are generally headed by two managers—those interviewed here managed more than one of the surveyed groups). In the three INRA cases, I also interviewed the panel leads:¹⁰ scientists who help the staff manage the panel (N = 3) (see Table 2 for the list of interviewees and their characteristics). Note that all panel managers interviewed are women (on this point, see section “Use of techniques conditional on knowledge and know-how”). None of them had contributory expertise in the disciplines relevant to the subject of the systematic expert review at the time of the review. Four of them had prior experience in panel management, while three did not. The questions covered the systematic expert review production process, the distribution of tasks and participant relations. Since respondents also spoke about their experiences of other systematic expert reviews than those studied here, these comments are occasionally used here. The interviews were supplemented by an online questionnaire to the 119 expert panel members, to which 46 members responded (RR = 39%). The questions covered the same topics. Both interviews and the questionnaire used the critical incident technique which, when applied to the study of trading zones, consists of asking actors about successes and failures in the interdisciplinary collaboration (Gorman, 2010: p. 292). In addition, I consulted INSERM and INRA records on the five systematic expert reviews: final reports, specifications, minutes of meetings, etc. The choice of methods—interview, questionnaire and records—was guided by their heuristic potential in the study of trading zones (Ibid.). The data collected were inductively analysed with a focus on identifying problems encountered by the expert panels and solutions found by their managers. This cross-analysis obviates the biases inherent in each of the methods (Ibid.).

Table 2 List of interviewees and their characteristics.

Two obstacles to fractionated trading zones: difficult interdisciplinary dialogue and reluctance to take a public position on a policy problem

These cases of fractionated trading zones put together to produce policy advice encountered two types of obstacles: difficult interdisciplinary dialogue and some scientists’ reluctance to take a public position on a policy problem. Scientists on each of the five panels studied reported debated issues, in particular when it came to drawing up the conclusions and recommendations.

[What were the points of debate at group meetings?] “The main points of debate were (…) the effectiveness of food subsidies and taxes” (Dietary Behaviours); “There was a great deal of debate over whether fish and birds can feel pain” (Animal Pain); “Debate on the conclusions” (Fruits and Vegetables); “We didn’t all agree on (…) the incidence of HCV among users” (Harm reduction); “The first point was the very definition of psychosocial risks” (Stress in the Workplace). (Scientists, Questionnaire, 2016).

These points of dissensus were associated with two distinct situations. As shown by the following comments, some points of debate were due to the fact that given that scientific disciplines have their own practices, language and representations, bringing them together is not self-evident (Galison, 1997).

[What obstacles did you encounter over the course of the systematic expert review?]: “Mutual understanding among scientists from traditionally compartmentalised disciplines (medicine and food sciences, social sciences and physiologists)” (Dietary Behaviours); “Combining the views of experts from different disciplinary fields” (Animal Pain); “Many questions about the evidence that each discipline can provide and how much evidence it takes for us to conclude that there is an effect” (Fruits and Vegetables); “A great deal of debate regarding the group’s heterogeneous make-up (psychologists, biologists, economists and epidemiologists) and the disciplines’ different views and understandings of key concepts such as stress” (Stress in the Workplace). (Scientists, Questionnaire, 2016).

Other points of debate were due to reluctance on the part of certain scientists to state an opinion for fear of pressure from interest groups formed on the subject of the systematic expert review. Some researchers are aware that an expert report can be used by policymakers to inform and substantiate a particular policy solution that may run counter to certain interest groups, who may attack the authors of the report in return (for an example, see Hilgartner (2000).

[What obstacles did you encounter over the course of the systematic expert review?]: “Self-censorship by certain experts for fear of political reactions” (Harm Reduction case study); “An obstacle encountered on many other occasions: scientists shy away from controversies” (Animal Pain) (Scientists, Questionnaire, 2016).

“The subject that might have been a bone of contention was (…) the question of price formation, the role of the supermarkets, and so on. But it fell a bit flat because [the scientists] hid behind the fact that there was no empirical study (…). At the end of the day, they’re quite fainthearted,” (Fruits and Vegetables). (Manager, Interview, 2016).

This reluctance is also found upstream, when the panel is put together. Some managers feel that it explains the refusal by certain scientists to take part in the systematic expert review.

“Putting the group together was complicated (…). [X] refused to take part in the systematic expert review. It’s a subject that really is highly controversial (…). People can sometimes take quite an aggressive stand on the status of animals” (Manager, Interview, 2016).

Note that a number of the interviewed panel managers consider overcoming these two obstacles to be part of their role.

“For [systematic expert reviews] to be a group outcome, there needs to be discussion and criticism among the contributors (…). But that’s clearly very hard (…). [How do you go about it?] That’s very much the job of the systematic expert review unit (…) who are there to challenge the group members.” (Manager, Interview, 2016).

This representation has been institutionalised. Both research institutes, INSERM and INRA, have stipulated in their systematic expert review charter that the exercise must answer the commissioning body’s question and be adversarial (or compare and consider different points of view in context).¹¹

Individual and group selection criteria

For the managers, part of their ability to overcome the abovementioned obstacles plays out upstream when the panel is put together. At this stage, they use different individual and group selection criteria. Some are formal and others not. These principles are applied in addition to those laid down in the two institutes’ charters (competence, reliability, impartiality, absence of conflicts of interest, neutrality and confidentiality for INSERM; and competence, plurality, impartiality and transparency for INRA). All these criteria stem from the managers’ own positive and negative experiences. The number one selection criterion used by the managers is the formal criterion stipulated in the charters of the scientist’s competence in their discipline. With in-depth knowledge of the literature in their sub-field, competent scientists are able to take an informed position on the subject of the systematic expert review. The managers assess this criterion based on the number and quality of the scientist’s publications listed in databases such as WOS, Scopus, and Pubmed with the help of specialised documentalists.

“It’s always the same procedure (…), what counts most is always scientific competence, as seen from the publications, to show that [the scientist] is able to analyse the international literature on a highly specific subject,” (Manager, Interview, 2016).

The importance of this criterion is borne out by negative experiences:

“The chapter was assigned to someone who’s not … [hesitates] who has a few publications (…). And as that person did not know an awful lot about the subject, they put together a synthesis for us (…). And that chapter was a nightmare, because you could see that they didn’t have a thorough grasp of the subject (…). They were in over their head.” (Manager, Interview, 2017).

Secondarily, the managers can use two other principles to select individuals. These principles are informal. First, there is how much effort the scientist is expected to put into the systematic expert review. Scientists have to put their knowledge of the sub-field to the test with a new, substantial review of the literature on the subject of the systematic expert review. Managers assess this criterion based on their own experience and/or the opinion of their colleagues:

“[Chapter X] was done, not very well, by [Scientist Y], who flogged us stuff already written (…). [Scientist Y] always says the same things and doesn’t do much (…). It might have been worth re-examining [such or such an observation] (…). But [Scientist Y] didn’t do it.” (Manager, Interview, 2016).

“We also sometimes say [to the manager], “Don’t take [X], because they’ll let you down halfway through, because they’re snowed under (…). They’ll tell you “yes”, but they won’t have the time.” (Superior, Interview, 2016).

Managers can use proxies to project a scientist’s level of commitment. In an interview, a manager said that scientists embarking on a project on the subject of the systematic expert review at the time the panel is put together (and therefore with few publications) can be viewed positively. She would expect them to take advantage of the systematic expert review to take forward their research and publish (2016). Second here, is how open-minded the scientist is expected to be. For the managers, the scientist should be capable of accepting a compromise and closing the discussion following the debate. As with the above, negative experiences underpin this criterion:

“You do get some cranks. We’ve seen a few. The report I did on [X] ten-odd years earlier: we had this guy (…) who was completely nuts, who talked to himself! So we kept our distance. We had to work at a table in our corner while he shouted at the board. It was a nightmare!” (Lead, Interview, 2017).

The managers can use other proxies to assess this criterion. One of the managers interviewed felt that a scientist who repeatedly takes positions in the media arena may be less open-minded than others (2015). Other respondents felt that some scientists at an advanced stage in their career are less inclined to discuss the subject of the systematic expert review than to defend their own research (2017).

The managers use a fourth criterion formally stipulated in the INSERM and INRA charters:¹² the plurality of the panel as a whole. The panel first of all needs to be diverse in terms of disciplines. To this end, the managers identify the relevant fields for the subject of the systematic expert review, with the help of documentalists who conduct searches on keywords in the scientific publication databases. This principle is underpinned by negative experiences, as illustrated by a manager’s reflection in a journal:¹³

“Eliciting an open debate therefore entails analysing how experts perceive the issue based on their own discipline and recruiting representatives of different schools of thought (…) “Animal pain” is a textbook case for which this upstream analysis was lacking. The initial selection of mostly zootechnics experts immediately made [the systematic expert review] somewhat one-sided”; “The conclusions were rushed by the late inclusion of publications from the cognitive sciences (…). This is how the debate finally emerged, but there was not enough time to unpack it entirely.”

As the excerpt suggests, the panel needs to be diverse in terms of disciplines¹⁴, but also in terms of schools of thought. One manager interviewed mentioned:

“‘Intellectual’ conflicts of interest: somewhat rigid cliques, schools of thought and theoretical frames. It’s the team’s job to reach beyond the disciplines (…). That way, we can discuss and things can emerge.” (2014).

The risk is that scientists exclude publications by other schools of thought from their literature review or that they dispute their conclusions—without the panel’s knowledge. The managers assess this criterion on the basis of their own experience:

“What bothered me was the economic issue (…). Should [products high in fat and sugar] be taxed? Now, there were actually some very, very important questions behind that. And I had one school in front of me. So they pretty much got their message across (…). I did feel that it wasn’t the only school. But what could I do? I was stuck.” (Lead, Interview, 2017).

They can also call on the documentalists who have scientometric software to map the networks of co-authors and citations on the subject of the systematic expert review and thereby identify the schools of thought¹⁵. Lastly, note that the managers are aware that these affiliations with schools of thought can take the form of attachments to organisations:

“You can’t just string together disciplines to get a diverse group! You have to work on (…) the institutional points of view. Take an ecologist from the CNRS or from IRSTEA … That’s not at all the same point of view.” (Manager, Interview, 2014).

Techniques to open the debate

Once the panel has been put together, the managers have a range of techniques at their disposal to open and construct the debate—and overcome the obstacles identified in section “Two obstacles to fractionated trading zones: difficult interdisciplinary dialogue and reluctance to take a public position on a policy problem”. The first method, described by Bijker et al. (2009: pp. 71–106), consists of the managers asking each of the scientists to present to the group their analysis of their own discipline’s literature on the subject of the systematic expert review. This individual presentation is followed by a group discussion, which can also be the occasion to examine the conclusions of articles published in peer-reviewed journals based on the evidence-based criteria of the different specialities represented on the panel.

“We ask the expert to analyse the articles (…) and draw up a review with five key points covering strengths and weaknesses (…). At the second meeting, they present this summary to everyone in a plenary session. And then, there are the discussions (…): ‘Oh yes, but your strong point, if I look at it statistically, it doesn’t hold up, because you can see that the correlation isn’t right’—discussions raised by other disciplines. Then, the experts go away and write their contribution taking on board what has been said.” (Manager, Interview, 2014).

This procedure is systematically used by the managers and plays a defining role in the expert review. The techniques described below are designed to bring to the surface disagreements among scientists during this discussion (rather than after). They are also intended to make it open and transparent. One method used upstream of the individual presentation is for the managers to share with the group the public declarations of interests (PDIs) that each member has to sign. This gives the panel the means to be on the lookout for cases where a scientist defends a position close to the interest group with which they are associated. The group can then ask the scientist if their position is supported by publications and which ones. This situation puts that particular individual’s reputation on the line and can lead to self-censorship.

“We had them sign a PDI (…). I found that [Scientist X] was linked [to a given interest group] (…). He’d done it [the PDI]. And certainly, in the expert group, he was quickly singled out. And, well, he kept a low profile, he kept quiet, because everyone knew and it didn’t look good,” (Manager, Interview, 2017).

Another technique, again upstream, is for the managers to give the scientists an analytic frame to help them with their literature review. This frame can prompt the scientists to systematically examine the strengths and weaknesses of the publications on the subject of the systematic expert review—and consequently adopt, express and justify a position:

“They [the experts] needed a fairly common analytic frame for them to gather the same info (…). So we tried to give them a frame, telling them, ‘For this type of study, you absolutely need this, this and this info—where it’s from, what country, the number of subjects.” (Manager, Interview, 2016).

Downstream of the individual presentation, one practice is for the managers to ask the scientists to give their grounds for not including a publication. The purpose here is, once again, to get them to explain their position.

“You don’t want them putting one over on you! (…) You have to say to an expert, ‘Tell me, the articles you’ve rejected, is it because you refute so-and-so’s hypothesis? (…) Here, you’ve given precedence to your theory. Where have you talked about so-and-so’s theory?’ And then they’re surprised, ‘Oh, you know so-and-so’s theory? But it’s rubbish!’ (…) And so we tell them, ‘That’s not the job! The job is to take stock of all the current hypotheses, not necessarily your own.” (Manager, Interview, 2014).

Following the group discussion, each of the scientists writes, individually and from one disciplinary angle, one of the chapters that will go into the final report. The writing of an interdisciplinary summary of these chapters by the managers is a technique in itself. Basically, this summary condenses and compares each of the positions. The submission of this summary to the panel can provide the opportunity to bring to light disagreements hitherto overlooked.

“The summary prompted comments, corrections and debates among the experts (…). As we are not experts ourselves, it’s easier to criticise us. Because we don’t have the kudos, we’re not senior researchers (…). It’s also an opportunity to lay it on the line and say, ‘Well, on that score, we’ve got nothing, there’s no knowledge,’ or ‘It’s highly controversial, everyone disagrees,’ (…). And then, all the experts say, ‘Oh, come on! That’s not entirely true,’ (…). In Dietary Behaviours, now there was a summary that prompted a lot of discussions.” (Manager, Interview, 2017).

Note that it appears to be less of a problem for the scientists to criticise the managers’ work than that of their peers (see section “Use of techniques conditional on knowledge and know-how on this point”).

Techniques to close the debate

To overcome the obstacles identified in section “Two obstacles to fractionated trading zones: difficult interdisciplinary dialogue and reluctance to take a public position on a policy problem”, the managers have another range of techniques to enable them to close the debate and secure the panel’s approval for the report at the end of the process. The first method is to moderate the discussion among the scientists, tempering vehement attitudes:

“It was tough going, especially for [Scientist X] who saw everything as biological [laughs] (…). But [Manager Y] was very good at trying to smooth things over,” (Manager, Interview, 2016).

Another technique consists of the managers ruling out as intuitions those scientists’ positions that are not supported by articles published in peer-reviewed journals.

“I think we had a lot of discussions about [Chapter X]. (…). For example, the meal structure: it’s out of sync, people snack … And then the sociological studies say, ‘Well no, actually, 90% of French people continue to eat their three meals, generally seated,’ (…). We had a lot of discussions about that, and vigilance, taking care to say things that came from the literature, and not things that we instinctively felt because that was our everyday experience.” (Manager, Interview, 2017).

The managers also organise peer rereads: a chapter written by one scientist is reread by another panel member. The reader may be asked to make general or specialist remarks depending on whether they work in a related or different discipline to the author. The author’s acknowledgement of the comments can be the moment the close the debate.

“There are two experts in the group, two rereaders (…). We take one close to the field and one in a different field. That way, we have one with a bit more of a lay eye and the other with a more specialist eye (…). The text can shuttle back and forth between the experts, it’s reworked,” (Manager, Interview, 2016).

The managers can also bring on board a figure of authority to manage to reach a consensus. For example, they might ask a scientist who is not on the panel, but who has a great deal of credibility in the discipline, to reread a section under debate.

“[Scientist X] didn’t like the conclusions (…). We had them validated by a seasoned epidemiologist [Z]. At this meeting (…), I said that between [Z]’s opinion, a prominent figure, and [X]’s opinion, I wouldn’t think twice,” (Manager, Interview, 2016).

Another technique that managers can use is to point out to scientists who think about leaving the group that doing so could have repercussions for their reputation. A person who resigned might be presented as a radical refusing to debate, fickle and capricious, or a malingerer shirking the work.

“Then, one of the experts said, ‘(…) Right! I quit!’ I said, ‘Oh, but you can’t do that now! You should have said so before, because now, we’re all in the same boat. You’ve written a report, now you need to write a summary.’ Interviewer: ‘Did everyone stay?’ Yes, you see, if you put to them, ‘If you say now you don’t agree when we’ve been working for eighteen months, you’re shooting yourself in the foot, because people will wonder why you chose this moment in time to say you disagreed when you could have said so [before],’” (Manager, Interview, 2017).

When none of the abovementioned techniques works and the managers fail to reach a consensus, there remains the possibility for them to state the intransigent positions in the final report:

“When things got too antagonistic and people dug in their heels, we basically put both hypotheses in, explaining that there were contradictory interpretations,” (Manager, Interview, 2017).

Use of techniques conditional on knowledge and know-how

These expert selection criteria and panel management techniques cannot be handled by everyone nor can they be put to use immediately. They require specific knowledge and skills that the managers acquire over the course of their experience in fractionated trading zones. Their descriptions of the first time they managed a panel speak volumes about this need to learn by trial and error:

“It was following a research programme on greenhouse gas emissions. The Ministry had asked for (…) the same group of scientists to draw up a sort of summary (…). We went back to the circle of scientists who couldn’t see what purpose we could serve them other than to book rooms (…). That was the first exercise for us, where we also had to try and see what our place could be in the scheme of things (…). It was a bit strange creating this new activity and assigning it (…) to three [female] engineers¹⁶ branded more as PR,” (Manager, Interview, 2015).

“In fact, it was the experts who ran the show. Needless to say, [Manager X] got taken hostage by the group [regarding X’s participation in her first systematic expert review],” (Manager, Interview, 2017).

In other words, the main stake for the managers is to gain recognition from the panel members. They have to overcome any prejudice the scientists may have with respect to their status and gender—as illustrated by the first of the above two quotes. In the five case studies, all the managers except one had a status of research engineer¹⁷ or systematic expert review officer, whereas the majority of the scientists were senior researchers or university professors. All of the managers interviewed were women, whereas most of the members of the panels studied were men.

The first element of knowledge that the managers need is scientific knowledge in their organisation’s meta-disciplinary field. All the INSERM managers interviewed had trained in biomedical research: in biophysics, immunology or haematology (see Table 2). The same applies to INRA where all, except one, held agricultural science degrees. This general knowledge can be coupled with more specific knowledge of one of the disciplines contained in the systematic expert review. This knowledge should facilitate the managers’ understanding of the subject and the dedicated literature, the recognition they gain from the scientists—and their relations with the scientists.

“[Superior X] asked me to do it [to manage this panel] (…). I had already done [the] mental health [report]. So I had some knowledge on the subject. I wasn’t completely clueless—it helps,” (Manager, Interview, 2016).

Other knowledge and know-how required of the managers is to be up to speed with the publication databases in their meta-disciplinary field (how they work, their qualities and their biases) so that they can conduct bibliographical research (even though they can count on the help of professional documentalists). Having a command of these databases is the condition for putting together the right panel and body of literature. Another skill required is for the managers and panel leads to be capable of facilitating panel meeting discussions:

“[Panel lead X] also got quite involved in running it (…), taking the group in hand, driving forward discussions, playing for time, just very dynamic, really, in facilitation,” (Manager, Interview, 2017).

They are expected to have the ability to be both open and firm with the scientists. The managers also need to be skilled writers and editors. They have to be capable of making corrections to the substance and form of the chapters and taking an active part in writing the summary and the conclusions:

“There was just this tiny paragraph on the supervised injection sites in the conclusions (…). We spent time writing it. We weighed each word,” (Manager, Interview, 2014).

The last element of know-how required is that the managers are supposed to be able to design and conduct a project. This entails translating an order (i.e., knowledge questions) into a set of tasks—timetabled and assigned to different individuals and groups—and overseeing their completion on time. The distribution of tasks among managers also suggests that handling the criteria and techniques identified in sections “Individual and group selection criteria”, “Techniques to open the debate”, and “Techniques to close the debate” is subject to having particular knowledge and know-how. In an interview, one manager said:

“My job, as I was telling you, is mainly to write summaries. Well, there is a thing, which is I’m not project leader. That’s [Manager X]. I currently work alongside a colleague in a two-person team, because (…) I’m not a leader, I’m not diplomatic,” (2015).

Note that this list of knowledge and skills required to handle the expert selection criteria and panel management techniques is formally specified in the manager job descriptions published by INRA and INSERM.¹⁸

Conclusion and discussion

This article undertook to pursue the third track of Gorman’s agenda and to identify which techniques interactional experts can use to overcome the obstacle of dialogue between scientists in different disciplines—in the context of interdisciplinary expert panels (which fall into the category of fractionated trading zones).

The literature review (section “Review of the literature on interactional expertise in trading zones”) revealed that there remain a number of untested hypotheses and first observations regarding the formation and management of an interdisciplinary panel that need to be further explored and integrated through empirical investigations. A comparative analysis was conducted of five interdisciplinary expert panels working on five systematic expert reviews (section “Sources and methods”). In these case studies, panel managers play the role of interactional experts who have no contributory expertise in the disciplines relevant to the public problem but who are able to speak fluently the language of the disciplines. In this last section, I summarise findings. I discuss their implications for the concept of interactional expertise—its constituent elements and its modes of acquisition. Drawing on insights from the literature on policy transfer and the critique of best practices, I carefully review their validity.

Which techniques interactional experts mediating a fractionated trading zone can use to overcome the obstacle of dialogue between scientists in different disciplines

Section “Individual and group selection criteria” showed that panel managers use different individual and group selection criteria based on their own positive and negative experience to overcome the two obstacles they encounter: difficult interdisciplinary dialogue and some scientists’ reluctance to take a public position on a public problem¹⁹. These criteria are scientific competence, the scientist’s projected engagement in the systematic expert review and predicted level of open-mindedness, and the plurality of the panel in terms of disciplines and schools of thought. These criteria (and the experiences on which they are based) support initial observations (Moosavi & Browne, 2021; Mormina, 2022) that forming more diverse groups in terms of epistemic and cognitive perspectives fosters better collaboration among panel members and eventually better scientific outcomes—a better and more shared understanding of the wicked problem at hand. Section “Individual and group selection criteria” also corroborates Norris’ suggestion (2016) that seeking balance across disciplines within the group facilitates the exploration of all avenues for understanding the problem. Future research should further explore the effects of the absence of a discipline from a panel.

Section “Techniques to open the debate” brought to light that the managers have a range of techniques at their disposal to open and construct the debate, also based on their own experience. A first method consists of the managers asking each of the scientists to present to the group their analysis of their own discipline’s literature on the subject of the systematic expert review, followed by a group discussion. This is consistent with Love et al.’s finding (2022) that practicing even turn-taking fosters collaborative work in interdisciplinary panels. The reason could be that researchers have to use plain English and avoid disciplinary jargon in this situation, as suggested by Brown et al. (2015)). Managers also use boundary objects others than metaphors (Gorman et al., 2009), planning documents (Gorman et al. 2012), and visual representations of scenario (Moosavi & Browne, 2021) to open and construct the debate. In INSERM and INRA systematic expert reviews, the draft interdisciplinary summary of the scientists’ presentations and group discussions and the common frame for analysing publications play the same role. Other methods include: sharing PDIs with the panel; asking a panel member their reasons for excluding a publication from the analysis.

Section “Techniques to close the debate” showed that the managers have another range of techniques to enable them to close the debate and secure the panel’s approval for the report. These methods are also based on the managers’ positive and negative experience. A first technique consists of pointing out the repercussions of a scientist’s resignation and thus appealing to the pressing and compelling superordinate goals (Gorman et al., 2009; 2012; 2013) of understanding the problem at hand and answering the commissioning body’s question. Managers also temper vehement attitudes in the group, thus fostering respectful interactions between panel members and epistemic perspectives, as recommended by Brown et al. (2015). Other methods include: ruling out scientists’ positions not supported by publications; organising peer rereads by other panel members; bringing on board a figure of authority.

On the constituent elements of interactional expertise

Section “Use of techniques conditional on knowledge and know-how” revealed that the use of the techniques identified in sections “Techniques to open the debate” and “Techniques to close the debate” calls for specific knowledge and skills that the managers acquire over the course of their experience in fractionated trading zones. This knowledge and know-how include: scientific knowledge in their organisation’s meta-disciplinary field and the field of the systematic expert review; bibliographic research skills, group management and facilitation, writing and editing qualities, and project management. Managers with a command of these skills and knowledge have the means to overcome any status- or gender-based prejudice the scientists may have in their regard and gain their recognition. These observations illustrate the point made by Conley et al. that interactional expertise can be broken down into knowledge, interpersonal skills, and know-how (2017). They echo Boix Mansilla et al.’s findings that successful interdisciplinary collaborations possess cognitive, emotional, and interactional dimensions, which ultimately depend on the organisational rules established for collaboration (2016). In addition, the previous skillset could inform the discussion on the measurement of interactional expertise, given the limitations of Turing tests (Collins & Evans, 2015).

On the modes of acquisition of interactional expertise

Section “Use of techniques conditional on knowledge and know-how” observations contradict the argument made by Collins & Evans (2015) that interactional expertise can only be acquired through a full and sustained immersion in one technical field and speaking the language of the field with contributory experts. At INSERM and INRA, the managers gain interactional expertise by engaging in a trial-and-error learning process. They first take an active part in a fractionated trading zone (they perform a bibliographic search; open and construct the debate; write a draft summary, etc.) and then reflect on what facilitated and what hindered collaborative work, before engaging in a new trading zone. This nuances Norris et al.’s assumption (Norris et al., 2016) that researchers with prior positive experience in interdisciplinary collaboration can provide solutions to resolve conflicts and deadlock situations, and Brown et al.’s recommendation (Brown et al., 2015) to reflect on what is working to foster interdisciplinary collaboration. Experiencing poor interdisciplinary collaboration and reflecting on what didn’t work seems to be equally important. Thus, if interactional expertise is to be taught in short university programmes, as a growing number of researchers advocate (McFadden et al., 2011; Seager et al., 2012; Stone, 2013; Henderson et al., 2016; Conley et al., 2017), such programmes could include in-depth case studies of failed interdisciplinary collaborations and their lessons. Future empirical investigations should determine the added value of this teaching approach.

Some final considerations regarding the validity of findings

In this article, I have endeavoured to take into account the methodological issues raised by the research community on policy transfer and their critique of best practices²⁰. Following Radaelli (2004) and Bulkeley (2006), I have sought to provide as many details as the format allowed on the practices I identified and the context of their production. In line with Mattocks (2018), I have highlighted what a collaborative practice is—a technique enmeshed in a mixture of knowledge, interpersonal skills, know-how, positive and negative experiences. I have illustrated when and how panel managers use which techniques. I have made explicit what makes these practices good practices in the eyes of panel managers. I have not silenced the obstacles they encountered, nor their negative experiences. In addition, I made explicit data collection and analysis methods and case selection criteria. I believe this wealth of details should allow readers to assess the internal validity of my observations. As for external validity, I have described the historical development of the INSERM systematic expert review method, and the objectives, principles, and procedure of this form of policy advice. I have emphasised that the practices I identified require panel managers to possess specific knowledge and skills. Thus, the practices presented here are not best practices that can be successfully replicated anyplace and anytime. They are good practices in the sense that different panel managers have found them to be conducive to better interdisciplinary dialogue across case studies—in the particular settings I described. Panel managers (and interactional experts mediating a fractionated zone) who would like to draw inspiration from this article should keep in mind that successful interdisciplinary collaborations have a holistic dimension (they result from the joint implementation of different techniques combined with the particularities of the local context of implementation), and that there are a variety of ways to achieve success (Radaelli, 2004; Gilligan, 2019).