Looking at disasters such as Hurricane Katrina in 2005 or the flood in the Ahr Valley in 2021, the lack of effective communication and coordination between emergency response teams is often cited as the reason why so many people lose their lives in such disasters (Dombrowsky, 2022; Moynihan, 2009). In all teams, particularly in interprofessional emergency response teams, effective communication among specialized experts is crucial for team coordination and effectiveness (DeChurch & Mesmer-Magnus, 2010). For example, firefighters, police, and paramedics must request the correct expert to minimize casualties and avoid life-threatening consequences. Studies on interprofessional emergency response teams have shown, however, that the flow of information between team members can be disturbed, and that in such cases, they are not connected enough to coordinate their actions effectively (Mohammedfam et al., 2015; Reddy et al., 2009). As technology advances, the question arises whether software agents acting as autonomous team members can be integrated in teams by taking over team communication among team members to further support effective teamwork (Fiore & Wiltshire, 2016).

Interprofessional emergency response teams can be described as a so-called transactive memory system in which members are specialized in specific domains to distribute the workload of an overarching common goal (Hollingshead et al., 2012). Previous research has shown that teams with a transactive memory system perform better than teams without a transactive memory system (e.g., Austin, 2003; Hinsz et al., 1997; Lewis, 2003, 2004). As a transactive memory system leads to high interdependence among team members, they must communicate with each other by requesting other experts in their team (Yan et al., 2021). For successful team communication (i.e., less information seeking about the experts and few mistakes in requesting experts), the importance of a meta-knowledge as a directory of “who knows what” that stores knowledge about the expertise of other team members has been emphasized (Ellwart & Antoni, 2017; Faraj & Sproull, 2000; Wegner, 1987, 1995). Team members without or incorrect meta-knowledge must spend time on gathering information about experts in the team (Brandon & Hollingshead, 2004; Su & Contractor, 2011) or are likely to make mistakes when requesting an expert (Austin, 2003). The importance of meta-knowledge in interprofessional emergency response or health care teams has also been repeatedly emphasized (Burtscher et al., 2011; Ford & Schmidt, 2000; Reddy et al., 2009).

To support team communication, a transactive memory system among humans and software agents has been discussed, for example, that software agents hold correct meta-knowledge to automate communication between specialized team members (cf. Fiore & Wiltshire, 2016). If team communication is automated by a software agent, the likelihood of breakdowns in the communication flow can be reduced and team performance can be improved. In hospitals, solutions with software agents to dispatch interprofessional emergency response teams are already used (AscomUK, 2015). Human factor research, however, emphasizes the need for flexibility in switching between manual control and automation, for example, due to system failures (Calhoun, 2022). Thus, the question arises whether teams with and without meta-knowledge differ in adapting to different levels of automated team communication as indicated by expert seeking, mistakes in requesting experts, and team performance.

This study contributes to existing research in three ways. First, in the context of interprofessional emergency response teams, we aim to replicate that teams that have learned meta-knowledge in advance perform better than teams without meta-knowledge. Second, we investigate the differences between these teams regarding team communication processes of expert seeking and mistakes in requesting experts, also considering these communication processes as mediators of team performance. Third, with respect to team adaptability in a volatile environment, we investigate whether teams with or without meta-knowledge can adapt best to different levels of automated team communication in terms of expert seeking, mistakes in requesting experts, and team performance.

Transactive Memory System

The concept of transactive memory “explains how people in collectives learn, store, use, and coordinate their knowledge to accomplish individual, group, and organizational goals” (Hollingshead et al., 2012, p. 421). The term transactive memory system stems from the research of Wegner (1987, 1995), where it is defined as “a set of individual memory systems in combination with the communication that takes place between individuals” (Wegner, 1987, p. 186). The positive effect of a transactive memory system on team performance is explained by the possibility of specialization. Specialization allows team members to focus on individual task-specific knowledge and reduces information-processing demands and workload (Brauner & Becker, 2006). Thus, in teams with a transactive memory system, team members only need to process the information related to their area of expertise, while other information can be neglected and passed on to the respective expert team member (Wegner, 1987). This distribution and reduction of workload positively influences team performance (DeChurch & Mesmer-Magnus, 2010; Hollingshead et al., 2012). Due to specialization in specific domains, team members rely on transaction or communication among each other to achieve the team goal (Lewis, 2003). Thus, team members must communicate to coordinate their information, knowledge, and resources to stay effective (Mohammedfam et al., 2015; Reddy et al., 2009; Wegner, 1987, 1995). A prerequisite for successful communication in teams with a transactive memory system is meta-knowledge.

Effects of Meta-Knowledge on Team Processes and Performance

Meta-knowledge describes the knowledge of team members about “who knows what” (Antoni & Ellwart, 2017; Hollingshead et al., 2012). Wegner, (1995) has described meta-knowledge by directories containing knowledge about the information “what” combined with the location of storage “where” (Wegner, 1995). Brandon and Hollingshead, (2004) describe this directory as task-expertise-person units (TEP units), defined as the knowledge of the connections between people and expertise. Directory, TEP units, and meta-knowledge describe the same phenomenon of knowing “who knows what.” Wegner, (1995) postulates that for successful use of an external storage (i.e., transactive memory system), meta-knowledge is a general requirement. Consequently, interprofessional teams with specialized team members without meta-knowledge have no awareness of team members’ expertise and are unable to successfully use their transactive memory system without additional effort.

To compensate for their lack of meta-knowledge, teams without meta-knowledge must invest effort in seeking information about the correct expert in the team (Brandon & Hollingshead, 2004). Expert seeking is defined as the behavior of gathering information about experts, for example, in knowledge repositories (cf. Su & Contractor, 2011). Expert seeking means an additional effort to team members’ actual task, as team members must first gather information about the correct expert before they can request this expert. This leads to inefficient and slow team communication and to reduced team performance (cf. Kirschner et al., 2018). If workload or time pressure of team members without meta-knowledge increase, they might not have time to gather information about the experts in their team. Thus, mistakes in communication (i.e., requesting the wrong experts) and reduced performance are likely to occur (cf. Eppler & Mengis, 2004). Mistakes in requesting experts are defined as incorrect requests among experts in a team.

The superiority of teams with meta-knowledge regarding expert seeking, mistakes in requesting experts, and team performance is due to the automation of learned schemas (Sweller et al., 2011). Learned schemas are cognitive constructs of multiple elements of information that are stored as one element in the long-term memory (Chi et al., 1982). When a schema is used in practice, the schema is processed as one and not as multiple elements. Thus, the processing of information becomes automated and allows for effortless processing of information (Sweller et al., 2011). Members of teams with meta-knowledge develop schemas of their own and others expertise. Thus, members with internalized meta-knowledge do not have to repeatedly search for information about other team members’ expertise during the task accomplishment because they know by heart which expert to request. Memorizing other team members’ expertise as part of meta-knowledge reduces errors in requesting help and enables team members to respond more quickly in emergencies. Members of teams without meta-knowledge only develop schemas of their own expertise. As a result, they must repeatedly search for information about other team members’ expertise or, if they fail to obtain that information, make mistakes in requesting experts.

  • Hypothesis 1: Teams without meta-knowledge (a) show more expert seeking, (b) make more mistakes in requesting experts, and (c) show a lower performance than teams with meta-knowledge.

Looking at the mechanism that explains the effect of meta-knowledge on team performance, mediators in this relationship may be expert seeking and mistakes in requesting experts. As in teams without meta-knowledge, team members must repeatedly search for information about other team members’ expertise; they may not request the correct experts. Consequently, team members must spend time resolving the incorrect request, and team performance is likely to be lower than in teams with meta-knowledge (Austin, 2003). For example, when firefighters request help from a police officer, but this task would be the paramedics’ responsibility, the police officer must realize that the requested help cannot be provided and must deny the request. As a result, the firefighters must search for information about the correct expert and request help from the paramedic, who can only then provide the requested help. This process binds a lot of time of at least two of the three team members mentioned here, affecting team performance negatively. Previous research on collaborative cognitive load theory (Kirschner et al., 2018) describes that additional transaction (e.g., due to resolving the incorrect expert request) leads to additional cognitive load, which will then foster mistakes, conflicts, and unnecessary duplication.

  • Hypothesis 2: The effect of meta-knowledge on team performance is mediated by (a) expert seeking and (b) mistakes in requesting experts.

Effects of Automated Team Communication on Team Processes and Performance

In the last years, technological progress has led to human-automation teamwork (Fiore & Wiltshire, 2016). In such teams, human and technological programs, also called software agents, act as autonomous team members (e.g., artificial intelligence to support decision-making, or to operate subtasks). Software agents are increasingly integrated in “teamwork activities involving coordination, task reallocation, and continuous interaction with humans” (O’Neill et al., 2022, p. 904). At the highest level of automation, the software agent decides and acts autonomously, and human team members are not able to interact with the agent. At the lowest level, namely the level of manual control, software agents offer no assistance and team members must decide and perform all tasks by themselves (O’Neill, 2022).

Those software agents can be implemented to take over team communication (cf. Fiore & Wiltshire, 2016; Yan et al., 2021; real-life example: Unite Alarm Agent, AscomUK, 2015). At the highest level of automated team communication, software agents take over team communication by requesting experts and answering requests from other team members. Human team members are no longer involved in this process and are not able to interact with the agent. As team communication is sourced out to the software agent, inefficient team communication (i.e., repeatedly gathering information about experts, mistakes in requesting experts) should be reduced when the software agent is functioning correctly. Furthermore, the workload for team members should be reduced (Kirschner et al., 2018), as they do not have to concentrate on team communication. Human team members can focus and work on their individual expert task (e.g., handle the emergencies more quickly). Thus, team performance should increase.

  • Hypothesis 3: When the level of automation changes from manual (T1) to automated (T2) team communication, (a) expert seeking decreases and (b) team performance increases.

Meta-knowledge is beneficial for human team members for efficient team communication (Brandon & Hollingshead, 2004; DeChurch & Mesmer-Magnus, 2010; Wegner, 1987). However, when software agents take over communication among human team members, team communication is no subtask for human team members anymore. Thus, there should be no differences between teams with and without meta-knowledge regarding their expert seeking and team performance, as meta-knowledge no longer offers a benefit. In both teams with and without meta-knowledge, team members must only work on their own individual expert task, thereby increasing the positive effect of specialization due to the transactive memory system (cf. Fiore & Wiltshire, 2016; Kirschner et al., 2018). Under manual control, teams without meta-knowledge engage more in seeking experts and perform worse compared to teams with meta-knowledge (see Hypothesis 1). During automated team communication, teams without meta-knowledge will show similar expert seeking and team performance to teams with meta-knowledge. Therefore, representing the interaction between meta-knowledge and the different levels of manual and automated team communication, the change in expert seeking and team performance from manual to automated team communication should be stronger for teams without meta-knowledge than for teams with meta-knowledge.

  • Hypothesis 4: When the level of automation changes from manual (T1) to automated (T2) team communication, (a) the decrease of expert seeking from T1 to T2 and (b) the increase of team performance from T1 to T2 should be stronger in teams without meta-knowledge than in teams with meta-knowledge.

Research on human factors in automated systems has shown that the ability to adapt to changing automation levels distinguishes high-performance systems (Feigh et al., 2012) and teams (Calhoun, 2022). When team communication is automated, external or internal triggers (e.g., software agent crashes, misunderstandings among team members) may force the team to switch back from automated to manual team communication. Thus, the question arises how well teams can adapt regarding expert seeking, expert requesting, and team performance when the automation level switches back from automated to manual team communication. If team communication switches back to manual control, expert seeking and mistakes in requesting experts should increase again. As a result of spending more time in seeking experts and resolving mistakes in requesting experts, team performance should decrease.

  • Hypothesis 5: When the level of automation changes from automated (T2) to manual (T3) team communication, (a) expert seeking and (b) mistakes in requesting experts increase, and (c) team performance decreases.

If team communication changes from automated to manual control, team members must adapt their behavior and require meta-knowledge to be able to take over the coordinated communication again (cf. Austin, 2003; Faraj & Sproull, 2000). After meta-knowledge about “who knows what” was not needed by human team members during the phase of automated team communication, meta-knowledge is now again the key factor for effective communication between team experts (cf. Wegner, 1987). Teams with and without meta-knowledge show similar behavior in expert seeking, mistakes in requesting experts, and team performance during the phase of automated team communication. During the phase of manual control, teams without meta-knowledge spend more time gathering information about experts (i.e., expert seeking) and make more mistakes in requesting experts than teams with meta-knowledge. As a result, the performance of teams without meta-knowledge should be worse than in teams with meta-knowledge, implying a worse team adaptation to the changed situation of manual team communication. Thus, representing the interaction between meta-knowledge and the different levels of automated and manual team communication, the change in seeking experts, mistakes in requesting experts, and team performance from automated to manual team communication should be stronger in teams without than in teams with meta-knowledge, as teams with meta-knowledge can draw on meta-knowledge from long-term memory (cf. Kirschner et al., 2018; Sweller et al., 2011). The stronger decrease in team performance from automated to manual team communication for teams without meta-knowledge than for teams with meta-knowledge represents a worse team adaptation due to the lack of meta-knowledge. Figure 1 presents the overall research model.

  • Hypothesis 6: When the level of automation changes from automated (T2) to manual (T3) team communication, (a) the increase in expert seeking from T2 to T3, (b) the increase of mistakes in requesting experts from T2 to T3, and (c) the decrease in team performance from T2 to T3 should be stronger in teams without meta-knowledge than in teams with meta-knowledge.

Fig. 1
figure 1

Overall research model including the proposed hypotheses

Full size image

Method

Participants

In total, 377 students in 127 teams from German universities studying different undergraduate and graduate programs participated in the experiment. For participation, students received course credit or monetary compensation. Seven teams had to be excluded because four teams consisted only of two instead of three students and the software program failed in three teams. After exclusion, 120 teams consisting of 360 members (69.4% females, M(SD)age = 23.17 (3.25) years) remained. Students were randomly assigned to teams of three and the teams were randomly assigned to two conditions of meta-knowledge (with vs. without). In total, 61 teams were assigned to the condition of learning meta-knowledge in advance and 59 teams were assigned to the condition of not learning meta-knowledge. Approximately 68% of the participants stated that they are somewhat or not at all experienced with computer games. Approximately 91% of the participants stated that they had not yet played the control-center simulation-task used. Approximately 40% of the participants stated that they did not know each other.

Team Task: Control-Center Simulation to Handle Emergencies as an Interprofessional Emergency Response Team

As it is difficult to investigate real-life interprofessional emergency response teams during emergencies, we decided to use an experimental design with a realistic simulation of a control-center, such as other studies on emergency response teams have done (Sanchez-Manzanares et al., 2020; Uitdewilligen et al., 2013, 2018). Such computer-based control-center simulations focusing on emergency response, action, or decision-making teams have often been used to study various teamwork variables (Pearsall et al., 2010).

In the experiment, students in an interprofessional emergency response control-center team played a computer-based control-center simulation (FCI; Fire, Crimes, and Injuries). The FCI is a serious game and uses the problem of resource allocation in interprofessional emergency response teams (i.e., limited resources, such as people and vehicles that must be allocated strategically to handle emergencies as quick as possible). The content and the procedure of the tasks in the FCI are based on the real context of control-centers. The FCI was developed based on official and freely available documents (e.g., fire department regulations) and has been adapted to the experimental context to ensure a simulation close to real control-centers. In real control-centers like in the FCI, emergency calls must be accepted and evaluated, the necessary resources must be made available (e.g., available vehicles and people) and dispatched, and the coordination of those resources must be checked and monitored (Rechenbach, 2013). This is done based on an available vehicle fleet and a staff plan. These tasks are also included in the FCI (see Fig. 2). Furthermore, the FCI is developed in such a way that interdependencies and communication among team members must take place for emergencies to be handled successfully. For more information about the FCI, please see the detailed description of the FCI by Timm et al., (2022).

Fig. 2
figure 2

FCI interface. Note. A = map of the city displaying all received emergencies, located stations, and all driving vehicles in the city. B = team score. C = received emergencies. D = own stations. E = own vehicles in the selected station (if no station is selected, this area is empty). F = own staff with competencies in the selected station (if no station is selected, this area is empty). G = button to send a request to the other two roles for a specific vehicle. H = list of all requests sent to other team members and indication if the request was accepted or declined from the other team members. I = list of all received requests from other team members and the option to see more details (i.e., which vehicle is requested), accept or decline this request. J = digital knowledge repositories including the symbols and names of the vehicles of the respective role (see Fig. 3 more details)

Full size image

Figure 2 represents the FCI interface for the role of the firefighter. Team members have distinct roles in the FCI: police officer, firefighter, and paramedic. The right side of the FCI interface displays four areas for incoming emergencies (C), respective stations (D = fire departments, police stations, or hospitals), role-specific vehicles (E), and people with specific competencies (F). Team members only see and have access to their role-specific stations, vehicles, and people. The left side of the FCI interface displays a map showing all emergencies of all roles (A) and a communication panel with functions to request help from other team members (G and H) and to accept/decline other team members’ requests (I). At the bottom left is a digital knowledge repository describing which role controls which vehicles, including the symbols and names of the vehicles (J; cf. Figure 3). The interface for the roles of police officers and paramedics looks similar, except for police stations and hospitals instead of fire departments as well as other role-specific vehicles and competencies of people.

Fig. 3
figure 3

Knowledge repositories showing which role has which vehicles. Note. Representing area J in Fig. 2. Klicks on these digital knowledge repositories were used to measure expert seeking

Full size image

In experimental phases 1 and 3 of manual control, the participants performed all subtasks. In experimental phase 2 of automated team communication, the software agent took over the tasks of G (i.e., requesting help from other team members for own emergencies) and I (i.e., accepting, or declining request from other team members). Thus, the areas of G, H, and I were no longer visible for participants.

Procedure and Manipulation of Meta-Knowledge and Team Communication

After being welcomed, team members were placed in one room at three separate computers with the FCI open with their respective roles (see Fig. 2). Privacy shields between the team members made face-to-face contact during the experiment impossible. The experiment contained an introduction phase, a manipulation phase, and three experimental phases.

In the introduction phase, team members received information about the aim of the experiment and signed a consent form. Thereafter, they were introduced to the FCI by a video for around 6 min. Afterwards, they played a FCI tutorial for 10 min. Here, the team members had to handle emergencies and could apply the knowledge they had learned from the video. In the tutorial, other emergencies (e.g., salvage operation, arson, mass panic) than in the experimental phases were used to avoid learning effects. At the end of the introduction phase, team members were asked to rate their skills using the FCI, the perceived task interdependence in the FCI, and the usefulness of the video and tutorial.

After the introduction phase, the manipulation of meta-knowledge at the team level followed. To acquire meta-knowledge, team members had to learn the symbols of vehicles they needed from the other roles to handle their own emergencies, as well as the symbols of their own vehicles that the other roles needed to handle their emergencies. For example, firefighters learned all symbols of their own emergencies and the symbols of vehicles needed from police and paramedics to handle them, as well as the symbols of vehicles of the firefighter that the police and paramedic will request. Thus, these teams had meta-knowledge in the form of “which role has which vehicle that I will need” and “which team member needs which vehicle from me.” In the condition of teams without meta-knowledge, team members only learned symbols of their own emergencies and vehicles but no symbols of other team member emergencies and vehicles. To learn the same amount of information, these teams had to additionally learn the street names where their stations were located. The learning phase lasted ten minutes for both conditions. The material to manipulate meta-knowledge is provided as supplemental material. After the learning phase, all participants had to fill out questions via a knowledge test for the symbols to ensure whether they really learned the presented symbols.

The manipulation of meta-knowledge was followed by three experimental phases of 10 min each. Similar scenarios were used in the three phases: The teams received twelve emergencies within 10 min, four emergencies for each role. The firefighter had to deal with two fires, one flood, and one hazardous material as quickly as possible, the police officer with two protests, one accident, and one crime, and the paramedic with two injuries, one pandemic, and one contamination. Seven vehicles were needed to handle each emergency: five owned vehicles and one vehicle from each of the other two roles. To send a vehicle to an emergency, a team member had to select the emergency and a station that had the correct and required vehicle. Before sending a vehicle to an emergency, the vehicle had to be staffed with three people with vehicle-specific competencies. This procedure had to be repeated until all required owned vehicles had been sent to the emergency. For the other two vehicles needed from the other team members, the team member had to send a request to the corresponding team member. Conversely, there were also requests for the team members’ own vehicles for emergencies that did not fall within the team members’ area of responsibility. Team members had to accept or decline these requests. If a request was accepted, the procedure for sending the vehicle to the emergency was the same as described above.

The three experimental phases differed in their level of automation (see also note of Fig. 2 for details). In the first experimental phase of manual team communication, team members had to decide and perform all subtasks (selecting emergencies, stations, vehicles, and people as well as requesting help and accepting or declining requests from other team members). In the second experimental phase of automated team communication, a software agent took over requesting help and accepting or declining requests from other team members so that team members only had to perform role-specific subtasks (selecting emergencies, stations, vehicles, and people). In the third experimental phase of manual team communication, team members must again decide and perform all subtasks by themselves, as in the first phase. During the experimental phases, a team performance score was displayed at the FCI interface to show the interdependence among team members. After each experimental phase, team members completed questionnaires. The whole experiment lasted around 80 to 90 min. Figure 4 represents the procedure of this study.

Fig. 4
figure 4

Procedure of the study

Full size image

Dependent Measures

To measure expert seeking, we used the log-files from the FCI. This score is composed of how many clicks an individual team member made on the digital knowledge repository of the roles’ carpools (showing symbols of each vehicle of each role, cf. Figure 3). Expert seeking was aggregated on team level by summing the number of clicks by each member of a team. For each experimental phase (T1, T2, and T3), a team score for expert seeking was calculated.

To measure mistakes in requesting experts, we used the log-files from the FCI. This score is composed of how many wrong requests each individual team member sent (e.g., a firefighter requested an ambulance from the police officer instead of from the paramedic). Mistakes in requesting experts were aggregated on team level by summing the number of wrong requests sent by each member of a team. As in the second experimental phase, the software agent took over the team communication, and there was no possibility for mistakes in requesting experts. Thus, a team score for mistakes in requesting experts was calculated for the first and the third experimental phase (T1 and T3).

To measure team performance, we used the log-files from the FCI of each experimental phase (T1, T2, and T3). The team performance score was computed as a team score, where one point was given for each properly dispatched vehicle, one point for each properly arrived vehicle at the emergency, and 30 points for the successful processing of an emergency (i.e., all required vehicles arrived at the emergency). This performance indicator represents efficiency, as the faster the teams handled emergencies, the more points they received. In each experimental phase, the highest team performance score possible was 528 points (= (12 emergencies * 7 dispatched vehicles) + (12 emergencies * 7 arrived vehicles) + (12 emergencies * 30 points for successful emergency handling) = 528 points), and the lowest was zero.

Analysis

To test our Hypotheses 1, 3, 4, 5, and 6, we calculated three mixed analyses of variance (mixed ANOVA) with meta-knowledge as a between-subject factor (with vs. without) and experimental phases as a within-subject factor (manual control = T1, automation = T2, manual control = T3) on the dependent variables of expert seeking, mistakes in requesting experts, and team performance using SPSS. To test Hypothesis 2, we analyzed two mediation models (for T1 and T3), with meta-knowledge as independent variable, team performance as dependent variable, and expert seeking and mistakes in requesting experts as mediators. For this analysis, we used the macro process 4.0. by Hayes for SPSS (Model 6). For all analyses, we will report exact p-values, except of p < 0.001, and interpret the effect sizes based on Cohen, (1988).

Results

Preliminary Analyses

As preliminary analyses, we examined whether the teams in the two conditions differed in demographical data, whether the manipulation of meta-knowledge worked, and analyzed the prerequisites of an ANOVA. The teams in the conditions did not differ significantly regarding age, familiarity with computer games (F < 1, p > 0.637), gender (χ2 (3, 360) = 1.95, p = 0.790) or familiarity with the FCI game (χ2 (1, 360) = 1.03, p = 0.363). Furthermore, the teams in the conditions did not significantly differ regarding self-assessed FCI skill after the tutorial, perceived task interdependence in the FCI, and usefulness of the video and tutorial (F < 2.271, p > 0.133). The teams in the two conditions differed significantly regarding familiarity with other team members (χ2 (2, 360) = 9.44, p = 0.009, φ = 0.162), which indicates a small effect (Cohen, 1988). Teams without meta-knowledge were composed more of team members who knew one or both team members than teams with meta-knowledge. As a result, we included familiarity with other team members as a covariate in the mixed ANOVA and mediation analyses.

To check whether the manipulation of meta-knowledge worked, we analyzed the knowledge test of the symbols learned during the learning phase. As only teams with meta-knowledge were asked questions about their own and other team members’ symbols and teams without meta-knowledge only questions about own symbols (to avoid learning effects in these teams), teams with meta-knowledge could reach up to 18, whereas teams without meta-knowledge up to eight points. The manipulation check was not intended to show the differences between the conditions in the knowledge test, but rather to show that teams in both conditions learned and internalized the presented symbols equally well. One hundred and sixty-nine team members with meta-knowledge (92.4% of participants) received eleven or more points in the knowledge test (i.e., 7.7% of participants received between eight and ten points), indicating that they learned their own and other team members’ symbols correctly and that they transferred this knowledge to long-term memory. One hundred and seventy-four team members without meta-knowledge (98.2% of participants) received six or more points, indicating that they learned their own symbols correctly and transferred this knowledge to long-term memory. Thus, the knowledge tests showed that almost all teams in both conditions learned the symbols correctly.

As there was no homogeneity of the error variances for expert seeking and mistakes in requesting experts assessed by Levene’s test (p < 0.05), we performed the Box-Cox power transformation for these dependent variables at T1, T2, and T3 (Hemmerich, 2016). Table 1 represents descriptive statistics and intercorrelations of all variables.

Table 1 Descriptive statistics and intercorrelations of all variables on team level
Full size table

Hypotheses Testing

We present the results ordered by the dependent variables, respectively the results of the mixed ANOVA, starting with expert seeking, followed by mistakes in requesting experts, team performance, and results of the mediation analyses.

Expert Seeking

The mixed ANOVA for expert seeking (Fig. 5) showed homogeneity of covariance, as assessed by Box’s test (p = 0.399). The Mauchly test indicated that the assumption of sphericity was violated, χ2(2) = 25.02, p < 0.05, so degrees of freedom were corrected using Huynh–Feldt estimates of sphericity (ε = 0.81). In line with Hypothesis 1a, there was a significant main effect for meta-knowledge, meaning that teams without meta-knowledge showed more expert seeking than teams with meta-knowledge, F(1, 117) = 11.522, p < 0.001, η2p = 0.090, which indicates a medium to large effect (Cohen, 1988). Familiarity with other team members had no significant effect on expert requesting (p = 0.142). In support of Hypotheses 3a and 5a, there was a significant main effect of experimental phases on expert seeking, Huynh–Feldt F(1.726, 201.892) = 52.773, p < 0.001, η2p = 0.311, which indicates a large effect (Cohen, 1988). Bonferroni-adjusted post hoc analysis revealed significantly (p < 0.001) higher expert seeking in phases 1 and 3 than in phase 2. In support of Hypotheses 4a and 6a, there was a significant interaction between meta-knowledge and experimental phases on expert seeking, Huynh–Feldt F(1.726, 201.892) = 3.317, p = 0.045, η2p = 0.028, which indicates a small to medium effect (Cohen, 1988). One factorial ANOVA showed that teams with and without meta-knowledge differed in their expert seeking at T1 (F(1, 119) = 13.227, p < 0.001) and at T3 (F(1, 119) = 10.534, p = 0.002), but not at T2. That is, the decrease from T1 to T2 and the increase from T2 to T3 in expert seeking were stronger for teams without meta-knowledge than with meta-knowledge. The interaction between familiarity with other team members and experimental phases on expert seeking became not significant (p = 0.138).

Fig. 5
figure 5

Interaction of meta-knowledge and experimental phases on expert seeking. Note. The covariates in the model are calculated using the following values: familiarity with other team members = 1.9250

Full size image

Mistakes in Requesting Experts

The mixed ANOVA for mistakes in requesting experts (Fig. 6) showed that the assumption of sphericity was not violated, as assessed by Mauchly test (p > 0.05). In line with Hypothesis 1b, there was a significant main effect for meta-knowledge, meaning that teams without meta-knowledge made more mistakes in requesting experts than teams with meta-knowledge, F(1, 117) = 14.339, p > 0.001, η2p = 0.109, which indicates a medium to large effect (Cohen, 1988). Familiarity with other team members had no significant effect on mistakes in requesting experts (p = 0.315). In support of Hypothesis 5b, there was a significant main effect of experimental phases on mistakes in requesting experts, F(2, 234) = 95.928, p < 0.001, η2p = 0.451, which indicates a large effect (Cohen, 1988). Bonferroni-adjusted post hoc analysis revealed significantly (p < 0.001) different mistakes in requesting experts in phases 1, 2, and 3, whereas the highest number of mistakes was made in phase 1, followed by phase 3, and then phase 2. In support of Hypothesis 6b, there was a significant interaction between meta-knowledge and experimental phases on mistakes in requesting experts, F(2, 234) = 6.299, p = 0.002, η2p = 0.051, which indicates a small to medium effect (Cohen, 1988). One factorial ANOVA showed that teams with and without meta-knowledge differed in their mistakes in requesting experts at T1 (F(1, 119) = 8.224, p = 0.005) and T3 (F(1, 119) = 10.997, p = 0.001), but not at T2. That is, the increase from T2 to T3 in mistakes in requesting experts was stronger for teams without meta-knowledge than with meta-knowledge. The interaction between familiarity with other team members and experimental phases on mistakes in requesting experts became not significant (p = 0.143).

Fig. 6
figure 6

Interaction of meta-knowledge and experimental phases on mistakes in requesting experts. Note. The covariates in the model are calculated using the following values: familiarity with other team members = 1.9250

Full size image

Team Performance

The mixed ANOVA for team performance (Fig. 7) showed homogeneity of covariance, as assessed by Box’s test (p > 0.05). The Mauchly test indicated that the assumption of sphericity was not violated (p > 0.05). There was homogeneity of the error variances, as assessed by Levene’s test (p > 0.05). In line with Hypothesis 1c, there was a significant main effect for meta-knowledge, meaning that teams without meta-knowledge showed lower team performance than teams with meta-knowledge, F(1, 117) = 9.975, p = 0.002, η2p = 0.079, which indicates a medium to large effect (Cohen, 1988). Familiarity with other team members had a significant effect on team performance, F(1, 117) = 7.350, p = 0.008, η2p = 0.059, indicating a medium effect (Cohen, 1988). In support of Hypotheses 3b and 5c, there was a significant main effect of experimental phases on team performance, F(2, 234) = 78.982, p < 0.001, η2p = 0.403, which indicates a large effect (Cohen, 1988). Bonferroni-adjusted post hoc analysis revealed significantly (p < 0.001) lower team performance in phases 1 and 3 than in phase 2. In support of Hypotheses 4b and 6c, there was a significant interaction between meta-knowledge and experimental phases on team performance, F(2, 234) = 4.705, p = 0.010, η2p = 0.039, which indicates a small to medium effect (Cohen, 1988). One factorial ANOVA showed that teams with and without meta-knowledge differed in their team performance at T1 (F(1, 119) = 6.921, p = 0.010) and at T3 (F(1, 119) = 13.244, p < 0.001), but not at T2. That is, the increase from T1 to T2 and the decrease from T2 to T3 in team performance were stronger for teams without meta-knowledge than with meta-knowledge. The interaction between familiarity with other team members and experimental phases on team performance became not significant (p = 0.281).

Fig. 7
figure 7

Interaction of meta-knowledge and experimental phases on team performance. Note. The covariates in the model are calculated using the following values: familiarity with other team members = 1.9250

Full size image

Mediation

Regression analysis for T1 showed a significant effect of meta-knowledge on expert seeking (β =  − 0.649, p < 0.001), mistakes in requesting experts (β =  − 0.647, p = 0.001), and team performance (β = 0.580, p = 0.005). Familiarity with other team members had no significant effect on expert seeking (p = 0.598) or mistakes in requesting experts (p = 0.096) but a significant effect on team performance (β = 0.191, p = 0.049). Contrary to Hypotheses 2a and b, there were no significant direct effects of expert seeking and mistakes in requesting experts on team performance. Furthermore, no indirect effect became significant.

Regression analysis for T3 showed a significant effect of meta-knowledge on expert seeking (β =  − 0.631, p < 0.001), mistakes in requesting experts (β =  − 0.553, p < 0.001), and team performance (β = 0.576, p = 0.002). Familiarity with other team members had no significant effect on expert seeking (p = 0.057) or mistakes in requesting experts (p > 0.895) but a significant effect on team performance (β = 0.274, p = 0.001). There was a significant direct effect of mistakes in requesting experts on team performance (β =  − 0.290, p = 0.002). The indirect effect of meta-knowledge on team performance via mistakes in requesting experts became significant (B = 13.340, SE = 5.072, 95% CI [3.920; 23.901], partially standardized β = 0.160). Thus, Hypothesis 2b is supported for T3.

Although teams with meta-knowledge showed less expert seeking than teams without meta-knowledge, this behavior does not seem to be the reason for the differences in team performance, as we could not find the mediation we expected in Hypothesis 2a. The mediating role of mistakes in requesting experts regarding the effect of meta-knowledge on team performance hypothesized in Hypothesis 2b is partially supported. Unlike expert seeking, mistakes in requesting experts have affected the speed with which teams handled emergencies, resulting in reduced team performance.

Discussion

The aim of this paper was to investigate the effects of meta-knowledge on expert seeking, mistakes in requesting experts, and (adaptive) team performance by comparing manual or automated agent-based team communication. We combined research findings on transactive memory system (e.g., Hollingshead et al., 2012), meta-knowledge (e.g., Wegner, 1995), and human-automation teaming (e.g., Fiore & Wiltshire, 2016) and transfer these to the context of interprofessional emergency response teams. In a laboratory setting, student teams played a computer-based control-center simulation. We manipulated meta-knowledge and the automation of team communication through a software agent by manual or automated team communication. While in the phase of manual team communication, teams with and without meta-knowledge differed in their performance, in the phase of automated team communication, no differences between teams with and without meta-knowledge were found. The stronger decrease in team performance from automated to manual team communication implies a worse team adaptation to the changed form of team communication in teams without meta-knowledge than in teams with meta-knowledge.

The result that teams with meta-knowledge perform better than teams without meta-knowledge under manual team communication is in line with previous research on the importance of meta-knowledge (Ellwart et al., 2014; Faraj & Sproull, 2000). However, we expand these findings of previous research by adding team communication processes of expert seeking and mistakes in requesting experts as important consequences of meta-knowledge. We included familiarity of other team members as a covariate, which had a positive association with team performance, indicating that the more team members know each other, the higher their team performance. This matches empirical research in teams, which shows that member familiarity has a positive influence on information elaboration, team communication, collaboration, and performance (e.g., Harrison et al., 2003; Janssen et al., 2009; Maynard et al., 2019). Thus, our findings that teams with meta-knowledge perform and adapt better than teams without meta-knowledge is conservative, as the effect is reduced by the higher familiarity of other team members in teams without meta-knowledge. These results underline the importance of meta-knowledge for successful team adaptation to changing conditions in the working environment of teams (cf. Burke et al., 2006), such as switching levels of automation. Thus, meta-knowledge of “who knows what” enables human-automation-teams to adapt to changing situational demands.

Theoretical Implications

As our study combines research on transactive memory system (e.g., Hollingshead et al., 2012; Wegner, 1995) and human-automation teaming (e.g., Fiore & Wiltshire, 2016), we can draw implications in these areas. As our mediations were only partially supported, other mediators in the relationship between meta-knowledge and team performance, such as the effort for automation and retrieval of meta-knowledge from long-term memory or individual cognitive load, could exist (cf. Hollingshead et al., 2012).

It is possible that the retrieval of meta-knowledge from long-term memory (i.e., think about or remember of meta-knowledge) might have taken the same amount of time as retrieving meta-knowledge from knowledge repositories, particularly when the meta-knowledge is newly acquired before teamwork. Sweller et al., (2011) explain that “newly acquired schemas must be processed consciously and sometimes with considerable effort” (p. 23). Therefore, teams that learned meta-knowledge just in advance have a schema but using this meta-knowledge during teamwork still takes time and deliberate information retrieval. Although the differences in expert seeking with manual team communication between teams with and without meta-knowledge were significant, they may not have affected the speed with which teams handled emergencies, as using the newly acquired schema and seeking for experts in the digital knowledge repository may have taken the same amount of time (cf. Sweller et al., 2011). However, theories of transactive memory system and meta-knowledge do not consider the time it takes for newly acquired meta-knowledge to become routine or for a transactive memory system to be developed in such a way that it does not require additional effort to use that knowledge and system efficiently (e.g., Hollingshead et al., 2012; Kennedy & McComb, 2010). Previous research on meta-knowledge has investigated whether team members update their meta-knowledge after a task change and that this update is beneficial for post-change team performance and team adaptation (Uitdewilligen et al., 2013). These theoretical assumptions and the empirical research, however, only describe how a transactive memory system and meta-knowledge develop, but time is not specified. Theories on transactive memory system should incorporate the temporal dynamics of meta-knowledge acquirement and transactive memory system development to make assumptions about the point at which meta-knowledge becomes more advantageous compared to using knowledge repositories.

Another explanation as to why teams with meta-knowledge performed better than teams without meta-knowledge is the amount of individual cognitive load for their remaining task (cf. Hollingshead et al., 2012). Hollingshead et al., (2012) have assumed that team members holding meta-knowledge benefit from a transactive memory system via a reduced cognitive workload. While empirical research has shown that information overload acts as a mediator (Whelan & Teigland, 2013), cognitive load has not been empirically investigated as a mediator between meta-knowledge and team performance until now. Teams with meta-knowledge have less cognitive load due to the learned schema (cf. Chi et al., 1982) and less workload due to less transaction-related activities (cf. Kirschner et al., 2018; Sweller et al., 2011). Thus, members of teams with meta-knowledge have more cognitive capacity available to focus on their remaining tasks. As a result, they are more effective and efficient. Thus, it might be worthwhile to examine whether team members’ cognitive capacity during a task acts as a potential mediator between meta-knowledge and team performance.

Furthermore, our results imply that in the early stage of teamwork, other variables might mediate the relationship between meta-knowledge and team performance than in the later stages of teamwork. Thus, theories on transactive memory system should consider different mediators in the relationship between meta-knowledge and team performance during different teamwork phases.

Our results provide theoretical implications for handling incomplete transactive memory system or incorrect meta-knowledge. Until now, research has focused on the positive effects of complete transactive memory system and correct meta-knowledge on team outcomes (e.g., DeChurch & Mesmer-Magnus, 2010). Strategies for teams to handle an incomplete transactive memory system or incorrect meta-knowledge have rarely been empirically investigated. What should teams do when they realize that their transactive memory system is incomplete, or their meta-knowledge is incorrect? Our results show that team members validated their meta-knowledge when they realized in advance that they made mistakes in using the transactive memory system correctly. To be more precise, mistakes in requesting experts led to increased expert seeking, which is reflected in the significant correlation between mistakes in requesting experts in the first teamwork phase and expert seeking in the second teamwork phase. That is, team members noticed their mistakes when requesting experts (i.e., their incorrect use of their transactive memory system) and subsequently adapted their behavior and engaged more in expert seeking. This adaptation was successful, represented by the significant decrease in mistakes in requesting experts from the first to the third teamwork phase, but only for teams without meta-knowledge. Su (2012) also provided evidence that the accuracy in expertise recognition increases the longer people use digital knowledge repositories. Su (2012) explained this result as individuals learn and internalize information about other team members’ expertise and build their meta-knowledge when using the digital knowledge repository. Thus, theories on transactive memory system development should incorporate strategies to handle incomplete transactive memory system or incorrect meta-knowledge.

Regarding the research of human-automation teaming, the negative and positive consequences of automated team communication on, for example, team members’ awareness or development of individual knowledge, skills, and abilities (KSAs), can be discussed. Because team communication is a means of establishing social presence and activity awareness about other team members (Haines, 2021), automated team communication could reduce the mutual awareness among team members. While social presence awareness describes the feeling of being connected to other team members (Short et al., 1976), activity awareness describes whether team members are aware of others’ activities (George, 1992). If team communication is automated, team members are not able to perceive other team members and their actions, thereby reducing this awareness (Endsley, 2017). While reduced social presence and activity awareness due to technology-mediated communication in virtual teams has been shown (Haines, 2021), the investigation of the effect of agent-based automated team communication on social presence and activity awareness is still missing. Thus, research combining transactive memory system and human-automation teaming should not only examine the positive but also the possible negative effects of integrating a software agent as an autonomous team member in a transactive memory system. Furthermore, research may focus on how reduced awareness among team members could be compensated.

In addition to potential negative effects of automated team communication, our results suggest other positive effects besides improved team performance. Convergence of expert seeking between teams with and without meta-knowledge with subsequent manual team communication shows that teams without meta-knowledge benefited more from implementing a software agent taking over team communication in the long term. This result shows that automatization of specific subtasks (i.e., team communication) has similar consequences as specialization due to a transactive memory system in a team (Wegner, 1987), as both enable team members to focus on their remaining tasks. Thus, theories and future research on human-automation teaming should, such as research on transactive memory system (e.g., Wegner, 1987), consider analyzing the effect of automated subtasks over time on learning effects, such as the development of expertise.

Practical Implications

As our study utilizes an experimental design, only cautious practical implications can be derived. The results emphasize the importance of internalized knowledge of “who knows what.” Expert seeking in knowledge repositories providing meta-knowledge is influenced by specific factors, which could inhibit expert seeking (cf. Su & Contractor, 2011). Therefore, the use of learned meta-knowledge is preferable because it is likely to be less inhibited since there is no known capacity limit of long-term memory (Sweller et al., 2011). Thus, to be optimally prepared for an emergency, interprofessional emergency response teams should receive enough time to get to know each other (e.g., their expertise) in advance (Ford & Schmidt, 2000). For example, in newly formed interprofessional emergency response teams, meetings with a focus on expertise and differences in skills could be helpful for team members to learn and internalize the knowledge of “who knows what” in the team (cf. Müller & Antoni, 2022).

Results of the study by Zambrano et al., (2019) support that an ongoing team should not be changed. If team members have worked together as a team before, they benefit more from a transactive memory system because they have already been able to learn meta-knowledge. Thus, coordinators of interprofessional emergency response teams or ad hoc action teams should try to assemble team members that have already worked together in previous emergency situations. If this is not possible and an interprofessional emergency response team changes during task accomplishment and there is no time to get to know each other, knowledge repositories of meta-knowledge provided in technologies are helpful to improve teamwork. Therefore, team members should be explicitly encouraged to seek experts in knowledge repositories (cf. Su & Contractor, 2011).

A further implication addresses the implementation of a software agent to automate individual subtasks or team processes. The automatization of team roles (i.e., human-automation teams) may be useful, as this can lead to team members focusing on their remaining tasks (e.g., extinguish fire, care for the injured, cordon off the scene). This can, in turn, improve learning and team performance. However, if the software agent becomes an autonomous team member included in the transactive memory system, the software agent should also learn and have correct meta-knowledge about its team members. Research has already emphasized that it is important that human team members should hold a correct mental model about a software agent for successful human-automation teaming (Schelble et al., 2022) but the opposite direction (i.e., a software agent must have a correct mental model about its human team members) has not been considered yet.

Our findings show that the automation of critical communication processes for coordination via meta-knowledge is transferable to a software agent. This becomes especially compelling when large teams are required to develop meta-knowledge (cf. Fiore & Wiltshire, 2016; Nevo et al., 2012). While further research should investigate how much time and costs are saved by such an implementation, for large teams, it may make sense to offload coordination processes that involve meta-knowledge about other team members to an agent to reduce mistakes and increase performance. This may also apply to large project teams or organizations, as they are often faced with inefficient communication and information-sharing processes (Steinheider et al., 2004).

Limitations and Future Research

When interpreting the results of this study, some limitations must be discussed, which stimulate future research. Our laboratory design allows for causal interpretation, but results require external validation. The limited external validation and generalizability is based on four aspects. First, although the FCI was developed based on documents from real control-centers, it is still an experimental platform to simulate the real context. The experimental platform, however, enabled us to collect enough data to analyze our hypotheses empirically. Second, our results are based on ad hoc student teams and not on real interprofessional teams in emergency response control-centers. Third, interprofessional teams in emergency response control-centers may not be comparable to other types of teams and settings, such as teams in nuclear power plant control-centers or project teams in companies. Fourth, the teamwork lasted only 30 min in total (3 × 10 min), which limits the generalizability to longer collaboration. There are also ad hoc action teams in field settings that meet only for a very short period and disband when the emergency has been successfully handled. However, with this study design, long-term effects of meta-knowledge and different levels of automation on team communication and performance could not be investigated. Furthermore, the learned meta-knowledge could not have been transferred to the long-term memory or could have been too low in its complexity compared to real-life meta-knowledge so that learning the meta-knowledge in advance does not have that big of an advantage compared to using knowledge repositories. Thus, the external validity and generalizability of our results has yet to be tested. Future field studies using (quasi-)experimental designs or longitudinal studies with real teams in emergency response control-centers as well as other types of teams with and without meta-knowledge that work together with software agents as a team or in whose teamwork specific subtasks are being automatized would be helpful to generalize our findings.

Furthermore, no control group was surveyed (i.e., without learning meta-knowledge beforehand and no automation by a software agent throughout the experiment). Therefore, we cannot analyze learning effects or fatigue symptoms of playing the control-center simulation. A control group (without meta-knowledge and with manual control of team communication in all three teamwork phases) could have shown whether participants became tired of playing the simulation, how fast they learned playing the FCI, and how fast they learned meta-knowledge during the experimental phases.

We manipulated the level of automation only by two levels (i.e., manual control and automation). According to O’Neill et al., (2022), who specified ten levels of automation, the level of automation in future experiments could be more differentiated. It would be interesting how other levels of automation (e.g., level of partial agent autonomy in which the agent executes a suggestion if the human has approved it) affect team performance and adaptability. Furthermore, the software agent used in our study worked flawless, trustworthy, and fast, so that participants could rely on it without double-checking its actions in the phase with automated team communication. However, in complex environments, it is possible that a software agent makes mistakes or does not run reliably. In such situations, it would be interesting to investigate whether human team members will engage in expert seeking to validate their meta-knowledge just for cases if the software agent makes mistakes or crashes and the human team members must take over subtasks again.

Finally, we have not investigated how other factors, such as satisfaction with work, frustration, perceived threat due to the software agent, cognitive overload, agent agency, or trust in the software agent, could have been influenced by the manipulation of meta-knowledge and level of automation of team communication. It might be interesting for future research to investigate the effects of meta-knowledge and level of automation of team communication by a software agent on other factors than on team performance and adaptability.

Conclusion

This study contributes to existing research on transactive memory system and human-automation teaming by showing the positive effect of meta-knowledge on expert seeking, mistakes in requesting experts, team performance, and adaptability. Results suggest that meta-knowledge improves performance and the ability to adapt to switching levels of automation of team communication, underlining its importance for flexible applications of team support systems. However, a software agent as an autonomous team member taking over the subtasks of team communication can compensate for the lack of meta-knowledge and its negative consequences. Future longitudinal field studies with real interprofessional emergency response teams using a more detailed manipulation of meta-knowledge and level of automation might be promising.