1 Introduction

Artificial Intelligence (AI) is reshaping our daily experiences and business landscapes. AI-driven technologies, currently at the core of contemporary lifestyles, are poised to fortify their future relevance. The escalating interest in AI’s role in education and training is set to drive investment in these technologies to an impressive $87 billion by 2030, marking a 13.6% annual growth (Grand View Research, 2023). AI promises enhanced personalization in training, automated management of recurring tasks, and heightened the effectiveness of instruction protocols (Diederich et al., 2022). For example, AI-driven virtual tutors and assistants can deliver tailored instruction to learners, while AI-integrated learning management systems leverage student data for personalized, instant feedback. Moreover, by streamlining routine administrative duties like grade tracking and class scheduling, AI empowers educators to concentrate on more essential instructional tasks.

The rapid evolution of AI technology, exemplified by OpenAI’s AI-based conversational agent (CA), ChatGPT, has ignited intriguing discourse about AI’s impending impact on higher education and training (Tate et al., 2023). These conversational AIs, designed to mimic human interaction using natural language processing, have become compelling facets of user interfaces. ChatGPT engages with users in natural language and generates human-like responses, and benefits wide applications like text summarization, question-answering, and language translation. This paradigm shift has induced a reevaluation of the relevance of many professions, particularly within higher education, thus prompting conversations around technology’s evolving role. However, the emulation of human-to-human interactions of such AI-driven systems would enhance human-AI cooperation within training environments (van den Bosch et al., 2019; Gmeiner et al., 2022). Hence, investigating such collaborative ventures in educational programs could catalyze effective learning.

The convergence of AI and team collaboration, particularly the inclusion of Artificial teammates, is a burgeoning area of research (Schelble et al., 2022). This fresh approach explores AI’s function in modern teamwork (O’Neill et al., 2022) and cooperation between humans and AI agents (Zhang et al., 2021). Our focus narrows down to the potential of AI, particularly OpenAI’s ChatGPT, in bolstering team learning in virtual settings. With an escalating reliance on virtual teams for functions like training and education, numerous challenges surface. Technology-facilitated virtual interaction, while convenient, curtails co-learning opportunities (He & Huang, 2017) and can heighten negative emotions (Hilliard et al., 2020). The cultural and spatial diversity inherent in virtual teams may also breed disparate expectations for online learning (Nordbäck & Espinosa, 2019). In such a scenario, AI-enabled systems like ChatGPT could play an instrumental role as human-like team member, offering guidance and real-time feedback to navigate the uncertainties and hurdles virtual teams face. Consequently, AI’s role in enhancing virtual team performance is seemingly indispensable.

However, early research has barely probed the potential of AI in training scenarios, with divergent views on its educational impact. Van den Bosch et al. (2019) and Gmeiner et al. (2022) adopt a macro view of AI’s emergent role in team learning. Tate et al. (2023) see promise in ChatGPT for educational purposes, but their hypothesis invites empirical examinations. Endsley (2023) further suggests that user perceptions of AI can impede its performance. Accordingly, this research investigates how ChatGPT can support project-based team learning. Our work makes four significant contributions. Utilizing research on human-agent teams (e.g., Lyons et al., 2023) and virtual project-based team learning (e.g., Darban, 2022a) to, firstly, develop a framework rooted in anthropomorphism — the attribution of human emotions, qualities, and behaviors to non-human objects (Epley et al., 2007). Unlike other interfaces, AI-enabled systems can produce an immanent degree of anthropomorphism due to their ability to emulate human-like properties (Seeger et al., 2018), which in turn can promote the AI’s role within teams (Cohen et al., 2021). Hence, research on AI’s perceptions and usage rooted in anthropomorphism can provide insights into the successful application of AI systems in teamwork.

Second, in human-AI interaction, humans’ perception of AI attributes can play an important role in the interaction outcomes (O’Neill et al., 2022). Using insights from AI technology and team learning studies to explore this lesser-known domain, we analyzed a range of features for AI-enhanced language models (specifically ChatGPT), concentrating on aspects of AI autonomy and explainability from a learner’s perspective. The third contribution stems from a multilevel approach. While current research on human-agent teams predominantly features single-level mechanisms, intra-team dynamics can also affect an individual’s learning journey through different group-level factors (Darban, 2022b). Therefore, drawing from recent studies that underscore the significance of virtuality perceptions in project-based teams (Costa & Handke, 2023; Handke et al., 2022), we synthesize individual and team-level characteristics to create a multilevel model, examining the interplay between AI attributes and learner’s perceptions. Specifically, the study conceptualizes the perceived virtuality at the team level (i.e., aggregated virtuality perceptions), and examined how its interplay with AI attributes impacts learners’ perceptions. Fourth, research, building upon the sequential updating mechanism, found that learner’s perception of knowledge evolves during various training stages and can be updated based on prior perceptions, a process termed perceived knowledge update (Darban et al., 2016). Hence, to contextualize AI in a learning setting, we examined how the abovementioned triggers impact perceived knowledge acquisition during the learning journey, specifically from pre- to post-training. Hence, we applied a longitudinal design to address the resulting research questions:

  • RQ1: How is the development of knowledge update in AI-assisted teams?

  • RQ2: What are the antecedents of team members’ perceived knowledge update and learning intention in virtual student teams?

  • RQ3: How do the team’s perceived virtuality and its interactions with the tested AI attributes impact the learner’s knowledge update?

2 Literature review

2.1 Virtual team learning: Human-AI teamwork

Prior research posits that successful team learning necessitates more than mere team formation—it is incumbent on the effective utilization of contemporary technologies to bolster team learning performance. For instance, with the advent of the internet and personal computing technologies in the 1990s, virtual teams emerged as a novel paradigm for training. This shift, however, introduced new challenges for educators and researchers. Notably, virtual team dynamics could lead to the loss of nonverbal cues, which demanded improved communication and coordination skills (Nordbäck & Espinosa, 2019) and effective organization of the learning setting (Hilliard et al., 2020). However, Virtual teams and remote work have become instrumental in the last twenty years, especially during events like the COVID-19 pandemic.

In parallel with these developments, AI technology has made significant strides in democratization and ubiquity, earning its status as a valuable team asset. Moving beyond its initial role as a mere support tool, AI now moves toward evolving to become an integrated group member (Hauptman et al., 2023). The latest studies have highlighted the potential of human-AI teamwork in amplifying results across various fields, like education and team learning (Sukhwal et al., 2023). For example, the adoption of CAs may foster group learning by enhancing knowledge transfer, improving team interaction (Ahmad et al., 2020), and even serving as a group moderator to identify members’ feelings (Benke et al., 2022).

This synergy between humans and AI has ushered in a new category of teams known as Human-AI Teams (HATs). These teams, consisting of at least one human and one AI member, represent a unique approach to teamwork (McNeese et al., 2021). In a virtual training scenario, AI can now assume some functions of a full-fledged team member, demonstrating logical reasoning and task performance on par with humans, which catalyzes co-learning and engenders shared mental models, thus facilitating team learning (Khakurel & Blomqvist, 2022). Importantly, recent studies underscore that AI agents can enhance the competencies of their human counterparts by extending the knowledge repository of teams (Seeber et al., 2020).

2.2 Anthropomorphism in HAT

Unpacking the factors that modulate user acceptance of AI systems, such as CA, is critical to overcome the existing reservations and hesitations surrounding their usage. Determining potential catalysts and obstacles to AI and CA utilization holds substantial implications for their practical applications and provides valuable insights for future research. One pivotal element in this context is anthropomorphism, the attribution of human-like attributes to non-human entities, which is a key feature of CA technology that can be leveraged to enhance user acceptance. Incorporating humanlike qualities and traits in the design of CA technology has increased familiarity and satisfaction with the technology, resulting in a more personalized user experience (Cohen et al., 2021). Indeed, ascribing human-like attributes, emotions, or intentions to AI tools is integral to AI development and deployment (Seeger et al., 2018).

Zitzewitz et al. (2013) conceptualized anthropomorphism as a multi-faceted construct, incorporating a diverse set of parameters that fall into static (appearance) and dynamic (behavior) categories. While the appearance category encompasses visual and auditory traits, the behavior category encapsulates movement, interaction, social behavior, and both verbal and non-verbal communication. For CAs like ChatGPT, designed to mimic human-like interaction, the dynamic facets enhancing anthropomorphism are predominantly tied to their linguistic competencies. These include generating natural, contextually relevant responses and employing conversational strategies like humor, empathy, and politeness. Moreover, implementing personalized greetings, responses, and suggestions—reflective of user preferences and past interactions—also enhances the anthropomorphic traits of language models. Overall, these dynamic characteristics contribute to the perceived humanness of language models and promote their seamless integration into human–machine interactions.

2.3 A research frontier: Anthropomorphism in virtual HATs

Research on virtual team learning has evolved alongside the rise of human-AI collaboration, with a focus on integrating artificial intelligence into team dynamics. This evolution, sparked by digital technologies, demands a better grasp of how teams communicate and organize, especially given the absence of nonverbal cues (Hilliard et al., 2020; Nordbäck & Espinosa, 2019). Concurrently, AI’s progression from a support mechanism to an integral team member has been pivotal (Hauptman et al., 2023), with emerging studies illustrating the profound impact of HATs on team learning outcomes (Darban, in press; Sukhwal et al., 2023; McNeese et al., 2021). The anthropomorphic design of CAs like ChatGPT, accentuating human-like interaction, represents a significant advancement in this domain (Cohen et al., 2021; Seeger et al., 2018). However, despite these advancements, the literature lacks a clear understanding how anthropomorphism in CAs can influence team learning in virtual settings, specifically in project-based environments. This gap is particularly salient in the context of leveraging AI to facilitate co-learning and engender shared mental models within diverse and dispersed teams. Our research seeks to fill this void by examining how the anthropomorphic features of CAs influence virtual team learning, thereby elucidating the potential of AI to enhance the competencies and knowledge repositories of human teams.

3 Research model and hypotheses

3.1 AI explainability

As mentioned above, anthropomorphism refers to the propensity to attribute human traits to non-human entities. Anthropomorphism typically refers to the tendency to ascribe human traits to non-human entities. One byproduct of this inclination is the expectation for the actions of agents, such as AI-enabled tools like CAs, to be explainable. AI explainability concerns the system’s ability to render its underlying logic and decision-making processes comprehensible to humans. When encountering a complex intelligent system, users may seek various types of explanatory information, each necessitating its distinct design (Mohseni et al., 2021). Hence, explainability is fundamentally a human-centric attribute, with human perceptions of AI explanations playing a critical role in optimizing and designing AI-enabled tools (Vaughan & Wallach, 2021). While often seen through an instrumental lens (Colaner, 2022), explainability is also intrinsically valuable in assuring users of the trustworthiness, fairness, and accuracy of the AI outputs (Lyons et al., 2023; Wulff & Finnestrand, 2023). Given the lack of a universal definition of what constitutes a good explanation, it is the perceptual alignment between a person’s reasoning and AI explanations that defines the machine’s explainability.

Research on AI explainability has mostly concentrated on technology and algorithmic transparency, exploring how AI can explain its reasoning (Wulff & Finnestrand, 2023). For example, studies in the field of machine learning aim to design new interpretable models and provide explanations for 'black-box' models with ad-hoc explainers (Miller, 2019). With a similar goal but a different approach, visual analytics researchers design tools and methodologies for data and domain experts to visualize and manipulate complex ‘black-box’ models. Yet, given the recipient-dependent nature of AI, AI explainability research should also embrace a human-centered perspective. This is because the interaction between the machine and the user fundamentally shapes the user’s perceptions and the performance of the AI system (Jin et al., 2023). People derive their understanding and perception of the system’s reliability and capabilities based on how the system explains the logic and data sources underpinning its outputs (Endsley, 2023). As individuals interpret technology characteristics differently, they may form diverse perceptions of AI characteristics. Therefore, addressing end-user needs, such as understanding machine-generated explanations, would benefit the successful application of AI.

3.1.1 AI explainability and learning

AI’s impact on education is profound and expanding. Previous studies have primarily investigated the implementation of AI in educational contexts from educators’ standpoint, often overlooking the learners’ perspective. However, recent research posits that AI explainability facilitates person’s AI usage and benefits human-AI task performance (Jin et al., 2023). As such, explainable AI (XAI) is geared toward enhancing individual-AI interactions and promoting a reliable rapport between the agents. XAI, by fostering the metacognitive processes of planning and self-reflection, affords learners the agency in their learning, potentially augmenting learning outcomes (Bull, 2020).

Recent advancements in AI, for instance, CAs like Chat GPT, are prompting a shift in how we view AI explainability. It is no longer merely a product feature but a process that facilitates human-AI interaction and knowledge transfer between the explainer and the explained (Ehsan et al., 2022; Miller, 2019). In team environments, multiple studies highlight the role of AI explanations in improving HAT performance by enhancing awareness about AI’s capabilities and limitations (Paleja et al., 2021). Particularly in learning teams, the existence of shared mental models about AI, or a uniform understanding of AI functioning across members, aids in conveying information without the need for intricate verbal communication, a known communication constraint of virtual teams (Endsley, 2023).

Thus, we assert that developing appropriate levels of AI explainability is key to optimizing learning outcomes and updating person’s knowledge across training stages, reflected by the difference between perceived knowledge before and after training (Darban et al., 2016). Accordingly, we hypothesize:

  • H1: Perceived AI explainability will have a positive association with knowledge update.

3.2 AI autonomy

HATs are typically differentiated from mere automation due to their enhanced independence, proactive behavior, and self-regulation (O’Neill et al., 2022). AI teammates, to fulfill a team role and function in intricate scenarios, are known to require a relatively high level of autonomy. The embedded autonomy of these AI teammates, determined by their design, presents a unique challenge to teams. In reality, the creators dictate the extent of autonomy in autonomous agents, and this might be perceived differently by the users (Hauptman et al., 2023).

AI’s autonomy, as demonstrated by systems like ChatGPT, encapsulates its capacity to operate, learn, interpret, and make decisions independently without continuous human supervision (O’Neill et al., 2022). Such proactive and responsive features fuel AI’s anthropomorphization, reinforcing its perception as a human-like entity (Wagner & Schramm-Klein, 2019). Studies show that an AI's competency and autonomy primarily shape a human teammate’s perception (Zhang et al., 2021), with higher levels of AI autonomy boosting the intent to adopt such technologies (Chao et al., 2016).

In essence, human–AI teaming pivots on the autonomous teammates’ robust self-governance, allowing them to act with independence and agency (O’Neill et al., 2022). Indeed, perceived agency, reflecting the degree of perceived autonomy in a system (O’Neill et al., 2022), is instrumental in shifting one’s perspective of technology from a mere tool to an autonomous teammate (Hauptman et al., 2023; Tokadlı & Dorneich, 2022). Consequently, to function effectively within HATs, AI agents should possess the capabilities and authority to adapt and respond to novel challenges in the task environment.

3.2.1 AI autonomy and learning

AI-grounded tools afford opportunities for team learning. The autonomous attribute of an AI-enabled tool can help identify diverse perspectives in evaluating and performing a task and improve the divergence and creativity of the output in a team learning context (Markauskaite et al., 2022). The perception of independency of AI teammate in HAT activities supports the democratic and unbiased participation of ideas, encouraging knowledge sharing (Hellwig & Maier, 2023). In fact, AI tools offer objective analysis, free from the inherent biases and preconceptions that humans might have, fostering knowledge sharing and team-based learning, which in turn can facilitate creative thinking, shared memory among various team members and peer learning (Jarrahi et al., 2023). Moreover, CA tools like ChatGPT, equipped with machine learning capabilities, can autonomously refine their understanding through continuous learning from interactions with human teammates. This iterative self-learning mechanism allows AI to contribute meaningfully to creative problem-solving in teams and assist in the discovery of unseen patterns and ideas.

Furthermore, AI tools, due to their inherent autonomy, can adapt to varying contexts, concepts, and meanings, providing learners with timely and pertinent information. This flexibility enables a dynamic learning experience, a crucial attribute in virtual team settings (Darban, 2022a). For instance, conversational AI tools can accommodate diverse learning preferences within student teams by being available around-the-clock and adapting to students’ unique learning needs (Sukhwal et al., 2023). CAs, such as ChatGPT, can autonomously engage students in interactive discussions, offering real-time responses, explanations, and examples based on student input. This autonomous interaction encourages active learning and engagement, stimulates critical thinking, and incites students to explore further, thereby enhancing knowledge transfer and interactive learning within virtual student teams. The impact of such an autonomous capability becomes particularly significant given the complexities of communication within virtual teams (Darban, 2022a). Thus, we hypothesize that:

  • H2: Perceived AI autonomy will have a positive association with knowledge update.

3.3 Team perceived virtuality

Researchers conceptualize virtuality as a spectrum, positing that the integration of communication technologies has characterized teams by the dimensions of virtuality (Brown et al., 2020). Recently, researchers have called for a more nuanced conceptualization of virtuality, one that goes beyond the structural properties and incorporates the experiential and perceptual aspects (e.g., Costa & Handke, 2023; Darban, 2022a; Handke et al., 2022). Accordingly, we adopt the definition of team perceived virtuality (TPV) as the collective virtuality experienced across team members, which emerges from shared affective-cognitive processes within the team (Handke et al., 2022).

3.3.1 Team perceived virtuality and HAT learning

Recent research suggests that team members’ perceptions of virtuality affect teamwork processes and group dynamics (Costa & Handke, 2023). In the realm of virtual teams, communication stands as a prominent challenge. The absence of face-to-face interaction and an over-reliance on digital platforms for information exchange, compounded by delays in information exchange and low-capacity communication tools (Nordbäck & Espinosa, 2019), all contribute to higher virtuality (Darban, 2022a). This notion is especially pertinent in HATs, where AI entities contribute to various facets of team tasks and decision-making processes (Sukhwal et al., 2023). One potential advantage of AI team members in virtual teams is their ability to mitigate communication challenges across human team members. For example, thanks to advances in Natural Language Processing (NLP), the responses of CAs and their human-like qualities are improving, positioning them as virtual group members (Diederich et al., 2022). AI tools such as CAs can facilitate information sharing, task coordination, and real-time feedback, enabling efficient communication within virtual teams (Benke et al., 2022).

However, high levels of virtuality can also introduce stress and complications, as they require team members to operate independently (Costa & Handke, 2023). Depending on the degree of perceived virtuality, members may need to engage in self-regulatory activities, like self-monitoring and self-instruction, to independently steer group work (Boekaerts & Corno, 2005). Under such conditions, affiliative coping theory suggests that learners may seek supportive coping mechanisms (Taylor et al., 2000). Viewing an artificial teammate with anthropomorphic qualities can alleviate such stresses and enhance the individual’s learning experience. Similarly, research has shown that as teams become more asynchronous, indicative of higher virtuality, members tend to become more independent in decision-making (Hoch & Dulebohn, 2017), thereby reducing communication-related anxiety (Hilliard et al., 2020) and fostering knowledge acquisition (Darban, 2022a).In such scenarios, human members would see the AI coupled with explainability qualities as a resourceful team member, rather than a tool, which further promotes the learning experience of the person. For instance, in a virtual team, AI explainability, that is AI’s comments and feedback are commonsensical, can mitigate the absence of peer feedback and foster self-regulation (Wulff & Finnestrand, 2023).

Therefore, we argue that in the presence of high TPV, perceiving the AI team member’s contributions are human-comprehensible and understandable (i.e., AI explainability) would improve self-regulatory activities and team processes, leading to enhanced learner performance. Thus:

  • H3: Team perceived virtuality will moderate the association between AI explainability and knowledge update of team members, such that the relationship is stronger for members with higher levels of team perceived virtuality.

Similarly, integrating AI into virtual HATs presents an additional layer of value due to its capacity for independent operation and autonomous contribution (Hellwig & Maier, 2023). Particularly the property of AI that enables it to provide alternative perspectives and real-time feedback (AI autonomy) can have a greater influence on individuals’ learning outcomes in higher virtualized environments, where immediate feedback, response, and peer learning are absent (Hilliard et al., 2020). Put differently, the autonomous operation of AI can potentially mitigate the unique challenges associated with virtual teams, including the need for self-regulation, independence, and coping with communication-related anxieties. Therefore, TPV amplifies the benefits and potential of AI autonomy, enriching students’ collective knowledge and learning experience and enhancing overall knowledge acquisition. Thus, we hypothesize that:

  • H4: Team perceived virtuality will moderate the relationship between AI autonomy and knowledge update of team members, such that the association is stronger for members with higher levels of team perceived virtuality.

3.4 Knowledge update and intention to learn

To ensure a holistic view, this paper incorporates a hypothesis examining the relationship between “knowledge update” and “intention to learn.” Based on the belief and attitude change literature (Angst & Agarwal, 2009), prior IS education research (Darban et al., 2016) and generative AI studies in training (Darban, in press) found evidence for the positive relationship between perceived knowledge update and intention to learn, thus obviating the need for an exhaustive hypothesis formulation. As such:

  • H5: Perceived knowledge update will have a positive influence on intention to learn.

The above hypotheses build the research model for this study, which Fig. 1. represents.

Fig. 1
figure 1

Research model

Full size image

4 Method

4.1 Subjects and procedure

Our sample was MBA students engaged in a Management Information Systems (MIS) class at a Midwestern US public university. These students were involved in a 16-week course that involved a group project. Students were randomly assigned to teams consisting of six to eight members at the start of the academic semester.

The students worked within their groups to study the application and impact of Information Systems (IS) in their selected organizations. This project involved various activities, such as interviewing individuals, applying IS concepts, generating reports at four major milestones, and creating virtual presentations. A crucial aspect of our study was the inclusion of ChatGPT-3.5 (hereafter ChatGPT) into each team. ChatGPT, a machine learning technology, learns via user interaction using unsupervised training, making it an ideal fit for a co-learning team environment. As illustrated in Fig. 2 (refer to Appendix A for more details), teams were able to incorporate ChatGPT into various phases of their group project workflow.

Fig. 2
figure 2

ChatGPT integration into group projects

Full size image

The students were first acquainted with the main features of ChatGPT through an introductory workshop (refer to Appendix B). Teams were encouraged to explore ChatGPT’s capabilities and distribute tasks among all members, ChatGPT included, to enhance team performance.

We gathered data in three phases over the semester. In the first week (Time1), we measured participants’ knowledge perceptions prior to the training, gathered demographic details, and assessed control variables through a survey. In week eight (Time2), we evaluated participants’ perceptions of AI autonomy, explainability, and team-perceived virtuality. Finally, eight weeks later (Time3), we measured participants’ post-training knowledge perceptions and intentions to learn. Hence, we measured perceived knowledge update twice: before the training (Time1) and after the training (Time3). Following the approach Angst and Agarwal (2009) outlined, we calculated the perceived knowledge update by determining the difference between the post-training and pre-training stages.

After excluding incomplete survey responses, our study analyzed data from 344 students (the original number of participants was 368) who were part of 48 project teams and enrolled in four sections of the same course, with females representing 43% of the sample. The average size of teams was seven members, and the average age of the participants was 29.9 years.

Despite the uniformity of course content, instructor, and project specifics across all sections, we employed control variables such as prior team experience and exposure to MIS to assess possible biases related to team size or assignment and account for any potential confounding effects arising from differences in course sections or team sizes. The results revealed no significant variances in the mean values of variables across the four-course sections (prior team experience: F = 0.42, p > 0.05; prior MIS knowledge: F = 0.39, p > 0.05) or across team sizes (prior team experience: F = 0.38, p > 0.05; prior MIS knowledge: F = 0.56, p > 0.05), indicating an absence of bias regarding team size or assignment.

4.2 Measurement

We applied seven-point Likert scales, ranging from 1 (strongly disagree) to 7 (strongly agree), validated in previous research, for all constructs (refer to Appendix C for details). We evaluated TPV using an eleven-item scale adapted from Brown et al. (2020), validated in virtual learning environments (Darban, 2022a). It assesses team members’ perceptions of virtuality and then aggregates them to the team-level, as team virtuality arises from the cumulative attributes of individual-level perceptions (Handke et al., 2022).

Perceived AI autonomy was gauged using a three-item scale borrowed from Hong and Williams (2019), while a five-item scale from Hamm et al. (2023) was utilized to measure perceived AI explainability. We employed Bassellier et al.’s (2003) instrument to determine participants’ perceived knowledge of MIS. The variation between pre- and post-training MIS knowledge represented the perceived knowledge update (Darban et al., 2016). The intention to learn about MIS was ascertained using a three-item scale from Davis et al. (1989).

To ensure our study adequately accounted for both individual and team-level variables that could impact team learning, we included controls for factors such as prior experience with teams and MIS, team gender proportion, and size, gender, and age.

4.3 Data analysis: Multilevel technique

The research model was evaluated using the hierarchical linear modeling (HLM) method, fitted for handling variances across different analytical levels and accounting for non-independent error structures. HLM8.1 is practical for assessing cross-level associations and nested data, generating estimation results utilizing a restricted maximum likelihood algorithm. It has preferred for real-world applications, such as group projects (Darban, 2022b).

We assessed TPV at team-level and justified the aggregation through diagnostic statistics. Firstly, within- and between-group heterogeneity were evaluated; ANOVA results showed significant team differences in perceived virtuality levels (p < 0.001), using team membership as the factor. Secondly, intra-team consistency and inter-team variability were validated by calculating the within-group agreement index (rwg(j)) and intraclass correlation coefficients (ICCs). With an agreement index of rwg(j) = 0.78 and ICC values of ICC(1) = 0.50 and ICC(2) = 0.71, the data aggregation was deemed acceptable (Bliese et al., 2018).

5 Results

5.1 Measurement model

We employed AMOS 29.0 and maximum likelihood estimation to perform a Confirmatory Factor Analysis (CFA) across all latent constructs (Fornell & Larcker, 1981). We analyzed various popular goodness-of-fit indices. The model fit well with the data: χ2/df = 1.73, CFI = 0.95, SRMR = 0.05, TLI = 0.96, RMSEA = 0.05, SRMR = 0.04 (Gefen et al., 2011).

All composite reliability estimates exceeded 0.70 (See Table 1). We reviewed item loadings to assess convergent validity, and all met the suggested 0.707 guidelines (Chin, 1998). The calculated average variance extracted (AVE) values exceeded the 0.50 threshold (Fornell & Larcker, 1981). To establish discriminant validity, we computed the square root of each construct’s AVE, which surpassed the construct’s correlations with other constructs.

Table 1 Correlations and descriptive statistics
Full size table

5.2 Common method bias

We conducted Harman’s single-factor test in SPSS, including all individual and team-level variables and controls for an exploratory factor analysis (Podsakoff & Organ, 1986). The first emerging factor accounted for just 13.2% of the variance, suggesting a minimal risk of common method bias (CMB). However, understanding the potential inadequacies of this approach in measuring CMB, we integrated an unmeasured latent method factor into our CFA model. All self-reported items were loaded onto this method factor and their respective theoretical constructs. The common variance computed was under 14%, and the item loadings were significantly lower on the method factor compared to their respective constructs and were statistically insignificant. In addition, the integration of a common factor did not influence the model fit (i.e., model without common latent factor: χ2/df = 1.33, model with common latent factor: χ2/df = 1.40). These results reaffirmed the minimal concern of CMB in this study.

5.3 Hypothesis testing

Due to the hierarchical nature of our data, with 344 members nested within 48 groups, we used the HLM8.1 package to analyze the multi-level model proposed (i.e., Model 1; H1 to H4) and employed ordinary least squares (OLS) regression to assess the individual-level model (i.e., Model 2; H5). The model accounts for 45% of the variance in knowledge update and 24% in intention to learn. As shown in Table 2 and Fig. 3, AI autonomy did not show a significant impact on knowledge update (H1: β = 0.09, p = 0.17). Perceived AI explainability (β = 0.42, p < 0.001) demonstrated a significant positive impact on knowledge update, supporting H2.

Table 2 Results
Full size table
Fig. 3
figure 3

Results of the research model

Full size image

The study derived Pseudo-R2 from the proportional reduction in error variance attributed to the predictor constructs.

We investigated cross-level interaction effects of PTV on the association between the tested AI characteristics– autonomy (H3) and explainability (H4)– and perceived knowledge update. The results confirmed both moderating effects as significant, aligning with our hypotheses (H3: β = 0.25, p < 0.05; H4: β = 0.19, p < 0.001), and TPV did not have any significant direct effect on the perceived knowledge update.

Figure 4a and b reflect the nature of these interactions. Lastly, knowledge update significantly influenced intention to learn (β = 0.25, p < 0.01).

Fig. 4
figure 4

Moderation effects

Full size image

6 Discussion

The rise of AI, such as CA, in educational contexts has sparked academic curiosity about its influence on team knowledge acquisition (Tate et al., 2023). However, the effectiveness of such teams in remote learning environments, where dynamics differ from face-to-face contexts, is questioned (Hilliard et al., 2020). Unlike typical human-technology interactions, human-AI collaboration presents a complex relationship due to its unique team composition (McNeese et al., 2021; Schelble et al., 2022). The improved AI enables assigning specific roles to it in virtual Human-AI Teams (HATs), transforming it from a mere tool to a quasi-human entity, adding complexity to the learning process.

This study, drawing on HATs and anthropomorphism literature, aims to extend our knowledge of human-AI teaming by outlining its success factors and the role of AI attributes in virtual learning teams. We developed and examined a multi-level and-phased model arguing that in learning HATs, AI autonomy, explainability, and their interplay with TPV impact the knowledge update of learners. The findings provide an insightful perspective on AI-mediated learning in virtual settings, showing that integrating an AI tool like ChatGPT into HATs can enhance knowledge acquisition and learning intentions.

6.1 Theoretical contributions

Our research presents multiple theoretical contributions. First, we advance the team learning literature by intertwining the recent dialogues between human and AI teammates (specifically, ChatGPT) and team performance into project-based learning. As AI’s capabilities and accessibility expand rapidly, so does the range of roles AI can adopt within teams. Despite the essential role of effective technology use in virtual teams for overall performance (Hauptman et al., 2023), the literature has largely overlooked the relationship between HAT and knowledge acquisition (Diederich et al., 2022). This study underscores AI’s potential as team members to accelerate knowledge updating among learners, thereby complementing the traditional roles of human team members, and proposes the empowering role of artificial tools in enhancing learners’ knowledge updates.

Second, our research furthers the nascent exploration into knowledge transfer in human-agent teams by identifying key factors (i.e., TPV and AI characteristics) in team learning. While the importance of artificial team members, particularly in knowledge transfer, is emerging (Sukhwal et al., 2023; Textor et al., 2022), empirical investigations into human-AI collaboration elements and their outcomes are limited (Diederich et al., 2022). Our study underscores the effectiveness of AI explainability (direct effect) and autonomy (coupled with TPV) as AI design features in facilitating knowledge transfer and enhancing team performance, aligning with literature on AI tools like natural language processing and conversational agents (Sukhwal et al., 2023). Interestingly, AI autonomy did not directly impact students’ knowledge acquisition. This finding aligns with the argument of recent scholarship that the direct impact of technical autonomy on knowledge acquisition may be negligible or even counterproductive in some cases, as the independence of AI might reduce human users’ perceived need to understand the underlying process, thereby limiting the depth of their knowledge acquisition (Harris-Watson et al., 2023).

Third, this study extends AI-in-education literature by elucidating AI characteristics’ conditional effect on knowledge acquisition. TPV emerged as a significant team-level moderator of increased knowledge acquisition in a virtual project-based learning setting. Our findings present intriguing insights into TPV’s role in shaping AI autonomy’s impact on knowledge acquisition. While the direct impact of AI autonomy was not found, its interplay with TPV significantly influenced students’ knowledge acquisition. Specifically, in high TVP, AI autonomy significantly contributes to knowledge updates among students. This finding aligns with the argument that the virtual environment stimulates and necessitates a greater reliance on autonomous systems to guide and facilitate learning (Xia et al., 2023). Accordingly, with the increase in perceived virtuality, the need for an autonomous AI system becomes more pronounced, assisting in the process of knowledge acquisition. Conversely, we found that AI autonomy leads to reduced knowledge acquisition when TPV is low. This finding agrees with a preference for human-led learning in less virtual environments, where autonomous AI could be seen as disruptive, inducing cognitive overload or discomfort (Chao et al., 2016).

Furthermore, our finding that AI explainability significantly bolsters knowledge acquisition aligns with prior research suggesting that within a predominantly virtual environment, transparency in AI operations (AI explainability) assumes paramount importance (Endsley, 2023). We extended AI explainability to an online educational setting where students rely on the AI for direction and learning support, necessitating a clear understanding of how the AI makes decisions or recommendations. This clear understanding or explainability encourages trust and engagement (Jin et al., 2023; Wulff & Finnestrand, 2023), which subsequently stimulates knowledge acquisition. The results imply that the elevated virtuality amplifies the need for AI explainability, as it replaces face-to-face clarification typically associated with complex concepts. Thus, the more understandable and interpretable the AI system is, the more effectively it enriches the learning experience in virtual settings.

Fourth, our study recognizes the dynamic nature of knowledge acquisition, enriching research on virtual training and its determinants by emphasizing the perceived knowledge update, as opposed to the self-perception of knowledge. By exploring the association between HAT characteristics and knowledge update, we extend our understanding of the belief updating theory, which illustrates the process of belief alteration through a sequential updating mechanism and recommends stage-based (i.e., pre-training and post-training) learning frameworks (Gupta & Bostrom, 2013). This further expands the applicability of the belief updating theory at the team level (Darban, 2022b).

Also, our multi-level approach addresses often-overlooked team-level facets of learning within virtual HAT settings. Studies into HATs in learning teams are infrequent, with existing literature on HATs primarily focusing on single-level analyses, i.e., individual or dyad levels (e.g., Bach et al., 2022; Textor et al., 2022). Consistent with teamwork technologies research (Darban, 2022a) and human-agent group dynamics (Diederich et al., 2022), virtual team learning studies necessitate investigating the social and interpersonal factors within teams through a multi-level perspective. Through a multi-level approach, novel to HAT research, we identified TVP’s moderating role between AI characteristics and learning outcomes. The results affirm our argument that AI characteristics significantly enhance student learning outcomes in teams with high virtuality perceptions, elucidating perceived virtuality’s crucial moderating role. This notion is thus intuitive that as highly virtual teams are inherently more reliant on digital tools (Darban, 2022a)—the realm where AI operates most effectively. Accordingly, the findings suggest that in a highly virtual environment, students might be inclined to engage more intensively with AI features, maximizing the AI’s potential to facilitate learning, which aligns with the extant literature suggesting that TPV influences technology use and team processes in virtual settings (Gilson et al., 2015).

6.2 Practical contributions

Prior studies have mainly examined the human-AI interface in laboratory conditions. This longitudinal, multi-level exploration of human-AI interaction in virtual learning environments endeavors to remedy these shortcomings, contributing practical insights applicable to real-world settings. Firstly, the results of our study reveal that incorporating AI, here ChatGPT, in human-AI teams, could enhance learners’ knowledge acquisition and stimulate their intention to learn. It has practical implications for educators and instructional designers who can leverage AI capabilities to foster knowledge update and stimulate learning within team-based settings. This finding aligns with the idea that practitioners should rethink the current team structure and explore the potential of AI as a team member rather than just a tool (Deloitte Insights, 2022).

Secondly, the crucial role of AI characteristics such as autonomy and explainability in facilitating knowledge acquisition underscores the necessity for AI developers to prioritize these features when designing AI for educational settings. Such an approach resonates with the recommendations of several scholars (e.g., Endsley, 2023) who emphasize the importance of autonomy and explainability in designing AI systems. Thirdly, TPV’s moderation suggests that both educators and managers need to factor in the degree of their team’s virtuality when integrating AI. To boost the connection between AI characteristics and knowledge acquisition (Darban, 2022a), teams should be encouraged to lean more heavily on virtual technologies (Brown et al., 2020). Ensuring teams have the necessary resources and skills to fully utilize collaboration tools is critical. Implementing suitable communication extensions in learning management systems (LMS), like Moodle, is recommended. Additionally, creating adaptable communication tools within the LMS could allow educators to manage their teams’ level of virtuality more effectively.

6.3 Limitations and future research

This study does have certain limitations, which open avenues for future exploration. Firstly, our sample consisted of MBA students enrolled in an information systems course. Although student samples accurately represent the target population of interest to advance the applicability of the results, it would be beneficial to conduct further research involving actual employees in real-world business scenarios. Secondly, while our results indicate successful knowledge acquisition in teams enabled by ChatGPT, we did not assess the performance of human-only teams versus those integrated with AI. Future research could enrich the literature by including both types of teams for comparison, thus exploring the effects on individual and group performance.

Thirdly, our focus was limited to one team-level factor. Future research could expand this scope by including other team or course-level variables to capture the diversity in virtual team learning environments. Lastly, the literature suggests that team members’ attributes (Darban, 2022b) and the nature of their collaborative tasks, for instance, the diversity of teamwork (Lee et al., 2015), may influence learning outcomes in virtual teams. Future research should identify specific conditions that may determine the success, failure, or possible harmful effects of incorporating an AI teammate.

6.4 Conclusion

This research delves into the key aspects influencing the role of AI in virtual learning teams. The study expands the literature on virtual team learning by magnifying the role of AI in project-based team learning and highlighting the significant contribution of AI teammates to the knowledge update process, going beyond the role typically played by human teammates. Additionally, the study emphasizes the importance of AI design attributes in promoting knowledge transfer and boosting overall team performance. A novel aspect of this study lies in its multi-level approach, which addresses the often-overlooked team-level factors that impact individual learning outcomes in virtual HAT settings. Moreover, it brings to light the significant role of TPV as a crucial team-level factor influencing the effectiveness of AI.