Introduction

In the year 2020, WHO declared the COVID-19 pandemic outbreak as a public health emergency of international concern (WHO 2020). The pandemic continued to wreak havoc on all industries and segments alike throughout the world all through 2020 and 2021. Along with affecting millions of people worldwide, one of the sectors that was majorly affected was education. When the crisis was at its peak, data from UNESCO reported that over 1.6 billion students in over 190 countries had to stay out of their educational institutions (Pokhrel and Chhetri 2021). Similarly, close to 100 million teachers and academic personnel had their jobs impacted as institutions suddenly closed down (UNESCO 2021). Amidst the crisis, the biggest shift in the education sector was shifting to a virtual first mode (Jena 2020). Students across different age groups resorted to remote learning as pedagogical methods changed from classroom-based to virtual communication platforms like Zoom, Microsoft Teams, and so on.

Among these were students pursuing management education in business schools (b-schools) across India and the world. Being one of the highest levels of education and one that acts as a bridge to build immediate future leaders and managers, the mode and method of information delivery are significantly different in b-schools. Whereas schools have a higher percentage of rote-learning integrated within their methodology, b-schools rely more on inculcating practical aspects of management within their students. When COVID-19 caused several countries to shut down face-to-face methods of learning completely, b-schools and management students were among the worst affected (Rawal 2021). During this time, virtual-learning took prevalence and was able to mitigate much of the challenges being faced by students because of the pandemic. It even gained traction among some students in terms of providing flexibility to schedules, higher access to global learning resources and new and exciting ways of learning delivery. On the flip side, whether or not it matched up to traditional pedagogical methods that revolved around hands-on practice, a significant pillar of management education, is something that remains ambiguous. In this study, we aim to fill this gap by developing a holistic scale comprising of acquisition, retention and application of knowledge that can help measure the impact of pandemic and virtual learning methods on b-school education.

Aims and objectives

This conceptual research aims to:

  • Explore the impact of the shift in pedagogy for management education from a conventional setting to a virtual mode of learning

  • Understand whether the learning outcome of management students characterized by acquisition and retention affected the application of knowledge as a result of the shift

  • Develop and validate a virtual learning outcome scale to measure students’ perception of their learning outcomes in the virtual mode of learning as compared to the physical mode of learning

Literature review

Types of learning delivery methodologies

Due to the ever-changing business landscape, management education is one of the most dynamic studies (Gupta et al. 2021). Due to this, the pedagogical methods followed by b-schools have always been globally inspired to say the least. The curriculum is driven by a mix of theories developed by researchers worldwide and practical aspects taken from real-life case studies of organizational successes and failures to derive valuable lessons. Therefore, management education has mostly been a mix of classroom-based discourse and virtual resources to aid the discourse. Broadly, the pedagogical methods in management education can be defined as Face-to-Face Mode of Delivery, Virtual Mode of Delivery, and Hybrid/Blended Mode of Delivery.

Face-to-face (F2F) mode of delivery

The F2F model of learning delivery is the most common method of education in b-schools and is characterized by real-time interaction between the facilitators and students. Along with imparting theoretical knowledge, students are exposed to various case study analyses in projects or assignments that involve verbal presentations, role-plays, or other creative forms of delivery. The exchange of information on the part of the facilitators and the derivation of concepts on the part of the students using real live presentations aids in creating better learning outcomes for the students. This setup helps in better student engagement and reach and helps in the simulation of management scenarios within a classroom which enables better understanding. (Goyal et al. 2020).

Virtual mode of delivery

The virtual mode of delivery started as an asynchronous learning method which included pre-recorded videos and other resources for the students to go over at their own pace. Programs other than regular management courses, such as distance education or correspondence programs, are the ones that make use of this methodology (Goyal et al. 2020). With time, this came to include the synchronous mode of learning where facilitators could hold virtual classes with real-time students on the opposite end (Dhawan 2020). With the disruption caused due to COVID-19, the virtual mode of learning delivery took prevalence in the face of the closure of institutions. This method provides the flexibility required for both facilitators and students regarding class timings (Xie et al. 2020). Session recordings also help students go over details later on that they might have missed during class hours. However, the virtual mode of delivery has also been characterized by lesser student engagement and comprehension of concepts riddled by technological disruptions such as internet connectivity issues in remote places which are not lucky enough to have seamless connectivity (Shahzad et al. 2020).

Hybrid/blended mode of delivery

A blended model of learning delivery comprises a mix of virtual and classroom-based learning. It is mainly implemented by executive management programs for working professionals or in some distance education programs where a certain percentage of facilitator interactions are F2F. At the same time, the rest of it is virtual (Goyal et al. 2020). This method of learning aims to provide the flexibility of schedule and convenience associated with self-paced learning along with a healthy mix of face-to-face experience (Paudel 2020).

Acquisition of knowledge in virtual mode

A standard measure of the effectiveness of pedagogical methodologies is learning intake and learning outcomes which can be subdivided into acquisition, comprehension, retention, interpretation, and application of knowledge as a loose translation of cognitive learning as per Bloom’s taxonomy (1964). Knowledge acquisition is the process of locating, collecting, and refining knowledge to develop knowledge-based systems (Harmon and King 1985). It is characterized by transferring information from one system to another (Grosso 1998). Due to COVID-19, as learning methods went virtual, student engagement levels varied depending on factors such as quality of facilitator and personal cognitive levels, which further affected their learning intake levels. This also made differences in how a student applies the acquired knowledge in situations that demand it.

Retention of knowledge in virtual mode

Studies have shown that knowledge is highly retained and interpreted by students when there is an organized interaction in virtual learning (Garrison and Cleveland-Innes 2005). Engaging interaction between the facilitators and learners is important, as it has the power to impact motivation levels and inspire learners to put an active effort to retain and interpret knowledge in virtual mode. (Lee 2018). With the pandemic and a subsequent shift to virtual learning, learners were not ready to learn virtual and when students are not ready and motivated for learning sessions, virtual education does not prove fruitful (Piskurich 2003, pp. 1–10). Retention of knowledge in virtual learning sessions (Vallerand and Blssonnette 1992), the perceived learning quality (Grolnick and Ryan 1987), as well as student performance (Saad´e et al. 2007) have been studied to be affected strongly by student motivation in the virtual learning environment.

Application of knowledge in virtual mode

Carl Rogers (1969) had theorized that in order to supplement the theoretical knowledge acquired in a cognitive learning process, an applied knowledge mechanism should also exist through experiential learning process. One of the major challenges for formal education is facilitating the transfer of acquired theoretical knowledge to apply in a practical domain such as work. Bellamy (1996) stated that learning would be only be fruitful when student collaboration is made an important part of learning as it goes a long way in helping learners acquire useful and comprehensible knowledge that they can apply in real life. A very relatable study by Hansen (2008) proposed three aspects of application of knowledge in an virtual scenario: a. Virtual learning enables learners to take higher ownership of the study material as it is a highly independent learning process; b. Virtual learning sessions cause an increase in the feeling of isolation making it even more important for learnings to try and continuously interact with facilitators and each other; c. According to a study by Iverson et al. (2005), learners in virtual sessions, have a stronger desire to apply their knowledge to newer domains. Whether or not the study held true in the pandemic situation amidst the sudden shift in pedagogy remains to be seen.

Research methodology

Scale development

To develop items for the constructs, an extensive review of literature articles was conducted, resulting in the construction of 14 items. Most of the items were drafted as relevant to our study and in the comparative context between virtual and conventional learning environments. The study involved not only the literature review but also conducted focused group discussions with management students who have the experience of the swift switch from conventional to the virtual mode of learning. We conducted two FGDs (n = 4) where we validated the understanding and readability of the items. The discussion also revealed two new items. We also merged a few overlapping items as supported by the FGD, narrowing it down to a total of 12 items.

Sample population

A survey instrument was used to collect the views and opinions of respondents across different b-schools in India. The sample population comprised current management students as well as recent graduates who had faced the transition from an utterly classroom-based setting to the virtual mode as a result of COVID-19. The list of management schools for data collection was decided based on NIRF rankings. Management students of B-schools under the NIRF rank 50 were contacted to fill out the survey. However, the selection of respondents was based on the snowball sampling method. Respondents were asked to fill out the survey anonymously to avoid exposing person-specific information. The rationale for choosing the sample population as management students was because management education is highly experiential and therefore is a ripe cohort to study the impact of the shift in pedagogy on learning outcomes. Additionally, due to the dynamic nature of management education, b-schools are one of the most prompt institutions in implementing newer learning methods. This makes management students a good focus group to understand the implications of the change in pedagogy for learning outcomes.

Data collection and interpretation

The questions in the survey instrument aimed to probe respondents regarding their experience and their perception of personal learning outcomes as a result of the shift in pedagogy that happened during the pandemic. Learning intake was further sub-divided into different aspects such as acquisition, interpretation, and retention of knowledge, and we aimed to understand whether they were in any way related to the application of knowledge in survey respondents (Templeton et al. 2002). Questions in the survey instrument revolved around these areas, and responses were captured using a 5-point Likert scale spanning the options Strongly Disagree, Disagree, Neutral, Agree, and Strongly Agree. A total of 121 responses were collected in a span of one month. Data cleansing was conducted, and datasets having incomplete responses were removed. The final analysis was performed with 104 complete responses that were found suitable for analysis.

Data analysis

Exploratory factor analysis

Using SPSS 25, exploratory factor analysis was performed. We performed factor analysis on the responses gathered to understand how our variables spanning across acquisition, retention, and application of knowledge are interrelated. We set the minimum factor loading as 0.5 (Leech et al. 2004). We then assessed the dimensionality using the varimax rotated factor solution.

As demonstrated in Table 1, KMO (Kaiser–Meyer–Olkin) measure and Bartlett’s test yielded a result of 0.741 indicating the adequacy of the sample in the analysis. Bartlett’s test being significant indicated that there were interrelated variables. Communality being (> 0.3) for each item indicated that all items fulfilled the criteria and were therefore retained for analysis. In the rotated component matrix, there were no non-loading items. All the factor loaded well in their respective constructs (Table 2).

Table 1 KMO-Bartlett test
Full size table
Table 2 Factor analysis of virtual learning outcome scale
Full size table

For inter-item reliability, Cronbach’s alpha is considered an adequate index. Cronbach’s alpha was found to be between 0.791 and 0.857 (demonstrated in Table 3), hence satisfying the minimum standard (Hair et al. 2014). There are three components identified with an eigenvalue greater than 1. All the items had good loadings on our intended constructs and indicated a cumulative variance of 67.497%, as shown in Table 4.

Table 3 Cronbach’s alpha
Full size table
Table 4 Variance summary
Full size table

Confirmatory factor analysis

After exploring factors, we tested measurement models using structural equation modeling. We employed AMOS 23 to perform confirmatory factor analysis. We assessed goodness of fit, convergent and discriminant validity, and composite reliability. To evaluate model fit, we observed good fit to the data (χ2 = 61.784, df = 51, SRMR = 0.071, CFI = 0.984, GFI = 0.909, TLI = 0.979, and RMSEA = 0.045). Based on the factor loading and constructs derived from factor analysis, we developed a measurement model that showed a good fit in CFA, as shown in Table 5. The standardized path diagram for the factor structure is displayed in (Fig. 1).

Fig. 1
figure 1

Diagram representing the structure coefficient for virtual learning scale items

Full size image
Table 5 Goodness of fit indices
Full size table

Reliability and validity analysis

We conducted a reliability analysis to be informed of the internal consistency of indicators for each construct. We also assessed Composite Reliability to report the reliability of factors. All the Composite Reliability values were found to be between 0.799 and 0.882. We also evaluated convergent validity by observing Average Variance Explained (AVE) for each construct. The values and CR, AVE, and the model’s outer loadings are reported in Table 6.

Table 6 Results of item loadings from CFA
Full size table

The discriminant validity of the scale was established Fornell-Larcker method by comparing constructs AVE (shown in bold along the diagonal in Table 7) with the inter-correlation of other constructs (Shown vertically below the value of AVE). AVE is established if the constructs AVE is found to be greater than the correlation. The result shows the evidence for discriminant validity.

Table 7 Discriminant validity
Full size table

Analysis/discussion

This conceptual research aimed to build a scale to understand student perceptions regarding whether the pedagogical changes due to COVID-19 impacted student learning intake, which impacted how students applied the acquired knowledge. The responses by the sample cohort brought out interesting themes which could be utilized to understand the perceptions of management students towards the observed change in pedagogical methods as a result of the pandemic. The loading of groups of variables into three different factors indicated a definite relation between clusters in the virtual learning scenario.

The first factor, which is comprised of acquisition of knowledge, falls under the category of learning intake of management students due to the pedagogical changes. The clearly distinguished factor shows that due to COVID-19, as learning methods went virtual, learning intake was impacted, as student engagement levels varied from one student to another.

Similarly, the second factor comprised of retention of knowledge by individuals. Insights from this factor reiterated that retention and interpretation of knowledge in virtual mode highly depends upon the engaging interaction between the facilitators and learners (Lee 2018).

The third component comprised of application of knowledge in virtual mode and mostly referred to exposing students to cases and encouraging them to step into the shoes of a manager to explore solutions either individually or in groups. Insights from the study brought out that group-level activity and interactions are impacted due to remote learning platform.

Combining the three, it can be presumed that there is a definite relation between pedagogical changes due to the pandemic and learning intake and outcomes for management students. The scale thus developed can be utilized to identify student perception towards virtual learning in the areas of acquisition, retention and application of knowledge. Further research in this area can be extended towards understanding other factors such as self-study, emotional intelligence and personal cognitive factors that play an important role in acquisition, retention and application of knowledge.

Implications and usefulness of the study

This research study has made an effort to develop a virtual learning outcome scale comprising of Acquisition of Knowledge, Retention of Knowledge, and Application of Knowledge among management students. The scale will prove to be helpful in identifying and measuring the learning intake and outcomes of management students in a virtual learning scenario compared to the conventional physical mode of learning. This will enable facilitators in b-schools to develop interesting ways to bridge the gaps in teaching and learning in virtual or hybrid methods of learning to improve the acquisition, retention, and application of knowledge.

As a response to COVID-19 and the transition to the virtual learning mode, several infrastructural interventions were undertaken to aid management students’ learning. However, the results and analysis of the survey responses ultimately raise questions on whether there need to be facilitator training interventions and infrastructural developments to inculcate the skill of imparting education to the students in virtual mode. Considering that management education relies largely on experiential learning, new and improved methods of student engagement need to be devised. Additionally, facilitators need to draw up teaching plans that feature a mixture of conceptual discourse, real-life examples by means of videos or sharing experiences by industry experts, and projects that require original thinking on the part of students. Additionally, the exercise of measuring learning outcomes will also help strengthen learning methodologies to enable b-school students to be better equipped for their future managerial practical endeavors.

Limitations and scope for future research

The current research does not cover facilitators’ points of view regarding challenges faced. Future research could probe into the challenges faced by the facilitators in the virtual mode of teaching, and the responses therein can be combined with those presented in this paper to come up with innovative interventions to benefit the entire management academia fraternity. The future scope of research also includes regression analysis of the components to understand the extent to which each component affects the other. This research study is generalizable only to the extent of higher education types that do not have a dependence on laboratory or field learning.