Theoretical model of investigating determinants for a successful Electronic Assessment System (EAS) in higher education

Abstract

Electronic assessment (e-assessment) is an essential part of higher education, not only used to manage a large class size of students’ learning performance and particularly in assessing the learning outcomes of students. The e-assessment data generated can not only be used to determine students’ study weaknesses to develop strategies for teaching and learning, but also in the development of essential teaching and learning pedagogies for online teaching and learning. Despite the wider adoption of Information and Communication Technology (ICT) technologies due to the COVID-19 pandemic, universities still encountered numerous problems during the transformation to electronic teaching as most educators struggled with the effective implementation of the Electronic Assessment System (EAS). The successful launch of EAS relied heavily on students’ use intention towards the new and unfamiliar electronic system, which was actually unknown to the project managers of EAS. It is therefore important to understand students’ views and concerns on EAS and the proactive measures taken by universities to enhance students’ acceptance and intention of usage. Although most studies investigate students’ acceptance of online learning, there is still little research on the adoption of e-assessment. In this regard, we propose to develop a theoretical model based on students’ perceptions of EAS. Based on the Technology Acceptance Model (TAM) and a major successor of TAM, an electronic assessment system acceptance model (EASA model) is developed with key measures including system adoption anxiety, e-assessment facilitation, risk reduction amid, etc. The data is obtained through a survey among current students at a local university, and structural equation modeling (SEM) is applied to analyze the quantitative data. This study has a significant impact on improving educators’ use of e-assessment in order to develop essential online teaching and learning pedagogy in the future.

Introduction

Recently, synchronous and asynchronous modes (Chau et al., 2021) of teaching and learning has attracted much attention due to the outbreak of the COVID-19 pandemic which has forced numerous schools to long-time campus lockdown due to the prevention measures implemented by governance authorities as group gathering and social interaction may present potential threats to public health. Under this special circumstance, digitalization in educational institutions has been accelerated as many universities have shifted to launch electronic teaching (e-teaching) to replace the traditional classroom teaching activities (Tang et al., 2021a). Actually, e-teaching has been adopted for more than a decade in many universities across the world as driven by the rapid development of Information and Communication Technology (ICT) (Parker et al., 2011; McQuiggan, 2012; Alexiou-Ray & Bentley, 2015), and which also had attracted tremendous attention in past research (Mac Callum & Jeffrey, 2014; Betts and Heaston, 2014; Mohammadyari & Singh, 2015). The e-teaching technologies including such as mobile devices (Tang & Yu, 2018), online learning platforms, MOOC (Bralić & Divjak, 2018), virtual reality (Tang et al., 2021b) and mixed reality (Tang et al., 2018, 2020) technologies, etc.

Although the adoption of e-teaching has been widely studied, limited research has focused on electronic assessment (e-assessment). In fact, e-assessment is another key component integrated as an essential part of education, not only for managing students’ performance, performing assessments on the students, supporting teaching and learning, but in particular for assessing the learning outcomes of students. These assessment data are critical in determining students’ weaknesses, developing an AI model for study recommendation, and developing online teaching and learning pedagogies to improve students’ academic performance. Compared with the traditional paper-based assessment, e-assessment has its unique features and advantages with supports from the rapid-developed ICT, such as fiber-optic communication and cloud technology. Da’asin (2016) defined e-assessment as the student using computers that have programs designed for the exam purpose and answers questions electronically. Hamsatu et al. (2016) broadened the definition as “the use of information technology for any assessment related activity”. In addition, the functionality of a system to carry out e-assessment had also been illustrated in previous studies (Ayo et al., 2007; Kuikka et al., 2014). In this study, e-assessment is defined as to use ICT to carry out assignments, examinations, and evaluation via the E-assessment System (EAS), and to improve the teaching process and learning outcomes through gathering and reviewing empirical data collected by the EAS. E-assessment can facilitate individualized educational interventions as it enables teachers to provide timely feedback to learners, which in turn improves the learning outcomes of students (Terzis et al., 2012; Wang 2014). In addition, the efficiencies of exam production, delivery, and administration in e-assessment are higher than traditional assessment (Wu et al., 2015). As mentioned by Ayo et al., (2007), e-assessment was more effective than traditional assessment as on one hand, it offered greater flexibility for the assessment time and location, on the other hand, it reduced a large proportion of workload during assessment, such as training, grading, and reviewing. While more importantly, the student also agreed and treated the e-assessment as an accurate and fair approach to conduct evaluation properly (Terzis et al., 2012). Moreover, e-assessment provides novel measurements that are hardly or even not possible to be undertaken in traditional assessment (Wu et al., 2015). As pointed out by Hochlehnert et al. (2011), e-assessment enabled easier control and editing of evaluation items, moreover, it combined advantages of multi-media content (e.g. with the integration of text, pictures, and audiovisual information), diverse evaluation formats, and rapid data analysis. As ICT is used for the whole process of e-assessment, information and data input and generated by such end-to-end electronic process could be collected and analyzed timely, efficiently, and accurately (Alruwais et al., 2018).

To obtain the benefits accompanying e-assessment, it is essential to understand users’ views and concerns regarding the function and implementation of EAS to ensure its successful launch. By understanding the affected factors for a successful launch of EAS, institutional interventions and measures such as training and marketing could be formulated and implemented proactively. Da’asin (2016) examined the attitudes of students at a university on e-assessment and found that positive attitudes (e.g. satisfaction with the arrangement and design of the e-assessment) mingled with negative attitudes (e.g. anxiety and stress of taking the e-assessment) among the students. However, student’s attitudes may also evolve with the changing circumstance. To understand university students’ views in a changing environment, Deutsch et al. (2012) investigated students’ attitude changes towards e-assessment and pointed out that the actual experience positively affected students’ attitudes toward e-assessment. While the study also indicated that negative effects generated from technical problems of e-assessment also existed, which should not be ignored when implementing an EAS. At present, the outbreak of COVID-19 could be a catalyst for the surging demand for e-assessment due to its capability to the reduction of travel needs and social interaction. Furthermore, it could also be expected that the transformation and acceleration from traditional paper-based assessment to e-assessment would continue in the post COVID-19 world due to the beneficial advantages of e-assessment. This study aims to investigate the key determinants for a successful EAS through the proposed ESAS model so as to provide suggestions on enhancing the adoption of the EA. This study has contributed not only to the development of a new ESAS model for supporting EAS adoption, but also to investigating students’ perceptions towards EAS adoption so as to support educators in developing strategies for enhancing online teaching and learning in the future.

The organization of the study is as follows. A literature review is provided in the next section, with the introduction of previous studies on e-assessment and EAS, TAM model and its major extension model for the evaluation of user’s acceptance towards a new technology would also be elaborated in Sect. 2. After illustrating the rationale for the investigation of intention regarding the implementation of EAS, a research model (i.e. EASA model) would be proposed in Sect. 3. Methodology, followed by the data analysis and results, would be presented in Sect. 4. After providing the model results, discussions on findings and implications of the research model come afterward in Sect. 5 and Sect. 6 respectively. Finally, conclusions, together with study limitations and future study, was presented in the last section of the paper.

Literature review

The assessment was at the heart of higher education and a core component for effective teaching and learning (Gikandi et al., 2011), and the study had shown that the well-designed assessment was able to increase the overall performance of students (Holmes, 2015). With the rapid development of ICT and digitalization, e-assessment has been conducted widely in educational institutions as it helps to enhance the procedure and quality of assessment, for example, it was cost-saving and was able to act as an essential supplement for traditional paper-based assessment (Stödberg, 2012). In addition, a more reliable assessment result compared against manual grading in traditional assessment could be achieved with the application of e-assessment (Wang et al., 2008). Besides the advantages of efficiency and accuracy in conducting evaluation, e-assessment was also convinced to be more applicable to stimulate the engagement and creativity of students (Fjørtoft, 2020). Which meant the continuous monitoring and evaluation of the stage growth and progress outcomes for students’ whole learning journeys could be documented through the means of e-assessment (Gikandi et al., 2011). In general, e-assessment was believed to be beneficial to students, teachers, and institutions as it provided the flexibility of assessment’s location and time, enhancement of learning outcome, immediate and higher-quality feedback from teachers, adoption of adaptive testing, and analysis of students’ cross-performance track records (Alruwais et al., 2018). In addition, the abundant information and data collected by EAS, such as student’s integrated performance across different related assessments (e.g. assignments, exams, and other educational activities) in one specific period or the whole study journey, would be a valuable asset to teachers and universities to systematically and individually analyze student’s performance and learning outcomes (Gañán et al., 2017). Despite the novel functionalities and advantages enabled by e-assessment, challenges for its adoption also exist, thus, plenty of efforts were needed for a successful implementation of EAS (Stödberg, 2012).

Actually, integrated efforts for both system aspect (e.g. infrastructure and functionality of the EAS) and user aspect (e.g. perceptions and using the experience of the student), were required to guarantee a satisfactory implementation of EAS. The successful introduction of an EAS was affected by a lot of factors, such as feature, usability, and reliability of the system, as well as related training and appropriate supports formulated to users (Kuikka et al., 2014). Further to the facilitation measures provided to users, various challenges such as lack of well-established technical infrastructure, insufficient use intention of inexperienced students, and inadequate confidence of users on system’s functions, were also needed to be carefully addressed for obtaining user’s acceptance towards a new EAS (Alruwais et al., 2018). User’s acceptance would be investigated by examining different kinds of variables, for example, an investigation of student’s perceptions on EAS was conducted in a university and positive attitudes were found towards a recently-launch EAS, while the degree of acceptance varied among different groups of the student (Sorensen, 2013). The examination for variables that affected students’ acceptance on a new EAS could be conducted by analyzing students’ responses in a survey with different models about user’s acceptance towards a new ICT application, such as the Technology Acceptance Model (TAM) and The Unified Theory of Acceptance and Use of Technology (UTAUT) Model (Terzis & Economides, 2011).

TAM was developed specifically suit to model users’ acceptance of computer-related technologies (Davis, 1989; Davis et al., 1989). TAM suggested that the acceptance of using a new ICT application was significantly determined by two factors: Perceived Usefulness (PU) and Perceived Ease of Use (PEOU). Under the definitions of TAM, PU was defined as the degree to which a person believed that using a particular system would enhance his/her job performance, and PEOU was defined as the degree to which a person believed that using a particular system would be free of effort (Davis, 1989). After its introduction, TAM and its successor models have been extensively applied to examine various types of factors that affected student’s behavior towards ICT application in various domains, such as electronic library (Jeong, 2011; Joo & Choi, 2015), electronic learning (Ngai et al., 2007; Mac Callum & Jeffrey, 2014), and electronic assessment (Nikou & Economides, 2017). The UTAUT model developed by Venkatesh et al. (2003) was a major successor among the TAM extension models. In UTAUT Model, supplementary factors such as social influence (SI) and computer anxiety (CA) were applied to examine the behavioral intention of users towards a new ICT application. For example, SI was found to positively affect university students’ acceptance of e-assessment (Terzis & Economides, 2011).

The outbreak of COVID-19 implied new potential factors that affected students’ behavior under such a special circumstance. The high transmission rate and mortality rate of COVID-19 had caused a high level of fear in the society and impacted people’s behavior solidly (Schimmenti et al., 2020; Jian et al., 2020), people may tend to reduce unnecessary travel and social interaction. As the adoption of ICT application always means more online interactions and less physical movement, therefore this study tries to fill the research gap by considering the impact of COVID-19 on the use intention of a new ICT application. In this study, factors like risk reduction and special facilitation measures amid the pandemic would be added to the TAM model to examine the behavioral intention of new ICT application potential users (i.e. current students) when introducing EAS in the university.

Research model

This study proposed a theoretical model for investigating the determinants of successful EAS in higher education. The proposed model was enhanced based on the TAM model introduced by Davis (1989) and its major successor, the UTAUT Model developed by Venkatesh et al. (2003). TAM is a robust model to examine users’ behavioral intention towards a new ICT application by considering the two most influential factors, which are the benefits and effortlessness of using the application. Thus, the variables of PU and PEOU of the TAM model were adopted and examined in this study as the directly affected factors determining the Usage Intention (UI) of students towards a new EAS. In addition, two external factors, namely Social Influence (SI) and System Adoption Anxiety (SAA), developed in the UTAUT Model were also applied in this study to investigate students’ acceptance and intention towards the EAS. Finally, one external factor, namely E-Assessment Facilitation (EAF), was developed from the original concept of Facilitating Conditions (FC) in the UTAUT Model and modified to reflect the pandemic outbreak, and one additional factor, namely Risk Reduction amid (RR), was formulated in the study to understand student’s views on risk reduction during COVID-19 by launching the EAS.

Students’ views on the above seven factors were collected through a self-administered questionnaire, and the questions for each factor were listed in Table 1. This was, students were asked to score, on a 5-point Likert-type scale, how much they agreed or disagreed with each statement in the questionnaire (i.e., 1 to 5 means from “strongly disagree” to “strongly agree”).

Table 1 Questionnaire Items in the Survey

After the formulation of variables (i.e. questions) for each potential factor, hypotheses were developed to examine factors’ impacts on students’ UI for an upcoming EAS. Therefore, a research model, namely Electronic Assessment System Acceptance model (EASA model), was developed to understand university students’ acceptance towards a novel EAS under the environment of a global pandemic. The conceptual model was shown in Fig. 1.

Fig. 1

EASA Model

Perceived Usefulness (PU) and Perceived Ease of Use (PEOU)

The core part of the proposed model was based on the TAM developed by Davis (1989), which was the inter-relationship between PU, PEOU, and UI. In other literature on acceptance of new ICT applications (e.g. Joo & Choi, 2015), PU and PEOU were also found to be the significantly affected factors on user’s intention to use new technology. In general, users tended to adopt the technology if they can perceive the functional advantages and effortlessness of adopting the technology. For example, the acceptance of a new system could be high if students expect the system was easy to handle and meanwhile can facilitate their performance improvement. The same effects existed on user’s adoption intention for a new electronic system (e.g. Mac Callum & Jeffrey, 2014). In addition, a previous study also revealed the positive influence of PEOU on PU (e.g. Rafique et al., 2019). Therefore, the following hypotheses are proposed:

Hypothesis 1

(H1): PEOU has positive effect on UI.

Hypothesis 2

(H2): PU has positive effect on UI.

Hypothesis 3

(H3): PEOU has positive effect on PU.

Social Influence (SI)

The affected factor SI was developed by Venkatesh et al. (2003) in UTAUT Model as the extension of the TAM model to understand user’s acceptance of a new ICT application, thus, to assess the likelihood of successful implementation of the new application. In UTAUT Model, SI was defined as “the degree to which an individual perceives that important others believe he or she should use the new system”, and the study found that SI was a significant determinant for users’ perception of usefulness for a new ICT system and their adoption intention towards it, especially during the initial implementation of the system (e.g. Nikou & Economides, 2017). Undertaking e-assessment may be a new approach for students, especially in the context of the pandemic, thus, it was valuable to understand to what extent, student’s peers and teachers, as well as the university they studied in, would influence their usefulness perception and behavioral intention for an upcoming EAS. Therefore, the following hypothesis is raised to test:

Hypothesis 4

(H4): SI has positive effect on PU.

System Adoption Anxiety (SAA)

The SAA factor was evolved from the factor of Computer Anxiety (CA) in the UTAUT Model (Venkatesh et al., 2003). CA depicted user’s sentiments regarding new technology in the UTAUT Model, such as feeling apprehensive about using new technology, scared of making mistakes when using a new ICT system, and so on. A study had found CA to be a significant affected factor for the acceptance of launching a new ICT application (Celik & Yesilyurt, 2013). Furthermore, CA could result in a less likely successful adoption of ICT application due to the negative relationship between CA and PEOU (Mac Callum & Jeffrey, 2014; Nikou & Economides, 2017). In the EASA model, SAA, as a modification of CA under the scope of this study, was defined as the degree of an individual’s apprehension for the adoption of the new EAS. In line with the literature, which implies that users were less likely to adopt a new ICT application due to the perception of anxiety, the following hypothesis is proposed:

Hypothesis 5

(H5): SAA has negative effect on PEOU.

E-Assessment Facilitation (EAF)

The EAF factor was developed from the concept of Facilitating Conditions (FC) in the UTAUT Model. In UTAUT Model, FC refers to “the degree to which an individual believes that an organizational and technical infrastructure exists to support the use of the system”. However, under the circumstance of COVID-19, all the supports and facilitations would only be available through the internet due to the lockdown of campus and the prevention measures for reducing transmission and infection, such as the reduction of travel and social interaction. Thus, in the EASA model, all the items related to this factor were re-designed to address the actual status amid the pandemic instead of applying the definition directly from the UTAUT Model. Specifically, EAF in this study was defined as the degree to which an individual believes there was suitable personal equipment and sufficient online supports from the organization for the adoption of a new EAS. It is expected that sufficient EAF could moderate student’s perception of difficulty when adopting a new EAS, therefore, the following hypothesis is proposed:

Hypothesis 6

(H6): EAF has positive effect on PEOU.

Risk Reduction amid (RR)

The RR was a factor to evaluate the impact of COVID-19 on student’s perception of usefulness for an ICT application through reducing travel needs and physical interaction with other people. Amid the pandemic, people were suggested, or even forced to stay at home to prevent virus infection and to reduce the rate of transmission. Facilitated by an EAS, students could participate in various forms of assessments at home via the internet instead of travelling to school for the on-site assessments or other activities. Thus, this factor was designed to understand how the student would value the benefits brought by EAS during an epidemic. In this study, RR was defined as the degree to which an individual perceived the efficacy of EAS on risk reduction and inflection prevention amid the epidemic. As RR was a factor analyzing user’s acceptance towards an ICT application by understanding their evaluation of the functionality of the application, this study focuses on the aspect of individual safety and public health which was not considered in any previous studies. Therefore, the following hypothesis is formulated to examine the relationship of RR and PU:

Hypothesis 7

(H7): RR has positive effect on PU.

Methodology

Data Collection

Considering the circumstance amid COVID-19, a self-administered questionnaire delivered via e-mail and simple random sampling was selected to conduct the study. Before the formal distribution of questionnaire, a pilot test was conducted to examine whether participants could clearly understand the questions and felt comfortable answering them. After continued revisions for the wordings of the questions, a modified questionnaire with good content validity was distributed randomly via group email address to 703 current undergraduate students in a university located in Hong Kong. A total of 452 responses were received representing a response rate of 64.3%. Before performing data input and analysis, a basic pre-processing was undertaken for all the answered questionnaires. Which was, respondents with a high percentage of the same answers against all the questions and a high percentage of unanswered questions were excluded for further analysis. Finally, 393 of the collected questionnaires, which was 87% of the returned questionnaires, were valid respondents and input into SPSS (i.e. Statistical Package for Social Science) and SPSS AMOS after coding.

Respondent Profile

The results of the data inferred that the research sample was representative as respondents had consisted proportionally of different schools of the university, and the demographic data of respondents are illustrated in Table 2. The gender distribution was 56% female and 44% male in the sample, which was aligned with the proportion of the whole university. For the age distribution, as undergraduate students were the target of the survey, 92% of respondents belonged to the age group of 18–22. It was found that senior students tended to be more willing to answer the questionnaire as the distribution of respondents’ year of study out of a four-year undergraduate study showed that 66% of year 3 and year 4 students participated in the survey, compared with 34% of year 1 and year 2 students.

Table 2 Demographic Data of Respondents

Data Screening

Data screening was performed to achieve a clean dataset before conducting further statistical analysis. Firstly, as the significant value of Little’s Chi-square test was 0.172, which was greater than 0.05, the missing values were “missing completely at random (MCAR)” and then replaced by regression imputation technique. Secondly, the outliner was not detected in the coded data as all the score is within 2 standard deviations under the Z-score test. Finally, normality was checked through kurtosis and skewness, and the results showed that the data was normally distributed without any problem of normality as the value of both kurtosis and skewness were less than |2|, which was within the acceptance ranges of normality (skewness within |3| and kurtosis within |10|) (Li, 2019). After the data screening process, a two-step factor analysis was conducted to evaluate and formulate the proposed conceptual model.

Factor Analysis

Factor analysis was performed through SPSS to purify the variables and verify their relationship with relevant factors. As the corrected item-total statistics of all the variables were bigger than the threshold value of 0.3, all 26 variables had passed the pre-test (Lin et al., 2020). The results of factor analysis were presented in Table 3. It could be noticed that the value of Cronbach’s α of all the factors were above 0.7, which implied the authenticity of close relationships between variables and relevant factors, thus, it indicated sound internal reliability of variables within a factor. Then principal component analysis (PCA) with a varimax rotation was performed during exploratory factor analysis (EFA) to identify factor’s underlying variables. Based on PCA, variables with a factor loading higher than 0.7 were extracted (Nikou & Economides, 2017; Rafique et al., 2019) and loaded onto the seven factors as displayed in Table 3. Therefore, the variables of PU5, SAA1, and RR4, with the factor loading of 0.647, 0.555, and 0.666 respectively, were removed from the model after EFA, and the seven factors being extracted totally explaining over 70% of the overall variance. The results of EFA further confirm the adaptation of the proposed EASA model to perform confirmatory factor analysis (CFA). CFA was performed using structural equation modelling (SEM) in this study. SEM was selected due to its capability of estimating relationships among latent factors with multiple variables in the multivariate environment, and to identify the relationship simultaneously instead of testing the association individually. Before running SEM, a validity test, which was comprised of convergent validation, structural validation, and discriminant validation, was conducted for the conceptual model. Composite Reliability (CR) and Average Variance Extracted (AVE) were performed for the test of convergent validity. CR measured variables internal consistency, while AVE was the measurement of variables explained by the factors compared to variables explained by measurement errors. As the results showed, the coefficients of CR and AVE were all above the recommended thresholds, which were 0.7 and 0.5, respectively (Feng et al., 2014; Yuan et al., 2020). Therefore, the convergent validity of the proposed model could be considered adequate. Furthermore, as the results of KMO were all greater than 0.7 (factor EAF was slightly less than 0.7), and Bartlett’s Test were all significant at p < 0.001 level, structural validation was passed which indicated that the collected data of variables was suitable for model formulation. Finally, with regard to the discriminant validity, the heterotrait-monotrait ratio of correlations (HTMT) (Henseler et al., 2015; Li 2019) was applied to test whether the constructs were distinguished with each other, which was, the ratio larger than 0.85 might indicate lack of discriminant validity. As shown in Table 4, all the values were less than 0.85, thus, satisfactory discriminant validity was also confirmed for the research model.

Table 3 Results of Reliability and Validity Analysis

Table 4 The HTMT Ratio of Correlation

Structural Equation Modeling (SEM)

The EASA model was performed the test of SEM by software SPSS AMOS to estimate the relationship between variables and factors, as well as to evaluate the relationship among the factors. The Goodness of Fit indices reported in Table 5 indicated that the EASA model fitted the data quite well. Therefore, the results of the EASA model and the proposed hypotheses were shown in Fig. 2.

Table 5 Goodness of Fit Statistics of EASA Model

Fig. 2

Results of EASA Model

Discussions

The purpose of this study was to examine the affected factors on student’s acceptance towards the use of a new ICT application (i.e. EAS in a university). Based on previous research and with the integration of the current situation of the COVID-19 pandemic, 26 variables of 7 factors were tested and finally 23 variables were confirmed to influence student’s acceptance for the launch of a new EAS. It could be found that the core part of the EASA model, which was the relationship between PU, PEOU, and UI, aligned with the structure of the traditional model of TAM, while the difference in the magnitude of the coefficient was also spotted. However, the findings for the extension variables adopted from the UTAUT Model were not similar to previous studies as SI and SAA were found not to be the significant factors (Celik & Yesilyurt, 2013; Nikou & Economides, 2017). Furthermore, both variables (i.e. EAF and RR) formulated to reflect the circumstance under the COVID-19 pandemic, especially RR, were found to greatly affect user’s acceptance of the new EAS. The results of the EASA model were discussed in detail in the following.

Both PEOU & PU Have Positive Effect on UI (H1 & H2 Accepted)

Based on the results of the EASA model, H1 and H2 (i.e. PEOU and PU both have a positive effect on UI) were confirmed at a significant level, which was aligned with the TAM model developed in previous studies (e.g., Davis, 1989; Joo & Choi, 2015; Mac Callum & Jeffrey, 2014). However, judging by the magnitude of coefficient, this study found that PU (with the coefficient of 0.447, i.e., ${{\beta }}_{\text{P}\text{U}\text{U}\text{I}}=0.447$) had a much greater influence on UI than PEOU (${{\beta }}_{\text{P}\text{E}\text{O}\text{U}\text{U}\text{I}}=0.173$), which meant users were much more concerned about the usefulness of the EAS than the easiness of using the EAS (e.g. Mohammadyari & Singh, 2015). This was a bit different from the findings of Davis, as the impact of PU was found only a little higher than PEOU in that study (Davis, 1989). The enhancement of the importance of PU may be due to the actual circumstance when this study was undertaken. Because of the outbreak of COVID-19, students tended to highly weigh the functionality of the EAS and less care the effortless of using the EAS. Thus, the result showed that, students were concerned less about the difficulties of using the EAS as they had the confidence to manage the new system and were willing to use it if it can bring them benefits through its functionality.

The above statements were further supported by the coefficient of variables. The coefficient of UI3 (${{\gamma }}_{\text{U}\text{I}3}=0.672$) was relatively smaller than UI1 (${{\gamma }}_{\text{U}\text{I}1}=0.795$) and UI2 (${{\gamma }}_{\text{U}\text{I}2}=0.742$), therefore, it could be deduced that the main reason for student’s intention of using EAS was due to its capability of providing convenient equipment or facility to arrange assessment amid the epidemic instead of ‘disliking’ the paper-based assessment. In addition, it could be noticed from coefficients of PEOU1 and PEOU2, which were 0.866 (${{\gamma }}_{\text{P}\text{E}\text{O}\text{U}1}=0.866$) and 0.916 (${{\gamma }}_{\text{P}\text{E}\text{O}\text{U}2}=0.916$) respectively, that students believed both the university and themselves would be quite well-prepared for the implementation of EAS. Although PEOU3 (${{\gamma }}_{\text{P}\text{E}\text{O}\text{U}3}=0.723$) was also a significant affected factor, a student might just treat the conventional user manual as an additional supporting measure as they perceived the real-time online assistant to be more valuable. As for PU, respondents prized the value of EAS on its assessment arrangement function most as the coefficient of PU2 (${{\gamma }}_{\text{P}\text{U}2}=0.812$) was the largest compared to other variables under PU. This reflected that students expected EAS would help to reduce the infection risk during the pandemic by offering another option on assessment arrangement. The coefficient of PU3 (${{\gamma }}_{\text{P}\text{U}3}=0.797$) and PU4 (${{\gamma }}_{\text{P}\text{U}4}=0.767$) showed that students did acknowledge the value of EAS on teaching & learning improvement although this element was in a less important position. Similar to the UI factor, the variable related to the characteristics of paper-based assessment (i.e. PU1, ${{\gamma }}_{\text{P}\text{U}1}=0.665$) has a relatively small coefficient, which further indicated that students provided their responses with close consideration for the surrounding circumstance instead of just focusing on the features of these two types of assessment.

PEOU Has Positive Effect on PU (H3 Accepted)

The identified positive relationship between PEOU and PU (i.e., ${{\beta }}_{\text{P}\text{E}\text{O}\text{U}\text{P}\text{U}}=0.434$) was aligned with previous research (e.g., Mac Callum & Jeffrey, 2014; Park et al., 2009; Rafique et al., 2019). This positive effect further supported the finding that students cared more about the functionalities of the EAS instead of the efforts they perceived when shifting from paper-based assessment to e-assessment. This means, in the users’ aspect, they were driven to adopt a new system primarily due to the functions accompanied by the system, in addition, the effortlessness on achieving those functions would further enhance their willingness to adopt the new system. In the scope of this study, students were willing to face some sort of difficulties for the new application of EAS on the premise that PU had been perceived by them. In the contrast, although difficulties may hinder students’ adoption intention to some extent, no amount of PEOU can compensate students for the functional disability of the EAS. Students’ expectations on the functions of EAS had been enlarged due to the outbreak of COVID-19 as this new form of assessment not only boosted their learning experience, but also provide some sort of health protection amid the pandemic. Thus, during the design and development stage of EAS, project managers should avoid overemphasizing the ease of use while the principal focus should be the functions of the system, especially under the circumstance of the COVID-19 pandemic. Furthermore, PU of EAS, instead of PEOU of the system, should be the focal point when launching promotions for this new ICT application.

SI Does Not Have Positive Effect on PU (H4 Rejected)

The EASA model indicated that SI was not a significant factor that affected PU. The SI variable mainly estimates the social impact from surrounding people (i.e. university, teacher, and other students) to users for their perception of usefulness on the new EAS. This kind of impact may be a considerable affected factor under normal circumstances, but not valid amid the COVID-19 pandemic. In the past studies (e.g. Chen & Huang, 2012; Terzis & Economides, 2011; Venkatesh et al., 2003), SI was found to be a significant factor due to potential users may need opinions and advice from influential people, which were normally some senior or experienced persons and experts, to assist users to attain information and understand functionalities of a novel ICT application. However, under special environments or different issues (Mohammadyari & Singh, 2015), the influence from senior persons or experts may eliminate as users were able to perceive the usefulness of a new application without consulting them. In this study, as the survey respondents were all university students (year 3 and year 4 students consist of nearly 70% of the total respondents), they were mature and informative enough to evaluate the usefulness and difference brought out by the EAS, especially on the aspect of risk reduction during the outbreak of COVID-19. This means they didn’t need teachers or other persons to arouse their worries and concerns on the current situation and then provide suggestions to tackle the problems. With the fast development of ICT, students may receive messages or suggestions easily and conveniently from traditional media, social media, and the government about the necessity to reduce travel and social contact amid the epidemic, and those suggestions were achievable with the functions enabled by an EAS. The finding of an insignificant SI on PU was aligned with the result of the newly developed factor in this study (i.e. RR), which was designed to investigate the impact of the outbreak of COVID-19 on student’s perception of usefulness towards an EAS.

RR Has Positive Effect on PU (H7 Accepted)

RR was a tailored factor developed to estimate the influence of COVID-19 on user’s perception of function for the EAS (i.e., ${{\beta }}_{\text{R}\text{R}\text{P}\text{U}}=0.212$), which would indirectly impact their adoption intention on this new system. There were four designed variables for this factor to evaluate student’s views on risk reduction by the adoption of EAS, which included risk diminishment through the reduction of travel needs (i.e. RR1), group gathering (i.e. RR2), social interaction (i.e. RR3), and onsite assessment (i.e. RR4). The results showed that all the variables except RR4, which had been eliminated after PCA although its load factor was very close to the benchmark threshold value of 0.7, were found to significantly affect RR. Among the confirmed variables, the largest coefficient belonged to RR3 (i.e. ${{\gamma }}_{\text{R}\text{R}3}=0.912$), which indicated that students believed the reduction in social interaction was the most crucial preventative measure to enhance personal safety amid the pandemic. The coefficient of RR1 (${{\gamma }}_{\text{R}\text{R}1}=0.830$) and RR2 (${{\gamma }}_{\text{R}\text{R}2}=0.836$), were close to each other, which reflected that students perceived the reduction in travel and group gathering had similar efficacy to lower the probability of getting an infection. It should be pointed out that all the three variables of RR actually indicated strong relationships between the variables and the RR factor, especially when it is compared to other variables and their related factors. From this point of view, the EASA model revealed that the feature of risk reduction brought by EAS was the most crucial exogenous factor-driven students to adopt the new EAS and this statement was supported by the significant positive relationship between RR and PU. Based on this relationship, this study discovered that user’s perception of usefulness evolved during the change of circumstance, which was aligned with the findings of Deutsch et al. (2012). As the changing environment may differ user’s weight and perception towards specific functions of a system, therefore, the source of impact shifts accordingly. In this study, for example, the weight on the system’s specific function shifts to its capability on risk reduction. Meanwhile, the EASA model also indicated that this altered focal point also eliminated user’s application anxiety towards a to-be-launched system.

SAA Has Negative Effect on PEOU (H5 Rejected)

SAA examined the negative effects of new system adoption anxiety on user’s perception of PEOU, which indirectly affects user’s acceptance of using a new system (Liaw & Huang, 2015). SAA mainly measured the emotional aspect of user acceptance towards a new application (Celik & Yesilyurt, 2013), such as adoption sentiment on the functional aspect (e.g. Did users welcome or resist the system?), technical aspect (e.g. Did users worry on system error?), and operational aspect (e.g. Did users have the confidence to perform the system well?). Aligned with the study of Venkatesh et al. (2003), the result of the EASA model indicated an insignificant relationship between SAA and PEOU, which meant the assumption of students’ anxiety on the new system negatively affected their adoption intention was not supported by the model. Possible explanations for this phenomenon may be, there were other concerns that outweigh users’ anxiety on the learning and adaption of a new application, or the users had confidence to overcome the difficulties and problems when adopting a new system. The former explanation was supported by the identification and confirmation of H7 as respondents’ consideration of risk reduction outweighs their anxiety on practical adoption. While the latter explanation could be certified by the positive relationship between EAF and PEOU (i.e. H6).

EAF Has Positive Effect on PEOU (H6 Accepted)

EAF could help to enhance users’ intention towards a new system as it referred to external supports to facilitate a smooth implementation of the system (e.g., Alhabeeb & Rowley, 2018; He & Wei, 2009). The supporting measures were percepted valuable for system users and could come from various forms (Menchaca & Bekele, 2008; Puri 2012; Venkatesh et al., 2003), practice such as inquiry hotline, face-to-face consultation, and onsite training could all be conducted under normal situations. However, under the conditions of the COVID-19 pandemic, all the supporting measures designed and examined in this study were adapted according to the actual circumstance. The outcome showed that students were not quite worried about whether their individual equipment could meet the requirements of the EAS as the coefficient of EAF1 (i.e., ${{\gamma }}_{\text{E}\text{A}\text{F}1}=0.699$) was the lowest among other variables. Align with this finding, the outcome also indicated that students were concerned more about the facilitation by user manual when adopting EAS, as the coefficient of EAF2 (i.e., ${{\gamma }}_{\text{E}\text{A}\text{F}2}=0.873$) was the highest among the three measurement variables. In general conditions, it could be expected that EAF3 (i.e., ${{\gamma }}_{\text{E}\text{A}\text{F}3}=0.732$), which was the interacted personnel support arranged by the university, to be the most preferred source for receiving assistance. However, under the condition of COVID-19, users’ expectations and preferences altered. Furthermore, a less significant level and a smaller coefficient of the positive relationship between EAF and PEOU (i.e., ${{\beta }}_{\text{E}\text{A}\text{F}\text{P}\text{E}\text{O}\text{U}}=0.140$) indicated that EAF was a less important affected factor compared to RR in the EASA model. Therefore, under the special circumstance amid epidemic, project managers may need to modify their strategy accordingly in the stage of system implementation.

Implications

The proposed EASA model was the extension of TAM model introduced by Davis (1989) and the major successor of TAM, namely UTAUT Model developed by Venkatesh et al. (2003). TAM has been testified a robust model to examine users’ acceptance and usage intention towards a new ICT application (Joo & Choi, 2015; Mac Callum & Jeffrey, 2014; Nikou & Economides, 2017). By adopting some of the most influential factors in privious studies (Celik & Yesilyurt, 2013; Davis, 1989; Rafique et al., 2019; Venkatesh et al., 2003), such as the benefits and effortlessness of using a ICT application, this study extended TAM model with factors under the circumstance of a global pandemic outbreak.

Theoretical Implications

The EASA model indicated that user’s perception evolved in a changing environment. The results of the EASA model revealed that although user’s behavioral acceptance towards a novel ICT application align with the traditional TAM model to some extent, new observations were found under new conditions. The study showed that user’s perception of values, as well as their preference, evolved as the surrounding environment changed. The reasons for this evolvement could be complicated, for example, the diminishing impact of influential people may be due to the development of ICT and raising the influential power of social media. However, this evolvement needed to be examined and understood as it would affect the successful implementation of a new ICT application. Thus, factors of the past studies should be re-examined to fit into the present circumstance, and more importantly, new factors addressing the significant change of environmental conditions should also be considered and investigated when formulating a new model. The findings of this paper suggested user’s behavior would change remarkably during crucial or critical circumstances, and the influence of the new environmental factors (i.e. RR and EAF) could surpass other important factors (i.e. SI and SAA) in normal circumstances.

Managerial Implications

In practical, the EASA model also provided insightful implications to project managers for the development and implementation of a new ICT application. As new influential factors were found to have significant effects on user’s behavior amid the COVID-19, project managers of the university should adjust their implementation strategy to respond to the new conditions and the evolving user expectations. In general, project managers ought to place their top priority on enhancing the functions of the EAS instead of the effortless of using the system, in addition, the focal points of the marketing campaigns should be system’s function on risk reduction facilitation and special supporting measures from the organization. Based on the EASA model, proactive measures could be formulated to ensure a smooth implementation of the new ICT application.

Enhanced the Functions of EAS. Aligned with previous studies (e.g., Davis, 1989; Joo & Choi, 2015; Mac Callum & Jeffrey, 2014; Rafique et al., 2019), both PU and PEOU were influencial determinents for uers’ acceptance of a new ICT application. The results of the EASA model further indicated that students were much more concerned with the functions (i.e. PU) of the EAS rather than the ease of use (i.e. PEOU) of the system. Specifically, the top priority demand of students was that they wished the new EAS could provide an alternative on arranging various kinds of assessments via the internet to reduce infection risk amid the COVID-19 pandemic. While in the meantime, served as additional benefits bringing with an ICT application, students also expected EAS could enhance their learning experience and outcomes in the long run. From the EASA model, it could be deduced that the students were actually eager to the EAS as this new form of assessment could reduce the risk of public health amid the epidemic. This statement was supported by the variables of UI as respondents were expected EAS could provide convenient and secure forms of assessment, which required fewer travel needs and social contact. This was also the explanation for the less importance of the PEOU factor found in the proposed model. Lastly, students may treat it as an extra bonus of the EAS if error reduction during the marking and grading procedure could be achieved greatly after the implementation of the system. Thus, project managers could focus on the above functions when developing and promoting the EAS.

Emphasized the Advantage of Risk Reduction Enabled by EAS. The EASA model revealed that students perceived a high value of EAS’s function on risk reduction during the outbreak of COVID-19. This function of risk reduction was a crucial element to enhance students’ perception of PU for the new EAS. To educate students about the functions and benefits of EAS, project managers should not rely on influential persons such as teachers (Chen & Huang, 2012; Terzis & Economides, 2011) to illustrate only its functional characteristics on learning facilitation, instead, the main direction on functionality demonstration of the EAS should be risk reduction and infection prevention amid the epidemic. As shown in the model, RR existed a robust positive relationship on user’s perception of PU, thus, the information on travel and social interaction reduction amid the epidemic should be delivered and emphasized clearly during the marketing campaigns of the new system. Unlike severe acute respiratory syndrome (SARS), which had been contained after few months since its outbreak in 2003 (WHO, 2003), COVID-19 had lasted almost one year, but still no convincing signal indicated effective control could be achieved in a short period of time. Therefore, project managers ought to accelerate the development and implementation process of EAS as the fears and uncertainty of pandemic provided a good timing for its implementation in university.

Illustrated the Transformation of Assignments and Examinations under EAS. The transformation on assignments and examinations enabled by the EAS was another important determinant that affected user’s PU for this new ICT application (Alruwais et al., 2018; Gikandi et al., 2011). Students needed to be fully aware of the behavior change on undertaking assignments and examinations (Gañán et al., 2017), in this case, they will simultaneously understand the potential benefits brought by the EAS. The clear illustration on the transformation of assignment and examination on one hand will reduce the problem or resistance in the implementation stage, on the other hand, it will intensify students’ overall acceptance towards the whole project (i.e. EAS implementation). The EASA model showed that although the students did not dislike the traditional paper-based assessment, they were eager to see a new form of assignment and exam to address their special needs amid COVID-19.

Established New Forms of Facilitation Measures under EAS. The traditional supporting and promoting measures may face difficulties during the COVID-19 as those measures heavily relied on onsite communication and social interaction, such as face-to-face training and physical consultation (Menchaca & Bekele, 2008; Puri 2012). Despite users’ requirement on PEOU was less due to the outbreak of COVID-19, sufficient measures to facilitate users’ adoption of a new ICT application could also encourage their usage towards the EAS (e.g., Alhabeeb & Rowley, 2018). However, the facilitation measures should be carefully selected as improper forms of measure may lead to users’ worry about virus infection. According to the results of the EASA model, it was recommended that the facilitation measures should concentrate on electronical methods, such as e-manual, online training, and online consultation.

Conclusions

The outbreak of COVID-19 had necessitated universities to employ ICT in educational activities due to the lockdown policy conducted by the government authorities or the self-prevention measures adopted by the organizations aiming to reduce travel and social interaction. The world is still being haunted by the anxiety and afraid of the new wave outbreak of COVID-19, and education activities always mean the gathering of people and close social interaction, therefore, it was important for society, especially the universities to respond timely and properly. However, universities may encounter complicated problems during their transformation to online educational activities, such as the implementation of EAS. Among the various challenges faced by the universities, student’s acceptance and use intention towards a novel ICT application (i.e. EAS) could be the most unpredictable and uncontrollable element. In this study, a conceptual model namely the EASA model was proposed and found to fit well to the data collected by the survey in a local university. The results of the EASA model confirmed PU and PEOU still being the valid factors affecting user’s acceptance of new ICT applications. Meanwhile, the functionality of EAS on risk reduction through diminishing travel and social interaction amid the epidemic was found to overweight other exogenous factors and significantly affect student’s intention of applying the EAS. The research model further revealed that the traditional facilitation measures during the initial implementation of an ICT application should adjust accordingly in order to address user’s particular needs under specific circumstances, which was to reduce onsite interaction and conduct educational activities via the internet. It could be expected that the wave of e-assessment transformation would continue even in the post COVID-19 world due to its tremendous benefits, thus, it is valuable to understand the investigation method to examine user’s acceptance and usage intention for the EAS. Therefore, this study could be applied to examine user’s acceptance towards other forms of novel ICT application during a pandemic if it is capable to shift communication and activities from onsite to online.

The study was conducted in a developed city and the proposed model assumed that the devices required for the adoption of EAS were affordable for students, however, this assumption may not be valid in the developing regions or countries. This was the major limitation of the study. In addition, this study mainly focused on the views and concerns of students who did not have EAS experience. However, the opinions of students with EAS experience were also important for the continuous improvement of EAS, and thereby should not be overlooked. Lastly, teachers, as another important stakeholder of the EAS, their views and concerns would also be valuable and worth to be examined and understood. Therefore, in the future study, teacher’s use intention towards an EAS should be investigated and compared with that of students. In this case, a comprehensive understanding could be achieved to further guarantee a successful implementation and enhancement of EAS.