Special issue on the current trends in E-learning Assessment
- 360 Downloads
In this full review paper, the recent emerging trends in E-learning Assessment have been reviewed and explored to address the recent topics and contributions in the era of Distance Education. This includes a set of rigorously reviewed world-class manuscripts addressing and detailing state-of-the-art, frameworks and techniques research projects in the area of E-learning Assessment, using different approaches such as Blockchain, Gamification, Process Mining, among others. Based on this systematic review, we have put some recommendations and suggestions for researchers, practitioners and scholars to improve their research quality in this area.
Introduction to the domain of research
Assessment is an essential element in a higher education e-learning scenario (Jafar Jalali et al. 2018; Daradoumis et al. 2013; Lara et al. 2014; Bulut 2019). E-learning is a form of distance learning that is completely virtualized through an electronic channel (medium), like the Internet, for example, using learning process support tools (Wong et al. 2019; Ng 2016; Barana and Marchisio 2016; Lizcano et al. 2019; Aljawarneh 2019; Esposito et al. 2018).
In e-learning environments, parties involved in the teaching–learning process do not share a physical space of interaction, which can give rise to a spatial and temporal gap that poses interesting challenges for assessing students’ acquisition of knowledge and skills (Daradoumis et al. 2013).
Assessment for e-learning requires effort on the part of the educational community to propose methods, strategies, and procedures in order to achieve effective and efficient processes (Bulut 2019). In particular, there are many open problems in E-learning Assessment related areas such as self-assessment (Wong et al. 2019; Dabowsa et al. 2018; Aljawarneh and Vangipuram 2018; Elberkawi et al. 2016; Aljawarneh et al. 2016a, c; Radhakrishna et al. 2017; Aljawarneh et al. 2007), peer-assessment (Ng 2016), automated assessment (Barana and Marchisio 2016), among others.
Machine learning and information security techniques (Radhakrishna et al. 2017; Aljawarneh et al. 2016a, b; Aljawarneh et al. 2007; Radhakrishna et al. 2016, 2018; Aljawarneh 2017; Kalpana et al. 2016; Elkobaisi et al. 2015; Aljawarneh et al. 2017) and approaches have been used in assessment for e-learning to improve the learning quality for higher education.
The purpose of this paper is to describe some important papers that authors have contributed to this special issue with respect to some of the areas mentioned above, which constitute the most advanced and recent vision on the topic of E-learning Assessment so far.
In this introductory paper, a related work includes the summaries of the selected papers have been explored in “Related work: the selected papers” section. “Discussions and recommendations” section includes a set of recommendations for researchers, practitioners and scholars to improve their research quality in this area. In “Conclusions” section, the conclusions have been dawn.
Related work: the selected papers
The purpose of this special issue is to present original contributions of studies on the design, analysis, and implementation of assessment in e-learning environments to deepen understanding of the performance and learning outcomes of individuals and groups, contribute new knowledge, and encourage innovation. We have included papers that include discussions of the implementation of software and/or hardware approaches but that also focus on the implications for the improvement of learning assessment. Priority has been given to papers that demonstrate a strong grounding in learning theory and/or rigorous educational research design. All accepted works include an exhaustive validation and include extraordinarily new ideas in the area.
The special issue includes 10 papers, which have been subject to a rigorous peer-review process. Each paper has been reviewed by two independent experts. The rest of this section includes a summary of the selected papers.
In the paper “A Supervised Learning Framework: Using assessment to identify students at risk of dropping out of a MOOC”, Monllaó et al. propose a method o ease the evaluation and usage of prediction models in Moodle. They abstract out the most relevant elements of prediction models and develop an analytics framework for Moodle. Apart from the software framework, they also present a case study model which uses variables based on assessments to predict students at risk of dropping out of a Massive Open Online Course (MOOC) that has been offered eight times from 2013 to 2018, including a total of 46,895 students. A neural network is trained with data from past courses and the framework generates insights about students at risk in ongoing courses. Predictions are then generated after the first, the second, and the third quarters of the course. The average accuracy that they achieve is 88.81% with a 0.9337 F1 score and a 73.12% of the area under the ROC curve.
The paper “Psychological Impact of E-learning on Social Network Sites: Online Students’ Attitudes and their Satisfaction with Life” by Vate-U-Lan, reports on the findings of a study pertaining to the psychological impact of e-learning on social network sites. The findings have resulted by means of a correlational analysis between attitude towards e-learning on social network sites and satisfaction with life of students experienced with e-learning experiences. It was based on an online survey of 607 valid responses with e-learning experiences gathered from 896 online respondents. The gender profile was balanced (males 50.7% and females 49.3% respectively). The analysis found that students who had experience of e-learning on social network sites also had a positive score on the Satisfaction with Life Scale (females 4.40 out of 6, SD = 0.91 and males 4.38 out of 6, SD = 0.9). The attitudes towards e-learning were also positive (female 4.34 out of 6, SD = 1.0 and male 4.12 out of 6, SD = 1.2). The relationship between e-learning attitudes and satisfaction with life was investigated using the Spearman Rank Order Correlation (rho) statistic. The research data shows a significant positive association between attitudes towards e-learning and satisfaction with life by females and males.
In “Effects of User Perceptions of SAP ERP System on User Learning and Skills”, Shim & Shim present a study that examines the effects of playfulness and anxiety as perceived by users in relation to SAP enterprise resource planning (ERP) system on users’ learning of business processes and users’ skills to use the system. Data was collected from a survey of college students who took a course on business process integration with ERP system where students used SAP ERP system to complete course works on business processes. System playfulness is found to have a small positive effect on user learning and skills without any controls but the positive effect disappears after controlling for gender and prior experiences. System anxiety is found to have a large negative effect on both user learning and skills. These results suggest that enhancing the playfulness of SAP ERP system can help improve the user’s learning of business processes and skills to use the system, but that reducing the anxiety of the system is far more important in improving the user’s learning of business processes and the user’s skills to use the system.
In the paper “Reviewing and exploring innovative ubiquitous learning tools in higher education”, by Aljawarneh, various e-learning tools like Wikipedia, MOODLE, Web 2.0, Web 3.0 and Blackboard have been evaluated. They also comment on key aims regarding each tool and investigate the disadvantages and advantages. Based on this analysis, a global view regarding the current as well as future tendencies pertaining to ubiquitous e-learning tools is obtained and thus possible key comments are provided for employing e-learning tools like MOODLE, Web 2.0 and Web 3.0 in the classroom. Based on their teaching experience, MOODLE was found to be efficient in the development of e-learning. MOODLE was favored by a majority of authors and practitioners rather than Blackboard. However, MOODLE cannot be considered a fully pure social software since it does not include social networks. In this review, the scope of employing ubiquitous learning environments has been presented in higher education contexts. However, it increases the requirement for transparent research that shows practical implications to generalize future development processes. Moreover, it was shown that e-learning 3.0 is one amongst the key trends employing Web 3.0 tools for social learning. Also, on the Internet, quick incorporation of new services into existing applications like integrating Wiki with Web 3.0 can be done easily. The primary risk here would be the fact that lecturers and students are not fully aware that these web services are not controlled by their universities. Since these servers have been installed in many different countries, the principles and privacy laws vary from country to country.
In the work “Process Mining for self-regulated learning assessment in eLearning”, Cerezo et al.’ objective was to discover students’ self-regulated learning processes during an eLearning course by using Process Mining Techniques. They applied a new algorithm in the educational domain called Inductive Miner over the interaction traces from 101 university students in a course given over one semester on the Moodle 2.0 platform. Data was extracted from the platform’s event logs with 21,629 traces in order to discover students’ self-regulation models that contribute to improving the instructional process. The Inductive Miner algorithm discovered optimal models in terms of fitness for both Pass and Fail students in this dataset, as well as models at a certain level of granularity that can be interpreted in educational terms, which are the most important achievement in model discovery. They can conclude that although students who passed did not follow the instructors’ suggestions exactly, they did follow the logic of a successful self regulated learning process as opposed to their failing classmates. The Process Mining models also allow examining which specific actions the students performed, and it was particularly interesting to see a high presence of actions related to forum-supported collaborative learning in the Pass group and an absence of those in the Fail group.
In the paper “E-learning Assessment for Tourism Education LISREL Assisted Intercultural Tourism Perception and Data Integrated Satisfaction Perspectives”, Qiu & Qi state that with the intensification of global integration, education internationalization has become one of the important indicators for evaluating the level of higher education development in a country. From the total income of tourism in recent years and its contribution to China’s GNP, it can be seen that the tourism industry has a strong development momentum. Tourism culture has become a mobile culture of which essence is cross-cultural tourism. Therefore, studying tourism from an intercultural perspective is an inevitable trend under the globalization of international tourism. Meanwhile, the contribution of tourism education talents is an important guarantee for the sustainable development of tourism. The dominant growth of the tourism industry has undoubtedly promoted the in-depth development of tourism education. Therefore, the development of tourism education and tourism industry should be a dynamic development pattern which promotes each other. This article regards the relationship between perception and satisfaction as the starting point and introduces the LISREL model into cross-cultural tourism research. This paper constructs a cross-cultural tourism research model and studies the relationship between perception and satisfaction, which can also be used to study other aspects of cross-cultural tourism.
In the work “Blockchain-based approach to create a model of trust in open and ubiquitous higher education”, Lizcano & Lara evaluate the benefits of the blockchain (or distributed ledger) technology and advocates a decentralised model of confidence for transactions based on an academic crypto currency. In this approach blockchain is used to manage transactions of content, teaching and competencies, assessed by consensus by students, trainers and employers, to eliminate once and for all the “gap” between the academic world and the working world. This paper aims to address the current challenges of an increasingly dispersed, open and ubiquitous higher education. The proposed model can be implemented in any training institution to adapt its teaching to the specific needs of professional profiles validated by employers in the sector. This model has been validated by means of a prototype with more than acceptable results.
The paper “Web Gamification with Problem Simulators for Teaching Engineering”, by De la Peña et al., present the development of a web board game with six categories of problems for the Industrial Systems Optimization Techniques subject, which is part of the Industrial Organization Engineering curriculum at Madrid Open University. The game relies on case-study simulators for six categories of problems in such way that the cases presented to the students are always different. Students receive instant feedback about the accurateness of their response as well as the correct solution. The results of the experience, based on data obtained and surveys carried out, indicate that the board game is dynamic and motivational as well as academically encouraging.
The paper “Investigating Variation in Learning Processes in a FutureLearn MOOC”, by Rizvi et al., investigates the intersection between learning design and the temporal process of engagement in MOOC courses. This study investigates the detailed processes of engagement using educational process mining (EPM) in a FutureLearn science course (N = 2086 learners) and applying an established taxonomy of learning design to classify learning activities. The analyses were performed on three groups of learners categorized based upon their clicking behaviour. The process-mining results show at least one dominant pathway in each of the three groups, though multiple popular additional pathways were identified within each group. All three groups remained interested and engaged in the various learning and assessment activities. The findings from this study suggest that in the analysis of voluminous MOOC data there is value in first clustering learners and then investigating detailed progressions within each cluster that take the order and type of learning activities into account. The approach is promising because it provides insight into variation in behavioural sequences based on learners’ intentions for earning a course certificate. These insights can inform the targeting of analytics-based interventions to support learners and inform MOOC designers about adapting learning activities to different groups of learners based on their goals.
Finally, Fernández-Chamorro et al., authors of the paper “Assessing prior knowledge of statistics in an online university: a case study”, analyze the relation between entry path to a degree and the prior statistical competence of the students on the Statistics and Probability course at an online University. This was done by assessing prior knowledge via the administration of a pretest on the course analyzed. The sample includes 108 students from different Faculties of an online University. The statistical analysis shows, on the one hand, that students have certain difficulties with some concepts related to Probability and Descriptive Statistics, and on the other, the influence of the entry path on the understanding of these concepts.
Discussions and recommendations
More importance should be given to E-learning Assessment in Data Science projects, since Assessment-related data have proved to be very useful to extract valuable knowledge and vice versa, that is, Data Science techniques have proved to provide important information to better understand Assessment processes.
Formal education is not the only environment where E-learning Assessment play a crucial role, since there are other domains (in company learning, tourism-related learning) where it is also important. Therefore, those domains should be explored in search of new findings with respect to E-learning Assessment.
It is important to take into account that nowadays E-learning Assessment takes place in a scenario of ubiquitous education and therefore new approaches, such as Blockchain technology, must be used to deal with this issue.
E-learning Assessment is a multidisciplinary area of importance in Social Network Analysis, Gamification strategies or Prior Knowledge evaluation, so some more research should be done in those areas.
In this special issue, 10 selected papers have been included that present important advancements in the area of E-learning Assessment. The selected papers present interesting studies about the development of this area, works about promising existing technologies and outstanding research about theories and methods that will play an important role in the future of this discipline.
As guest editors, we are aware of the fact that this issue cannot completely cover all the advancements in this area, but we expect that this special issue can stimulate further research in the domain of E-learning Assessment.
We would like to thank the referees who have reviewed the papers for providing a valuable feedback to authors. We would also like to thank the authors for their manuscripts that represent an important contribution to the existing knowledge in the area. Finally, we would like to thank Professor Stephanie L. Moore, Editor-in-Chief, Professor M. J. Bishop, former Editor-in-Chief, and editorial assistants of Journal of Computing in Higher Education for their support during the preparation of this issue.
- Aljawarneh, S. A. (2019). Reviewing and exploring innovative ubiquitous learning tools in higher education. Journal of Computing in Higher Education. https://doi.org/10.1007/s12528-019-09207-0.
- Aljawarneh, S. A., Jaradat, R., Maatuk, A. M., & Alhaj, A. (2016a). Gene profile classification: A proposed solution for predicting possible diseases and initial results. In International conference on engineering & MIS (ICEMIS) (pp. 1–7).Google Scholar
- Aljawarneh, S., Laing, C., & Vickers, P. (2007). Verification of web content integrity: A new approach to protect servers against tampering. In: The 8th annual postgraduate symposium on the convergence of telecommunications, networking and broadcasting, 28–29 June 2007, Liverpool.Google Scholar
- Aljawarneh, S., Radhakrishna, V., Kumar, P. V., & Janaki, V. (2016c). A similarity measure for temporal pattern discovery in time series data generated by IoT. In International conference on engineering & MIS (ICEMIS) (pp. 1–4).Google Scholar
- Bulut, S. (2019). Assessing online learners’ academic self-efficacy in a symbiotic learning environment. http://doi.org/10.2139/ssrn.3370615.
- Dabowsa, N. I. A., Amaitik, N. M., Maatuk, A. M., & Aljawarneh, S. A. (2018). A hybrid intelligent system for skin disease diagnosis. In Proceedings of 2017 international conference on engineering and technology, ICET 2017 (Vol. 2018–Jan). https://doi.org/10.1109/ICEngTechnol.2017.8308157.
- Daradoumis, T., Bassi, R., Xhafa, F., & Caballé, S. (2013) A review on massive e-learning (MOOC) design, delivery and assessment. In 2013 eighth international conference on P2P, parallel, grid, cloud and internet computing (pp. 208–213), Compiegne.Google Scholar
- Elberkawi, E. K., El-firjani, N. F. M., Maatuk, A. M., & Aljawarneh, S. A. (2016). Usability evaluation of web-based systems: A new method and results. In International conference on engineering & MIS (ICEMIS) (pp. 1–5).Google Scholar
- Elkobaisi, M. R., Maatuk, A. M., & Aljawarneh, S. (2015). A proposed method to recognize the research trends using web-based search engines. In ACM international conference proceeding series (Vol. 24–26–Sept). https://doi.org/10.1145/2832987.2833012.
- Lizcano, D., Lara, J. A., White, B., & Aljawarneh, S. (2019). Blockchain-based approach to create a model of trust in open and ubiquitous higher education. Journal of Computing in Higher Education. https://doi.org/10.1007/s12528-019-09209-y.
- Radhakrishna, V., Aljawarneh, S. A., Kumar, P. V., & Choo, K.-K. R. (2018). A novel fuzzy gaussian-based dissimilarity measure for discovering similarity temporal association patterns. Soft Computing, 22, 1903–1919.Google Scholar
- Radhakrishna, V., Kumar, P. V., & Janaki, V. (2016). Mining of outlier temporal patterns. In International conference on engineering & MIS (ICEMIS) (pp. 1–6).Google Scholar
- Wong, T.-K., Xie, H., Zou, D., Wang, F. L., Tai Tang, J. K., Kong, A., et al. (2019) How to facilitate self-regulated learning? A case study on open educational resources. Journal of Computers in Education. https://doi.org/10.1007/s40692-019-00138-4.