In contemporary organizations, multiple variants of the same business process are often considerable. Such business process variability has caused considerable challenges, both while modeling processes and in their execution. In order to develop a new approach to managing process variants, or extend an existing one, in this research, we review the state of the art in a particular area: online examination processes. We show to what extent variability should be considered in exam processes, whether this is due to special legal restrictions and regulations, different exam frameworks, or even different technical infrastructure. This could be the foundation for developing an approach to managing process variability in the field of e-assessment. Initial findings indicate that examination processes have many similarities, but also considerable differentiation. Therefore, there an appropriate model needs to be developed in order to manage variability in e-assessment and the developed approach must then be validated in identifying faculties. This paper constitutes a first step in this direction.
- Process variability
- Online examination
- E-assessment process model
In today’s dynamic world, there are often multiple variations of identical business processes. Rosemann and colleagues noted, for instance, that SAP offers 27 different industry solutions with corresponding business process reference models (Rosemann and van der Aalst 2007). These models usually include decisions in the workflow, which could be made before executing process instances. It is impossible for both variants of such decisions to coexist in a certain domain or process context. However, conventional modeling approaches do not offer the opportunity to differentiate between such decisions and regular decisions during the runtime of a process instance. An important element of controlling the variability in business process models is to separate the usual runtime decisions from decisions at configuration time, called variation points. The results of such steps are complex artifacts. The number of artifacts makes the manageability of related workflows more complex. Based on the reviewed literature, organizations take different approaches to managing process variability (Ayora Esteras 2012). The existing approaches have limitations in terms of supporting an entire set of elements like control flow, rules, and legal regulations during the construction and execution of business processes.
For this paper, we chose an educational field as an example of process variability, in order to observe effects and causes of variability. The goal is to have a comprehensive overview to address the problem of process variability in online examination processes at German universities and show the necessity of managing it through an appropriate business process model. To achieve this goal, we set out the state of the art in research regarding the process variability of online examinations from different perspectives. This could be the basis for developing a process model to manage existing variability in this field. We evaluate existing approaches and concepts in the context of e-assessment in the literature and clarify current accreditation processes in educational fields. This will help to identify to what extent existing examination procedures reflect variability and demonstrate the necessity of developing a unified e-assessment model to cover all variability in the learning-teaching process.
The paper is organized as follows: After illustrating motivation through existing studies in the following section, the research method is explained. After exploring the literature and data collected in the identified domains, the results are evaluated. Finally, the need for further work is explained in the conclusion.
In this section, we have a close look at process variability and its challenges, identifying the importance of variability management in the organizations. Basically, process models capture an organization’s activities in achieving certain business goals. The aim is to better understand the process, its implementation, and its execution in a workflow (Becker et al. 2013). However, there are a lot of possible variants for one process. Such business process variability creates considerable challenges in process modeling and execution:
the variants may be modeled in a highly redundant way, so there are many identical or similar parts;
there is no strong relation between the variants, so there is no support for automatically combining existing variants;
manual modeling in every single process variant would be time-consuming and error-prone.
In recent years, the proper management of business process variability has been the subject of numerous scientific studies. A very comprehensive survey article about business process variability can be found in Valenca et al. (2013), which describes more than 80 primary sources. Based on this study, significant numbers of variability approaches exist, where each one addresses different issues in terms of process variability. Valenca and colleagues observed 57 new approaches to different aspects of variability in processes (Valenca et al. 2013). They divided these references into five categories: business process configuration, to capture an instance of the reference model; business process correctness, to semantically support correction of the process model; business process flexibility, to change process models fast and easily; business process modeling, to visualize variability in process models; business process similarities, to investigate differentiations between business process models. It is argued that only 30% of solutions are practically evaluated through case studies and surveys, especially of industry: The lack of empirical studies in process variability is considerable, with implications for executing process variability (Valenca et al. 2013).
In the public sector, as in business, processes have a lot in common, but significant differences due to the local conditions and legal regulations are considerable. Vogelaar and colleagues analyze and compare the different processes of ten Dutch municipalities which are found to vary in terms of classical standardization processes (Vogelaar et al. 2012).
In education, Arnold and Laue studied controllability of variability in examination process models (Arnold and Laue 2014). They investigated six different courses at three different universities in the German Federal State of Saxony, to achieve better comparability. Based on this research, considerable variability in examination processes exists, even in one university between different fields of study. The authors tried to provide a solution based on existing variability approaches, in order to manage examination processes. They argued that appropriate process variability modeling requires modeling skills and significant experience in the identified domain (Arnold and Laue 2014).
We focus on online examination processes in higher education, presenting the state of the art in three different domains and observing existing process variability in order to gain a comprehensive overview of process variability in this field, highlighting the need for this to be managed.
3 Research Method
The purpose is to evaluate the existing variability in higher educational e-assessment processes, as a basis for further research into variability management in this field.
The state of the art is identified in five phases (Cooper 1998):
Problem formulation: The academic goal and research relevance are defined;
Literature search and data collection: Literature and data related to the problem formulation are identified;
Literature evaluation: The acquired literature is assessed for relevance and categorized;
Interpretation: The results are analyzed and interpreted;
Presentation: The results are presented in a suitable fashion.
Assessments are an important part of the educational cycle (Ferrão 2010) and have a great impact on the learning process. They provide valuable information about the effectiveness of a study course in increasing the students’ knowledge (Primiano et al. 2004). An appropriate assessment process is not only important in terms of teaching and learning, but also for accreditation processes and educational standards (Ferrão 2010). Recent developments in e-learning can be seen as an accelerator to developing e-assessment alternatives. It is therefore becoming more important to develop methods for e-assessment and to gain feedback on learning and teaching (Sangi and Malik 2007). Furthermore, Dermo has shown that e-assessment can offer different forms of assessment with immediate feedback to both students and lecturers, so it can be recognized as a complementary tool in the learning framework (Dermo 2009). Exam regulation documents, which are the basis for accreditation processes in higher educational institutions, have a lot in common. But in some points, they differ from one university to another or even from one course to another within the same institution. Therefore, there is variability in assessment processes, which is an obstacle to developing a unified process model for e-assessment.
We reviewed the state of the art in three identified domains within e-assessment:
designing study courses and e-assessment concepts;
In the following, the results of our literature review and data collection in each domain are explained separately.
3.1 Literature Search and Data Collection in Three Domains
3.1.1 Domain: IT Approaches
Data sources: AISel and EBSCO
Research period: 2000–2013
Search terms and keywords: e-assessment, education, online examination, e-test, computer-based exam (in abstract and title)
Number of related articles:
AISel: 51 after reviewing and removing doublets and non-related articles: 14
AISel: 51 after reviewing and removing doublets and non-related articles: 14
EBSCO: 62 after reviewing and removing doublets and non-related articles: 18
Based on the reviewed articles, a main classification can be recognized in the context of e-assessment in higher education:
pedagogical issues (educational view)
Different kinds of terms and concepts are used, based on pedagogical and technical approaches: Each one addresses one or more aspects of e-assessment. These issues are summarized in the following Tables 1 and 2.Footnote 1
Most of these references include multiple issues from the technical and pedagogical perspectives. These issues are connected and cannot be separated.
Jacob and colleagues deploy an e-assessment tool, the Black Board Learning System (BBLS), as a comprehensive e-learning software to facilitate continuous assessment and evaluate its effects on learning processes (Jacob et al. 2006). It reveals that the biggest advantage of e-assessment in this system is immediate feedback, which bolsters the formative assessment.Footnote 2 One weakness of the system is the lack of the automatic evaluation of essay-writing exams. Kehily analyzed the impact of a Web-based e-learning platform that can support effective teaching (a course management system for lecturers) and formative assessment (a computer-assisted learning tool for students) in a case study (Kehily 2011). Venkatraman developed a four-step student-centered approach to an effective e-learning process and, in a case study of information system (IS) courses, evaluated this approach for different assessment methods including individual, group, peer, and self-assessment (Venkatraman 2007). These four steps are:
understanding the students’ learning style and their learning expectations;
identifying suitable assessment models;
designing a set of assessments;
evaluating the impact of the assessment on the learning process.
Dermo evaluates the possible risks in planning e-assessments such as computer stress, fairness of choosing questions randomly from a bank, accessibility, and the contribution of e-assessment to students’ learning, through six dimensions in a case study (Dermo 2009). These six dimensions are: affective factors, reliability and fairness, validity, security, practical issues, and teaching and learning terms, which are a mixture of pedagogical and technical issues. Daly and colleagues argue that existing e-assessment solutions focus on developing technical and infrastructural issues more than educational aspects (Daly et al. 2010). McCann identifies different factors which affect real implementations of e-assessment systems based on two IS theories: Roger’s theoryFootnote 3 and Eckel and Kezar’s theoryFootnote 4 (McCann 2010).
3.1.2 Domain: Designing Study Courses and E-Assessment Concepts
Data sources: German university homepages
Research period: 2000–2013
Search terms and keywords: e-assessment, online examination, project, computer-based exam, e-exam, e-test
In this domain, we began by finding some case studies of e-assessment or online examination at German universities. In order to have an appropriate sample, we selected universities which conducted online examinations or had a project to recognize a unified approach or process regarding e-assessment in Germany. Different kinds of projects in terms of computer-based examinations have been in progress from the year 2000 onward. Table 3 summarizes all these projects with their functionality and their relation with exam regulation documents.
Of all the universities studied, only the University of Duisburg-Essen proposes a process model for implementing online examination. Proposing such a process model for online exams has the following advantages:
improves the speed of feedback to students;
motivates students and lecturers to increase computer skills through the integration of multimedia elements such as audio and video, and using complex digital systems;
significant time saving for lecturers by automatic correcting process.
By reviewing the exam regulations and conditions, it becomes obvious that there is no identified exam process model in the administrative processes at different universities. A process model not only supports understanding the complexities of processes properly, but also contributes to advancing and improving defined processes (Irani et al. 2000). Therefore, in order to understand the examination workflows carried in the universities, a process model is required to analyze exam processes comprehensively.
3.1.3 Domain: Accreditation Process
Data sources: Accreditation agencies authorized by AISelFootnote 5 and EBSCO
Research period: 2000–2013
Search terms and keywords: e-assessment, education, online examination, computer-based exam, accreditation process, accreditation criteria (in abstract and title)
As a definition, accreditation is a criteria-based procedure to assess and evaluate the admissibility of an educational program in terms of quality (Gorgone 2006; Impagliazzo and Gorgone 2002; Reichgelt and Yaverbaum 2007). The main goal of accreditation is to assess the educational quality of an academic program to ensure that it meets certain quality standards, called accreditation criteria (Reichgelt 2007).
Based on the European Network for Quality (ENQA), each educational program should fulfill the minimum in the following set of requirements to be accredited:
requirements and objectives;
monitoring, analysis, and overview.
The assessment or examinations are placed in the teaching–learning process, which is the most complex aspect of this model because it includes a mixture of technical, pedagogical, and social competences and, furthermore, there is a great freedom to manage courses in order to achieve identified objectives.
According to Reichgelt and colleagues, there are two accreditation types:
institutional accreditation is applied to an academic institution, like an entire university;
specialized accreditation is applied to a subunit in an institution and includes two levels: program accreditation focuses on the content of the program, while administrative accreditation concentrates on the administrative process in a subunit of an institution (Reichgelt and Yaverbaum 2007).
The authors explain that there are two main approaches to accreditation processes. The first one is the input-based approach which measures various minimal standards through a checklist based on the learning-input processes such as curriculum, teaching resources, library, laboratory, and other facilities. The second is the outcomes-based approach, which considers the program’s outcomes, such as the institution’s educational objectives and student learning. Reichgelt and colleagues argue that a significant shift from the input-based to the outcomes-based approach has occurred in recent years and academic institutions attempt to conform themselves with outcome criteria (Reichgelt and Yaverbaum 2007).
3.2 Accreditation Processes in Germany
In Germany, the federal states are responsible for accreditation processes and at least 11 authorized accreditation agencies in different fields of education (medical, natural science, engineering, economics, etc.) are in operation at present.Footnote 6 Educational accreditation in Germany is based on two issues:
standards of study courses and degrees, which are based on regulations for study and examinations;
accreditation in teaching, which is based on quality improvement measures.
The requirements in the examination accreditation process are as follows:
examinations are coordinated so that students have sufficient time to prepare themselves;
it must be possible to move directly from the bachelor’s degree to the master’s degree without loss of time;
the form of examination is laid down in the description of each module;
examinations should not cause extensions to the period of study;
the evaluation criteria are transparent for both lecturers and students;
the degree program ends with a final thesis;
it is checked whether students are capable of oral discussion in their specialist area;
the supervision of the final thesis is subject to precise regulations in the curriculum.
The archived documents in the accreditation process are test results, drop-out rates, any quantity results of examinations, as well as feedbacks from the courses.
This survey of three domains indicates that multiple perspectives exist, which cause variability in performing and evaluating e-assessment processes. It is therefore essential to develop an appropriate model to account for this variability in online examination processes.
4 Literature and Results
This literature review was performed in order to demonstrate existing process variability in e-assessment in different domains of higher education, which occurs for different reasons. The results of each domain and its relations to process variability are analyzed and presented in the following.
4.1 Evaluation of IT Approaches
Based on reviewed IT approaches, it can be argued that due to the great advantages of and positive impacts of online exams on learning processes, there is a considerable movement from traditional to electronic assessment in academia today. Furthermore, studiesFootnote 7 show that different issues, from pedagogical to technical and even social matters, cause process variability in higher education examinations. To identify a way of managing this inherent variability and to construct unified e-assessment procedures, it seems necessary to have a cross-functional view of e-assessment projects. Some of the issues related to e-assessment are: the impacts on student learning; effects on the teaching method; formative and collaborative e-assessment; immediate feedback and legal issues; evaluating possible risks in planning such as computer stress or fairness impression; developing infrastructural issues, such as security and question banks.
4.2 Evaluation of Study Courses
Thirty German universities from different federal states and different study courses were reviewed.Footnote 8 The results show that e-assessment or online examination projects are currently in progress in 18 universities, but just nine are performing such assessments in practice. It should be noted that although online examinations are in use at some universities, there exists considerable process variability, too, which makes it difficult to extract a unified platform in this area.
Furthermore, legal conditions in regulation documents create practical limitations on performing online examinations. In order to make it an acceptable form of assessment, electronic examination should be referred to in a paragraph or even a sentence in the corresponding exam regulation document. Based on reviewing examination rules documents in all these universities, Table 4 reveals that only three German universities have a paragraph stating that online examination is an admissible examination form.
4.3 Evaluation of Accreditation
The results revealed that accredited educational programs are subject to a variety of quality criteria. These criteria depend on the level and the goals of the courses. Based on the Information Model from the European Network for Quality (ENQA), assessment and examination processes are placed within the teaching–learning process, which is one of the complex parts of accreditation. It involves different aspects such as pedagogical, technical, legal, and even social issues.
No framework for examination forms (traditional or computer-based) was identified in this study of the German accreditation process. In exam regulation documents, assessments were only described in the terms listed at the end of Sect. 3.3 above. The form of examinations is not restricted by the accreditation process, but is under the authority of the educational systems and based on the identified objectives of courses. This causes process variability from one educational institution to another.
4.4 Summary of the Results
For clarity, we summarize the results of the three domains in Table 5.
In sum, exam processes show variability for the following reasons:
variability in legal restrictions and regulations in different educational institutions;
variability in exam framework, not only in different universities but also in one university between different courses;
variability in study course design;
variability in technical infrastructure for online examination processes.
Based on the obtained results, it can be argued that an appropriate design for variability management has to be aligned with the specified domain of projects and conditions to cover all variability within the identified domain. There is only one process model for performing online examinations in some study courses at University of Duisburg-Essen, which could be an appropriate model for conducting online examinations at German universities.
We studied existing process variability (effects and causes) in online examination processes from different perspectives. We found that to manage process variability, an appropriate business process model is necessary to cover all examination processes and features of educational institutions.
According to the existing literature, one approach to process variability management is the single model approach, which models all known variants of the process in one common model (Hallerbach et al. 2010; Kumar and Yao 2012). The alternative is to model every variant of a single model, which is called the multi-model approach. The latter models will have a simpler structure (Hallerbach et al. 2010; Kumar and Yao 2012). Some advanced modeling approaches explicitly deal with families of process models, such as configurable event-driven process chains (C-EPCs) (Rosemann and van der Aalst 2007), PROcess Variants by OPtions (PROVOP) (Hallerbach et al. 2010), ConDec (Pesic and Aalst 2006), and feature modeling, which is normally used in software engineering. Each of these approaches has their own advantages and drawbacks but there is a lack of adaptability between different approaches (Valenca et al. 2013).
The next step in this research is to conduct a comprehensive overview of available variability approaches, to identify which of them are more appropriate for online examinations in higher education. Further research is needed to evaluate how existing process variability approaches can be compatible with exam processes, and to what extent existing process variability in this field can be controlled and managed.
5 Conclusion and Further Work
The aim of this research is to demonstrate existing variability in examination processes and emphasize the need for variability management. This could be the basis for developing a new approach to business process variability, or extending an existing one. To reach this goal, we concentrated on online examination processes in higher education.
As a preliminary stage, we performed a literature review in three different domains: IT approaches; concepts; course design and the accreditation process. This paper demonstrates the important role of business process management in improving and promoting the design of e-assessment processes.
Combined with this literature review, an analysis of current e-assessment case studies and projects in German universities yielded the following results. Although different kinds of projects under the name of e-assessment or online examination are in progress in German universities, the variability in these processes is recognizable. In other words, similar processes exist, but distinctions and variability are observable as well. Furthermore, the review of the exam regulation documents for various study courses at different universities revealed that these regulations do not yet mention an acceptable framework for performing e-assessments.
Process variability exists in e-assessment at German universities. It is necessary to manage this variability through an appropriate business process model to support online examination procedures.
This paper describes research in progress which clarifies and identifies the necessity of developing an existing approach to manage and control the process variability in university e-assessment. The next step is to study process variability management to identify an appropriate model in the context of e-assessment. This could be followed by the development of a prototype for the identified approach and finally the validation of the developed method.
Numbers in brackets refer to the references in Appendix 1.
Formative assessment encourages deeper engagement with learning and is a motivation and progressive force in learning. The key element of formative assessment is feedback.
It identifies five variables to demonstrate how and why new ideas are adopted: relative advantage, compatibility, trialability, observability, and complexity (McCann 2010).
It identifies five core strategies that explain change across institutions: senior administrative support, collaborative leadership, flexible vision, staff development, and visible actions (McCann 2010).
Association for Information Systems eLibrary.
All these studies are summarized in a table in Appendix 1.
These universities are listed in Appendix 2.
Arnold, O., Laue, R.: Untersuchungen zur Beherrschbarkeit von Variabilität in Prüfungsprozessmodellen deutscher Hochschulen. In INFORMATIK 2014, 22–26. September 2014: GI-Edition—Lecture Notes in Informatics (LNI), P-232, S. 1135–1148. Bonner Köllen Verlag, Stuttgart (2014)
Accountants find e-assessment adds up., e-Learning Age news, p. 4 (2011)
Attali, Y., Burstein, J.: Automated essay scoring with e-rater® V. 2. J. Technol. Learn. Assess. 4(3) (2006)
Ayora Esteras, C.: Modeling and Managing Variability in Business Process Models (2012)
Becker, J., Bergener, P., Kleist, S., Pfeiffer, D., Raeckers, M.: Business process model-based evaluation of ICT investments in public administrations. In: AMCIS 2008 Proceedings. Paper 353 (2008)
Becker, J., Eggert, M., Bruning, D., Saat, J.: Applicability of business process model analysis approaches—a case study in financial services consulting. In: Wirtschaftsinformatik Proceedings 2013. Paper 74 (2013)
Boyle, A.: Some forecasts of the diffusion of e-assessment using a model. Innov. J. 15(1), 2–30 (2010)
Boyle, A., Hutchison, D.: Sophisticated tasks in e-assessment: what are they and what are their benefits? Assess. Eval. High. Educ. 34(3), 305–319 (2009)
Braun, G., Crable, E.: Administering exams electronically: issues, techniques, and assessment. In: AMCIS 1998 Proceedings. Paper 353 (1998)
Campbell, A.: Performance enhancement of the task assessment process through the application of an electronic performance support system (2008)
Charoen, D.: Challenges and opportunities of elearning: a case study of higher education in Thailand. In: AMCIS 2009 Proceedings. Paper 132 (2009)
Cooper, H.M.: Synthesizing Research: A Guide for Literature Review. Sage, Thousand Oaks (1998)
Coyle, E.: Engineering Education in the US and the EU. Chapter 5. In: Engineering in Context (2009)
Daly, C., Pachler, N., Mor, Y., Mellar, H.: Exploring formative e-assessment: using case stories and design patterns. Assess. Eval. High. Educ. 35(5), 619–636 (2010)
Dascalu, M., Bodea, C.: Challenges in building e-assessment services from project management knowledge perspective. Inter. J. Glob Manag. Stud. Prof. 2(1), 35–50 (2010)
Davis, R.: Business Process Modeling with ARIS. Springer, London (2001)
Davis, R., Brabänder, E.: ARIS Design Platform: Getting Started with BPM. Springer, London (2007)
Dermo, J.: e-Assessment and the student learning experience: a survey of student perceptions of e-assessment. Br. J. Edu. Technol. 40(2), 203–214 (2009)
Dittman, D., Deokar, A.V.: Collaboration process patterns and integrated assessment in e-learning environments. In: MWAIS 2008 Proceedings. Paper 15 (2008)
El-Ashy, A.K.: A methodology using UML & SADT to develop an effective business model in healthcare. Paper 449 (2006)
Ferrão, M.: E-assessment within the Bologna paradigm: evidence from Portugal. Assess. Eval. High. Educ. 35(7), 819–830 (2010)
Gorgone, J.T.: Information systems accreditation: preparation, process, and standards. In: Communications of the Association for Information Systems, Vol. 17, Article 19 (2006)
Gruhn, V., Laue, R.: Approaches for business process model complexity metrics. In: Technologies for Business Information Systems, pp. 13–24. Springer, Heidelberg (2007)
Hall, J.D.: Fourth and Fifth Grade Student Achievement and Possible Relationship to Three Teacher Selection Tools (2010)
Hallerbach, A., Bauer, T., Reichert, M.: Configuration and management of process variants. In: vom Brocke, J., Rosemann, M. (eds.) Handbook on Business Process Management 1, pp. 237–255. Springer, Berlin Heidelberg (2010)
Hallerbach, A., Bauer, T., Reichert, M.: Issues in modeling process variants with provop. In: Business Process Management Workshops—BPM 2008 International Workshops, Milano, Italy, 1–4 September. Revised Papers, pp. 56–67. Springer, Berlin Heidelberg (2009)
Hodgson, P., Pang, M.: Effective formative e-assessment of student learning: a study on a statistics course. Assess. Eval. High. Educ. 37(2), 215–225 (2012)
Impagliazzo, J., Gorgone, J.: Professional accreditation of information systems programs. In: Communications of the Association for Information Systems, Vol. 9, Article 3 (2002)
Irani, Z., Hlupic, V., Giaglis, G.: Business process reengineering: a design perspective. Inter. J. Flex. Manuf. 12(4), 247–252 (2000)
Jacob, S.M., Issac, B., Sebastian, Y.: Impact on student learning from traditional continuous assessment and an e-assessment proposal. In: PACIS 2006 Proceedings. Paper 63 (2006)
Johannsen, F., Leist, S.: Erratum to: Wand and Weber’s decomposition model in the context of business process modeling. Bus. Inf. Sys. Eng. 4(6): 375–375 (2012)
Johnson-Glenberg, M.C.: Embedded formative e-assessment: who benefits, who falters. Educ. Media Inter. 47(2), 153–171 (2010)
Jordan, S., Mitchell, T.: e-Assessment for learning? The potential of short-answer free-text questions with tailored feedback. Br. J. Edu. Technol. 40(2), 371–385 (2009)
Kehily, D.: Review of Dublin Institute of Technology’s E-Learning Platform for Effective Teaching Formative Assessment and Feedback. In: EDUCLEARN 2011, Barcelona, Spain, 4–6 July (2011)
Kumar, A., Yao, W.: Design and management of flexible process variants using templates and rules. Comput. Ind. 63(2), 112–130 (2012)
McCann, A.L.: Factors affecting the adoption of an e-assessment system. Assess. Eval High Educ 35(7), 799–818 (2010)
Pesic, M., Aalst, W. van der: A declarative approach for flexible business processes management. In: BPM’06 Proceedings of the 2006 International conference on Business Process Management Workshops, pp. 169–180 (2006)
Primiano, D.N., Pirjo-Liisa, O., Schade, A.: Accreditation models in higher education: experiences and perspectives. In: ENQA Workshop Report 3. Helsinki (2004)
Reichgelt, H., Yaverbaum, G.: Accountability and accreditation: putting information systems accreditation into perspective. In: Communications of the Association for Information Systems, Vol. 20, Article 27 (2007)
Rosemann, M., van der Aalst, W.M.P.: Configurable reference modelling language. Inf. Sys. Databases Creation Manage Utilization 32(1), 1–23 (2007)
Sangi, N. A., Malik, I.: E-assessment models and methods for student evaluation. In: 21st Asian Association of Open Universities Conference Putrajaya: Malaysia: Kuala Lumpur (Malaysia) University (2007)
Valenca, G., et al.: A systematic mapping study on business process variability. Inter. J. Comput Sci. Inf. Technol. 5(1), 1–21 (2013)
Venkatraman, S.: Student-centred assessments and their impact on learning—an IS case study. In: PACIS 2007 Proceedings. Paper 50 (2007)
Vogelaar, J., Verbeek, H.M.W., Luka, B., van der Aalst, W.M.P.: Comparing business processes to determine the feasibility of configurable models: a case study. In: Business Process Management Workshops, BPM 2011 International Workshops, Clermont Ferrand, 2011. 2012; S. 50–61. Springer, Berlin Heidelberg (2012)
Editors and Affiliations
Appendix 1: Summary of IT Approaches to e-Assessment
Reference (author, year)
F. Braun, 1998
It introduces some web-based tools for online assessment such as Learning space, Web forms, Microtest, and Quizmaker.
It indicates the benefits and limitations of implementing an e-learning system in an educational institution in Thailand based on expert interviews.
It introduces a structure, the collaborative e-learning template (CET), to improve collaborative activities in the learning process such as collaborative assessments through a course.
It indicates different critical factors in distance learning.
Squires et al., 2004
It is about education policy analysis in Arizona University.
Shen et al., 2000
It analyzes the effectiveness of an online collaborative examina-tion process through a questionnaire.
D. Hall, 2010
It introduces some tools to assess teachers.
Jacob et al., 2006
It reveals the importance of assessments in the learning process through different learning approaches. The Black Board Learning System (BBLS) is used as a comprehensive e-learning software platform in the context of continuous assessments.
It proposes an application called the Electronic Performance Support System (EPSS) to enhance the performance of assess-ment tasks.
It uses a web-based platform, DIT learning teaching and technol-ogy center, as a course management system for lecturers and as a computer-assisted learning tool for students. It is a case study in the Dublin Institute.
Attali and Burstein, 2006
It proposes a flexible modeling procedure which can be used to as a basis for expert judgment.
It is a case study which develops a four-step student-centered approach to increase positive impacts on the learning process. It analyzes the strengths and weaknesses of assessments in IS courses through a survey.
It is an empirical study of technical problems in web-based sys-tems for assisting education.
It investigates how use of an online assessment tool, the Com-puter-Assisted Personalized Approach (CAPA), helps achieve teacher and students learning goals. It is a case study of urban courses in Florida University.
It evaluates how digital technologies such as tablet PCs, wireless technology, and Web 2.0, can facilitate formative and collabora-tive assessment effectively.
e-learning age magazine, 2011
It is a case study of The Association of Chartered Certified Ac-countants (ACCA), which wants to deliver all accounting examina-tions electronically through an e-assessment program.
It is a case study of the positive role of aesthetics design on learner perceptions and task performance in an e-assessment environment.
Dascalu and Bodea, 2010
It proposes a qualitative analysis of an e-assessment and its role to increase knowledge creation and knowledge management.
Sainburg and Benton, 2010
It is a case study of the importance of formative assessment in the learning process in schools.
It evaluates the possible risks in planning e-assessment through six identified dimensions in a case study.
Jordan and Mitchell, 2009
It explains some methods to facilitate answer and assessment of free-text questions in e-assessment. It examines its model (IAT) in the UK Open University, using information extraction tech-niques and NLP.
It shows that an e-assessment system can be a good alternative to encourage students and has positive effects on the learning process.
British Journal of educational studies, 2009
It summarizes different works which all encourage educational institutes to use e-assessment technologies to support the learn-ing process, with emphasis on feedback from e-assessments.
Hodgson and Y. C. Pang, 2012
It is a study of statistics students to evaluate the advantage of formative assessment.
It examines the impact of formative quizzes on knowledge con-struction.
Daly et al., 2010
It argues that existing assessment solutions focus on developing technical and infrastructure issues more than educational as-pects of assessment through different educational ases.
It is a case study of a US campus to explore how a new e-assessment system is implemented and to identify factors which affect how it is adopted.
It suggests a set of principles for the effective design and evalua-tion of a formative assessment and feedback process through two different case studies.
It is about forecasting models like the Bass model, used to evalu-ate how educational systems can adopt themselves to e-assessment methods.
Boyle and Hutchison, 2009
It makes the case for the substantial effects of e-assessment on the assessing process and the technical difficulties involved in developing sophisticated assessment tasks.
McNeil et al., 2011
It looks at e-assessment from both sides: educational and tech-nical. It develops a model for analyzing the life cycle of institu-tional assessments.
Sangi and Malik, 2007
It reviews existing e-assessment models and practices and identi-fies the challenges facing developers in South Asia.
Appendix 2: List of German universities reviewed
Freie Universität Berlin
Johannes Gutenberg-Universität Mainz
Karlsruher Institut für Technologie
Rheinisch-Westfälische Technische Hochschule Aachen
Leibniz Universität Hannover
Westfälische Wilhelms-Universität Münster
Technische Hochschule Ostwestfalen-Lippe
Technische Hochschule Wildau
Universität zu Köln
Eberhard Karls Universität Tübingen
Rheinische Friedrich-Wilhelms-Universität Bonn
Technische Universität Dresden
Technische Universität Chemnitz
© 2021 The Author(s)
About this chapter
Cite this chapter
Heidari, M., Arnold, O. (2021). Toward Process Variability Management in Online Examination Process in German Universities: A State of the Art. In: Koschtial, C., Köhler, T., Felden, C. (eds) e-Science. Progress in IS. Springer, Cham. https://doi.org/10.1007/978-3-030-66262-2_8
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-66261-5
Online ISBN: 978-3-030-66262-2