Using a Student Authentication and Authorship Checking System as a Catalyst for Developing an Academic Integrity Culture: a Bulgarian Case Study

Abstract

This paper presents a case study carried out at Sofia University in Bulgaria, describing the relationship between two developments, firstly an expanding involvement with online learning and e-assessment, and secondly the development of institutional approaches to academic integrity. The two developments interact, the widening use of e-learning and e-assessment raising new issues for academic integrity, and the technology providing new tools to support academic integrity, with the involvement in technological developments acting as a catalyst for changes in approaches to academic integrity. The aim of this study is to describe in what ways the integration of technologies for student authentication and authorship checking in this university has begun to influence teachers’ approach to academic integrity, and has also helped to identify specific issues that need to be resolved for the future of academic integrity in the university. Data collected during the implementation of pilots for the project TeSLA - An adaptive trust-based e-assessment system - enabled an examination of the perspectives of administrators, teachers and students on approaches to cheating and plagiarism, and on possible future directions. The data suggests that the piloting of the TeSLA system has triggered a deepening consideration of approaches to academic integrity, and has also helped to identify important issues for future developments.

Introduction

The development of academic integrity is a vital part of the development of higher education. The Handbook of Academic Integrity (Bretag 2016) includes accounts of the present state of academic integrity implementation in a wide variety of countries, from which it is clear that universities in many countries are having difficulties in devising and implementing appropriate policies, a process made harder by the increasing use of the internet in completing assignments, and the rise of ‘contract cheating’ (where a student pays a third party to produce work for them for an assessment). A variety of approaches to addressing academic integrity have been suggested, and these are described in the section Developing academic integrity below, however it is widely acknowledged in that literature that significant problems still remain. In this paper, we provide a description of a novel approach adopted in a university in Bulgaria, a country with an acknowledged low level of implementation of academic integrity. The aim of this study is to describe in what ways the integration of technologies for student authentication and authorship checking in that university has begun to influence teachers’ approach to academic integrity, and has also helped to identify specific issues that need to be resolved for the future development of academic integrity.

The paper first provides a more detailed account of the context of the study, and then an outline of existing literature on academic integrity implementation in European and, specifically, in Bulgarian universities. The next sections discuss approaches to developing academic integrity, focusing in particular on the use of assessment design to reduce opportunities for cheating and the use of technology to detect cheating, including a brief description of the use of student authentication and authorship checking in the TeSLA system. The central sections of the paper present the aims, objectives and methods of the study followed by a report of the findings from interviews with institutional leaders, questionnaires completed by teachers and students, and a focus group with a small number of teachers. The final two sections contain, firstly, a discussion putting the results of the empirical work in the context of the existing literature, and then a conclusion setting out the implications for the development of academic integrity within this university, and highlighting aspects which may have a wider significance.

Context

This paper presents a case study carried out at Sofia University in Bulgaria, which describes the relationship between two interacting developments, firstly an expanding involvement with blended learning and online learning, and secondly the development of institutional approaches to academic integrity. The two developments interact, the development of e-learning and e-assessment raises new issues for academic integrity, and the technology provides new tools to support academic integrity. The involvement in technological developments can act as a catalyst for changes in institutional approaches to academic integrity.

The development of e-learning and online learning at Sofia University has been significantly supported through involvement with EU funded projects, resulting in interactions with other European universities and quality assurance agencies which have highlighted issues about quality and academic integrity. Most recently, Sofia University has participated in the TeSLA project which is developing an integrated system of student authentication and authorship checking for e-assessment. Involvement in these projects has encouraged institutional reflection about academic integrity, and as a result Sofia University is now at an important stage of transition in this area: an Ethical Committee was established recently, a license for plagiarism detection software has been obtained, and the university is in the process of developing policies for academic integrity.

Academic Integrity

This section provides an introduction to some issues related to academic integrity in higher education, and in particular an account of studies of academic integrity policies in European universities.

The European Network for Academic Integrity defines academic integrity as: ‘Compliance with ethical and professional principles, standards and practices by individuals or institutions in education, research and scholarship’ (Tauginienė et al. 2018). This present paper seeks to approach academic integrity from a positive perspective, as something embedded in university culture, rather than simply as a negative response to cheating (Gallant 2015).

In recent years, there has been some public concern that cheating and plagiarism in universities are increasing, in part as a consequence of the use of the internet, however, researchers have questioned both the perception of an increasing problem and of the negative impact of the internet, for example, McCabe (2016) in his longitudinal study provides some evidence of a decrease in cheating, and Davies and Howard (2016) argue that there is no empirical evidence to support the argument that the internet is a contributor to plagiarism.

For some time the main focus of concern about cheating in the academic literature has been about plagiarism. This literature (see for example: Howard 2016; Flint et al. 2006; Selwyn 2008; Sutherland-Smith 2016; Vehviläinen et al. 2018) has explored the extent of plagiarism and its impact, but has also raised issues about the nature of plagiarism and its relation to scholarship more generally, and argued for ways of addressing plagiarism through improved approaches to academic literacies. More recently, the phenomenon of ‘contract cheating’ has perhaps overtaken plagiarism as the area of most concern (see Harper et al. 2018; Lancaster and Clarke 2016; QAA 2016).

Academic Integrity Policies in European Universities

Two recent projects have examined the state of academic integrity within European universities: Impact of Policies for Plagiarism in Higher Education across Europe (IPPHEAE) (Glendinning et al. 2013) and South-East European Project on Policies for Academic Integrity (SEEPPAI) (Foltýnek et al. 2018). These studies present an Academic Integrity Maturity Model (AIMM), reporting AIMM scores for each country along nine dimensions: Research, Training, Knowledge, Communication, Prevention, Software, Sanctions, Policies, and Transparency. Glendinning (2014) concluded that all the participating institutions in the IPPHEAE viewed plagiarism and academic dishonesty as a serious issue, and that there were many examples of innovative practice, but that universities in many parts of Europe lacked effective policies and systems for assuring academic integrity.

Of particular interest for this study is the IPPHEAE project national report for Bulgaria (Glendinning 2013). Bulgaria had the lowest AIMM score of all the countries in the IPPHEAE study, and remained the lowest when the countries in the SEEPPAI project were also included. Glendinning (2013) found that policies and practices varied greatly between Bulgarian universities, that there was a lack of research, statistics and guidelines related to academic integrity, and little use of plagiarism detection software. Though a range of sanctions were available for cheating and plagiarism in Bulgarian universities, teachers felt that there was a lack of consistency in their application, and 54% of students and 50% of teachers believed it was likely that no action would be taken over plagiarism in an assignment. Glendinning (2013) reports that respondents had a mature understanding of academic integrity and of possible ways to improve academic standards, though when examining the responses of both teachers and students to detailed questions on whether specific kinds of behavior constituted plagiarism or not, the author comments that the result “suggests that students’ confidence in understanding academic writing conventions may be misplaced and that some teachers may themselves be inadvertently plagiarizing” (Glendinning 2013, 11–12).

Developing Academic Integrity

A literature has grown up over the last ten years looking at approaches for developing academic integrity. Approaches that have been explored include those related to policy (Bretag et al. 2011), teaching and learning approaches (E.J. Morris 2016), restorative justice (Kara and MacAlister 2010) and the use of technology (Okada et al. 2019b).

It has been widely recognized that universities need to adopt an integrated approach involving a range of interrelated measures. The Higher Education Academy (HEA) stressed the importance of: induction and learning support provision including opportunities to develop study and academic writing skills; emphasizing authentic assessment tasks that can help to deter plagiarism; and developing, implementing and monitoring formal policies and procedures (Higher Education Academy 2010, 6). In discussing contract cheating, the Quality Assurance Agency also stressed the need for a multi-stranded approach, identifying four main approaches: information, training and culture; assessment design; penalties; and detection through technology:

In reviewing some of the options for tackling this threat, we acknowledge that there is no single solution. Instead, we need a multi-faceted approach that builds on published research and the steps that universities and colleges are already taking to promote good academic practice by students, to ‘design out’ opportunities for plagiarism in their assessments, and to detect and penalise academic misconduct. (QAA 2016, 1)

All four of these approaches have some relevance to the present study; however, this paper is particularly concerned with assessment design and the use of technology for the detection of cheating as it focuses on the use of technologies for student authentication and authorship checking for assessment activities, and the use of these technologies necessarily has a close link with the design of those assessments.

Using Assessment Design to Reduce Opportunities for Cheating

A number of authors have proposed the use of assessment design in order to reduce the opportunities for cheating (Mahmood 2009; Morris 2016; Morris and Carroll 2016; Newton and Lang 2016; M. Walker and Townley 2012; C. Walker and White 2014). The project Contract Cheating and Assessment Design: Exploring the Connection (https://cheatingandassessment.edu.au) has explored the issue of assessment design in some depth, collecting data from a large sample of students and teachers as well as examining procurement requests to cheat sites and examples of purchased assessment items identified by universities (Bretag et al. 2018; Bretag et al. 2019; Harper et al. 2018). The findings of this project challenge the view that assessment design alone might be the solution to the issue of cheating. When the students were asked about the likelihood of contract cheating in a wide variety of assessment tasks then all assessment tasks were shown to be vulnerable. The four assessment activities that the students thought the least likely to be outsourced (i.e. ‘contact cheating’) were in-class tasks, personalized and unique tasks, vivas and reflections on practical placements, and, interestingly, these four assessment types were found to be the types of assessment least used by teachers in the study (Bretag et al. 2019). Assessments with short turnaround times and heavily weighted assessments were seen by the students as the most likely to be outsourced.

Most staff who had had suspicions about outsourced assessment referred the matter to the appropriate decision maker in accordance with the university policy (55.8%), but 7.7% did nothing, and 29% applied their own penalties (e.g. warning, refer to counseling, resubmission, penalty mark) (Harper et al. 2018).

Bretag et al. (2019) also found that the more satisfied students were with the teaching and learning in their courses the less likely they were to think that contract cheating would take place. There were, however, differences in the perceptions of teachers and students about the level of support they received: “Staff agreed at much higher levels than students that assignment grading is consistent, that staff teach scholarship in their discipline, that staff consistently monitor and penalize academic integrity breaches, and that staff teach referencing” (Harper et al. 2018, 10).

Using Technology to Detect Cheating

Technology offers a number of approaches which might play a role in addressing cheating. The use of technology for plagiarism detection in universities is well established and other technologies such as student authentication, proctoring and style checking are becoming increasing available (Okada et al. 2019b). This paper is principally concerned with two groups of such technologies: those related to student authentication and those related to authorship checking.

Authentication technologies, in this context, seek to establish that the student taking the assessment really is who they say they are. Apampa et al. (2010) make a distinction between the user and password paradigm for authentication which is the most popular method of protecting access to computer systems (including assessment systems), but which is open to many possible attacks, and the biometric paradigm which consists of an enrolment stage capturing the student’s biometric data (e.g. a fingerprint) which creates a template, and an authentication stage when the student wishes to access the system and where the new biometric data is compared with the template. The use of a single biometric approach is often sufficient, but may be less accurate when used on a large scale than laboratory testing predicts, and is also less secure than the use of two or more biometric instruments (e.g. fingerprint and face recognition together). Karim and Shukur (2015) describe the use of a wide range of possible biometric instruments, including those using physiological characteristics such as the iris, face and finger prints, and those using behavioral characteristics such as voice, key stroke dynamics, and handwriting. The authors also highlight the potential expense involved in capturing some biometric data (e.g. iris recognition requires additional hardware, though face recognition can be done using commonly available, and cheap, web cams) and the privacy and data security issues which arise with the storage of biometric data. Present day use of student authentication systems in higher education is most commonly seen in the use of online proctoring systems for assessment, which are now being used increasingly in the USA, though their use in Europe is still quite restricted (Draaijer et al. 2017).

Authorship checking technologies, on the other hand, seek to establish whether a document was actually written by the student who submits it. The use of text-matching software for authorship checking and plagiarism detection is now a well established practice in many European universities, though it has been found to be accompanied by a number of problems (see for example: Davies and Howard 2016; S.M. Morris and Stommel 2017; Nilsson 2016; Weber-Wulff 2016). Weber-Wulff (2016) identifies the following problems: the reports can be difficult to interpret; correctly referenced material may be marked as plagiarism; some plagiarized documents are not recognized as plagiarized; and different systems report varying degrees of plagiarism for the same document, but the lack of transparency about the algorithms used means that these differences are difficult to interpret. Davies and Howard (2016) argue that writing using internet sources is a difficult process and that students are not being properly prepared for this task and, therefore, the use of text matching software for plagiarism detection is not an appropriate response, and that a better response would be to devote more effort to mentoring students’ critical reading practices.

A different approach to authorship checking is the use of linguistic analysis tools such as stylometrics which have used for some time in areas such as determination of authorship of literary works and for legal documents (Juola 2007). In recent years stylometrics has begun to be applied to addressing contract cheating and plagiarism (Juola 2017; Sousa-Silva 2017). This is a promising direction in which a lot of hope is being placed but, whilst small scale experimental studies exist, there is as yet no evidence in the research literature of their effective use at the institutional level in education.

The TeSLA Project

The Horizon 2020 funded TeSLA project (2015–2019) involved a consortium of technological, educational and quality assurance institutions in order to develop a technological system and an assessment design approach to address cheating and plagiarism through the development of a modular system for student authentication and authorship checking (Mellar et al. 2018; Noguera et al. 2017; Okada et al. 2019a, b). The TeSLA system brings together a number of technological strands for addressing cheating, offering a choice of Face Recognition, Voice Recognition and Keystroke Dynamics instruments to authenticate students’ identity, and Forensic Analysis (a stylometric approach) and Plagiarism Detection instruments for authorship checking. In its implementation in trials in universities in Bulgaria, Finland, Netherlands, Spain, Turkey and the UK the use of the TeSLA instruments was accompanied by the redesign of assessments to ensure greater integrity and variety in the assessment process.

The Study

This section sets out the aims of the study, followed by a description of the methods of data collection and analysis.

Aims

The work of the IPPHEAE and SEEPPAI projects (Foltýnek et al. 2018; Glendinning et al. 2013) has shown that the level of development of academic integrity policies in universities in many European countries is quite low, with Bulgaria having the lowest AIMM score. Universities are facing increasing challenges to academic integrity in the form of contract cheating, but whilst they are being called on to improve their academic integrity policies to address these challenges many of them have no clear path to making such improvements. This paper examines the situation in one leading Bulgarian University (Sofia University) which, like many others, is seeking to improve its academic integrity approach whilst at the same time it is increasing its use of blended learning and online learning in order to address student demands for more flexible provision, which in turn poses new challenges for academic integrity.

The aim of this study is to describe in what ways the integration of technologies for student authentication and authorship checking in this university has begun to influence teachers’ approach to academic integrity, and has also helped to identify specific issues that need to be resolved. We wish to examine how the introduction of technology for student authentication and authorship checking is impacted by existing academic integrity norms and then in turn influences those norms. The specific objectives of the study are to ask:

• How is academic integrity presently implemented in terms of university policies and regulations, training and support for students and teachers?

• What is the level of awareness of students and teachers of cheating and plagiarism, of existing academic integrity policies and their level of implementation, and of institutional expectations in relation to sanctions?

• What are the views of students and teachers on reducing the occurrence of cheating and plagiarism?

• How would teachers use the feedback they obtain from student authentication and authorship checking instruments, and how would this impact on the implementation of academic integrity?

On the basis of this study, conclusions related to the further development of academic integrity in the university will be presented, which it is argued will be of wider relevance in the development of academic integrity in higher education more generally, especially for other countries like Bulgaria which are in the early stages of implementation of academic integrity policies.

Data Collection and Analysis

The empirical work reported in this paper is based on data collected during the implementation of the second of three pilots of the TeSLA project. The data captures the perspectives of administrators, teachers and students on cheating and plagiarism, on possible ways of supporting the development of an academic integrity culture, and on ways in which TeSLA might impact on this.

Background information about Sofia University was obtained from senior staff, who were asked to provide an overview of the regulations related to academic integrity and to comment on the advice and support given to students, the scale of cheating, and the use of sanctions. The main data collection methods of the study built on existing TeSLA project evaluation tools – online questionnaires and face-to-face focus groups – extending them to explore the areas the study wished to examine in more depth.

The questionnaires were based on the student and teacher questionnaires developed for the TeSLA project (Okada et al. 2019a, b). These questionnaires had been constructed by one of the seven TeSLA partner institutions and then reviewed by academic staff in the other six partner institutions to ensure a degree of face validity. They were used in the first TeSLA pilot and modified as a result of experience, and the work reported here built on this second version. These questionnaires were modified for this study by incorporating some more detailed questions relating to academic integrity and removing some of the questions which were less relevant for this study in order to reduce the length of the questionnaire. The additional questions relating to academic integrity were developed from the student and teacher questionnaires used in the IPPHEAE project (available at http://plagiarism.cz/ippheae/), the development of which is described in Glendinning et al. (2013). The final questionnaires were reviewed by academics from other TeSLA partner institutions.

This final version of the questionnaire was checked for test-retest reliability with a small sample of 15 students. Aside from the demographic data, this paper reports on twelve items that were asked to both students and teachers. A further four items (related to teachers’ responses to cheating) were only asked to teachers and so were not part of this test-retest reliability check. The average level of agreement between test and re-test responses across nine of the items was 84% (with values for Cohen’s kappa for all greater than 0.55). The remaining three items had lower levels of agreement (a little over 50%), and so whilst results for these items will be reported they will not be used in statistical tests. These three items were: a) support for academic integrity including opposition to cheating and plagiarism is well established in the university – both among students and academic staff; b) the university has clear regulations in relation to cheating and plagiarism in assessment; and c) I understand the consequences of cheating and plagiarism at the university.

Ethical approval for the study was obtained from the Ethics Committee at Sofia University. Students and teachers were provided with written information about the data collection and the TeSLA project and were assured that questionnaires were anonymous, and that they could withdraw from the study at any point and that their data would then be deleted. Students completed an online consent form for participation in the project relating both to the questionnaires and the collection of biometric data by the TeSLA instruments, and teachers gave their consent for the collection and use of survey and focus group data.

The student questionnaire was completed by 218 students who were involved in the TeSLA study and had experience of the use of at least one of the TeSLA instruments. The teacher questionnaire was completed by 100 teachers, of who six had been involved with TeSLA. In the questionnaires, both students and teachers provided basic demographic information, previous qualifications (for students), their experience of cheating and plagiarism, whether they themselves had cheated (for students) or found students cheating (for teachers), their opinions about cheating and plagiarism, the implementation of academic integrity in Sofia University (support, information, regulations, understanding, procedures for identifying and sanctioning cheating and plagiarism), and teachers were also asked how they would deal with a variety of situations where cheating and plagiarism were identified. Both teachers and students were asked to respond to an open question about how cheating and plagiarism might be prevented or reduced.

In this paper the quantitative data from the questionnaires is presented descriptively together with one test of association between teacher and student data, and the responses to the open questions in the questionnaires are analyzed thematically.

Eight teachers from four faculties, one male and seven female, took part in a focus group. Two of the teachers had been involved with the TeSLA project, and the others were provided with a detailed introduction to the TeSLA system. The aims of the focus group were to get a deeper picture of teachers’ understanding of the culture of academic integrity, to obtain their proposals for changes to the regulation of academic integrity, and their views on how they would interpret and deal with the feedback provided by TeSLA. The initial guiding questions presented in the focus group were:

• 1. What do you understand by academic integrity?

• 2. What changes would you wish to see in relation to regulation of academic integrity?

• 3. How would you interpret and deal with the feedback provided by TeSLA?

These questions were then followed up with additional prompts. The discussion was recorded and then transcribed for analysis. For topics 1 and 3, the transcript for each topic was analyzed into inductively derived categories; the analysis started with simple content codes which were then revised and amalgamated into a smaller number of larger categories, and then the analysis was reviewed by a second coder and additional revisions made. In the case of topic 2, the analysis was informed by the categories identified in the literature related to approaches to preventing cheating (i.e. information, training and culture; assessment design; penalties; and detection through technology). The categories identified together with examples of the discussion are presented in the section Findings – Focus Group below.

Findings – Institutional Perspectives

This section presents an overview of the existing academic integrity regime at Sofia University as provided by senior staff in the university. It first presents a summary of the responses of the Vice-Rector of Education, the Senior Legal Expert and the Chair of the Ethics Committee to a set of questions asking for background information about the university and its academic integrity regime, and then provides a description of the external quality assurance review for e-assessment for Sofia University that was carried out as part of the TeSLA project.

The Vice-Rector said that the university has 21,000 on-campus students, that during the previous 5–6 years it has started to introduce blended and fully online learning, and has recently received accreditation for distance education. All final exams use face-to-face written examinations, and use of oral exams has been discontinued because of concerns that teachers were too subjective in their assessments.

The Senior Legal Expert said that the regulations identified sanctions for cheating that included: change of mark, warning of exclusion from the university or actual exclusion either temporally or permanently. Accusations of cheating are referred to the Rector via a Dean’s report based on a report from the teacher identifying the possible cheating. In the period 2012–2017, 57 cases were reported from nine of the 16 faculties. The most common type of cheating during examinations was the preparation by students of written answers to a range of expected questions and attempting to bring them into the exam room (either on paper or using electronic devices), and for theses and coursework was plagiarism. The Senior Legal Expert commented that there were no well-established mechanisms to stop cheating and plagiarism, no use of electronic systems to check text materials, and that cheating and plagiarism was a problem that was somewhat neglected by the teachers.

The Chair of the Ethics Committee confirmed that issues related to academic integrity are subject to regulation by the University, and that in March 2016 the university adopted a Code of Ethics and established an Ethics Committee. Ethics training courses are provided for doctoral students, and the Code of Ethics specifies that all teachers should introduce their students to the ethical norms in their area and clarify requirements for writing a paper. Plagiarism, copying and contract cheating are all regarded as breaches of academic integrity and should be penalized by return of submitted written papers, or a fail grade. The Chair of the Ethics Committee felt that the university response in the case of violations of academic ethics was adequate.

As part of the TeSLA project the implementation of e-assessment in the project was subject to a quality review by a review panel from European Association for Quality Assurance in Higher Education (ENQA) whose methodology is explained in Huertas et al. (2017). The review panel collected information about the implementation of quality assurance for e-learning and e-assessment, and then reported back to a head panel which collated the reviews for the seven institutions involved in the TeSLA project. The review panel provided detailed feedback and recommendations for improvement to the Sofia University TeSLA team. This report was then presented to the Rector and senior management, providing an important stimulus for change in the form of a well-grounded comparison with other institutions, and with the ENQA quality standards.

Findings - Questionnaires

This section provides an analysis of the questionnaire data, describing the participants, their opinions about cheating and plagiarism, the existing academic regime, and teachers’ views on the management of cheating and plagiarism. It also presents an analysis of the teachers’ and students’ views about the prevention of cheating and plagiarism, as given in response to an open question in the questionnaires.

Participants

The demographic data for the teachers who took part in the study showed that most of the teachers were female (65%) and aged over 31 (roughly equally divided into those in their 30’s, 40’s and 50’s). They came from a variety of faculties, all had considerable experience of face-to-face teaching, and most had taught a course where at least part of the assessment had been conducted on-line.

Most of the students were female (90%), under 21 (69%), and were full-time students taking their first degree (97%). The proportion of female students in the university overall is 65%, and so the proportion of female participants in this study is somewhat higher than for the university as a whole. The students who participated in the study were drawn from courses in education either in one of the two education faculties, or from teacher training departments in other faculties, and the majority of students in these courses are female. It is likely that students in the area of education would be more aware of issues around cheating and plagiarism than other students as it is something that impacts on them as future teachers.

Cheating and Plagiarism

In this section, the views of teachers and students about cheating and plagiarism are compared. Teachers (N = 100) were asked ‘How many students have cheated or plagiarized other work in at least one assessment in the courses you teach?’ and given the options many/some/few/none, but these options were not further defined. The responses were: 6% many, 26% some, 37% few, 0% said none, 2% did not know, and 29% did not respond. Students (N = 218) were asked ‘How often have you cheated or plagiarized other work in your university courses?’ and given the options often/sometimes/occasionally/rarely/never, but these options were not further defined. The responses were: 1% often, 5% sometimes, 5% occasionally, 31% rarely, 31% never, and 28% did not answer. It is clear that cheating and plagiarism is a significant issue for both teachers and students.

Most teachers and students disagreed with the statement that ‘cheating and plagiarism are fairly innocent ways to get help’ and agreed that ‘cheating and plagiarism are unethical’ (see Tables 1 and 2).

Table 1 Students’ responses (N = 218) and teachers’ responses (N = 100) to the statement ‘Cheating and plagiarism are fairly innocent ways to get help’

Table 2 Students’ responses (N = 218) and teachers’ responses (N = 100) to the statement ‘Cheating and plagiarism are unethical’

As for the students, 11% agreed that cheating and plagiarism are fairly innocent ways to get help, and 9% disagreed with the statement that cheating and plagiarism are unethical. So, whilst the majority were clearly aware of the seriousness of cheating and plagiarism a minority were not. It is likely that this is a result of an inconsistent level of teaching about this issue for these undergraduate students. As for the teachers, whilst no teachers agreed that that cheating and plagiarism are fairly innocent ways to get help, some 11% disagreed with the statement that cheating and plagiarism are unethical. This unwillingness on the part of a minority of teachers to class this behavior as unethical, even though they accepted that it was not ‘a fairly innocent way to get help’, is interesting. It is possible that these teachers were taking into consideration the inconsistent level of preparation of students about this issue, and in particular a lack of teaching about how to avoid plagiarism (some support for this speculation can be found in the comments of teachers in the focus group reported in the section Findings – Focus Group below).

Two follow-up questions probed their understanding of plagiarism, by asking whether they agreed with each of two statements. Table 3 shows that most students agreed with the first statement and comparison with results from other institutions in the TeSLA project indicated that most students in those institutions also agreed with this statement. However, the students in Sofia University tended to disagree with the second statement, as did students in five of the seven institutions in the TeSLA project. The teacher questionnaires did not incorporate the two questions shown in Table 3, however the six teachers involved in the TeSLA pilot did answer these questions as part of the TeSLA evaluation carried out by the TeSLA evaluation team, and the evaluation report indicates that three did not agree with the second statement, one was unsure and two agreed. The level of disagreement with this second statement demonstrates a somewhat weak understanding of plagiarism on the part of both students and teachers at Sofia University.

Table 3 Student’s responses (N = 218) to statements about plagiarism

Existing Academic Integrity Regime

Teachers and students were asked about the implementation of academic integrity at Sofia University. The results are shown in Tables 4 and 5 - the number of responses to this latter part of the questionnaire was lower than that for the earlier part of the questionnaire, the number of teachers responding was 71, and for students it was 156. For all but question 3 the students are more likely than the teachers to be uncertain. Further, students were more likely than teachers to agree (or, less likely to disagree) that:

• Support for academic integrity was well established;

• The university had clear regulations in relationship to cheating;

• They understood the consequences of cheating and plagiarism.

However, these three items had been found to have low test-retest reliability scores.

Students were less likely than teachers to agree that:

• The regulations were applied consistently;

• They were well informed about the concept of academic integrity;

• They had been introduced to the regulations at the start of the course;

• The methods used to identify cheating were adequate.

Table 4 Teachers’ responses (N = 71) about the implementation of academic integrity

Table 5 Students’ responses (N = 156) about the implementation of academic integrity

Chi-squared tests were calculated for the null hypotheses that academic status (teacher/student) and level of agreement (Agree & strongly agree/Neither agree nor disagree/Disagree & strongly disagree) with the proposition are independent for these four items, and the results are shown in Table 6. In all cases the null hypothesis can be rejected with a significance level of p < 0.05.

Table 6 Chi-squared test for independence (H_o: academic status and level of agreement with the proposition are independent)

Questions 1–4 refer to the institution, on these four questions an average of 40% of teachers agreed with each proposition and an average of 38% disagreed, indicating a degree of difference in views about the institutional situation amongst the teachers. Questions 5–7 refer to individual teacher behavior and attitudes, and here the teachers are more consistent as a group in their responses.

Teachers’ Management of Cheating

The teachers were asked how often they had reported cheating and plagiarism: 10% said often, 15% sometimes, 14% occasionally, 24% rarely and 37% never. They were then asked whether they would be less likely to report cheating and plagiarism in each of four hypothetical situations, the results are shown in Table 7. Though there is reasonable agreement amongst teachers on the first proposition, it is interesting to see that, as with the questions about the institutional approach to academic integrity reported above, quite different approaches to handling cheating and plagiarism were being adopted by different teachers, with an average of 46% of teachers agreeing with propositions 2–4, and an average of 36% disagreeing with them.

Table 7 Teachers’ responses (N = 71) to cheating scenarios

Preventing Cheating and Plagiarism

The responses to the open question ‘Briefly describe how you think cheating and plagiarism might be prevented or reduced in your university courses’ provided a number of valuable insights into the thinking of both teachers and students about approaches to academic integrity. An analysis of these responses identified four categories which closely correspond to the categories of approaches identified in the literature:

• Pedagogy and assessment design (corresponding to ‘assessment design’)

• Regulations and sanctions (corresponding to ‘penalties’)

• Use of technology (corresponding to ‘detection’)

• Information and training (corresponding to ‘information, training and culture’).

For teachers, responses in the categories pedagogy and assessment design and use of technology were most common, with responses in the categories information and training and regulations and sanctions less common, though still widely found. For students, responses in the categories pedagogy and assessment design and regulations and sanctions were most common, with responses in the category use of technology also widely found, but with very few responses in the category information and training.

Pedagogy and Assessment Design

Teachers and students identified quite different aspects of pedagogy and assessment design as important. Teachers put emphasis on forms of assessment that would make cheating and plagiarism more difficult to carry out (e.g. oral exams, personalized assessment). Students put emphasis on aspects of pedagogy and assessment that would make it less likely that they would want to cheat (e.g. more engaging teaching and assessment, reduction in work and assessment load, dropping unnecessary assessments).

Teachers proposed approaches to the design of assessment which could be described as trying to ‘design out’ opportunities for cheating and plagiarism, for example:

• Use tasks with a creative element, which do not simply ask for the reproduction of knowledge;

• Employ continuous control over the assessment process so that the teacher can get a more holistic view of the student’s work;

• Use a variety of assessment methods;

• Combine written and oral tests;

• Give unique assignments for each student;

• Individualize and randomize the test materials.

Students’ comments related to two topics: quality of education, and design and organization of assessment tasks. Most comments were about how the quality of teaching, learning and assessment could be improved in order to make it less likely that students would want to cheat.

When a student feels confident enough in his/her knowledge of the subject matter, which is based on exercises and lectures in the course, and devotes less time to self-education and self-training, then cheating and plagiarism will be insignificant.

Giving more affordable materials and explaining to the students where they can get information. Just giving a list of books which are not even in the university library is just a mocking attitude, and so the students do the same.

Teachers should be ready to assist students if needed and to encourage them more.

Students also saw their workload as contributing to cheating and plagiarism, complaining about ‘three tests in a week’, ‘too much homework’, and ‘meaningless assessments of material that do not have to be known, but are only recommended’.

A number of students commented that they were more likely to cheat when they thought that the course was unnecessary for their specialty.

Some disciplines are taught in close relation to the specialty, but when this is not the case the students do not feel that it is worthwhile to strive, is important, or put an effort to it, so therefore, he/she resorts to plagiarism.

The students also commented on the format of assessment tasks, some of which echo the comments made by teachers, being suggestions for assessment design that can make cheating and plagiarism more difficult. They suggested that tasks should be ‘small in size and concise instead of being given as a few big tasks’, ‘individual to everyone’, stimulating/thought-provoking and directly ‘practice oriented’, and ‘different from previously assigned tasks’. They also suggested ‘more group assignments’ so that ‘students have an incentive to have fun and do their work’, ‘more written tasks, such as essays, etc., for which more time for preparation is given’, and ‘more oral exams’. Some students suggested that the assessment process should involve greater use of continuous and formative assessment.

Regulations and Sanctions

Both teachers and students commented on sanctions and regulations. Teachers mentioned:

• Imposing more severe penalties and sanctions;

• The need for clear procedures against plagiarism and for a common university policy;

• Requiring students to sign an enrolment statement that explains the meaning of cheating and plagiarism and their consequences.

Some students suggested stricter examination procedures with all exams carried out on the university premises, strict control accompanied by strict sanctions.

Through increased security measures, including a greater number of exam invigilators in the face-to-face form of assessment or more security measures, and student authentication in online assessment.

By penalizing with a low mark or rejecting the student’s work for an assessment and expulsion.

Use of Technology

Both students and teachers mentioned the use of technology to identify cheating and plagiarism and also to act as a deterrent. Plagiarism detection was widely mentioned, and other suggestions included: video for surveillance in exams, discontinuation of internet access and use of cell phone jammers in exam rooms, and the use of authentication software.

Information and Training

Neither students nor teachers said that they themselves needed additional training, though teachers believed that students needed additional training about plagiarism:

• Prevention through introducing students to what plagiarism is, how to avoid it, how to quote foreign language materials;

• Explaining the consequences of cheating at the beginning of the course and repeating it before the exam;

• Enhancement of teaching about academic writing.

In summary, teachers and students identified a range of possible approaches to addressing cheating and plagiarism which reflected those discussed in the literature, though a striking feature of this data was the differences between students and teachers in their proposals for redesign of pedagogy and assessment. Whilst neither students nor teachers pointed to a need for training for teachers, the data shown above about teachers’ judgments as to what constitutes plagiarism, the differences between the approaches taken by students and teachers to redesign of pedagogy and assessment to reduce cheating and plagiarism, and the differences in teachers’ views about existing institutional practice and their differing approaches to dealing with the plagiarism scenarios presented in the questionnaire all point to a weakness in their preparedness for dealing with this issue.

Findings - Focus Group

Issues about academic integrity were explored in more depth with a small group of teachers in a focus group discussion. An analysis of their responses is provided below, firstly looking at the present situation with respect to academic integrity, then proposals for changes in relation to the regulation of academic integrity, and finally looking at the impact of the use of the TeSLA system for student authentication and authorship checking.

Perspectives on Academic Integrity

Teachers saw academic integrity as a cultural issue.

Academic integrity should be based on principles of cooperation that are negotiated for the narrative communication […]. Those principles assume a negotiated common goal, negotiated results, so the communication is successfully based on trust, everybody works as a team for the negotiated results. Thus, for me, of primary importance is to motivate the student to take part in the academic dialogue and to know what kind of behavior is acceptable and not just to have a sanctioning or disallowing approach.

The teachers said that they felt sufficiently well prepared in the area of academic integrity, and that education and training regarding academic integrity issues and the ways to deal with cheating were made available by the university.

Regulations

Seeing academic integrity as a matter of culture and of morality was thought of as being in some tension with the idea of regulation.

For me the idea of academic integrity is related to morality and the idea of establishing law or regulations to control morality is doubtful in terms of implementation.

Teachers recognized the need for rules and regulations to help students internalize the ideas of academic integrity, though they tended to look first to pedagogical instruments to educate the students and only reluctantly, if at all, to turn to administrative instruments to discipline and impose rules.

I keep close watch on the students, I rarely miss someone cheating. I am one of those who don’t report, i.e. I don’t write a report for attempted cheating which I can prove undeniably … I make the student repeat the exam until they show satisfactory performance.

The teachers applied a variety of what they described as ‘pedagogical’ sanctions when they identified cheating: termination of the assessment, redoing the work, and repeating the assessment, but administrative sanctions were rarely applied. Teachers were aware of the rules and regulations and of the procedures for escalating cases by submitting a report to the Dean, but they felt that these regulations were not clearly operationalized in terms of actionable procedures.

Student Awareness

The teachers said that there were a number of established approaches to familiarize students with academic values, rules and regulations, including a system of academic advisers. However, it was felt that this information is not systematized or coordinated by the institution, rather teachers are left to select the content to be delivered and this varies from one teacher to another.

In our courses in Moodle […] there is a presentation about what constitutes plagiarism, we familiarize the students with this at the first class and we explain that there will be consequences for those who don’t comply with the rules.

There is a file developed by me which explains how to quote according the requirements of Sofia University Year Book. There are examples for Bulgarian and English, for books in Bulgarian and English, for magazines, for collection of articles … it is very interesting that we show them this file and when they submit their papers, there is nothing of that sort there.

Preventing Cheating and Plagiarism

The teachers believed that cheating is becoming more frequent, that new technologies facilitate cheating, and that there is an increasing use of contract cheating. However, they thought that many of those that cheat are negligent and careless, and as a result easy to identify (e.g. they submit somebody else’s paper together with the feedback from the teacher).

The analysis of the open questions in the questionnaire described earlier identified four categories of responses to the issue of preventing cheating and plagiarism. The responses in the focus group fell into the same categories, though the discussion of detection via the use of technologies tended to be subsumed in the discussion of TeSLA and so will be reported in the section Using the TeSLA System below.

Pedagogy and Assessment Design

Teachers described techniques such as using a combination of several types of assignments, the use of more creative tasks, and the use of personal interaction with the students.

There are ways for the good teacher to reduce the possibilities for students to copy […] I combine different forms of assessment … for instance, besides tests, besides assignments, I give methodical essays ‘What does it mean to teach Bulgarian’ and ‘Who is the real teacher?’ combined with … concrete cases – ‘Your students ask you how to prepare for the final exams, what are you going to advise them?’

When it is a face-to-face exam it is very easy for me … I ask them why you have chosen this answer and it is very easy to me … Nobody can cheat me because I know the capabilities of those people.

Regulations and Sanctions

The teachers expressed confidence in the effectiveness of the approaches they adopted, but they also felt that they should be able to rely on effective institutional regulations when these educational approaches did not give satisfactory results.

I agree it is better to have described procedures … for punishment, or more precisely actions the professors should take when they witness noncompliance with the rules.

However, some teachers appreciated the existing freedom of action in dealing with cases of cheating in a flexible and individualistic way.

I am OK with the current situation and I am very content. I have the freedom given me by the institution, i.e. there is a regulation if I decide to use it … If they make me report all the cases, I will be the first one to protest because I make my decision for each case individually.

Information and Training

Several teachers commented that it would be useful if students signed an agreement to abide by the university rules, so that they became more aware of these rules. The teachers also thought that students should receive further training about academic integrity, and that a university wide approach would be valuable.

Maybe it makes sense for us to come to an agreement about what is acceptable, how it should be discussed with students and find working solutions which will help us overcome the issues of conflict.

The teachers thought that the students lacked sufficient training and practice in academic writing, which meant that they frequently did not realize they were plagiarizing. The teachers drew a distinction between consciously plagiarizing, thus cheating, and unconsciously plagiarizing due to lack of knowledge and training, and approached the two situations differently, adopting a sanctioning approach in the first case, educative in the latter.

Maybe the fault is not only in the students because there are students who take our course in the third year and they haven’t done a course paper before … Students are not trained on how to write course papers and this takes time.

Using the TeSLA System

In discussing the way in which the feedback from the TeSLA instruments would be used, the teachers presented a number of perspectives on how the introduction of a student authentication and authorship checking system might impact on academic integrity.

Firstly, teachers saw the use of such a system as improving academic integrity by imposing more discipline among students through their knowledge that the system was being used.

This will enable the creation of new anti-plagiarism culture, because when you know that you are watched, when you know there are ways to be registered, it is more probable that the anonymity of the informational society will be reduced, just knowing that you might be caught.

However, it was important for the teachers that they remained the primary decision-makers in the assessment process and the judges of students’ performance whilst making use of the TeSLA feedback to support their evaluation process.

First I would make a standard search in Google … If my search confirms the information that the system gives me, I will just make the student repeat the assessment, again and again. If I cannot confirm it, let’s say the system gives me 60-70% but I cannot confirm it with my search, I would invite the student to tell me more about his paper.

Secondly, teachers were keen to use the data from the TeSLA system to inform the assessment process, by providing them with a more informed view of the extent of any problems. In this way they hoped to be able to adjust their approaches to assessment by identifying the source of the problem, and then either revising the information given to students, or changing the assessment format.

The system will give a very good basis for feedback.

I would change the assessment form, either a test or a case-study, but in a way to prevent using existing materials.

I would change the approach so as to reduce the plagiarism percentage¹ […] either some more independent work, or a creative assignment.

Discussion

This section seeks to put the empirical findings from this study within the context of the literature that was reviewed earlier, and the next section will examine the implications of the findings for the implementation of academic integrity in higher education.

The rather low level of support for academic integrity found in Sofia University reflects the findings of the IPPHEAE (Glendinning et al. 2013) and SEEPPAI (Foltýnek et al. 2018) studies which showed that Bulgaria had the lowest score of all European countries on the Academic Integrity Maturity Model. The evidence that both teachers and students have a weak understanding of plagiarism corresponds with the result from the questionnaires reported by Glendinning (2013). However, the discussions of the teachers in the focus group indicated an increasing awareness of the issues and a strong desire to address them.

This study has highlighted some clear differences of perception between students and teachers with respect to the support that was provided to students by their teachers. Whilst the questions used in the survey in this study are not the same as those used to investigate this topic in the Contract Cheating and Assessment Design project (Bretag et al. 2018; Bretag et al. 2019; Harper et al. 2018), it is interesting to note that that project also found that students described themselves as receiving a lower level of support than the teachers claimed to provide.

In the present study, we found differences between students and teachers in their approach to thinking about assessment design for reducing cheating. The Contract Cheating and Assessment Design project did not explore this topic directly with students; however they did find that the assessment methods most used by teachers were the same as the ones students indicated as being most likely to be subject to contract cheating (Bretag et al. 2019). The students in the present study stressed increased teacher support and more relevant assessment as ways of improving assessment design in order to reduce cheating, which corresponds with the finding by Bretag et al. (2019) that higher quality teaching environments reduced cheating.

In this study, we found that teachers had a variety of different views of how the university approached academic integrity, and on the appropriate use of sanctions. For comparison, Glendinning (2013) found that 50% of the Bulgarian teachers in the IPPHEAE study believed that no action would be taken by the university if cheating was identified, and 50% believing that sanctions would be applied. The Australian universities involved in the Contract Cheating and Assessment Design project had more developed institutional approaches to academic integrity than those in the Bulgarian universities, but it is interesting to note that, even here, there was considerable variation in teachers’ practices with only 55.8% of the teachers referring suspected examples of contract cheating through the institutional processes (Harper et al. 2018). This variation in teachers’ responses presents a significant challenge to the establishment of a consistent academic integrity regime. The analysis of focus group discussions in this study suggested that the teachers were ambivalent about relinquishing their autonomy in favor of clearer institutional regulation and explicit procedures, and it may well be that it is this ambivalence that lies at the root of these differences.

Conclusions

This case study of Sofia University’s development of approaches to academic integrity, stimulated by its involvement in international e-learning and e-assessment projects and the use of a student authentication and authorship checking system, illustrates a development path that has potential relevance to other institutions looking to develop their academic integrity cultures from a similar low base level. The data from the focus group discussions, in particular, provided evidence that the increasing involvement in European and e-learning projects is influencing the university to improve its procedures in this area. Involvement in the TeSLA project, which brought with it the use of technologies for detection of cheating and plagiarism as well as involvement with EU quality agencies has done much to stimulate wider discussion, and the development of plagiarism training and new regulations and procedures.

Whilst the wider use of technologies (particularly anti-plagiarism software) and better student training, both in the nature of plagiarism, and the process of academic writing, were widely accepted, and can be readily implemented, the responses of the students and the teachers presented earlier highlight a number of other important issues which have direct implications for the development of academic integrity in Sofia University, and are likely to be of relevance for implementation in other universities.

Firstly, whilst both teachers and students saw cheating and plagiarism as a very real problem, it was evident that that many students (and some teachers) did not have a clear view as to what actually constitutes plagiarism. Any implementation needs to investigate the present understanding of teachers and students, and to provide an appropriate training programme on the nature of plagiarism.

Secondly, in developing academic integrity policy, institutions need to recognize that the intuitional and teachers’ perspectives may not be shared by students. This study found that students and teachers had quite different perceptions of the approaches adopted by the institution. This indicates a need for improved training for students about institutional policies, but also probably suggests a need for greater consultation with students about the policies and their implementation.

Thirdly, there was a lack of consistency of approach at the institutional level, with teachers developing their own approaches, and with institutional procedures initiated in only a small number of cases. This inconsistency, no doubt, contributed to students’ differing perceptions of policies and support, but it was also clear that the teachers themselves differed in their experiences of academic integrity in Sofia University, which, in turn, resulted in their adoption of different approaches on how to sanction cheating and plagiarism. In moving forward in this area, it will be important to address these differing perceptions rather than addressing the teachers as if they are a homogenous group. It was not possible from this data to determine any underlying reasons for these differences between teachers and this is something that should be explored in further studies.

Fourthly, in implementing academic integrity policies the institution needs to reconsider pedagogy and assessment design. This study found that the students and teachers had quite different ideas about what would constitute effective pedagogy and assessment design in order to reduce cheating and plagiarism. The students looked for more creative and interesting teaching, better alignment of assessment with their needs, and increased teacher support. The teachers looked to designing assessments that are cheat proof. In recognizing academic integrity as a shared cultural issue, institutions and teachers will need to pay more attention to the students’ perspectives in this area.

Fifthly, this study found that there was a tension in the teachers’ thinking between, on the one hand, a desire for clearer regulation and explicit procedures and, on the other hand, a wish to maintain autonomy in decision making with respect to cheating. This tension leads to a degree of resistance to institutionalization of academic integrity policies. We did not find direct comparison with this aspect of the study in earlier studies of academic integrity polices, though we have no reason to believe that this issue is unique to this university. Further, the responses by the teachers in discussing the use of the student authentication and authorship checking system indicated awareness that the adoption of such systems will increase the pressure towards adopting a consistent approach. Institutions need to recognize the likelihood of this form of resistance and to explore how to deal with this tension between clear regulation and explicit procedures and teacher autonomy.

Whilst these implications well have relevance across many higher education institutions they are perhaps particularly relevant to other higher education institutions in a similar situation to Sofia University in Southern and Eastern Europe in the European Higher Education Area, where the widely perceived threats to academic integrity by contract cheating and plagiarism, and the lack of effective policy response provides a context in which the lessons learned may be readily employed. The introduction of technology for student authentication and authorship checking is a relatively straightforward step to take in response to the widely perceived threats, and this, then, can be a catalyst for further reflection on academic integrity more widely leading to changes in institutional academic integrity policy.