We used a scoping review methodology (Tricco et al. 2016) following the guidance published in the Joanna Briggs Institute (JBI) Review's Manual (Peters et al. 2015).
Concept and Context
The concept of this review was that there is a wide range of existing practices/guidance documents in RPOs and RFOs with implications on RI promotion and avoiding research misconduct, as well as that these guidance documents may vary in their scope, means of addressing RI issues and stakeholders (e.g. policymakers, researchers, reviewers, students, committees and boards) to which they are directed.
This review examined the practices/guidance documents for RI promotion and avoiding research misconduct related to RPOs, RFOs, and other various stakeholders involved in research (policymakers, researchers, reviewers, students, committees and boards) with the aim of building an overarching view of the current situation regarding RI guidance. Moreover, the review examined RI guidance documents that exist in different research fields and are related to different research phases (research planning, conducting, dissemination and evaluation). It also explored the guiding principles presented across documents, as these principles could serve RPOs and RFOs in creating and preserving the RI environment (NASEM 2017).
The main eligibility criterion for the documents from peer-reviewed journals and grey literature was that these documents addressed any aspect of RI in RPOs and/or RFOs. By any aspect we meant RI issues related to different phases of the research process and with the different RI focus. For example, authorship issues, data management issues, investigations of research misconduct, RI education and other.
A description or summary of RI practices had to be provided in these documents in order for them to be included in the analysis. Editorials and commentaries were included as well when they met the above mentioned criterion.
We included all types of guidance documents on RI issues as ‘practices’. This included guidance in the form of codes, guidelines, checklists, and standard operating procedures but did not exclude other types of guidance documents. Hence, the list of the different forms in which guidance for RI was presented was updated during the process of document screening and analysis.
Although the majority of documents contained the type of guidance on RI issues in their title or description, for documents that were not defined regarding the type of guidance we used the following criteria:
Code—a document providing general, rather than detailed guidance on ethical standards, principles, values, and rules of behaviour;
Guideline—a document more specific than code in providing guidance; a document providing specific instructions for performing a certain task or achieving a certain goal;
Checklist—a document presented as a clear list of items to be done, checked, or considered in performing a specific task;
Standard operating procedure (SOP) —a document providing detailed, step-by-step instructions for carrying out routine tasks and aimed at achieving uniformity and efficiency;
Flowchart—a document presenting guidance in the form of a diagram representing a workflow or process;
Legal document—a document established by a government or other authority, empowered by law, and outlining legal consequences; and
Policy—a document established and implemented by an organisation, containing adopted principles, rules, and procedures for conducting certain actions.
Other types of guidance used as a category in this review included reports, statements, declarations, white papers, as they had such a term set out in the title or description of the document.
Since academic integrity comprises fundamental values relevant for researchers and their work (Fishman 2014), documents related to academic integrity were included into our analysis whenever they reflected on research performance or researchers’ behaviour, be it professional or unprofessional. Further, documents related to research ethics (RE) were also included if they addressed issues similar to RI, since RE and RI are not always clearly distinguished (Komić et al. 2015).
The search addressed practices relating to different scientific disciplines, categorised in advance as—medical sciences (including biomedicine), natural sciences (including engineering), social sciences, humanities, and ‘research in general’. The latter term was used to map the practices that were not developed for RI in a specific field, but rather to be applicable across different scientific fields.
The search of bibliographic databases did not have geographical or language restrictions, while the grey literature search was limited to documents in English because of the possibility of retrieving a large number of documents that would need to be translated in order to be analysed. Since research misconduct emerged as an important problem in the late 1980s and 1990s (Resnik and Shamoo 2017), only the materials dating from 1990 onward were included in the screening process. The reason for this was based on the need for ensuring applicability and contemporaneity of identified practices and exploring currently existing gaps in knowledge.
Search of Bibliographical Databases
The search strategy was developed by three researchers who were assisted by a librarian specialised in systematic review search methodology. The development of the search strategy aimed at high sensitivity and included a broad approach to the field, based on the need for the identification of as many relevant documents as possible. As a starting point in the development of the search strategy, we used terms from the European Code of Conduct for Research Integrity (ALLEA 2017). The search strategy is available in Appendix 1 (Electronic Supplementary Material). We searched Scopus, Web of Science (WOS), Medline and PsycINFO bibliographical database. The search of Medline, WOS, and Scopus was performed on 18 February 2019, while the search of PsycINFO was performed on 12 February 2019. The obtained data were exported to the EndNoteTM tool (Clarivate Analytics, Philadelphia, PA, USA).
Search of Grey Literature Sources
The search of grey literature encompassed several different sources: Open Grey database (Open Grey, INIST-CNRS), World Conferences on Research Integrity (WCRI) (The World Conferences on Research Integrity) website, the Community Research and Development Information Service (CORDIS) database (European Commission), Office of Research Integrity (ORI) (The Office of Research Integrity) website, European Network of Research Integrity Offices (ENRIO) (The European Network of Research Integrity Offices) website, the National Academies of Sciences, Engineering, and Medicine (NASEM) publications (The National Academies of Sciences, Engineering and Medicine), Science Europe publications (Science Europe), Mutual Learning Exercises (MLE) on Research Integrity reports (European Commission), and the League of European Research Universities (LERU) publication (The League of European Research Universities). Details of the search of grey literature sources are presented in Appendix 2 (Electronic Supplementary Material).
Selection of Documents
For documents that were retrieved by the search of bibliographic databases, duplicates and articles dating before 1990 were first removed and then the screening of the titles and abstracts was performed. The screening was conducted independently by two reviewers. In order to precisely define the criteria and the screening process, as well as to ensure that both reviewers would perform the task in the same manner, the reviewers first performed a pilot screening of the titles and abstracts of 100 records. After the pilot screening, they proceeded with the screening of the titles and abstracts of all the documents, after which they compared and discussed the obtained results in order to decide which documents would be included in the full-text analysis. In cases of disagreement, the final consensus decision was reached after a discussion with the third reviewer. In the following step, the three reviewers performed a full-text assessment of the documents in order to decide whether they were eligible for inclusion into the final analysis. To be included in the final analysis, a consensus had to be reached by at least two reviewers. In cases of major disagreements, the material was discussed with an additional reviewer. Documents that were not written in English were translated using tools such as Google Translate to explore whether they fulfilled the eligibility criteria. Reference lists of the documents included in the final analysis were screened by one reviewer to identify additional documents (sources of practices).
For grey literature sources, one researcher performed the search to identify documents that specifically met the set eligibility criteria. This means that all available documents were not extracted and screened, but rather the full-text screening was performed simultaneously with the search.
Data Extraction Process
For the documents from the bibliographic databases included in the final analysis (Fig. 1), two researchers performed the data extraction. The list of categories to be extracted was defined in advance and was continually updated by each researcher during the charting process. The list is available in Appendix 3 (Electronic Supplementary Material). The categories were discussed by authors to reach the consensus on the final list. The data extraction of the material obtained from the grey literature search was performed by one researcher.
The following data were extracted: author(s) (for documents from bibliographic databases); title (for documents from bibliographic databases); year of publication; reference type, i.e. journal article, book, book section (for documents from bibliographic databases); journal (for documents from bibliographic databases); country of origin; research fields, i.e. humanities, social sciences, natural sciences (including engineering), medical sciences (including biomedicine), research in general; name of the practice; type of practice (type of guidance on RI issues), i.e. code, guideline, checklist, SOP, legal document, report, declaration, statement, flowchart, white paper, policy; whether the practice was more related to RPOs or RFOs or both; whether the practice was more related to institutions (organisations) or individuals or equally to both; target audience in practice, i.e. researchers, research groups, policymakers, funders, students, mentors and supervisors, committees and members of committees, RI offices and officers, RI advisors, ombudsman, reviewers, administrators, whistle-blowers; description of the source of practice (for grey literature); principles addressed in practices. Documents were also categorised according to the phase of the research process—planning, conducting, dissemination, evaluation—as well as RI violations and resolutions and RI promotion. Within each research process, several RI topics were identified based on their relatedness to the process. Since the main research processes were defined broadly, the grouping of RI topics which were more related to the specific issue enabled us to capture the most prevalent RI issues addressed across practices. Two researchers independently developed the lists of RI topics during the extraction process. After finalising the analysis, the lists of RI topics were compared to detect any overlaps. The list of topics was finalised through a discussion and consensus between two researchers and in consultation with a third researcher.
After the data extraction, all the documents were summarised and analysed based on their geographical origin, the scientific field and organisational (RPO or RFO) origin of the identified practices, the types of practice (the type of guidance), and the target group to which the practices were directed. We also categorised the documents based on the research processes and RI topics addressed in them.
Furthermore, we extracted the guiding RI principles that were explicitly addressed in the documents. This means that RI principles had to be explicitly mentioned and explained in the chapters or parts of the text. The documents just stating RI principles without further elaboration were not included in the analysis. We mapped the extracted principles to the principles presented by the All European Academies in the European Code of Conduct for Research Integrity (ALLEA 2017) and those presented by the US National Academies of Sciences, Engineering, and Medicine in the book Fostering Integrity in Research (NASEM 2017). The aim was to observe the similarity in principles, and terms used to address the guiding principles. We used these two documents because of their wide recognition and acceptance, as well as their up-to-dateness (both were updated in 2017). The extracted principles were mapped by one researcher and checked with the second researcher, upon which the agreement was reached for the final mapping.