Introduction

Research performance is a concept that is broadly associated with resources (e.g. funding, faculty support), the research process and output (e.g. publications, bibliometric indicators, teaching) (Bazeley 2010). Research academics are evaluated by their contribution to knowledge and ideas through research performance. Research performance in the form of publications is one of the most critical indicators of performing scholarly activities, producing knowledge and gaining recognition among peers (Fox 1983; Ramsden 1994). Assessment of university academics’ research performance remains an important issue, with the use of measured criteria dating back to the late 20th century (Lundberg 2006; Smith 2015). In recent years, research performance based on measured criteria (e.g. publications, bibliometric indicators) has been used to assess who gets research grants or funding as well as to determine who qualifies for promotion (Lundberg 2006; Smith 2015; Schneider 2009).

Research takes place in a work environment that may limit or stimulate the development of ideas and the production of knowledge (Fox 1992). Research is conducted within the framework of organizational practices and policies but also relies heavily on the work environment. The work environment includes conditions related to the organizational and psychosocial aspects as well as ergonomic factors (e.g. laboratory environment, office space) in general. This review addresses the organizational and psychosocial work characteristics which have been acknowledged to be among the most potential factors influencing workers risk of ill-health and productivity in organizations. However, studies on the organizational and psychosocial work characteristics as they relate to productivity in academic settings where publication productivity is the most central indicator of performance are scanty (Fox and Mohapatra 2007).

In most countries, regulations and recommendations apply to promoting the organizational and psychosocial work features where employers have the responsibility to promote a good work environment for e.g., (The Swedish Work Environment Authority 2015). The organizational environment are conditions for the work that include but not limited to 1. Management and governance; 2. Communication; 3. Participation, ability to decide for oneself; 4. Assignment of tasks; and 5. Requirements, resources, and responsibilities (The Swedish Work Environment Authority 2015). In previous studies, the organizational work environment have referred to factors such as policies, structure and resources for the job (including department support, incentives, reasonable and clear goals, skills and staffing) which are necessary for any sort of significant research (Bland et al. 2005). The psychosocial work environment are conditions for the work that include social interaction, collaboration and social support from managers and colleagues (The Swedish Work Environment Authority 2015). The job factors related to social interactions are victimization or harassment, job demands, and work climate (Bakker and Demerouti 2007; The Swedish Work Environment Authority 2015).

Numerous studies relate research performance to a set of identified characteristics that influence research performance. For instance, in a recent study, university researchers who had a poor psychosocial work environment were shown to experience ill-health, impaired work performance and increased costs related to attendance problems (Lohela-Karlsson et al. 2018). One of the earliest critical reviews investigated the determinants of research performance (Fox 1983). This was followed by Bland and Ruffin (1992), whose review examined the characteristics of productive research environments from the mid-1960s to 1990. The studies concluded that organizational factors such as goal clarity, research orientation, group climate and culture, organizational structure, communication and resources, as well as the size and diversity of the research group, and leadership were associated with research performance. Other studies support the finding that type of leadership is related to high performance among researchers (Ryan and Hurley 2007). In Widenberg (2003), aspects of the psychosocial work environment including the work climate, researcher’s network and leadership were also found to be important for research performance.

Most of the studies conducted so far typically investigate the impact of a few work-environment characteristics suspected of influencing the research performance of academics across different institutions. Some studies have suggested general models of how these characteristics together impact research performance (Bland et al. 2002; Brocato 2001; Dundar and Lewis 1998; Teodorescu 2000). A few studies have been able to test these models (Bland et al. 2005). Thus, syntheses of existing research looking into the relationship between publication performance and the work environment have been called for in previous reviews. Such study findings can be important for helping university administrators and policymakers to make informed decisions about where to focus efforts to support research performance, since research institutions continue to rely on publication performance.

This systematic review summarizes the evidence about the association between publication performance and the organizational and psychosocial work environment of research academics in a university setting.

Method

This review adheres to the preferred reporting items for systematic review and meta-analysis (PRISMA) guidelines (Moher et al. 2009; Shamseer et al. 2015). The review is not registered because it does not meet the eligibility criterion of dealing with clinical or health outcomes which existing review registers such as the International prospective register of systematic reviews (PROSPERO) aim at.

The PEO framework

The PEO (i.e., population, exposure and outcome) framework was used for the present search. PEO is a framework that can, for example, be used in prognostic studies (Schardt et al. 2007; Richardson et al. 1995). The population was defined as university research academics. The term ‘research academic’ can also refer to clinical researchers while excluding researchers working in institutions not affiliated to any university. There are considerable challenges in designing and conducting intervention studies to capture performance outcomes in relation to the work environment. It may, for example, be necessary to conduct observational rather than randomized studies. Thus, in place of interventions, we refer to exposure to characteristics of the work environment restricted to the organizational and psychosocial work environment in a university context. With this information, this review includes evidence from studies with a range of study designs, including intervention studies where available. The main outcome of interest is publication performance defined as the number of publications, bibliometric indicators, quality assessment of publications (see also Table 1 showing the PEO framework only including population, exposure, and outcome (Khan et al. 2003).

Table 1 The PEO framework
Full size table

Search strategy

With the help of librarians, we formulated a search strategy to identify relevant literature based on the PEO framework. Previous reviews indicated that few studies have previously been conducted on this topic. We therefore decided to set no limitations for the study design, study duration, intervention strategies, follow-up period, control condition, or whether research performance was assessed subjectively or objectively when we conducted the preliminary search. Separate test searches were conducted by a librarian in four databases: 1. Ovid MEDLINE, 2. Embase, 3. Web of Science Core Collection, and 4. ERIC (ProQuest).

After the test searches, a final systematic literature search, developed by expert university librarians, was performed in the Ovid MEDLINE and Web of science databases. A combination of controlled search words and free-text words was used. Ovid MEDLINE was the preferred platform because it is the standard interface for Cochrane Reviews. Further, the databases Psycinfo and Global Health could be accessed via Ovid. Web of Science was also preferred because it is a multi-disciplinary database that covers many research fields and free-text searches are possible. Further supplementary literature searches were performed in specific journals such as Scientometrics, Higher Education, Journal of Higher Education and Studies in Higher Education, in addition to the two databases. The search strategy and search in the databases was completed in March 2018 (S1 Search strategy). The literature search returned 1474 abstracts after duplicates were removed.

Study records

The identified records were collated in the Endnote reference manager version X9. The Endnote record was exported in RIS file format to the evidence synthesis tool CADIMA (Kohl et al. 2018) for further analysis and to facilitate independent screening and cataloging of disagreements between reviewers.

Study selection process

The study selection process was conducted in two stages. Firstly, two reviewers (EA and CB) independently scanned all titles and abstracts for potentially relevant studies. Titles and abstracts were included for full text reading if inclusion criteria were met. Any ambiguity about the eligibility of an article was flagged and discussed with the principal investigators of the team (IJ and GB) until consensus was reached. The second stage of the study selection process was reading the full text of all potentially eligible articles, which were then independently screened by four reviewers (EA, CB, OPE and EBB) in accordance with the a priori inclusion criteria. The articles deemed to be relevant continued to the risk of bias assessment. Disagreements between reviewers about eligibility were resolved by discussions in the research team until consensus was reached.

The main inclusion criteria for identifying relevant literature were:

  1. 1.

    The population study participants should be clearly described and relevant, i.e. research academics.

  2. 2.

    The exposure investigated should be clearly described, measured and relevant, i.e. the organizational and psychosocial work environment in a university context, including faculties, institutions, departments and/or divisions.

  3. 3.

    The investigated outcome should be clearly described, measured and relevant, i.e. publication performance measured as number of publications, bibliometric indicators, quality assessments based on publications)

  4. 4.

    The study should examine the association between the work environment and publication performance.

  5. 5.

    The study should be written in English and published between 1990 and 2017 as original study in a scientific journal. The period 1990–2017 was chosen based on the assumption that working life conditions and the assessment of research performance have so dramatically changed during the last 25 years that the external validity and relevance of older studies may be questioned.

Previous reviews on the topic, studies that focus only on non-university research, and studies that did not meet the above inclusion criteria were excluded. Qualitative study designs were also excluded. This review only examines quantitative study designs.

Data extraction process

All eligible articles were summarized using a pre-set form to extract data from each study. In order to facilitate easy data collection and analysis processes, we assigned a unique identifying number to each variable field so they can be programmed into fillable form fields in the CADIMA software that was used for data extraction. The number was also used to generate coded data for analytical procedures.

The main domains on the extraction form, adapted from the Cochrane Collaboration, consist of article details, study characteristics (e.g. methodological steps), details of exposure, outcome measurement and study results. The specifics of each domain are outlined in Table 2.

Table 2 Data extraction form
Full size table

Full data extraction began only after we obtained satisfactory agreement between the authors, after some rounds of pilot testing. Subsequently, each included study was summarized through data extraction by one team member and verified by another reviewer. The reviewers resolved disagreements by discussion, and one of two arbitrators (IJ or GB) adjudicated unresolved disagreements until consensus was reached.

Critical appraisal of methodological risk of bias

We used a pre-set assessment form developed by the Swedish Council on Health Technology Assessment (SBU) to assess risk of bias (SBU 2019). This risk of bias assessment form is divided into several sections. These include potential selection bias; potential bias in exposure; potential bias in outcome measures; potential bias in loss to follow-up; potential bias in reporting results; and conflict of interest bias.

Each article was graded by the reviewers (EA, CB, OPE and EBB) as having a low, moderate or high overall risk of bias. The authors were trained at group meetings in how to assess the study’s risk of bias and how to reach consensus. After the training sessions, each member of the pair performed the assessments independently, after which disagreements were discussed by the reviewers. If disagreements remained, a joint discussion with all the members of the research team was held until consensus was reached. For a more detailed description of the criteria appraisal form, (see S2 Criteria appraisal form).

Data synthesis

The authors used a descriptive or qualitative summary of studies in exposures and outcomes. Data synthesis was not conducted by applying meta-analysis due to the differences in study design, population (for example, researchers in different disciplines), exposure (i.e. any type of work-environment characteristics), and outcome (i.e. any number of publication type or bibliometric indicators with varying time span in years or publication quality assessment).

Results

After applying the inclusion and exclusion criteria to 1473 identified titles and abstracts, 32 articles were judged to be relevant and were critically reviewed. An adapted PRISMA flow diagram (Fig. 1) shows the final numbers in the resulting study publication. The reasons for excluding articles were recorded at the full-text review stage.

Fig. 1
figure 1

PRISMA flow chart describing the process of study selection

Full size image

Description of included studies

These 32 studies form the basis of the findings. Most are retrospective cross-sectional studies, but there are also five prospective cohort studies and one pre-post study, (S3 Description of included studies). No intervention studies were found. The source populations for the studies varied geographically, with 14 studies from the US and Canada combined, two from the Netherlands, three each from the UK and Australia/New Zealand, and one each from, Nigeria/Ghana, Portugal, Italy, Japan, Norway and Sweden. Other studies had combined populations from different continents. The study participants were mostly staff affiliated with university departments covering academic fields such as the biological, life, agricultural and chemical sciences, medicine and the social sciences. Six of the studies focused on staff who divided their time between research and clinical work. The unit of analysis for one study was at the aggregated university department level. That sample was therefore not reported. The sample sizes ranged from 21 to 21,840 (48,277 in total; mean sample size: 1557; median sample size: 470). Four of the studies were conducted in the 1990s, 14 in the following decade, while 14 were published after 2010.

An overall description of the details of type and the level of exposure are presented in S3 Description of included studies. The types of exposure looked at were organizational (such as research management practices, appointment type, faculty recruitment, department orientation and size) and psychological (such as climate, culture, support, collaboration, leadership). Some studies included both exposures. Most studies examined exposure at (1) departmental/organizational level (n = 6), with the aim of aggregating data on work environment, (2) individual level (n = 23), with the aim of generating data on the perception of the work environment, and (3) organizational- and individual level (n = 3), with the aim of generating data on the work environment at the individual level and aggregating them to explain differences in team and individual outcomes.

From the studies we identified nine broad work-environment concepts, classified as organizational and psychosocial factors, that affect research productivity (see list of points 1‒9). Most of the factors seem to be organizational in character (points 1‒8). However, there are some factors that can be characterized as psychosocial factors in their influence (point 4) but may also be associated with organizational factors—for example, if the management of a department has a clear policy on how to facilitate communication in the organization (support structures in point 6).

Organizational and psychosocial work-environment factors identified.

  • 1. Appointment type joint appointment such as academic-clinician appointee, appointment status such as tenure or non-tenure, senior or junior faculty

  • 2. Type of contract temporary or permanent, working week, teaching load, clinical workload, administrative activities.

  • 3. Faculty recruitment strategic recruitment of PhD and postdocs and research scientists.

  • 4. Communication and climate characteristics collaboration, networks, research contacts, research interaction, information exchange with peers, collegiality, competition, openness.

  • 5. Department size and staff composition total number of research scientists, quota of senior researchers, quota of PhDs and postdocs, research group size, research unit size, team composition, number of PhDs produced.

  • 6. Research management practices performance monitoring, performance-based funding, benchmarking and concentration, division of labor, individual incentives (staff appraisal and performance rewards), support structure (workshops, mentoring or mentorship, additional funding opportunities), department support, and upgrading research qualifications, structure—control, autonomy, hierarchical, leadership styles.

  • 7. Department rewards and funding support research grants, research benefits, incentives, training grants, scholarship support or research-related gifts grants.

  • 8. Department orientation teaching-oriented departments, research-oriented departments.

  • 9. Psychosocial characteristics discrimination, culture, job satisfaction, work-family life balance.

Publication performance was defined by most studies as published research and research quality ratings by an external peer reviewer. The number of peer-reviewed articles and books from the previous years were frequently used to measure research performance. Articles produced by academics in the previous 2 years, the previous three or more years or even lifetime publications were also used as outcome most frequently. The qualitative rating of performance (i.e. publications from previous years) was performed by external evaluators who aggregated them in order to compare departments. Other studies also converted data, especially records of published work of researchers, to outcome measures such as impact factor, efficiency scores and other weighted measures of publication performance, such as dividing total publication by number of authors. Publication performance was measured by various methods such as surveys or questionnaires and performance records. Studies that used performance records consulted publication records in a subject field and in well-known databases such as Web of Science or MEDLINE and peer reviewed publication quality ratings.

Association between the work environment and publication productivity

Organizational work environment characteristics that demonstrated a positive association with group publication productivity include factors such as a department having a defined research agenda and expectations; strategically recruiting enough academic staff to achieve goals; formally assigned mentors; a well-developed network of colleagues outside the department with whom to discuss research; less teaching, and satisfactory department resources.

Some of the studies concentrate on the connection between aspects of the psychosocial work environment and publication productivity (see Table 3 and S4 References of included studies). The results based on data from 15 studies generally show positive associations. These include job stress, organizational culture and cooperative climate, job satisfaction, work-family life balance, and collaboration and research interaction with colleagues outside the department. Some studies demonstrate a negative or no direct association between organizational culture and publication productivity. Here, it is rather job satisfaction that mediates the observed relationship between organizational culture and publication productivity. However, some studies also show no direct relationship between job satisfaction and publication output. Some leadership traits that researchers perceived as effective, such as fostering autonomy, also positively affect publication performance.

Table 3 Description of included studies and exposures measured
Full size table

Quality assessment

The overview of risk of bias in the 32 studies is summarized in Table 4. Most of the studies were assessed to have a moderate risk of bias, mostly based on the outcome measure bias. According to the SBU risk of bias form recommendations, there are four important criteria for evaluating the quality of a study (sample selection, exposure, outcome, and non-response bias). If there is a lack of information in one or more of these key domains of bias it is difficult to evaluate risk of bias. As a result, these studies are classified as moderate or high risk based on ratings in the sub domains. Thus, the studies were evaluated taking into consideration all the domains except bias due to slight deviations from reporting and conflict of interest. These domains were not suspected to have a great effect on the quality of the study. Using the SBU risk of bias form recommendations, we defined an article as having a high risk of bias if there was a high risk of bias in one or more of the key domains of bias. About 24 studies were evaluated as having moderate risk of bias, as most of the studies used either poor sample selection, self-reported publication through questionnaires or interviews and/or had a high dropout rate. A moderate risk of bias due to self-reported publication does not provide a valid measure of publication performance that is comparable to reliable publication records in terms of quality or quantity. Seven studies were rated as having a high risk of bias, while only one study was rated as having a low risk.

Table 4 Critical appraisal outcomes
Full size table

Discussion

In this study we have identified relevant literature from independent database sources to investigate which aggregate work-environment characteristics can explain the publication performance of research academics, both in terms of quality and quantity. This review study gives new information about the characteristics of productive research environments from the mid-1960s to 1990 that a previous review by (Bland and Ruffin 1992) had also examined. The present review is an important contribution and has a practical relevance as it highlights where academics might need support and what areas of the work environment can be improved to bolster publication productivity.

Main findings

The results suggest that the factors which affect publication performance differ markedly for different research fields, institution or department types, and even countries. Nevertheless, the commonalties in the findings offer a greater insight into the relationship between the work environment of academic researchers and their publication performance. From here, we discuss the most important findings of those studies that were evaluated as having a low to moderate risk of bias.

The main factors covering organizational work environment concerns management practices and managers’ relations with all their employees. The former suggests that high management involvement is important, while the latter underlines the role the employer’s awareness of leader relations plays in enhancing workplace conditions. Management practices such as performance monitoring, strategic recruitment and research agenda and cooperative leadership styles that support autonomy were associated with good publication productivity. In short, management practices and leadership are without question influential factors that affect all other organization characteristics and in turn affect academics’ publication productivity. This supports the findings of Bland and Ruffin (1992).

We found that the effects of team composition, collaborative patterns, workplace climate, and employees’ perception of job satisfaction, explain inter-departmental publication productivity differentials. The composition of the team encompasses aspects of team structure, membership, staffing, and diversity (Taylor et al. 1996). Studies that looked at variable team composition found that a high proportion of senior research staff and postdoctoral researchers—as opposed to PhD students—seems to positively affect a department’s publication productivity (Felisberti and Sear 2014; Fox and Mohapatra 2007).

In the literature, communication and collaboration among academics are often discussed together. Previous reviews have found that internal and external communication among academic colleagues has a positive impact on publication performance. In this review, staff who have frequent contacts with other departments and/or peers were more likely to be highly productive. In fact, research collaboration outside an academic’s own university department seems to be a better predictor of publication productivity than collaboration within one’s department (Fox and Mohapatra 2007; Lee and Bozeman 2005; van Kessel et al. 2014). This finding is supported by previous studies that have suggested that collaboration with other departments or universities is the type of collegial collaboration that is most important for publication productivity (Bland and Ruffin 1992).

Although it is quite clear that department climate influences publication productivity, it is not possible at this point to suggest that organizational culture has the same effect. Previous studies have, similarly, concluded that there may be a link between publication productivity and group climate (Bland and Ruffin 1992). This also implies that productive staff members influence their colleagues in a work environment where there is open communication and a good exchange of ideas with peers. Although in the corporate world the culture of an organization (i.e. what makes it distinctive) is important for productivity, the literature about academia is not in line with this. Some studies find a direct association between organization culture and publication productivity while other see no connection at all (Desselle et al. 2018; van Kessel et al. 2014). Some mediation studies performed on the relationships indicate that it is rather job satisfaction that is mediated in the observed relationship between organizational culture and publication productivity (Kato-Nitta and Maeda 2016).

Apart from team composition, one other frequently mentioned factor related to publication productivity is research group size. The literature covers different types of research groups ranging from a few people (at least three) to whole departments or institutes with at least one leader (Bland and Ruffin 1992). Previous reviews have concluded that research performance increases as research group size increases. However, this is only the case up to an inflection point at which the benefits of being large become deleterious—a diminishing effect. When studies in our review control for the growth in group size, they consistently find that the size effect accounts for a decreasing number of publications (Cook et al. 2015; Groot and Garcia-Valderrama 2006; Smeby and Try 2005). Large research group sizes are associated with higher publication quality, but after a certain point a growth in size of the research group can negatively affect publication quantity.

Other factors that seem to have been less researched in the time frame of this review but have a bearing on the work environment are work practices (e.g. number of projects), workload or task, proportion of female academics, the appointment type and/or contract and financial resources. We found only a few studies that examined academics’ publication productivity in relation to the research orientation of the department, appointment type, and subtle gender discrimination (Eagan and Garvey 2015; Rothausen-Vange et al. 2005; Sax et al. 2002). Studies found no differences between men and women in more research-oriented departments in terms of the impact of work environment on publication performance. Nevertheless, women's publication performance tends to lag when they have a work-family life imbalance and encounter subtle discrimination. For example, the productivity of specific groups such as women with a minority background tends to decrease when they are discriminated against in the workplace. There is thus a risk that women's performance will be evaluated incorrectly, with an underestimation of the effect of work-environment factors.

Further, a very heavy workload alongside research activities negatively impacts academics’ publication performance. We see examples of this whether it be teaching load, administrative load or volume of work for clinical academics (Kessler et al. 2014; Taylor 2001). This also implies that teaching and clinically oriented departments may have a lower publication volume than research-oriented departments. However, having a joint appointment (i.e. both clinical and research) may not affect publication performance if the staff member is contractually allowed to allocate a certain amount of time to research (Jensen et al. 2020). In a study of job resources for academic productivity measured by publication and credit points, it was shown that administrative and technical support could stimulate research publications. However, such support had adverse effects on credit points from teaching if it is skewed towards research (Christensen et al. 2018). Research resources such as colleagues, assistants, technical consultants and other core support facilities have also been shown to positively influence publication performance. This finding may not be that surprising since the support of especially the human resource unit tend to be tied into how effectively staff can work, communicate, divide task as well as have a high level of wellbeing and job satisfaction.

Financial resource referring to internally or externally sourced funding are considered as essential for publication quality indicators. In Groot and Garcia-Valderrama (2006), external funding from the national level was associated with research quality indicators. Furthermore, in a recent study of external funding it was shown that not only does external funding influence publication quality indicators (e.g. citations) directly, but it also influences the relationship between the organizational and psychosocial work environment and publication productivity (Jensen et al. 2020). There is support for this finding in research showing that performance-based funding has positive effects on publication productivity (Aghion et al. 2010; Christensen et al. 2018).

Limitations of the review

In this study we have reviewed several studies that examined the relationship between publication productivity and several independent factors in the work environment. The studies use a variety of methods to measure these independent work-environment factors. There may be a tendency for researchers to prioritize data on factors that are easy to study in large population groups. A consequence of this may be that research may tend to focus on organizational and psychosocial factors that can be investigated by means of questionnaires.

Secondly, the choice of measure of work-environment factors also has significance for the reliability of data on exposure in the work environment. It also matters how much data is collected and how data collection is organized over time. More data perhaps gives more secure information, but how many people are studied, how many measurements are made per person, and how these measurements are distributed over time also play a role. For instance, many studies in the review use cross-sectional designs but also measure exposure and outcome at time points, which can give misleading results since the outcome is mostly retrospective and the exposure is now. With this method it is difficult to determine what influences or interferes with what. The work environment, especially organizational and psychosocial factors is important for academics’ publication performance. Thus, measurement improvement of work environment factors is needed.

Thirdly, the measures of perceived work-environment characteristics are also of significant concern because studies use self-reported questionnaires to capture organizational and psychosocial factors. This notwithstanding, the findings of this review, which is aimed particularly at policy makers, those working in higher education administration or research institutions, and other stakeholders, add to our understanding of the direct impact of work-environment factors on the publication performance of research academics.

Conclusion

The findings in the studies reviewed here suggest that highly productive research academics tend to be found in departments that emphasize proactive management practices which support, monitor and reward publication performance. Effective leadership characteristics which positively affect publication performance tend to be cooperative or participative, specify and coordinate a clear research orientation, have clear expectations, and encourage autonomy. Departments that recruit academic staff strategically and help to create a balanced workload besides research and publishing can also influence their publication productivity positively. Furthermore, the studies suggest that improving faculty perceptions of psychosocial factors (including cooperative climate, atmosphere of wellness and non-discrimination, and encouraging research collaboration with colleagues outside the department) might improve publication performance.