Unsustainable Growth, Hyper-Competition, and Worth in Life Science Research: Narrowing Evaluative Repertoires in Doctoral and Postdoctoral Scientists’ Work and Lives

The new field of valuation studies is concerned with valuation as a social process, that is, with how the worth (both economic and social) of people, objects and practices is produced, assessed, distributed and negotiated (Kjellberg et al. 2013; Lamont 2012)². Many contributions in this line of work share an interest in viewing valuation as practice and in examining how acts of valuation relate to actors’ practices and decisions.

This raises the issue of reactivity, which is how actors react to the fact that their practices are being evaluated and how they change their behaviors to improve their evaluation performance. Espeland and Sauder (2007) describe how academic institutions adapt their behavior to achieve better rankings. They show that the reflexive consideration of the evaluation process and the resulting re-shaping of institutional practices become an inextricable part of institutional work. Writers who comment on the impact of New Public Management on academic institutions have pointed to the multitude of metric assemblages that measure the conduct of individual researchers (Burrows 2012; Felt and Stöckelová 2009), the normative power that they exert (Stöckelová 2012), and the emotional burdens that they can cause (Sparkes 2007). Numerous studies have documented the impact that these new ways of measuring and defining research excellence have on collective actors, such as higher education institutions and scientific fields. They have found that considerable diversity exists in how these collective actors are affected by and react to these changes (Gläser et al. 2010; Paradeise and Thoenig 2013). However, little is known about how such changes affect different status groups within a field or institution. Only a few studies investigate how researchers relate to being evaluated (Linková 2014), how they engage in valuing their own work (Rushforth and de Rijcke 2015), and how this valuation shapes the way that they work and live as researchers and their decisions about whether to stay in or leave academia.

Studying the nexus between acts of (e)valuation and research practices raises the question of how to conceptualize the figures of valuation that actors in a specific domain (habitually) draw on to inform, orient or justify their actions. Valuation studies scholars have developed concepts to analyze how actors account for their valuations and their actions related to them. Boltanski and Thévenot (2006) have proposed six different orders of worth that societal actors refer to in justifying their actions, each with its own distinctive principles of evaluation, its own grammar of associating ideas and concepts and its own material tests for determining the worth of individuals or objects. They stress that actors’ capacities to reflexively relate to different orders of worth are acquired through socialization experiences in which they and their practices are evaluated according to specific orders of worth. Subsequent authors have taken up Boltanski and Thevenot’s interest in studying how actors refer to figures of valuation in their practices, but they have mostly found that their deductive scheme is too rigid for the empirical study of valuation as practice (Heuts and Mol 2013; Lamont 2012; Stark 2009). Rather than limiting their interest to how a fixed set of pre-defined orders of worth is referred to in social interactions, they have proposed empirically re-constructing the evaluative principles (Stark 2009) or registers of valuation (Heuts and Mol 2013) that are specific to particular empirical situations and studying situations in which it is uncertain which evaluative principle should be applied. In analyzing valuation practices in specific empirical settings, valuation studies scholars have distinguished between hierarchical and heterarchical constellations, depending on whether a specific situation is dominated by one specific evaluative principle or if a larger repertoire is available to the actors (Lamont 2012; Stark 2009). Heterarchical contexts are assumed to provide better conditions for social resilience because they offer a plurality of repertoires in which individual worth can legitimately be claimed. Studying high-tech companies, Stark (2009) argues that heterarchies also support intellectual innovation. His work shows that these companies thrive on ambiguity among coexisting evaluative principles at work, which creates uncertainty, enables entrepreneurial action and ultimately fosters organizational reflexivity.

Our approach in this paper shares these authors’ concerns to empirically re-construct rather than deduct patterns in how researchers ascribe value. Prior work has argued that researchers experience increasingly normative ways in which research in general and their own work and lives in particular are valued (Burrows 2012; Sparkes 2007). We will not limit our analytic gaze to the normative principles that are criticized in this literature or to young researchers’ reactions to them. Instead, we will attempt to map the breadth of different evaluative principles that young researchers employ to ascribe value to people, objects or actions when they discuss their current and future lives as researchers. We will define an evaluative principle as any logic or set of rules that our respondents explicitly or implicitly refer to when making a statement about worth in a particular situation. Our analytical work will focus on describing these evaluative principles, and on studying which group of young researchers (PhD students or postdocs) applies which principles in which situations. For this purpose, we find it important to consider that young researchers not only use evaluative principles to evaluate - that is, to assess the worth of their work and those of others. Researchers also employ evaluative principles in actions that aim to valorize (Heuts and Mol 2013; Vatin 2013) - that is, to build and increase the worth of their work and lives in the sciences. For example, when planning and conducting their next project, researchers consider the principles along which their work will be evaluated. Therefore, evaluative principles become important elements in how researchers think about and plan their biographical futures in and beyond science.

Evaluative principles will be the central focus of our empirical analysis. However, although speaking of evaluative principles or registersof valuation aptly stresses the situated and situational character of valuation practices, the evaluative principles used in acts of valuation also draw on wider moral/discursive and material infrastructures and institutionalized social practices of valuation (Kjellberg et al. 2013). Prime examples include the metric assemblages that pervade contemporary academia, the calculative infrastructures that they build on and institutions’ heterogeneous ways of using them (Burrows 2012). Therefore, we will use regimes of valuation as a concept that complements our empirical analysis of evaluative principles. Thus, in this paper, evaluative principles denote how worth is ascribed and argued for in a concrete situation, and regimes of valuation point to the broader discursive, material and institutional background this concrete evaluation draws on. We assume that regimes of valuation are comprised not only of institutionalized discourses, practices and material and digital infrastructures, but also of people living in, complying with and resisting these very regimes. These acts of compliance, resistance and debate make regimes dynamic, perhaps causing them to change³. In our understanding of the term, speaking of regimes sensitizes us to consider the material and discursive structures that make patterns of valuation durable (Kjellberg et al. 2013), but also to take into account actors’ own agency in relation to these structures⁴. Regimes of valuation suggest particular evaluative principles to actors who then apply, modify or reject them in concrete situations. Some regimes of valuation have more normative purchase on actors’ practices than others do, and they may thus restrict the actors’ capacities to act against the evaluative logic suggested by this dominant regime. A regime’s normative power may be expected to correspond with the degree of its institutionalization in discourses and practices. In analyzing our material and re-constructing regimes of valuation, we must look for traces of how our respondents ground the concrete valuations that they make and the principles that they use to do so in a broader narrative frame, which legitimizes this particular form of valuation.

This paper attempts to study young life science researchers’ cultures of knowledge production through the lens of valuation practices. In doing so, it utilizes a less comprehensive approach than classical studies of cultures of knowledge production, which aim to analyze the co-production of epistemic and social orders in entire fields of research (Knorr-Cetina 1999; Traweek 1988). This line of work has been surprisingly reluctant to engage with institutional and organizational changes and the different ways in which they affect specific cultures of knowledge production (Garforth and Kerr 2010). Additionally, it has contributed less to understanding how individual researchers navigate and cope with the complex realities of contemporary research landscapes. A more person-centered approach seems better suited to tackle this complexity.

In a study of contemporary knowledge cultures in academia, Felt (2009) recently suggested studying “epistemic living spaces” or researchers’ individual or collective perceptions of “the multi-dimensional structures—symbolic, social, intellectual, temporal and material—which mold, guide and delimit in more or less subtle ways researchers’ (inter)actions, what they aim to know, the degrees of agency they have and how they can produce knowledge” (ibid., 19). Focusing on the person as the agent of knowledge production, this concept aims to explore how ways of knowing in academia (i.e., epistemic approaches) are coproduced with ways of living in academia (i.e., the cultural specificities of particular research fields and geographic contexts). This concept allows for an analysis of how societal imaginaries and policy framings as well as changes in research assessment and monitoring exercises are experienced by individual researchers and how they affect their practices. From our perspective, regimes of valuation constitute one particularly crucial structure that molds practices in contemporary research.

Following this perspective, we have developed a specific methodological approach to capture the interrelations between the epistemic and social aspects of researchers’ epistemic living spaces. We call this approach the “reflexive peer-to-peer interview.” In line with recent work in very different traditions of qualitative research (Lynch 2000; Mills and Ratcliffe 2012; Plesner 2011; Stark 2009), we conceptualize reflexivity not only as a crucial virtue for social science researchers, but also as a capacity of social actors in general that constitutes a productive resource for qualitative studies of social phenomena. Our approach prompts our interlocutors to reflect on how the biographical, institutional and epistemic dimensions of the decisions they take related to their past, current and future careers are entangled. In this paper, our analysis will focus on respondents’ narrations of the role of valuations in their current practices and in their plans for the future. Due to the limited scope of a journal article, we cannot relate respondents’ valuation practices to their wider biographical backgrounds, and we will only bring in past experiences when they are crucial for analyzing present valuation practices.

The basic structure of the interview conversation follows that of a biographical or life story interview. However, rather than inviting only one linear narration, we used interviewer interventions to foster explicit reflections on the conditions that had shaped each individual’s perspectives, plans and decisions. These interventions were accomplished by (a) temporalizing, i.e., inviting the interviewees to relate past, present and future contexts and decisions to one another; (b) comparing, i.e., inviting the interviewees to compare their perspectives and arguments with those of other scientists who were relevant to them (e.g., a supervisor or a colleague who chose a very different career strategy); (c) perspectivizing, i.e., asking the interlocutors to retell particular aspects of their biographies from different viewpoints (e.g., adding a narration of the social context of their development to a story of seemingly purely epistemic choices) and (d) relating their experiences and practices to specific buzzwords in current science and science policy discourses, such as “mobility” or “competition.” Each of these strategies not only fosters reflection on a general level but also invites the interview partners to provide more explicit accounts of the evaluative principles related to their perspectives, actions and decisions. In this process, the interviewer plays a very active role (Holstein and Gubrium 1995) by attempting to induce the respondent to reflexively explore her or his story and self-presentation. In our experience, this process is aided by the shared role of the interviewer and the interviewee as academic peers, although this particular role constellation also introduces limitations that must be reflected in the analysis (e.g., that the interviewer and interviewee might tacitly share a particular definition of the quality of research).

Our analysis is based on interviews with 16 PhD students (9 female, 7 male) and 16 postdocs (8 female, 8 male). All respondents were employed in research groups at public universities and academies, which a project advisory board from the life science community had identified to be relevant research laboratories in a key subfield of the life sciences in Austria. These subfields included “red” (biomedical), “green” (plant) and “grey” (microbial) life sciences and bio-informatics. The interviews are part of a larger sample of 51 interviews in total, in which senior researchers were also interviewed. As the sampling process was based on the research groups, senior researchers, postdocs and PhD students were interviewed in each research group, which allowed us to better compare their perspectives, e.g., on living and working together in this specific group.

Approximately half the post-docs were born in Austria, while the other half were born elsewhere. In addition, seven of the 16 PhD students were not born in Austria, and they had come to Austria to earn their PhD. Whereas all interviewed PhD students were well into the second halves of their theses, the postdoc sample included respondents with quite different research experiences, ranging from researchers in their first postdoc year to others who were returning to Austria after two postdocs abroad. PhD students’ ages ranged from mid-20s to early 30s, with the highest proportion of PhD students being in their late 20s. Postdocs’ ages ranged from late 20s to early 40s, with most being in their early 30s⁵.

The interviews were conducted in English or German between 2007 and 2009. They lasted between 90 and 150 minutes and were recorded and fully transcribed. Informed consent was obtained before each interview. The informed consent form assured the participant of strict anonymity, which means that any details, such as a specific research field, university or nationality, that might make a person recognizable will be obscured in this analysis.

For the analysis of the material, the research team followed a grounded theory approach (Charmaz 2006), with multiple successive rounds of open coding. Questions about how to ascribe value to scientific practices, objects, institutions and people (including themselves) emerged as an interesting research theme during the analysis. Initially, our coding process did not focus on comparison of PhD students and postdocs; it instead sought to map young researchers’ statements about valuation practices. The major difference between PhD students and postdocs that this paper will report emerged as a result of the analysis.

Austria presents an interesting case for studying how researchers discuss the value of their work and lives in academia. In recent years, Austrian academia has seen a strong shift in both academic career structures and the discourses and practices that aim to define and measure the quality of research. Both in policy and within institutions, debates have focused on improving the quality of research by adhering more closely to what are framed as international standards, particularly in awarding research funds and in employing and promoting faculty. At the same time, there has been a shift away from the block funding of researchers and research organizations and toward competitive third-party funding and a focus on quantitative performance indicators, such as publication numbers and impact factors, in hiring practices. In this general effort to catch up with countries that are regarded as “innovation leaders,” specific research fields have received particular attention. The life sciences constitute a key area of Austrian specialization, and they have attracted considerable national financial support in the form of new research institutes and a dedicated government research program.

In addition, career structures have gradually but fundamentally changed since the mid-1990s. Traditional academic career trajectories, in which university faculty below the full professor level could achieve civil-servant status and hence secure a lifetime position by successfully concluding a habilitation, were repealed. For nearly a decade, no clear career trajectory existed within Austrian academic institutions, and all junior staff were employed with temporary contracts only. In 2009, a new collective bargaining agreement established a career trajectory that was similar to a tenure-track model, but which only led to the rank of associate professor. However, in a period of relative financial austerity, the number of these new positions remains low, and a strong discourse emphasizes the importance of a good balance between tenured and non-tenured staff. Formal evaluations of individual staff performance remain a rare exception beyond tenure track assessment procedures. Legally, individual performance should be discussed in formal annual appraisal meetings between staff and direct superior (group leader or head of department). However, if and how these meetings are conducted and if any binding agreements result from them varies widely between and within institutions. Often, they remain informal in character and may even be omitted particularly in project employment contexts.

Although this background is important for our study, our interviewees’ relationships to the Austrian context varied. Their main frame of reference was generally an internationalized academic job market rather than an Austrian one, even though the issue of being able to stay or to return to one’s home country was raised in our interviews. Among both PhD students and postdocs, nearly half of our interview partners had not completed their undergraduate studies in Austria and had come to Austria for specific research appointments, and many of the Austrian-born interviewees planned to take their next career steps abroad. At least in the European life sciences, the early stages of research careers do not typically take place within a national academic labor market and instead occur in the flows and markets constituted by international academic mobility.

Much of our analytical work in this paper has focused on how our respondents implicitly referred to evaluative principles in discussing important aspects of their work and lives as researchers. In this section, we will briefly discuss whether and how both PhD students and postdocs explicitly and critically reflected on how evaluative principles structured their practices.

This paper has offered some analytic insights into the current epistemic and social crisis of the academic life sciences (Alberts et al. 2014) through the perspectives of doctoral and postdoctoral researchers working in Austria. Its central observation is a stark narrowing down of the repertoire of evaluative principles that are available to researchers to conceptualize and prove the worth of their work as they advance in academic socialization.

Our empirical analysis has shown that life science PhD students refer to a range of different evaluative principles when discussing their current and future lives in research. This plurality of evaluative principles leads us to characterize their epistemic living spaces as heterarchical. As a prime example, we have described how this plurality enables PhD students to engage in risky epistemic questions.

By contrast, postdocs’ narratives mainly refer to one regime of valuation in which the worth of individuals is defined by their ability to succeed in competition based on productivity in terms of acquiring internationally accepted and transferable tokens of academic quality, that is, indexed publications, grant money and recorded citations. Other evaluative principles were hardly mentioned, and if they were, they were mostly discussed either in instrumental relation to this dominant regime or as obstacles in conflict with this regime. Postdocs’ epistemic living spaces may thus be described as being structured by a strongly hierarchical form of attributing worth. Virtually all decisions—epistemic, career-related or even private—were scrutinized in relation to their potential effects on the postdocs’ individual productivity and thus their chances of obtaining future faculty positions. Epistemically, this led them to potentially make rather conservative choices, avoiding unorthodox, risky or lengthy projects.

In growing into the life sciences as an academic field, young researchers are hence socialized into an ever-narrower regime of valuing their work and that of others. Those who choose to remain in academia move from a heterarchical epistemic living space into a hierarchical environment.

Both PhD students and postdocs did not see themselves as having significant agency to critically engage with the dominant regime of valuation. While PhD students located their agency in choosing between compliance and exit, most postdocs tended to treat the dominant regime of valuation as a quasi-natural order without alternative. The postdocs who did criticize the negative effects of this dominant regime lacked imagination with regard to potential changes—they saw no alternative regimes to which they could legitimately refer.

Our findings may be read to connect with broader debates about the re-structuring of academic work along the neoliberal agenda of the New Public Management (Shore 2008; Sparkes 2007). The increasing metric measurement of research performance, the temporalization of research work and the emphasis on competition are key elements of the dominant regime of valuation that we have analyzed for the postdoc period. Prior literature has pointed to the fact that the actual impact of changes in the governance of research on specific institutions or fields of research is often not as linear as might be assumed at first glance (Gläser et al. 2010; Paradeise and Thoenig 2013). With this caveat in mind, we would like to propose five hypotheses on the potential wider problematic implications of the narrowing of evaluative repertoires that we have analyzed in this paper—for science as both a social and epistemic endeavor.

First, the valuation dynamics that we have observed may create deeply unsustainable biographies for those opting for a career in research. Researchers’ intense focus on a distinctly inner-academic system of valuing research potentially keeps them from acquiring skills that might be valuable in other employment contexts. Additionally, to succeed in competing in the dominant regime of valuation in the postdoctoral period, they sacrifice other aims that have worth for them in their wider biographic plans, i.e., personal fulfillment not only in their epistemic and professional lives but also in their private lives. Such sacrifices may be expected to have dire consequences for the predictably high percentage of postdocs who will not remain in academia in the long term (European Science Foundation 2015).

Second, for the research system, the narrowing of evaluative repertoires could create problematic dynamics of selection and self-selection. Many of the doctoral students we interviewed planned to opt out of an academic career because they did not want to work and live like their postdoc colleagues. Research, at least in the life sciences, could indeed be becoming less attractive for young people whose biographic imaginations have no fit with the dominant regime of valuation. In addition to existing selection effects, such self-selection processes may further challenge the aim of fostering diversity in the scientific workforce. Particularly the role of gender, class and ethnic background merits deeper exploration than we could provide in this paper.

Third, the highly individualistic dynamics fostered by the current regime of valuation, at least for postdocs, are likely to challenge the social structures of the research system. The cohesion of local research groups and departments is likely to suffer as individuals are structurally encouraged to place their individual interests above the work needed to build and maintain collective intellectual structures.

Fourth, the narrowing down to one regime of valuation also may segregate academic science from other societal values and concerns. As our material shows, care for the solution of societal problems is still an important issue for PhD students, but it gradually loses importance in the environment in which postdocs work. Other than currently demanded by policy discourses and academic commentators alike (Felt et al. 2013; Stilgoe et al. 2013), academic socialization in the life sciences seems to structurally inhibit engaging with societal responsibility rather than fostering it.

Fifth, research driven by productivity concerns rather than other values also runs the risk of hampering its epistemic development. On the one hand, the conservative approach induced by the synonymy of epistemic and career risks (Sigl 2015) is likely to lead to a mainstreaming of research, to researchers choosing safe topics rather than pursuing risky intellectual breakthroughs and to a systematic under-reporting of negative results (Fanelli 2012). On the other hand, the need to stand out in a crowd may lead to deviant behavior, such as the premature publication of results, the exaggeration of the significance of findings, or even the fraudulent fabrication of data.

Further research is needed to explore the salience of these hypotheses. In identifying directions for future studies, reflecting the context of our case and its potential limitations may be useful. Our study focused on the life sciences because they have been at the center of research policy imaginaries in recent decades, not only providing them with ample funding but also exposing them to new governance logics. We argue that this specific constellation makes the life science a good case for observing general trends; however, studies are needed to explore the dynamics in other fields, which will likely be different. An additional important context in our study is Austria, with its strong impetus to internationalize research careers and to distance itself from a more parochial past. Again, comparative work with other nations seems necessary to better grasp the respective importance of local governance and global standardization processes (Paradeise and Thoenig 2013). Institutionally, our sample focused on well performing research groups with clear international perspectives. Studies that analyze institutions and groups with different profiles may reveal different dynamics.

This paper has also aimed to offer new theoretical tools for future studies of academic valuation practices. Our approach joins recent valuation studies in stressing the importance of empirically re-constructing the situated use of specific evaluative principles. However, it also stresses the importance of considering the institutionalization of these principles in wider regimes of valuation and the structuring and even coercive effects that they may exert on individuals’ valuation practices. In offering these tools, this paper aims to contribute to the wider agenda of research on academic valuation practices and their effects on the relationship between science and society.