There had been in our institute for over fifteen years already a strong focus on research. In line with what at these days was pretty normal, the more fundamental pre-clinical science was regarded the best, as was measured by the JIF of the venue of publication. Publication output, citations, JIF and top 10% of journals of the field and in addition high profile personal grants and the amount of grant money that was brought in were used to score the research performance of divisions. This type of metrics was every three-year period used to determine the number of professors each of the divisions was entitled to have. As the total number of professorships was capped, this was a zero-sum game where every three years some divisions lost, some gained. Fortunately, only a limited fraction of intramural money was allocated to the divisions based on these indicators, in addition to monetary rewards for the number of awarded PhD’s. As we have seen in Chap. 3, this was since the years 2000 common practice in the Dutch and the wider European and international research landscape. UMC Utrecht in that respect was not at all atypical. In this strategy we did very well in publications, numbers of PhD’s and grant money that was brought in. With our new strategy a year or two underway, however, we after some time had to admit that this incentive and reward system was not aligned with the different forms of science and academic output in the six multidisciplinary programs. In fact, we also realized that our research evaluation system did not acknowledge top professionals and clinicians engaged in more practical patient-driven research where journal impact factors are lower and no prestigious personal career grants are to be won. The more your research was to the left of the innovation loop, the better your chances were for high JIF publications and thus academic promotion. Of course, there were exceptions. When regarding clinicians who performed extremely well and were scarce because they did and thought surgery at stellar levels, after fierce debates in academic appointment advisory committees, their publication lists and grants won were regarded less important compared to their professional academic performance and impact.
The Higher Purpose
In 2014, the need to change the system of research evaluation forced itself upon us in UMC Utrecht. This was a couple of years after we had made the change in the organization of our research environment. For us this was quite logical, conceptually and in time but, to be honest, it had not been planned in 2010. This struck me again in January 2017, at a Washington DC bookstore-restaurant having a breakfast meeting with Paul Wouters and Dan Sarewitz. The three of us attended a special one-day meeting on incentive and rewards organized by Metrics Stanford. Sarewitz made it quite clear, he was not much into the problem of metrics, but had been thinking for decades about the organization of science and how to effectively change it. Sarewitz is well-known from his critical well-informed pieces about the science system in Nature and his book, Frontiers of Illusion, his book chapters and his opinionated excellent long read ‘Saving Science’ (Sarewitz, 1996, 2016). His work has focused on the politics of science and how all kinds of forces and powers keep science from living up to the promise to optimally contribute to society and the good life. He is highly critical of those who are pursuing intellectual interests of ‘blues skies’ research with reference to the linear model of innovation and value-free inquiry. It is an endless frontier, but in his analysis with a lot of illusion indeed.
Sarewitz, asked me why and how we had been able to agree on and then implement a new system of research evaluation at UMC Utrecht. I told him the story of our intervention in UMCU 3.0 and that it thus was a logical consequence of our strategy. It was the diversity of goals and academic roles defined in the six strategic programs, that after a couple of years forced us to implement a research evaluation system that matched with these goals and with the ‘higher purpose’ of UMC Utrecht. We had assigned this task to a group of midcareer young researchers and clinical professionals chaired by Marieke Schuurmans, professor of Nursing Science and secretarially supported by Rinze Benedictus who was by then already quite an expert on Incentives and Rewards. We invited them in August 2015 to start on the question ‘How do you want to be judged and evaluated?’ After six months they presented a more inclusive and less metrics-driven evaluation protocol. The result was a very open and generic scheme which allowed to honour the pluriform excellences related to the diversity of academic roles in the system. Not only papers published, or funding obtained had to be considered, but also results being used and applied closer to users by peers or by users and stakeholders themselves. Think of application in the clinic, in medical products and technical appliances via private partners, in a treatment advise by the Health Council, in the organisation of health care in the region, or in policy making of any kind. In addition, a lot of emphasis was on the ex-ante, or ‘how’ the research was organised in order to enhance on beforehand its potential impact. For instance, we asked, if there was early engagement of stakeholders. The scheme and its implementation were not uncontested. Some warned that ‘it would come at a cost to the quality of our research, was to hurt basic science and the reputation of our institute. ‘It very much depends on how, and who defines ‘research quality’, was my response. Of course, although we all believed we were moving in the right direction, I very well understood the issue. The risk of a first-mover disadvantage posed a serious and realistic worry in those days when DORA was barely known and there was a global addiction amongst academics and university administrators to JIF, h-indexes and the Shanghai Ranking. Even in 2020, when a lot has happened regarding Incentives and Rewards, nationally and internationally, understandably this is the worry still most frequently aired by young research professionals.
The worry about basic science, as we have seen in the previous chapters is of all times. Here I refer to Stoke’s ‘Pasteur’s Quadrant where the concept of ‘user-inspired basic science’ is explained as the kind of research most researchers in basic science do (Stokes, 1997).
User-inspired basic science takes on problems in the context of a larger problem in a given practice and investigates ‘blind spots’ and missing links in knowledge and understanding in that particular field. As we have seen, basic science has a higher standing than applied science, even with the public, and this still feels like a problem for the investigator. In a typical early evening show that until recently ran on Dutch TV and was famous for a host with boundless admiration of scientists, we often see the invited scientist first explain how terribly fundamental the work is, to demonstrate its scientific quality, in order to then proudly explain how it can be used to solve a clinical or social need. Even our recent Nobelist, the synthetic organic chemist Ben Feringa, who started his career at Shell Research, did not escape this knee-jerk reflex when in 2016 he explained in the Dutch evening news his price-winning work as totally ‘blue skies’, but a moment later proudly explained that his molecular motors once may be used to direct medicines to the right spot in the body of patients among other applications in practice.
How to Make the Right Choices
One day, at my job in the department of immunology of UMC Utrecht, I got a phone call from my sister that my brother, who was more than ten years my senior, had suffered a very serious stroke and was in hospital. I went to see him at the hospital, near where he lived. He was in very bad shape. It was a devasting sight. He was paralysed on his left side, but the most terrible thing was that he could not speak and probably had serious cognitive problems. He was moved to a well-known rehabilitation centre in the heart of Amsterdam. During visits we sat in a common room, with a view of the Vondelpark. His ability to move the left leg and arm returned pretty quickly. His speech did not return and communicating with him during visits and ever thereafter was very difficult, which frustrated him enormously. Looking around at the facility, its ambiance, shocked also by the sight of also relatively young patients and their visitors, I was reading the information leaflets about the rehabilitation therapy my brother was receiving. I could not help myself to think of the enormous investments made over the years in research on the pathogenesis of stroke, involving numerous PhD positions, sophisticated animal models, laboratory equipment and large expensive devices and the most innovating molecular and imaging technologies. As the fast majority of patients survive stroke but badly need medical rehabilitation for recovery of speech and mobility and cognitive recovery, the low academic priority and very modest investments in innovation in research and development of rehabilitation and mobility research, I realised, were a disgrace.
After a few years I became the dean and I was confronted with this problem in my own UMC Utrecht and later realised it had been noted at that time by the national Health Council. Because of the reward system, its metrics and definitions of excellence, rehabilitation sciences were suffering. Typical career advice to young MD’s therefore was: ‘go for a PhD on a topic of ‘hard science’ such as molecular pathogenesis. It has more esteem, gives better papers and a better CV than to work on applied problems of mobility and rehabilitation’. Be sure, such problems caused by ‘the system’ is nobody’s fault. People, even highly educated people ‘read the system’, behave according to the system and adept strategically to seek possibilities of advantage for themselves and their set up. I could fill many pages with similar problems of agenda setting being distorted by the incentive and reward system. Molecular cancer biology versus research on living with adverse effects of chemo, total immune ablation, radiotherapy and a bone marrow transplantation. The tumour is hopefully gone, but the patient is still there struggling with her poor quality of life.