Introduction

The award of a doctoral degree has been a strong signal that the holder of the qualification can join the academy’s latest generation of scholars. Possession provides the evidence of a successful attempt to make a knowledge claim and acquisition of research skills and capabilities including much tacit knowledge of the culture of the academy. While doctoral theses produced within scientific disciplines traditionally include bodies of papers reporting scientific discovery, in the Social Sciences and Humanities in the UK higher education sector this has not been so until recently. Now, however, doctoral education is transforming, and the thesis is being reconceived with greater emphasis upon it qua output and as a set of knowledge claims.

This paper considers aspects of this transformation. It begins with a discussion of the importance of distinctions within the academy. The paper then moves to review developments in the status of the doctoral thesis which have seen universities changing their rules on allowable thesis format to permit a submission for the doctoral degree to comprise an assemblage of papers, often three or four in number, each intended to be of publishable quality, and taken together forming a coherent body of work. The changes now being observed are then related to two developments both of which are underpinned by quantification, the first being the adoption of citation as the pivotal organizing concept which conceives knowledge as quanta with the value of knowledge considered quantitatively, and the second being the marketization of higher education which emphasizes commodification that re-frames activity as product (de Rosa 2016). The resulting transformation in the thesis is critically approached with a conceptual framing using ideas from realist philosophy (Searle 1995) and new institutionalist theory (Scott 2014), particularly the notions of constitutive rules, and institutional development and change. Such a framing helps to illuminate the tensions that exist as knowledge claims made within the context of one important academic institution, the doctoral thesis, are redefined in the terms of another, if not the central academic institution, the peer-reviewed paper. Our study then reports on an investigation of practice in the use of traditional and new thesis formats in two representative disciplines of the Social Sciences (Sociology) and the Humanities (History) at a random sample of 54 departments at British universities. The paper approaches the topic with an exploratory methodology, but with a limited number of prior expectations: (a) that on certain measures of research performance there might be a positive relationship with adoption;(b) that departmental size might be related to adoption practice; (c) that disciplines might show differences in adoption practice; and (d) that more established organisations, which we defined by membership of the Russell Group of universities, would show differences in practice. The paper provides a range of data and analysis and concludes with a discussion of issues raised by the research, proposing areas for further enquiry.

Literature

The academy—a place of distinctions

Understanding knowledge production as a system [‘metascience’ Ziman (2002)] is a topic of long-standing and continuing interest across both the Sciences and the Humanities. Within debates about how such systems operate there has been much discussion of the way control is exercised over the outputs of the system and in particular the definition of quality. Key contributions in the debates about academic quality, what it is, and how it can be recognized, have been made by some of the most eminent researchers in the whole of social science, see for example the extensive contributions of Merton (1973), Merton and Sztompka (1996) and Bourdieu (1988).

Debate on how the quality and relevance of academic work, and of academics themselves, should be judged has commonly heeded the verdict of philosophers of science who have avoided the construction of explicit and objectivist criteria [see for example, Feyerabend (1993) on the diverse practices and forms of science of defining methods]. Instead, the approaches in metascience have attended to three main questions, see for example Merton (1973): (a) what, in scientific practice counts or can count as, or be measured as, or considered to exhibit scientific quality; (b) how, in practice, does it count (in the sense of how do such distinctions arise?); and (c) who should make such judgements? The second question is the one that has most occupied scholars and it has been understood and dealt with in two main ways: how should it count, i.e., normatively, or what, in practice, happens when the rules of the game are devised and or further applied. Both Bourdieu’s notion of academic capital (Naidoo 2004), and Merton’s reflection on the Matthew Effect (Merton 1968) have reflected this awareness of a connection between the question of who makes the judgement about quality, what criteria are in use to make judgements, and then how such criteria are subsequently applied and with what effect. Thus, while Merton and Bourdieu were concerned with what might give rise to distinctions within academic culture or the academic field (Merton’s emphasis is frequently upon the priority and novelty of the discoveries of the scientist, while Bourdieu identifies performance in the agrégation as one form whereby academic excellence is established), the main emphasis of their work has been upon what and how the reputation of those who possess it allows them to control and to exert influence in the systems they inhabit.

The emphasis in the work of Merton and in the work of Bourdieu therefore has been upon the social processes which underpin such distinctions. Bourdieu’s highly detailed analysis of academic hierarchies and distinctions has emphasised the importance of bodies of rules or the ‘rules of the game’ (Bourdieu 1988) which might change over time, but the approach to definition is largely a static one.

Across this wide range of work, there is great reluctance to state categorically what means should be chosen to designate excellence. Merton identifies many of the subtleties of distinction, and is mindful of the many difficulties of defining excellence; but his discussion covers the issue of what is prior to achievement, not in what achievement actually consists. Thus, in his framework, he refers to potential (‘talent’) while his approach to the question of excellence relies ultimately upon an identity: “For immediate purposes, it is enough to note that two of the meanings implied by the word excellence correspond loosely to the two meanings of recognition that have been briefly examined’ (Merton 1973, p. 423). For him excellence is defined teleologically: there is potential on the one hand—in his terms a ‘quality’—and there is excellence in what is delivered—in his terms ‘performance’.

Within the academy, the position of the doctoral thesis within the hierarchy of academic knowledge has been little considered. Major writers on the subject of knowledge production have given little consideration to the doctoral thesis, either as knowledge product or in terms of a position or status in the academic hierarchy. Merton in his major works does not particularly consider students, nor does Ziman. The more recent analysis by Whitley (2000) of the intellectual and social organisation of the sciences does not once mention students as contributing to or creating scientific knowledge.

Where there has in fact been consideration of the doctoral thesis using citation-based bibliometric analysis, this has tended to treat the thesis as sui generis and not as part of the rest of the scientific literature. Important studies in this area using bibliometric analysis of the thesis have tended to study topic focus, see for an examination of the field of library and information studies Sugimoto et al. (2011) and more recently Sugimoto (2014) on academic heritages.

The valuation of the thesis as outside the normal knowledge producing process is evident also in the behaviour of citation indexers which do not consider the thesis as a type of output, and theses are not currently included in the Web of Knowledge (Clarivate 2019). Databases of theses have, though, developed as a result of initiatives taken by national bodies (Weisser and Walker 1997) and some national systems are now in place to record the output of students, see for example the UK’s British Library system (British Library 2018). The Spanish Government (Spanish Ministry 2018) also maintains its own database although with limited functionality (Hernandez Serrano 2020) and there is widespread recognition of the virtues of such databases for providing access to knowledge, see for example the work of Grigas et al. (2017). Other repositories can be found around the world at university level, see for example in Israel where Ben-Gurion University of the Negev has its own resource inspired by the global ETD movement (Asner and Polani 2008).

A number of reflections on how to turn the thesis into a paper (Singh 2015; Thomas and Skinner 2012) reflect a widespread view of the thesis as of a lower standard, a corollary of which is that such work needs ‘conversion’ (Thomas and Skinner 2012, p. 2) to turn it into a proper piece of academic work, but that the effort is worth making to produce more papers in a regulatory climate that seeks to incentivize the production of texts. Currently, within the UK context, the UK Research Excellence Framework (the REF) ignores the thesis entirely qua knowledge claim (Higher Education Funding Agency for England (HEFCE)/Research England 2018), although other research systems do take notice of theses, for example the Danish one (Wilsdon et al. 2015).

A new model thesis

However, there have been strong arguments mounted over a long period in favour of a thesis structured in such a way that its contents are presented as papers, and, therefore, as a set of very explicit knowledge claims. For an account of the origins of the thesis by published work, see Bao et al. (2018), who note its origins in the so-called ‘cumulative dissertation’ of the German higher education system. Early work to establish the case for a publication based thesis is made by a number of writers with a leading figure being Rudy (1962). Later work in favour of the thesis qua contribution to knowledge (through a published papers format), and which takes a critical stance against the traditional monograph, includes most importantly that by Breimer and Breimer (1995, p. 175) which dismissively terms the traditional format a ‘tome-based system’. Other proponents of the thesis that is formed from papers include Thomas et al. (1986) whose use of the term ‘alternative format’ begins to establish one of a number of new naming conventions, although ‘publication based system’ (Breimer and Mikhailidis 1993, p. 406) is a more accurate description. Writing about the production of doctoral theses in the context of medical research, Breimer and Mikhailidis (1993) describe the difference between the UK system for a Ph.D. in medicine, which does not require a thesis to include papers, and the Swedish system, in which the thesis is required to be made up of papers. The authors however, note discrepancies even within systems: they observe that at a leading UK university, in the Faculty of Medicine (The University of Oxford), theses are indeed allowed to be presented as papers.

Characteristic of the debate about the definition of the thesis is a paradoxical approach to the status of the thesis as knowledge claim. Thus, despite arguing for the superiority of the thesis by published work through claiming it as a better test of the capacity of a student to produce knowledge of significance (Breimer and Breimer 1995, 1996), the authors incongruously propose a database of theses that, in effect, formally segregates the thesis as knowledge product from the other scientific literature, even when it is the case the papers produced within a thesis are published. Again, to further demonstrate the difficulty of making a decision on which format of thesis is the better, rather than seeking to impose a system, the authors, perhaps rather paradoxically, argue for the freedom to choose which format to use, although they indicate that the market will decide, and do so in favour of the thesis by published papers.

The Social Sciences and the Humanities

While the debate over the format of the doctoral degree is long-running, and, as we note above, different systems operate in different countries, in recent years, the arguments in favour of the thesis by published papers have grown. In the UK higher education system, where the use of the thesis by published papers is still limited (Christianson et al. 2015), there is rising interest in the Social Sciences and Humanities in the adoption of that format (Jump 2015). According to de Rosa (2016), interest within the Social Sciences and Humanities in the thesis by published work is the result of two developments: (a) a change in our understanding of scientific knowledge brought about by the citation mapping revolution and; (b) the new public management. We elaborate on these two influences briefly below.

Citation—mapping and then measuring: changing cultures of valuation

In the same decade (the 1970s) that the pioneering sociological investigations into the field of scientific enquiry by Bourdieu and Merton were taking place, including into how hierarchies and distinctions formed and were reproduced, important developments were occurring in the technologies of information science, especially citation indexing (De Bellis 2009). To implement a large-scale indexing of papers, Eugene Garfield established the Institute of Scientific Information (ISI), later the Web of Science/Web of Knowledge. Central to approaches developed within the ISI’s activities was co-citation (Small 1973) which associates publications into aggregations illustrative of scientific fields and research fronts. Citation also provided a means of measuring the recognition/impact of papers, but also the status of the journals which published them. While the idea of the journal impact factor had been developed earlier, it was in this period that it became more widely used. This quantifying approach to scientific information, anticipated in 1955 by Garfield (1955, 1972, 2006) and then made possible through computerization of citation indexing by the ISI, created possibilities for the appreciation and delineation of the quantitative dimensions of scientific achievement which had been shown to exist by Lotka (1926) and which had been more recently noted by Shockley (1957) and by De Solla Price (1963).

It is now clear that the quantitative paradigm for conceiving of the quality of academic work has become an increasingly important approach across the whole of the academy. For more detailed treatment of this subject, see Haley (2013), Egghe (2006) and Hirsch (2005), the latter work now having over eight thousand citations (on Google Scholar). Attempts to refine the quantitative approach have followed, Franceschini et al. (2012) who use an approach which they term the success index, while Waltman et al. (2011) use a field normalized recursive counting approach employing PageRank counting methods.

A context for quantification

The quantifying approach to knowledge production has developed if not flourished in a period that has seen a number of reforms of public sector organisations, including significant changes within the higher education sector across the world. These changes have arisen from two contrasting and yet related attempts to revise and re-frame the relationship between universities and the outside world: (a) attempts to impose notions of economic efficiency upon the sector, including upon its research functions, a set of changes that are often referred to as the new public management (NPM) and resulting from the neoliberal revolution; (b) attempts to engage the universities more broadly with societal concerns, a position outlined most notably by Gibbons et al. (1994) and in a later work Nowotny et al. (2001) and pursued under the notion of grand challenges by research funders such as the European Union (Cagnin et al. 2012; van Oost et al. 2016). The first of these two changes has been the more significant although the second cannot be denied as having importance.

The NPM has brought to bear upon the higher education sector not only a range of mechanisms to promote competition between institutions such as performance indicators of many kinds (Christensen 2012; de Boer et al. 2007; Erno-Kjolhede and Hansson 2011; Guthrie and Neumann 2007; Hicks 2012; Humphrey and Miller 2012; Olssen and Peters 2005; Vinokur 2017; Wollmann 2017) with attendant criticisms of their effects (Chandler et al. 2002; Christensen 2011, 2012; Diefenbach 2009; Lorenz 2012; Moosa 2018; Rebora and Turri 2013; Renaudie 2018; Ryan et al. 2017; Schimank 2005; Schubert 2009; Shore 2008; ter Bogt and Scapens 2012) but has imposed notions of commodification upon the whole area of activity in which universities engage (McCowan 2017; Zacharakis and Holloway 2016). Commodification is not as Komljenovic and Robertson (2016) argue a static process but a continuous and dynamic activity to which the individual institutions of the sector are subject.

Institutional change in the academy

To examine the emergence, development, and adoption of the papers-based thesis, we believe it is productive to draw upon the work of institutionalist and realist scholars as this broad body of work provides concepts helpful in understanding the bases upon which social institutions, such as the doctoral thesis, rest, and how such institutions relate to other institutions and develop over time (and potentially across distance).

Institutionalist theory generally considers that social institutions are underpinned by three ‘pillars’ (regulative, normative, cultural-cognitive), and below the pillars are considered to lie informing assumptions about the creation of social reality that have been termed ‘constitutive rules’. Taking the form of statements, ‘X counts as a Y in context C’ (Scott 2014, p. 76), they are definitional statements stating what is the case, not what should be the case or why something is the case (normative statements). Nor are they an expression of a rule or of the consequences of the breaking of a rule (regulative statements). Recent changes observed to the way in which the thesis qua social institution is defined within some of the disciplines is, we would argue, a clear example of the changing of the ‘constitutive rules’ of an institution. As we suggest, below, such a change is profound and significant.

In the example we are considering of changes to the rules that define a thesis, an immediate question concerns the nature of the effect of changing a second institution when other social institutions [they can be ‘institutional facts’ after earlier realist attempts to define social objects (Searle 1995)] are affected by the re-definition of the constitutive rules of a first institution. Our argument is that if one institution is re-defined in such a way that its relation to another institution alters, then the constitutive rules of the second institution also change, by implication, or, least a tension arises from ambiguity. In the case we consider of institutional change, the process of redefinition of the thesis has begun to occur by reference to another social institution: the peer-reviewed academic paper. In summary form, if institution A [e.g. the thesis] is redefined in such a way that its property X [the content of the thesis] is now deemed to be a Y [published papers], and that institution B [the peer-reviewed paper] has this same property Y [published papers] which was not connected previously to X (and to institution A) in any way, but is now connected by virtue of the redefinition of A, a change to B has been implied through an attempt to change the properties of A.

A further concern relevant to our case with which institutionalist scholarship has been engaged is how such changes are brought about? Theorizing on the subject of change in the institutionalist literature identifies two contrasting processes to explain how changes happen—both of which may however be present in any given case. Where changes to institutions—in this case the definition of the thesis—proceed on an uncertain basis with a general open-ended approach, and where, as Scott indicates, outcomes ‘emerge from the collective sense-making and problem solving behaviour of actors confronting similar situations’ (Scott 2014, p. 114), a naturalistic account is plausible. Where institutional change is led by specific agents, the term agent-based change is employed.

Our reading of the case of the redefinition of the thesis in light of the institutionalist scholarship therefore raises questions about relations between different institutions when their respective constitutive rules overlap or come into conflict i.e. have common reference points, and also questions about how such changes occur. Our view is that such topics invite a more detailed investigation. At this stage, it is appropriate to conduct, as we have done, a short exploratory investigation of actual practice of thesis formats, to document change to the rules regarding format, and to explore the extent of new practices, and relate changes to known features of the organisations and institutions of academe, such as their size, and performance.

Exploring changes in practice: an investigation

The foregoing review of context and literature suggested an agenda for empirical work as follows: (a) a general documenting of practice and intention as regards the use of the thesis as a means to publish work; and (b) identifying differences between disciplines, which a previous study on the general topic had not examined in detail (Christianson et al. 2015), and exploring how differences in practice and intention might have arisen. To provide empirical evidence about changes in practice in terms of thesis submission format adoption, a short exploratory study was conceived to obtain information by telephone and or email questionnaire on the use of theses by publication in universities in the United Kingdom in two disciplines in the Social Sciences and Humanities, namely Social Science and History respectively and to incorporate in the analysis of such data features of departmental organisation and performance that could conceivably shed light on such research practice and changes that had occurred to it. While initial review of university thesis documents provided to us at the start of the study suggested that an investigation of a taxonomic kind (or even the development of typologies) might be possible, our focus was upon obtaining an overview of developments.

Evidence from the REF2014 database was considered potentially relevant to explaining departmental practice and investigating differences. This database, created as part of the United Kingdom’s periodic review of university research known as the Research Excellence Framework or REF (—prior to 2014 the review was known as the Research Assessment Exercise or RAE), contains information at unit of assessment level (quasi departmental information) about the size of departments and various performance measures including the quality of academic outputs [‘the output measures’], the research culture [‘the research environment’] and the impact of the research upon society and the economy [‘impact’]. From this data it is also possible to generate a composite measure reflecting the amount of resource (financial resource won from the funding council funds) available to the unit of assessment on the basis of the entire REF submission (at REF2014).

Our expectation was that departmental size, i.e. larger size, might influence adoption of new practices, following the view that the more communication channels through which information about an innovation could pass, the higher the diffusion rate (Rogers 2003), although we note the relationship between size and performance is problematic and that above a certain size threshold, performance might decline (Kenna and Berche 2011). A further but weak expectation was that departments with a higher proportion of staff producing work regarded as of higher overall quality would be different from those at the other extreme—and would, furthermore, have adopted the new style—i.e. thesis by published papers, on the assumption that if flexibility on thesis format is advantageous to an organisation, it will occur more frequently in those which, on some yardstick of performance measurement, have fared better.

As disciplinary cultures vary significantly, in regard to citation amongst other aspects (Ziman 2002, p. 260), it was our view that it could be reasonably expected that differences between Sociology and History would exist in thesis format: in particular, that more theses by publication might exist in departments where the use of ‘theory’ is more common, i.e. in Sociology, than in departments where narrative exposition and extended analysis is more common, i.e. History. For that reason, our approach was to consider the two disciplines separately.

Our sample included a number of newer universities and very well-established ones. We sought to investigate whether Russell or non-Russell Group universities were different in their use of the thesis by published papers as Russell Group universities have advantages of size, stronger research tradition and, through being often older, more experienced staff (Karran and Mallinson 2018).

Methodology

Selection of cases and sources of data

We wrote to or phoned departments or quasi departments in 21 educational institutions where there had been return in the unit of assessment ‘Sociology’ in the last Research Excellence Framework (REF) exercise of 2014. We also wrote to or phoned 40 education institutions where there had been a return in the unit of assessment ‘History’. We selected institutions randomly. From 29 institutions submitting in the unit of assessment Sociology we sought replies from 21, while from the 83 institutions submitting in the unit of assessment history, we sought replies from 40 educational institutions.

While the REF data applies to units of assessment, our study sought responses from academic departments/divisions or schools, as disciplinary practice was the focus. We recognize that our study relates data from responses given to a questionnaire about practice in 2019 to data from 2014 (published in 2014, the REF data). The relevance of what could be regarded as contemporary data about practice to historical data of research performance could be questioned, and we accept that previous performance might no longer be strongly associated with and relate to current departmental practice. The universities in our sample were from varying ages, with 15 founded prior to 1900, five founded in the period 1900–1950, ten in the period 1951–1990, and 23 in the period 1991–2020.

Ethics review statement

An ethical review manager (ERM) software tool provided by our university was used to assess the study’s requirement for formal review by one of its ethics committees. The tool indicated that the study would not require formal ethical review as personal data was not being collected and the guarantee of anonymity to respondents and to their institutions was to be provided verbally and in writing.

Research focus and questionnaire

We noted in our enquiry, which was carried out by email and by telephone survey, that in some institutions, the term journal format thesis was used instead of thesis by published papers format. But we also drew attention to the fact that our enquiry sought information only about the thesis format that was an alternative to the traditional doctoral monograph thesis and not the form of thesis which is intended for later career staff who might wish to earn a doctoral degree from work published over a long (sometimes a career-long) period (i.e. already published) which is in many institutions termed a ‘thesis by published work’. Our reasons for excluding this second form of thesis involving publications was that (a) in cases where academic staff are submitting papers for the degree of doctor of philosophy they are doing so with papers that are often already published, (b) this is a less common practice, and (c) importantly, this is not the major form of doctoral degree production. Our study aims therefore to examine the issue of whether and how the alternative to the traditional monograph thesis is emerging as a rival form in which the thesis is comprised of papers of publishable quality which are intended to constitute knowledge claims. We also sought to identify what other factors might be associated with such changes in practice. In each enquiry we made to departments, we aimed to contact administrators or academic staff with knowledge of the history of submission format practice who could answer the following questions in the table below Table 1 questionnaire on thesis type. In both email correspondence which accompanied the questions and when speaking on the phone to respondents (if a reply had been delayed), we were careful to state that we were not focusing our research on theses produced by staff based on published papers.

Table 1 Questionnaire on thesis type
Full size table

Findings

Responses

For the discipline of Sociology, all 21 departments we wrote to replied. 33 replies were received from 40 History departments. Our coverage of the sector was therefore, in Sociology, 21 from 29 submitting in the last REF (72%) and in History, 33 responses from 83 submitting in the last REF (40%).

Data was analysed in order to establish the following and the results are displayed in the following Tables 2, 3, 4, 5, 6, 7, 8 and 9.

Table 2 Thesis format adoption behaviour
Full size table
Table 3 Year (4 year periods) of introduction of TBPP and count of departments adopting
Full size table
Table 4 Thesis adoption behaviour
Full size table
Table 5 Thesis submission format and intra-university agreement
Full size table
Table 6 The effect of size upon thesis format use—Sociology
Full size table
Table 7 The effect of size upon thesis format use—History
Full size table
Table 8 Performance based measures comparison of departments—Sociology
Full size table
Table 9 Performance based measures comparison of departments—History
Full size table
  1. (a)

    In how many institutions has thesis submission been allowed in both monograph and journal format, and the current level of usage of the different formats, and current intentions.

  2. (b)

    Evidence of changes recently in the adoption of thesis by published papers.

  3. (c)

    The existence of statistically significant differences between disciplines in respect of the acceptance of monograph and thesis by published papers format.

  4. (d)

    The existence of different levels of submission in the two formats differ, and the presence of differences between disciplines?

  5. (e)

    If published papers are allowed in a particular discipline at a certain institution, when did changes occur and are there differences between subject disciplines?

  6. (f)

    For those institutions yet to adopt the practice of thesis submission in published papers format, what were current intentions?

  7. (g)

    If intentions were to introduce a thesis in published papers format, when would this be allowed?

  8. (h)

    Were there differences between subject fields?

Transformation of practice—current rules, practice, intention

Amongst Sociology departments, eight out of 21 now allow the submission of both a monograph thesis and thesis by published papers with five departments allowing the practice since 2001. Amongst History departments, 15 out of 33 departments now allow both formats with eight departments starting to allow the practice since 2001. In our sample of departments, there is no statistically significant difference in practice between the uses of the two formats between disciplines. The Fisher exact test statistic value is 0.7784 for the comparison of allowed practice by discipline. In terms of usage, the monograph is more common in both disciplines. Only in one department is the thesis by published papers more common than the Monograph. This occurs in Sociology.

Recent introductions and plans to introduce

As to intentions to allow both the monograph and the thesis by published papers, no Sociology departments in our study indicated that they were planning to introduce rules to permit students to submit theses in both formats currently, although one department replied that intentions were not known. Four History departments indicated that they were planning to permit both formats with two further departments of History indicating that their future plans were not known at the time of our request for information from them. Of those History departments intending to introduce both formats, two were intending to introduce them in 2019 and two in 2020.

Below we note the introduction of thesis by published papers (TBPP) and the number of departments adopting. In Sociology it is possible to see that in total, five departments have either adopted the TBPP or have plans to introduce TBPP since 2001, which is 17% of all departments, while in History 12 twelve departments have adopted since 2001 or plan soon to adopt, which is 14%.

Exploring differences in practice: Russell Group and non-Russell Group membership

Our samples of Sociology and of History departments included Russell and non-Russell Group universities. The proportions of each group were similar in each discipline. In Sociology, there were 8 Russell Group universities in the sample of 21 (38%), and in History, there were 13 Russell Group universities (33%) from a sample of 33. We used Fisher’s exact test to establish if there were differences in practice at the level of the whole sample of both disciplines [N = 54] between Russell Group departments and non-Russell Group departments, and also within subject disciplines (Sociology, N = 21; History, N = 33). No significant differences were found in any of the 3 tests.

Exploring the influence of the university upon departmental practice

When random samples of departments were chosen from both disciplines, a small number [N = 8] were located in the same education institution. It was therefore possible to examine the effect of institutional location on those universities that had departments in both. One of the departments in the group of eight did not however reply to the question and was removed. In the event, seven departments were examined and in three cases, there was departmental agreement, while in four cases, departments in the same institution operated different practices in regard to thesis submission format. A binomial test can be made and a null hypothesis of no influence can be tested. In the event of there being seven cases with four non-matches, the probability of four or more non-matches has a probability of 0.5, and a 95% confidence interval of 0.18–0.90, showing that the null hypothesis of no-influence cannot be rejected.

Exploring differences in practice: effect of size of department

Our analysis of the adoption of thesis by published papers as a distinct practice examined the role of the size of departments upon adoption. New practices may be more likely to arise where departments have more contact with the outside world. A crude indicator of a department’s exposure to outside influences would be its size in terms of the number of staff. While the number of staff submitted in the REF2014, which is the information we have used, is not necessarily a reliable guide to the number of actual members of staff in a department, for the purposes of this enquiry it might be a relevant indicator as those counted are the department’s active researchers with, potentially the stronger external contacts. A ranks test was used to assess the association between the adoption of the TBPP and departmental size. The analysis showed that in the case of Sociology, there was a statistically significant difference with larger departments more likely to adopt the TBPP. History departments showed no effect of size upon adoption of the practice.

Exploring differences in practice: the link with quality measured by the REF2014

We used information from the Research Excellence Framework 2014 [‘REF2014’] to explore the relationship between the performance of the department in the last major formal evaluation of university research and thesis submission practices. Our view was that judgements formed by the REF panels relate to departmental performance over a long period and, to that extent, are valid as assessments of departmental capability and achievement in a number of areas. Furthermore, as measures of organisational achievement in the area of research which are widely reported and of considerable importance to a university’s reputation, they might be expected to have an association with departmental practices, including the practice of interest to us, namely thesis submission format.

To assess the relationship between departmental capability and achievement and the practice of thesis adoption, we used REF data on the financial resources ‘earned’ by the department (UOA) on three measures assessed by the REF. These measures are all made at unit of assessment level and we have combined them to create a composite indicator. We made the assumption that the unit of assessment was an acceptable proxy for a department. The three measures assessed by REF are impact, environment, and output. We then combined three measures to form a compositive indicator to reflect the financial reward earned by a department. Our indicator follows the United Kingdom Research and Innovation (UKRI) methodology in measuring the reward in terms of full-time equivalent employees (FTE) and reflects the Quality of Research (QR) money earned by units of assessment from their performance in the REF.

Publicly available data was obtained from the last REF (REF2014) on the three main measures of unit of assessment performance to create our indicator. We term that indicator, ‘UOA_Total_Financial_Resource_REF2014’. The weightings at the last REF were as follows for each of the measures: 0.2 for the Impact score, 0.15 for the Environment score, and 0.65 for the Output score (Higher Education Funding Agency for England (HEFCE)/Research England 2011). To create the assessment of financial reward at university department level, these weightings are multiplied by the proportion of staff achieving category four status in the assessment and the proportion achieving category three status. Research impact, environment and output scores below three carry no financial reward and do not need to be included. Category four research is defined as ‘Quality that is world-leading in terms of originality, significance and rigour’. Category three research is considered to be “internationally excellent in terms of originality, significance and rigour’ (UK Research and Innovation 2019).

Our steps in creating the individual and overall measures were to calculate each of the three individual measures in terms of impact, environment and output, and then to weight each according to the REF model to create a Grand Total. The financial return to the department was the Grand Total with the London weightings applied (inner London 12%, outer London 8%). Our measures therefore give the notional financial reward (of impact, environment and output) which and that this financial reward may to some extent proxy for departmental capability and achievement.

$$\begin{aligned} & {\text{Imp}} = \left( {\left( {{\text{FTE}} \times {\text{Imp}}4/100 \times 4} \right)} \right) + \left( {\left( {{\text{FTE}} \times {\text{Imp}}3/100} \right) \times 1} \right) \\ & {\text{Env}} = \left( {\left( {{\text{FTE}} \times {\text{Env}}4/100 \times 4} \right)} \right) + \left( {\left( {{\text{FTE}} \times {\text{Env}}3/100} \right) \times 1} \right) \\ & {\text{Out}} = \left( {\left( {{\text{FTE}} \times {\text{Out}}4/100 \times 4} \right)} \right) + \left( {\left( {{\text{FTE}} \times {\text{Out}}3/100} \right) \times 1} \right) \\ & {\text{Grand}}\;{\text{Total}} = \left( {{\text{Imp}} \times 0.2} \right) + \left( {{\text{Env}} \times 0.15} \right) + \left( {{\text{Out}} \times 0.65} \right) \\ & {\text{UOA}}\_{\text{Total}}\_{\text{Financial}}\_{\text{Resource}}\_{\text{REF}}2014\left( {{\text{Per}}\;{\text{department}}/{\text{Unit}}\;{\text{of}}\;{\text{Assessment}}} \right) \\ & \quad = {\text{Grand}}\;{\text{Total}} \times {\text{Weighting}}\;{\text{Factor}}\;\left( {{\text{Inner}}\;{\text{London}}\;{\text{Weightings}}} \right) \\ \end{aligned}$$

Departments were then assessed on these three measures of performance and our UOA Total Financial Resource indicator and those operating both forms of thesis were compared with those operating only the monograph. A ranks test was used on each discipline. The results of the tests are shown below in Table 8 performance based measures comparison of departments—Sociology and Table 9 performance based measures comparison of departments—History.

We should note that while individual measures were analysed separately and are different data sets, analysis of the combined data set of total financial resources and the individual component data sets when taken together should be considered to be subject to the risk of an increased familywise error rate. There is no general agreement on a single approach to control the rate (Perneger 1998) and there are many suggestions on how to avoid the risks. Our view is to follow the common practice informed by the Bonferroni principle and divide alpha by the number of actual tests involved. In this case, that denominator is 2 as there are in effect two tests, one test with the whole data and a further 3 tests all of which contribute 1/3 to the first test, i.e. 1 + (3 * 1/3) = 2.

In the field of Sociology, three of these QR based measures of performance were statistically significantly related to thesis practice before correction, while after our correction we considered two were. The measures related to the overall research quality of the department, and the measure of the department’s quality of research were all significantly higher amongst the departments that had adopted the thesis than those who had not. The measure focusing on the impact of research was not significant. The measure focusing on the environment was significant when considered separately, and we consider is likely to be weakly associated.

In the field of History, no measure of performance was significantly related to the adoption of the thesis, and there was therefore no need to carry out correction of alpha. The measure of performance assessing the impact of research was, however, close to significant. Our interpretation of the results is that in Sociology, there is an association between the performance of the department, in certain aspects, and thesis practice. In History, there is no association.

Discussion

Change in practice and regulation

Our study provides evidence from a systematic investigation of changes in the practices regarding thesis submission in both Sociology and History departments in the UK Higher Education sector. It uses a number of non-parametric statistics as the incidence of cases/data points in our sample is low. As yet however, we can only note changes in the rules that apply to format of thesis submission, rather than to the actual incidence of usage of different formats since there is no publicly available information, and responses volunteered from interviewees suggested that few if any organisational records were kept.

In both disciplines, more departments still permit thesis submission in the monograph format exclusively: amongst Sociology departments, 13 departments require the monograph as compared with eight departments where both formats are allowed; while in History, 18 departments require the monograph exclusively, compared with 15 departments that allow both formats. Despite changes to the rules affecting practice, it is only in one department that the thesis by published papers is more common than the monograph. Change however is occurring to the rules affecting thesis submission, but not rapidly.

Factors affecting change

Our view was that some factors associated with the adoption of innovations might be investigated to identify a link with the requirements to use different thesis types. We considered departmental size, the organisation to which the department belonged, and a small number of measures of performance of the department (obtained from the REF2014) as potentially relating to thesis adoption.

Departmental size

Departmental size, for which a proxy is available as an FTE measure in the REF2014, is a simple but readily available measure that is plausibly related to the scope for the adoption of new practices. And in Sociology, size of department is indeed statistically significantly related to adoption of the thesis by published papers, whereas in History, size has no effect. In departments of Sociology, the mean rank of departments that adopt both forms of thesis—i.e. that have adopted the thesis by published papers is nearly twice that of departments that have not adopted. The difference is statistically significant in the case of Sociology.

Organisational effect

We were able to consider if the university in which the department was located had any effect upon the adoption of the thesis by published papers. In our sample of universities there was a small number [N = 7] where departments of both Sociology and History had returned in the last REF and it was therefore possible for us to examine if the organisation itself in which the department (the Unit of Assessment) was located had an influence upon the types of thesis allowed. As we have noted above, four universities had differences in their thesis submission practice between their departments of Sociology and History, while three universities had the same practice in both disciplines. With such a small number of cases, it is not possible to be confident that university-wide policy has had any effect. However, we do note that there was more difference than there was similarity in thesis formats allowed. We tentatively suggest therefore that the organisations (i.e. universities) in which departments are located are not having a strong effect upon thesis format policy at departmental level.

Departmental performance

Our investigation also examined departmental performance along a number of dimensions and its association with thesis format types allowed. Our evidence suggested that in the discipline of Sociology, departments that are more successful, measured by our metric of QR money earned overall, and in terms of output measures, and possibly environment measures, were more likely to allow both types of thesis. In the discipline of History, there was no association between success measured using our metric and thesis adoption practice.

We had assumed that in departments that were more likely to achieve higher performance levels when judged by outside observers, there might be more alignment with notions of disciplinary good practice, including those relating to the adoption of new forms of thesis. This might be most evident when comparing departments on the Environment measure of performance. Our grounds for considering this is that funding council guidance on the way in which panels assess environment refers directly to the supervision of research students in Section C4: Assessment criteria, Paragraph 107, Section 3 subsection (ii) (Higher Education Funding Agency for England (HEFCE)/Research England 2012).

As we note above, in departments of Sociology such a link is very weakly evident but in departments of History, the link is not present. We might consider on the basis of this evidence that good practice in Sociology departments may be to use both forms of thesis, whereas in History, there is no consensus. This contention should though be considered in light of the fact that in Sociology, the thesis by published papers is less established (in our sample) at 38% of the cases, compared with History, where 15 departments (45% of our sample) have allowed both forms of thesis.

Conclusions

We note on the basis of our study of the UK that important changes to the institution of the doctoral thesis are occurring. But the changes we have observed may not be complete and, missing from our picture of practice is the number of theses which are produced in different formats in departments. We should also note that our study reached its conclusions without our examining university policy documents, and we relied upon respondent’s knowledge of each organisation’s policy and practice. The change we have evidence of appears to be piecemeal across disciplines. In our view, this is suggestive of more a more naturalistic form of institutional change than one, and here our limited evidence on the role of universities qua organisations is also suggestive, that is actor-led.

Our theoretical discussion noted that changes to the definition of the thesis concerned two central institutions within the academic field, the doctoral thesis and the academic paper that have historically served different purposes, but which today, are being brought, for various reasons, into a closer alignment. In the past, in the UK system, while the thesis has indeed been a knowledge claim clearly vital to the progression of the student forwards from the position of scholar to academic staff member, its knowledge claims have not necessarily been considered of sufficient strength or in an appropriate form to be considered suitable for incorporation into the formal bodies of codified knowledge. A cordon sanitaire had remained around the thesis, with the thesis remaining proof of a capacity to generate knowledge, and a claim of credentialization of the individual but not certification of the knowledge claim or claims made inside the thesis. Whether the two institutions can have a closer identity as some current practices envisage is very open to debate, and should be the object of further research.

A number of other questions are raised by the alignment of these two institutions: how should knowledge claims made within papers that comprise a thesis be linked to and indexed in the formal bodies of codified knowledge? Will the link be made from the thesis to its cited papers or from the papers themselves that comprise the thesis to those papers that are cited? If both routes are chosen, duplication will surely result. In the examination of the thesis, who now holds sway in passing the judgement of quality of a thesis by published papers, an examiner or the various peer-reviewers of the papers submitted within the thesis? What will be the quality of papers that are produced within a thesis by published papers? Will such papers be widely accepted within formal evaluative processes such as the research excellence framework? Will changes to the rules affecting the thesis lead to an increase in the production of papers? We respectfully submit that the topic expects more research and that taxonomic and typological approaches may be helpful in developing understanding.