Why Does the Government get Defeated in the House of Lords?: The Lords, the Party System and British Politics

Abstract

Although the threat of defeat in the House of Commons attracts significant attention, we hear a lot less about defeats in the House of Lords. Yet in this chamber of parliament, government is defeated regularly — over 400 times since 1997. We analyse what contributes to these defeats, using voting data from six full parliamentary sessions since the (majority of) hereditary peers were removed from the chamber in 1999, supplemented by data from a survey of peers. There are various groups — Labour rebels, independent ‘Crossbenchers’, Bishops, Conservatives and Liberal Democrats — who may contribute to defeat. But the chamber also has a reputation for being independent and non-partisan. We find that this does not translate into voting patterns, and that cross-voting, and the votes of independents, tend not to be influential. In most cases, the pivotal groups in the House of Lords are the opposition parties, who vote cohesively, albeit with high levels of absenteeism. The 1999 reform created a chamber in which no party had overall control, and has thus increased the number of veto players in the British political system. In particular, the Liberal Democrats have gained a new importance in British politics that has not yet been widely appreciated.

Introduction

Voting patterns in the British House of Commons have been much studied over the years (e.g., see Finer et al., 1961; Berrington, 1973; Norton, 1975, 1980; Cowley, 2002, 2005a). The House of Lords, in contrast, is a far less well-studied institution (exceptions include Morgan, 1975; Shell, 1992; Shell and Beamish, 1993; Norton, 2003). Indeed in British politics, the term ‘parliament’ is often used synonymously with the House of Commons — the Lords being considered an unreformed curiosity, which may not be with us for long, and which is not where the important decisions are taken. Government defeats in the House of Commons are rare, and never go unnoticed. House of Lords defeats are also only occasionally brought to public attention. Yet the government is defeated frequently in the House of Lords, and the number and severity of these defeats appears to be on an upward trajectory.

Labour governments have always been defeated more often than Conservative governments in the House of Lords, due to the chamber's peculiar composition. In keeping with this tradition, the Blair government suffered a total of 432 defeats in the Lords in the 10 years from 1 May 1997. However, reform removed the majority of hereditary peers from the chamber at the end of the parliamentary session 1998–1999, and thus ended the Conservative Party's predominance (Shell, 2000). What was left was a chamber where no party could command an overall majority, and most members were life peers.¹ The removal of the bulk of hereditaries has given the Lords a greater confidence, and peers' belief that they are now more legitimate is shared by the public and MPs (Russell and Sciara, 2006b). Government defeats in the Lords, therefore, now take place in a very different context. Since the reform in 1999, some significant proposals have been blocked, and the argument between the two chambers over the Prevention of Terrorism Bill in 2005 was the largest since the early 20th century.

This paper sets out to answer the simple question of why the government gets defeated in the House of Lords, in terms of which actors in the chamber contribute to these defeats. The chamber has a reputation for being independent and non-partisan, and in party terms remains most associated with the Conservatives. Through detailed study of voting patterns over a 6-year period since reform, we provide a picture of the behaviour of different groups in the chamber as it is now, and their impact on policy proposals. This leads us to draw some significant and surprising conclusions about the impact that reform has had on the dynamics of British politics.

We have constructed a database of all members and all votes in the House of Lords since the start of the 1999–2000 session — the point at which the hereditary peers departed.² This paper is based on the data from the first six parliamentary sessions since then — stretching from November 1999 to the dissolution shortly before the general election of May 2005. Data on voting (derived from Lords Hansard) is supplemented by the views of peers as expressed in a postal questionnaire survey circulated in February 2005.³ We also in places compare with data from the Lords in the 1980s.

Our results show that despite its reputation as a non-partisan chamber, and the presence of a large group of independent ‘Crossbenchers’, neither cross-voting nor independents have much impact in bringing about Lords defeats. The main players are the opposition political parties, who vote relatively cohesively and whose combined strength is normally sufficient to overwhelm other forces. In particular, the third party, the Liberal Democrats, is a pivotal group in a way which it was not prior to reform. Reform has therefore increased the number of veto players in British politics (Tsebelis, 2002), and changed the dynamics of the party system at Westminster — although other groups in the chamber also remain important. We review the implications of these findings in the final section of the paper, after examining the empirical evidence with respect to each of the main groups in the Lords in turn. Before turning to these matters, however, it is worth pausing briefly to ask what we mean by a government defeat.

When is a Defeat a Defeat?

At one level, it is straightforward to describe how many defeats there are in the House of Lords. The Lords website itself openly cites the total defeats in each session dating back to 1975.⁴ This comprises all those occasions on which the government whipped a division, but the number of peers backing its position in the lobbies was fewer than the number opposing it. The number of such defeats in each of the sessions we are dealing with, and the number of other divisions, is given in Table 1. By this definition, the government was defeated 283 times in the chamber during the first six sessions after it was reformed.

Table 1 Defeats and divisions by session, 1999–2005

This is the basic definition that we will work with in the paper. However, three qualifications should be put on this as a description of when the government is defeated in the Lords.

The first is that the definition seeks to capture clear conflicts between the government and the House of Lords, rather than between the two chambers of parliament. On occasions where the government does not whip a division (i.e., allows a ‘free vote’), clashes between the two chambers cannot be considered as government defeats. However, some free votes are more ‘free’ than others, and there are various borderline ‘conscience’ issues which may attract the label on some, but not all, occasions (see Cowley, 1998, 180–181; 2002, 234–235). We have excluded from our analysis votes on procedural matters, which are by convention unwhipped, and others where it was clear that the government side had officially allowed a free vote (e.g., through the pattern of ‘tellers’ appointed, through public government statements, or — in the case of doubt — through enquiries to the Labour whips' office). As shown in Table 1, this applied to 67 of 873 divisions during the period. Although it seems obvious that these divisions should be excluded, it should be noted that it results in some high-profile and highly political issues being removed from the analysis, which under a wider definition might be argued to be government defeats. For example, 23 divisions related to hunting and seven to House of Lords reform.⁵ At the same time some high-profile ‘conscience’-type issues remain in the analysis — for example, votes on the abolition of Section 28, on rights of unmarried couples to adopt, and the Civil Partnerships Bill. All of these were whipped on the government side.

The second qualification relates to the nature of parliamentary influence. Defeats in the division lobbies are a blunt instrument by which to extract concessions from government. Peers are generally more inclined to deploy powers of persuasion where they can.⁶ A common tactic is to raise an issue at the early stages of a bill, but not press an amendment to the vote if the relevant minister states that s/he will go away and consider the matter again. This often happens at committee stage in the Lords, where there are relatively few divisions. If the minister's response proves unsatisfactory the matter can then be pressed again at report stage, and the option remains to inflict a defeat at third reading if a compromise cannot be reached. This pattern helps explain the figures in Table 2 which shows how the 283 defeats on whipped divisions broke down by legislative stage.

Table 2 Defeats by legislative stage

It is very common for this process to result in government concessions in the Lords (Griffith, 1974; Shell, 1992; Miers and Brock, 1993). Indeed this may prove a more effective mechanism than pressing an issue to the vote, as it is low profile and thus enables ministers to concede policy changes without losing face. However, such concessions cannot be classified as defeats, even where an amendment was formally tabled at three stages under threat of defeat, and a similar government amendment followed. This simply illustrates that the Lords' impact on policy cannot be fully captured by a quantitative study of vote outcomes, but that a fuller analysis of the chamber's policy influence would be a long and laborious process.

The third qualification to the standard definition of defeats is that they do not necessarily result in real policy change. Just as government may concede without being defeated, it can also be defeated without having to concede. This makes defeats in the Lords very different from defeats in the House of Commons. As the Commons has the last word, the government frequently attempts to overturn Lords defeats when a bill returns there for approval. The common perception may be that this is what happens most of the time (though our research — as touched on later — shows this not to be the case). Due to this process of negotiation it could be argued that the number of defeats in the Lords, per se, is unimportant. What is of greater interest is the number of defeats that ‘stick’ and thus result in real policy change. We return to this issue briefly in the final section of the paper.

First, however, we consider in more detail the 283 ‘standard’ defeats that occurred, and the factors explaining why the government lost these divisions.

Potential Contributors to Defeat: Political Groups in the House of Lords

The short answer to why government gets defeated in the House of Lords is that Labour does not have a majority in the chamber. The breakdown of membership in the Lords at the end of our period of study is given in Table 3. This shows that it contained three roughly evenly balanced groups: Labour, the Conservatives and the independent ‘Crossbenchers’, each with around 200 seats. The Liberal Democrats then had 69 seats. In addition the Church of England Bishops hold 26 seats,⁸ and a small number of other non-aligned members make up the total. Although numbers fluctuated slightly, the proportions held by these groups were relatively stable over the period we are considering. Only following the 2005 general election did the dissolution honours list, which included 16 former Labour MPs, make Labour narrowly the largest party in the chamber for the first time (Russell and Sciara, 2006a). In early 2007 the appointment of a new group of Crossbench peers resulted in the Conservatives becoming the third largest group.

Table 3 Party breakdown in the house at the end of the 2004–2005 session

This is a rather different situation to that which applied in the House of Lords throughout the 20th century, when the Conservative Party was dominant in the chamber. Immediately before reform in 1999 the Conservatives held 471 seats to Labour's 179, while the Liberal Democrats held 72 seats and the Crossbenchers 353. Given the low turnout among Crossbench peers (discussed below), the Conservatives were well placed to win divisions without the support of other forces when they were determined to do so.⁹ In practice this meant that the party acted with considerable restraint during periods of Labour government, and did not force large numbers of divisions (Bromhead, 1958; Morgan, 1975).

In the post-1999 house, Labour has to secure the support (or at least the abstention) of a significant number of members from other groups in order to win a vote in the House of Lords. It is important to note that many decisions are taken without a division — for example, it is rare for the House to divide on the second reading of bills.¹⁰ But where there is a division, any group has the potential to act as a veto player, depending on the behaviour of other groups. This situation is made more unpredictable and complex by the widely recognised ‘less-partisan setting’ of the House of Lords (Judge, 2005, 76), where the whips have few sanctions and many members may choose not to vote at all. Particularly given that the government lost 51 divisions over this period by fewer than 10 votes, it is clear that the behaviour of Labour rebels, Conservatives, Crossbenchers, Liberal Democrats or Bishops could each potentially make the difference between victory and defeat.

We therefore structure the main body of the paper by looking at the behaviour of each of these groups in turn, and their influence on the outcome of divisions. Summary data, and a discussion, is included in the final section of the paper.

Labour

In the House of Commons it is the behaviour of the government's own backbenchers that is most critical to legislative outcomes (King, 1976; Cowley, 2005a). While holding a majority, governments can generally discount the votes of opposition parties, so long as they can depend on the support of their own MPs. In the contemporary House of Commons it is, therefore, Labour rebels, in most instances, who are the pivotal voters.

In the House of Lords, Labour rebels are only one group among many that could contribute to government defeat. This makes them a less obvious focus than they are in the House of Commons. However, given the government's weak numerical position in the upper house, it potentially needs even more to maximise its own vote in Lords divisions. A few abstentions or rebels can better be afforded in the Commons, where the government's majority has been comfortable since 1997. This is a challenge in an environment where party discipline is generally considered to be weaker.

As Norton (2003) has shown, party voting in the House of Lords is actually remarkably cohesive, despite the relatively weak sanctions available to the whips. Of the 806 whipped divisions over the period we are studying, dissenting votes were cast by Labour members in 206 cases (26%). This was only a slightly higher proportion than the 21% in the House of Commons during 2001–2005 (Cowley, 2005b). And most of the time these rebellions in the Lords were very small — as shown in Table 4. On only 13 occasions over the period did more than 10 Labour members rebel.¹¹ Of these, six could be considered to have an element of ‘conscience’, despite there being a government whip (e.g., the repeal of Section 28 or allowing unmarried couples to adopt). Of the remainder, three related to the 2005 Prevention of Terrorism Bill — the subject of the greatest argument between the Commons and the Lords since the early 20th century (Russell and Sciara, 2006a), one to the protection of pension rights when the National Air Traffic Services (NATS) was in privatised in 2000, and one to the attempt to restrict trial by jury in the Criminal Justice Bill of 2003. The bill which attracted the largest rebellion (and also a second rebellion of over 10 votes) was the Communications Bill in June 2003, with 33 Labour members seeking to rebalance the interests of the citizen and the consumer in the bill.

Table 4 Divisions including Labour dissenting votes, 1999–2005

Of our 283 defeats, 105 included some Labour rebels. However, given the small numbers involved, they were rarely enough on their own to make the difference. For example, on the Communications Bill the 33 Labour rebels were joined by sufficient Conservatives, Liberal Democrats and Crossbenchers to ensure that the government lost the division by 105 votes. Indeed if all Labour members had always voted loyally, only eight government defeats would have been prevented (assuming that all other votes remained unchanged).¹² The most dramatic occasion was that relating to pension rights of NATS staff, when the government lost by one vote but would have won by 21 had it been supported in the division lobbies by all the Labour peers who rebelled.

The influence of rebellious Labour peers, however, cannot be fully captured by such an analysis, as the voices of rebels may be influential with other members in the chamber. Crossbenchers, in particular, may be swayed to vote against the government if it is clear that there are splits in its own ranks. This again demonstrates the very different politics that applies in the Lords when compared to the House of Commons, and also the limits of a wholly quantitative approach. An amendment moved by a Labour backbencher is obviously a particularly effective way of signalling to Crossbenchers, opposition peers and the outside world that there is disquiet on the government side. For example, the amendment to the Communications Bill was moved by Lord Puttnam (who had chaired a joint committee on the draft bill), the amendment on NATS was moved by Labour's Lord Brett, and a key amendment to add a sunset clause to the Prevention of Terrorism Bill was moved by Baroness Hayman (then a Labour peer). All of these resulted in defeat, and the last in one of the largest defeats the government has suffered in the chamber.¹³

More frequently, a Labour member will add their name to an amendment moved by a member of another party (or a Crossbencher) to indicate that concern extends across the House, or express their dissent in debate or in statements outside the chamber. In this context, it is worth noting that though the number of Labour rebels in the Lords is small, some members rebel very publicly and in spectacular style. In the House of Commons, the most rebellious member is Jeremy Corbyn, who cast 148 votes against the government in the 2001–2005 session, but still voted against his party only 12% of the time (Cowley, 2005a). In the Lords three Labour members cast a greater proportion of their votes than this against the government. The leading rebel was Baroness (Helena) Kennedy, who cast 51 votes against the government over this period and only 103 votes in its favour (a dissention rate of 33%). Her outspoken criticisms of government policy on various civil liberty issues helped set the media agenda and, particularly given her legal expertise, were likely to have been influential with Crossbench members.

Potentially as important as Labour rebellions are abstentions or absences among the Labour ranks. Overall, the level of absence from votes is far higher in the House of Lords than in the Commons, for a number of reasons. Members of the Lords are not paid a salary, and many have jobs outside parliament which prevent them from attending. Some have taken a peerage as an honour, rather than a job, and choose to attend rarely if at all (e.g., former Labour Deputy Leader Lord Hattersley cast only 31 votes out of a possible 873 over this period). Others are elderly or in poor health, but remain members (since there is no means of retirement from the House)¹⁴: at the start of 2006 there were 22 Labour peers aged over 80. These factors affect the turnout among all groups in the chamber. In the parties, however, a further factor is that members who disagree with their frontbench line are strongly encouraged by their whips to stay away, rather than to vote against. This saves the party embarrassment, and also loses it only one vote (rather than losing it one and gaining its opponent one). Given the many possible reasons for absence from divisions, and the fact that members cannot request that an abstention is formally recorded, it is impossible to know how much deliberate abstention goes on in the Lords.

Over this period the average Labour turnout in whipped divisions was 53%. It was slightly higher (57%) in divisions where the government was defeated than that in divisions which it won (51%). This can largely be accounted for by whipping: where there is a threat of defeat party managers will try harder to encourage Labour peers to attend. Many more divisions might have been won had Labour whips been able to rely on higher attendance among their party colleagues. For example, if Labour turnout in all divisions had been at least 67% (and all additional members voted loyally, all other factors remained the same), 93 defeats would have been averted. One way of improving turnout is adding new, younger, members to the chamber. But even among the 140 Labour peers who joined the Lords between 1997 and the 2005 election (most of whom were appointed as ‘working peers’, with an increasing emphasis on attendance) turnout averaged only 61%.

The Crossbenchers

The existence of the Crossbenchers in the House of Lords can be explained by its history. Until the passage of the Life Peerages Act 1958 the vast majority of members entered the chamber as a result of inheriting a title. Many had no firm party allegiance; hence it was natural for a non-aligned group to form, and this took its name from the benches which are neither on the government nor opposition side. With the arrival of life peerages the tradition of adding independent members to the chamber then continued. The Crossbenches include many retired senior civil servants, diplomats and military personnel, as well as distinguished scientists, lawyers and academics who choose not to take a party whip. Since its establishment in 2000, the House of Lords Appointments Commission has largely taken responsibility for selecting non-party peers, providing most, though not all, new Crossbench members.¹⁵ The group also includes a number of peers who previously represented political parties, and a small number of minority party representatives (at the end of the 2004–2005 session these numbered eight from the Ulster Unionist Party and one from Plaid Cymru). Under the ‘Weatherill compromise’ of 1999, whereby 10% of hereditary peers remained in the chamber, 32 Crossbench hereditaries were retained.

Although not a party group, the Crossbenchers do have their own organisation, including an elected ‘convener’ who has some office support, and weekly meetings and mailings through which they share information on business in the chamber (see Russell and Sciara (2006c) for a fuller discussion). Nonetheless the group takes no collective position on matters discussed. Given that it has around 200 members, it has the potential to be very influential on legislative outcomes; but there are a number of factors which make its impact much less significant than the raw numbers might suggest.

The first factor is turnout among Crossbench peers in divisions, which is far lower than that among the party groups. In government-whipped divisions across this period, the average Crossbench turnout was only 18 members (i.e., 10% of the group). Of all the 806 whipped divisions over this period, Crossbench turnout exceeded 30 members only 95 times, and exceeded 50 only 19 times. The highest turnouts were all on constitutional and civil liberties issues (in part due to the large number of lawyers on their benches), or ‘family values’/sexual morality issues.¹⁶ The highest was over the retention of Section 28 in July 2000 (82 Crossbench members). Others included votes on rights of unmarried couples to adopt children in November 2002 (75 Crossbenchers), the retention of the Lord Chancellor in the Constitutional Reform Bill in 2004 (72), and inserting a sunset clause in the Prevention of Terrorism Bill in 2005 (67).

The low average turnout of this group is not particularly surprising. Unlike party representatives, the Crossbenchers have no whips pressurising them to attend the chamber, and nobody in authority telling them which way to vote. This means absorbing numerous papers and briefings, and actually sitting and listening to debates. Added to the fact that members receive no salary, have no electorate to answer to, and many have important responsibilities outside the House, the Crossbenchers' low participation in Lords divisions is easily explained.

The other factor which restricts the Crossbenchers' influence on legislative outcomes is obviously that when members of this group do vote, they do not all vote the same way. Thus even a high turnout may have little impact, if Crossbench votes in effect cancel each other out. This is a significant factor. So, for example, in the vote on unmarried couples' right to adopt, 30 Crossbenchers supported the government (which sought to extend this right) and 45 voted against. This pattern is replicated on a smaller scale in many other divisions. Nonetheless, there are trends discernable in the pattern of Crossbench votes. Of the 14,626 votes cast by Crossbenchers over this period, 9,252 (63%) were against the government and 5,374 (37%) in its favour. At the individual level the voting pattern of Crossbenchers varies widely: many vote against the government the great majority of the time, some largely vote with the government, while others divide their votes relatively evenly (Russell and Sciara, 2006c). But using the standard measure of legislative party cohesion, the Rice index, which equals 100% when a group is perfectly cohesive and zero when it is completely split, the Crossbench group scores 50%.¹⁷ Crossbenchers argue that the overall imbalance of votes does not show a party bias, but a bias against bad government legislation. Bearing in mind that divisions are called in the Lords only when there is some controversy, this is difficult to test.

The combination of the low Crossbench turnout and frequently divided votes means that this group holds the balance of power in the House of Lords far less often than its size would suggest. We calculate the group's voting power by looking at how many times an outcome would have changed if none of its members were present. This obviously does not capture the many other forms of impact which the Crossbenchers can have: for example, in sponsoring amendments and contributing to debates which (particularly where the sponsor is a recognised expert) may be influential on the government or on party peers. But in this relatively crude sense, we find that Crossbenchers affected the outcome of only 50 of the 806 divisions over the period. The breakdown of these outcomes is given in Table 5.

Table 5 Outcome of whipped divisions 1999–2005 if Crossbenchers had not voted

The Crossbenchers held the balance in 47 defeats over this period (17% of the total). In 37 cases the government would have won had they been absent, and in 10 there would have been a tied vote (which in practice generally means that the government would have won).¹⁸ In just two additional cases the government won, but would have lost without the support of the Crossbenchers. In one final case a government victory would have been turned into a tied vote, which in practice would not have affected the outcome.

The Crossbenchers' impact is largely felt in divisions which are close to the line between victory and defeat. In only half of the defeats where they made a difference did the government lose by more than five votes. For example in 2003, the Lords voted to remove a clause from the Criminal Justice Bill to allow hearsay evidence in court. The government was defeated by four votes but would have won by four if no Crossbenchers had voted (two Crossbenchers having voted with the government and 10 against). The size of Crossbench turnout does have some effect on the group's impact. So, for example, in five defeats on the Constitutional Reform Bill in 2004–2005 over 50 Crossbenchers voted, and each time their votes were crucial to the outcome. But since high Crossbench turnouts are often accompanied by internal divisions, and also by high turnouts across the chamber as a whole, these votes are often still not decisive. For example 72 Crossbenchers contributed to the government's largest defeat over the period, on the sunset clause to the Prevention of Terrorism Bill (voting 60–12 against the government), but the government lost by 187 votes. Similarly the Crossbenchers voted 45–11 to remove a religious hatred clause from the Anti-Terrorism, Crime and Security Bill in 2001, but this did not affect the outcome, as the government was defeated by 99 votes. We conclude that the Crossbenchers can affect policy outcomes on a limited set of issues, or where the size of defeat is small.

The Bishops

The Bishops are interesting because — like the Crossbenchers — they do not take a party whip, but also because their continued presence in the chamber is controversial. The Bishops' impact, however, is limited by the fact that they are a small group, and that like the Crossbenchers they vote relatively little, and do not vote as a cohesive block. On most occasions there is one Bishop ‘on duty’ in the House (they have a formal rota) and the average turnout from the group in government-whipped divisions is only 3.2% (i.e., less than one person). On only 10 occasions over this period did more than five Bishops participate in whipped divisions, and indeed on only 66 occasions did more than two Bishops take part. The largest turnouts were 11 votes on the balloting of grammar schools in 2000 (when nine Bishops supported the government and two opposed), on the Civil Partnerships Bill in 2004 (eight supporting, two opposed), and on the Learning and Skills Bill in 2000 with respect to sex education guidelines to replace Section 28.¹⁹ In the last of these cases the compromise amendment agreed with the government was moved by a Bishop, and nine supported this with one voting against.

Unsurprisingly then, the Bishops' votes only rarely make a difference to legislative outcomes. Of our 806 divisions this occurred only three times. Twice the government was defeated by one vote, with the vote of one Bishop making the difference between this and a tied vote — which in practice would have meant a government victory.²⁰ Only once, on the Nationality, Immigration and Asylum Bill in 2003, was the difference a clear one between government victory and defeat. This was over the issue of education of asylum seekers' children, where an amendment moved by the Bishop of Portsmouth required that such children are taught in a school and not separately in a detention centre. Three Bishops attended and voted against the government, and it lost the division by one vote. This division also attracted 19 Crossbench votes, 17 of them cast against the government, plus four Labour rebels, while the Conservatives abstained. In this case, therefore, there were three pivotal groups. Clearly, however, the Bishops' influence came not just through voting but also through influencing the votes of others. This is where their presence is generally more important.

The Conservatives

Not surprisingly, the Conservatives contributed to almost every government defeat in the House of Lords over this period. On the one defeat just mentioned the party abstained, but in the other 282 cases its peers always voted against rather than for the government's position. This is not to say that Conservative voting itself assures government defeat: in 426 cases the Conservatives opposed the government in the division lobbies but it nonetheless won the vote. In other cases the group abstained. But in just 17 cases (all resulting in victory) did the Conservatives cast their votes with the government.

The opposition of the Conservatives is generally required in order to bring about a government defeat. However there were five defeats which would have occurred (all else being equal) even if the Conservatives had not voted. Four were the four largest defeats over the period, all on the Prevention of Terrorism Bill 2005, where the range of forces against the government meant that it was defeated by between 127 and 187 votes. The fifth was on Lord Puttnam's amendment to the Communications Bill, where the government would have lost by 31 votes even without the Conservatives' participation, in part because of the 33 Labour members who rebelled.

The Conservative group benefited from its large size and also its cohesive voting behaviour. There were only four occasions over this period where more than 10 Conservative peers rebelled, and the Rice Index for the group was over 99%. This is a change from the old hereditary days, when many Conservative members had a far weaker association with the party. However, one tradition that has proved more enduring is poor attendance, which weakened the Conservatives even more than Labour. There were relatively few appointments to the Conservative side of the House between 1997 and 2005: just 58 new peers compared to Labour's 140. As a result Conservative peers were on average older and had served in the chamber for a longer time (see Table 7). The lack of renewal also helped retain the group's old ethos, including caution in inflicting government defeats. Over this period, the party's turnout in whipped divisions (excluding divisions on which it abstained) was just 34%. That represents an average Conservative turnout of only 73 members, out of an (average) available group of 214. When turnout was higher, defeat was far more likely: of the divisions where Conservative turnout was over 50%, 76% were defeats, compared to 29% of all other divisions. As with Labour, this can be put down to heavier whipping. In general, the Conservatives continue to whip far more lightly than the other parties, and thus consciously choose not to defeat the government as much as they could. Had the group maintained a turnout of 50% in all whipped divisions, Labour would have lost a further 218 votes. Although Conservative peers, therefore, played an important role in defeat, their impact could have been far greater. With more renewal on the Conservative benches, a Labour government could expect to be defeated substantially more often.

The Liberal Democrats

The Liberal Democrats are a far smaller group than either the Crossbenchers or the Conservatives, but play a highly significant role. They also have some distinct characteristics compared to those of the other parties. While the Conservative benches have seen relatively little renewal since Labour came to power, two-thirds of the Liberal Democrat peers sitting in the chamber at the end of the 2004–2005 session were new since 1997 (see Table 7). This matches the level of turnover on the Labour benches, and results in part from the close relationship which the two parties enjoyed prior to the 1997 election. Early Blair appointments sought to end the Liberal Democrats' previous underrepresentation in the chamber in comparison with their general election vote. As a result the group is the most youthful in the chamber, with an average age of 64 (again, see Table 7) and fewer problems with attendance among members who are elderly and infirm. Although average turnout in whipped divisions where the party took a position was only 51%, this compared favourably with the turnout on the government benches and far exceeded that of the Conservatives. In defeats the average Liberal Democrat turnout was 63%, and the party actually managed a turnout of over 80% on 50 occasions (a performance matched on just four such occasions by Labour and three by the Conservatives). Of these 50 divisions with the highest Liberal Democrat turnout, 38 were government defeats.

The results of our survey showed some strikingly different attitudes among Liberal Democrat peers when compared to the other parties, and particularly to the Conservatives. When asked whether ‘party peers have a duty to attend the House as much as they can’, 35% of Liberal Democrat peers strongly agreed, compared to just 17% of Conservatives (on the government benches, where the pressure to attend is naturally highest, the proportion sharing this view was 52%). Similarly when asked whether ‘when party peers vote they have a duty to support the party line all or most of the time’, 89% of Liberal Democrat peers agreed (20% strongly) compared to 58% of Conservatives and 83% of Labour peers. In organisational terms the Liberal Democrats showed an impressive degree of linkage between their party groups in the two chambers (made possible in part due to their smaller size), with 83% agreeing that relations between the two groups were good, compared to just 49% of Conservatives and 42% of Labour peers who said the same of their parties.

The Liberal Democrat group is also strongly ideologically cohesive. When asked in our survey to place themselves and their party leader on a 10-point left–right scale, only 17% of Liberal Democrat peers placed themselves more than one point away from the position they attributed to their then leader, Charles Kennedy. In comparison 46% of Conservatives placed themselves this far from Michael Howard and 53% of Labour peers placed themselves this far from Tony Blair (in both cases largely to the leader's left). This cohesiveness showed itself in voting in the chamber, with a Rice index of 99%. There were just 39 divisions in which any dissenting Liberal Democrat votes were cast, compared to Labour's 101 and the Conservatives' 62. On the majority (22) of these occasions there was only one Liberal Democrat rebel.

As a result, in terms of legislative outcomes, this group certainly ‘punches above its weight’. Of the 283 defeats in the chamber during the period we are studying, the Liberal Democrats supported the government on just 12 occasions. In a further seven defeats they abstained or their position was unclear, but on the remaining 264 they opposed the government position. In the overwhelming majority of cases it is, therefore, Liberal Democrat votes, coupled with those of the Conservatives, that are responsible for inflicting government defeat. Of the 264 defeats where the Liberal Democrats opposed the government, it would have won in 179 if they had abstained, or in 256 had they voted with it (as always, assuming that the behaviour of other groups remained unchanged).

The 12 divisions when the government was defeated despite Liberal Democrat support included six on the Constitutional Reform Bill (to abolish the position of Lord Chancellor) and two on Section 28. As noted above, these were both issues on which there was a high Crossbench turnout, and most Crossbenchers voted against the government. On the second issue, there were also large numbers of Labour rebels (11 and 18, respectively). It is, therefore, on occasions when the Liberal Democrats support the government that these sets of voters can come into play, in coalition with the Conservatives, and make the difference between government victory and defeat. But it is rare that such a coalition can succeed, and an unusually high Conservative turnout is necessary for it to do so. If the Liberal Democrats abstain, the Conservatives have a better chance of inflicting defeat with the help of a handful of other voters, as they did in the other seven cases.

The Liberal Democrats are, therefore, the most important pivotal group in the House of Lords. But given the small size of the group, it is obviously almost impossible for it to defeat the government on its own. (The government won 198 divisions despite Liberal Democrat opposition, and only once did the Liberal Democrats contribute to defeat without the support of the Conservatives. This was on the amendment on education of asylum seekers' children moved by the Bishop of Portsmouth.) In general, defeat depends on both opposition parties uniting against the government. In total there were just 20 defeats — this one on asylum seekers' children by the Liberal Democrats and the 19 by the Conservatives referred to in the previous paragraph — where one opposition party, but not the other, voted against the government's position.

While the Conservatives have fairly consistently opposed the government in the House of Lords, the position of the Liberal Democrats has changed markedly during the 6 years since 1999, as shown in Table 6. In the early years of the Blair government the party tended to support it on divisions called by the Conservatives. In more recent years, this pattern has been reversed, with Liberal Democrats increasingly tending to oppose the government in most divisions and often taking the lead on pressing for votes on issues such as civil liberties. This is the main explanation for the upward trend in defeats over this period that was seen in Table 1. The change of attitude towards the government has been noted with respect to the Liberal Democrats in the House of Commons (where it makes little practical difference), and on the election trail (Denver, 2001; Cowley and Stuart, 2003; Russell, 2005). But its impact in the House of Lords, where it can have a real difference on policy outcomes, has largely gone unnoticed. This is a new phenomenon, brought about by the reform of the chamber in 1999.²¹ It has significantly changed the relationships between the parties in the upper house. In earlier periods, when Labour was in power, the Conservatives could inflict defeat easily if they chose to, whereas it is now difficult for them to do so without Liberal Democrat support. The two parties have, therefore, become more interdependent. In recent years, Blair's government has had to court the Liberal Democrats in order to gain support for key social and economic policies, but also depended increasingly on the Conservatives on criminal justice and security issues. This pattern was seen, for example, over the Prevention of Terrorism Bill of 2005, and over the Identity Cards Bill in 2006. Here the Liberal Democrats held firm in their opposition to the government, while the Conservatives in the Lords ultimately abstained allowing the policy through.

Table 6 Liberal democrat position in government whipped divisions, 1999–2005

It is notable that it is Liberal Democrat leaders who have become the most vocal defenders of the House of Lords' right to block government legislation. Party leaders have repeatedly renounced the ‘Salisbury convention’, under which government manifesto measures get an easy passage through the Lords, and have pledged to resist any reduction in the chamber's powers (see, e.g., Lord McNally: Lords Hansard 17 May 2005, cl. 21; 6 June 2005, cls. 759–760). They have also frequently pointed out that the party balance in the Lords is more proportional to vote shares than is that in the House of Commons (see, e.g., Lord Goodhart: Lords Hansard 17 May 2006, cl. 265; Lord McNally: Lords Hansard 19 June 2006, cl. 510). Conservative leaders, who see themselves as the government in waiting, have been markedly more cautious on all of these points.

Discussion

The analysis in this paper challenges some of the common assumptions about the House of Lords. But our findings also challenge more fundamental assumptions about the British political system, with respect to the relationship between the parties, and between government and parliament.

In terms of the House of Lords, Table 7 summarises some of the key data discussed above, including turnout, cohesiveness and profile of members in different groups and the extent to which these groups contributed to defeat. As already noted, only eight defeats would have been averted if all Labour members voting had voted loyally (and three if Labour rebels had simply abstained). With respect to Crossbenchers, 47 defeats (mostly narrow ones) would have been averted had they all abstained — and even in the unlikely event that those voting had cohesively supported the government only 152 defeats would have been averted. The main influence, therefore, lies with the opposition parties. In particular, the Liberal Democrats, despite their far smaller size, influenced the outcome on almost as many occasions as the Conservatives. Had this group consistently supported the government (as it did most of the time in its early years) there would have been only 27, rather than 283, defeats in the Lords over this period. These results demonstrate that the ‘non-partisan nature’ of the House of Lords can be overstated, or at least misunderstood. Cross-voting makes relatively little impact, and the influence of Crossbenchers is proportionately far smaller than the size of this group suggests. The main players in the chamber are highly cohesive party blocks.

Table 7 Summary information on groups in the Lords 1999–2005

This is not to suggest that the chamber mimics the House of Commons or other ordinary parliamentary chambers in its voting behaviour. It does have a ‘less-partisan’ nature in important ways. The outcome of votes cannot be wholly determined by looking at the size of the party groups alone, due to high levels of absenteeism. Some absences are due to abstention, but these cannot be distinguished from absences for other reasons. Labour absences accounted for more defeats than Labour rebellions, and the Conservatives could have inflicted far more defeats had their members participated more regularly in divisions. Influence may be exerted by high-profile Labour rebels, Crossbenchers or Bishops whose networking and debate contributions persuade others to vote or to stay at home. This creates unpredictability, though a basic fact remains. Over 90% of defeats depended on the combined forces of the two opposition parties.

At the start of the paper, we suggested that the importance of Lords defeats could be dismissed if these were routinely overturned in the House of Commons. In this case they might be no more than an irritation to government. But this is not the case, and many defeats go on to result in real and significant policy change. It is often less costly for government to concede a point to the Lords than to seek to overturn it, as this could use precious legislative time and result in damaging media exposure. Lords defeats over this period that were not reversed included limitations on control orders in the Prevention of Terrorism Bill (2005), blocking the offence of religious hatred (2001), preventing limitation on trial by jury (2000 and 2003) and ensuring the office of Lord Chancellor was retained (2004). These were all high profile, but there are many others, for example, in the Tax Credits Bill (2001) to require the Chancellor to lay an annual report before parliament, in the Adoption and Children Bill (2002), to ensure that a child has legal representation during the adoption process, and in the Higher Education Bill (2004) to ensure that student fees would not apply to students who were offered a university place in 2005 that they did not take up until 2006. An analysis of all 283 defeats over this period shows that only four out of 10 were completely reversed by the government, while the remainder were influential on the final outcome to some extent (Russell and Sciara, 2007). Not only is government regularly defeated in parliament, in defeats which are rarely brought to the attention of the public, but it is also now often forced to compromise with opposition parties in order to get its legislation. We can quantify the compromises which come through Lords defeats. But probably more significant is the compromise that goes on between the parties ‘in the shadow’ of Lords defeat, which is far more difficult to measure.

The British political system is traditionally seen as one with few formal checks on the executive, sustained by a single party majority in parliament and a weak upper house (Lijphart, 1999). Tsebelis (2002, 78–79) classified Britain as having only one veto player — the governing party — since coalitions are rare and the Lords was considered insignificant. But while the Lords does not have an absolute veto in the policy process, the examples above demonstrate that the chamber can have a real impact. Lords reform has strengthened its influence in two interrelated ways. First, it has created a ‘no overall control’ chamber, where various sets of actors can combine to create a majority against the government. Second, it has created a more confident chamber, which the government itself has described as ‘more legitimate’ (Russell and Sciara, 2006b). The sum result is that the effective number of veto players in the British system has increased, despite the Lords' lack of an absolute veto. The Liberal Democrats and Conservatives, individually and collectively powerless in the House of Commons, are both partisan veto players in the House of Lords (Tsebelis, 2002). The Crossbenchers, Labour rebels and even Bishops also have a (lesser) potential to play this role. Negotiation between all groups in the Lords is, therefore, substantial and regular. But the main winners from this situation are the opposition parties and, given the cautious approach taken by the Conservatives and particularly the Liberal Democrats. The desire to avert Lords defeat has resulted in negotiation at earlier stages of the policy process, and almost certainly some proposals not being advanced at all when it is clear that compromise cannot be reached. The extent to which the government is now working with both parties was evidenced, for example, by talks prior to the introduction of the Prevention of Terrorism Bill, and later its successor the Terrorism Bill, in 2005. Here the Liberal Democrats were included on an equal basis to the Conservatives, in clear recognition of both parties' power in the House of Lords.

Reform of the House of Lords in 1999 has, therefore, fundamentally changed the dynamic between the parties, enhancing the position of the opposition and, particularly, of the third party. It has led to greater negotiation between government and opposition parties, and strengthened links between the two opposition parties, thus creating a far more plural system at Westminster. This falls short of what might be expected if elections for the Commons moved to a proportional system, but it has some similar effects. Furthermore, it is a dynamic that is likely to endure, and become established, in the longer term. All parties have stated a commitment to there being no partisan majority in the Lords, whether it remains appointed or is reformed to introduce elections. In future Conservative governments, as well as Labour ones, will, therefore, be forced to negotiate policy with a range of forces in the chamber. It seems clear that, despite its continuing unelected basis, the House of Lords has a growing role in the policy process. It also has the potential to change some of the fundamentals of British politics in significant ways.