Contextuality, Pigeonholes, Cheshire Cats, Mean Kings, and Weak Values

Abstract

The Kochen–Specker (KS) theorem shows that noncontextual hidden variable models of reality that allow random choice are inconsistent with quantum mechanics. Such noncontextual models predict certain outcomes for specific experiments that are not observed in practice, and this is how the theorem is proved. A realist hidden-variable model suggested by the Aharonov–Bergmann–Lebowitz reformulation of quantum mechanics is introduced to explain why those outcomes are never observed. Just as the KS theorem requires them due to noncontextuality, this model requires independent truth-value assignments for each observable, but now allows that the entire set of assignments depends on both the pre-selected and post-selected quantum states in a time-symmetric manner. Using sets that prove the KS theorem, along with pre- and post-selected states, we find that particular projectors within the set cannot be assigned logically consistent truth-values, and furthermore that the weak values of these projectors have a corresponding signature. The contextual behavior has effectively been confined to a particular context by the pre- and post-selection. This inconsistency is never observed because it is never in the pre-selected or post-selected quantum state, but its signature can be experimentally revealed through weak measurements. We also show that for specific cases where the logical inconsistency is in a ‘classical basis,’ this gives rise to the quantum pigeonhole effect. As a related issue, we show using weak values that any KS set can be used to ensure that the Mean King always wins the game.

Appendix

In the following appendices we derive some general formulae for working with the ABL reformulation, weak values, stabilizer states, IDs, and KS sets within the N-qubit Pauli group, and how to use them to show the pigeonhole effect. Using these formulae, we proceed to carefully work through several important examples, deriving other results along the way.

1.1 The ABL reformulation of quantum mechanics

Here we give a brief review and generalization of some relevant parts of the ABL formalism.

To begin, let us define some general notation to refer to projectors of different rank. We define a rank-1 projector as

$$\begin{aligned} \Pi _i^1 \equiv |i\rangle \langle i|, \end{aligned}$$

(15)

and subsequently define a rank-r projector in terms of rank-1 projectors as

$$\begin{aligned} \Pi _i^{r_i} = \sum _j^{r_i} |j_i\rangle \langle j_i| = \sum _j^{r_i} \Pi _{ij}^1. \end{aligned}$$

(16)

where $\{|j_i\rangle \}$ is any orthonormal basis (kets forming rank-1 projectors) that spans that $r_i$-dimensional subspace $\Pi _i^{r_i}$ projects onto. The arbitrary choice of $\{|j_i\rangle \}$ is analogous to preparation-independence of mixed-state density matrices.

We define a complete measurement basis ${\vec {B}}$ on a system of Hilbert-space dimension d, as any set of mutually orthogonal projectors that span the space—regardless of their individual ranks.

$$\begin{aligned} {\vec {B}} = \{\Pi _1^{r_1},\Pi _2^{r_2},\ldots ,\Pi _{d'}^{r_{d'}}\}, \end{aligned}$$

(17)

where $d'$ is the cardinality of the set ${\vec {B}}$, $\sum _i^{d'} r_{i} = d$, and $\sum _i^{d'} \Pi _{i}^{r_i} = I$. Thus it is clear that ${\vec {B}}$ completely defines the POVM that will actually be measured during an experiment. This experiment truly measures all observables that have ${\vec {B}}$ as an eigenbasis, which is obvious when such an observable is spectrally decomposed,

$$\begin{aligned} A = \sum _i^{d'} a_i \Pi _{i}^{r_i}, \end{aligned}$$

(18)

where $a_i$ are the eigenvalues.

For pre-selected state $|\Psi \rangle$, post-selected state $|\Phi \rangle$, the weak value of a rank-1 projector is given by,

$$\begin{aligned} (\Pi _i^1)_w = \frac{\langle \Phi | i \rangle \langle i | \Psi \rangle }{\langle \Phi |\Psi \rangle }, \end{aligned}$$

(19)

whereas for a rank-r projector it is,

$$\begin{aligned} (\Pi _i^{r_i})_w = \frac{\sum _j^{r_i} \langle \Phi | j_i \rangle \langle j_i | \Psi \rangle }{\langle \Phi |\Psi \rangle } = \sum _j^{r_i} (\Pi _{ij}^1)_w. \end{aligned}$$

(20)

Note that weak values of projectors are defined without reference to any specific basis ${\vec {B}}$, which is one common definition of noncontextuality. It is important to note that the weak values also obey the sum rule, $\sum _i^{d'} (\Pi _{i}^{r_i})_w = 1$, for all possible bases ${\vec {B}}$.

Now, the ABL formula gives the probability to obtain a particular outcome when an intermediate measurement is made in basis ${\vec {B}}$ between the pre-selection of $|\Psi \rangle$ and post-selection of $|\Phi \rangle$.

$$\begin{aligned} P(|\Pi _i^{r_i}= & {} 1 / |\Psi \rangle , |\Phi \rangle , {\vec {B}}) = \frac{ |\sum _j^{r_i} \langle \Phi | \Pi ^1_{ij} | \Psi \rangle |^2 }{ \sum _k^{d'} | \sum _j^{r_k}\langle \Phi | \Pi ^1_{kj} | \Psi \rangle |^2 },\nonumber \\= & {} \frac{ |\sum _j^{r_i} (\Pi ^1_{ij})_w|^2 }{ \sum _k^{d'} | \sum _j^{r_k} (\Pi ^1_{kj})_w|^2 } = \frac{ |(\Pi ^{r_i}_{i})_w|^2 }{ \sum _k^{d'} | (\Pi ^{r_k}_{k})_w|^2 }, \end{aligned}$$

(21)

Now we have everything we need to derive the ABL rule. If the ABL formula gives probability 1 to obtain $\Pi _i^{r_i} = 1$ when measured in basis ${\vec {B}}$, it follows that $\langle \Phi | \Pi _k^{r_k} | \Psi \rangle = 0$ for all $k \ne i$. Then from the sum rule for weak values we find that,

$$\begin{aligned} (\Pi _i^{r_i})_w = \frac{\langle \Phi | \Pi _i^{r_i} | \Psi \rangle }{\langle \Phi |\Psi \rangle } = 1, \end{aligned}$$

(22)

which is the ABL rule. Specifically, if the ABL formula predicts unit probability to obtain a given projector by an intermediate strong measurement, then that projector has a weak value of 1. It is also trivial to see that when the ABL formula predicts probability 0 for a given projector, then that projector has a weak value of 0.

The reverse ABL rule is slightly more subtle. Given that $(\Pi _i^{r_i})_w=1$, then from the sum rule $\sum _{k \ne i}^{d'} (\Pi _k^{r_k})_w = 0$, for any basis that contains $\Pi _i^{r_i}$. If $d'=2$ for some basis ${\vec {B}}$, then we know both weak values, and we can see that the ABL formula gives unit probability to obtain outcome $\Pi _i^{r_i}=1$, which is the reverse ABL rule. For larger cardinalities $d' = 2+n$, it must also be given that $(\Pi _k^{r_k})_w=0$ for n of the projectors in ${\vec {B}}$ in order to ensure unit probability to obtain $\Pi _i^{r_i}=1$. It is also trivial to see that if the weak value of a projector is 0, then the ABL formula gives 0 probability to obtain that outcome by an intermediate measurement, and this is true for systems of all dimensions.

1.2 The Mean King’s problem

The examination of the weak values of all projectors in a KS set is also useful for seeing why the Mean King [28,29,30] will always win the game if he chooses measurements from a KS set. In order for the physicist (the other player of the game) to succeed, he must be able to predict with certainty the King’s outcome for an intermediate measurement of any observable from among a specified set of observables—that need not mutually commute. In other words, he must be able to choose a special pre-selected state and post-selected basis such that for each measurement outcome the King can obtain, the ABL formula predicts unit probability for that outcome, and zero for all other outcomes in that measurement basis. From the ABL rule, it then follows that every projector the suitor can measure must have a weak value of 0 or 1. Now, a KS set by definition cannot admit any noncontextual truth-value assignment of 0s and 1s to all projectors in the set without the sum rule being violated in some bases. As we have discussed above, the weak values of the projectors in any set, $v_w$, are noncontextual and obey the sum rule by definition, and therefore they cannot all be 0 and 1 for any KS set. It then follows from Eq. (21) that if the Mean King is allowed to make measurements from a KS set then he will always win the game, since there can be no pre- and post-selection for which the ABL probabilities to obtain the projectors by an intermediate measurement are all 0 or 1.

This simple new argument using weak values generalizes Mermin’s original observation [25] to all possible KS sets, and indeed to any set for which all 0/1 weak values are impossible. This fails to include many state-dependent proofs of contextuality, but it does apply to the 13-ray KS proof of Yu and Oh [31]. This is an interesting case because the set is colorable in the KS sense, but there is a geometric feature common to all legitimate colorings of the set that disagrees with a quantum mechanical prediction, allowing the KS theorem to be proved. This suggests that there could be a novel method for using weak values to identify many new colorable sets that prove the KS theorem, which will be worth exploring in the future.

It is worth mentioning that this application of the weak value has absolutely nothing to do with weak measurements—the weak value is simply a mathematical property of the pre- and post-selected quantum system that helps to make the situation more transparent.

It is also likely that the complete family of KS sets is more general than the group that can be explained by the diagonal-PPS proposition of [7], which also warrants further inquiry.

1.3 The N-qubit Pauli group

The first rule to note is that any two N-qubit Pauli observables must either commute or anticommute. As a result, if one observable is applied as a similarity transformation to another, then if the two observables commute the transformation does nothing, and if they anticommute it introduces a negative sign.

Let us now consider what happens if we act with an N-qubit Pauli operation U on such an N-qubit Pauli state. Applying a unitary to a quantum state also applies that unitary as a similarity transformation to all of the observables in that state’s stabilizer group. We must also be careful to track the relative phase a that is introduced by such a transformation.

$$\begin{aligned} U|\lambda _A A, \lambda _B B, \lambda _C C \rangle= & {} a|\lambda _A a U A U a^*, \lambda _B a U B U a^*, \lambda _C a U C U a^* \rangle \nonumber \\= & {} a|c_{AU}\lambda _A A, c_{BU}\lambda _B B, c_{CU}\lambda _C C \rangle = a|\lambda _A' A, \lambda _B' B, \lambda _C' C \rangle \end{aligned}$$

(23)

where $c_{QP} = 1$ if $[Q,P]=0$ and $c_{QP} = -1$ if $\{Q,P\}=0$, and these signs are simply absorbed into the eigenvalues of the new state as $\lambda _i' = c_{iU}\lambda _i$. Thus for all possible aU, the state remains an eigenstate of the same stabilizer group—though it may now be a different state in the eigenbasis. For $U=A$, it is easy to see that $a=\lambda _A$.

At this point we can derive a useful formula that we will need later. Suppose that V is another Pauli observable, such that $c_{QU}=c_{QV}$ for all $Q=\{A,B,C\}$, so that,

$$\begin{aligned} V|\lambda _A A, \lambda _B B, \lambda _C C \rangle = b|\lambda _A' A, \lambda _B' B, \lambda _C' C \rangle . \end{aligned}$$

(24)

Furthermore suppose that (A, U, V) is an ID with sign $s_{_{AUV}}$. Taking the inner product of the last two expressions we obtain,

$$\begin{aligned} ab^*= & {} \langle \lambda _A A, \lambda _B B, \lambda _C C|VU|\lambda _A A, \lambda _B B, \lambda _C C \rangle \nonumber \\= & {} s_{_{AUV}}\langle \lambda _A A, \lambda _B B, \lambda _C C|A|\lambda _A A, \lambda _B B, \lambda _C C \rangle \nonumber \\= & {} \lambda _A s_{_{AUV}}. \end{aligned}$$

(25)

We can use this formula to find relations between the weak values of different projectors in the same eigenbasis, and in some cases this is enough to calculate all of the weak values exactly.

1.4 KS sets within the N-qubit Pauli group

There exists a family of KS sets composed of observables and contexts from within the N-qubit Pauli group. The observables are simply tensor products of the Pauli spin matrices and the identity, while the contexts are sets of mutually commuting N-qubit observables. A maximal set of mutually commuting observables is called a stabilizer group, and we will further call any set of mutually commuting observables that is closed under multiplication a sub-stabilizer group. Within a given stabilizer group, one can find minimal sets of M observables whose product is $\pm I^{\otimes N}$ (identity in the space of all N qubits). Such sets contain $M-1$ independent generators, and the last observable is the product of those. We call these minimal subgroups IDs or Identity Products, and characterize them by the symbol ID$M^N$ for a set of M observables from the N-qubit Pauli group [18, 19]. Any ID is also a measurement context.

A minimal KS set [15,16,17], is a set of observables and IDs with the following properties: 1) each observable in the set appears in an even number of the IDs in the set, and 2) the number of IDs in the set whose product is $-I$ (negative IDs) is odd.

To see how such a set proves the KS theorem, consider the overall product of all of the noncontextual truth-values (${\pm }1$) for all the IDs in the set. If the product rule is to be obeyed, then this product must be −1, since there are an odd number of negative IDs in the set. However, the truth-value for each observable appears an even number of times in this product, since each observable belongs to an even number of IDs in the set, and thus noncontextuality implies that the product must be 1, and the theorem is proved.

There are many examples of such KS sets based on observables from the N-qubit Pauli group. In Figs. 1, 5, 9, 6, and 7, we give diagrams for some example KS sets that can show the pigeonhole effect for certain pre-and post-selections, as discussed below. The observables along a line or arc form an ID, and thick lines denote negative IDs, while thin lines denote positive IDs. These diagrams make the proof of the KS theorem transparent because it is easy to see that each observable lies in an even number of lines, while the number of thick lines is odd.

We can obtain another type of KS set composed of projectors, and contexts that are complete orthogonal bases of these projectors. A KS set of projectors and bases is a set that cannot admit a noncontextual truth-value assignment (0 or 1) to the projectors in the set in such a way that the sum rule is satisfied for all of the bases. Furthermore, a parity KS set is a set of projectors and bases in which each projector in the set appears in an even number of the bases in the set, and the total number of bases is odd [12,13,14, 32,33,34,35,36].

To see how a parity KS set proves the KS theorem, consider the overall sum of all of the noncontextual truth-values for all of the bases in the set. If the sum rule is to be obeyed, this sum must be equal to the odd number of bases. However, the truth-value for each observable appears an even number of times in this sum, and thus noncontextuality implies that the sum must be even, and the theorem is proved.

A projector-based KS set is also called critical, or minimal, if the KS proof would fail if any context were removed from the set.

Each ID also gives rise to a joint eigenbasis of projectors. The complete set of projectors taken from all IDs in a KS set of the former (observables) type always gives a KS set of the latter (projectors) type. This KS set is never a critical parity KS set, but always contains them as subsets.

1.5 3-Qubit square (or Wheel)

Let us now return to, and generalize, the example case of the 3-qubit square KS set. Specifically we are interested in the weak values of the projectors in the conflict basis. Our pre- and post-selected states are

$$\begin{aligned} |\Psi \rangle \equiv |\lambda _D D, \lambda _E E, \lambda _F F\rangle , \end{aligned}$$

(26)

and

$$\begin{aligned} |\Phi \rangle = |\lambda _G G, \lambda _H H, \lambda _I I\rangle , \end{aligned}$$

(27)

such that $s_{_{DEF}} = \lambda _D \lambda _E \lambda _F = +1$ and $s_{_{GHI}}=\lambda _G \lambda _H \lambda _I = +1$. This PPS always leads to a conflict in the classical basis. We define the maximum conflict projector for which all eigenvalues are opposite to the forced values.

$$\begin{aligned} |\mathcal {C}|= & {} |\lambda _D \lambda _G A, \lambda _E \lambda _H B, \lambda _F \lambda _I C|,\nonumber \\\equiv & {} |\lambda _A A, \lambda _B B, \lambda _C C|, \end{aligned}$$

(28)

noting also that $s_{_{ABC}} = \lambda _A \lambda _B \lambda _C = +1$ (which can be verified by direct substitution of $\lambda _A = \lambda _D \lambda _G,\ldots$).

The weak value of this projector is

$$\begin{aligned} |\lambda _A A, \lambda _B B, \lambda _C C|_w= & {} \frac{\langle \Phi | \mathcal {C} | \Psi \rangle }{\langle \Phi | \Psi \rangle }\nonumber \\= & {} \frac{\langle \lambda _G G, \lambda _H H, \lambda _I I|\lambda _A A, \lambda _B B, \lambda _C C|\lambda _D D, \lambda _E E, \lambda _F F\rangle }{\langle \lambda _G G, \lambda _H H, \lambda _I I|\lambda _D D, \lambda _E E, \lambda _F F\rangle }. \end{aligned}$$

(29)

Expanding $|\mathcal {C}|$ as a sum of rank-1 projectors and using Eq. (25), we can see that

$$\begin{aligned} |\lambda _A A, \lambda _B B, \lambda _C C|= & {} \lambda _A s_{_{ADG}} (G|\lambda _A A, -\lambda _B B, -\lambda _C C|D)\nonumber \\= & {} \lambda _B s_{_{BEH}} (H|-\lambda _A A, \lambda _B B, -\lambda _C C|E) \nonumber \\= & {} \lambda _C s_{_{CFI}} (I|-\lambda _A A, -\lambda _B B, \lambda _C C|F), \end{aligned}$$

(30)

where we have made explicit use of the commutation relations among these nine observables in order to obtain the values of $c_{QP}$.

Taking the weak value we obtain,

$$\begin{aligned} |\lambda _A A, \lambda _B B, \lambda _C C|_w= & {} \lambda _A \lambda _D \lambda _G s_{_{ADG}}(|\lambda _A A, -\lambda _B B, -\lambda _C C|_w)\nonumber \\= & {} \lambda _B \lambda _E \lambda _H s_{_{BEH}} (|-\lambda _A A, \lambda _B B, -\lambda _C C|_w) \nonumber \\= & {} \lambda _C \lambda _F \lambda _I s_{_{CFI}} (|-\lambda _A A, -\lambda _B B, \lambda _C C|_w), \end{aligned}$$

(31)

a relation between the weak values of all four projectors in the conflict eigenbasis of the ID (A, B, C), and note that $\lambda _A \lambda _D \lambda _G s_{_{ADG}}=\lambda _B \lambda _E \lambda _H s_{_{BEH}} = \lambda _C \lambda _F \lambda _I s_{_{CFI}} -1.$ We see that all four weak values have the same magnitude and differ only up to a sign. From this, and the fact that these four weak values obey the sum rule (i.e. they add up to 1), we find that all four weak values are real and have magnitude 1/2, and that the maximum conflict projector $|\mathcal {C}|$ has a negative (anomalous) weak value.

Finally, noting that the product of the signs of all four weak values is,

$$\begin{aligned} \lambda _A \lambda _D \lambda _G\lambda _B \lambda _E \lambda _H\lambda _C \lambda _F \lambda _I s_{_{ADG}} s_{_{BEH}} s_{_{CFI}}=s_{_{ABC}}s_{_{DEF}}s_{_{GHI}} s_{_{ADG}} s_{_{BEH}} s_{_{CFI}} = -1, \end{aligned}$$

(32)

we see that the negative weak value occurs exactly because this is a KS set, and the product of the signs of all IDs in a KS set must be −1.

It is important to point out that we obtained this result, even up to the exact weak values, without making use of the Hilbert-space form of the Pauli operators. Only the Pauli algebra and the structure of the KS set itself is required to obtain this result. Furthermore, the anomalous weak value follows directly from the fact that the KS set must have an odd number of negative IDs. To understand why this is important, consider that the proof we have just given applies just as well to the 2-qubit Peres–Mermin square as it does to the 3-qubit version, and to any other square configuration one might find—regardless of what observables and IDs may be used to legally populate that structure. It is also interesting to note that unlike the 3-qubit cases, the anomalous projector for the 2-qubit square depends on entanglement between the qubits.

The same method can be generalized for several other classes of KS structures to find the exact weak values. The details vary, but there are two general features that are always obeyed within these classes. First, in a conflict basis, the maximum real part of the weak value is 1/2, and second, every conflict eigenbasis contains at least one anomalous weak value. In all cases, the calculation can be performed entirely in the observable-based manner we have demonstrated.

The Kite family of KS sets is by far the most general and numerous, and the 2-qubit Peres–Mermin square is the simplest member. Unsurprisingly, the complete logic presented above extends to this entire family of KS sets (for all N), with the same four weak values as for the square. For this argument to hold with certain PPSs, the tail IDs must be reduced by multiplying all tail observables together, which yields eigenbases of higher-rank projectors that may be less entangled—essentially reducing the Kite to a square.

Fig. 7

A 5-qubit Kite

We can also give the general solution for the complete family of Wheels for all N, of which the 3-qubit Peres–Mermin square is the simplest member. We should point out that the Wheel set is only a KS set for odd N, while for even N it fails to have an odd number of negative IDs. Given the same product PPS as before, now generalized to arbitrary N, we find that the Wheel for even N is also noncolorable because of the 3-qubit Wheels that overlap with it. It is then no surprise that we find a classical projector with an anomalous weak value for Wheels of all N.

The case of Wheels with odd N is a trivial generalization of the 3-qubit case given above. In each case, the magnitude of the weak values is given by $w_N = 1/2^{\frac{N-1}{2}}$. The weak values of all $2^{N-1}$ rank-2 projectors in the classical basis must add up to 1, and therefore there must be $2^{\frac{N-1}{2}}$ more positive weak values than negative ones. This means that the number of negative weak values in the basis is $\nu _N = 2^{N-2} -2^{\frac{N-3}{2}}$. It is interesting that while the magnitude of the anomalous weak values decreases exponentially with N, the cumulative anomaly, which we define as the sum of all negative weak values in the basis, increases with N as $\mathcal {A}_N =\nu _N w_N = 2^{\frac{N-3}{2}} - 1/2$.

The case of Wheels with even N is slightly more complicated. In these cases, applying the observables of the IDs does not allow one to cycle a given eigenstate into every other eigenstate in the basis. Instead, these operations define two distinct orbits of $2^{N-2}$ projectors, with half of the eigenbasis in each. It can be shown that the weak values of all of the projectors in one of the orbits are zero, while the weak values in the other orbit are identical to the (odd) $N-1$ case.

1.6 4-Qubit Wheel

We demonstrate the calculation for the 4-qubit case, from which the generalization to all even N is straightforward. The 4-qubit Wheel can be represented as shown in Fig. 8. Note that unlike the horizontal IDs of Fig. 1, the rows of Fig. 8 are not complete sub-stabilizers, and we can obtain additional observables by taking products of those within each ID. We define a generalized set as a set in which all sub-stabilizers are complete. Completing the sub-stabilizers of Fig. 8 adds three observables to each row, forming two more negative ID columns and one positive ID column. The generalized 4-qubit Wheel set now contains four overlapping 3-qubit Wheels as explicit subsets, and in this way the generalized set proves the KS theorem.

Fig. 8

The 4-qubit Wheel set. Each row is a positive ID4. Each column is a negative ID3 a Pauli Observables and b Labels

The pre-selected state is $|\Psi \rangle = |\lambda _E E,\lambda _F F,\lambda _G G,\lambda _H H\rangle$ and the post-selected state is $|\Phi \rangle = |\lambda _I I,\lambda _J J,\lambda _K K,\lambda _L L\rangle$, such that $\lambda _E\lambda _F\lambda _G\lambda _H = s_{_{EFGH}}=+1$ and $\lambda _I\lambda _J\lambda _K\lambda _L = s_{_{IJKL}}=+1$. Because the 4-qubit Wheel is not a KS set, there is no explicit conflict basis. Instead there is a particular state in the classical basis that seems to be forced (this is a secondary forcing, not an ABL rule forcing),

$$\begin{aligned} |\mathcal {F}|= & {} |-\lambda _E\lambda _I A,-\lambda _F\lambda _J B,-\lambda _G\lambda _K C,-\lambda _H\lambda _L D|\nonumber \\\equiv & {} |\lambda _A A, \lambda _B B,\lambda _C C,\lambda _D D|, \end{aligned}$$

(33)

with $\lambda _A\lambda _B\lambda _C\lambda _D = s_{_{ABCD}}=+1$ (which can be verified by direct substitution of $\lambda _A = -\lambda _E\lambda _I, \ldots$). As above, we will find a relation between the weak values of this projector and several others in its basis,

$$\begin{aligned} |\lambda _A A, \lambda _B B,\lambda _C C,\lambda _D D|_w= & {} \lambda _A\lambda _E \lambda _I s_{_{AEI}}|\lambda _A A, -\lambda _B B,\lambda _C C,-\lambda _D D|_w \nonumber \\= & {} \lambda _B\lambda _F \lambda _J s_{_{BFJ}}|-\lambda _A A, \lambda _B B,-\lambda _C C,\lambda _D D|_w \nonumber \\= & {} -\lambda _A\lambda _B\lambda _E \lambda _I\lambda _F \lambda _J s_{_{AEI}}s_{_{BFJ}}|-\lambda _A A, -\lambda _B B,-\lambda _C C,-\lambda _D D|_w, \end{aligned}$$

(34)

where we have used the explicit values of $c_{QP}$ to obtain the signs of the eigenvalues. Note also that in this case $\lambda _A\lambda _E \lambda _I s_{_{AEI}} = \lambda _B\lambda _F \lambda _J s_{_{BFJ}} = +1$. It is important to note that Eq. (34) was obtained by using the two IDs (A, E, I), (B, F, J) in sequence, which is identical to multiplying their observables together into a new ID (AB, EF, IJ) and applying it. These three IDs then form one of the 3-qubit Wheels within the generalized 4-qubit Wheel set.

Applying (C, G, K) or (D, H, L) reproduces the same orbit of four eigenstates shown above, and so we omit these cases.

What about the other four states in this basis? Let us begin with an arbitrary element that did not belong to the first orbit,

$$\begin{aligned} |\mathcal {G}| = |\lambda _A A, \lambda _B B,-\lambda _C C,-\lambda _D D|. \end{aligned}$$

(35)

Again we find the relation between the weak values of this projector and several others in its basis,

$$\begin{aligned} |\lambda _A A, \lambda _B B,-\lambda _C C,-\lambda _D D|_w= & {} \lambda _A\lambda _E \lambda _I s_{_{AEI}}|\lambda _A A, -\lambda _B B,-\lambda _C C,\lambda _D D|_w \nonumber \\= & {} \lambda _B\lambda _F \lambda _J s_{_{BFJ}}|-\lambda _A A, \lambda _B B,\lambda _C C,-\lambda _D D|_w \nonumber \\= & {} -\lambda _A\lambda _B\lambda _E \lambda _I\lambda _F \lambda _J s_{_{AEI}}s_{_{BFJ}}|-\lambda _A A, -\lambda _B B,\lambda _C C,\lambda _D D|_w, \end{aligned}$$

(36)

This is our second orbit of four eigenstates, obtained using the same method as before.

Before we use these relations to compute weak values, we need to note that some of the eigenstates in this basis are orthogonal to the PPS, but in a way that is not immediately obvious. In particular, we note that $R \equiv AC=BD=ZZZZ$.

To construct rank-2 projectors, one takes a sum of any two orthogonal rank-1 projectors within the space of the rank-2 projector. One way to find two orthogonal rank-1 projectors within the space of the rank-2 projectors in the classical basis is to add any new observable that commutes with, and is independent of, (A, B, C, D). We will choose to add the observable $S = XXXX$, and the rank-2 projector will take the following form,

$$\begin{aligned}&|\lambda _A' A, \lambda _B' B,\lambda _C' C,\lambda _D' D|= |\lambda _A' A, \lambda _B' B,\lambda _C' C,\lambda _D' D, +S|+ |\lambda _A' A, \lambda _B' B,\lambda _C' C,\lambda _D' D, -S|. \end{aligned}$$

(37)

Noting that the sub-stabilizer already contains $R \equiv AC=BD=ZZZZ$, once we add in S the expanded stabilizer now also contains $T \equiv RS = YYYY$ and the positive ID (R, S, T). Omitting all of the identical labels for (A, B, C, D) for compactness, and noting that $\lambda _R' = \lambda _A' \lambda _C'$ we can rewrite Eq. (37) as

$$\begin{aligned} |\lambda _R' R| = |\lambda _R' R, +S, \lambda _R' T| + |\lambda _R' R, -S, -\lambda _R' T|. \end{aligned}$$

(38)

Rewriting the PPS as $|\Psi \rangle = |\lambda _E \lambda _G S\rangle$ and $|\Phi \rangle = |\lambda _I \lambda _K T\rangle$, we can see that the weak values will depend explicitly on the sign of $\lambda _S \equiv \lambda _E \lambda _G$ in the PPS. If $\lambda _S = +1$, then only projectors with $\lambda _R'=\lambda _T \equiv \lambda _I \lambda _K$ will have nonzero weak values, while if $\lambda _S = -1$, then only projectors with $\lambda _R'= - \lambda _T$ will have nonzero weak values. One can then see that the orbit around $|\mathcal {F}|$ is always nonzero because $\lambda _R' = \lambda _A' \lambda _C' = \lambda _S \lambda _T = \lambda _E \lambda _G\lambda _I \lambda _K$, while the orbit around $|\mathcal {G}|$ is always zero because $\lambda _R' = \lambda _A' \lambda _C' = -\lambda _S \lambda _T = - \lambda _E \lambda _G\lambda _I \lambda _K$. The remaining orbit has the same four weak values as the 3-qubit Wheel, and the anomalous weak value always belongs to the projector of Eq. (34), which has all opposite eigenvalues to $|\mathcal {F}|$.

This calculation can be straightforwardly generalized to the case of arbitrary even N to obtain the formulae given above.

This example also reveals an important general restriction on conflict projectors for which the realist truth-value assignment $v_e$ is forced to be 0. In a general set, it may be that the projectors in a conflict basis break up into several distinct orbits (each belonging to a KS subset), and each orbit can have its own magnitude of weak value. If multiple orbits have nonzero weak value, then there is no guarantee that the weak values in each orbit will have a positive sum, and because of this, there is no general maximum weak value for a conflict projector in a basis with multiple nonzero orbits.

For a set and PPS with just one nonzero orbit in the conflict basis, we obtain the rule that the upper bound on the weak value of the conflict projectors is $w_{\max } = 1/2$. To see this, consider that there must be an even number m of projectors in the orbit, all with the same magnitude weak value w, but with potentially arbitrary signs. It is easy to see that the smallest possible positive sum of these m weak values is 2w, and the upper bound then follows from the sum rule. Furthermore, it follows from the overall negative sign of a KS set, that every orbit must also contain at least one negative weak value.

The basis containing this orbit is always an eigenbasis of the conflict ID in the set. There may be other hybrid conflict bases with all 0 realist truth-value assignments $v_e$ composed of projectors from several different eigenbases, as shown in Fig. 3. As a result, these conflict bases may need not contain any of the negative weak values from conflict eigenbasis, but the upper bound of still applies.

Fig. 9

The 6-qubit Arch

1.7 The 6-qubit Arch

We work through one more KS set in order to make sure the general features are apparent. The 6-qubit Arch of Fig. 9 is a remarkably compact KS set that can also be used to show the quantum pigeonhole effect by choosing the PPS from among the two horizontal IDs and the curved ID. We show the brief calculation for the weak values in the case that the conflict occurs in a product basis (which can be converted into the classical basis by local unitary operations).

To begin, we label the observables as in the other examples. Starting from the bottom left, we label the curved ID in the order (A, B, C, D, E). The two horizontal IDs are then labeled from left to right in the order (B, F, G, H) and (A, I, J, K).

We choose as our pre-selection as $|\Psi \rangle = |\lambda _A A, \lambda _B B, \lambda _C C, \lambda _D D, \lambda _E E\rangle$, with $\lambda _A \lambda _B \lambda _C \lambda _D \lambda _E = s_{_{ABCDE}} = -1$, and our post-selection $|\Psi \rangle = | \lambda _B B, \lambda _F F, \lambda _G G, \lambda _H H\rangle$, with with $\lambda _B \lambda _F \lambda _G \lambda _H = s_{_{BFGH}} = +1$.

This forces the states $|f_1\rangle = |\lambda _C C, \lambda _F F, \lambda _C \lambda _F I \rangle$, $|f_2\rangle = |\lambda _D D, \lambda _G G, \lambda _D \lambda _G J \rangle$, and $|f_3\rangle = |\lambda _E E, \lambda _H H, \lambda _E \lambda _H K \rangle$. These three forced states then induce a conflict basis, with the maximum conflict state given by,

$$\begin{aligned} |\mathcal {C}|= & {} |\lambda _A A, -\lambda _C \lambda _F I, -\lambda _D \lambda _G J, -\lambda _E \lambda _H K|\equiv |\lambda _A A, \lambda _I I, \lambda _J J, \lambda _K K|, \end{aligned}$$

(39)

with $\lambda _A \lambda _I \lambda _J \lambda _K = s_{_{AIJK}} = +1$ (which can be verified by direct substitution). Notice also that in this case $\lambda _A$ is fixed by the PPS, and thus the weak value of projectors with $\lambda _A' = -\lambda _A$ is zero. We now proceed to find the relation between the weak values of the other four projectors in the conflict basis,

$$\begin{aligned} |\lambda _A A, \lambda _I I, \lambda _J J, \lambda _K K|_w= & {} \lambda _C \lambda _F \lambda _I s_{_{CFI}}|\lambda _A A, \lambda _I I, -\lambda _J J, -\lambda _K K|_w \nonumber \\= & {} \lambda _D \lambda _G \lambda _J s_{_{DGJ}}|\lambda _A A, -\lambda _I I, \lambda _J J, -\lambda _K K|_w \nonumber \\= & {} \lambda _E \lambda _H \lambda _K s_{_{EHK}} |\lambda _A A, -\lambda _I I, -\lambda _J J, \lambda _K K|_w, \end{aligned}$$

(40)

and note that $\lambda _C \lambda _F \lambda _I s_{_{CFI}} = \lambda _D \lambda _G \lambda _J s_{_{DGJ}} = \lambda _E \lambda _H \lambda _K s_{_{EHK}} = -1$. So once again we find that the maximum conflict state $|\mathcal {C}|$ is the projector with the negative weak value, and in this case all four weak values have magnitude 1 / 2.

Finally, the product of the signs of the all four weak values can be written as,

$$\begin{aligned} \lambda _C \lambda _D \lambda _E\lambda _F \lambda _G \lambda _H \lambda _I \lambda _J \lambda _K s_{_{CFI}}s_{_{DGJ}}s_{_{EHK}}= & {} \lambda _A\lambda _B\lambda _C \lambda _D \lambda _E s_{_{CFI}}s_{_{DGJ}}s_{_{EHK}}s_{_{AIJK}}s_{_{BFGH}} \nonumber \\= & {} s_{_{CFI}}s_{_{DGJ}}s_{_{EHK}}s_{_{AIJK}}s_{_{BFGH}}s_{_{ABCDE}} = - 1, \end{aligned}$$

(41)

showing again that the negative weak value occurs exactly because this is a KS set.

1.8 KS sets and PPSs that do not lead to conflict bases

As we have discussed, not every KS set of PPS leads to a conflict basis.

For example, consider the 3-qubit square case we explored in detail. If we had chosen a row and a column of the square as our PPS, instead of two different rows, there would be no forced values, and thus no conflict basis.

There are also KS sets that do not produce a conflict basis for any PPS, like the 3-qubit GHZ-Mermin Star of Fig. 10a. It is easy to see that the PPS shown in Fig. 10b is symmetric to any other choice of PPS from within this set, and does not force any additional values.

Fig. 10

The 3-qubit GHZ-Mermin Star cannot give rise to a conflict basis a no assignment and b pre- and post-selection

This is not to say that this PPS does not give rise to any conflict bases in the entire 3-qubit Pauli group, only that those contexts do not belong to this KS set. With a fairly simple modification of this KS set, we can obtain a different KS set that does produce a conflict basis for this same PPS. The modified set and the steps leading to the conflict are shown in Fig. 11. Again, we find the conflict in the classical basis, and we have the pigeonhole effect, but in this case our PPS included a maximally entangled GHZ state, so this is not the product pigeonhole effect.

Fig. 11

This modified version of the GHZ-Mermin star does give rise to a conflict basis for the same PPS as in Fig. 10 a no assignment, b pre- and post-selection and c the ABL rule and violation of the product rule (shown by the blue dashed line) (color figure online)

This conflict basis was first obtained using this PPS by Tollaksen [7].

1.9 The N-qubit quantum pigeonhole effect without a KS set

Throughout this paper we have focussed on the connection between the quantum pigeonhole effect and KS sets, but the KS theorem is not the only demonstration of quantum contextuality, and likewise it is not the only demonstration of the quantum pigeonhole principle.

In this section we give an $N+1$-qubit (N even) proof of quantum contextuality that uses product states for the PPS, and forces a conflict in the classical basis (i.e. the same trick we can play with the N-qubit Wheels), but does not depend on any complete KS set that we can identify. We were made aware of this proof by Yakir Aharonov, and what follows is truly only a reproduction and slight generalization of his ideas.

For this proof, we consider the positive ID of Fig. 12, which has the classical basis as its eigenbasis.

Fig. 12

An $(N+1)$-qubit positive ID with $N+1$ observables a $N=4$ and b any even N

We will choose product states as our pre- and post-selected states, and thus we can consider each qubit individually, and then multiply the results together. To begin we consider the case that each qubit is pre-selected in the state $|\Psi \rangle = |+X\rangle$, and post-selected in the state $|\Phi \rangle = \cos {(n\pi /2N)}|0\rangle - i \sin {(n\pi /2N)}|1\rangle$, where n is any odd integer and $\{|0\rangle \equiv |+Z\rangle ,|1\rangle \equiv |-Z\rangle \}$ is the computational basis. The weak value of each single-qubit Pauli Z observable is then $Z_w = \mathrm{e}^{in\pi /N}$. It then follows that the weak value of each $(N+1)$-qubit observable in Fig. 12 is −1. Using the reverse ABL rule to force these values, we find a conflict in the classical basis, since Fig. 12 is a positive ID.

We can also show that these cases always give rise to projectors with negative real parts for their weak values, which is another direct verification of contextuality. Specifically we consider the maximum conflict projector, $|\mathcal {C}| \equiv |\mathcal {C}\rangle \langle \mathcal {C}|$, that maximally opposes the PPS by having eigenvalue +1 for all $N+1$ observables. This is actually a rank-2 projector, and therefore we first decompose it into a sum of orthogonal rank-1 projector as follows,

$$\begin{aligned} |\mathcal {C}| = |+Z_1, +Z_2, \ldots , +Z_{N+1}| + |-Z_1, -Z_2, \ldots , -Z_{N+1}|. \end{aligned}$$

(42)

Each term in this sum can be factored into the tensor product of $|{\pm }Z|$ for each individual qubit, and thus the weak value of the projector is,

$$\begin{aligned} |\mathcal {C}|_w = \mathrm{e}^{ -i n\pi (N+1) /2N} \left[ \left( \cos {\frac{n\pi }{2N}}\right) ^{N+1} + \left( i \sin {\frac{n\pi }{2N}}\right) ^{N+1}\right] , \end{aligned}$$

(43)

which has a negative real part that approaches zero as $N\rightarrow \infty$. It is also noteworthy that the weak values of the classical projectors generally do not have the same magnitudes with this PPS, as they do with KS sets. Thus, the uniform ABL probabilities for the projectors in a conflict basis appears to be a feature that is unique to KS sets—perhaps due to their additional symmetry properties.

To extend this proof to full generality, we will now consider any pre-selected state in the XZ plane, $|\Psi \rangle = \cos {(\theta /2)}|0\rangle + \sin {(\theta /2)}|1\rangle$, which combined with the (unnormalized) post-selected state $|\Phi \rangle = \sin {\frac{\theta }{2}}\cos {\frac{n\pi }{2N}}|0\rangle - i \cos {\frac{\theta }{2}}\sin {\frac{n\pi }{2N}}|1\rangle$ (which is in the YZ plane), we again obtain $Z_w = \mathrm{e}^{in\pi /N}$, which leads to the conflict just as before.

The proof we have shown here fails for odd N, but just as with the Wheels, the proof can be recovered by considering all of the different $(N-1)$-qubit IDs like Fig. 12 that fall within the classical stabilizer group.

As a final generalization, note that each observable of Fig. 12 is composed of N Pauli Z observables and one single-qubit identity (I) operator. There exists a much more general set of IDs from within the classical stabilizer group for which the above PPSs will work, and these are IDs composed of N Pauli-Z observables for any even N, and m single-qubit identity (I) operators, for any odd m. From symmetry, these positive IDs must contain an odd number of observables, and thus the PPS (for all $N+m$ qubits) forces a conflict in the classical basis.