Abstract
Quantum games represent the really twenty-first century branch of game theory, tightly linked to the modern development of quantum computing and quantum technologies. The main accent in these developments so far was made on stationary or repeated games. In this paper, we aim at initiating the truly dynamic theory with strategies chosen by players in real time. Since direct continuous observations are known to destroy quantum evolutions (so-called quantum Zeno paradox), the necessary new ingredient for quantum dynamic games must be the theory of non-direct observations and the corresponding quantum filtering. Apart from the technical problems in organizing feedback quantum control in real time, the difficulty in applying this theory for obtaining mathematically amenable control systems is due partially to the fact that it leads usually to rather non-trivial jump-type Markov processes and/or degenerate diffusions on manifolds, for which the corresponding control is very difficult to handle. The starting point for the present research is the remarkable discovery (quite unexpected, at least to the author) that there exists a very natural class of homodyne detections such that the diffusion processes on projective spaces resulting by filtering under such arrangements coincide exactly with the standard Brownian motions (BM) on these spaces. In some cases, one can even reduce the process to the plain BM on Euclidean spaces or tori. The theory of such motions is well studied making it possible to develop a tractable theory of related control and games, which can be at the same time practically implemented on quantum optical devices.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Quantum games represent the really twenty-first century branch of game theory, tightly linked to the modern development of quantum computing and quantum technologies. Initiated by Meyer [33], Eisert, Wilkens and Lewenstein [15], and Marinatto and Weber [31], the theory now boasts of many beautiful results obtained by various authors in numerous publications, see, e.g., surveys [17, 20], and a mathematically oriented survey in textbook [28]. However, the main accent in these developments was made on stationary or repeated games. In this paper, we aim at initiating the truly dynamic theory with strategies chosen by players in real time. Since direct continuous observations are known to destroy quantum evolutions (so-called quantum Zeno paradox), the necessary new ingredient for quantum dynamic games must be the theory of non-direct observations and the corresponding quantum filtering. This theory was essentially developed by Belavkin in the 1980s of the last century, in [5,6,7], see [11] for a readable modern account. There is an important work under way on the technical side of organizing feedback quantum control in real time, see, e.g., [2, 12] and [36]. The difficulty in applying this theory for obtaining mathematically amenable control systems is due partially to the fact that it leads usually to rather non-trivial jump-type Markov processes and/or degenerate diffusions on manifolds, for which the corresponding control (an even more so games) is very difficult to handle.
The starting point for the present research was the remarkable discovery (quite unexpected, at least to the author) that there exists a very natural class of homodyne detections such that the diffusion processes on spheres or projective spaces resulting by filtering under such arrangements coincide exactly with the standard Brownian motions (BM) on these Riemannian manifolds, that is, the processes generated by the invariant Laplace–Beltrami operator. For qubits the basic example of such special arrangements is the choice of the three Pauli matrices as the coupling operators governing the interaction with the optical measuring devices. For qudits, the corresponding matrices can be chosen as the generalized Pauli or Gell-Mann matrices. Another unexpected feature of these special arrangements is that the corresponding diffusions written with respect to the output process coincide exactly with the diffusions written with respect to the so-called innovation process that plays the key role in the theory of quantum feedback control. The theory of the BM on compact Riemannian manifolds is well studied in stochastic analysis and operator theory on manifolds, making it possible to develop a tractable theory of related control and games, which can be at the same time practically implemented on quantum optical devices. This theory is based on the ability to build classical or mild solutions to the corresponding Hamilton–Jacobi–Bellman–Isaacs (HJB-Isaacs) equations on compact Riemannian manifolds, which makes it more elementary than the approach to stochastic control based on the viscosity solutions, for which we refer to [38] and references therein. Moreover, in some cases (essentially when all controlled Hamiltonian operators commute), the filtered dynamics turns out to be governed by the standard Brownian motion on Euclidean spaces and tori, that is by the diffusion processes generated by the standard plain Laplacian in \(\mathbf {R}^d\) or a torus.
The content of the paper is as follows. In the next section, we briefly explain the necessary tools from the theory of continuous quantum measurement and filtering. In the following two sections, we introduce our main homodyne detection schemes (first for qubits and then for qudits) that allow one to turn the problems of dynamic quantum filtering, control and games into the problems of the drift control of the standard Brownian motions on the complex projective spaces. In Sects. 5 and 6, we build the theory of classical and mild solutions of the HJB-Isaacs equations on Riemannian manifolds leading to the theory of dynamic control and games on compact Riemannian manifolds and thus automatically to the theory of quantum dynamic control and games under the special homodyne detection schemes. In Sect. 7, we introduce yet another homodyne detection scheme that leads to the simpler drift controls on tori, which works, however, only in case when all controlled Hamiltonian operators commute. In Sect. 8, an exactly solvable model is presented, which can be considered as a kind of dynamic extension of the initial quantum coin flipping game of Meyer. In Sect. 9, yet another detection scheme is developed that turns the problem of quantum dynamic control to the drift control of the standard BM in Euclidean spaces. In Sects. 10 and 11, a version of the theory is developed for players acting on different atoms, thus for the dynamic games set in the spirit of papers [15] and [31]. Some conclusions and perspectives are drawn and open questions posed in Sect. 12.
2 Prerequisites: Non-demolition Observation and Quantum Filtering
The general theory of quantum non-demolition observation, filtering and resulting feedback control was built essentially in papers [5,6,7]. A very good readable introduction is given in [11]. We shall describe briefly the main result of this theory.
The non-demolition measurement of quantum systems can be organized in two versions: photon counting and homodyne detection. One of the first mathematical results on the control with photon counting measurement was given in [22], which can be used to develop the corresponding game theoretical version, see [26]. But here we fully concentrate on the homodyne (mathematically speaking, diffusive type) detection. Under this type of measurement, the output process \(Y_t\) is a usual Brownian motion (under appropriate probability distribution). There are several (by now standard) ways of writing down the quantum filtering equation for states resulting from the outcome of such process. The one which is the most convenient to our purposes is the following linear Belavkin filtering equation (which is a particular version of the stochastic Schrödinger equation) describing the a posteriori (pure but not normalized) state:
where the unknown vector \(\chi \) is from the Hilbert space of the observed quantum system, which we shall loosely referred to everywhere as the atom, the self-adjoint operator H is the Hamiltonian of the corresponding initial (non-observed) quantum evolution, and the operator L is the coupling operator of the atom to the optical measurement device specifying the chosen version of the homodyne detection. Very often the operator L is chosen to be self-adjoint, in which case Eq. (1) reduces to the simpler form:
An important part in the theory is played by the so-called innovation process
where for an operator A and a vector v in a Hilbert space we use the (more or less standard) notation for the average value of A in v:
The innovation process is in some sense a more natural driving noise to deal with, because it turns out to be the standard Brownian motion (or the Wiener process) with respect to the fixed (initial vacuum) state of the homodyne detector, while the output process \(Y_t\) is a Brownian motion with respect to the states transformed by the (quite complicated) interaction of the quantum system and optical device, which can also be obtained by the Girsanov transformation from the innovation process \(B_t\). Therefore, another well-used version of Eq. (1) is the nonlinear equation on the normalized vector \(\phi =\chi /|\chi |\), which can be obtained directly from (1) by the classical Ito formula (using the classical Ito rule for the differentials of the Wiener processes, \(dY_t dY_t=dt\)), but written in terms of the innovation process \(B_t\).
The theory extends naturally to the case of several, say N, coupling operators \(\{L_j\}\), where the quantum filtering is described by the following direct extension of Eq. (1):
with the N-dimensional output process \(Y_t=\{Y^j_t\}\). The corresponding innovation process is the standard N-dimensional Wiener process with the coordinate differentials
The theory of quantum filtering reduces the analysis of quantum dynamic control and games to the controlled version of evolutions (4). The simplest situation concerns the case when the homodyne device is fixed, that is, the operators \(L_j\) are fixed, and the players can control the Hamiltonian H, say, by applying appropriate electric or magnetic fields to the atom. Thus, Eq. (4) becomes modified by allowing H to depend on one or several control parameters. One can even prove a rigorous mathematical result, the so-called separation principle (see [10]), that shows that the effective control of an observed quantum system (that can be based in principle on the whole history of the interaction of the atom and optical devices) can be reduced to the Markovian feedback control of the quantum filtering equation, with the feedback at each moment depending only on the current (filtered) state of the atom.
Remark 1
The filtering Eq. (1) was initially derived from the interaction of the atom and optic devices described by the unitary evolution solving the quantum stochastic equation
where \(A_t, A_t^*\) are the Hudson–Parthasarathy differentials of the quantum stochastic Wiener noise (built from the annihilation and creation operators). It can be shown (see, e.g., [1]) that this evolution represents the Markovian approximation to the more realistic quantum dynamics
driven by a stationary Gaussian wide-band noise of the annihilation operators a(t, r). More elementary derivations of the main filtering equation (bypassing heavy use of quantum stochastic calculus) are also available. It can be obtained from an appropriate limit of sequential discrete observation scheme, see, e.g., [8, 34] or [27]. A derivation from the theory of instruments was given in [3] and [19].
Remark 2
In this paper, the players are supposed to have the same information based on the whole (filtered) quantum state (even if they have access to different controlling tools). The case when they make their decision on individual states can be most naturally achieved by allowing them to have access to their partial traces (natural quantum analogues of classical individual states). Such modeling is developed by the author in separate papers in the framework of quantum mean field games, see [26] and [25].
3 Special Homodyne Detection Leading to the Laplace–Beltrami Operator on a Sphere (for Qubits)
For a qubit the Hilbert space of an atom is \(\mathbf {C}^2\). Since the pure state of a quantum system is specified by a vector in the Hilbert space up to a multiplier, the actual state space is the one-dimensional complex projective space or a two-dimensional sphere, often referred to as the Bloch sphere. Hence, as the natural coordinate outside the state vector (0, 1), one can take the complex number \(w=\chi _1/\chi _0\). It is straightforward to rewrite evolution (4) in \(\mathbf {C}^2\) in terms of w. Namely, from the equation for the first coordinate \(\chi _0\) (and Ito’s rule for the function 1/x), we find the equation for \(\chi _0^{-1}\):
and then using the Ito product rule for the product \(\chi _1\chi _0^{-1}\), we find that
and finally, the quantum filtering equation in terms of the projective coordinates w:
where, for convenience, we have introduced the vector \(W=(1,w)=\chi /\chi _0\). Equivalently, it can be rewritten in terms of the innovation processes expressed in terms of w as
Remark 3
Though this is not of use for us, let us mention that coordinates w can be obtained by the stereographic projection from the Stokes parameters \((x_1,x_2,x_3)\) (or a polar vector) describing in the most common way the Bloch sphere of the pure quantum states of a qubit.
We are interested in choosing \(\{L_j\}\) in a way to make the diffusion on the Bloch sphere defined by Eq. (5) as simple as possible, at least to make it non-degenerate, that is, with the second-order part of the diffusion operator being elliptic.
The Hamiltonian operator does not enter the noise term, and consequently, it does not play role in this question. If L consists of just one operator, the diffusion is definitely degenerate, though it may be hypoelliptic (see [23]). If there are two operators \(L_j\), one usually gets diffusions that are elliptic almost everywhere (see Sects. 9 and 12). Turning to the case of three operators \(L_j\), it is natural to try the simplest three operators on qubits, namely the three Pauli operators
Since \(\sigma _j\) are self-adjoint and \(\sigma _j^2=\mathbf {1}\), the second term in equation in this case is seen directly to vanish. Moreover, explicit calculation of the third term shows that it vanishes as well, so that the filtering Eq. (5) simplifies to
where \(h_{jk}\) denote the entries of the \(2 \times 2\)-matrix H.
With this equation two remarkable effects occur.
Proposition 3.1
(i) Writing Eq. (7) in terms of the innovation process \(dB^j=dY^j-2\langle \sigma _j \rangle _W \, dt\), it takes exactly the same form (7) with \(B^j\) instead of \(Y^j\) (all new terms with the differential dt cancel).
(ii) The diffusion operator D corresponding to Eq. (7) with vanishing H takes the form
in terms of the real coordinates x, y, where \(w=x+iy\), so that \(D=2\Delta _{sp}\), where \(\Delta _{sp}\) is the Laplace–Beltrami operator on the two-dimensional sphere written in stereographic coordinates.
Proof
This is done by direct inspection. For instance, to prove (ii), we can write Eq. (7) with vanishing H in terms of the real and imaginary parts of w as
By Ito’s formula, the corresponding second-order operator is found to be
as was claimed. \(\square \)
Remark 4
Thus, Eq. (7) gives a method to express the curvilinear two-dimensional Brownian motion on a sphere in terms of the three-dimensional standard (plain) Brownian motion.
It is natural to ask what is the general class of the triples of operators \(L_1,L_2, L_3\), where the same effects hold.
Reducing the attention to the case of self-adjoint matrices \(L_j\) let us write them as
with \(l^{00}_j, l^{11}_j \in \mathbf {R}\), \(l^{01}_j\in \mathbf {C}\). Let us introduce the three-dimensional real vectors \(L^0, L^1, L^{\delta }, L_j^R, L^I\) defined by their coordinates
Proposition 3.2
(i) The second-order part of the diffusion operator arising from the stochastic Eq. (5) is isothermic, that is, it has the form
with some positive function \(\omega (x,y)\) if and only if the vectors \(L^{\delta }, L_j^R, L^I\) form an orthonormal basis in \(\mathbf {R}^3\), up to a common constant multiplier. If this is the case, then this operator actually coincides with the Laplace–Beltrami operator (8) (again of course up to a constant multiplier).
(ii) The whole diffusion operator arising from the stochastic Eq. (5) with vanishing H is isothermic (that is, additionally to (i), all the first-order terms cancel as in the case of the Pauli matrices) if and only if the vectors \(L^{\delta }, L_j^R, L^I\) form an orthonormal basis in \(\mathbf {R}^3\) (up to a common constant multiplier) and \(L^0=-L^1\). Moreover, under these conditions, the diffusion operator of stochastic equation (5) coincides with the diffusion operator arising from Eq. (5) rewritten in terms of the innovation process.
Proof
This is done by lengthy explicit calculations, which we omit. \(\square \)
Since the transpose of an orthogonal matrix (in our case the matrix with the columns built from the vectors \(L^0, L_j^R, L^I\)) is also orthogonal, Proposition 3.2 can be formulated in the following more transparent way.
Proposition 3.3
The diffusion operator arising from Eq. (5) with the 3 self-adjoint matrices \(L_j\) coincides with the Laplacian on a sphere (up to a multiplier), if and only if three matrices \(L_j\) form a basis in the space of traceless self-adjoint matrices, which is orthogonal in the sense that
with a constant a. In the case of the Pauli matrices \(a=2\). The exact Laplacian arises from \(a=1\).
4 Special Homodyne Detections Leading to the Laplace–Beltrami Operator on Projective Spaces (for Qudits)
In this section, we extend the previous results to quantum systems in \(\mathbf {C}^{n+1}\) with arbitrary n (a qudit with \(d=n+1\)).
As in the case of qubit, let us start by writing the corresponding filtering Eq. (4) in terms of the vector \(W=(1,w_1,\cdots , w_n)=\chi /\chi _0\), that is, in the projective coordinates \(w_1,\cdots , w_n\). We have
Hence, by the Ito formula
Consequently, by the Ito product rule, we find for \(k>0\) that
and thus, the quantum filtering equation for qudits (with \(d=n+1\)) in terms of the projective coordinate W:
To reduce complexity, let us discuss in more detail the case of a three-dimensional Hilbert space (a qutrit) of the vectors \(\chi =(\chi _0,\chi _1, \chi _2)\) (the general case being quite similar). Extending the case of qubit, it is natural to choose \(L_j\) to be the 8 generalized Pauli or Gell-Mann matrices: 3 symmetric \(\sigma ^s_{jk}\) (with 1 on places jk and kj and zero otherwise ), \(0\le j<k\le 2\), 3 antisymmetric \(\sigma ^a_{jk}\) (with \(-i\) on the place jk and i on the place kj, and zero otherwise), \(0\le j<k\le 2\), and 2 diagonal matrices \(\sigma ^d_k\):
Thus,
In arbitrary dimension, it is more convenient to work directly in complex coordinates \(w_k, \bar{w}_k\) (rather than playing with their real and imaginary parts). Again direct substitution of the above Gell-Mann matrices into Eq. (10) (we omit the lengthy by direct calculations) shows the following analog of Proposition 3.1.
Proposition 4.1
Equation (10) with \(L_j\) chosen as the 8 Gell-Mann matrices and written for vanishing H takes the form
(with all terms with dt vanishing), and exactly the same form has this equation when rewritten in terms of the innovation process, which is now an eight-dimensional standard Wiener process with the coordinates
Finally, the diffusion operator D arising from the stochastic differential Eq. (11) has the form \(D=2 \Delta _{pro}\), where \( \Delta _{pro}\) is the major (second order) part of the Laplace–Beltrami operator on the complex projective space \(P\mathbf {C}^2\):
Of course, there exists a characterization of all collections of \(L_j\) with the same property, analogous to Proposition 3.3.
For a quantum system in \(\mathbf {C}^{n+1}\), there are \((n^2+2n)\) generalized Pauli matrices. Choosing these matrices as the coupling operators in a homodyne detection scheme will lead analogously to the invariant BM on the complex projective space \(P\mathbf {C}^n\).
5 Theory of Drift Control and Games on Riemannian Manifolds
Now we shall develop the theory of the classical or mild solutions to the Hamilton–Jacobi–Bellman–Isaacs equations arising in the stochastic control and differential games on compact Riemannian manifolds with a controlled drift and the fixed underlying Markov process being the standard Brownian motion on M. In the next section, it will be used to build the theory of dynamic quantum games that can be reduced to such stochastic games under the special arrangement homodyne detection as shown in previous sections. Let
denote the Laplace–Beltrami operator on a compact Riemannian manifold (M, g) of dimension N, with the Riemannian metric given by the matrix \(g=(g_{jk})\) and its inverse matrix \(G=(g^{jk})\). Let K(t, x, y) be the corresponding heat kernel, that is, K(t, x, y) is the solution of the corresponding heat equation \((\partial K/\partial t)=\Delta _{LB} K\) as a function of \((t>0,x\in M)\) and has the Dirac initial condition \(K(0,x,y)=\delta _y(x)\). It is well known that the Cauchy problem for this heat equation is well posed in M and the resolving operators
where dv(y) is the Riemannian volume on M, form a strongly continuous semigroup of contractions (the Markovian semigroup of the Brownian motion in M) in the space C(M) of bounded continuous functions on M, equipped with the sup-norm. Let \(C^1(M)\) denote the space of continuously differentiable functions on M equipped with the norm
where in local coordinates
The key properties of this semigroup needed for our theory are the following smoothing and smoothness preservation properties.
Proposition 5.1
(i) The operators \(S_t\) are smoothing:
with a constant C, uniformly for any compact interval of time.
(ii) The operators \(S_t\) are smoothness preserving:
with a constant C, uniformly for any compact interval of time.
Remark 5
This result is possibly known, but the author did not find any precise reference. It is standard for diffusions in \(\mathbf {R}^d\), but seemingly not so standard for manifolds. We sketch a proof briefly. An alternative proof of (ii) (by-passing estimates from (i)) and its extension to higher derivatives can be built on the theory of SDEs on (M, g).
Proof
(i) This is a consequence of the well-known estimate for the derivatives of the heat kernel on a compact Riemannian manifold (see Theorem 6 in [13]):
with any \(\delta >0\) and a constant \(C(\delta ,N)\), where the derivative \(\nabla \) is taken with respect to x, and where d is the Riemannian distance in M. In fact, differentiating (14) and using (17) yield (15).
(ii) This follows from (17) and the method of parametrix (frozen coefficients) approximation. This method (see, e.g., formula (5.60) in [24]) starts by representing K in terms of its asymptotics \(K_{as}\) and the integral correction as
where F is the error term in the equation for \(K_{as}\), that is
From (17), it follows that the derivative of the second term in (18) is bounded and thus the estimate for the derivative reduces to the derivatives arising from \(K_{as}\), and these estimates are standard and are performed as in the case of heat equations in \(\mathbf {R}^d\). \(\square \)
For the stochastic control of diffusions on (M, g) with the second-order part being fixed as \(\Delta _{LB}\), and where control is carried out via the drift only, the corresponding HJB equation is the equation
where the Hamiltonian function is of the form
where U is the set of possible controls and \(\psi ,J\) are some continuous functions. In case of zero-sum stochastic two-player games with the so-called Isaac’s condition, the Hamiltonian function takes the form
The possibility to exchange \(\sup \) and \(\inf \) here is called Isaac’s condition. It is fulfilled, in particular, when the control of two players can be separated in the sense that the Hamiltonian becomes
It is worth recalling here that though the theory of HJB is often built (for simplicity) for the Cauchy problem of Eq. (19) in forward time, in the control theory it appears more naturally as the backward Cauchy problem for the equation
with a given terminal condition \(f_T\) at some time T. This way of writing the HJB equation becomes unavoidable whenever any of the parameters of the problem are explicitly time dependent.
Let us now consider the general Hamilton–Jacobi–Bellman–Isaacs Eq. (19) with H being a Lipschitz continuous function of its two variables. It is well known (and easy to see) that if f is a classical solution of (19) with the initial condition Y, then f solves also the following integral equation
referred to as the mild form of (19). Solutions to the mild Eq. (24) (which may not solve (19), because of the lack of sufficient smoothness) are often referred to as mild solutions to (19).
The following result gives the well-posedness of the HJB-Isaacs Eq. (19) with explicit estimates for the growth of solutions and their continuous dependence on initial data.
Theorem 5.1
Let H(x, p) be a continuous function on the cotangent bundle \(T^*M\) to the compact Riemannian manifold (M, g) such that
with a constant \(L_H\). Then, for any \(Y\in C^1(M)\) there exists a unique solution \(f_. \in C([0,T], C^1(M))\) of Eq. (24). Moreover, for all \(t\le T\),
where \(h=\sup _x|H(x,0)|\) and C is from Proposition 5.1. And the solutions \(f_t(Y_1)\) and \(f_t(Y_2)\) with different initial data \(Y_1,Y_2\) enjoy the estimate
where \(E_{1/2}\) denotes the Mittag–Leffler function.
The proof of the theorem is identical to the corresponding proof given for the equations in \(\mathbf {R}^d\) in [24] (Section 6.1), and it follows essentially from the fixed point argument and Proposition 5.1.
Assuming additionally that H is Lipschitz continuous in the first argument so that
one can improve the result of Theorem 5.1 by showing (in exactly the same way as for \(\mathbf {R}^d\), see again [24]) that with the initial condition Y from \(C^2(M)\) the solution \(f_t\) will belong to \(C^2(M)\) for all t and hence, will be a (unique) classical solution to the Cauchy problem of the HJB-Isaacs Eq. (19).
Finally, the standard result of the stochastic control theory (called the verification theorem, see, e.g., [16]) states that a classical solution to the HJB-Isaacs equation yields in fact the optimal cost for the corresponding stochastic control problem or the minimax solution in case of zero-sum games.
6 Toward the Theory of Dynamic Games Under the Special Homodyne Detections
Applying the results of the previous section in conjunction with the detection schemes of Sects. 3, 4 leads automatically to the theory of dynamic quantum games under these detection schemes.
In fact, assume that the Hamiltonian operator H decomposes into the three parts \(H=H_0+uH_1+vH_2\), where \(H_0\) is the Hamiltonian operator of the “free” motion of an atom and the strength u and v of the application of the operators \(H_1\) and \(H_2\) can be chosen strategically by the two players I and II, respectively. To simplify the formulas let us assume that allowed values of the control parameters lie in certain symmetric intervals: \(u\in [-U,U]\), \(v\in [-V,V]\) with some constants \(U,V\ge 0\), The case of a pure control (not a game) corresponds to the choice \(V=0\) and is thus automatically included. Assume that players I and II play a standard dynamic zero-sum game with a finite time horizon T meaning that the objective of I is to maximize the payoff
where J and F are some operators expressing the current and the terminal costs of the game (they may depend on u and v, but we exclude this case just for simplicity) and \(\mathbf {E}\) denotes the expectation with respect to the random trajectories W(s) arising from dynamic (10) under the strategic choice of the controls u and v by the players.
The Isaacs condition (21) is fulfilled under our assumptions. Assuming \(\{L_j\}\) are chosen with our special detection scheme such that the corresponding diffusion operator with vanishing H coincides with the second-order part of the Laplace–Beltrami operator on the complex projective space, the HJB-Isaacs Eq. (23) for the minimax value
of the game takes the form
where \(\alpha \) includes the contributions arising from \(H_0\) and from the first-order terms of the Laplace–Beltrami operator (if any). Equivalently, it can be rewritten as
In our setting, the Hamiltonian depends quadratically on w. It follows that the Lipschitz continuity (28) holds locally. But the whole compact manifold can be covered by a finite number of local charts implying the global Lipschitz continuity, which is required to conclude, according to the results of the previous section, that the backward Cauchy problem for Eq. (32) with the terminal condition \(\langle F\rangle _{W}\) has the unique classical solution that yields the minimax value of our zero-sum game.
Remark 6
Here and in the following examples, the controls u and v are not penalized in the cost. This covers the special case of cheap control. This assumption is made for simplicity. The choice of linear control also makes sense physically (see, e.g., [10] and [9]). In fact, operators contributed to the Hamiltonian stand for various magnetic or electric fields that can be applied, and choosing the linear dependence on the control parameters means the ability of the controlling agents to choose the strength of the fields inside given bounds. Including quadratic penalty costs would not make much difference for our analysis.
7 Special Homodyne Detections Leading to Standard BM on Tori
An alternative approach to simplify (10) is to choose \(L_j\) in a way allowing for invariant manifolds and thus reducing the complexity (the dimension) of resulting controlled process. Suppose we choose the coordinates \(\chi \), where H is diagonal, that is \(h_{km}=0\) for \(k\ne m\). It turns out (which seen by direct inspection) that if one chooses n operators \(L_j\), \(j=1,\cdots , n\), as anti-Hermitian diagonal operators with only one non-vanishing elements, namely with the entries
with some real numbers \(r_j\), then (10) decomposes into the system of uncoupled equations
from which it follows that \(d(w_k\bar{w}_k)=0\) for all k, and thus, the whole process lives on the n-dimensional torus \(T_n^C=\{(w_1, \cdots , w_n):|w_k|=C_k\}\), where the vector \(C=\{c_m\}\) is specified by the initial condition.
Moreover, since all \(L_j\) are anti-Hermitian, the innovation process \(B_t=(B^k_t)\) coincides with the output process \(Y_t\), both forming the standard Brownian motion.
Remark 7
Therefore, in case of a qubit, only one operator L (with an element ir in the right low corner and zeros otherwise) is sufficient to get \(d(w\bar{w})=0\). It is not difficult to see that for a qubit any operator leading to this effect has the form \(L+\alpha \mathbf {1}\) with a constant \(\alpha \in \mathbf {C}\).
Writing \(w_k=|w_k|\exp \{i\phi _k\}\) we can rewrite (33) (using Ito’s formula, of course) in terms of the angles as follows:
which is the BM on the torus \(T_n^C\) with a constant drift.
This is of course simpler, than the invariant BM on the projective space \(P\mathbf {C}^n\) obtained by choosing \((n^2+2n)\) generalized Pauli operators. However, from the point of the application to control and games, this homodyne detection can be used only in case when under any choice of control parameters \(u\in U\) the resulting family of possible Hamiltonians H(u) is a commuting family. Only in this case we can choose a basis where all H(u) are diagonal and thus treat them all with a single choice of anti-Hermitian operators \(L_j\) above. The general theory of games is then exactly the same as in Sect. 6, with the projective spaces substituted by the tori. An exactly solvable example of such case will be given below.
8 Example of Exactly Solvable Model
In the case of a qubit, Eq. (34) reduces to the equation
with real constants \(h_0, h_1,r\) and the standard one-dimensional BM \(B_t\), thus describing the BM on a circle with a drift.
Let us now choose \(r=1\) and assume that two players I and II can control the strength and direction of the field yielding the first and the second entries \(h_0\) and \(h_1\), respectively. Assume further that the goal of player I is to maximize the integral cost
with \(W(t)=C e^{i\phi (t)}\) and some Hermitian operators \(J=(J_{jk})\) and \(F=(F_{jk})\), so that
Ignoring irrelevant constants the current cost function \(\langle J\rangle _{W}\) rewrites as \((a \cos \phi +b\sin \phi )\), or, by shifting \(\phi \), even simpler as just \(\cos \phi \). Hence, the corresponding HJB-Isaacs equation for this game becomes
Assuming as above that u and v can be chosen from some intervals, this equation is rewritten as
with a real constant \(\alpha \).
Remark 8
Player I has an advantage if \(\alpha >0\).
Instead of the fixed time horizon problem, let us consider the corresponding stationary problem, when one looks for the average payoff per unit time for a long lasting game. Analytically this means searching for a solution to Eq. (36) with S linearly dependent on t: \(S(t,\phi )=\lambda (T-t)+S(\phi )\) with some function \(S(\phi )\). Here, \(\lambda \) is the average payoff per unit time and \(S(\phi )\) solves the stationary HJB equation
From the symmetry of the problem, it is clear that \(S(\phi )\) is an even periodic function of \(\phi \), and thus, it has the boundary conditions \(S'(0)=S'(\pi )=0\). Moreover, S is decreasing in \(\phi \) on the interval \(\phi \in [0,\pi ]\) (since \(\cos \phi \) has maximum at \(\phi =0\)), so that on this interval Eq. (37) turns to the equation
Hence, for the function \(f=S'\) we get the problem
The stationary solution S itself is defined up to a constant, as it should be, since what one is looking for is actually the average cost \(\lambda \).
The solution to Eq. in (39) with the initial condition \(f(0)=0\) is
Hence, from \(f(\pi )=0\) one finds
Similarly, one can solve the infinite horizon problem with discount, for which one searches for S of the form \(S(t,\phi ) =e^{-t\delta }S(\phi )\), with a fixed discount factor \(\delta >0\), and the stationary HJB equation writes down as
The same monotonicity and evenness conditions as above lead now to the problem
on the interval \([0,\pi ]\). This linear problem is easy to solve. First one finds the general solution of the equation in the form
with
and then, the boundary condition gives the linear system
from which A and B are found.
9 Special Homodyne Detections Leading to the Standard BM on Euclidean Spaces
As the third model of special homodyne detections, we introduce the arrangements, under which the resulting filtering process is the standard BM in \(\mathbf {R}^n\), that is a process govern by the plain Laplacian, though perturbed by a drift with unbounded coefficients. We need here 2n operators \(L_j\) for a quantum system in \(\mathbf {C}^{n+1}\) (unlike \((n^2+2n)\) Pauli matrices leading to the invariant BM and n matrices leading to the tori). We shall take n operators \(L_{j1}=(l_{j1}^{kl})\) and n operators \(L_{j2}=(l_{j2}^{kl})\) \(j=1, \cdots , n\), such that the only non-vanishing entries of their matrices are the elements of the first column \(l_{j1}^{k0}\) and \(l_{j2}^{k0}\) with \(k=1, \cdots , n\). More concretely, let
Under this choice \((L_{j1}\chi )_0=(L_{j2}\chi )_0=0\) and
for all j, and Eq. (10) takes the form
The corresponding diffusion operator for vanishing H gets the form
in the real coordinates \(x_k,y_k\) such that \(w_k=x_k+iy_k\), that is, it defines a Gaussian (Ornstein–Uhlenbeck) diffusion in \(\mathbf {R}^{2n}\) and its major second-order part is just the standard Laplacian in \(\mathbf {R}^{2n}\).
From the first sight, this third homodyne arrangement seems to be the simplest one. However, the catch is that, unlike the cases of compact spaces above (projective spaces and tori), where all coefficients are automatically bounded, here the Hamiltonian controlled part has unbounded drift (generally speaking of quadratic growth), which complicates the investigation of the corresponding HJB equations in \(\mathbf {R}^{2n}\). In the case of a commuting set of controlled Hamiltonians, when all matrices H have only diagonal coefficients, the quadratic term disappears and the controlled drift coefficient grows linearly, which makes it amenable to analysis, see, e.g., [18]. In this paper, we avoid dealing systematically with unbounded coefficients and will not develop a theory of control in this case.
Remark 9
Unlike the previous cases with the projective spaces and tori, in the scheme leading to (46) the diffusion operator written in terms of the innovation processes will be different from (46) leading to another complication of the theory, and even to possibly two different formulations of the control problems.
10 Zero-Sum Games on Two Coupled Atoms
The examples above can be looked at the dynamic extensions of the initial game of Meyer [33] in the sense that two players are acting on the same atom. In the popular EWL protocol [15] and the MW protocol [31], the players act simultaneously on two different qubits, with the interaction between the qubits taken into account by choosing entangled initial states and taking appropriate measurement.
The theory of Sect. 6 is general enough to accommodate games of two players on different atoms, moreover with the genuine interaction of atoms taken into account. Namely, suppose the atoms of players I and II, playing a zero-sum game, are two quantum systems, in \(\mathbf {C}^{n+1}\) each. The combined Hilbert space is thus \(\mathbf {C}^{n+1}\otimes \mathbf {C}^{n+1}\), so that its vectors can be written as \(\chi =\sum \chi _{jk} e_j\otimes e_k\) with \(\{e_k\}\) the standard basis in \(\mathbf {C}^{n+1}\). Suppose player I (respectively II) can act on the first (resp. second) atom by the controlled Hamiltonian operators \(H_I(u)=(h^I_{jk}(u))\) (resp. \(H_{II}(v)=(h^{II}_{jk}(v))\)), and the interaction between the atoms is given by an operator \(A=(A_{jk,pq})\).
Remark 10
The standard physics choice of the interaction is the operator arising from possible exchange of photons, \(A=a_1^*a_2+a_2^*a_1\), with the annihilation operators \(a_1\) and \(a_2\) of the two atoms.
The filtering Eq. (4) takes the form
where by \(\cdots \) we denoted the terms arising from the coupling with optical devices or from the uncontrolled Hamiltonian operators of the atoms. As previously, we rewrite this equation in terms of the projective coordinates \(w_{jk}=\chi _{jk}/\chi _{00}\) as follows (where it is understood that \(w_{00}=1\)):
Choosing for observation our special homodyne detection scheme from Sect. 4 (with \((2n+1)^2+2(2n+1)\) generalized Pauli operators) we get the HJB-Isaacs Eq. (31) in the form
where \(\alpha \) includes the contributions arising from the uncontrolled Hamiltonian operators (if any) and from the first-order terms of the Laplace–Beltrami operator \(\Delta _{LB} \) on the projective space \(P\mathbf {C}^{2n+1}\) (if any; there are no such terms for \(n=1\)). We are fully in the setting of Sect. 6 implying the well-posedness of the backward Cauchy problem for the HJB-Isaacs Eq. (49) in classical and mild solutions that yield the minimax value of the corresponding zero-sum game.
11 Nonzero-Sum Games
In the previous section, zero-sum games of two players were analyzed. However, the initial EWL and MW protocols are dealing with more general, nonzero-sum games. These games can be also accommodated in our setting with continuous observations. Let us consider for simplicity the case of two players playing on two coupled atoms. N players on N atoms can be looked at analogously.
As in the previous section, assume that players I and II can act on two atoms with the combined Hilbert space \(\mathbf {C}^{n+1}\otimes \mathbf {C}^{n+1}\). To simplify the story, we shall assume that the Hamiltonian operators \(H_I(u)\) and \(H_{II}(v)\) depend linearly on their control parameters \(u\in [-U,U]\) and \(v\in [-V,V]\). Suppose player I (respectively II) can act on the first (resp. second) atom by the controlled Hamiltonian operators \(H_I(u)=(h^I_{jk}(u))\) (resp. \(H_{II}(v)=(h^{II}_{jk}(v))\)), and the interaction between the atoms is given by an operator \(A=(A_{jk,pq})\). Hence, the controlled filtering Eq. (48) will be written as
Unlike the previous section with a single cost function, we assume now that the players have different cost functions, namely that players I and II aim at maximizing the costs
respectively. We again assume for simplicity that the current costs \(J^{I,II}\) do not depend on control, though this is really not essential.
If player I acts according to some strategy \(u=u(t,W)\), the optimal payoff of player II can be defined from the backward Cauchy problem for the HJB equation
where \(\alpha \) includes the contributions arising from the uncontrolled Hamiltonian operators (if any) and from the first-order terms of the Laplace–Beltrami operator \(\Delta _{LB} \) on the projective space \(P\mathbf {C}^{2n+1}\). Similarly for player I. Since \(\sup _v\) and \(\sup _u\) depend only on the signs (\(\mathrm{sgn}\)) of the corresponding sums, the pair of costs functions \(S^I\) and \(S^{II}\) satisfy the coupled system of two equations (a vector-valued HJB):
with
Since u, v depend Lipschitz continuously on the gradients of \(S^{I,II}\) and are uniformly bounded, Theorem 5.1 applies (more exactly, its straightforward vector-valued extension) leading to the well-posedness of system (54)-(55) in the sense of mild and/or classical solutions. By the verification theorem (see, e.g., [16]; note that for checking the Nash condition one has to verify the optimality for each single player, that is the verification theorem of the standard control theory is applicable) the solution of the backward Cauchy problem for system (54)–(55) yields the subgame-perfect Nash equilibrium for the corresponding game. For the recent results on general nonzero-sum differential games, we refer to [30] and references therein.
12 Conclusions
We introduced the special homodyne detection schemes that turn the problems of dynamic quantum filtering, control and games into the problems of the drift control of the standard Brownian motions on the complex projective spaces, tori and Euclidean spaces allowing for the effective theory of quantum dynamic games based on the classical and mild solutions of the HJB-Isaacs equations on Riemannian manifolds. An explicitly solved example is presented.
This approach opens the road to the effective application of the recent advanced numeric approaches to solving HJB equations, see [32, 35] and references therein, as well as to the methods of finding explicit solutions from [14]. Of course, additional work is required for the concrete applications of these methods to the present setting.
An interesting question arises from our construction. What is the minimal number N of the operators \(L_j\) (physically, of optical measuring devices) for a quantum system in \(\mathbf {C}^{n+1}\) that can ensure that the resulting diffusion on \(P\mathbf {C}^n\) is everywhere non-degenerate (and hence the theory of Sect. 5 applies)? From Sect. 4, it follows that \(2n\le N \le n^2+2n\). In particular \(2\le N\le 3\) for a qubit. Notice that the scheme of Sect. 9 does not solve the problem (as may be thought superficially), as it constructs the scheme with \(N=2n\), which is non-degenerate everywhere on the chart specified by finite W, but not outside it (in particular, with the exception of one point for a qubit).
Additionally, one can look at unbounded coefficients control problems arising from the homodyne schemes of Sect. 9.
Of interest is also a proper investigation of the long time behavior of controlled quantum processes, which can lead to some kind of turnpike behavior (see [29]) of stationary solutions. Some steps in this direction were made in [22] and [4, 23] for the jump-type and diffusive filtering, respectively. Some application of these ideas can be found in [37].
References
Accardi L, Frigerio A, Lu Y (1990) The weak coupling limit as a quantum functional central limit. Comm. Math. Phys. 131:537–570
Armen MA, Au JK, Stockton JK, Doherty AC, Mabuchi H (2002) Adaptive homodyne measurement of optical phase. Phys Rev Lett 89
Barchielli A, Belavkin VP (1991) Measurements continuous in time and a posteriori states in quantum mechanics. J. Phys A: Math. Gen. 24:1495–1514
Barchielli A, Paganoni AM (2003) On the asymptotic behaviour of some stochastic differential equations for quantum states. Infin Dimens Anal Quantum Probab Relat Top 6(2):223–243
Belavkin VP (1987) Nondemolition measurement and control in quantum dynamical systems. In: Information Complexity and Control in Quantum Physics. CISM Courses and Lectures 294, S. Diner and G. Lochak, eds., Springer-Verlag, Vienna, pp 331–336
Belavkin, VP (1988) Nondemolition stochastic calculus in Fock space and nonlinear filtering and control in quantum systems. Proceedings XXIV Karpacz winter school (R. Guelerak and W. Karwowski, eds.), Stochastic methods in mathematics and physics. World Scientific, Singapore, pp 310–324
Belavkin VP (1992) Quantum stochastic calculus and quantum nonlinear filtering. J Multivar Anal 42:171–201
Belavkin VP, Kolokoltsov VN (2002) Stochastic evolution as interaction representation of a boundary value problem for Dirac type equation. Infinite Dimens Anal Quantum Probab Relat Fields 5(1):61–92
Botina J, Rabitz H, Rahman N (1996) A simplified approach to optimally controlled quantum dynamics. J Chem Phys 104:4031–4040
Bouten L, Van Handel R (2006) On the separation principle of quantum control. arXiv: math-ph/0511021v2
Bouten L, Van Handel R, James M (2007) An introduction to quantum filtering. SIAM J Control Optim 46(6):2199–2241
Bushev P et al (2006) Feedback cooling of a single trapped ion. Phys.Rev Lett, p 96
Davies EB (1989) Pointwise bounds on the space and time derivatives of the heat kernel. J Oper Theory 21:367–378
Duncan TE (2018) Some solvable stochastic differential games in compact and noncompact rank one symmetric spaces. Intern J Control 91:2445–2450
Eisert J, Wilkens M, Lewenstein M (1999) Quantum games and quantum strategies. Phys Rev Lett 83(15):3077–3080
Fleming WH, Soner HM (2006) Controlled Markov Processes and Viscosity Solutions. Sec Ed. Sptinger
Guo H, Zhang J, Koehler GJ (2008) A survey of quantum games. Decis Supp Syst 46:318–332
Hamadène S, Mu R (2015) Existence of Nash equilibrium points for Markovian non-zero-sum stochastic differential games with unbounded coefficients. Stochastics 87(1):85–111
Holevo AS (1991) Statistical Inference, for quantum processes Quanum Aspects of Optical communications. Springer, Berlin, pp 127–137
Khan FS, Solmeyer N, Balu R, Humble T (2018) Quantum games: a review of the history, current state, and interpretation. Quantum Inf Process 17:309
Kolokoltsov VN (2000) Semiclassical analysis for diffusions and stochastic processes. Springer Lecture Notes Math., v. 1724, Springer
Kolokoltsov VN (1992) The stochastic Bellman equation as a nonlinear equation in Maslov spaces. Perturbation theory. Dokl. Akad. Nauk 323(2): 223-228. Engl. transl. in Sov. Math. Dokl. 45:2 (1992), 294-300
Kolokoltsov VN (1995) Long time behavior of the solutions of the Belavkin quantum filtering equation. Proc. Intern. Workshop on Quantum Communications and measurement, Nottingham, 11-16 July 1994 (Eds. V.P. Belavkin, O. Hirota, R.L. Hudson). Plenum Press, N.Y., pp 429–439
Kolokoltsov VN (2019) Differential equations on measures and functional spaces. Birkhäuser Advanced Texts, Birkhäuser
Vassili N. Kolokoltsov. Quantum mean field games. https://arxiv.org/pdf/2005.02350.pdf
Kolokoltsov VN (2021) Quantum mean-field games with the observations of counting type. Games 12:7. https://doi.org/10.3390/g12010007
Kolokoltsov VN Continuous time random walks modeling of quantum measurement and fractional equations of quantum stochastic filtering and control. arXiv:2008.07355
Kolokoltsov VN, Malafeyev OA (2020) Understanding Game theory, 2nd edn. World Scientific, Singapore
Kolokoltsov V, Yang W (2012) The turnpike theorems for Markov games. Dyn Games Appl 2(3):294–312
Li J, Li W (2019) Nash equilibrium payoffs for non-zero-sum stochastic differential games without Isaacs condition. Stochastics 91(1):1–36
Marinatto L, Weber T (2000) A quantum approach to static games of complete information. Phys Lett A 272:291–303
McEneaney WM, Kluberg LJ (2009) Convergence rate for a curse-of-dimensionality-free method for a class of HJB PDEs. SIAM J Control and Opt, p 48
Meyer DA (1999) Quantum strategies. Phys Rev Lett 82(5):1052–1055
Pellegrini C (2009) Poisson and diffusion approximation of stochastic Schrödinger equations with control. Ann Henri Poincaré 10(5):995–1025
Sridharan S, Gu M, James MR, McEneaney WM (2010) A reduced complexity numerical method for optimal gate synthesis. Phys Rev A
Wiseman HM, Milburn GJ (2010) Quantum measurement and control. Cambridge Univesity Press, Cambridge
Yur’ev, DV (1996) Belavkin-Kolokoltsov watch-dog effect in interactively controlled stochastic computer-graphic dynamical systems. Teoret. Mat. Fiz. 106(2):333-352; translation in Theoret. and Math. Phys. 106:2 (1996), 276–290
Zhu X (2014) The optimal control related to Riemannian manifolds and the viscosity solutions to Hamilton-Jacobi equations. Syst Control Lett 69:7–15
Acknowledgements
The author gratefully acknowledges support by the Russian Academic Excellence project ’5-100’.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Kolokoltsov, V.N. Dynamic Quantum Games. Dyn Games Appl 12, 552–573 (2022). https://doi.org/10.1007/s13235-021-00389-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13235-021-00389-w
Keywords
- Quantum dynamic games
- Quantum control
- Quantum filtering
- Belavkin equation
- Stochastic Schrödinger equation
- Output process and innovation process
- Brownian motion on sphere and complex projective spaces
- Controlled diffusion on Riemannian manifolds
- Hamilton–Jacobi–Bellman–Isaacs equation on manifolds
- Classical and mild solutions
- Ito’s formula