On the operator origins of classical and quantum wave functions

Abstract

We investigate operator algebraic origins of the classical Koopman–von Neumann wave function $\psi _{KvN}$ as well as the quantum-mechanical one $\psi _{QM}$. We introduce a formalism of Operator Mechanics (OM) based on a noncommutative Poisson, symplectic, and noncommutative differential structures. OM serves as a pre-quantum algebra from which algebraic structures relevant to real-world classical and quantum mechanics follow. In particular, $\psi _{KvN}$ and $\psi _{QM}$ are both consequences of this pre-quantum formalism. No a priori Hilbert space is needed. OM admits an algebraic notion of operator expectation values without invoking states. A phase space bundle ${{\mathcal {E}}}$ follows from this. $\psi _{KvN}$ and $\psi _{QM}$ are shown to be sections in ${{\mathcal {E}}}$. The difference between $\psi _{KvN}$ and $\psi _{QM}$ originates from a quantization map interpreted as “twisting” of sections over ${{\mathcal {E}}}$. We also show that the Schrödinger equation is obtained from the Koopman–von Neumann equation. What this suggests is that neither the Schrödinger equation nor the quantum wave function are fundamental structures. Rather, they both originate from a pre-quantum operator algebra. Finally, we comment on how entanglement between these operators suggests emergence of space; and possible extensions of this formalism to field theories.

1 Introduction

Many traditional formulations of quantum mechanics typically assume a Hilbert space of states ${{\mathcal {H}}}$ upon which one admits an algebra of linear operators over ${{\mathcal {H}}}$. Observables quantities correspond to Hermitian operators in this algebra and measurements correspond to expectation values of these operators with respect to a suitably defined density operator. In other words, states take precedence over operators with the Hilbert space of states as the primary structure upon which everything else is defined or constructed.

There certainly do exist approaches to quantum mechanics where operators take precedence over states. An example being ‘Categorical Quantum Mechanics’ (CQM) [1], where one typically considers the compact closed monoidal category of finite-dimensional Hilbert spaces (referred to as FHilb). The objects of FHilb are finite-dimensional Hilbert spaces used to describe ‘states’, and whose morphisms are linear operators used to describe ‘processes’.

Another related example, where operators take precedence, is Connes’ Noncommutative Geometry (NCG), as an attempt to describe the quantum geometry of spaces using noncommutative coordinates [25]. In this case, the underlying base structure of the formalism is a Fredholm module or a spectral triple $(\mathcal{A}, {{\mathcal {H}}}, F)$, which consists of a representation of a noncommutative ${{\mathcal {C}}}^\star$ operator algebra on a Hilbert space together with an unbounded self-adjoint operator. However, as mentioned, even in these examples, a Hilbert space of states is a crucial part of the initial structure of the formalism. In other words, this dichotomy between states and operators is very much one between geometry and algebra, respectively. The original motivation for the NCG program was to investigate whether one can indeed construct geometric data of a base manifold ${{\mathcal {M}}}$ from a noncommutative operator algebra. The cotangent bundle $\mathcal{T}^\star {{\mathcal {M}}}$ of ${{\mathcal {M}}}$ or rather its configuration space provides the support of the Hilbert space of $L^2$-integrable functions. Observables then correspond to Hermitian operators on this Hilbert space.

On the other hand, what we would like to investigate in this work is whether one can solely work with an abstract operator algebra, independent of any specific representation or a priori Hilbert space, and rather construct the space of states from these operators alone. In other words, absolutely without taking recourse to any geometric overheads. A purely abstract algebraic formalism where the space of states itself is obtained as a derived entity would arguably be a promising candidate for a pre-quantum theory, or rather for pre-quantum geometry. What can be useful with such pre-quantum structures is that they allow for background-independent formulations of potential quantum gravity models. In particular, these methods may help concretize pregeometric approaches to quantum theory, of the kind advocated by Wheeler [53].

In contrast to some of the above-mentioned programs, the objectives of our work here are modest. As a first step toward a pre-quantum theory, we introduce a formalism of Operator Mechanics (OM), based on a Lie algebra of canonically conjugate operators. This algebra is equipped with a noncommutative Poisson structure, a symplectic structure, and a noncommutative differential structure. The OM framework introduced here builds upon previous work on noncommutative geometry discussed in [39,40,41,42]. OM is presented as a formal operator algebra and this framework does not rely on any specific representation theory of the algebra, nor its action on any Hilbert space. In this sense, OM assumes far less than the noncommutative geometry formalism of Connes’, whose starting point is a spectral triple. In OM, we recover the operator version of Hamilton’s classical equations of motion. This follows simply as a consequence of the symplectic structure. An important ingredient of OM that we introduce here is a definition of a density operator defined using the symplectic structure of the algebra (without alluding to any notion of states). This allows us to compute expectation values of operators without invoking any notion of wave functions or states. In the OM framework, wave functions are not fundamental entities, but are derived from structures built upon the operator algebra.¹

We will investigate both the classical Koopman–von Neumann’s $\psi _{KvN}$ [45, 48] and the quantum-mechanical wave function $\psi _{QM}$. We do not assume any a priori Hilbert space of states. OM admits an algebraic notion of operator expectation values without invoking wave functions or states. A phase space bundle ${{\mathcal {E}}}$ follows from this. Both $\psi _{KvN}$ and $\psi _{QM}$ are shown to be sections in ${{\mathcal {E}}}$. The difference between $\psi _{KvN}$ and $\psi _{QM}$ appears in the form of a quantization scheme interpreted as “twisting” of sections over ${{\mathcal {E}}}$. Implementing this quantization map, we will also show that the Schrödinger equation itself can be obtained from the Koopman–von Neumann equation. What this suggests is that neither the Schrödinger equation nor the quantum-mechanical wave function are fundamental structures. Rather, these are built upon a pre-quantum operator algebra and ensuing algebraic homomorphisms. In this sense, the OM formalism presented here serves as a candidate for a pre-quantum operator algebra.

The outline of this paper is as follows: In Sect. 2, we introduce our definitions of the OM formalism. In Sect. 3, we discuss the operator equations of motion resulting in OM and its pre-quantum nature. In Sect. 4, we introduce a new density operator and show that it satisfies the usual requirements of a density operator; we also introduce a new operator expectation value map. In Sect. 5, we discuss structures leading to wave functions, observables, and expectation values of the second kind. In Sect. 6, we discuss the pre-quantum origins of Ehrenfest’s theorem. In Sect. 7, we show how the Schrödinger equation itself follows from the Koopman–von Neumann equation via quantization. In Sect. 8, we discuss further implications of pre-quantum mechanics and conclude mentioning connections to matrix models, noncommutative geometry, emergence of space from entanglement and other category-theoretic approaches.

2 Definitions of Operator Mechanics (OM)

We now introduce our framework of Operator Mechanics (OM). The mathematical formulation of OM is based on a noncommutative Poisson algebra [14, 15, 26, 58], a symplectic structure, and a noncommutative differential structure.

Recall that a commutative Poisson algebra is a commutative (unital) algebra (${{\mathcal {L}}}, \, +, \, \cdot$) with an associative product ‘$\cdot$’, and at the same time also a Lie algebra (${{\mathcal {L}}}, \, +, \, [ \, -, - \, ]$) with a non-associative product $[ \, -, - \, ]: {{\mathcal {L}}} \times {{\mathcal {L}}} \rightarrow {{\mathcal {L}}}$, that is bilinear and anti-commutative. Furthermore, this algebra satisfies the Jacobi identity and the Leibniz rule, respectively

$$\begin{aligned} \, [f, \, [ g, h] ]= & {} [ [f, g], \, h] + [g, \, [f, h] ] \end{aligned}$$

(1)

$$\begin{aligned} \, [f, \, g \cdot h]= & {} \,\, [ f, g] \cdot h \, + \, g \cdot [f, h], \end{aligned}$$

(2)

where f, g, h belong to ${{\mathcal {L}}}$. The Jacobi identity itself can be thought of as a Leibniz rule with respect to the Lie product. Notice that if one defines a derivative $D (g) = [f, g]$ using Lie brackets and a given f in this algebra, then the Leibniz rule stated above yields the usual product rule for derivatives. On the other hand, without the product ‘$\cdot$’, one simply recovers the definition of a Lie algebra. The archetypical example of a commutative Poisson algebra is the algebra of ${{\mathcal {C}}}^{\infty }$ functions with the Poisson bracket as the Lie bracket. This forms the backbone of classical mechanics.

To define a noncommutative Poisson algebra, the product ‘$\cdot$’ is now replaced by a noncommutative associative product. ${{\mathcal {L}}}$ can be an operator ring or an ${{\mathcal {R}}}$-module and the bracket

$$\begin{aligned}{}[A, B] = A \cdot B - B \cdot A \end{aligned}$$

(3)

realizes a noncommutative Poisson structure.² Only a few other instances of noncommutative Poisson structures are known. These have been found on moduli spaces classifying semi-simple modules [14, 26]. Examples of such systems include path algebras of doubled quivers, pre-projective algebras, and multiplicative pre-projective algebras discussed in [14, 26]. In what follows, we will remain largely agnostic to the choice of Lie brackets. In OM, $[ \, -, - \, ]$ will denote a generic Lie bracket, whose form could be as in Eq. (3) or one appearing in aforementioned cases of path algebras of doubled quivers and pre-projective algebras that have noncommutative Poisson structures given by the necklace Lie algebra [14, 15, 26].

Furthermore, in addition to this noncommutative Poisson structure, we will include a symplectic structure on (${{\mathcal {L}}}, \, +, \, \cdot$). Let this algebra be generated by operators $\{ X^i \}$ and $\{ P_i \}$, for $1 \le i \le n$, plus the multiplicative identity. These generators satisfy the following canonical commutation relations:

$$\begin{aligned}{}[X^i, P_j]= & {} \kappa \; \delta ^i_j \end{aligned}$$

(4)

$$\begin{aligned} \, [X^i, X^j]= & {} 0 \end{aligned}$$

(5)

$$\begin{aligned} \, [P_i, P_j]= & {} 0, \end{aligned}$$

(6)

where $\kappa$ is an arbitrary constant at this point (which need not be the $i \, \hbar$ of quantum mechanics, but may be associated to a more fundamental theory).

Let us make a small remark on the dimensions of $\kappa$, which in general will depend on specific representations of the Lie bracket that one works with. When the Lie bracket $[ \, -, - \, ]$ is chosen as in Eq. (3), then $\kappa$ has dimensions M L$^2$T$^{-1}$. On the other hand, if one replaces (${{\mathcal {L}}}, \, +, \, \cdot$) by a commutative algebra, then the Lie bracket is the Poisson bracket. In that case, $\kappa$ would equal 1 and be dimensionless.

The symplectic structure underlying the canonical commutation relations in Eqs. (4), (5), and (6) can be seen by packaging operators $X^i$ and $P_i$ into “vectors” (or rather elements of an operator module)

$$\begin{aligned} V_i = ( X^i, \, P_i )^T. \end{aligned}$$

(7)

The components of this object $V_i^a$ for $a = \{ 1, 2 \}$ then satisfy the following:

$$\begin{aligned} \frac{1}{\kappa } \, [V_i^a, V_j^b]= & {} \omega ^{a b} \; \delta _{i j}, \end{aligned}$$

(8)

where $\omega ^{a b}$ is the symplectic form with matrix representation

$$\begin{aligned} \omega= & {} \left( \begin{matrix} 0 &{} 1 \\ -1 &{} 0 \end{matrix} \right) . \end{aligned}$$

(9)

Given a noncommutative Poisson and symplectic structure, OM admits the following differential structure:

Definition 2.1

Partial derivatives of operators $F \in {{\mathcal {L}}}$ are maps Der

$$\begin{aligned} Der: {{\mathcal {L}}} \rightarrow {{\mathcal {L}}} \end{aligned}$$

(10)

where Der satisfies the Leibniz Rule

$$\begin{aligned} Der (a \cdot b) = Der (a) \cdot b + a \cdot Der (b). \end{aligned}$$

(11)

With respect to $X^i$ and $P_i$, the generators of the algebra, these partial derivatives are defined as

$$\begin{aligned} \kappa \, \frac{\partial F}{\partial X^i}= & {} [F, P_i] \end{aligned}$$

(12)

$$\begin{aligned} \kappa \, \frac{\partial F}{\partial P_i}= & {} [X^i, F]. \end{aligned}$$

(13)

Definition 2.2

The time derivative of an operators $F \in {{\mathcal {L}}}$ is defined as a Lie product with a special operator H, which we call the Hamiltonian

$$\begin{aligned} \kappa \, \frac{d F}{d t}= & {} [F, H]. \end{aligned}$$

(14)

Note that this form of the time derivative is structurally analogous to the Hamiltonian flow generated by a Lie algebra. The brackets $[ \, -, - \, ]$ in both the above definitions are generic Lie brackets, which algebraically capture the noncommutative differential structure of OM.

3 OM as a pre-quantum algebra

As detailed above, our formalism of OM is the noncommutative analogue of the Poisson algebra (plus a symplectic structure). The commutative case is defined over ${{\mathcal {C}}}^{\infty }$ functions, whereas here we will work with a noncommutative operator algebra. In OM, we take the conjugate variables $X^i$ and $P_i$ to be abstract operators (that is, those defined independent of an operator representation or upon a Hilbert space) satisfying the canonical commutation relations. We replace Poisson brackets with Lie brackets admitting the Leibniz rule over noncommutative algebras.

We will take the perspective that abstract operator algebras be deemed neither classical nor quantum entities as they carry insufficient structure to completely specify either, yet they demonstrate certain properties referring to both (as we shall now discuss). Special realizations of ${{\mathcal {L}}}$ (examples may also include a sub-algebra involving a commutative ring or other noncommutative deformations of the Poisson bracket) with a given specification of the Lie bracket, and appropriate $\kappa$ values, may accordingly yield classical or quantum-mechanical characteristics based on subsequent structures admissible over the operator algebra. In this sense, we refer to OM as a pre-quantum framework upon which the laws of classical and quantum mechanics arise following the identification of additional algebraic structures. This confronts us with an important fundamental question: Where does the wave function and algebra of observables (in both, classical and quantum mechanics) come from? In this work, we investigate algebraic structures that may help shed light on this question.

We now show how the operator analogues of Hamilton’s equations of motion can be derived in our formalism. A special case of this result was shown in [39,40,41,42] when one considers the commutator form of the Lie bracket as in Eq. (3). Here, we extend that result for any generic Lie bracket as follows:

$$\begin{aligned} \kappa \, \frac{d P_i}{d t}= & {} [P_i, H] \, = \, - [H, P_i] \, = \, - \kappa \, \frac{\partial H}{\partial X^i} \end{aligned}$$

(15)

$$\begin{aligned} \kappa \, \frac{d X^i}{d t}= & {} [X^i, H] \, = \, \kappa \, \frac{\partial H}{\partial P_i}, \end{aligned}$$

(16)

where only the definitions of derivatives and axioms of a Lie algebra have been used, without alluding to any representation of the Lie bracket. The Hamilton’s equations thus derived are also independent of the value of $\kappa$ (with the constant dropping off on both sides).

From the above, it is also straightforward to see how Newton’s laws (in operator form) can be obtained in our noncommutative OM formalism for any Hamiltonian quadratic in momentum. Consider the Hamiltonian

$$\begin{aligned} H = \frac{1}{2 m} \sum _{i,j = 1}^n \eta ^{ij} P_i P_j + V (\{ X^i \}) \end{aligned}$$

(17)

with mass m, $\eta ^{ij}$ a flat Euclidean metric, and $V (\{ X^i \})$ an arbitrary operator-valued potential. Substituting H in Hamilton’s equations above gives

$$\begin{aligned} \frac{d P_i}{d t}= & {} - \frac{\partial V (X)}{\partial X^i} \end{aligned}$$

(18)

$$\begin{aligned} \frac{d X^i}{d t}= & {} \frac{\eta ^{ij}}{m} P_j, \end{aligned}$$

(19)

where the derivatives are the noncommutative differentials defined in Sect. 2 above. Putting the above equations together, we get the following operator-theoretic version of Newton’s law of motion:

$$\begin{aligned} m \ddot{X^i} = { F^i}, \end{aligned}$$

(20)

where the force ${ F^i}$ is defined as the negative gradient of the potential $- \partial V (X) / \partial X^i$ (the partial derivative being an OM commutator).³

Note that Eqs. (15)–(20) are all operator equations. Naive application of standard calculus or functional analysis to solve these equations of motion will not work. Instead, one has to work with spectral properties and admissible representations of these operators to find solutions. This is because with noncommutativity, one is confronted with what is sometimes referred to as “pointless geometry”, where localized notions of coordinates on phase space are replaced by operators.

On the one hand, the above noncommutative formalism seems to bear partial resemblance to classical mechanics; on the other hand, it also resembles quantum mechanics. The latter can be seen from the OM time evolution equation [Eq. (14)], which has the form of Heisenberg’s equation (for operators without explicit time dependence) when $\kappa$ is taken to be $i \, \hbar$. However, by itself, the algebra of OM is strictly-speaking neither classical nor quantum. It is an abstract algebraic structure defined independently of both, background spaces and state spaces (classical or quantum). We will argue that the familiar structures of classical and quantum mechanics originate from OM. In this sense, one may regard OM as a pre-quantum algebra.

4 Expectation values of the first kind

In the standard quantum-mechanical framework, expectation values of Hermitian operators can be found for a given quantum state in Hilbert space. The expectation value refers to the probabilistic expected value of an experimentally measured quantity. For mixed states, one may construct the density matrix as a positive trace-class operator $\rho = \sum _{i = 1}^n p_i |\psi _i\rangle \langle \psi _i|$. The expectation value of self-adjoint operators ${{\mathcal {O}}}$ associated with observables is given by $Trace \left( \rho {{\mathcal {O}}} \right)$. Note that $\rho$ itself is defined in the Schrödinger picture and requires the a priori specification of a complete set of basis states in the Hilbert space upon which the action of operators ${{\mathcal {O}}}$ is defined. On the other hand, if one starts with a pre-quantum operator algebra devoid of a Hilbert space as discussed in Sects. 2 and 3, how should one construct a density operator? Also, how does one interpret the trace of a given operator weighted by such a density operator in the absence of a state space?

Let us proceed to define additional constructions within our OM framework that will enable us to answer the questions raised above. Within our operator algebra, let us consider two sub-spaces LinOp(X) and LinOp(P) whose elements are linear combinations of operators X and P, respectively, with scalar multiplication given by a ring ${{\mathcal {R}}}$.

Definition 4.1

We define maps

$$\begin{aligned} \phi (X): X \rightarrow LinOp (X) \qquad \chi (P): P \rightarrow LinOp (P), \end{aligned}$$

(21)

where $\phi (X)$ and $\chi (P)$, respectively, take values in an $\mathcal{R}$-module whose basis are the generators of ${{\mathcal {L}}}$.

Here, we will consider the field of scalars ${{\mathcal {R}}}$ to be ${{\mathbb {C}}}$-valued (though that can be generalized to noncommutative scalars in an even more general setting, for instance, when one wants to discuss spinor-valued operators).

Furthermore, the symplectic structure in OM endows the above maps with the following property:

Lemma 4.1

For every $\phi (X)$, there exists an operator $\chi (P)$, such that

$$\begin{aligned} \frac{1}{\kappa } \, {\mathbf{{Pr}}_m} \, [ \phi (X), \, \chi (P) ] \in {{\mathbb {R}}}_{\ge 0} < \infty , \end{aligned}$$

(22)

where ${\textbf{Pr}_m}$ denotes an eigenvalue projection operator whose action on any $F \in {{\mathcal {L}}}$ yields the $m^{th}$ eigenvalue of F, that is

$$\begin{aligned} {\textbf{Pr}_m} \, F = f_m \end{aligned}$$

(23)

for any given an N-dimensional representation of F with eigenvalues $\{ f_m \}$ for $1 \le m \le N$.

The proof of this lemma involves substituting $X^i \rightarrow P_i$ and $k \rightarrow k^{*}$ in $\phi (X) = \sum _{i = 1}^n k \, X^i$ for scalars $k \in {{\mathbb {C}}}$.

Note that given an N-dimensional identity operator, we have ${\textbf{Pr}_m} \, {{\textbf{1}}}_N = 1$. Alternatively, instead of the eigenvalue projection operator ${\textbf{Pr}_m}$, we could have also implemented the standard operator trace in Eq. (22), which would simply sum over all eigenvalues rather than projecting onto a specific one. That would also work for defining the density operator below. The operation ${\textbf{Pr}_m}$ is more useful when working with given operators $X^i$ and $P_i$ as we do here. On the other hand, the trace operation can be useful when considering an ensemble of $X^i$ and $P_i$ and performing a matrix model analysis.

Remark 4.1

From Eq. (22), we see that for given $\phi (X)$, a map $\chi (P)$ can be constructed in such a way that the above lemma holds. $\chi (P)$ thus constructed can be thought of as the involution of $\phi (X)$. We will denote this as $\overline{\phi (X)}$. Equation (22) then defines a norm on LinOp(X).

The right-hand side of Eq. (22) can be verified using the canonical commutation relations.

Remark 4.2

Furthermore, LinOp(X) almost carries the structure of a Hilbert space, except that the $\phi (X)$ are operators in an $\mathcal{R}$-module and we have not imposed completeness under the operator topology. The space LinOp(X) of operators (along with its involution operation and norm) is in fact a ‘pre-Hilbert’ space ${{\mathcal {H}}}_{pre}$ (one would still need to show completeness under the operator topology in order for this to be a full-fledged Hilbert space).

Equipped with ${{\mathcal {H}}}_{pre}$, we can now define a new density operator $\hat{\rho }$ in OM

Definition 4.2

$$\begin{aligned} \hat{\rho } = \frac{1}{\kappa ^2 } \sum _i^n w_i \, | \, P_i \, ] \, [ \, X^i \, |, \end{aligned}$$

(24)

which is expressed in the basis of ${{\mathcal {H}}}_{pre}$. The weights $w_i$ are real coefficients satisfying $w_i \ge 0$ and $\sum _{i = 1}^n w_i = 1$.

The notation $[ \, X^i \, |$ indicates a basis operator with left-action on any operator F, such that taking the trace over ${{\mathcal {H}}}_{pre}$ results in the commutator $[ \, X^i, \, F \, P_i \, ]$. Likewise, $| \, P_i \, ]$ denotes the conjugate operator, which, due to the involution map, can also be expressed as $| \, \overline{X^i} \, ]$. Notice that these are in fact the operator analogues of Dirac’s bra–ket vectors, defined directly using the canonical commutation relations of the Lie algebra ${{\mathcal {L}}}$. For that reason, $[ \, X^i, \, F \, P_i \, ]$ is not a complex amplitude, but is operator-valued in ${{\mathcal {L}}}$.

Given the above definition of $\hat{\rho }$, we then define operator expectation values as follows:

Definition 4.3

Given an operator F, we define its ‘expectation value of the first kind’ as

$$\begin{aligned} \langle F \rangle _m = {\textbf{Pr}_m} \, Tr_{{{\mathcal {H}}}_{pre}} (\hat{\rho } F), \end{aligned}$$

(25)

where, we first take the trace over the bases of ${\mathcal H}_{pre}$, that is $[ \, X^i, \, \hat{\rho } F \, \overline{X^i} \, ]$, and then, apply the eigenvalue projection operator. The expectation value so defined depends on the eigenvalue index m.

Remark 4.3

The density operator $\hat{\rho }$ defined in Eq. (24) satisfies

$$\begin{aligned} Tr_{{{\mathcal {H}}}_{pre}} (\hat{\rho }) = {{\textbf{1}}}_N. \end{aligned}$$

(26)

Moreover, $\hat{\rho }$ is also positive semi-definite and self-adjoint (where the adjoint operation is defined using the involution map on ${{\mathcal {H}}}_{pre}$ as conjugation, and the transpose is realized via flipping the operator bra and ket in $\hat{\rho }$), which justifies its use as a density operator for computing expectation values as in Eq. (25).

For Hermitian operators $F \in {{\mathcal {L}}}$ (that is, those with real eigenvalues), $\langle F \rangle _m$ will yield real values (for each m) that reflect the spectrum of F. The above constructions make use of the symplectic structure in OM and do not require any a priori notion of a Hilbert space of state vectors upon which operators act.

Let us now examine the expectation value of the first kind for the operator $X^i$. Consider any given N-dimensional representation of $X^i$ with spectrum $\{ \lambda _m^i \}$ for $1 \le m \le N$. This may be taken to be an $N \times N$ matrix. Furthermore, the $X^i$ are Hermitian (as required for a Hermitian Hamiltonian). Therefore, the $\lambda _m^i$ are all real. The expectation value of $X^i$ then is

$$\begin{aligned} \langle X^i \rangle _m = {\textbf{Pr}_m} \, Tr_{{{\mathcal {H}}}_{pre}} (\hat{\rho } X^i) = w_i \, \lambda _m^i, \end{aligned}$$

(27)

which is now a function of m, the eigenvalue index (note that no summation over common indices is implied above). Unlike the usual notion of an expectation value, which depends on the states of a system, $\langle X^i \rangle _m$ is a distribution depending on the eigenvalue index of the operator. It will be instructive to now consider the continuum limit of the operator spectrum, where $\langle X^i \rangle _m$ for each given instance of m can be thought of as defining a coordinate of a classical configuration space. To take the continuum limit of the eigenvalue spectrum of this operator, we will use the same method typically employed in Hermitian matrix models (see [17] or [3] for a more recent survey). This limit can be obtained by taking the large N limit of the operator $X^i$ (using the representation of $N \times N$ Hermitian matrices). Consider functions $\lambda ^i (x)$, such that

$$\begin{aligned} \lambda ^i (m/N) = \frac{\lambda _m^i}{N} \end{aligned}$$

(28)

with the eigenvalues $\lambda _m^i$ being arranged in non-decreasing order $\lambda _1^i \le \lambda _2^i \le \cdots \le \lambda _N^i$. Then, in the large N limit, we get $- \alpha \le \lambda ^i (x) \le \alpha$ for some positive $\alpha$ and $0 \le x \le 1$. The precise value of $\alpha$ does not concern us here, but it can be determined based on how the distributions of eigenvalues of these operators are constrained. Alternatively, one may consider an ensemble of random Hermitian matrices with an appropriate path integral and determine $\alpha$ from the resolvent of the corresponding loop equations as in [17]. When $N \rightarrow \infty$, the scaling of eigenvalues in Eq. (28) yields a continuous spectrum over a finite support, as shown in [17].

Remark 4.4

The main reason for considering the aforementioned large N limit of the operator expectation values (of the first kind) of $X^i$ (and likewise $P_i$) is that this procedure allows us to define new continuum variables in terms of the spectrum of the operator $X^i$ (likewise for operators $P_i$). In the following sections, we show that the new variables

$$\begin{aligned} {{\mathcal {X}}}^i (x) \equiv \lim _{N \rightarrow \infty } \, \frac{1}{N} \langle X^i \rangle _x \qquad \qquad {{\mathcal {P}}}_i (x) \equiv \lim _{N \rightarrow \infty } \, \frac{1}{N} \langle P_i \rangle _x \end{aligned}$$

(29)

turn out to be a natural choice as coordinates of phase space. Here, ${{\mathcal {X}}}^i$ and ${{\mathcal {P}}}_i$ are both continuous variables taking values over a subset of ${{\mathbb {R}}}^{2n}$.

Alternatively, in a matrix model analysis, the above phase space variables can also be obtained from the eigenvalue distribution of the matrix ensemble, in the large N limit.

Furthermore, in Sect. 8, we have provided an additional perspective (not completely unrelated to the operator eigenvalue description above) whereby the set of points (${{\mathcal {X}}}^i$, $\mathcal{P}_i$) acquire a spatial structure with a well-defined metric. This is achieved via entanglements of modes of operators $X^i$ and $P_i$, and is computed using the entanglement entropy based on a reduced density operator extracted from $\hat{\rho }$ defined above. In other words, that demonstrates how space emerges in OM from entanglements of operator eigenmodes in the pre-Hilbert space ${{\mathcal {H}}}_{pre}$.

5 Wave functions, observables, and expectation values of the second kind

The operator expectation values of the first kind defined above enables construction of new variables $\langle X^i \rangle _m$ and $\langle P_i \rangle _m$, which in the continuum limit, yield the coordinates that will become relevant for classical phase space. In this sense, classical phase space is not a pre-existing fundamental space, but one that results from the pre-Hilbert structure within the OM framework.

Remark 5.1

Algebraically, the expectation value of the first kind is a mapping from the noncommutative operator algebra associated with ${{\mathcal {L}}}$ to a commutative algebra. This can be described in terms of the following commutative diagram⁴:

(30)

If we require the commutative algebra on the right-hand side of the above diagram to preserve a Lie-algebraic structure, then we will require that

$$\begin{aligned} \langle \frac{1}{\kappa } \, [X^i, P_j] \, \rangle\cong & {} \big \{ \langle X^i \rangle , \, \langle P_j \rangle \big \}_{PB}, \end{aligned}$$

(31)

since the Poisson bracket is a natural choice for a Lie bracket associated with a commutative algebra. Equation (31) is a Lie algebra-preserving compatibility condition when going from a noncommutative Poisson structure to a commutative Poisson structure. The diagram in Eq. (30) along with the above compatibility condition is formally a ‘dequantization’ map from a noncommutative ${{\mathcal {C}}}^*$ algebra to a commutative one⁵ [13, 38, 57].

Remark 5.2

We can extend the diagram in Eq. (30) to a second commutative square which admits complex-valued functions $f \in {{\mathcal {F}}}$ on the space spanned by the $\langle X^i \rangle$ (and in general, also spanned by $\langle P_i \rangle$). Partial derivatives of these functions are expressed as their Poisson bracket with $\langle P_i \rangle$

(32)

Admitting an algebra of functions on the space spanned by $\langle X^i \rangle$ and $\langle P_i \rangle$ yields a bundle over ($\langle X^i \rangle$, $\langle P_i \rangle$), with $\frac{\partial }{\partial \langle X^i \rangle }$ and $\frac{\partial }{\partial \langle P_i \rangle }$ as tangent vectors. We will call this the phase space bundle ${{\mathcal {E}}}$ over the base over ($\langle X^i \rangle$, $\langle P_i \rangle$). The aforementioned functions can be thought of as sections in $\mathcal{E}$. Furthermore, working in the large N limit, we replace $\langle X^i \rangle \rightarrow {{\mathcal {X}}}^i$ and $\langle P_i \rangle \rightarrow {{\mathcal {P}}}_i$. Within this continuum limit, both classical and quantum wave functions $\psi _{KvN}$ and $\psi _{QM}$, respectively, are sections in ${{\mathcal {E}}}$. The difference between these sections is related to quantization (discussed in Sect. 7).

Hence, wave functions are complex-valued constructs existing in a phase space bundle ${{\mathcal {E}}}$. There are two kinds of wave functions alluded to above are: (i) the classical Koopman-von Neumann wave function $\psi _{KvN} \in {{\mathcal {F}}} \left( {{\mathcal {X}}}, {{\mathcal {P}}} \right)$, which depends on both ${{\mathcal {X}}}$ and ${{\mathcal {P}}}$; and (ii) the standard quantum-mechanical wave function $\psi _{QM} \in {{\mathcal {F}}} \left( {{\mathcal {X}}} \right)$, which depends only on $\langle X \rangle$.

Remark 5.3

The Koopman–von Neumann (KvN) formulation of mechanics was originally developed in the 1930s by Koopman and von Neumann as a theory of classical ensembles, with a particular focus toward ergodic theory [45, 48]. It makes use of a similar technical machinery to quantum mechanics, namely, Hilbert spaces, wave functions as states, and operators acting upon wave functions. For this reason, Koopman–von Neumann mechanics has also been relevant to the conceptual foundations of quantum theory [16, 44, 54]. The wave function $\psi _{KvN}$ in Koopman–von Neumann mechanics is a complex-valued function defined using the classical phase space distribution function $\tilde{\rho } ({{\mathcal {X}}}, {{\mathcal {P}}}; t) = \psi _{KvN}^*\, \psi _{KvN}$, and $\psi _{KvN}$ depends on both ${{\mathcal {X}}}$ and ${{\mathcal {P}}}$. Since $\tilde{\rho } ({{\mathcal {X}}}, \mathcal{P}; t)$ is a probability density, this implies a normalization condition on $\psi _{KvN}$. The latter induces an inner product on the space of Koopman–von Neumann wave functions (see Eq. (34)), thus leading to a Hilbert space of states ${{\mathcal {H}}}_{KvN}$. The Koopman–von Neumann equation is the classical analogue of the Schrödinger’s equation and follows from Liouville’s theorem (this is derived in Eq. (59) below).

Now, let us investigate the algebra of operators acting on functions in ${{\mathcal {E}}}$. These are $\langle X^i \rangle$ and $\langle P_i \rangle$, which act by ordinary multiplication; and the derivatives indicated in Eq. (32). The latter can equivalently be expressed as the following Poisson brackets: $\big \{ \bullet , \, \langle P_i \rangle \big \}_{PB}$ and $\big \{ \langle X^i \rangle , \, \bullet \big \}_{PB}$. One easily checks that these operators satisfy the following commutation relations:

$$\begin{aligned}{}[{{\mathcal {X}}}^i, \, {{\mathcal {P}}}_j] = 0 \qquad \qquad [{{\mathcal {X}}}^i, \, \tilde{\kappa } \big \{ {{\mathcal {P}}}_j, \, \bullet \big \}_{PB}]= & {} \tilde{\kappa } \, \delta ^i_j \nonumber \\ \, [ \tilde{\kappa } \big \{ {{\mathcal {X}}}^i, \, \bullet \big \}_{PB}, \, \tilde{\kappa } \big \{ {{\mathcal {P}}}_j, \, \bullet \big \}_{PB} ] = 0 \qquad \qquad [ \tilde{\kappa } \big \{ {{\mathcal {X}}}^i, \, \bullet \big \}_{PB}, \, {{\mathcal {P}}}_j ]= & {} \tilde{\kappa } \, \delta ^i_j \nonumber \\ \! [{{\mathcal {X}}}^i, \, \tilde{\kappa } \big \{ {{\mathcal {X}}}^j, \, \bullet \big \}_{PB}] = 0 \, \qquad \qquad [ {{\mathcal {P}}}_i, \, \tilde{\kappa } \big \{ {{\mathcal {P}}}_j, \, \bullet \big \}_{PB} ]= & {} 0, \end{aligned}$$

(33)

where we will be working with these operators in the continuum limit of their respective variables. The constant $\tilde{\kappa }$ has been introduced here to match dimensions. Note that $\tilde{\kappa }$ need not be the same as the $\kappa$ in OM. The operators in Eq. (33) act on wave functions in ${{\mathcal {E}}}$, whereas those in the OM canonical commutation relations are defined without any reference to a space of states. When we arrive at the Schrödinger’s equation, $\tilde{\kappa }$ will be related to $i \, \hbar$, whereas $\kappa$ is still a free parameter that may depend on another fundamental scale.

From the perspective of ${{\mathcal {E}}}$, the above operator algebra in Eq. (33) and the associated commutation relations hold for both, classical Koopman–von Neumann mechanics as well as traditional quantum mechanics. The difference between the two will be identified as a “twisting” of sections following a different identification of momentum in the two cases.

Being ${{\mathbb {C}}}$-valued functions, both $\psi _{KvN}$ and $\psi _{QM}$ admit the $L^2$-norm over phase space and configuration space, respectively

$$\begin{aligned} \int d {{\mathcal {X}}} d {{\mathcal {P}}} \, \psi _{KvN}^*\, \psi _{KvN} = 1 \qquad \qquad \int d {{\mathcal {X}}} \, \psi _{QM}^*\, \psi _{QM} = 1. \end{aligned}$$

(34)

And it is precisely this norm that endows the space of wave functions with the structure of a Hilbert space. In other words, this is a Hilbert space of sections over the phase space bundle ${{\mathcal {E}}}$. We will denote this Hilbert space as $\mathcal{H}_{KvN/QM}$. It is this $L^2$-norm in ${{\mathcal {H}}}_{KvN/QM}$ with respect to which the familiar density operator for a system in a mixed state is defined as

$$\begin{aligned} \rho = \sum _j p_j \, | \, {\psi _j}_{KvN/QM} \, > \, < \, {\psi _j}_{KvN/QM} \, |, \end{aligned}$$

(35)

where each of the pure states ${\psi _i}_{KvN/QM}$ occurs with probability $p_j$. Using this, expectation values of observables F are then defined as

$$\begin{aligned} \langle F \rangle = tr \left( \rho \, F \right) . \end{aligned}$$

(36)

We will call these expectation values of the second kind to distinguish them from the pre-quantum expectation values of the first kind. The above Hilbert space and expectation values exist for both Koopman–von Neumann classical mechanics as well as standard quantum mechanics. On this Hilbert space, an expectation value (of the second kind) corresponding to an observable is a function over states. However, this object also has a geometric interpretation in ${{\mathcal {E}}}$, where it quantifies the extent to which an operator $\hat{A}$ deforms a section $\psi _{KvN/QM}$ over ${{\mathcal {E}}}$ (with undeformed sections yielding the identity).

Hence, expectation values of the second kind are simply the standard expectation values with respect to a Hilbert space wave function that appear in Koopman–von Neumann mechanics as well as quantum mechanics.

Remark 5.4

The map from expectation values of the first kind to those of the second kind lead to the following commutative diagram:

(37)

This diagram represents the transition from “points” in ${{\mathcal {E}}}$ to ensemble averages.

All the above three commutative diagrams realize algebraic homomorphisms from ${{\mathcal {L}}}$ to ${{\mathcal {E}}}$ to $\mathcal{H}_{KvN/QM}$. In principle, these maps can also be formalized using functorial constructions from the appropriate categories of noncommuting operators to those of commuting entities. In particular, ${{\mathcal {F}}} \left( \langle X^i \rangle \right)$ would be replaced by the relevant category of Hilbert spaces. The categorification of the commutative diagrams above would presumably encapsulate the formalism of categorical quantum mechanics as a special case [1].

The maps constructed above also suggests an interesting generalization of the wave function itself. Instead of considering only the algebra of functions over expectation values of $X^i$, one may investigate the algebra of functions over expectation values of operator products, in other words, functions of n-point operator products. Given that the $\langle X^i \rangle$ have an interpretation as points of space, higher products refer to n-point geometric correlations such as extended objects (geodesics, membranes, etc).⁶

6 Pre-quantum origins of Ehrenfest’s theorem

Before we discuss quantization and Schrödinger’s equation in the next section, let us discuss one more result whose generalization turns out to be relevant to pre-quantum formalizations: Ehrenfest’s theorem. The standard version of Ehrenfest’s theorem is usually stated as a prescription to realize classical-like laws for expectation values of quantum-mechanical observables. Here, we demonstrate that there are in fact multiple manifestations of Ehrenfest’s theorem, or what could be called Ehrenfest-like theorems, obtained by applying expectation values of the first and second kind, starting from the OM pre-quantum algebra.

Firstly, recall that the Hamilton’s equations in operator form already show up at the level of the OM algebra, without recourse to classical calculus. Furthermore, for a Hamiltonian quadratic in momentum, this yields

$$\begin{aligned} \frac{d X^i}{d t}= & {} \frac{1}{m} P_i \end{aligned}$$

(38)

$$\begin{aligned} \frac{d P_i}{d t}= & {} - \frac{\partial V}{\partial X^i}, \end{aligned}$$

(39)

which is simply the first-order representation of Newton’s laws for operators $X^i$ and $P_i$. What we demonstrate below is that all manifestations of Ehrenfest’s theorem originate from these operator relations by applying the algebraic homomorphisms, discussed in the previous section, from the noncommutative algebra to the commutative one.

Now, let us apply the expectation value map of the first kind to the operatorial Hamilton’s equations above. This will map (alternatively, one may think of this as a functorial lift) these equations to dynamics in the regime of commutative algebras, otherwise known as classical mechanics. To see this, let us use the following generic form of the non-relativistic Hamiltonian operator that is quadratic in momentum and carries an arbitrary potential

$$\begin{aligned} H = \frac{1}{2 m} \sum _{i,j = 1}^n g^{ij} P_i P_j + V (\{ X^i \}), \end{aligned}$$

(40)

where the inner product $g^{ij}$ will refer to the flat metric $\eta ^{i j}$ for what follows (and is not operator valued here). Using this Hamiltonian in the OM equation of motion, Eq. (16), and applying the operator expectation of the first kind on both sides of this equation yield

$$\begin{aligned} \kappa \, \Big \langle \frac{d X^i}{d t} \Big \rangle= & {} \big \langle [X^i, H] \big \rangle \, = \, \kappa \, \frac{\eta ^{i j}}{m} \langle P_j \rangle . \end{aligned}$$

(41)

While the right-hand side is proportional to $\langle P_j \rangle$, we now want to understand how the left-hand side of this equation relates to $\langle X^i \rangle$. To see this, we first need a definition for time derivatives of commuting quantities f, which are functions of continuous variables ${{\mathcal {X}}}$ and $\mathcal{P}$ (working in the large N limit will ensure continuity of our phase space variables).

In the noncommutative OM calculus, the time derivative was defined as a Lie bracket of noncommutative operators. However, to work with a commutative calculus that carries an analogous Lie-algebraic structure, we need to define the following commutative time derivative:

Definition 6.1

The total time derivative of ${{\mathbb {C}}}$-valued functions of variables ${{\mathcal {X}}}$, ${{\mathcal {P}}}$ and time is defined via a Poisson bracket involving a Hamilton function in the variables ${{\mathcal {X}}}$ and ${{\mathcal {P}}}$, such that

$$\begin{aligned} \frac{d \, f }{d t_{c}}= & {} \frac{\partial \, f }{\partial \, t_{c}} \, + \, \big \{ f, \, H_{cl} \big \}_{PB}, \end{aligned}$$

(42)

where the partial derivative $\partial / \partial t_{c}$ is the same as partial derivatives in standard analysis.

The Hamilton function $H_{cl}$ above refers to a classical system, expressed in terms of continuous commuting phase space variables ${{\mathcal {X}}}^i$ and ${{\mathcal {P}}}_i$, given by

$$\begin{aligned} H_{cl} \, = \, \frac{1}{2 \, m} \sum _{i = 1}^n {{\mathcal {P}}}_i^2 \, + \, V \left( \{ {{\mathcal {X}}}^i \} \right) , \end{aligned}$$

(43)

and the classical Poisson bracket is expressed in terms of classical partial derivatives

$$\begin{aligned} \big \{ f, \, g \big \}_{PB}= & {} \sum _{i = 1}^n \left( \frac{\partial \, f }{\partial \, {{\mathcal {X}}}^i} \frac{\partial \, g }{\partial \, {{\mathcal {P}}}_i} \, - \, \frac{\partial \, f }{\partial \, {{\mathcal {P}}}_i} \frac{\partial \, g }{\partial \, {{\mathcal {X}}}^i} \right) . \end{aligned}$$

(44)

Here, we will use the notation $d / dt_{c}$ and $\partial / \partial t_{c}$ for commutative time derivatives (as opposed to d/dt, which denotes the time derivative defined for a noncommutative calculus referring to derivatives of operators). This notation is simply meant to distinguish between different types of derivatives associated to the algebra that they act upon. The definition in Eq. (42) is a Lie algebra-preserving homomorphism from ${{\mathcal {L}}}$ to a commutative algebra, where the expectation value of the first kind maps operators in OM to functions $\langle \cdots \rangle : F \rightarrow f$. The partial derivative $\partial / \partial t_{c}$ in the above definition has been included to account for functions f with explicit time-dependence.

Note that instead of proposing Eq. (42) as a definition, one could have also derived it from classical calculus. However, the derivation would involve assuming Hamilton’s equations to get Poisson brackets on the right-hand side. The advantage of taking Eq. (42) as a definition is that Hamilton’s equations then follow from this commutative algebra. To see this, one can simply substitute ${{\mathcal {X}}}^i$, respectively, ${{\mathcal {P}}}_i$ into Eq. (42) to get precisely the two Hamilton’s equations in classical mechanics.

Now returning to the Ehrenfest’s theorem. Given the above definition of commutative derivatives, the time derivative of $\mathcal{X}^i$ is obtained as

$$\begin{aligned} \frac{d \, {{\mathcal {X}}}^i }{d t_{c}} \, = \, \frac{1}{m} {{\mathcal {P}}}_i \, = \, \frac{1}{m} \frac{1}{N} \langle P_i \rangle , \end{aligned}$$

(45)

where the first equality is just Newton’s law for the commutative variable ${{\mathcal {X}}}^i$ (for a Hamiltonian quadratic in momentum). Compare this to Eq. (38). The second equality in Eq. (45) holds only in the large N limit. In that limit, the right-hand side of Eq. (45) is comparable to the right-hand side of Eq. (41). This implies that the expectation value of the operator time derivative of $X^i$ can be replaced by the commutative time derivative of the expectation value of $X^i$ in the continuum limit. That is

$$\begin{aligned} \frac{1}{N} \Big \langle \frac{d X^i}{d t} \Big \rangle \, \rightarrow \, \frac{d \, {{\mathcal {X}}}^i }{d t_{c}} \end{aligned}$$

(46)

in the large N limit.

Now implementing this mapping in Eq. (41) and expressing everything in continuum variables ${{\mathcal {X}}}^i$ and ${{\mathcal {P}}}_i$ give Newton’s law for the commutative variable ${{\mathcal {X}}}^i$ (for a Hamiltonian quadratic in momentum) from the OM expectation value equation (Eq. (41)). This serves as a consistency check for the roles of ${{\mathcal {X}}}^i$ and ${{\mathcal {P}}}_i$ as classical phase space variables.

In a similar manner, an Ehrenfest-like theorem for the time derivative of the momentum operator in OM is obtained by inserting the Hamiltonian operator above into Eq. (15) and taking the operator expectation values of the first kind on of both sides to get

$$\begin{aligned} \kappa \, \Big \langle \frac{d P_i}{d t} \Big \rangle= & {} - \, \kappa \, \Big \langle \frac{\partial \, V( \{ X^i \} ) }{\partial X^i} \Big \rangle . \end{aligned}$$

(47)

This is the statement of an Ehrenfest-like theorem for the momentum operator in OM. Given that the expectation value map of the first kind relates to classical phase space variables, the above statement can be cast in the traditional form of Newton’s law for quadratic potentials. This can be seen as follows. Consider the special case when the potential is of the form $V ( \{ X^i \} ) \, = \, g \, \delta _{i_1 i_2 \cdots i_l} \, X^{i_1} X^{i_2} \cdots X^{i_l}$ with coupling constant g. We then have

$$\begin{aligned} \Big \langle \frac{d P_i}{d t} \Big \rangle= & {} - \, g \, l \, \delta _{i j \cdots j} \, \big \langle \left( X^j \right) ^{ l - 1 } \big \rangle . \end{aligned}$$

(48)

On the other hand, operating the commutative time derivative on ${{\mathcal {P}}}_i$ yields

$$\begin{aligned} \frac{d \, {{\mathcal {P}}}_i }{d t_{c}}= & {} - \, \frac{\partial \, V \left( \{ {{\mathcal {X}}}^j \} \right) }{\partial \, {{\mathcal {X}}}^i } \, = \, - g \, l \, \delta _{i j \cdots j} \, \left( {{\mathcal {X}}}^j \right) ^{l - 1} \, = \, - g \, l \, \delta _{i j \cdots j} \, \frac{1}{N^{l - 1}} \big \langle X^j \big \rangle ^{l - 1}, \end{aligned}$$

(49)

where the last equality holds in the large N limit.

Now, when $l = 2$, we have

$$\begin{aligned} \frac{1}{N} \Big \langle \frac{d P_i}{d t} \Big \rangle \, \rightarrow \, \frac{d \, {{\mathcal {P}}}_i }{d t_{c}} \end{aligned}$$

(50)

in the large N limit; thus replacing the expectation value of the OM time derivative of $P_i$ with the commutative time derivative of the expectation value of $P_i$ in the continuum limit. For other values of l, this relation only holds modulo statistical (higher order) corrections. Hence, implementing Eq. (50) in Eq. (47) with the above-mentioned polynomial potential yields Newton’s law for the commutative variable ${{\mathcal {P}}}_i$ upon applying expectation values of the first kind to the OM equation of motion and taking the large N limit (when $l = 2$ in the polynomial potential).

Equations (41) and (47) are Ehrenfest-like theorems with respect to expectation values of the first kind in OM. In the large N limit, these equations are equivalent to standard Newton’s laws in the case of a quadratic Hamiltonian.

On the other hand, the origins of the classical Hamilton’s equations can be directly traced to the Lie algebra-preserving homomorphism from OM to the commutative setting where operators map to their expectation values of the first kind and Lie brackets map to Poisson brackets.

Furthermore, subsequent homomorphisms from expectation values of the first kind to those of the second kind lead to other realizations of Ehrenfest-like theorems. We will now examine these. Let us first consider the case for Koopman–von Neumann mechanics, where expectation values depend on wave functions in the Hilbert space ${{\mathcal {H}}}_{KvN}$. There are two ways to apply the two kinds of expectation values. The first way involves starting with the equation of motion in OM and applying successive expectation values of the first and second kind to get

$$\begin{aligned} \langle \langle \frac{d X^i}{d t} \rangle \rangle= & {} \frac{\eta ^{i j}}{m} \langle \langle P_j \rangle \rangle \end{aligned}$$

(51)

which, in the large N limit, is equivalent to

$$\begin{aligned} \frac{d \, \langle {{\mathcal {X}}}^i \rangle }{d t_{c}} \, = \, \frac{1}{m} \langle {{\mathcal {P}}}_i \rangle \end{aligned}$$

(52)

computed using Poisson brackets in term of variables $\langle \langle X^i \rangle \rangle$ and $\langle \langle P_j \rangle \rangle$; and the Hamilton function of an “average” particle from a classical ensemble

$$\begin{aligned} H_{avg} \, = \, \frac{1}{2 \, m} \sum _{i = 1}^n \langle \mathcal{P}_i \rangle ^2 \, + \, V \left( \{ \langle {{\mathcal {X}}}^i \rangle \} \right) , \end{aligned}$$

(53)

where the $\langle \langle X^i \rangle \rangle$ and $\langle \langle P_j \rangle \rangle$ are expectation values with respect to the Koopman–von Neumann wave function $\psi _{KvN}$.

Likewise, for the time derivative of the momentum operator, we have the following Ehrenfest-like theorem:

$$\begin{aligned} \langle \langle \frac{d P_i}{d t} \rangle \rangle= & {} - \, \langle \langle \frac{\partial \, V( \{ X^i \} ) }{\partial X^i} \rangle \rangle \end{aligned}$$

(54)

which in the special case of a fully quadratic Hamiltonian is equivalent to (in the large N limit)

$$\begin{aligned} \frac{d \, \langle {{\mathcal {P}}}_i \rangle }{d t_{c}} \, = \, - \frac{\partial \, V \left( \{ \langle {{\mathcal {X}}}^i \rangle \} \right) }{\partial \, \langle {{\mathcal {X}}}^i \rangle } \end{aligned}$$

(55)

for an “average” particle from a classical ensemble.

The other way to apply the two kinds of expectation values would be to start with ${{\mathcal {X}}}$ and ${{\mathcal {P}}}$, which are already expectation values of the first kind, and apply expectation values of the second kind to their time derivatives (with respect to the wave function $\psi _{KvN}$). In Koopman–von Neumann mechanics, ${{\mathcal {X}}}$ and ${{\mathcal {P}}}$ also play the role of classical operators, whose time derivatives are defined via the Poisson bracket. With this prescription, we get

$$\begin{aligned} \langle \frac{d \, {{\mathcal {X}}}^i }{d t_{c}} \rangle \, = \, \frac{1}{m} \langle {{\mathcal {P}}}_i \rangle \qquad \qquad \langle \frac{d \, \mathcal{P}_i }{d t_{c}} \rangle \, = \, - \langle \frac{\partial \, V \left( \{ {{\mathcal {X}}}^i \} \right) }{\partial \, {{\mathcal {X}}}^i } \rangle . \end{aligned}$$

(56)

Furthermore, notice that in the so-called Schrödinger picture, operators carry a time-dependence, but states do not. We can therefore pull the time derivatives out of the expectation value operation (defined in terms of $\psi _{KvN}$) to get

$$\begin{aligned} \langle \frac{d \, {{\mathcal {X}}}^i }{d t_{c}} \rangle \, = \, \frac{d \, \langle {{\mathcal {X}}}^i \rangle }{d t_{c}} \qquad \qquad \langle \frac{d \, {{\mathcal {P}}}_i }{d t_{c}} \rangle \, = \, \frac{d \, \langle \mathcal{P}_i \rangle }{d t_{c}}. \end{aligned}$$

(57)

Substituting this into Eq. (56) above gives the standard Ehrenfest’s theorem for Koopman–von Neumann mechanics. In the next section we show that similar considerations yield the standard form of Ehrenfest’s theorem for the quantum-mechanical wave function $\psi _{QM}$.

In summary, the Ehrenfest’s theorem and its related manifestations all originate from the operator form of Hamilton’s equations and successive Lie algebra-preserving homomorphisms. The insight that this exercise gives is that Hamilton’s equations are not merely classical laws, but universal laws of dynamics capturing evolution of symplectic variables, which manifest at the level of the pre-quantum operator algebra, the classical commutative algebra (for individual particles), the classical ensemble algebra (for Koopman–von Neumann distributions), and the algebra of quantum mechanical observables (we shall see this case in the following section). In fact, in [16], it was even suggested that the Ehrenfest’s theorem should be thought of as more fundamental than both, the Koopman–von Neumann’s as well as the Schrödinger’s equation; and those authors seek to derive the latter two equations from a version of Ehrenfest’s theorem. Here, we will demonstrate a different route to obtaining Koopman–von Neumann’s and Schrödinger’s equation, which nonetheless makes use of Hamilton’s equations in one of its many guises.

7 From Koopman–von Neumann’s to Schrödinger’s equation

In Sect. 5, we identified the Koopman–von Neumann wave function $\psi _{KvN}$ as belonging to the algebra of functions $f \left( {{\mathcal {X}}}, {{\mathcal {P}}} \right)$ over continuum variables ${{\mathcal {X}}}$ and ${{\mathcal {P}}}$, where the latter are obtained as expectation values of the first kind associated with operators X, respectively, P of the OM pre-quantum algebra⁷. In this section, we discuss how $\psi _{KvN}$ relates to the quantum wave function $\psi _{QM}$, and in the process, we will recover Schrödinger’s equation from the classical Koopman–von Neumann equation.

Geometrically, $\psi _{KvN}$ forms a section of the phase space bundle ${{\mathcal {E}}}$, and the space of these sections along with the norm induced from $L^2$ functions yields the Hilbert space $\mathcal{H}_{KvN}$. In that sense, the origins of $\psi _{KvN}$ can be traced to the OM pre-Hilbert space ${{\mathcal {H}}}_{pre}$ of operators and successive homomorphisms bring us to the familiar space of wave functions ${{\mathcal {H}}}_{KvN}$.

Analogously, the same is true for the wave function $\psi _{QM}$ in quantum mechanics and its familiar home in ${{\mathcal {H}}}_{QM}$, with the caveat that the former depends only on ${{\mathcal {X}}}$ and not on ${{\mathcal {P}}}$. We will trace this subtle difference between $\psi _{KvN}$ and $\psi _{QM}$ to a quantization scheme interpreted as a “twisting” of sections over ${{\mathcal {E}}}$. We will first write down the Koopman–von Neumann equation for $\psi _{KvN}$ and then the Schrödinger equation from it via quantization.

Note that the Koopman–von Neumann equation follows from Liouville’s theorem, which posits the constancy of the phase space distribution function $\tilde{\rho } ({{\mathcal {X}}}, {{\mathcal {P}}}; t)$ along any trajectory. Expressing $\tilde{\rho } ({{\mathcal {X}}}, {{\mathcal {P}}}; t)$ in terms of a complex-valued function $\psi _{KvN}$ as $\tilde{\rho } ({{\mathcal {X}}}, {{\mathcal {P}}}; t) = \psi _{KvN}^*\, \psi _{KvN}$ yields the Koopman–von Neumann equation (and its conjugate). Hence, we have

$$\begin{aligned} \frac{\partial \, \psi _{KvN} ({{\mathcal {X}}}, {{\mathcal {P}}}) }{\partial t_{c}}= & {} - \sum _{i = 1}^n \left( \frac{d {{\mathcal {X}}}^i }{d t_{c}} \, \frac{\partial \, \psi _{KvN} }{\partial \, {{\mathcal {X}}}^i } \, + \, \frac{d {{\mathcal {P}}}_i }{d t_{c}} \, \frac{\partial \, \psi _{KvN} }{\partial \, {{\mathcal {P}}}_i } \right) . \end{aligned}$$

(58)

For the time derivatives of ${{\mathcal {X}}}^i$ and ${{\mathcal {P}}}_i$, we then substitute the version of Hamilton’s equations for a commutative algebra. This yields precisely the Poisson bracket in terms of the classical Hamilton function $H_{cl}$ on the right-hand side

$$\begin{aligned} \frac{\partial \, \psi _{KvN} }{\partial t_{c}}= & {} \big \{ H_{cl}, \, \psi _{KvN} \big \}_{PB}, \end{aligned}$$

(59)

which is the familiar form of the Koopman–von Neumann equation, expressed in terms of the Liouville operator, acting on the classical wave function $\psi _{KvN}$. The Koopman–von Neumann equation pertains to classical mechanics, albeit from a statistical perspective (with the classical Liouville equation expressing the statistical density function). Notice the important role Hamilton’s equations play here. In our framework, Hamilton’s equations result from mapping structures in ${{\mathcal {L}}}$ to the corresponding commutative setting, following which the wave function $\psi _{KvN}$ originates from sections in ${{\mathcal {E}}}$.

Of course, there are also other ways to derive or generalize the Koopman–von Neumann equation (for instance, see [16, 44]). However, here, our objective is in investigating the underlying algebraic structures behind this equation and, ultimately, its relation to the Schrödinger’s equation.

We now show that a simple quantization map applied to the Koopman–von Neumann equation leads to Schrödinger’s equation. To see how this works, let us recall the operators identified earlier in Sect. 5, which act on sections of the bundle $\mathcal{E}$. These are ${{\mathcal {X}}}^i$, ${{\mathcal {P}}}_i$, $\tilde{\kappa } \big \{ {{\mathcal {X}}}^i, \, \bullet \big \}_{PB}$ and $\tilde{\kappa } \big \{ {{\mathcal {P}}}_i, \, \bullet \big \}_{PB}$. They satisfy the algebra given in Eq. (33). Notice that this algebra contains remnants of both classical and quantum mechanics and is applicable to generic sections within the bundle ${{\mathcal {E}}}$. To understand what constitutes a classical versus a quantum wave function, the key issue is to identify the relevant observables. Given that the wave function represents the physical state of the system, it should only depend on quantities that are observable. The classical wave function $\psi _{KvN}$ only depends on ${{\mathcal {X}}}^i$ and ${{\mathcal {P}}}_i$ corresponding to the observable position and momentum operators. And given that these operators commute, they are simultaneously diagonalizable, resulting in joint eigenstates of ${{\mathcal {X}}}^i$ and ${{\mathcal {P}}}_i$ for the wave function $\psi _{KvN}$.

The key observation that guides us in transitioning to the quantum wave function $\psi _{QM}$ is that the observable momentum operator from the algebra in Eq. (33) is now $\tilde{\kappa } \big \{ {{\mathcal {P}}}_i, \, \bullet \big \}_{PB}$ and not ${{\mathcal {P}}}_i$. And given that ${{\mathcal {X}}}^i$ does not commute with the new momentum operator, these two observables are complementary. Hence, $\psi _{QM}$ can be expressed either in the eigenbasis of $\mathcal{X}^i$ or $\tilde{\kappa } \big \{ {{\mathcal {P}}}_i, \, \bullet \big \}_{PB}$, with the former choice often referred to as a “projection to configuration space”. From the above perspective, quantization is simply a mapping from one sub-algebra of Eq. (33) to another

$$\begin{aligned} {{\mathcal {P}}}_i \rightarrow \frac{\tilde{\kappa }}{2} \big \{ {{\mathcal {P}}}_i, \, \bullet \big \}_{PB} \qquad \qquad \tilde{\kappa } \big \{ {{\mathcal {X}}}^i, \, \bullet \big \}_{PB} \rightarrow {{\mathcal {X}}}^i, \end{aligned}$$

(60)

which from the point of view of sections of the bundle ${{\mathcal {E}}}$ can be geometrically interpreted as “twisting” a classical section that can be deformed in the directions of ${{\mathcal {X}}}^i$, $\mathcal{P}_i$, $\tilde{\kappa } \big \{ {{\mathcal {X}}}^i, \, \bullet \big \}_{PB}$ and $\tilde{\kappa } \big \{ {{\mathcal {P}}}_i, \, \bullet \big \}_{PB}$ to a quantum section that can only be deformed in the directions of ${{\mathcal {X}}}^i$ and $\tilde{\kappa } \big \{ \mathcal{P}_i, \, \bullet \big \}_{PB}$ (which forces the projection of the wave function to configuration space). The quantized system does not depend on ${{\mathcal {P}}}_i$ anymore, and this is now not the physical momentum. Instead, $\tilde{\kappa } \big \{ {{\mathcal {P}}}_i, \, \bullet \big \}_{PB}$ is the new momentum (operator).

Implementing the quantization scheme above into the Koopman-von Neumann equation (Eq. (59)) directly leads to

$$\begin{aligned} \frac{\partial \, \psi _{QM} ( {{\mathcal {X}}}, \tilde{\kappa } \big \{ {{\mathcal {P}}}, \, \bullet \big \}_{PB} ) }{\partial t_{c}}= & {} - \sum _{i = 1}^n \left( \frac{\tilde{\kappa }}{2 m} \big \{ \mathcal{P}_i, \, \bullet \big \}_{PB} \, \frac{\partial \, \psi _{QM} }{\partial \, {{\mathcal {X}}}^i } \, - \, \frac{\partial \, V ({{\mathcal {X}}}) }{\partial \, {{\mathcal {X}}}^i } \, \frac{ {{\mathcal {X}}}^i }{\tilde{\kappa }} \, \psi _{QM} \right) , \end{aligned}$$

(61)

which when $\tilde{\kappa } = i \, \hbar$, is exactly the Schrödinger’s equation

$$\begin{aligned} \tilde{\kappa } \, \frac{\partial \, \psi _{QM} }{\partial t_{c}}= & {} \sum _{i = 1}^n \frac{\eta ^{i j} \, \tilde{\kappa }^2 }{2 m} \big \{ {{\mathcal {P}}}_i, \, \bullet \big \}_{PB} \, \big \{ \mathcal{P}_j, \, \bullet \big \}_{PB} \, \psi _{QM} \, + \, \widetilde{V} ({{\mathcal {X}}}) \, \psi _{QM}, \end{aligned}$$

(62)

with a shifted potential

$$\begin{aligned} \widetilde{V} ({{\mathcal {X}}}) \, = \, - V ({{\mathcal {X}}}) \, + \, \sum _{i = 1}^n \frac{\partial \, V ({{\mathcal {X}}}) \cdot {{\mathcal {X}}}^i }{\partial \, {{\mathcal {X}}}^i }. \end{aligned}$$

(63)

For polynomial potentials, this shift amounts to a scaling of couplings of the potential. For instance, consider the case with $i = 1$. Then

$$\begin{aligned} \sum _{k = 1}^m a_k \, ( {{\mathcal {X}}}^1 )^k \, \rightarrow \, \sum _{k = 1}^m \, k \, a_k \, ( {{\mathcal {X}}}^1 )^k. \end{aligned}$$

(64)

Hence, with the above quantization scheme, we get the following Hamiltonian operator for the quantized system:

$$\begin{aligned} H_{qm} \, = \, \frac{\eta ^{i j}}{2 \, m} \sum _{i = 1}^n \tilde{\kappa }^2 \big \{ {{\mathcal {P}}}_i, \, \bullet \big \}_{PB} \, \big \{ {{\mathcal {P}}}_j, \, \bullet \big \}_{PB} \, + \, \widetilde{V} \left( \{ {{\mathcal {X}}}^i \} \right) . \end{aligned}$$

(65)

The Hamiltonian flow for operators acting on ${{\mathcal {H}}}_{QM}$ is associated to the Hamiltonian $H_{qm}$. This flow is induced by the symplectic structure governing the canonical commutation relations of the algebra of observables ${{\mathcal {X}}}^i$ and $\tilde{\kappa } \big \{ {{\mathcal {P}}}_i, \, \bullet \big \}_{PB}$ (which itself is a sub-algebra of the algebra in Eq. (33)). The resulting operator evolution respects the noncommutative Poisson structure and hence takes the form

$$\begin{aligned} \frac{d F}{d t}= & {} \frac{\partial F}{\partial t} + \frac{1}{\tilde{\kappa }} [ F, H_{qm} ], \end{aligned}$$

(66)

with an additional term $\frac{\partial F}{\partial t}$ for operators that carry an explicit time-dependence. Notice that when $\tilde{\kappa } = i \, \hbar$, this is exactly the Heisenberg equation of motion for F. Using this to compute time derivatives of the quantum operators ${{\mathcal {X}}}^i$ and $\tilde{\kappa } \big \{ {{\mathcal {P}}}_i, \, \bullet \big \}_{PB}$ gives

$$\begin{aligned} \tilde{\kappa } \, \frac{d {{\mathcal {X}}}^i }{d t} \, = \, [ \mathcal{X}^i, H_{qm} ] \, = \, \frac{ \tilde{\kappa }^2 }{ m} \, \big \{ {{\mathcal {P}}}_i, \, \bullet \big \}_{PB} \end{aligned}$$

(67)

$$\begin{aligned} \tilde{\kappa } \, \frac{d \, \tilde{\kappa } \big \{ {{\mathcal {P}}}_i, \, \bullet \big \}_{PB} }{d t} \, = \, [ \tilde{\kappa } \big \{ {{\mathcal {P}}}_i, \, \bullet \big \}_{PB}, H_{qm} ] \, = \, - \tilde{\kappa } \, \frac{\partial \, \widetilde{V} ( \{ {{\mathcal {X}}}^i \} ) }{\partial \, {{\mathcal {X}}}^i }. \end{aligned}$$

(68)

Notice that these are yet again realizations of Hamilton’s equations for the quantum operator algebra, and their expectation values will yield the familiar version of Ehrenfest’s theorem in quantum mechanics. To see this, we apply expectation value of the second kind to the above equations with respect to the quantum-mechanical wave function $\psi _{QM}$ to get

$$\begin{aligned} \langle \frac{d \, {{\mathcal {X}}}^i }{ d t } \rangle \, = \, \frac{1}{m} \langle \tilde{\kappa } \big \{ {{\mathcal {P}}}_i, \, \bullet \big \}_{PB} \rangle \qquad \qquad \langle \frac{d \, \tilde{\kappa } \big \{ {{\mathcal {P}}}_i, \, \bullet \big \}_{PB} }{ d t } \rangle \, = \, - \langle \frac{\partial \, V \left( \{ {{\mathcal {X}}}^i \} \right) }{\partial \, {{\mathcal {X}}}^i } \rangle , \end{aligned}$$

(69)

where working in the Schrödinger picture, the time derivative can be pulled out of the expectation value brackets $\langle \cdots \rangle$ to retrieve the standard form of the Ehrenfest’s theorem.

What we have shown above is that quantization realized as “twisting” of sections of the bundle ${{\mathcal {E}}}$ gives quantum mechanics from Koopman–von Neumann’s classical mechanics. At the level of the algebra Eq. (33), the map Eq. (60) identifying one sub-algebra with another is equivalent to performing a quotient on the algebra of classical operators to obtain a quantum algebra which is half of the classical one. From this perspective, the well known identification of the quantum momentum operator as a derivative $\hat{p} = - i \, \hbar \, \partial /\partial x$ is simply an instance of this quotient map.

It is worth noting that other quantization schemes slightly different from the one we have introduced here are also plausible when working with modifications or generalizations of the Koopman–von Neumann equation. In particular, a specific generalization of the Koopman–von Neumann equation is considered in [44]. This work generalizes the Liouville operator (also suggesting an infinite family of possible generalizations based on integrability). Upon considering one such choice of a generalized Koopman–von Neumann equation, the authors apply the following quantization map to recover the time-dependent Schrödinger equation:

$$\begin{aligned} {{\mathcal {P}}}_i \rightarrow \tilde{\kappa } \, \big \{ {{\mathcal {P}}}_i, \, \bullet \big \}_{PB} \qquad \qquad \tilde{\kappa } \, \big \{ \mathcal{X}^i, \, \bullet \big \}_{PB} \rightarrow 0. \end{aligned}$$

(70)

This form of quantization was identified as “polarization” [44, 57] and was interpreted as effecting a projection of the classical wave function to configuration space, thus yielding the quantum wave function. In contrast, we have considered the standard form of the Koopman–on Neumann equation and observed that this is enough to recover the Schrödinger equation. Apart from the aforementioned works, yet another operator approach was pursued in [16] which was shown to yield both, the Koopman–von Neumann equation as well as the Schrödinger equation. Even though [16] do not directly delve into the issue of quantization, the algebra of operators upon which their Liouville operator and their Hamiltonian depend on shows the same feature we find here via quantization: that the quantum algebra is half of the classical algebra.

8 Conclusions and outlook

8.1 Summary

In this work, we have argued that an abstract noncommutative operator algebra defined by a noncommutative Poisson structure, a symplectic structure,and a noncommutative differential structure serves as a plausible candidate for a pre-quantum theory. To the extent that no a priori assumption of a Hilbert space of states or wave functions is even necessary to start with. All those aforementioned structures are constructions built upon our pre-quantum operator algebra, which may therefore be viewed as a foundational starting point of a quantum theory. This formalism was referred to as Operator Mechanics (OM). OM serves as a pre-quantum algebra from which algebraic structures relevant to real-world classical and quantum mechanics follow. In this work, we have shown systematic constructions of homomorphisms from OM to classical structures, to quantum structures. In particular, the classical Koopman–von Neumann wave function as well as the quantum wave function are both consequences of this pre-quantum formalism.

With OM, we derived the operator versions of Hamilton’s equations and Newton’s law of motion. These follow from the underlying noncommutative differential and symplectic structures. We then identified a pre-Hilbert space ${{\mathcal {H}}}_{pre}$ within this operator algebra, which enabled us to define a density operator based purely on operators. This allowed for a computation of expectation values of operators without invoking any notion of wave functions or states. These were referred to as expectation values of the first kind. Using this operation, expectation values of $X^i$ and $P_i$ in the large N limit (of operator dimension) led to coordinates ${{\mathcal {X}}}^i$ and ${{\mathcal {P}}}_i$ which provide the spatial support for an algebra of functions $f \left( {{\mathcal {X}}}, {{\mathcal {P}}} \right)$ over these variables. The coordinates $({{\mathcal {X}}}^i, \, {{\mathcal {P}}}_i)$ define a phase space and the above-mentioned functions can be thought of as sections over the phase space bundle ${{\mathcal {E}}}$. The classical Koopman–von Neumann wave function $\psi _{KvN}$ is precisely such a section of ${{\mathcal {E}}}$, and the space of these sections along with the norm induced from $L^2$ functions, yields the Hilbert space ${{\mathcal {H}}}_{KvN}$. In this sense, the origins of $\psi _{KvN}$ can be traced to the OM pre-Hilbert space $\mathcal{H}_{pre}$ of operators. Successive homomorphisms then take us to the familiar space of wave functions ${{\mathcal {H}}}_{KvN}$ and the algebra of observables acting on ${{\mathcal {H}}}_{KvN}$. Along the way we also encountered the Hamilton’s equations in various guises; from the operator version, to the classical commutative version, to those appearing in Ehrenfest’s theorems; suggesting a certain functorial property of these equations that is preserved across algebraic structures.

An analogous picture emerges for the quantum-mechanical wave function $\psi _{QM}$ and its familiar home in ${{\mathcal {H}}}_{QM}$, with the caveat that the former depends only on ${{\mathcal {X}}}$ and not on ${{\mathcal {P}}}$. We showed that the difference between $\psi _{KvN}$ and $\psi _{QM}$ comes from a quantization scheme interpreted as “twisting” of sections over ${{\mathcal {E}}}$. Alternatively, this twisting can also be viewed as the quantum algebra of observables being half the classical one. Using such a quantization map, we showed that the Schrödinger equation itself can be obtained starting from the Koopman–von Neumann equation. What all this suggests is that neither the Schrödinger equation nor the quantum-mechanical wave function are fundamental structures. All these are simply built upon a pre-quantum operator algebra and ensuing algebraic homomorphisms.

Below, we summarize the key algebraic structures involved in OM, classical mechanics and quantum mechanics. This comparison also serves to illustrate the role of OM as a pre-quantum operator algebra from which real-world classical and quantum mechanics follow upon loosening specific facets of its algebraic structure:

• Operator Mechanics (OM) is formalized using a noncommutative Poisson structure, a symplectic structure, and a noncommutative differential structure.

• Classical Mechanics on the other hand is formalized using a commutative Poisson structure, a symplectic structure, and a commutative differential structure from standard analysis. The transition from OM to classical mechanics involves homomorphisms from the noncommutative Poisson structure of OM to a commutative Poisson structure, and from noncommutative calculus to the classical one.

• Quantum Mechanics is also formalized using a noncommutative Poisson structure and a symplectic structure. However, the differential structure realized here is, in part, that of classical calculus inherited from a projection of classical phase space onto configuration space. While the transition from OM to quantum mechanics retains noncommutativity of the Poisson structure, it partly loses OM’s noncommutative differential structure. The latter is because the differential operator $\partial / \partial \mathcal{X}^i$ in quantum mechanics is the ordinary derivative of configuration space (in contrast to $\partial / \partial X^i$, the noncommutative derivative in OM). The Schrödinger operator in quantum mechanics is expressed in terms of this differential operator from classical analysis.

Note that the above-mentioned noncommutative differential structure of OM strongly suggests an underlying noncommutative pre-geometry associated to this operator algebra. This noncommutative pre-geometry would presumably be required for investigating structures relevant to classical and quantum theories of gravity.

8.2 A route to quantum field theories using matrix models

Our work here suggests several possible routes of extension. The obvious ones being formalizing OM for relativistic quantum mechanics and deriving the Klein–Gordon equation (see [22] for some recent progress in this direction). Another interesting possibility would be to compute sub-leading 1/N corrections to the Schrödinger equation. Yet another exciting extension would be to formalize an operator algebraic field theory using matrix models. Let us outline this prospective application of OM.

The objective in this case would be to construct an operator algebraic formalization of quantum field theories without assuming an a priori classical geometry or Hilbert space. We present the outline of the idea and will soon report details in an upcoming work. A plausible extension of OM to a field theory may be realized by considering, instead, an ensemble of Hermitian operators $X^i$ and $P_i$ defined with respect to an appropriate measure. In fact, a precise realization of such an ensemble and its associated measure can be obtained by representing $X^i$ and $P_i$ as $N \times N$ Hermitian matrices. We propose that the path integral of this field theory is a special type of matrix model. In fact, this is what we will call a ‘Constrained One-dimensional 2n-Matrix Model’. The constraints refer to the OM canonical conjugation relations between the 2n matrix operators $X^i$ and $P_i$. The action of this one-dimensional matrix model for the non-relativistic case can be expressed as

$$\begin{aligned} S_N [\phi (X)] = N \, Tr \int dt \left( \frac{m}{2} \dot{\phi }(X)^2 \, - \, V \left( \phi (X) \right) \right) , \end{aligned}$$

(71)

where the OM constraints corresponding to the canonical conjugation relations and subsequent partial derivatives have been used to cast the above action $S_N$ in terms of $\phi (X)$ and $\dot{\phi }(X)$. The integral above is a formal antiderivative defined using the operator inverse of Eq. (14). As indicated, the constraints due to the canonical commutation relations help remove the $\chi (P)$ dependence from the action, thus simplifying the problem. The operators $\phi (X)$ are chosen as elements of the pre-Hilbert space ${{\mathcal {H}}}_{pre}$ satisfying Eq. (22). The partition sum for this action is the following matrix integral:

$$\begin{aligned} {{\mathcal {Z}}}_N = \int {{\mathcal {D}}} \phi (X) \, {{\mathcal {D}}} \chi (P) \, \, e^{\frac{i}{\hbar } S_N [\phi (X)]}. \end{aligned}$$

(72)

The measure ${{\mathcal {D}}} \phi (X) \, {{\mathcal {D}}} \chi (P)$ runs over Hermitian matrices $X^i$ and $P_i$, as well as the space of coefficients of the relevant ${{\mathcal {R}}}$-module.

A special case of the above matrix model, where one sets $\phi (X) \rightarrow X^i$, would give precisely the standard one-dimensional matrix model in $X^i$ (see [3]). The above path integral is a natural extension of OM to a field theoretic setting. Presumably, the large N limit of this matrix model approximates certain classes of low-energy scattering amplitudes of a scalar field theory, and its 1/N corrections might help open a window into non-perturbative contributions in a manner similar to previously known correspondences of matrix models to topological field theories [27]. Let us mention another closely related matrix dynamics approach which also seeks to formulate quantum field theories and ultimately gravity without classical spacetimes is Adler’s ‘trace dynamics’ [2, 50]. This is based on Grassmann-valued matrix operators rather than Hermitian matrices. Like OM, this program also posits noncommutativity at the fundamental scale. It will be interesting to explore connections of OM to trace dynamics in follow-up work.

8.3 Connections to noncommutative geometry

As suggested earlier, a potential avenue for future exploration would be possible connections to Noncommutative Geometry (NCG) [25]. Even though the objectives of our work in this paper did not concern geometric considerations, the OM formalism shares conceptual similarities to Connes’ noncommutative geometry, in that differential operators are defined as commutators with respect to a compact operator. The central thesis of Connes’ work seeks to generalizes the Gelfand-Naimark Theorem and posits that geometric data can be fully constructible from the $\mathcal{C}^\star$ algebra of noncommutative operators over a Hilbert space. Unlike NCG, in OM, we assume less than the full spectral triple (starting with only the operator algebra and a designated Hamiltonian), and obtain the Hilbert space of states as a by-product of the algebra. In a sense, this means OM is agnostic to the choice of representations of the algebra, which can have interesting implications for exploring the moduli space of possible geometries.

Besides the NCG program, other realizations of noncommutative geometry include ‘Sub-Riemannian Geometries’ associated with Carnot–Caratheodory groups [18, 19]. The Heisenberg group being a special case of a Carnot-Caratheodory group, potentially suggests a sub-Riemannian geometry for modular spaces associated to representations of the Heisenberg algebra discussed in [33].

8.4 Relation to quantum foundations and spaces from entanglement

In the current work, we have not said anything about the various interpretations of quantum mechanics or the measurement problem. Rather the focus of our work here has been on elucidating formal structures that provide the foundations for spaces upon which contemporary formulations of classical and quantum mechanics are described. It is hoped that such structural foundations may shed insights on on-going debates concerning ontological questions, such as wave function realism, fundamentality of Hilbert spaces, and emergence of classical spaces via entanglement. For now, let us comment only on these three issues.

Recall that wave functions are not exclusive to quantum mechanics. This was already demonstrated since the work of Koopman and von Neumann in 1932 [45, 48]. What we have emphasized in our work here is that both the classical and quantum wave functions $\psi _{KvN/QM}$ are sections of the same phase space bundle ${{\mathcal {E}}}$. The difference between the two being a geometric twist of sections, otherwise interpreted as a quantization scheme on the algebra of observables. Furthermore, the Hilbert space ${{\mathcal {H}}}_{KvN/QM}$ in which these wave functions reside is again not exclusive to quantum mechanics. In this view, neither the wave functions $\psi _{KvN/QM}$ nor their associated Hilbert spaces ${{\mathcal {H}}}_{KvN/QM}$ are fundamental structures. Rather, these are merely “front-ends” supported on a tower of structures built upon a pre-quantum algebra. Both $\psi _{KvN}$ and $\psi _{QM}$ belong to the algebra of functions over phase space (${{\mathcal {X}}}$ and ${{\mathcal {P}}}$), respectively, configuration space ($\mathcal{X}$) variables, which themselves are obtained as expectation values of the first kind associated with operators X and P of the OM pre-quantum algebra. The familiar Hilbert spaces ${{\mathcal {H}}}_{KvN}$ and ${{\mathcal {H}}}_{QM}$ arise from the space of these sections of the phase space bundle ${{\mathcal {E}}}$ along with the norm induced from $L^2$ functions. Given that the operator expectation values of the first kind originate from a pre-Hilbert space ${{\mathcal {H}}}_{pre}$ of operators in OM, the origins of both, the classical and quantum wave functions $\psi _{KvN}$ and $\psi _{QM}$ can be traced to this pre-Hilbert structure and successive algebraic homomorphisms. Under this unified perspective, the Koopman–von Neumann’s and Schrödinger’s equation, both appear on an equal footing. We showed how the latter may be viewed as a quantization of the former.

What insights does our framework provide on emergence of conventional spaces upon which physics is “operationalized”? In particular, we noted that operator expectation values of the first kind defined in OM led to a characterization of phase space involving variables ${{\mathcal {X}}}$ and ${{\mathcal {P}}}$ as coordinates. However, this is just a set of “points” (${{\mathcal {X}}}$, ${{\mathcal {P}}}$). How do topological or spatial structures such as a metric get endowed upon this set? One possible route is via entanglements of blocks or modes of the operators $X^i$ and $P_i$. To see how this works, let us recall the density operator in ${{\mathcal {H}}}_{pre}$ expressed in the basis of the algebra ${{\mathcal {L}}}$

$$\begin{aligned} \hat{\rho } = \frac{1}{\kappa ^2 } \sum _i^n w_i \, | \, \overline{X^i} \, ] \, [ \, X^i \, |. \end{aligned}$$

(73)

For operators of dimension N (as before, thinking of them as $N \times N$ matrices), $\hat{\rho }$ is an $N \times N$ matrix. To quantify the entanglement entropy between operator blocks, we need a reduced density matrix $\hat{\rho }_{M}$ corresponding to an $M < N$ dimensional eigenmode of the matrix operators. This can be expressed by way of a partial trace over the complementary $N - M$-dimensional operator modes $\overline{M}$

$$\begin{aligned} \hat{\rho }_{M} = Tr_{\overline{M}} \, \hat{\rho }, \end{aligned}$$

(74)

where the partial trace here involves $\overline{M}$-dimensional operator eigenmodes in ${{\mathcal {H}}}_{pre}$ to obtain an $M \times M$ reduced density matrix. Using this operator, the entanglement entropy between operator eigenmodes M and $\overline{M}$ is given by

$$\begin{aligned} S_M = - \, Tr_{M} \, \left( \hat{\rho }_{M} \, log \, \hat{\rho }_{M} \right) . \end{aligned}$$

(75)

Given $S_M$, one readily obtains the mutual information $I (M_1: M_2)$ between any two blocks $M_1$ and $M_2$. While it is known that mutual information itself is not a metric, using mutual information and entropy, one can define several distance functions between blocks $M_1$ and $M_2$, such as

$$\begin{aligned} d (M_1, M_2) = S_{M_1, M_2} - I (M_1: M_2) \end{aligned}$$

(76)

known as Variation of Information [46]; or

$$\begin{aligned} d (M_1, M_2) = 1 - \frac{ I (M_1: M_2) }{ S_{M_1, M_2} } \end{aligned}$$

(77)

known as the Rajski distance [49], which is closely related to the Jaccard distance from a set-theoretic interpretation. In our particular case, by considering one-dimensional operator blocks M, expectation values of the first kind corresponding to operators $X^i$ and $P_i$ led to $n \times N$ points $\left( \langle X^i \rangle , \langle P_i \rangle \right)$ denoting a phase space. Now, the above definitions of distance endow this set with the structure of a metric space. And in the large N limit of operator dimension, we obtain the continuum limit of this space with coordinates (${{\mathcal {X}}}$, ${{\mathcal {P}}}$). This is how we obtain phase space from entanglement starting from eigenmodes of operators in the pre-Hilbert space ${{\mathcal {H}}}_{pre}$ of OM.

The entanglement entropies of the operator blocks defined above do not depend on states of any system, but follow exclusively from the operator algebra. A similar argument proposing emergence of spaces from entanglement was made in [20, 21], with the important difference that here we do not even need states, wave functions, or the Hilbert space of standard quantum mechanics. In light of the program being pursued in [20], as to ‘What might be the minimal starting point of quantum mechanics?’, our investigations here show that the rabbit-hole of minimalism seems to run much deeper that was previously anticipated.

8.5 Connections to geometry from graphical and category-theoretic models

Operator algebraic approaches such as OM and its extensions may also be useful tools for formally bridging discrete models of quantum geometry to their continuum limits, and in the process systematically extract non-perturbative corrections to continuum observables stemming from the underlying discretization. This is because by themselves operator algebras are neither continuous nor discrete. The topology that the spectrum of operators inherits depends on the representation one works with, and this can be either continuous or discrete with formal techniques to understand transitions from one regime to another.

In particular, operator algebras similar to the ones we have discussed here may have potential foundational implications for investigating geometry from random matrix models [6, 8, 31], emergent spacetime from entanglement entropy and dualities [4, 5, 51, 52], geometric structures and limiting behaviors of random graphs and tensor networks [28, 30], category-theoretic approaches to quantum mechanics [24], and combinatorial models of spacetime based on rewriting systems such as the Wolfram model [7, 10,11,12, 29, 55, 56]. Since we have already mentioned about connections to matrix models and emergent spaces from entanglement earlier, let us now briefly comment on how operator algebras may serve as useful tools for probing (emergent) geometry from graphical and combinatorial models. A key feature common to the latter is that they are based on diagrammatic calculi (as examples of diagrammatic reasoning, see [23, 24, 30, 43, 56, 59, 60]) that offer flexible theorem proving and formal interpretability techniques including diagrammatic reasoning and categorical semantics. Applications of diagrammatic approaches include tensor networks and a host of other diagrammatically representable operator calculi used in the investigation of holographic dualities [9, 51], string diagrams used in categorical quantum mechanics and quantum circuit simplification [23, 24], and hypergraph rewriting systems used in discrete models of spacetime and pregeometric spaces appearing in the Wolfram model [7, 10, 11]. Many of these approaches find a common home in category (and higher category) theory.

For instance, consider formal rewriting systems as the ones espoused by the Wolfram model. A rewriting rule such as

$$\begin{aligned} {{\mathcal {G}}}_1 \rightarrow {{\mathcal {G}}}_2 \end{aligned}$$

(78)

involving graphs or hypergraphs, when applied to an initial state generates a multi-history evolution network, respecting local causal structure. This is the Wolfram model ‘Multiway System’. Rewriting rules are unary operations applied on graphs, character strings, or other operators. This resulting evolution sequences are formally comparable to those generated by the operator derivative Eq. (14). The latter also suggests a potential graphical operator playing the role of a Hamiltonian, which would generalize unary rewriting to n-ary operations involving a sequence of graphs (or hypergraphs)

$$\begin{aligned} {{\mathcal {G}}}_1 \times \cdots \times {{\mathcal {G}}}_n \rightarrow {{\mathcal {G}}}_{n+1} \times \cdots \times {{\mathcal {G}}}_m \end{aligned}$$

(79)

as higher order rewriting rules captured via an operator algebra of graph actions. Furthermore, multiway systems are natural constructions to express tensor networks of categorical quantum mechanics and ZX-calculus as was shown in [34,35,36]. In [10,11,12], this categorical formulation of multiway systems was extended n-fold categories and homotopy types for the purpose of constructing synthetic pregeometric spaces. With higher order rewriting rules, what was shown in that work was that such a limiting multiway system was formally identifiable with the $\infty$-groupoid. This nexus of higher category theory, tensor networks, and homotopy type theory suggests new ways to probe synthetic geometry and pregeometric structures as higher categorical constructions.