Quantum hidden Markov models based on transition operation matrices

In this work, we extend the idea of quantum Markov chains (Gudder in J Math Phys 49(7):072105 [3]) in order to propose quantum hidden Markov models (QHMMs). For that, we use the notions of transition operation matrices and vector states, which are an extension of classical stochastic matrices and probability distributions. Our main result is the Mealy QHMM formulation and proofs of algorithms needed for application of this model: Forward for general case and Vitterbi for a restricted class of QHMMs. We show the relations of the proposed model to other quantum HMM propositions and present an example of application.


Introduction
The classical Hidden Markov Model was introduced as a method of modelling signal sources observed in noise.It is now extensively used, e.g. in speech and gesture recognition or biological sequence analysis.Their popularity is a result of their rich structure, which is able to model wide variety of problems and effective algorithms that facilitate their application.The HMM is characterized by three fundamental problems [10]: given a sequence of symbols of length T , O = (o 1 , o 2 , . . ., o T ), and a HMM parametrized by λ, 1. Compute the P (O|λ), probability of the sequence O given a HMM with parameters λ.
2. Select the sequence of states N T = (n 0 , n 1 , . . ., n T ) that maximizes the probability P (O|λ, N T ); in other words the most likely state sequence in HMM λ that produces O.
The above problems are solved, respectively, the Forward, Vitterbi and Baum-Welch algorithms.The effectiveness of those algorithms is based on optimized procedure of computation, which uses a 'trellis': a two dimensional lattice structure of observations and states.This formulation is based on the Markov property of model evolution and reduces the complexity from exponential O(T N T ) to polynomial O(N 2 T ), where T is the number of observations and N the number of model states [10].
Depending on the formulation, there are two definitions of a Hidden Markov Model: Mealy and Moore.In the former, the probability of next state n t+1 depends both on the current state n t and the generated output symbol o t .In the latter, the symbol generation is independent from state switch, i.e.P (S(t + 1) = S i |o t = o, S(t) = S j ) = P (S(t + 1) = S i |S(t) = S j ).While the expressive power of Moore and Mealy models is the same, i.e. process can be realized with Moore model if and only if it is realizable by Mealy model, the minimal model order for the realization is lower in Mealy models [15].In this work we focus only on Mealy models.

Related work
In this work we follow the scheme proposed by Gudder in [5] and extend it in order to construct Quantum Hidden Markov Models (QHMMs).Gudder introduced the notions of Transition Operation Matrices and Vector States, which give an elegant extension of classical stochastic matrices and probability distributions.These notions allows to define Markov processes that exhibit both classical and quantum behaviour.
Below we review two areas of research most closely related to our work: open quantum walks and Hidden quantum Markov models.

Open quantum walks
In recent years a new sub-field of quantum walks has emerged.In series of papers [9,3,12,13,2,14] Attal, Sabot, Sinasky, and Petrucione introduced the notion of Open Quantum Walks.Theorems for limit distributions of open quantum random walks were provided in [6].In [1] the average position and the symmetry of distribution in th SU(2) Open Quantum Walk is studied.
The notion of open quantum walks is generalised to quantum operations of any rank in [8] and analysed in [11].In first of these two papers the notion of mean first passage time for a generalised quantum walk is introduced and studied for class of walks on Apollonian networks.In the second paper a central limit theorem for reducible and irreducible open quantum walks is provided.
Quantum hidden Markov models Hidden quantum Markov models were introduced in [7].The construction provided there by the authors is different from ours.In their work the hidden quantum Markov model consists of a set of quantum operations associated with emission symbols.The evolution of the system is governed by the application of quantum operations on a quantum state.The sequence of emitted symbols defines the sequence of quantum operations being applied on the initial state of the hidden quantum Markov model.

Our contribution
In this work we propose a Quantum Hidden Markov model formulation using the notions of Transition Operation Matrices.We focus on Mealy models, for which we derive first the Forward algorithm in general case, then the Vitterbi algorithm, for models restricted to those with sub-TOMs elements are trace-monotonicity preserving quantum operations.
The paper is organised as follows: in Section 2 we collect the basic mathematical objects and their properties, in Section 3 we define Quantum Hidden Markov Models and provide Forward na Viterbi algorithm for these models, and finally in Section 4 we conclude.

Transition Operation Matrices
In what follows we provide basic elements of quantum information theory and summarize definitions and properties of objects introduced by Gudder in [5].

Quantum theory
Let H be a complex finite Hilbert space and L(H) be the set of linear operators on H.We also denote set of positive operators on H as P + (H) and the set of positive semi-definite operators on H as P(H).
With each measurement µ we associate non-negative functional p : Θ → R + ∪ {0} which maps measurement outcome a for given positive operator ρ and measurement µ to non-negative real number in the following way p(a) ρ = tr µ(a)ρ.If tr ρ = 1, for given ρ and µ the value of p can be interpreted as probability of obtaining measurement outcome a in quantum state ρ.
If ρ is a sub-normalized state the trivial measurement µ : a e → 1 measures the probability p(a e ) ρ = tr ρ that the state ρ exists.One should note that this kind of measurement commutes with any other measurement and thus does not disturb the quantum system.

Transition Operation Matrices
The core object of the Gudder's scheme is Transition Operation Matrix (TOM) which generalizes the idea of stochastic matrix.

Definition 10 (Transition Operation Matrix).
Let H 1 , H 2 denote two finite dimensional Hilbert spaces and Ω(H 1 ), Ω(H 2 ) denote sets of quantum states acting on those spaces respectively.
A TOM is a matrix in form , where E ij is completely positive map in L(L(H 1 ), L(H 2 )) such that for every j and ρ ∈ Ω(H Alternatively one can say that E = {E ij } M,N i,j=1 is a TOM if and only if for every column j i E ij is a quantum channel (Completely Positive Trace Preserving map).A simple implication of this definition is that each E ij is CP-TNI mapping.
Note that in this definition TOM has four parameters: • size of matrix "output" (number of rows) -M, • size of matrix "input" (number of columns) -N, • "input" Hilbert space -H 1 , • "output" Hilbert space -H 2 , The set of TOMs we will denote as Γ M,N (H , where E ij is completely positive map in L(L(H 1 ), L(H 2 )) such that for every j and ρ ∈ Ω(H 1 ).
Definition 12 (Quantum Markov chain).Let a TOM E = {E ij } M,N i,j=1 be given.Quantum Markov chain is a finite directed graph G = (E, V ) labelled by E ij for e ∈ E and by zero operator for e / ∈ E.

Remark 2. Product of two sub-TOMs is a sub-TOM.
Product of TOMs that have same parameters is associative ie.(EF )G = E(F G) and (EF )(α) = E(F (α)).
For any square TOM E ∈ Γ M,M (H, H) whose both input and output Hilbert spaces are equal one can define integer exponent.By E (n) we understand a TOM that is equivalent to n-fold application of TOM E. Vector state

Quantum Hidden Markov Model
In order to explain the idea of QHMM we can form following analogy.QHMM might by understood as a system consisting of a particle that can has an internal quantum state ρ ∈ Ω ≤ and it occupies a classical state S i .This particle hops from one classical state into another passing trough a quantum operation associated with a sub-TOM element P V k S j ,S i .With each transition a symbol V k is emitted from the system.
We will now define the classical and quantum version of the Mealy Hidden Markov Model.

Mealy HMM and QHMM
Definition 15 (Mealy Hidden Markov Model).Let S and V be set of states and an alphabet respectively.Mealy HMM is specified by tuple λ = (S, V, Π, π), where: • π ∈ [0, 1] N is a stochastic vector representing initial states, where π i is probability that initial states is S i ; Definition 16 (Mealy Quantum Hidden Markov Model).Let S and V be set of states and an alphabet respectively.Mealy QHMM is specified by tuple λ = (S, V, P, π), where: A graphical representation of three-state two-symbol Mealy QHMM is presented in Fig. 1.

Properties of QHMMs
Remark 3.For dim H = 1 QHMM reduces to classical HMM.In this case TOMs reduce to stochastic matrices, sub-TOMs to sub-stochastic matrices, the vector states to probability vectors, sub-vector states to sub-normalized probability vectors.
Let us consider sub-normalized vector states α T ∈ Ξ M (H) such that α T,i .Equation (1) we call the Forward algorithm for QHMMs.It allows us to formulate the following theorem: Theorem 1.Let V * be the set of sequences over alphabet V. Let V T ⊂ V * be a set of all sequences of length T .For any QHMM λ we have Proof: We will proceed with induction on T: (*) For T = 1 But P * 1 = o∈V P o is TOM, therefore M i=1 P(π) ∈ Ω(H).
. The sub-normalized state that maximizes trace over set of all B w s with w ∈ N S i k is Then the following holds Proof: Let us denote as the sequence of states maximizing trace of B w , so that We now have Let us assume that it is not true.That would mean, that states that lead to a given emission sequence.We have also proposed a formulation of the Forward algorithm that is applicable for general QHMMs.
Because of the fact that the structure of Quantum Hidden Markov Models is more complicated than their classical counterparts, the most likely sequence of states leading to a given emissions sequence has to be calculated using extensive search.
We believe that proposed model can find applications in modelling systems that posses both quantum and classical features.
that is, probability of going from state i to j while generating the output o.Denote set of all finite strings as O and let o = o 1 o 2 , . . .o T ∈ O. Let then P : O → [0, 1] be string probabilities, defined as P (o) = p(o(1) = o 1 , o(2) = o 2 , . . ., o(T ) = o T ).Of course function P satisfies P (O) = 1 and o A ∈O P (oo A ) = P (o), where oo A is concatenation of strings o and o A .The string probabilities generated by Mealy HMM (S, V, Π, π) are given by Obviously tr A k,S i = tr P o kWe will now prove that for n* k = S i w * k,S i = (n * 0 , . . ., n * k−1 , n * k ) =⇒ w * k−1,n * k−1 = (n * 0 , . . ., n * k−1 )