Coalgebra Learning via Duality

Automata learning is a popular technique for inferring minimal automata through membership and equivalence queries. In this paper, we generalise learning to the theory of coalgebras. The approach relies on the use of logical formulas as tests, based on a dual adjunction between states and logical theories. This allows us to learn, e.g., labelled transition systems, using Hennessy-Milner logic. Our main contribution is an abstract learning algorithm, together with a proof of correctness and termination.


Introduction
In recent years, automata learning is applied with considerable success to infer models of systems and in order to analyse and verify them.Most current approaches to active automata learning are ultimately based on the original algorithm due to Angluin [4], although numerous improvements have been made, in practical performance and in extending the techniques to different models [30].
Our aim is to move from automata to coalgebras [26,14], providing a generalisation of learning to a wide range of state-based systems.The key insight underlying our work is that dual adjunctions connecting coalgebras and tailor-made logical languages [21,12,19,26,22] allow us to devise a generic learning algorithm for coalgebras that is parametric in the type of system under consideration.Our approach gives rise to a fundamental distinction between states of the learned system and tests, modelled as logical formulas.This distinction is blurred in the classical DFA algorithm, where tests are also used to specify the (reachable) states.It is precisely the distinction between tests and states which allows us to move beyond classical automata, and use, for instance, Hennessy-Milner logic to learn bisimilarity quotients of labelled transition systems.
To present learning via duality we need to introduce new notions and refine existing ones.First, in the setting of coalgebraic modal logic, we introduce the new notion of sub-formula closed collections of formulas, generalising suffixclosed sets of words in Angluin's algorithm (Section 4).Second, we import the abstract notion of base of a functor from [8], which allows us to speak about 'successor states' (Section 5).In particular, the base allows us to characterise ⋆ Partially supported by EPSRC grant EP/N015843/1.reachability of coalgebras in a clear and concise way.This yields a canonical procedure for computing the reachable part from a given initial state in a coalgebra, thus generalising the notion of a generated subframe from modal logic.
We then rephrase coalgebra learning as the problem of inferring a coalgebra which is reachable, minimal and which cannot be distinguished from the original coalgebra held by the teacher using tests.This requires suitably adapting the computation of the reachable part to incorporate tests, and only learn 'up to logical equivalence'.We formulate the notion of closed table, and an associated procedure to close tables.With all these notions in place, we can finally define our abstract algorithm for coalgebra learning, together with a proof of correctness and termination (Section 6).Overall, we consider this correctness and termination proof as the main contribution of the paper; other contributions are the computation of reachability via the base and the notion of sub-formula closedness.At a more conceptual level, our paper shows how states and tests interact in automata learning, by rephrasing it in the context of a dual adjunction connecting coalgebra (systems) and algebra (logical theories).As such, we provide a new foundation of learning state-based systems.
Related work.The idea that tests in the learning algorithm should be formulas of a distinct logical language was proposed first in [6].However, the work in loc.cit. is quite ad-hoc, confined to Boolean-valued modal logics, and did not explicitly use duality.This paper is a significant improvement: the dual adjunction framework and the definition of the base [8] enables us to present a description of Angluin's algorithm in purely categorical terms, including a proof of correctness and, crucially, termination.Our abstract notion of logic also enables us to recover exactly the standard DFA algorithm (where tests are words) and the algorithm for learning Mealy machines (where test are many-valued), something that is not possible in [6] where tests are modal formulas.Closely related to our work is also the line of research initiated by [15] and followed up within the CALF project [11,12,13] which applies ideas from category theory to automata learning.Our approach is orthogonal to CALF: the latter focuses on learning a general version of automata, whereas our work is geared towards learning bisimilarity quotients of state-based transition systems.While CALF lends itself to studying automata in a large variety of base categories, our work thus far is concerned with varying the type of transition structures.

Learning by Example
The aim of this section is twofold: (i) to remind the reader of the key elements of Angluin's L * algorithm [4] and (ii) to motivate and outline our generalisation.
In the classical L * algorithm, the learner tries to learn a regular language L over some alphabet A or, equivalently, a DFA A accepting that language.Learning proceeds by asking queries to a teacher who has access to this automaton.Membership queries allow the learner to test whether a given word is in the language, and equivalence queries to test whether the correct DFA has been learned already.The algorithm constructs so-called tables (S, E) where S, E ⊆ A * are the rows and columns of the table, respectively.The value at position (s, e) of the table is the answer to the membership query "se ∈ L?".
Words play a double role: On the one hand, a word w ∈ S represents the state which is reached when reading w at the initial state.On the other hand, the set E represents the set of membership queries that the learner is asking about the states in S. A table is closed if for all w ∈ S and all a ∈ A either wa ∈ S or there is a state v ∈ S such that wa is equivalent to v w.r.t.membership queries of words in E. If a table is not closed we extend S by adding words of the form wa for w ∈ S and a ∈ A. Once it is closed, one can define a conjecture, 3 i.e., a DFA with states in S. The learner now asks the teacher whether the conjecture is correct.If it is, the algorithm terminates.Otherwise the teacher provides a counterexample: a word on which the conjecture is incorrect.The table is now extended using the counterexample.As a result, the table is not closed anymore and the algorithm continues again by closing the table.
Our version of L * introduces some key conceptual differences: tables are pairs (S, Ψ ) such that S (set of rows) is a selection of states of A and Ψ (set of columns) is a collection of tests/formulas.Membership queries become checks of tests in Ψ at states in S and equivalence queries verify whether or not the learned structure is logically equivalent to the original one.A table (S, Ψ ) is closed if for all successors x ′ of elements of S there exists an x ∈ S such that x and x ′ are equivalent w.r.t.formulas in Ψ .The clear distinction between states and tests in our algorithm means that counterexamples are formulas that have to be added to Ψ .Crucially, the move from words to formulas allows us to use the rich theory of coalgebra and coalgebraic logic to devise a generic algorithm.
We consider two examples within our generic framework: classical DFAs, yielding essentially the L * algorithm, and labelled transition systems, which is to the best of our knowledge not covered by standard automata learning algorithms.
For the DFA case, let L = {u ∈ {a, b} * | number of a's mod 3 = 0} and assume that the teacher uses the following (infinite) automaton describing L: As outlined above, the learner starts to construct tables (S, Ψ ) where S is a selection of states of the automaton and Ψ are formulas.For DFAs we will see (Ex. 1) that our formulas are just words in {a, b} * .Our starting table is ({q 0 }, ∅), i.e., we select the initial state and do not check any logical properties.This table is trivially closed, as all states are equivalent w.r.t.∅.The first conjecture is the automaton consisting of one accepting state q 0 with a-and b-loops, whose language is {a, b} * .This is incorrect and the teacher provides, e.g., aa as counterexample.
The resulting table is ({q 0 }, {ε, a, aa}) where the second component was generated by closing {aa} under suffixes.Suffix closedness features both in the original L * algorithm and in our framework (Section 4).The table ({q 0 }, {ε, a, aa}) is not closed as q 1 , the a-successor of q 0 , does not accept ε whereas q 0 does.Therefore we extend the table to ({q 0 , q 1 }, {ε, a, aa}).Note that, unlike in the classical setting, exploring successors of already selected states cannot be achieved by appending letters to words, but we need to locally employ the transition structure on the automaton A instead.A similar argument shows that we need to extend the table further to ({q 0 , q 1 , q 2 }, {ε, a, aa}) which is closed.This leads to the (correct) conjecture depicted on the right below.The acceptance condition and transition structure has been read off from the original automaton, where the transition from q 2 to q 0 is obtained by realising that q 2 's successor q 3 is represented by the equivalent state q 0 ∈ S.
A key feature of our work is that the L * algorithm can be systematically generalised to new settings, in particular, to the learning of bisimulation quotients of transition systems.Consider the following labelled transition system (LTS).We would like to learn its minimal representation, i.e., its quotient modulo bisimulation.Our setting allows us to choose a suitable logical language.For LTSs, the language consists of the formulas of standard multi-modal logic (cf.Ex. 3).The semantics is as usual where a φ holds at a state if it has an a-successor that makes φ true.
As above, the algorithm constructs tables, starting with (S = {x 0 }, Ψ = ∅).The table is closed, so the first conjecture is a single state with an a-loop with no proposition letter true (note that x 0 has no b or c successor and no proposition is true at x 0 ).It is, however, easy for the teacher to find a counterexample.For example, the formula a b ⊤ is true at the root of the original LTS but false in the conjecture.We add the counterexample and all its subformulas to Ψ and obtain a new table ({x 0 }, Ψ ′ } with Ψ ′ = { a b ⊤, b ⊤, ⊤}.Now, the table is not closed, as x 0 has successor x 1 that satisfies b ⊤ whereas x 0 does not satisfy b ⊤.Therefore we add x 1 to the table to obtain ({x 0 , x 1 }, Ψ ′ ).Similar arguments will lead to the closed table ({x 0 , x 1 , x 3 , x 4 }, Ψ ′ ) which also yields the correct conjecture.Note that the state x 2 does not get added to the table as it is equivalent to x 1 and thus already represented.This demonstrates a remarkable fact: we computed the bisimulation quotient of the LTS without inspecting the (infinite) right-hand side of the LTS.
Another important example that fits smoothly into our framework is the wellknown variant of Angluin's algorithm to learn Mealy machines (Ex.2).Thanks to our general notion of logic, our framework allows to use an intuitive language, where a formula is simply an input word w whose truth value at a state x is the observed output after entering w at x.This is in contrast to [6] where formulas had to be Boolean valued.Multi-valued logics fit naturally in our setting; this is expected to be useful to deal with systems with quantitative information.

Preliminaries
The general learning algorithm in this paper is based on the theory of coalgebras, which provides an abstract framework for representing state-based transition systems.In what follows we assume that the reader is familiar with basic notions of category theory and coalgebras [14,26].We briefly recall the notion of pointed coalgebra, modelling a coalgebra with an initial state.Let C be a category with a terminal object 1 and let B : C → C be a functor.A pointed B-coalgebra is a triple (X, γ, x 0 ) where X ∈ C and γ : X → BX and x 0 : 1 → X, specifying the coalgebra structure and the point ("initial state") of the coalgebra, respectively.
Coalgebraic modal logic.Modal logics are used to describe properties of statebased systems, modelled here as coalgebras.The close relationship between coalgebras and their logics is described elegantly via dual adjunctions [20,18,24,21].
Our basic setting consists of two categories C, D connected by functors P, Q forming a dual adjunction P ⊣ Q : C ⇆ D op .In other words, we have a natural bijection C(X, Q∆) ∼ = D(∆, P X) for X ∈ C, ∆ ∈ D.Moreover, we assume two functors, B : . The functor L represents the syntax of the (modalities in the) logic: assuming that L has an initial algebra α : LΦ → Φ we think of Φ as the collection of formulas, or tests.In this logical perspective, the functor P maps an object X of C to the collection of predicates and the functor Q maps an object ∆ of D to the collection Q∆ of ∆-theories.
The connection between coalgebras and their logics is specified via a natural transformation δ : LP ⇒ P B, sometimes referred to as the one-step semantics of the logic.The δ is used to define the semantics of the logic on a B-coalgebra (X, γ) by initiality, as in (2).Furthermore, using the bijective correspondence of the dual adjunction between P and Q, the map corresponds to a map th γ : X → QΦ that we will refer to as the theory map of (X, γ).

BX
The theory map can be expressed directly via a universal property, by making use of the so-called mate δ ♭ : BQ ⇒ QL of the one-step semantics δ (cf.[18,24]).More precisely, we have δ ♭ = QLε • QδQ • ηBQ, where η, ε are the unit and counit of the adjunction.Then th γ : X → QΦ is the unique morphism making (3) commute.
Example 1.Let C = D = Set, P = Q = 2 − the contravariant power set functor, B = 2 × − A and L = 1 + A × −.In this case B-coalgebras can be thought of as deterministic automata with input alphabet A (e.g., [25]).It is well-known that the initial L-algebra is Φ = A * with structure α = [ε, cons] : 1 + A × A * → A * where ε selects the empty word and cons maps a pair (a, w) ∈ A×A * to the word aw ∈ A * , i.e., in this example our tests are words with the intuitive meaning that a test succeeds if the word is accepted by the given automaton.For X ∈ C, the X-component of the (one-step) semantics δ : LP ⇒ P B is defined as follows: where π 1 and π 2 are the projection maps.The theory map th γ sends a state to the language accepted by that state in the usual way.
where A and O are fixed sets, thought of as input and output alphabet, respectively.Then B-coalgebras are Mealy machines and the initial L-algebra is given by the set A + of finite non-empty words over A. For X ∈ C, the one-step semantics δ X : Concretely, formulas are words in A + ; the (O-valued) semantics of w ∈ A + at state x is the output o ∈ O that is produced after processing the input w from state x.
Example 3. Let C = Set and D = BA, where the latter denotes the category of Boolean algebras.Again P = 2 − , but this time 2 X is interpreted as a Boolean algebra.The functor Q maps a Boolean algebra to the collection of ultrafilters over it [7].Furthermore B = (P−) A where P denotes covariant power set and A a set of actions.Coalgebras for this functor correspond to labelled transition systems, where a state has a set of successors that depends on the action/input from A. The dual functor L : BA → BA is defined as LY := F BA ({ a y | a ∈ A, y ∈ Y })/ ≡ where F BA : Set → BA denotes the free Boolean algebra functor and where, roughly speaking, ≡ is the congruence generated from the axioms a ⊥ ≡ ⊥ and a (y 1 ∨ y 2 ) ≡ a (y 1 ) ∨ a (y 2 ) for each a ∈ A. This is explained in more detail in [21].The initial algebra for this functor is the so-called Lindenbaum-Tarski algebra [7] of modal formulas (φ ::=⊥| φ ∨ φ | ¬φ | a φ) quotiented by logical equivalence.The definition of an appropriate δ can be found in, e.g., [21]-the semantics of a formula then amounts to the standard one [7].
Different types of probabilistic transition systems also fit into the dual adjunction framework, see, e.g, [17].
Subobjects and intersection-preserving functors.We denote by Sub(X) the collection of subobjects of an object X ∈ C. Let ≤ be the order on subobjects s : S X, s ′ : S ′ X given by s ≤ s ′ iff there is m : S → S ′ s.t.s = s ′ • m.The intersection J X of a family J = {s i : S i → X} i∈I is defined as the greatest lower bound w.r.t. the order ≤.In a complete category, it can be computed by (wide) pullback.We denote the maps in the limiting cone by x i : J S i .
For a functor B : C → D, we say B preserves (wide) intersections if it preserves these wide pullbacks, i.e., if (B( J), {Bx i } i∈I ) is the pullback of {Bs i : BS i → BX} i∈I .By [2, Lemma 3.53] (building on [29]), finitary functors on Set 'almost' preserve wide intersections: for every such functor B there is a functor B ′ which preserves wide intersections and agrees with B on all non-empty sets.Finally, if B preserves intersections, then it preserves monos.
Minimality notions.The algorithm that we will describe in this paper learns a minimal and reachable representation of an object.The intuitive notions of minimality and reachability are formalised as follows.
Definition 4. We call a B-coalgebra (X, γ) minimal w.r.t.logical equivalence if the theory map th γ : X → QΦ is a monomorphism.Definition 5. We call a pointed B-coalgebra (X, γ, x 0 ) reachable if for any subobject s : S → X and s 0 : 1 → S with x 0 = s • s 0 : if S is a subcoalgebra of (X, γ) then s is an isomorphism.
For expressive logics [27], behavioural equivalence concides with logical equivalence.Hence, in that case, our algorithm learns a "well-pointed coalgebra" in the terminology of [2], i.e., a pointed coalgebra that is reachable and minimal w.r.t.behavioural equivalence.All logics appearing in this paper are expressive.
Assumption on C and Factorisation System.Throughout the paper we will assume that C is a complete and well-powered category.Well-powered means that for each X ∈ C the collection Sub(X) of subobjects of a given object forms a set.Our assumptions imply [10, Proposition 4.

Subformula Closed Collections of Formulas
Our learning algorithm will construct conjectures that are "partially" correct, i.e., correct with respect to a subobject of the collection of all formulas/tests.Recall this collection of all tests are formalised in our setting as the initial L-algebra (Φ, α : LΦ → Φ).To define a notion of partial correctness we need to consider subobjects of Φ to which we can restrict the theory map.This is formalised via the notion of "subformula closed" subobject of Φ.

LX LY X Y
The definition of such subobjects is based on the notion of recursive coalgebra.For L : D → D an endofunctor, a coalgebra f : X → LX is called recursive if for every L-algebra g : LY → Y there is a unique 'coalgebra-to-algebra' map g † making (5) commute.Definition 6.A subobject j : Ψ → Φ is called a subformula closed collection (of formulas) if there is a unique L-coalgebra structure σ : Ψ → LΨ such that (Ψ, σ) is a recursive L-coalgebra and j is the (necessarily unique) coalgebra-to-algebra map from (Ψ, σ) to the initial algebra (Φ, α).
Remark 7. The uniqueness of σ in Definition 6 is implied if L preserves monomorphisms.This is the case in our examples.The notion of recursive coalgebra goes back to [28,23].The paper [1] contains a claim that the first item of our definition of subformula closed collection is implied by the second one if L preserves preimages.In our examples both properties of (Ψ, σ) are verified directly, rather than by relying on general categorical results.
Example 8.In the setting of Example 1, where the initial L-algebra is based on the set A * of words over the set (of inputs) A, a subset Ψ ⊆ A * is subformulaclosed if it is suffix-closed, i.e., if for all aw ∈ Ψ we have w ∈ Ψ as well.
Example 9.In the setting that B = (P−) A for some set of actions A, C = Set and D = BA, the logic is given as a functor L on Boolean algebras as discussed in Example 3. As a subformula closed collection is an object in Ψ , we are not simply dealing with a set of formulas, but with a Boolean algebra.The connection to the standard notion of being closed under taking subformulas in modal logic [7] can be sketched as follows: given a set ∆ of modal formulas that is closed under taking subformulas, we define a Boolean algebra Ψ ∆ ⊆ Φ as the smallest Boolean subalgebra of Φ that is generated by the set ∆ It is then not difficult to define a suitable σ : Ψ ∆ → LΨ ∆ .As Ψ ∆ is generated by closing ∆ under Boolean operations, any two states x 1 , x 2 in a given coalgebra (X, γ) satisfy (∀b In other words, equivalence w.r.t.Ψ ∆ coincides with equivalence w.r.t. the set of formulas ∆.This explains why in the concrete algorithm, we do not deal with Boolean algebras explicitly, but with subformula closed sets of formulas instead. The key property of subformula closed collections Ψ is that we can restrict our attention to the so-called Ψ -theory map.Intuitively, subformula closedness is what allows us to define this theory map inductively.Lemma 10.Let Ψ j Φ be a sub-formula closed collection, with coalgebra structure σ : Ψ → LΨ .Then th γ Ψ = Qj • th γ Φ is the unique map making (6) commute.We call th γ Ψ the Ψ -theory map, and omit the Ψ if it is clear from the context.

Reachability and the Base
In this section, we define the notion of base of an endofunctor, taken from [8].This allows us to speak about the (direct) successors of states in a coalgebra, and about reachability, which are essential ingredients of the learning algorithm.
Definition 11.Let B : C → C be an endofunctor.We say B has a base if for every arrow f : X → BY there exist g : X → BZ and m : Z Y with m a monomorphism such that f = Bm • g, and for any pair (7).We call (Z, g, m) the (B)-base of the morphism f .
We sometimes refer to m : Z Y as the base of f , omitting the g when it is irrelevant, or clear from the context.Note that the terminology 'the' base is justified, as it is easily seen to be unique up to isomorphism.
For example, let B : and m is the inclusion.The associated g : X → BZ is the corestriction of f to BZ.
For B = (P−) The B-base provides an elegant way to relate reachability within a coalgebra to a monotone operator on the (complete) lattice of subobjects of the carrier of the coalgebra.Moreover, we will see that the least subcoalgebra that contains a given subobject of the carrier can be obtained via a standard least fixpoint construction.Finally, we will introduce the notion of prefix closed subobject of a coalgebra, generalising the prefix closedness condition from Angluin's algorithm.
By our assumption on C at the end of Section 3, the collection of subobjects (Sub(X), ≤) ordered as usual (cf.page 3) forms a complete lattice.Recall that the meet on Sub(X) (intersection) is defined via pullbacks.In categories with coproducts, the join s 1 ∨ s 2 of subobjects s 1 , s 2 ∈ Sub(X) is defined as the mono part of the factorisation of the map [s 1 , s 2 ] : • e for a strong epi e.In Set, this amounts to taking the union of subsets.
For a binary join s 1 ∨ s 2 we denote by inl ∨ : S 1 → (S 1 ∨ S 2 ) and inr ∨ : S 2 → (S 1 ∨ S 2 ) the embeddings that exist by s i ≤ s 1 ∨ s 2 for i = {1, 2}.Let us now define the key operator of this section.Definition 14.Let B be a functor that has a base, s : S X a subobject of some X ∈ C and let (X, γ) be a B-coalgebra.Let (Γ (S), g, Γ B γ (s)) be the B-base of γ • s, see Diagram (8).Whenever B and γ are clear from the context, we write Γ (s) instead of Γ B γ (s).Lemma 15.Let B : C → C be a functor with a base and let (X, γ) be a Bcoalgebra.The operator Γ : Sub(X) → Sub(X) defined by s → Γ (s) is monotone.
Intuitively, Γ computes for a given set of states S the set of "immediate successors", i.e., the set of states that can be reached by applying γ to an element of S. We will see that pre-fixpoints of Γ correspond to subcoalgebras.Furthermore, Γ is the key to formulate our notion of closed table in the learning algorithm.
Proposition 16.Let s : S X be a subobject and (X, γ) ∈ Coalg(B) for X ∈ C and B : C → C a functor that has a base.Then s is a subcoalgebra of (X, γ) if and only if Γ (s) ≤ s.Consequently, the collection of subcoalgebras of a given B-coalgebra forms a complete lattice.
Using this connection, reachability of a pointed coalgebra (Definition 5) can be expressed in terms of the least fixpoint lfp of an operator defined in terms of Γ .
Theorem 17.Let B : C → C be a functor that has a base.A pointed B-coalgebra This justifies defining the reachable part from an initial state x 0 : 1 X as the least fixpoint of the monotone operator Γ ∨ x 0 .Standard means of computing the least fixpoint by iterating this operator then give us a way to compute this subcoalgebra.Further, Γ provides a way to generalise the notion of "prefixed closedness" from Angluin's L * algorithm to our categorical setting.Definition 18.Let s 0 , s ∈ Sub(X) for some X ∈ C and let (X, γ) be a Bcoalgebra.We call s s 0 -prefix closed w.r.t.γ if s = n i=0 s i for some n ≥ 0 and a collection {s i | i = 1, . . ., n} with s j+1 ≤ Γ ( j i=0 s i ) for all j with 0 ≤ j < n.
The first point means that the learned coalgebra is 'correct', that is, it agrees with the coalgebra of the teacher on all possible tests from the initial state.For instance, in case of deterministic automata and their logic in Example 1, this just means that the language of the learned automaton is the correct one.
In the learning game, we are only provided limited access to the coalgebra γ : X → BX.Concretely, the teacher gives us: for any subobject S X and sub-formula closed subobject Ψ of Φ, the composite theory map S X QΨ for (S, γ, ŝ0 ) a pointed coalgebra, whether or not it is correct w.r.t. the collection Φ of all tests; -in case of a negative answer to the previous question, a counterexample, which essentially is a subobject Ψ ′ of Φ representing some tests on which the learned coalgebra is wrong (defined more precisely below); -for a given subobject S of X, the 'next states'; formally, the computation of the B-base of the composite arrow S X BX γ .
The first three points correspond respectively to the standard notions of membership query ('filling in' the table with rows S and columns Ψ ), equivalence query and counterexample generation.The last point, about the base, is more unusual: it does not occur in the standard algorithm, since there a canonical choice of (X, γ) is used, which allows to represent next states in a fixed manner.It is required in our setting of an arbitrary coalgebra (X, γ).
In the remainder of this section, we describe the abstract learning algorithm and its correctness.First, we describe the basic ingredients needed for the algorithm: tables, closedness, counterexamples and a procedure to close a given table (Section 6.1).Based on these notions, the actual algorithm is presented (Section 6.2), followed by proofs of correctness and termination (Section 6.3).Moreover, we assume a pointed B-coalgebra (X, γ, s 0 ).

Assumption 19 Throughout this section, we assume
Remark 20.We restrict to C = Set, but see it as a key contribution to state the algorithm in categorical terms: the assumptions cover a wide class of functors on Set, which is the main direction of generalisation.Further, the categorical approach will enable future generalisations.The assumptions on the category C are: it is complete, well-powered and satisfies that for all (strong) epis q : S → S ∈ C and all monos i : S ′ → S such that q • i is mono there is a morphism q −1 : S → S such that (i) q • q −1 = id and q −1 • q • i = i.

Tables and counterexamples Definition
To make the notation a bit lighter, we sometimes refer to a table by (S, Ψ ), using s and i respectively to refer to the actual subobjects.The pair (S, Ψ ) represents 'rows' and 'columns' respectively, in the table; the 'elements' of the table are given abstractly by the map th γ Ψ • s.In particular, if C = D = Set and Q = 2 − , then this is a map S → 2 Ψ , assigning a Boolean value to every pair of a row (state) and a column (formula).
For the definition of closedness, we use the operator Γ (S) from Definition 14, which characterises the successors of a subobject S X.
A conjecture is a coalgebra on S, which is not quite a subcoalgebra of X: instead, it is a subcoalgebra 'up to equivalence w.r.t.Ψ ', that is, the successors agree up to logical equivalence.The following describes, for a given table, how to extend it with the successors (in X) of all states in S. As we will see below, by repeatedly applying this construction, one eventually obtains a closed table.Definition 29.Let (S, Ψ ) be a sharp table.Let (S, q, r) be the (strong epi, mono)-factorisation of the map th γ • (s ∨ Γ (s)), as in the diagram: We define close(S, Ψ ) := {s: S X | th γ • s = r, s ≤ s ≤ s ∨ Γ (s)}.For each s ∈ close(S, Ψ ) we have s ≤ s and thus s = s • κ for some κ : S → S.
We will refer to κ = q • inl ∨ as the connecting map from s to s.
By our assumptions, the hypothesis of Lemma 31 is satisfied (Remark 20), hence close(S, Ψ ) is non-empty.It is precisely (and only) at this point that we need the strong condition about existence of right inverses to epimorphisms.

The algorithm
Having defined closedness, counterexamples and a procedure for closing a table, we are ready to define the abstract algorithm.In the algorithm, the teacher has access to a function counter((S, γ, ŝ0 ), Ψ ), which returns the set of all counterexamples (extending Ψ ) for the conjecture (S, γ, ŝ0 ).If this set is empty, the coalgebra (S, γ, ŝ0 ) is correct (see Lemma 28), otherwise the teacher picks one of its elements Ψ ′ .We also make use of close(S, Ψ ), as given in Definition 29.
Algorithm 1 Abstract learning algorithm The algorithm takes as input the coalgebra (X, γ, s 0 ) (which we fixed throughout this section).In every iteration of the outside loop, the table is first closed by repeatedly applying the procedure in Definition 29.Then, if the conjecture corresponding to the closed table is correct, the algorithm returns it (Line 12).Otherwise, a counterexample is chosen (Line 14), and the algorithm continues.

Correctness and Termination
Correctness is stated in Theorem 33.It relies on establishing loop invariants: Theorem 32.The following is an invariant of both loops in Algorithm 6.2: 1. (S, Ψ ) is sharp, 2. s • ŝ0 = s 0 , and 3. s is s 0 -prefix closed w.r.t.γ.Theorem 33.If Algorithm 6.2 terminates, then it returns a pointed coalgebra (S, γ, ŝ0 ) which is minimal w.r.t.logical equivalence, reachable and correct w.r.t.Φ.
In our termination arguments, we have to make an assumption about the coalgebra which is to be learned.It does not need to be finite itself, but it should be finite up to logical equivalence-in the case of deterministic automata, for instance, this means the teacher has a (possibly infinite) automaton representing a regular language.To speak about this precisely, let Ψ be a subobject of Φ.We take a (strong epi, mono)-factorisation of the theory map, i.e., th γ Ψ = X eΨ / / / / |X| Ψ / / mΨ / / QΨ for some strong epi e and mono m.We call the object |X| Ψ in the middle the Ψ -logical quotient.For the termination result (Theorem 37), |X| Φ is assumed to have finitely many quotients and subobjects, which just amounts to finiteness, in Set.
We start with termination of the inner while loop (Corollary 36).This relies on two results: first, that once the connecting map κ is an iso, the table is closed, and second, that-under a suitable assumption on the coalgebra (X, γ)-during execution of the inner while loop, the map κ will eventually be an iso.
Theorem 34.Let (S, Ψ ) be a sharp table, let S ∈ close(S, Ψ ) and let κ : S → S be the connecting map.If κ is an isomorphism, then (S, Ψ ) is closed.
Lemma 35.Consider a sequence of sharp tables (S i si X, Ψ ) i∈N such that s i+1 ∈ close(S i , Ψ ) for all i.Moreover, let (κ i : S i → S i+1 ) i∈N be the connecting maps (Definition 29).If the logical quotient |X| Φ of X has finitely many subobjects, then κ i is an isomorphism for some i ∈ N.

Corollary 36. If the Φ-logical quotient |X| Φ has finitely many subobjects, then the inner while loop of Algorithm 1 terminates.
For the outer loop, we assume that |X| Φ has finitely many quotients, ensuring that every sequence of counterexamples proposed by the teacher is finite.

Future Work
We showed how duality plays a natural role in automata learning, through the central connection between states and tests.Based on this foundation, we proved correctness and termination of an abstract algorithm for coalgebra learning.The generality is not so much in the base category (which, for the algorithm, we take to be Set) but rather in the functor used; we only require a few mild conditions on the functor, and make no assumptions about its shape.The approach is thus considered coalgebra learning rather than automata learning.
Returning to automata, an interesting direction is to extend the present work to cover learning of, e.g., non-deterministic or alternating automata [9,5] for a regular language.This would require explicitly handling branching in the type of coalgebra.One promising direction would be to incorporate the forgetful logics of [19], which are defined within the same framework of coalgebraic logic as the current work.It is not difficult to define in this setting what it means for a table to be closed 'up to the branching part', stating, e.g., that even though the table is not closed, all the successors of rows are present as combinations of other rows.
Another approach would be to integrate monads into our framework, which are also used to handle branching within the theory of coalgebras [16].It is an intriguing question whether the current approach, which allows to move beyond automata-like examples, can be combined with the CALF framework [13], which is very far in handling branching occuring in various kinds of automata.

A Proofs of Section 5
Proof (Proof of Proposition 12).Let f : X → B(Y ).Consider the collection of all pairs of maps g k : be the intersection of all the m k -this is a (small) set since C is well-powered.We abbreviate {m k } k∈K by I.
Since B preserves intersections, B(m) : B(I) → B(Y ) is the intersection of all the subobjects B(m k ).Now the g k 's form a cone over the B(m k )'s, so we get a unique g : X → B(I) from the universal property of the pullback B(I).
We claim that (I, g, m) is the base of f .To see this, first of all, note that i is mono, and B(m) • g = f by definition of m and g.Further, if there is any and m ′ monic then it is (up to isomorphism) one of the g k ,m k pairs.Hence, there is the map x k : I → U k in the limiting cone, i.e., m k • x k = i, and we have B(x k ) • g = g k .Finally x k is unique among such maps, since B preserves monos (as it preserves intersections).Lemma 13).By our assumption on Z there exists a morphism g : S → BZ such that Bm • g = f .Therefore we have Bm ′ • Be • g = Bh • f which shows that m ′ is a candidate for the base of Bh • f .We still need to check the universal property of the base.To this aim let g ′ : S → BU and n : U → Y be the base of Bh • f :

Proof (Proof of
By the universal property of the base there is a morphism j : U → W making the lower right diagram commute.Now, consider the following pullback: This is a preimage because n is mono and by assumption on B we have that this pullback is preserved under application of B. S forms a cone over the diagram with S → BX, S → BU .So there exists a map from S to the pullback.
p n is mono, so Bp n • w is a base factorization.So, we get an arrow from BZ to BP , i.e., k : Z → P such that: Consider the following diagram.According to the diagonal filling property, we have By the universal property of the base we have d and because m ′ is monic we have j • d = id W .
Lemma 38.If C is complete and well-powered, then for each X ∈ C, we have Sub(X) has arbitrary meets.Consequently, Sub(X) is a complete lattice.
Proof (Proof of Lemma 38).Consider some X ∈ C and an arbitrary family of subobjects {m i : S 1 → X} i∈I .Let P be the pullback with pullback maps {p 1 : P → S i } i∈I .As the m i are mono, the p i are mono as well, so let define p := m i • p i : P → X ∈ Sub(X).Obviously, we have p ≤ m i ∀i ∈ I. So, P is a lower bound.To see that P is the greatest lower bound, consider an arbitrary P ′ with p ′ : P ′ → X that is a lower bound of the same family of subobjects.By definition of lower bound we have for each i ∈ I a map p ′ i : By universal property of the (wide) pullback, there exists a unique map c : P ′ → P s.t.p • c = p ′ , i.e., p ′ ≤ p.As P ′ was an arbitrary lower bound, we showed that P is the greatest lower bound.This finishes the proof of the fact that Sub(X) has arbitrary meets.Completeness of the lattice can now be proven in a standard way by defining the join of an arbitrary collection of subobjects as the meet of all upper bounds of this collection.Lemma 15).It is obvious that Γ is well-defined.To check monotonicity, we consider the following diagram for subobjects s : S → X and s ′ : S ′ → X such that s ≤ s ′ .

Proof (Proof of
Here j exists by the definition of ≤ and h exists by the universal property of the base of γ • s.Therefore we have Γ (s) ≤ Γ (s ′ ) as required.

Bj Bs
As s is a subcoalgebra there exists σ : S → BS s.t. the outer square commutes.By the universal property of the base there exists j : Γ (S) → S s.t.s • j = Γ (s).In other words, Γ (s) ≤ s as required.⇐ By assumption there exists a j : Γ (S) → S such that s • j = Γ (s).We define a B-coalgebra structure on S by putting σ := j • e.We have to show that the outer square in the above diagram commutes, but this is easy to show because the inner square commutes by definition of the base, the left triangle commutes by definition of σ and the right one by assumption on j.
Finally, that the collection of subcoalgebras of (X, γ) forms a complete lattice is now a direct consequence of the fact that the collection of pre-fixpoints of the monotone operator Γ forms a complete lattice (Knaster-Tarski theorem).

B Proofs of Section 6
Proof (Proof of Theorem 24).Given a table (S, Ψ ) that is closed, it is straightforward to construct a conjecture γ as composite of g : S → BΓ (S) and the arrow Bk : BΓ (S) → S, where g is part of the base (Γ (S), g, Γ (s)) of γ • s and k : Γ (S) → S is the morphism that exists by closedness of (S, Ψ ).For the converse, consider a conjecture (S, γ) for a sharp table (S, Ψ ), let (Γ (S), g, Γ (s)) be the base of γ • s and let (h : Γ (S) → Y, m : Y → QΨ ) be the factorisation of th γ • Γ (s).By Lemma 13, as h is epi, we have that (Y, Bh • g, m) is the base of Bth γ • γ • s.The situation is depicted in the (commuting) upper square of the diagram below).The bigger outer square commutes by the fact that γ is assumed to be a conjecture.As the table is sharp, we have th γ • s is mono.Therefore the universal property of the base yields existence of a morphism j : Y → S such that th γ • s • j = m.We define k := j • h and claim that this k is a witness for (S, Ψ ), i.e., that k makes the relevant diagram from Definition 22 commute.To see this, we calculate: Proof (Proof of Lemma 31).Given the assumption of the lemma we are able to define a morphism s : S → X by putting s := (s ∨ Γ (s)) • q −1 .Obviously s is a mono as it is defined as composition of monos.Furthermore, by definition, we have s ≤ s ∨ Γ (s).To see that s ≤ s, we calculate Finally, the condition concerning the theory map also follows easily: This finishes the proof of the lemma.
We need a few auxiliary lemma's in the proofs below.
2. (Holds at entry of the outer loop.)Follows immediately from the first two lines of the algorithm.(Preserved by the inner loop.)Suppose ŝ0 • s = s 0 , and let s ∈ close(S, Ψ ).We need to prove that s 0 •κ• ŝ0 = s 0 where κ : S → S is the connecting map.Indeed, we have s 0 • κ • ŝ0 = s • ŝ0 = s 0 , by definition of κ and assumption, respectively.
(Preserved by the outer loop.)This follows immediately from preservation by the inner loop.3. Clearly the initial configuration (S 0 , 0) is s 0 -prefix closed.Suppose now that (S, Ψ ) is a table with s being s 0 -prefix closed.We need to check that any s ∈ close(S, Ψ ) is s 0 -prefix closed as well.By assumption on (S, Ψ ) we have s = n i=0 s i for a suitable family of subobjects s 0 , . . ., s n .Let s ∈ close(S, Ψ ).Then by definition we have s ≤ s ∨ Γ (s), so we put s n+1 := Γ (s) ∧ s.It is then easy to check that s is s 0 -prefixclosed: where the last equality follows from s ≤ s ≤ s ∨ Γ (s) .By definition we have Proof (Proof of Theorem 33).Minimality w.r.t logical equivalence follows from the fact that sharpness of the table is maintained throughout.As the algorithm terminated there is no counterexample, which means by Lemma 28 that the coalgebra is correct w.r.t.Φ.For reachability we show that the pointed coalgebra that is returned by the algorithm is reachable by showing that any conjecture that is constructed during the run of the algorithm is reachable.While running the algorithm we will only encounter conjectures that are built from tables that are both sharp and closed.Therefore we consider an arbitrary sharp and closed table (S, Ψ ) together with the conjecture (S, γ) that exists according to Theorem 24.We are going to prove that (S, γ, ŝ0 ) is reachable.
By Theorem 32 we know that (S, Ψ ) is s 0 -prefix closed.This means that s = n i=0 s i for suitable subobjects s 1 , . . ., s n ∈ Sub(X).Suppose now that (S, γ, s 0 ) is a subcoalgebra of (S, γ, ŝ0 ) with inclusion j : S → S such that j • s 0 = ŝ0 .We prove by induction on i that s i ≤ s for all i ∈ {0, . . ., n} and thus s ≤ s -this will imply s = s and thus, as s was assumed to be an arbitrary (pointed) subcoalgebra, reachability of (S, γ, ŝ0 ).
where we slightly abuse notation by writing f ≤ g for arbitrary morphisms f : X 1 → Y and g : X 2 → Y is there exists a morphism m : X 1 → X 2 such that g • m = f .The inequality implies that there is a map k The inner shapes commute, from top to bottom: (1) by definition the base, (2) by definition of inr ∨ , (3) by definition of (q, r); for the bottom triangle (4), we have which suffices since κ is an iso.Since the entire diagram commutes, the coalgebra structure on S gives a conjecture for (S, Ψ ).Hence, by Theorem By assumption, |X| Φ has finitely many subobjects, so κ i must be an isomorphism for some i.

⊓ ⊔
Proof (Proof of Corollary 36).The while loop computes a chain of subobjects of X as in Lemma 35; in particular, each of these forms a sharp table (with Ψ ), since sharpness is an invariant (Theorem 32).Hence, after a finite number of iterations, κ is an iso.By Theorem 34 this implies that (S, Ψ ) is closed, which means the guard of the while loop is false.

⊓ ⊔
For termination of the outer loop, we need several auxiliary lemmas.
Lemma 42.Let (S, Ψ ) be table, and let Ψ ′ be a subformula-closed subobject of Φ, such that Ψ is a subcoalgebra of Ψ ′ .Then there is a unique map q making the following diagram commute: Moreover, this map q is an epimorphism.
Proof.The outside of the diagram commutes by Lemma 40.The map q arises by the unique fill-in property.That q is an epi follows since e Ψ is an epi, and e Ψ = q • e Ψ ′ .⊓ ⊔ Lemma 43.Let (S s X, Ψ ) be a closed table, and (S, γ, ŝ0 ) a pointed coalgebra, such that (S, γ) is a conjecture and s • ŝ0 = s 0 .If Ψ ′ is a counterexample for (S, γ, ŝ0 ), then the map q : |X| Ψ ′ → |X| Ψ from Lemma 42 is not an isomorphism.
Proof.Suppose that q is an iso; we prove that, in that case, (S, γ, ŝ0 ) is correct w.r.t.Ψ ′ .Let q −1 be the inverse of q.Since q•e Ψ ′ = e Ψ we also have e Ψ ′ = q −1 •e Ψ .Hence, the two shapes on the lower right in the following diagram commute: The rectangle commutes since γ is a conjecture for the closed s i • ŝ0 = s 0 , and -Ψ i+1 is a counterexample for (S i , γ, ŝ0 ).
We will show that such a sequence is necessarily finite.
By the last point and Lemma 42, for each i, there exists a map q i+1,i making the diagram on the left-hand side commute: Moreover, again by Lemma 42, for each i, there is a map q i making the diagram on the right-hand side above commute.For each i, we have and since e Φ is epic, we obtain q i+1,i • q i+1 = q i .Hence, we get the following sequence of quotients: It follows from Lemma 43 and the previous assumptions that none of the quotients q i+1,i can be an iso.But since for each i, |X| Ψi is a quotient of |X| Φ , and the latter has only finitely many quotients, the sequence of counterexamples must be finite.⊓ ⊔

Example 2 .
Again let C = D = Set and consider the functors P 4.3] that every morphism f in C up to isomorphism) as f = m • e with m a mono and e a strong epi.Recall that an epimorphism e : X → Y is strong if for every commutative square in(4) where the bottom arrow is a monomorphism, there exists a unique diagonal morphism d such that the entire diagram commutes.

Proposition 12 .
Suppose C is complete and well-powered, and B : C → C preserves (wide) intersections.Then B has a base.If C is a locally presentable category, then it is complete and well-powered[3,  Remark 1.56].Hence, in that case, any functor B : C → C which preserves intersections has a base.The following lemma will be useful in proofs.Lemma 13.Let B : C → C be a functor that has a base and that preserves preimages.Let f : S → BX and h : X → Y be morphisms, let (Z, g, m) be the base of f and let e : Z → W, m ′ : W → Y be the (strong epi, mono)-factorisation of h • m.Then (W, Be • g, m ′ ) is the base of Bh • f .

-
that we deal with coalgebras over the base category C = Set; -a functor B : C → C that preserves pre-images and wide intersections; -a category D with an initial object 0 s.t.arrows with domain 0 are monic; -a functor L : D → D with an initial algebra LΦ ∼ = → Φ; -an adjunction P ⊣ Q : C ⇆ D op , and a logic δ : LP ⇒ P B.

)
On the other hand, we have s • in j+1 = s j+1 where in j+1 = s j+1 denotes the inclusion of s j+1 into s.Therefore we have th γ • s • in j+1 = th γ • s j+1 .Together with(13) this implies th γ • s • in j+1 = th γ • s • j • k j+1 .By sharpness of the table s we obtain in j+1 = j • k j+1 and finally s j+1 = s • in j+1 = s • j • k j+1 = s ′ • k j+1 which shows that s j+1 ≤ s ′ .This finishes the induction proof.⊓ ⊔Proof (Proof of Theorem 34).Suppose κ is an isomorphism.Consider the following diagram (using the notation from Definition 29), where g : S → BX is the map which forms the base of γ • s together with Γ (s) : Γ (S) → X.
Definition 22.A table (S, Ψ ) is closed if there exists a map k : Γ (S) → S such that Diagram (9) commutes.A table (S, Ψ ) is sharp if the composite map Thus, a table (S, Ψ ) is closed if all the successors of states (elements of Γ (S)) are already represented in S, up to equivalence w.r.t. the tests in Ψ .In other terms, the rows corresponding to successors of existing rows are already in the table.Sharpness amounts to minimality w.r.t.logical equivalence: every row has a unique value.The latter will be an invariant of the algorithm (Theorem 32).
It is essential to be able to construct a conjecture from a closed table.The following, stronger result is a variation of Proposition 16.A sharp table is closed iff there exists a conjecture for it.Moreover, if the table is sharp and B preserves monos, then this conjecture is unique.Suppose (S, Ψ ) is closed, and γ is a conjecture.Then th γΨ • s = th γ Ψ : S → QΨ .If ŝ0 : 1 → S satisfies s• ŝ0 = s 0 then (S, γ, ŝ0 ) is correct w.r.t.Ψ .We next define the crucial notion of counterexample to a pointed coalgebra: a subobject Ψ ′ of Ψ on which it is 'incorrect'.Let (S, Ψ ) be a table, and let (S, γ, ŝ0 ) be a pointed B-coalgebra on S. Let Ψ ′ be a subformula closed subobject of Φ, such that Ψ is a subcoalgebra of Ψ ′ .We say Ψ ′ is a counterexample (for (S, γ, ŝ0 ), extendingΨ ) if (S, γ, ŝ0 ) is not correct w.r.t.Ψ ′ .The following elementary lemma states that if there are no more counterexamples for a coalgebra, then it is correct w.r.t. the object Φ of all formulas.Let (S, Ψ ) be a table, and let (S, γ, ŝ0 ) be a pointed B-coalgebra on S. Suppose that there are no counterexamples for (S, γ, ŝ0 ) extending Ψ .Then Definition 23.Let (S, Ψ ) be a table.A coalgebra structure γ : S → BS is called a conjecture (for (S, Ψ )) if Diagram (10) commutes.Definition 25.Let (S, Ψ ) be a table, and let (S, γ, ŝ0 ) be a pointed B-coalgebra on S. We say (S, γ, ŝ0 ) is correct w.r.t.Ψ if Diagram (11) commutes.(S, γ, ŝ0 ) is correct w.r.t.Φ.
Proof (Proof ofLemma 26).The map th γ is, by definition, the unique map making the following diagram commute.The rectangle on the right commutes by definition of th γ Ψ .Together with γ being a conjecture, it follows that the outside of the diagram commutes.Since th γ Ψ is the unique such map, we have th γ Ψ • s = th γ Ψ .⊓⊔Proof(ProofofLemma28).If there is no counterexample, then in particular Φ is not a counterexample.The object Φ is subformula-closed subobject of itself, and Ψ is a subcoalgebra of Φ.Hence, by definition of counterexamples, it must be the case that (S, γ, ŝ0 ) correct w.r.t.Φ.⊓ ⊔Proof (Proof ofLemma 30).Let s ∈ close(S, Ψ ).We calculate: 24, the table is closed.⊓ ⊔ Proof (Proof of Lemma 35).First, observe that the S i 's form an increasing chain of subobjects of X.Since all these tables (S i , Ψ ) are sharp, they give rise to an increasing chain of subobjects of Q(Ψ ), by composition with th γ Ψ , given by th γ Ψ • s i : S i → Q(Ψ ).By Lemma 40, it follows that each th γ Φ • s i : S i → QΦ is monic, and we obtain a sequence of subobjects of Φ: It follows that this induces a chain of subobjects of |X| Φ : table (S, Ψ ).Since the entire diagram commutes, it shows that (S, γ) is a conjecture for the closed table (S, Ψ ′ ) as well.Together with s • ŝ0 = s 0 , by Lemma 26, we obtain that (S, γ, ŝ0 ) is correct w.r.t.Ψ ′ .⊓ ⊔ Proof (Proof of Theorem 37).The inner while loop terminates in each iteration of the outer loop by Corollary 36.The outer loop generates a sequence Ψ 0 , Ψ 1 , Ψ 2 , . . . of subobjects, such that for each i, there is a pointed coalgebra (S i , γ, ŝ0 ) such that -(S i si X, Ψ i ) is a closed table, -(S i , γ) is a conjecture for this table,