Analytic Combinatorics of Lattice Paths with Forbidden Patterns, the Vectorial Kernel Method, and Generating Functions for Pushdown Automata
 130 Downloads
Abstract
In this article we develop a vectorial kernel method—a powerful method which solves in a unified framework all the problems related to the enumeration of words generated by a pushdown automaton. We apply it for the enumeration of lattice paths that avoid a fixed word (a pattern), or for counting the occurrences of a given pattern. We unify results from numerous articles concerning patterns like peaks, valleys, humps, etc., in Dyck and Motzkin paths. This refines the study by Banderier and Flajolet from 2002 on enumeration and asymptotics of lattice paths: we extend here their results to patternavoiding walks/bridges/meanders/excursions. We show that the autocorrelation polynomial of this forbidden pattern, as introduced by Guibas and Odlyzko in 1981 in the context of rational languages, still plays a crucial role for our algebraic languages. En passant, our results give the enumeration of some classes of selfavoiding walks, and prove several conjectures from the OnLine Encyclopedia of Integer Sequences. Finally, we also give the trivariate generating function (length, final altitude, number of occurrences of the pattern p), and we prove that the number of occurrences is normally distributed and linear with respect to the length of the walk: this is what Flajolet and Sedgewick call an instance of Borges’s theorem.
Keywords
Lattice paths Dyck paths Motzkin paths Łukasiewicz paths Pattern avoidance Autocorrelation Finite automata Markov chains Pushdown automata Generating functions Wiener–Hopf factorization Kernel method Asymptotic analysis Gaussian limit law Borges’ theorem1 Introduction
Combinatorial structures having a rational or an algebraic generating function play a key role in many fields: computer science (e.g. for the analysis of algorithms involving trees, lists, words), computational geometry (integer points in polytopes, maps, graph decomposition), bioinformatics (RNA structure, pattern matching), number theory (integer compositions, automatic sequences and modular properties, integer solutions of varieties), probability theory (Markov chains, directed random walks); see e.g. [8, 22, 42, 78]. Rational functions are often the trace of a structure encodable by an automaton, while algebraic generating functions are often the trace of a structure which has a treelike recursive specification (typically, a contextfree grammar), or which satisfies a functional equation solvable by variants of the kernel method.
One of the origin of this method goes back to 1968, when Knuth introduced it to enumerate permutations sortable by a stack; see the solution to Exercise 2.2.1–4 in The Art of Computer Programming [52, pp. 536–537], which presents a “new method for solving the ballot problem”. For this problem, the solution involves the root of a quadratic polynomial (the socalled kernel). The method was later extended to more general equations (see e.g. [9, 24, 39, 40]). We refer to [14] for more on the long history and the numerous evolutions of this method, which found many applications e.g. for planar maps, permutations, lattice paths, directed animals, polymers, may it be in combinatorics, statistical mechanics, computer algebra, or in probability theory.
In this article, we show how a new extension of this method, which we call vectorial kernel method, allows us to solve the enumeration of languages generated by any pushdown automaton. Since the seminal article by Chomsky and Schützenberger on the link between contextfree grammars and algebraic functions [27], which also holds for pushdown automata [76], many articles tackled the enumeration of combinatorial structures via a formal language approach. See e.g. [36, 56, 61] for such an approach on the socalled generalized Dyck languages. The words generated by these languages are in bijection with directed lattice paths, and we show how these fundamental objects can be enumerated when they have the additional constraint to avoid a given pattern. For sure, such a class of objects can be described as the intersection of a contextfree language and a rational language; therefore, classical closure properties imply that they are directly generated by another (but huge and clumsy) contextfree language. Unfortunately, despite the fact that the algebraic system associated with the corresponding contextfree grammar is in theory solvable by a resultant computation or by Gröbner bases, this leads in practice to equations which are so big that no current computer could handle them in memory, even for generalized Dyck languages with only 20 different letters.
Our vectorial kernel method offers a generic and efficient way to tackle such enumeration and bypass these intractable equations. Our approach thus generalizes the enumeration and asymptotics obtained by Banderier and Flajolet [9] for classical lattice paths to lattice paths avoiding a given pattern. This work continues the tradition of investigation of enumerative and asymptotic properties of lattice paths via analytic combinatorics [9, 10, 12, 14]. This allows us to unify the considerations of many articles which investigated natural patterns like peaks, valleys, humps, etc., in Dyck and Motzkin paths, corresponding patterns in trees, compositions, ...; see e.g. [5, 16, 25, 33, 38, 53, 57, 60, 62, 66, 69] and all the examples mentioned in our Sect. 8.
2 Definitions, Notations, Autocorrelation Polynomial
Let \({\mathcal {S}}\), the set of steps, be some finite subset of \({\mathbb {Z}}\), that contains at least one negative and at least one positive number.^{1} A lattice path with steps from\({\mathcal {S}}\) is a finite word \(w=[v_1, v_2, \ldots , v_n]\) in which all letters \(v_i\) belong to \({\mathcal {S}}\), visualized as a directed polygonal line in the plane, which starts at the origin and is formed by successive appending of vectors \((1, v_1), (1, v_2), \ldots , (1, v_n)\). The letters that form the path are referred to as its steps. The length of \(w=[v_1, v_2, \ldots , v_n]\), to be denoted by w, is the number of steps in w (that is, n).
The final altitude of w, to be denoted by \({\text {alt}}(w)\), is the sum of all steps in w (that is, \(v_1 + v_2 + \cdots + v_n\)). Visually, \((w, {\text {alt}}(w))\) is the point where w terminates.
 1.
A walk is any path as described above.
The corresponding generating function is denoted by W(t, u).
Nota bene: in all our generating functions, the variable t encodes the length of the paths, and the variable u the final altitude.
 2.
A bridge is a path that terminates at the xaxis.
The corresponding generating function is denoted by B(t). One has^{2}\(B(t)=[u^0]W(t,u)\).
 3.
A meander is a path that stays (weakly) above the xaxis.
The corresponding generating function is denoted by M(t, u).
 4.
An excursion is a path that stays (weakly) above the xaxis and also terminates at the xaxis. In other words, an excursion satisfies both restrictions.
The corresponding generating function is denoted by E(t). One has \(E(t)=M(t,0)\).
Before we state our results, we introduce some notations.
A presuffix of p is a nonempty string that occurs in p both as a prefix and as a suffix. In particular, the whole word p is a (trivial) presuffix of itself. If p has one or several nontrivial presuffixes, we say that p exhibits an autocorrelation phenomenon. For example, the pattern \(p=[1, 1, 2, 1, 2]\) has no autocorrelation. In contrast, the pattern \(p=[1,1,2,3,1,1,2,3,1,1]\) has three nontrivial presuffixes: [1], [1, 1], and \([1,1,2,3,1,1]\), and thus in this case we have autocorrelation.
Presuffix  Length of its complement  Final altitude of its complement 

q  \({\bar{q}}\)  \({\text {alt}}({\bar{q}})\) 
[1]  9  15 
[1, 1]  8  14 
[1, 1, 2, 3, 1, 1]  4  7 
[1, 1, 2, 3, 1, 1, 2, 3, 1, 1]  0  0 
Therefore, for this p we have \(R(t,u) = 1 + t^4u^7 + t^8u^{14} + t^9u^{15}\).
Notice that if for some p no autocorrelation occurs, then we have \({\mathcal {Q}}=\{p\}\) and therefore \(R(t,u)=1\).
Summary of our results
In the next section, we present in a more detailed way the generating functions of all these paths.
3 Lattice Paths with Forbidden Patterns
In this section, we state our main results for the generating functions W / B / M / E of walks, bridges, meanders, excursions, constrained to avoid a given pattern p. In all the theorems, we denote by \(\ell \) the length of the pattern p, and by \({\text {alt}}(p)\) its final altitude.
Theorem 3.1
 1.The bivariate generating function for walks avoiding the pattern p isIf one does not keep track of the final altitude, this yields$$\begin{aligned} W(t,u)=\frac{R(t,u)}{K(t,u)}. \end{aligned}$$(5)$$\begin{aligned} W(t) = W(t,1)=\frac{1}{1 tP(1) + t^\ell /R(t,1)}. \end{aligned}$$(6)
 2.The generating function for bridges avoiding the pattern p iswhere \(u_1(t), \ldots , u_e(t)\) are the small roots of the kernel K(t, u), as defined in (4).$$\begin{aligned} B(t)=  \sum _{i=1}^e \frac{u_i'(t)}{u_i(t)} \frac{R(t,u_i)}{K_t(t,u_i)} \text{, } \end{aligned}$$(7)
In order to find the generating functions of meanders and excursions, we shall use a novel approach that we call vectorial kernel method. With this method, we obtain formulas that generalize those from the Banderier–Flajolet study (in which they considered classical paths, without the additional pattern constraint).
Theorem 3.2
For excursions, just take the above closed form for \(u\rightarrow 0\).
Although we present here this formula under the paradigm of patterns in lattice paths, it holds in fact much more generally for the enumeration of words generated by any pushdown automaton. This shall become transparent via the proof given Sect. 5 and via the examples from Sect. 8.
We now introduce two classes of patterns for which the factor G(t, u) in Formula (8) has a very nice simple shape.
Definition 3.3
(Quasimeanders, reversed meanders) A quasimeander is a lattice path which does not cross the xaxis, except, possibly, at the last step. A reversed meander is a lattice path whose terminal point has a (strictly) smaller ycoordinate than all other points. (Notice that the empty path is both a quasimeander and a reversed meander.)
Theorem 3.4
 1.The bivariate generating function of meanders avoiding the pattern p iswhere \(u_1(t), \ldots , u_c(t)\) are the small roots^{5} of \(K(t, u)=0\). If one does not keep track of the final altitude, this yields$$\begin{aligned} M(t,u) = \frac{R(t,u)}{u^c \, K(t,u)} \prod _{i=1}^c \bigl (uu_i(t)\bigr ), \end{aligned}$$(9)$$\begin{aligned} M(t) = M(t,1)= \frac{R(t,1)}{K(t,1)} \prod _{i=1}^c \bigl (1u_i(t)\bigr ). \end{aligned}$$(10)
 2.The generating function for excursions avoiding the pattern p is$$\begin{aligned} E(t)=M(t,0)= {\left\{ \begin{array}{ll} \displaystyle {\frac{(1)^{c}}{t} \prod _{i=1}^c u_i(t)} &{} \text { if } {\text {alt}}(p)>c\text{, } \\ &{} \\ \displaystyle {\frac{(1)^{c}}{t^\ell t} \prod _{i=1}^c u_i(t)} &{} \text { if } {\text {alt}}(p)=c. \end{array}\right. } \end{aligned}$$(11)
Theorem 3.5
 1.The bivariate generating function for meanders avoiding the pattern p iswhere \(u_1(t), \ldots , u_e(t)\) are the small roots of \(K(t, u)=0\). If one does not keep track of the final altitude, this yields$$\begin{aligned} M(t,u) = \frac{1}{u^e \, K(t,u)} \prod _{i=1}^e \bigl (uu_i(t)\bigr )\text{, } \end{aligned}$$(12)$$\begin{aligned} M(t) = M(t,1)= \frac{1}{K(t,1)} \prod _{i=1}^e \bigl (1u_i(t)\bigr )\text{. } \end{aligned}$$(13)
 2.The generating function for excursions avoiding the pattern p iswhere \(D(t):=[u^0] u^e K(t,u)\) is either some power of t, or a difference of two powers of t (similarly to (11), but with more cases that will be specified in the proof in Sect. 5).$$\begin{aligned} E(t)=M(t,0)= \displaystyle {\frac{(1)^{e}}{D(t)} \prod _{i=1}^e u_i(t)}\text{, } \end{aligned}$$(14)
Remark 3.6
(Compatibility with the literature on classical lattice paths) Notice that for these four classes of lattice paths, if one forbids a pattern of length 1 or if one uses symbolic weights for each step, this recovers the formulas from Banderier and Flajolet [9].
The proof will involve the adjacency matrix of a finite automaton which encodes accumulating the prefixes of the forbidden pattern. This will be introduced in the next section.
4 Automaton, Adjacency Matrix A, and Kernel \(K=ItA\)
4.1 The Automaton and Its Adjacency Matrix

For all i, j such that \(j > i+1\), we have \(A_{i,j}=0\).

The superdiagonal entries are \(A_{i,i+1}=u^{a_i}\).

For each j such that \(2 \le j \le \ell \), every entry in the jth column is either 0 or \(u^{a_{j1}}\), depending on a procedure explained below.

The first column is such that all rows sum to P(u), except for the last row, where this entry is \(P(u)u^{a_{\ell }}\) (because the transition with \(a_\ell \) is forbidden in this row, as it would create an occurrence of the pattern p).
 1.
If \([a_1, a_2, \ldots , a_{\ell d1}]=[a_{d+2}, a_{d+3}, \ldots , a_{\ell }]\), then all the entries \(A_{i,j}\) with \(ij=d\), \(j \ge 2\), are 0.
 2.
Otherwise, if k is the smallest number such that \(a_{k} \ne a_{d+1+k}\), then \(A_{d+k+1, k+1} = u^{k}\), unless a smaller d yielded \(u^{k}\) in the same row, to the right of this position (if this happens, \(A_{d+k+1, k+1}=0\)).
4.2 Algebraic Properties of the Kernel: Link with the Autocorrelation Polynomial
It is well known that the matrix \(1/(ItA)= \mathrm {adj}(ItA) / \det (ItA)\) (where I is the identity matrix) plays an important role in the enumeration of walks. We will see that this adjoint and this determinant also play a fundamental role in the enumeration of meanders. In fact, the role of \(\det (ItA)\) in our study is the analogue of the role played by \(1tP(u)\) in the study of Banderier and Flajolet [9], but, as we shall see, the situation is more involved in our case.
Proposition 4.1
Another important quantity related to the adjacency matrix is what we call the autocorrelation vector, defined as \(\vec {{\mathbf {v}}}:=\mathrm {adj}(ItA)\cdot \vec {{\mathbf {1}}}\), where \(\vec {{\mathbf {1}}}\) denotes the column vector \((1, 1, \ldots , 1)^{\top }\) of size \(\ell \times 1\). In Proposition 4.2 we give a simple combinatorial description of this vector \(\vec {{\mathbf {v}}}\); in particular, this description has the advantage of smaller computational cost than getting it via a casebycase matrix inversion.
Proposition 4.2
 1.
w is any walk starting in state \(X_{j}\) and ending in state \(X_{\ell }\),
 2.
\(a_\ell \) is the last step of the pattern p,
 3.
\({{\bar{q}}}\) corresponds to a term of R(t, u) (that is, \({{\bar{q}}}\) is the complement of some presuffix q of p).
As this proposition is a little bit hard to digest, we give an example after the proof (Example 4.3), and the reader will see that the situation is in fact simpler than what she may have feared.
Proof
First note that \((ItA)\vec {{\mathbf {v}}}=K(t,u)\vec {{\mathbf {1}}}\) and \(A\vec {{\mathbf {1}}}=P(u)\vec {{\mathbf {1}}}u^{a_\ell }\vec {{\mathbf {e}}_\ell }\), where \(\vec {{\mathbf {e}}_\ell }=(0,\dots ,0,1)^\top \).
The entries of \((ItA)^{1}\) are the generating functions of lattice paths not containing the pattern \(p=[a_1,a_2,\dots ,a_\ell ]\), specifically: the (i, j)entry of this matrix is the generating function of such walks that start in state \(X_{i}\) and end in state \(X_{j}\). Therefore, the jth component of \((ItA)^{1} \vec {{\mathbf {e}}_\ell }\, tu^{a_\ell } R(t,u)\) is the generating function of all lattice paths that are composed as follows: first, start with a pavoiding walk w that begins in the state \(X_{j}\) and ends in state \(X_{\ell }\), followed by the single step \(a_\ell \), and then finally by some complement \({{\bar{q}}}\) of some presuffix q of the pattern p. Note that if the walk w is long enough and \(\bar{q}\) is not the empty sequence, then adding the step \(a_{\ell }\) to w makes it end with q. So, \(w.a_{\ell }.\bar{q}\) has an occurrence of p at the very end. Furthermore, having added the step \(a_{\ell }\) to w an occurrence of p is created unless w was shorter than \(\ell \).
The second term, \((ItA)^{1} \vec {{\mathbf {1}}}t^\ell u^{{\text {alt}}(p)}\), is the generating function of all walks avoiding p to which we add p at their end. We may have a single occurrence of p at the end of the walk or two overlapping patterns if the first part of the walk ended with some prefix of p, which needs only a presuffix to complete p; the completion is done by the final occurrence of p.
We observed that the two terms on the righthand side of (17) correspond to sets of lattice paths. Clearly, any walk ending with p and having only one occurrence of p is in both sets. And so is any walk having two overlapping occurrences of p at its very end. However, the first set contains in addition paths being too short for having an occurrence of p: they are precisely the set of paths described in the assertion. Since this set is finite, its generating function is a polynomial. \(\square \)
Example 4.3

For the first component of \(\vec {{\mathbf {s}}}\), the subtracted set of walks is in fact empty. Indeed, each path w from \(X_1\) to \(X_6\), followed by the step \(a_{\ell }\), has at least length 6. So, there are no walks that satisfy the condition of being shorter than \(\ell \). Therefore, we have \(\vec {{\mathbf {s}}}_1=0\) (which holds in full generality, implying \(\vec {{\mathbf {v}}}_1=R(t,u)\) for any pattern).

For the second component of \(\vec {{\mathbf {s}}}\), we start in state \(X_2\). The length of the prefactor w has to be smaller than \(\ell 1\), and the only way to reach \(X_6\) in strictly fewer than 5 steps is the path \(X_2 \rightarrow X_3 \rightarrow X_4 \rightarrow X_5 \rightarrow X_6\), which generates 2112, to which we append the last letter \(a_\ell =1\). Then the only possible \({{\bar{q}}}\) is the empty one (otherwise the path will be too long). Thus, the subtracted set of walks is \({\mathcal {S}}_2=\{2112.1.\varepsilon \}\), and its generating function is \(t^5u^7\).

For the third component of \(\vec {{\mathbf {s}}}\), the only sufficiently short walk from \(X_3\) to \(X_6\) is \(X_3 \rightarrow X_4 \rightarrow X_5 \rightarrow X_6\), and the only possible choice of \({{\bar{q}}}\) is \(\varepsilon \). Thus, the subtracted set of walks is \({\mathcal {S}}_3=\{112.1.\varepsilon \}\), and its generating function is \(t^4u^5\).

For the fourth component of \(\vec {{\mathbf {s}}}\), we have two ways to go from \(X_4\) to \(X_6\): \( X_4 \rightarrow X_5 \rightarrow X_6\) and \(X_4 \rightarrow X_3 \rightarrow X_4 \rightarrow X_5 \rightarrow X_6\). In both cases, the only possible choice of \({{\bar{q}}}\) is \(\varepsilon \). Thus, the subtracted set of walks is \({\mathcal {S}}_4=\{12.1.\varepsilon ,2112.1.\varepsilon \}\), and its generating function is \(t^3u^4+t^5u^7\).

For the fifth component of \(\vec {{\mathbf {s}}}\), starting in \(X_5\), only one short path to \(X_6\) is possible: \(X_5 \rightarrow X_6\), but in this case we have two choices for \({{\bar{q}}}\): \(\varepsilon \) and 121. This gives the subtracted set of walks \({\mathcal {S}}_5=\{2.1.\varepsilon , 2.1.121\}\), and its generating function is \(t^2u^3+t^5u^7\).

For the sixth component of \(\vec {{\mathbf {s}}}\), starting in \(X_6\), the subtracted set of walks is \({\mathcal {S}}_6=\{\varepsilon .1.\varepsilon ,\varepsilon .1.121\}\), and its generating function is \(tu+t^4u^5\).
4.3 Analytic Properties of the Kernel: Newton Polygons and Geometry of Branches
Let us end this section with the proof of an important property of the kernel, the number of “small” and “large” roots u(t) of \(K(t, u)=0\):
Proposition 4.4
(Small and large roots of the kernel K) All roots u(t) of K(t, u) are either small (meaning \(\lim _{t \rightarrow 0} u(t)=0\)) or large (meaning \(\lim _{t \rightarrow 0} u(t)= \infty \)). Let \(d_K\) denote the degree of K(t, u) in u, i.e., \(d_K=\max \{j\,\mid \, [u^j]K(t,u)\ne 0\}\), and let \(l_K\) denote the lowest power of u in the monomials of K, i.e., \(l_K=\min \{j\,\mid \, [u^j]K(t,u)\ne 0\}\). Then, K has e small roots and f large roots, where \(e=\max (0, l_K)\) and \(f=\max (0,d_K)\)
Remark 4.5
The nested \(\max /\min \) is needed for the cases where either \(l_K>0\) or \(d_K<0\). An example for such a model is \({\mathcal {S}}=\{1,3\}\) and \(p=[1,1]\), where we have \(K=1u^3tu^2t^2\).
Proof
Recall that the kernel K is defined as \(K(t,u)=(1tP(u))R(u)+t^\ell u^{{\text {alt}}(p)}.\) Now, consider \(u^e K(t,u)\), which is a polynomial, and draw for each of its monomials \(t^{r_1} u^{r_2}\) a point \((r_1,r_2)\) in the plane. This gives a set of points \({\mathcal {P}}\). The Newton polygon^{6} of K is the boundary of the convex hull of \({\mathcal {P}}\); of particular interest to us are the segments of this polygon that are visible when we look from the left: their slope (it is always a rational number) will give the exponent of the Puiseux expansions of each root of K. Figure 3 shows schematically all possible shapes for the case \(R(t,u)=1\) (if \(R(t,u)\ne 1\), the possible shapes are more diverse). Each segment (in green) which lies below the point (0, e) (they all have a negative slope) gives the Puiseux expansion of a set of small roots. Each segment (in red) which lies above the point (0, e) (they all have a positive slope) gives the Puiseux expansion of a set of large roots. We now focus on the small roots and prove the proposition for them. (The proof for large roots is similar and can be obtained from this one by the replacement \(u \mapsto \frac{1}{u}\), which corresponds to the vertical reflection of the Newton polygon.)
If the Newton polygon has no segment of negative slope (the ones drawn in green in Fig. 3), then K is a polynomial in u (and in t as well) having the constant term 1. Hence, \(K(t,u)=1+Q(t,u)\) where Q(t, u) is a polynomial in t and u with \(\lim _{(t,u)\rightarrow (0,0)}Q(t,u)=0\). This implies that small functions u(t) can never compensate the constant term near \(t=0\), and so there are zero small roots and \(d_K\) large roots, in accordance with \(e=0\) and \(f=d_K\).
Several examples that demonstrate the diversity of the behaviour of real branches of \(K(t,u)=0\)
Among all monomials of the form \(t^j u^{e}\) in K(t, u), let \(t^\lambda u^{e}\) be the one where \(\lambda \) is minimal. If we draw the Newton polygon of the polynomial \(u^e K(t,u)\), then it contains the points (0, e) and \((\lambda ,0)\). Assume that the polygonal line we obtain when traversing the boundary of the Newton polygon from (0, e) to \((\lambda ,0)\) consists of line segments \(\Sigma _1,\dots , \Sigma _r\) of respective slopes \(\beta _1/\alpha _1\), ..., \(\beta _r/\alpha _r\). Then, the line segment \(\Sigma _j\) corresponds to a set of small roots of K(t, u) each satisfying \(u(t)\approx t^{\alpha _j/\beta _j}\) as \(t\rightarrow 0\). The number of such roots is equal to the height of \(\Sigma _j\), which is the difference of the ycoordinates of the two endpoints of \(\Sigma _j\). In particular, this implies that the number of small roots of K(t, u) is equal to e.
Using the same reasoning for the line segments (drawn in red in Fig. 3) above (0, e), we obtain that the number of large roots of the kernel is indeed f. \(\square \)
Table 2 on the next page gives several examples of plots of these small and large roots. Then, equipped with all these notions, we can give the proofs of our main theorems.
5 Proofs of the Generating Functions for Walks, Bridges, Meanders, and Excursions
In addition to our usual notations for generating functions of different classes of paths, we denote by \(W_{\alpha }=W_{\alpha }(t,u)\) (where \(1 \le \alpha \le \ell \)) the bivariate generating function of those walks avoiding the pattern p that terminate in state \(\alpha \); similarly \(M_{\alpha }=M_{\alpha }(t,u)\) for meanders avoiding the pattern p that terminate in state \(\alpha \).
Proof of Theorem 3.1
We now turn to the consequences of this formula for W(t, u) when one considers bridges.
Proof of Theorem 3.1
We now consider the nonnegativity constraint for the paths.
Proof
Thus, if we multiply (23) by \(\vec {{\mathbf {v}}}_{u=u_i}\), the lefthand side vanishes. In other words, the equation \((F_1(t,u), \ \ldots , \ F_{\ell }(t,u)) \, \vec {{\mathbf {v}}}(t, u)=0\) is satisfied by every small root \(u_i(t)\) of K(t, u).
If the degree of \(\Phi (t,u)\) is precisely e, the formula simplifies as G is then just the leading term (in u) of \(\Phi (t,u)\). As we shall show now, this happens if p is a quasimeander (as introduced in Definition 3.3).
Proof of Theorem 3.4
(Generating function of meanders, whenpis a quasimeander) First, we notice that all the powers of u in R(t, u) are nonnegative, and \({\text {alt}}(p) \ge c\). Moreover, if \({\text {alt}}(p)=c\), the cancellation of terms with \(u^{c}\) in K(t, u) is not possible.^{7} Therefore, the lowest power of u in K(t, u) is c, and thus we have \(e=c\) by Proposition 4.4.
Let us now simplify this formula for excursions, i.e., when \(u=0\).
Proof of Theorem 3.4
(Generating function of excursions, whenpis a quasimeander) The generating function of excursions is given by \(E(t) = M(t,0)\). If \({\text {alt}}(p) >c\), we have, as u tends to 0, \(K(t,u)\sim tu^{c}R(t,0)\) from (16). If \({\text {alt}}(p) = c\), then we have \(R(t,u) = 1\) and \(K(t,u)\sim tu^{c} + t^\ell u^{c}\). In both cases, (11) follows. \(\square \)
We now handle the next interesting class of patterns leading to generating functions with a nice closed form: the case of reversed meanders. Recall that a reversed meander is a lattice path whose terminal point has a (strictly) smaller ycoordinate than all other points. Moreover, we define a positive meander to be a meander that never returns to the xaxis.
Proof of Theorem 3.5
(Generating function of meanders, whenpis a reversed meander) If p is a reversed meander, then all the terms of R(t, u), except for the monomial 1, contain negative powers of u, and we also have \({\text {alt}}(p)<0\). Therefore, the highest power of u in K is d, and (by Proposition 4.4) the number of large roots of \(K(t, u)=0\) is d: we denote them as above by \(v_1, \ldots , v_d\). The number of small roots can be in general higher than c: as usual, we denote it by e, and the roots themselves by \(u_1, \ldots , u_e\).
Remark 5.1
It is interesting to notice that though M(t, u), for p being a quasimeander [as given in (29)], is similar to M(t, u), for p being a reversed meander [as given in (39)], the latter does not contain the factor R(t, u) even if p has a nontrivial autocorrelation.
Remark 5.2
It is also worth mentioning that if only the terminal point of the pattern p has negative ycoordinate, then p is both a quasimeander and a reversed meander, and \(R=1\). Therefore, we have \({M(t,u)= \frac{1}{u^e \, K(t,u)} \, \prod _{i=1}^e (uu_i(t))}\) by both Theorems 3.4 and 3.5.
Proof of Theorem 3.5

In all the terms of R(t, u), the powers of u are nonpositive.

Moreover, if \(t^{m_1}/u^{\gamma _1}\) and \(t^{m_2}/u^{\gamma _2}\) are two distinct terms in R(t, u) such that \(0 \le m_1 < m_2\), then we have \(0 \le \gamma _1 < \gamma _2\).

Case 1: \(c+\gamma > {\text {alt}}(p)\). Then \(e=c+\gamma \) and we have \(D(t) = t^{m+1}\).

Case 2: \(c+\gamma < {\text {alt}}(p)\). Then \(e={\text {alt}}(p)\) and we have \(D(t)= t^{\ell }\).

Case 3: \(c+\gamma = {\text {alt}}(p)\) and \(\ell \ne m+1\). Then \(e=c+\gamma = {\text {alt}}(p)\) and \(D(t) = t^{\ell }t^{m+1}\).

Case 4: \(c+\gamma = {\text {alt}}(p)\) and \(\ell = m+1\). If \(\ell \ge 2\), then \(m\ge 1\), and therefore \(R(t,u) \ne 1\). Then \(e=c+\gamma '\) and \(D(t) = t^{m'+1}\). As usual, we ignore the degenerate case \(\ell =1\).
Now that we have proven these closed forms for the generating functions, we can turn to the asymptotics of their coefficients.
6 Asymptotics of Lattice Paths Avoiding a Given Pattern
The aim of this section is to characterize the asymptotics of the number of paths (walks, bridges, meanders, excursions) with steps from \({\mathcal {S}}\) avoiding a given pattern p.
In order to avoid pathological cases, we now focus on “generic” walks.
Definition 6.1

Property 1. The generating functions B(t), M(t) and E(t) are algebraic, not rational.

Property 2. They have a unique dominant singularity, which is algebraic, not a pole.

Property 3. The factor G(t, u) in Eq. (8) is a polynomial in t.

Property 4. Let \(\rho \) be the smallest positive real number such that a large branch meets a small branch at \(t=\rho \). No large negative branch (i.e., a branch of \(K(t,u)=0\) such that \(\lim _{t\rightarrow 0^+} u(t)=\infty \)) meets a small negative branch at \(t=\rho \).

Property 5. The smallest positive root of K(t, 1) is simple.

For Property 1, it can be the case that the forbidden pattern leads to a degenerate model, in the sense that it is no more involving any stack. Thus, we have words generated by a regular automaton (hence, the generating functions are rational and the asymptotics are well understood). Example: \({\mathcal {S}}=\{1,1\}\) with \(p=[1,1]\) or \(p=[1,1]\).

For Property 2, it is proven in [7] that several dominant singularities appear if and only if the gcd of the pairwise differences of the steps is not 1. In this case, the asymptotics are obtained via [14, Theorem 8.8]. Moreover, polar singularities are possible, but these are easy to handle.

For Property 3, it is satisfied in many natural cases (like e.g. in Theorems 3.4 and 3.5) and we analyse in the followup article [3] what happens otherwise.

For Property 4, we conjecture that it always holds. In fact, we have a proof for many classes of walks, but some remaining cases are open. Note that it is possible to exhibit cases where one small negative root meets a large negative root, at some \(\rho '>\rho \): this is e.g. the case for \({\mathcal {S}}=\{2, 1, 0, 1, 2\}\) with \(p=[0, 1, 2]\). Moreover, it is also possible that two small negative roots meet at \(\rho \): e.g. for \({\mathcal {S}}=\{2,1\}\) with \(p=[1,2,1,2]\).

For Property 5, an example of a double root for K is given by \({\mathcal {S}}=\{1,1\}\) and \(p=[1,1]\) (this example corresponds to the very last drawing in Table 2). Double or higher multiplicity roots would just create additional subcases (trivial to handle) in the following theorems.
Table 2 from Sect. 4 illustrates that we always have a small branch and one large branch whose shape in general resembles that of \(u_1 \cup v_1\) observed for classical paths by Banderier and Flajolet. In one sense, the geometry of the branches of K observed for classical paths is now perturbed by the pattern avoidance constraint: this perturbation adds new branches. In the next section, we introduce the generating function W(t, u, v) where v encodes the number of occurrences of the pattern p. One can then play with v like if it would encode a Boltzmann weight/Gibbs measure (a typical point of view in statistical mechanics): moving the parameter v in a continuous way from 1 to 0 gives a rigorous explanation of this perturbation phenomenon, and shows the coherence with the emergence of new branches. More information about these branches (and their Puiseux expansions) can be derived from the Newton polygon associated with the kernel (see Proposition 4.4 and [34]).
Lemma 6.2
(Location and nature of the dominant singularity) For any generic model, the dominant singularity of B(t) and E(t) is \(\rho \), the smallest real positive number such that a small branch meets a large branch at \(t=\rho \). (The branches refer to the roots of \(K(t,u)=0\), as defined in (4)). We call these branches \(u_1\) and \(v_1\). Additionally, their branching point is a square root singularity.
Proof
Lattice paths avoiding a given pattern can be generated by a pushdown automaton (see Fig. 1). Accordingly, they can be generated by a contextfree grammar, and their generating functions thus satisfy a “positive” system of algebraic equations (see [27]). Therefore, the asymptotic number of words of length n in such languages is of the form \(C \rho ^{n} n^\alpha \). When the system is not strongly connected, \(\alpha \) is either an integer (if \(\rho \) is a pole), either a dyadic number (if one has an iterated square root Puiseux singularity at \(\rho \)), as proven by Banderier and Drmota in [8]. For excursions, one has a strongly connected dependency graph (see Fig. 1); the dominant singularity \(\rho \) (or, possibly, the dominant singularities) thus behaves like a square root, as we have generic walks (and not a degenerate case where we face a polar singularity).
Now, for generic walks, because of the product formula (8) for excursions, one (or several) of the small roots have to follow this square root Puiseux behaviour. By Pringsheim’s theorem, this has to be at a place \(0<\rho \le 1\). Note that the Pólya–Fatou–Carlson theorem [26] on pure algebraic functions with integer coefficients says that they cannot have radius of convergence 1. Therefore, the first crossing between a small and large branch is at \(0<\rho <1\) (i.e., \(\rho =1\) or any other root of \(tt^\ell \), cannot be the dominant singularity). Now, by Proposition 4.1, the geometry of the branches implies that this branching point is at a location where a large branch meets a small branch, because if the branching point comes from the intersection of small roots only (see the examples 1, 6, 8 in Table 2 for such a case), then their product will be regular. So, \(\rho \) has to be the smallest real positive number where a small branch meets a large branch.
When one does not take into account occurrences of a pattern, the generating function of bridges is essentially the logarithmic derivative of the generating function of excursions, and they have the same radius of convergence (the cycle lemma, the identity \(B=1+ E t \partial _t A\), Spitzer’s and Sparre Andersen’s formulas are alter egos of this relation, see the paragraph “On the relation between bridges and excursions” in [9, Theorem 5]). For walks with a forbidden pattern, this simple relation is not holding anymore and there is no apparent equation linking the two generating functions. Nevertheless, the numbers \(e_n\) of excursions and \(b_n\) of bridges of length n still satisfy \(e_n\le b_n \le n e_n\) (this is easily seen by doing the n cyclic shifts of each excursion). This implies that E(t) and B(t) have the same radius of convergence. \(\square \)
Equipped with this additional information on the roots and the way they cross, we can derive the following asymptotic results. Note that we use the notations \(K_t(t,u)\) for \((\partial _t K)(t,u)\), and \(K_{uu}(t,u)\) for \((\partial _u^2 K)(t,u)\). We start with the asymptotics of walks on \({\mathbb {Z}}\) with a forbidden pattern.
Theorem 6.3
Proof
This follows from the partial fraction decomposition of \(W(t)=\frac{R(t,1)}{K(t,1)}\), where \(\rho _K\) is a simple pole as the model is generic. \(\square \)
Now, for excursions and bridges, the corresponding generating functions have an algebraic dominant singularity; this leads to the following theorems.
Theorem 6.4
Proof
Theorem 6.5
Proof
We now introduce the notion of drift, which plays a role for the asymptotics of meanders.
Definition 6.6
As usual, the drift is not playing a role for the asymptotics of excursions and bridges. Indeed, the constraint to force the walk to end at altitude zero is there “killing” the drift. This is best seen by a “time reversal” argument: under this transformation, a bridge stays a bridge (which then avoids the reverse forbidden pattern), one thus gets the same generating function (note that K(t, u) then becomes K(t, 1 / u)); therefore, the asymptotics have to be independent of the drift. A similar reasoning holds for excursions. Now, the next theorem shows how the drift does play a role for the asymptotics of meanders. For meanders with negative or zero drift, the quantity \(\rho \) from Lemma 6.2 is also the radius of convergence of M(t). For meanders with positive drift, the radius of convergence of M(t) is the dominant pole of 1 / K(t, 1).
Theorem 6.7

If \(\tau =1\) and \(\rho _K=\rho \), then we are in the “zero drift” case.

If \(\tau >1\) and \(u_1(\rho _K)=1\) and all large roots v satisfy \(v(t)\ne 1\) for \(\rho _K<t<\rho \), then we are in the “negative drift” case.

If either \(\tau <1\), or \(\tau =1\) but \(\rho _K<\rho \), or \(\tau >1\) but some large root v satisfies \(v(\rho _K)=1\), then we are in the “positive drift” case.
Proof
7 Limit Law for the Number of Occurrences of a Pattern
Our approach also allows us to count the number of occurrences of a pattern in paths. As usual, an occurrence of p in w is any substring of w that coincides with p, and when we count them we do not require that the occurrences will be disjoint. For example, the number of occurrences of 11 in 1111 is 3. One has
Theorem 7.1
Proof
We give two proofs, each of them having its own interest. Both of them are of wider applicability, see [42, pp. 60 and 212].
First proof, via symbolic inclusionexclusion. Define a cluster as a sequence of repetitions of the pattern p (possibly overlapping), where each occurrence of p is marked by the variable v, so the set \({\mathcal {C}}\) of clusters is given by \({\mathcal {C}}= v p {\text {Seq}}(v (\mathcal {{{\bar{Q}}}}\epsilon ))\), where \(\mathcal {{{\bar{Q}}}}\epsilon \) is the set of nonempty complements of presuffixes of p [the generating function of which is \(R(t,u)1\), see Formula (3)]. Obviously, \(W(t,u,v+1)= {\text {Seq}}({\mathcal {S}} +{\mathcal {C}})\). This directly gives (42).
Accordingly, the trivariate kernel is thus defined as \(K(t,u,v):=\det (ItA)\). Note that for \(v=0\) we get the kernel from the avoidance case [see eq. (4)], and for \(v=1\) we get \(1tP\) (which is, as expected, the kernel from [9]). The formulas for the trivariate generating functions of bridges, meanders, and excursions are thus like in Theorems 3.1 and 3.2, where the \(u_i\)’s are now the small roots of the trivariate kernel. This allows us to prove a universal asymptotic behaviour, an instance of what Flajolet and Sedgewick pleasantly called Borges’s theorem (we comment more on it in the conclusion).
Theorem 7.2
Proof
All these combinatorial structures are generated by contextfree grammars [7, 36, 56, 61]. Accordingly, their generating functions satisfy a positive algebraic system. Thus, it leads to Gaussian limit laws, as proven in [8, Theorem 9]: it comes from following the dependencies in the graph associated with the system, and applying Hwang’s quasipower theorem to each component. To apply it, a positive variance condition has to be checked: this is done via the trivariate version of the formulas of Theorems 3.1 and 3.2, with the additional variable v counting the number of occurrences of the pattern. \(\square \)
8 Examples, Pushdown Automata

acceptance by empty stack, this corresponds to the generation of excursions;

acceptance by final state (whatever the value of the stack is), this corresponds to the generation of meanders.
Example 8.1
(Forbidden patterns) We start with our initial problem of lattice paths with forbidden patterns. Our theorems rediscover, in a uniform way, numerous results by different authors obtained in the last years by different methods. We present some of them in Table 3.
A few examples in which our theorems produce sequences from the OnLine Encyclopedia of Integer Sequences
Steps, pattern of length \(\ell \), model  Generating function  OEIS reference\(^{\mathrm{a}}\) 

\(\begin{array}{l}{\mathcal {S}}=\{1, 0, 1\}\\ p=[1, 0, \ldots , 0, 1] \\ \hbox {Bridges}\end{array}\)  \(\begin{array}{l}\displaystyle {\frac{1}{\sqrt{12t3t^2+2t^{\ell }2t^{\ell +1}+t^{2\ell }}}}\end{array}\)  
\(\begin{array}{l}{\mathcal {S}}=\{1, 0, 1\}\\ p=[1, 0, \ldots , 0, 1] \\ \hbox {Meanders}\end{array}\)  \(\begin{array}{l}\frac{13t+t^{\ell }\sqrt{12t3t^2+2t^{\ell }2t^{\ell +1}+t^{2{\ell }}}}{2t(13t+t^{\ell })}\end{array}\)  
\(\begin{array}{l}{\mathcal {S}}=\{1, 0, 1\}\\ p=[1, 0, \ldots , 0, 1] \\ \hbox {Excursions}\end{array}\)  \(\begin{array}{l}\frac{1t+t^\ell \sqrt{12t3t^2+2t^\ell 2t^{\ell +1}+t^{2\ell }}}{2t^2}\end{array}\)  
\(\begin{array}{l}{\mathcal {S}}=\{1, 1\}\\ p=[1, 1, 1, 1, \ldots , 1] \\ \hbox {Excursions}\end{array}\)  \(\begin{array}{l}\frac{1t^{\ell +1}\sqrt{14t^2+2t^{\ell +1}+4t^{\ell +3}3t^{2\ell +2}}}{2t^2(1t^{\ell 1})}\end{array}\)  
\(\begin{array}{l}{\mathcal {S}}=\{1, 1\}\\ p=[1, 1, 1, 1,\dots , 1] \\ \hbox {Excursions}\end{array}\)  \(\begin{array}{l}\frac{1t^{\ell +2}\sqrt{14t^2+6t^{\ell +2}4t^{2\ell +2}+t^{2\ell +4}}}{2t^2(1t^{\ell })}\end{array}\) 
Next we now give several examples of a wide variety of questions on lattice paths, that can also be tackled with our approach. In some cases, we omit the closed forms: they are just a direct application of our approach, once the adjacency matrix A is known. Some pushdown automata lead to a system for which an additional argument is needed to identify the factor G in Theorem 3.2; we study this situation in more detail in [3].
Example 8.2
Example 8.3
(Basketball with alternation of team scoring)
Example 8.4
(Walks/meanders/excursions of bounded height) If one considers meanders of altitude bounded by h, they are easily generated by an automaton with h states. In fact, the reader now familiar with our approach will realize that a pushdown automaton with just one state is already enough: the kernel equation will encode the boundedness by h and the positivity constraint! See [7, 12, 23] for further considerations on the closed forms obtained. This bypasses the quickly intractable resolution of a system \(h\times h\).
Example 8.5

in the first model, the halfplane is the one over the line \(x = 0\); the heights of the steps are \({\text {alt}}(\mathsf {n}) = 1\), \({\text {alt}}(\mathsf {e}) = 0\) and \({\text {alt}}(\mathsf {s}) = 1\);

in the second model, the halfplane is the one over the line \(x = y\); the heights are \({\text {alt}}(\mathsf {n}) = 1\), \({\text {alt}}(\mathsf {e}) = 1\) and \({\text {alt}}(\mathsf {s}) = 1\);

in the third model, the halfplane is the one over the line \(x = y\); the heights are \({\text {alt}}(\mathsf {n}) = 1\), \({\text {alt}}(\mathsf {e}) = 1\) and \({\text {alt}}(\mathsf {s}) = 1\).
These models are illustrated in Fig. 7. Each of them leads to an algebraic generating function, expressible via our method as a closed form involving the roots of the kernel. \(\square \)
Example 8.6
(Motzkin paths with horizontal steps only at even/odd altitude) If one wants to allow some specific set of steps depending on the altitude modulo some period, the vectorial kernel method will do the job! Let us illustrate this with Motzkin paths (\({\mathcal {S}}=\{1,0,1\}\)) with horizontal steps only at even altitude.
If we want to enumerate Motzkin paths with horizontal steps only at odd altitude, we adjust the automaton as shown on the left, and obtain the sequences A327421 (for meanders) and A090345 (for excursions), see e.g. [18, 32]. \(\square \)
Example 8.7
(Duchon’s club without lonely visitors)
The terminology of Duchon’s club was introduced in [9] as a playful reference to the nice article [36]. As some readers may wonder, we have to add that it is in no way an allusion to some fancy habit of our highly honourable and very respectable colleague Philippe Duchon! Here is the story. In the early evening, the club opens empty, and then, during the full night, people are entering in couples and leaving in groups of three people. Finally, the club closes empty at the end of the night. For sure, you do not want to be alone in such a club! What is the number of possible scenarios? These are excursions with step set \({\mathcal {S}}=\{2, 3\}\) and never going to altitude 1. This enumeration is encoded by a single state automaton, and the kernel equation then fully handles all the constraints. This leads to the sequences A327422 (for meanders) and A327423 (for excursions). \(\square \)
Example 8.8
(Counting/avoiding humps and peaks in Motzkin paths) Consider Motzkin walks: \({\mathcal {S}} = \{ 1, 0, 1 \}\). A peak is the pattern \([1,1]\). A hump is an occurrence of the pattern \([1,0^*,1]\), that is, 1 followed by a (possibly empty) sequence of 0s, followed by \(1\). Humps were considered e.g. in [35, 60, 72]. We first consider the generating function of walks counting the number of humps, similarly to our approach from Sect. 7.
The transition which was forbidden in the pattern avoidance setting is here represented with a variable v in the adjacency matrix A. This yields the trivariate generating function with respect to length (t), final altitude (u), and number of occurrences of humps (v). In particular, this leads to a Gaussian distribution, see Borges’s Theorem 7.2.
OEIS entries for the sequences that enumerate Motzkin meanders (M) and excursions (E) with avoidance and/or parity constraints on the number of peaks and humps.
Even number of humps  Odd number of humps  Any number of humps  

No peaks  
Even number of peaks  
Odd number of peaks  
Any number of peaks 

Dyck excursions with up steps colored in two colors (blue and red, denoted \({\mathsf {U}}_B\) and \({\mathsf {U}}_R\)), and avoiding the pattern \({\mathsf {U}}_B {\mathsf {U}}_R\) (A006013, which also enumerates many other objects!),

Motzkin excursions avoiding UD, UU and DD (see A023432 and [31]),

Schröder bridges (also called Delannoy paths) with k occurrences of \(\mathsf {DD}\) that cross \(y=0\) (see A110121 and [13, 80]),

excursions from (0, 0) to (3n, 0) that use steps \({\mathsf {u}}=(2,1)\), \({\mathsf {U}}=(1,2)\), and \({\mathsf {D}}=(1,1)\), and have k peaks \(\mathsf {uD}\) or \(\mathsf {UD}\) (see A108425 and [30]),

Łukasiewicz paths with k\(\mathsf {U}\)steps that start at an even level (see A091894 and [16, 20]).
For some of these examples, it is possible to tackle the problem by using some ad hoc decompositions, bijections with trees, or some variation of a classical kernel method [28, 37, 43, 46, 54, 55, 63, 64, 65, 68, 71, 75, 79]. Some examples require to mark several patterns at once, or to take a diagonal of our multivariate generating function (e.g. when the number of occurrences of a given pattern is required to be proportional to the length n, or to the number of occurrences of another pattern), etc. The method is really flexible and allows playing with variants of patterns, step sets, colors, weights, values of the stack, constraints modulo 2, 3, ... an endless game!
We emphasize that the full approach is automatable (sic Open image in new window ), so all such studies can now morally be considered as computationally solved by our vectorial kernel method, with some subtleties which will be presented in the followup article [3]. Thus, in a style dear to the heart of Shalosh B. Ekhad and Philippe Flajolet, the vectorial kernel method can be considered as a method to automatically prove enumerative and asymptotic results for pattern occurrences in structures given by an algebraic system of functional equations.
9 Conclusion
In this article, we presented a unifying way which gives the generating functions and asymptotics of all families of lattice paths with a forbidden pattern, and we proved that the number of occurrences of a given pattern is normally distributed. The same techniques would, for instance, allow us to do the asymptotics of walks having exactly m occurrences of a given pattern, or to consider patterns which are no longer a word but a regular expression.
It is also nice that our approach gives a method to solve in an efficient way the question of the enumeration and asymptotics of words generated by a pushdown automaton (or words belonging to the intersection of an algebraic language and a rational language). What is more, it is possible to use our functional equation approach to analyse the intersection of two algebraic languages. Note that testing whether this intersection is empty is known to be an undecidable problem, even for deterministic contextfree grammars (see e.g. [48]), so we cannot expect too much from a generic method in this case. However, we can specify the type of system of functional equations that we get: indeed, this problem is related to automata with two stacks, which, in turn, are known to have the same power as Turing machines; the evolution of these two stacks corresponds to lattice paths in the quarter plane (with steps of arbitrary length), the complexity of the problem is reflected by the fact that one can then get generating functions which are no more algebraic, Dfinite, or differentiallyalgebraic, and we do not expect some nice universal enumerative closedform results here but a wider zoo of formulas. On the other hand, some universal asymptotic behaviour is expected; it is highly believable that all these cases will be new instances of what Flajolet and Sedgewick called Borges’s Theorem: Any pattern which is not forbidden by design will appear a linear number of times in large enough structures, with Gaussian fluctuations.
For sure, it is more a metatheorem, a natural credo, and the claim should be established rigorously in each case. Naturally, may it be with tools of probability theory or of analytic combinatorics, there is always some technical conditions to check to ensure this claim. In this article, our closed form expressions for the generating functions were one of the keys, together with the universal behaviour of the small roots. This allowed us to prove this Gaussian behaviour for the number of occurrences of any given pattern. Year after year, this claim is established for more and more combinatorial structures (it was done for patterns in Markov chains, trees, maps, permutations, contextfree grammars, and now... lattice paths!).
Footnotes
 1.
Without this assumption, the corresponding models are easy to solve: they are leading to rational generating functions. Our enumeration results still hold in these easy cases, but are then just classical folklore.
 2.
The notation \([t^n] F(t)\) stands for the coefficient of \(t^n\) in the power series F(t).
 3.
Some weights (or probabilities) could be associated with each step. Most of the results would generalize accordingly, we omit them in this article to keep readability.
 4.
A similar notion also appears in the work of Schützenberger on synchronizing words [77]. It should also be added that the autocorrelations of a pattern allow to explain some famous paradoxes, e.g. why, in a random sequence of coin tosses, the pattern HTT is likely to occur much sooner (after 8 tosses on average) than the pattern HHH (needing 14 tosses on average): “All patterns are not born equal!” as Flajolet and Sedgewick pleasantly write in [42, Example IV.11].
 5.
It will be shown in the proof that in this case we have \(e=c\).
 6.
 7.
The only exception is the case of \(p=[c]\). The case of patterns of length 1 is not interesting, so we assume from now on that \(\ell \ge 2\). Yet, one can check that our formula is also valid for \(\ell =1\), with some adjustments in the special case \(p=[c]\).
Notes
Acknowledgements
Open access funding provided by University of Klagenfurt. We thank the organizers and referees of the conferences AofA 2018 and LATA 2018, where we presented some aspects of this work (see [1, 2]). This work was supported by the Austrian Science Fund (FWF), in the framework of the Special Research Program Algorithmic and Enumerative Combinatorics (Grant SFB F5003) and of the project Analytic Combinatorics: Digits, Automata and Trees (Grant P 28466N35).
References
 1.Asinowski, A., Bacher, A., Banderier, C., Gittenberger, B.: Analytic combinatorics of lattice paths with forbidden patterns: asymptotic aspects. In: 29th International Conference on Probabilistic, Combinatorial and Asymptotic Methods for the Analysis of Algorithms (AofA 2018). Leibniz International Proceedings in Informatics (LIPIcs), vol. 110, pp. 10.1–10.13 (2018)Google Scholar
 2.Asinowski, A., Bacher, A., Banderier, C., Gittenberger, B.: Analytic combinatorics of lattice paths with forbidden patterns: enumerative aspects. In: Language and Automata Theory and Applications. LATA 2018, volume 10782 of Lecture Notes in Computer Science, pp. 195–206. Springer (2018)Google Scholar
 3.Asinowski, A., Bacher, A., Banderier, C., Gittenberger, B.: Pushdown automata, the vectorial kernel method, and underdetermined functional equations. In preparation (2019)Google Scholar
 4.Ayyer, A., Zeilberger, D.: The number of [oldtime] basketball games with final score \(n:n\) where the home team was never losing but also never ahead by more than \(w\) points. Electron. J. Comb. 14(1), R19 (2007)MathSciNetzbMATHGoogle Scholar
 5.Bacher, A., Bernini, A., Ferrari, L., Gunby, B., Pinzani, R., West, J.: The Dyck pattern poset. Discrete Math. 321, 12–23 (2014)MathSciNetzbMATHCrossRefGoogle Scholar
 6.Bacher, A., BousquetMélou, M.: Weakly directed selfavoiding walks. J. Comb. Theory Ser. A 118(8), 2365–2391 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
 7.Banderier, C.: Combinatoire analytique des chemins et des cartes. Ph.D. thesis, Université Paris VI (2001)Google Scholar
 8.Banderier, C., Drmota, M.: Formulae and asymptotics for coefficients of algebraic functions. Comb. Probab. Comput. 24(1), 1–53 (2015)MathSciNetzbMATHCrossRefGoogle Scholar
 9.Banderier, C., Flajolet, P.: Basic analytic combinatorics of directed lattice paths. Theor. Comput. Sci. 281(1–2), 37–80 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
 10.Banderier, C., Gittenberger, B.: Analytic combinatorics of lattice paths: enumeration and asymptotics for the area. Discrete Math. Theor. Comput. Sci. Proc. AG, 345–355 (2006)MathSciNetzbMATHGoogle Scholar
 11.Banderier, C., Krattenthaler, C., Krinik, A., Kruchinin, D., Kruchinin, V., Nguyen, D., Wallner, M.: Explicit formulas for enumeration of lattice paths: basketball and the kernel method. In: Lattice Paths Combinatorics and Applications. Developments in Mathematics Series, vol. 58, pp. 78–118. Springer (2019)Google Scholar
 12.Banderier, C., Nicodème, P.: Bounded discrete walks. Discrete Math. Theor. Comput. Sci. AM, 35–48 (2010)MathSciNetzbMATHGoogle Scholar
 13.Banderier, C., Schwer, S.: Why Delannoy numbers? J. Stat. Plan. Inference 135(1), 40–54 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
 14.Banderier, C., Wallner, M.: The kernel method for lattice paths below a rational slope. In: Lattice Paths Combinatorics and Applications. Developments in Mathematics Series, vol. 58, pp. 119–154. Springer (2019)Google Scholar
 15.Baril, J.L.: Avoiding patterns in irreducible permutations. Discrete Math. Theor. Comput. Sci. 17(3), 13–30 (2016)MathSciNetzbMATHGoogle Scholar
 16.Baril, J.L., Kirgizov, S., Petrossian, A.: Enumeration of Łukasiewicz paths modulo some patterns. Discrete Math. 342(4), 997–1005 (2019)MathSciNetzbMATHCrossRefGoogle Scholar
 17.Baril, J.L., Pallo, J.M.: Motzkin subposets and Motzkin geodesics in Tamari lattices. Inf. Process. Lett. 114(1–2), 31–37 (2014)MathSciNetzbMATHCrossRefGoogle Scholar
 18.Barry, P.: Continued fractions and transformations of integer sequences. J. Integer Seq. 12(7), Article 09.7.6 (2009)Google Scholar
 19.Bendkowski, M., Grygiel, K., Lescanne, P., Zaionc, M.: Combinatorics of \(\lambda \)terms: a natural approach. J. Log. Comput. 27(8), 2611–2630 (2017)MathSciNetzbMATHCrossRefGoogle Scholar
 20.Bevan, D.: Permutations avoiding 1324 and patterns in Łukasiewicz paths. J. Lond. Math. Soc. (2) 92(1), 105–122 (2015)MathSciNetzbMATHCrossRefGoogle Scholar
 21.Bóna, M., Knopfmacher, A.: On the probability that certain compositions have the same number of parts. Ann. Comb. 14(3), 291–306 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
 22.BousquetMélou, M.: Rational and algebraic series in combinatorial enumeration. In: International Congress of Mathematicians, vol. III, pp. 789–826. EMS (2006)Google Scholar
 23.BousquetMélou, M.: Discrete excursions. Sém. Lothar. Comb. 57, Article B57d (2008)Google Scholar
 24.BousquetMélou, M., Jehanne, A.: Polynomial equations with one catalytic variable, algebraic series and map enumeration. J. Comb. Theory Ser. B 96(5), 623–672 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
 25.Brennan, C., Mavhungu, S.: Visits to level \(r\) by Dyck paths. Fund. Inform. 117(1–4), 127–145 (2012)MathSciNetzbMATHGoogle Scholar
 26.Carlson, F.: Über Potenzreihen mit ganzzahligen Koeffizienten. Math. Z. 9(1–2), 1–13 (1921)MathSciNetzbMATHGoogle Scholar
 27.Chomsky, N., Schützenberger, M.P.: The algebraic theory of contextfree languages. In: Computer Programming and Formal Systems, pp. 118–161. NorthHolland, Amsterdam (1963)CrossRefGoogle Scholar
 28.Dershowitz, N.: Nonleaf patterns in trees: protected nodes and Fine numbers. Submitted to J. Integer Seq. arXiv:1908.04329 (2019)
 29.Dershowitz, N., Zaks, S.: More patterns in trees: up and down, young and old, odd and even. SIAM J. Discrete Math. 23(1), 447–465 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
 30.Deutsch, E.: Another type of lattice path. Am. Math. Mon. 107(4), 368–370 (2000). Problem 10658, with solution by D. Callan, M. Beck, W. Bohm, R.F. McCoart, and GCHQ Problems GroupGoogle Scholar
 31.Deutsch, E., Elizalde, S.: Statistics on bargraphs viewed as cornerless Motzkin paths. Discrete Appl. Math. 221, 54–66 (2017)MathSciNetzbMATHCrossRefGoogle Scholar
 32.Deutsch, E., Munarini, E., Rinaldi, S.: Skew Dyck paths, area, and superdiagonal bargraphs. J. Stat. Plan. Inference 140(6), 1550–1562 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
 33.Deutsch, E., Shapiro, L.W.: A bijection between ordered trees and 2Motzkin paths and its many consequences. Discrete Math. 256(3), 655–670 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
 34.Dieudonné, J.: Calcul infinitésimal, 2 edn. Hermann, Paris (1980). 1st edition in 1968: 479 pp, there is also an English translation of the 1st edition in 1971, 427 ppGoogle Scholar
 35.Ding, Y., Du, R.R.X.: Counting humps in Motzkin paths. Discrete Appl. Math. 160, 187–191 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
 36.Duchon, P.: On the enumeration and generation of generalized Dyck words. Discrete Math. 225(1–3), 121–135 (2000)MathSciNetzbMATHCrossRefGoogle Scholar
 37.Dziemiańczuk, M.: On directed lattice paths with vertical steps. Discrete Math. 339(3), 1116–1139 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
 38.Eu, S.P., Liu, S.C., Yeh, Y.N.: Dyck paths with peaks avoiding or restricted to a given set. Stud. Appl. Math. 111(4), 453–465 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
 39.Eynard, B.: Counting Surfaces. Progress in Mathematical Physics, vol. 70. Springer, Berlin (2016). CRM Aisenstadt chair lectureszbMATHCrossRefGoogle Scholar
 40.Fayolle, G., Iasnogorodski, R., Malyshev, V.: Random Walks in the QuarterPlane. Applications of Mathematics, vol. 40. Springer, Berlin (1999)zbMATHCrossRefGoogle Scholar
 41.Flajolet, P.: Combinatorial aspects of continued fractions. Discrete Math. 32(2), 125–161 (1980)MathSciNetzbMATHCrossRefGoogle Scholar
 42.Flajolet, P., Sedgewick, R.: Analytic Combinatorics. Cambridge University Press, Cambridge (2009)zbMATHCrossRefGoogle Scholar
 43.Georgiadis, E., Callan, D., Hou, Q.H.: Circular digraph walks, \(k\)balanced strings, lattice paths and Chebychev polynomials. Electron. J. Comb. 15(1), R108 (2008)MathSciNetzbMATHGoogle Scholar
 44.Gessel, I.M.: A factorization for formal Laurent series and lattice path enumeration. J. Comb. Theory Ser. A 28(3), 321–337 (1980)MathSciNetCrossRefGoogle Scholar
 45.Guibas, L.J., Odlyzko, A.M.: String overlaps, pattern matching, and nontransitive games. J. Comb. Theory Ser. A 30(2), 183–208 (1981)MathSciNetzbMATHCrossRefGoogle Scholar
 46.Hackl, B., Heuberger, C., Prodinger, H.: Ascents in nonnegative lattice paths. arXiv:1801.02996 (2018). (Long version of Counting ascents in generalized Dyck paths, in Proceedings of Analysis of Algorithms 2018.)
 47.Hofacker, I.L., Reidys, C.M., Stadler, P.F.: Symmetric circular matchings and RNA folding. Discrete Math. 312(1), 100–112 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
 48.Hopcroft, J.E., Motwani, R., Ullman, J.D.: Introduction to Automata Theory, Languages, and Computation, 3rd edn. AddisonWesley, Boston (2006). (First edition in 1979)zbMATHGoogle Scholar
 49.Irvine, V., Melczer, S., Ruskey, F.: Vertically constrained Motzkinlike paths inspired by bobbin lace. Electron. J. Comb. 26(2), P2.16 (2019)MathSciNetzbMATHGoogle Scholar
 50.Jin, E.Y., Qin, J., Reidys, C.M.: Combinatorics of RNA structures with pseudoknots. Bull. Math. Biol. 70(1), 45–67 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
 51.Kauers, M., Paule, P.: The Concrete Tetrahedron: Symbolic Sums, Recurrence Equations, Generating Functions, Asymptotic Estimates. Texts and Monographs in Symbolic Computation. Springer, Berlin (2011)zbMATHCrossRefGoogle Scholar
 52.Knuth, D.E.: The Art of Computer Programming. Vol 1: Fundamental Algorithms. AddisonWesley, Boston (1968)zbMATHGoogle Scholar
 53.Krattenthaler, C.: Lattice path enumeration. In: Handbook of Enumerative Combinatorics, pp. 589–678. Discrete Math. Appl. (Boca Raton), CRC PressGoogle Scholar
 54.Kreweras, G., Moszkowski, P.: A new enumerative property of the Narayana numbers. J. Stat. Plan. Inference 14(1), 63–67 (1986)MathSciNetzbMATHCrossRefGoogle Scholar
 55.Kreweras, G., Poupard, Y.: Subdivision des nombres de Narayana suivant deux paramètres supplémentaires. Eur. J. Comb. 7(2), 141–149 (1986)zbMATHCrossRefGoogle Scholar
 56.Labelle, J., Yeh, Y.N.: Generalized Dyck paths. Discrete Math. 82(1), 1–6 (1990)MathSciNetzbMATHCrossRefGoogle Scholar
 57.Manes, K., Sapounakis, A., Tasoulas, I., Tsikouras, P.: Strings of length 3 in GrandDyck paths and the Chung–Feller property. Electron. J. Comb. 19(2), P2 (2012)MathSciNetzbMATHGoogle Scholar
 58.Manes, K., Sapounakis, A., Tasoulas, I., Tsikouras, P.: Equivalence classes of ballot paths modulo strings of length 2 and 3. Discrete Math. 339(10), 2557–2572 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
 59.Mansour, T.: Statistics on Dyck paths. J. Integer Seq. 9, Article 06.1.5 (2006)Google Scholar
 60.Mansour, T., Shattuck, M.: Counting humps and peaks in generalized Motzkin paths. Discrete Appl. Math. 161(13–14), 2213–2216 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
 61.Merlini, D., Rogers, D.G., Sprugnoli, R., Verri, M.C.: Underdiagonal lattice paths with unrestricted steps. Discrete Appl. Math. 91(1–3), 197–213 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
 62.Mohanty, S.G.: Lattice Path Counting and Applications. Academic Press, Boston (1979)zbMATHGoogle Scholar
 63.Munarini, E., Salvi, N.Z.: Binary strings without zigzags. Sém. Lothar. Comb. 49, Article B49h (2002)Google Scholar
 64.Niederhausen, H., Sullivan, S.: Ballot paths avoiding depth zero patterns. J. Comb. Math. Comb. Comput. 74, 181–192 (2010)MathSciNetzbMATHGoogle Scholar
 65.Niederhausen, H., Sullivan, S.: Pattern avoiding ballot paths and finite operator calculus. J. Stat. Plan. Inference 140(8), 2312–2320 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
 66.Niederhausen, H., Sullivan, S.: Counting depth zero patterns in ballot paths. Integers 12(2), 215–236 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
 67.Pan, R., Qiu, D., Remmel, J.: Counting consecutive pattern matches in \({{\cal{S}}}_n(132)\) and \({{\cal{S}}}_n(123)\). Adv. Appl. Math. 105, 130–167 (2019)zbMATHCrossRefGoogle Scholar
 68.Pan, R., Remmel, J.B.: Paired patterns in lattices paths. In: Lattice Paths Combinatorics and Applications. Developments in Mathematics Series, vol. 58, pp. 382–418. Springer (2019)Google Scholar
 69.Park, Y., Park, S.K.: Enumeration of generalized lattice paths by string types, peaks, and ascents. Discrete Math. 339(11), 2652–2659 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
 70.Parviainen, R.: Lattice path enumeration of permutations with \(k\) occurrences of the pattern 2–13. J. Integer Seq. 9(3), Article 06.3.2 (2006)Google Scholar
 71.Qiu, D., Remmel, J.: Quadrant marked mesh patterns in 123avoiding permutations. Discrete Math. Theor. Comput. Sci. 19(2), Paper No. 12 (2018)Google Scholar
 72.Regev, A.: Identities for the number of standard Young tableaux in some \((k,\ell )\)hooks. Sém. Lothar. Comb. 63, Article B63c (2010)Google Scholar
 73.Righi, C.: Number of “\(udu\)”s of a Dyck path and \(ad\)nilpotent ideals of parabolic subalgebras of \(sl_{\ell +1}({\mathbb{C}})\). Sém. Lothar. Comb. 59, Article B59c (2008)Google Scholar
 74.Riordan, J.: Enumeration of plane trees by branches and endpoints. J. Comb. Theory Ser. A 19(2), 214–222 (1975)MathSciNetzbMATHCrossRefGoogle Scholar
 75.Sapounakis, A., Tasoulas, I., Tsikouras, P.: Counting strings in Dyck paths. Discrete Math. 307(23), 2909–2924 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
 76.Schützenberger, M.P.: On contextfree languages and pushdown automata. Inf. Control 6, 246–264 (1963)MathSciNetzbMATHCrossRefGoogle Scholar
 77.Schützenberger, M.P.: On the synchronizing properties of certain prefix codes. Inf. Control 7, 23–36 (1964)zbMATHCrossRefGoogle Scholar
 78.Stanley, R.P.: Enumerative Combinatorics. Vol. 1. Cambridge Studies in Advanced Mathematics, vol. 49, 2nd edn. Cambridge University Press, Cambridge (2011)CrossRefGoogle Scholar
 79.Stein, P.R., Waterman, M.S.: On some new sequences generalizing the Catalan and Motzkin numbers. Discrete Math. 26(3), 261–272 (1979)MathSciNetzbMATHCrossRefGoogle Scholar
 80.Sulanke, R.A.: Objects counted by the central Delannoy numbers. J. Integer Seq. 6(1), Article 03.1.5 (2003)Google Scholar
 81.Sun, Y.: The statistic number of udu’s in Dyck paths. Discrete Math. 287(1–3), 177–186 (2004)MathSciNetzbMATHCrossRefGoogle Scholar
 82.Zhuang, Y.: A generalized Goulden–Jackson cluster method and lattice path enumeration. Discrete Math. 341(2), 358–379 (2018)MathSciNetzbMATHCrossRefGoogle Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.