Humanoid Robotics: A Reference pp 1-31 | Cite as

# Virtual Constraints and Hybrid Zero Dynamics for Realizing Underactuated Bipedal Locomotion

## Abstract

Underactuation is ubiquitous in human locomotion and should be ubiquitous in bipedal robotic locomotion as well. This chapter presents a coherent theory for the design of feedback controllers that achieve stable walking gaits in underactuated bipedal robots. Two fundamental tools are introduced, virtual constraints and hybrid zero dynamics. Virtual constraints are relations on the state variables of a mechanical model that are imposed through a time-invariant feedback controller. One of their roles is to synchronize the robot’s joints to an internal gait phasing variable. A second role is to induce a low-dimensional system, the zero dynamics, that captures the underactuated aspects of a robot’s model, without any approximations. To enhance intuition, the relation between physical constraints and virtual constraints is first established. From here, the hybrid zero dynamics of an underactuated bipedal model is developed, and its fundamental role in the design of asymptotically stable walking motions is established. The chapter includes numerous references to robots on which the highlighted techniques have been implemented.

## Keywords

Feedback control Stabilization Poincaré map Invariant manifold Impacts Experiments## 1 Introduction

Models of bipedal robots are hybrid, nonlinear, and, typically, high dimensional. In addition, as it will be motivated shortly, the continuous portion of the dynamics is often underactuated. A further complication is that a steady walking cycle is a nontrivial periodic motion . This means that standard stability tools for static equilibria do not apply. Instead, one must use tools appropriate for the study of periodic orbits, such as Poincaré return maps. The overall complexity in the modeling and analysis of bipedal locomotion has in turn motivated a host of gait design methods that are built around low-dimensional approximations to the dynamics, often based on approximating the system as an inverted pendulum and also often approximating the legs as massless.

This chapter outlines an approach to gait design that applies to high-dimensional underactuated models, without making approximations to the dynamics. As with approximate design methods, lower-dimensional models do appear, but unlike approximate design methods, the lower-dimensional models are exact, meaning that solutions of the low-dimensional model are also solutions of the high-dimensional model evolving in a low-dimensional invariant surface. To get an initial sense of what this may mean, consider a floating-base model of a bipedal robot, and then consider the model with a point or link of the robot, such as a leg end or foot, constrained to maintain a constant position with respect to the ground. The given contact constraint is holonomic and constant rank, and thus using Lagrange multipliers (from the principle of virtual work), a reduced-order model compatible with the (holonomic) contact constraint is easily computed. When computing the reduced-order model, no approximations are involved, and solutions of the reduced-order model are solutions of the original floating-base model, with inputs (ground reaction forces and moments) determined by the Lagrange multiplier.

Virtual constraints are relations (i.e., constraints) on the state variables of the robot’s model that are achieved through the action of actuators and feedback control instead of physical contact forces. They are called *virtual* because they can be reprogrammed on the fly without modifying any physical connections among the links of the robot or its environment. Virtual constraints can be used to synchronize the evolution of a robot’s links to create stable periodic motion. Like physical constraints, under certain regularity conditions, they induce a low-dimensional invariant model, called the *zero dynamics*, due to the highly influential paper [8]. Each virtual constraint imposes a relation between joint variables, and by differentiation a relation between joint velocities. As a consequence, for the cases studied in this chapter, the dimension of the zero dynamics is the initial number of states in the robot’s model minus twice the number of virtual constraints (which can be at most the number of independent actuators). The main novelty required for the study of this reduced-dimensional constrained system in bipedal locomotion arises from the *hybrid* or multi-phase nature of locomotion models, such as alternating single support phases and impacts. The hybrid nature of the models gives rise to the *hybrid zero dynamics* (HZD).

Virtual constraints and hybrid zero dynamics originated in the study of underactuated, planar bipedal locomotion in [20, 49]; a synthesis of these methods can be found in [37, 48], with many experiments and extensions reported in [3, 10, 11, 28, 31, 42, 43, 47, 50, 51] . The methods are currently being developed for and applied to underactuated 3D robots; see [1, 2, 12, 21, 35, 44, 46] with experiments just beginning to be reported [7, 19]. Virtual constraints and hybrid zero dynamics are also being used in the control of lower-limb prostheses [15, 16, 24].

## 2 Why Study Underactuation?

An important source of complexity in a bipedal robot is the degree of actuation of the model with respect to the number of degrees of freedom. When there are fewer independent actuators than degrees of freedom, the model is *underactuated*. It is a common theme in robotics that underactuated models are more challenging to control than fully actuated models.

The primary motivation of this chapter is the *effective underactuation* that arises in a humanoid robot. Due to the finite size of its feet and the unilateral nature of the contact forces between the foot and ground, most humanoids can all too easily roll forward on the foot, creating an axis of rotation without actuation. Because feet are typically narrower than they are long, it is even easier to roll laterally on the foot. Keeping the foot flat on the ground is difficult and imposes severe restrictions on ankle torque, which is what is meant by “effective underactuation.” This chapter studies the case when the size of the feet is taken to a limiting value of zero, namely, point feet, resulting in a truly underactuated model. Control designs that allow a robot with point feet to achieve asymptotically stable walking gaits are developed.

Focusing on underactuation is important for at least two reasons. On one hand, it is interesting to prove, both theoretically and experimentally, that elegant walking and running motions are possible with a mechanically simple robot (no feet). On the other hand, if human walking is taken as the de facto standard against which mechanical bipedal walking is to be compared, then the flat-footed walking achieved by current robots needs to be improved. In particular, toe roll toward the end of the single support phase needs to be allowed as part of the gait design. Currently, this is specifically avoided because, as mentioned above, it leads to underactuation, namely, *rotation of the foot about the toe introduces an axis of rotation with no actuation, as does lateral rotation of the foot*. A nominal gait design method that produces feasible stable motions without *requiring the use of ankle torque* can always produce stable motions when powered ankles are available, even in cases where the torque that can be produced may be very limited. Underactuation and “effective underactuation” (severe bounds on ankle torque) are extremely challenging for a control design philosophy based on trajectory tracking and a quasi-static stability criterion, such as the zero moment point (ZMP) [45], as is currently practiced widely in the bipedal robotic community. Moreover, the results developed in this chapter for point feet can easily be extended to the case of finite size foot [11, 46]. Actuation at the foot can be used to improve the convergence toward a periodic motion or to achieve a humanlike evolution of the ZMP.

There is considerable freedom when choosing an underactuated model. One passive joint can be considered in the sagittal plane (at ankle or toe) for studying planar [10] or 3D biped [44, 46] gaits. Two passive joints, one in the sagittal plane and another in the frontal plane, can be introduced for 3D bipeds [12] when yaw rotation is assumed to be avoided by friction. Three passive joints are introduced to model point feet in [39]. In the present chapter, all of these sources of underactuation are handled.

## 3 Hybrid Model of a Bipedal Walker

This section introduces a hybrid dynamic model for walking motions of a bipedal robot. A hybrid model is required because walking consists of alternating phases of one foot on the ground and two feet on the ground.

*N*≥ 2 rigid links with mass connected via rigid, frictionless revolute joints. The contact with the ground is modeled as a passive point. A typical robot is depicted in Fig. 1a, which is intentionally suggestive of a human form. All motions are assumed to consist of successive phases of

*single support*(stance leg on the ground and swing leg in the air) and

*double support*(both legs on the ground). The double support phase is assumed to be instantaneous. Conditions that guarantee the leg ends alternate in ground contact without slipping – while other links such as the torso or arms remain free – must be imposed during control design. A rigid impact is used to model the contact of the swing leg with the ground.

The distinct phases of walking naturally lead to mathematical models that are comprised of two parts: the differential equations describing the dynamics during the swing phase and a model that describes the dynamics when a leg end impacts the ground. For simplicity, in the models developed here, the ground is assumed to be flat [18, 34].

### 3.1 Lagrangian Swing Phase Model

The swing phase model corresponds to a pinned open kinematic chain. For simplicity, it is assumed that only symmetric gaits are of interest, and hence it does not matter which leg end is pinned. The swapping of the roles of the legs can be accounted for in the impact model.

*N*-dimensional configuration manifold of the robot when the stance leg end is acting as a pivot and let \(q:=(q_1; \cdots ; q_N) \in {\mathcal {Q}}\) be a set of generalized coordinates. Denote the potential and kinetic energies by

*V*(

*q*) and \(K(q,\dot q)=\frac {1}{2}\dot {q}^T D(q) \dot q\), respectively, where the inertia matrix

*D*is positive definite on \({\mathcal {Q}}\). The dynamic model is easily obtained with the method of Lagrange , yielding the mechanical model

*B*, the torque distribution matrix, is assumed to be constant with rank 1 ≤

*k*<

*N*. Recall that the foot is modeled as an unactuated point contact, with one, two, or there degrees of freedom.

*g*(

*x*) is a 2

*N*×

*k*matrix. In natural coordinates \((q;\dot {q})\) for \(T{\mathcal {Q}}\),

*g*is independent of \(\dot {q}\).

### 3.2 Impact Model

*relabeling*of the generalized coordinates is given by a matrix,

*R*, acting on

*q*with the property that

*RR*=

*I*, i.e.,

*R*is a circular matrix. The result of the impact and the relabeling of the states provides an expression:

### 3.3 Overall Hybrid Model

A *step* of the robot is a solution of (6) that starts with the robot in double support, ends in double support with the configurations of the legs swapped, and contains only one impact event. *Walking* is a sequence of steps.

## 4 Physical Constraints and Reduced-Order Models

*regular holonomic constraint*is a twice continuously differentiable function \(h:{\mathcal {Q}} \to \mathbb {R}^k\) such that

*N*−

*k*) DOF. To compute it, the principle of virtual work says to augment (8) with a Lagrange multiplier

*λ*is obtained by setting the second derivative of the constraint along solutions of the model to zero, that is,

*automatically invertible*from the regularity of

*h*and the positive definiteness of

*D*(

*q*).

*q*

_{c},

*q*

_{f}), with the constrained coordinates \(q_{\mathrm {c}} \in {\mathcal {Q}}_{\mathrm {c}} \subset \mathbb {R}^k\), the free coordinates \(q_{\mathrm {f}} \in {\mathcal {Q}}_{\mathrm {f}} \subset \mathbb {R}^{N-k}\), and such that a diffeomorphism \(F:{\mathcal {Q}}_{\mathrm {c}} \times {\mathcal {Q}}_{\mathrm {f}} \to {\mathcal {Q}}\) exists. In this case, the constraint is always regular, and the mapping \(F_c: {\mathcal {Q}}_{\mathrm {f}} \to {\mathcal {Q}}\) given by

*F*

_{ c }.

## 5 Virtual Constraints and Zero Dynamics

Physical constraints in a mechanism guide the motion along a constraint surface, but do not impose a specific evolution with respect to time. This property seems particularly well adapted to locomotion. First of all, any attempt to describe walking, even something as simple as the difference between humanlike walking (knees bent forward) and bird-like walking (knees bent backward), inevitably leads to a description of the *posture* or *shape* of the robot throughout a step. In other words, a description of walking involves at least a partial specification of the path followed in the configuration space of the biped.

Secondly, in a controller based upon tracking of a time trajectory, if a disturbance were to affect the robot and causes its motion to be retarded with respect to the planned motion, the feedback system is then obliged to play catch up in order to regain synchrony with the reference trajectory. Presumably, what is more important is the orbit of the robot’s motion, that is, the path in state space traced out by the robot, and not the slavish notion of time imposed by a reference trajectory (think about how you respond to a heavy gust of wind when walking). A preferable situation, therefore, would be for the control system in response to a disturbance to drive the motion back to the periodic orbit, but not to attempt otherwise re-synchronizing the motion with respect to time.

With these motivations in mind, a controller based on virtual constraints is now presented. This section parallels the developments in the previous section but this time for a model with actuation. In particular, the action of contact forces and moments is replaced by actuator torques or forces. The imposed constraints will be virtual because they exist as lines of code in an embedded controller and can be modified on the fly without any physical changes to the robot. While the constraints imposed are virtual, in the authors’ opinion, they are as natural as physical constraints.

### 5.1 Virtual Constraints

*k*independent actuators,

*k*virtual constraints can be generated by the actuators. The virtual constraints are expressed as outputs applied to the model (3), and a feedback controller must be designed that drives asymptotically to zero the output function. In order to emphasize the parallels between virtual and physical constraints, the output is written as

*q*

_{c},

*q*

_{f}) forming a set of generalized configuration variables for the robot (i.e., such that a diffeomorphism \(F:{\mathcal {Q}}_{\mathrm {c}} \times {\mathcal {Q}}_{\mathrm {f}} \to {\mathcal {Q}}\) exists). The interpretation is that

*q*

_{c}represents a collection of variables that one wishes to “control” or “regulate,” while

*q*

_{f}is a complementary set of variables that remain “free.” Later, a special case of

*h*

_{ d }(

*q*

_{f}) will be introduced that highlights a

*gait phasing variable*, which makes it easier to interpret the virtual constraint in many instances.

*zero dynamics manifold*:

*N*−

*k*).

*u*

^{∗}required to remain on the zero dynamics manifold is computed by substituting the controlled model (1) into (12) and solving for the input yielding \(\ddot {y}=0\). This gives

*u*

^{∗}to be well defined, the decoupling matrix

*the invertibility of the decoupling matrix is not automatic*and must be checked. Numerous examples exist, nevertheless, that show it is straightforward to meet this condition. Later in the chapter, it is shown how to compute the decoupling matrix without inverting the inertia matrix.

### Remark 1

In the control literature, the output (23) is said to have vector relative degree two. More general constraints can be handled.

*u*

^{∗}can be modified to an input-output linearizing controller [48, Chap. 5]:

Like physical constraints, under certain regularity conditions, virtual constraints induce a low-dimensional invariant model of the swing phase dynamics. The low-dimensional model is called the *zero dynamics* in the nonlinear control literature [8]. The main novelty required in biped locomotion arises from the hybrid nature of the models, which gives rise to the *hybrid zero dynamics* [49].

The zero dynamics of the hybrid model (6) with output (23) are developed in a two-step process. First, the zero dynamics of the (non-hybrid) nonlinear model consisting of the swing phase dynamics (3) and the output (23) are characterized, and then, second, an impact invariance condition is imposed on the swing phase zero dynamics manifold to obtain the hybrid zero dynamics.

### 5.2 The Swing Phase Zero Dynamics

The objective is to characterize the swing phase model (1) restricted to the constraint surface (24). The zero dynamics, by definition, reflects the internal dynamics when the output is identically zero, meaning the system evolves on the constraint surface. The development is analogous to the case of physical constraints, once the invertibility of a key matrix is established.

### Proposition 1

*Let B*

^{⊥}

*be a*(

*N*−

*k*) ×

*N matrix of rank N*−

*k such that B*

^{⊥}

*B*= 0

*, and suppose that the decoupling matrix*(26)

*is invertible at a point q. Then the following matrices are each N*×

*N and have rank N:*

The proof is sketched. Multiplying the leftmost matrix by the inverse of *D*(*q*) gives the matrix in the middle. Multiplying the matrix in the middle by the full rank matrix \([B~|\left (B^\perp \right )^T]\) gives the rightmost matrix. This matrix is full rank because it is block upper triangular with the upper left block being the decoupling matrix, which has full rank by assumption, and the lower right block has full rank by standard properties of matrices. This completes the proof.

*J*

_{ c }(

*q*

_{f}) has rank

*N*−

*k*give the following result.

### Corollary 1

*Let B*

^{⊥}

*be a*(

*N*−

*k*) ×

*N matrix of rank N*−

*k such that B*

^{⊥}

*B*= 0

*, and suppose that the decoupling matrix*(26)

*is invertible at a point q*=

*F*

_{ c }(

*q*

_{f})

*. Then the matrix*

*is invertible.*

*B*

^{⊥}as in [40, 41] gives

*B*

^{⊥}

*Bu*= 0. The term \(B^\perp \frac {\partial L}{\partial \dot {q} }\) is a form of

*generalized angular momentum*. Restricting to the zero dynamics manifold gives

### 5.3 The Hybrid Zero Dynamics

*invariant under the impact map*, that is

### Definition 1 (Hybrid zero dynamics [49])

*hybrid zero dynamics*.

*k*=

*N*− 1), however, it is known that if a single nontrivial solution of the zero dynamics satisfies these conditions, then all solutions of the zero dynamics will satisfy them [48, Thm. 5.2]. In the case of systems with more than one degree of underactuation, systematic methods have been developed which modify the virtual constraints “at the boundary” and allow the conditions to be met [30]. Very straightforward implementations of the result are presented for 3D robots in [12] and [18].

### Remark 2

When low-dimensional pendulum models are used for gait design, they *approximate* the swing phase dynamics and the impact map is ignored. The zero dynamics is an *exact* low-dimensional model that captures the underactuated nature of the robot. Presumably, one could use it without insisting on the impact invariance condition (36).

### 5.4 Calculating the Feedback Controller

*u*

^{∗}on the zero dynamics manifold can be given. Let

*B*

^{+}be the pseudoinverse of the full rank matrix

*B*. Then

A similar expression can be developed to calculate the torque ensuring convergence toward the zero dynamics surface, once again without explicitly computing the decoupling matrix. The important advantage over (27) is that the matrix to invert has dimension equal to the unactuated degrees of freedom in the model, which is typically much smaller than *N* or *k*.

*y*and \(\dot {y}\) are computed in (44) and (45).

*N*−

*k*) × (

*N*−

*k*) matrix, where the size corresponds to the number of unactuated coordinates. Corollary 1 guarantees that the matrix is invertible near the zero dynamics manifold. A uniqueness result in [8] implies that

*u*in (54) when restricted to \(\mathcal {Z}\) is equal to

*u*

^{∗}in (25) and (42).

### Remark 3

Commonly, *q* _{c} and *q* _{f} are a linear function of *q*, which then greatly simplifies many of the above equations.

## 6 Stability Analysis with Hybrid Zero Dynamics

This section first presents the relation between the stability of a periodic solution in the full-order model and in the hybrid zero dynamics. The stability of periodic orbits within the hybrid zero dynamics is subsequently analyzed.

### 6.1 Closed-Loop Stability with Zero Dynamics

*𝜖*> 0, and apply it to the model (6). Let

*P*

^{ 𝜖 }:

*S*→

*S*be the Poincaré (return) map (It is in general a partial map and depends on the tuning parameter through the feedback (56). It is defined by starting with a point in \(\mathcal {S}\), applying the impact map, using this as an initial condition of the closed-loop ODE, and following the flow until it first crosses \(\mathcal {S}\) at time

*T*

_{ I }; see [48, Chap. 4] for a careful definition.) of the closed-loop system, defined in the usual way. For a given

*𝜖*> 0,

*x*

^{∗}∈

*S*is a fixed point if

*P*

^{ 𝜖 }(

*x*

^{∗}) =

*x*

^{∗}, which is well-known to be equivalent to the existence of a periodic solution of the closed-loop system.

*compatible with the zero dynamics*if

*Δ*(

*x*

^{∗}) ∈

*Z*, then the periodic solution is independent of the tuning parameter

*𝜖*. Based on [5], the feedback is said to

*drive the virtual constraints rapidly exponentially to zero*if there exist constants

*β*> 0 and

*γ*> 0 such that, for each

*𝜖*> 0, there exists

*δ*> 0, such for all

*x*

_{0}∈

*B*

_{ δ }(

*Δ*(

*x*

^{∗})):

### Theorem 1 ( [5, 30])

**Determining Closed-loop Stability from the HZD***Suppose that the virtual constraints*(23)

*satisfy:*

- (a)
*the decoupling matrix is invertible,* - (b)
*the associated zero dynamics manifold*\(\mathcal {Z}\)*is hybrid invariant, and* - (c)
*there exists a point x*^{∗}∈*S*∩*Z giving rise to a periodic orbit.*

*Assume moreover that the feedback*(56)

*:*

- (d)
*is compatible with the zero dynamics in the sense of*(57)*and* - (e)
*drives the virtual constraints rapidly exponentially to zero.*

*Then there exists*\(\bar {\epsilon } > 0\)

*such that for*\(0 < \epsilon < \bar {\epsilon }\)

*, the following are equivalent:*

- (i)
*the periodic orbit is locally exponentially stable;* - (ii)
*x*^{∗}*is an exponentially stable fixed point of P*^{ 𝜖 };*and* - (iii)
*x*^{∗}*is an exponentially stable fixed point of ρ,*

*where the restricted Poincaré map*

*is the Poincaré map of the hybrid zero dynamics, is independent of the feedback, and hence is also independent of 𝜖.* \(\Box \)

### Remark 4

It is reemphasized that periodic orbits of the hybrid zero dynamics are periodic orbits of the full-dimensional model. Two feedback controllers are provided in [29, 49] for exponentially stabilizing these orbits in the full-dimensional model, (3), and a third family of feedback controllers is presented in [5].

### 6.2 Special Case of One Degree of Underactuation

When there is only one degree of underactuation , the restricted Poincaré map \(\rho := P^{\epsilon }|{ }_{\mathcal {S} \cap \mathcal {Z}}\) is one-dimensional and can be computed in closed form. While one degree of underactuation is primarily a “planar” (2D) robot phenomenon, it has also been used in 3D robots [44] with both single and (nontrivial) double support phases.

*θ*be a configuration variable tied to the world frame, such as the absolute angle of the line connecting the stance hip to the end of the stance leg, and let

*σ*be the generalized angular momentum conjugate to

*θ*. Then in the coordinates (

*θ*,

*σ*), (35) becomes

*κ*

_{2}is now

*independent of angular momentum*. Assume the virtual constraints have been selected such that the zero dynamics admit a periodic orbit, as in Theorem 1. It can be shown that \( \dot \theta \) and

*σ*will not change signs. Assume furthermore there exists a single point

*θ*

^{−}such that the swing foot height decreases to zero, with a strictly negative impact velocity in the vertical direction. In this case, [48, Chap. 6] shows that the impact surface in the zero dynamics can be written:

*d*is the distance between the feet (measured along the

*x*-axis) and \(\dot {z}^{-}\) the vertical velocity of the center of mass just before impact. In general, \(\dot {z}\) is linear with respect to joint velocities, and on \(\mathcal {Z}\), it is therefore proportional to \(\dot {\theta }\). Hence, using (60a) and the chain rule

*δ*

_{zero}< 1 when \(\dot z^-<0\), that is, when the vertical velocity of the center of mass is directed downward at the end of the step.

*θ*

^{+}≤

*ξ*

_{1}≤

*θ*

^{−}, define

*Note that the energy*\({\mathcal {E}}_{\mathrm {zero}}\)

*is not the total energy of the initial system evaluated on the zero dynamics.*Indeed, the total energy of the original system is not constant along solutions because actuator power is being injected to create the virtual constraints. One might call \({\mathcal {E}}_{\mathrm {zero}} \) pseudo-energy; its relation to stability is illustrated in Fig. 3.

### Remark 5

In the case of a single degree of underactuation, (62), (63), and (64) imply that the *stability of the periodic orbit in the zero dynamics is determined by the direction of the vertical velocity of the center of mass at the end of the step and is* essentially independent *of the particular choice of virtual constraints used to realize the orbit*. Stated a bit more precisely, different choices of *q* _{c}, *q* _{f}, and *h* _{ d } in (23), each with an invertible decoupling matrix and each defining a virtual constraint that is identically zero along the same periodic orbit, will result in a hybrid zero dynamics with the same stability properties. Other aspects of the transient behavior, such as the basin of attraction or how much control effort is required to return to the periodic orbit, may be different.

### Remark 6

The motion corresponding to the 3D linear inverted pendulum (3D-LIP), widely used in control of humanoid robots, constrains the center of mass such that *z*(*t*) is constant. From (63), a similar motion in the zero dynamics would result in *δ* _{zero} = 1 and, consequently, would not be exponentially stable.

### 6.3 Higher Degrees of Underactuation

When two or more degrees of underactuation are considered, the dimension of the zero dynamics is then greater than or equal to four, and the restricted Poincaré map (59) can no longer be computed in closed form. While its numerical calculation remains a valid means to analyze the stability of a periodic solution, it is more challenging to extract information for synthesis of a gait. In addition, when there are two or more degrees of underactuation, it has been shown that stability of a periodic motion within the zero dynamics does indeed depend on the choice of the virtual constraints, in contrast to the case of one degree of underactuation analyzed above. Reference [12] gives two choices of virtual constraints that result in the *same periodic solution of the robot*, and yet the periodic orbit is asymptotically stable for one of the choices and unstable for the other; see [7] for additional examples. In each of the cited examples, stability was recovered by judiciously modifying a virtual constraint in the “frontal plane.”

In the case of one degree of underactuation, the freedom coming from the fact that the joint path is controlled but not the time evolution along the path compensates the underactuation: any path can be followed, and the corresponding time evolution along it is unique [9], though not freely assignable. Different virtual constraints restraining the robot’s configuration to the same path result in the same dynamical behavior. With higher degrees of underactuation, the virtual constraint does not define a unique path in the configuration space, but instead a surface of higher dimension. A periodic trajectory induces a periodic orbit in the state space that belongs to a continuum of virtual constraints defining different surfaces each of which includes this path. As a consequence, different dynamic behaviors will be obtained.

In a certain sense, the fact that stability depends on the choice of virtual constraints is the more natural situation. The surprise was really that for one degree of underactuation, the periodic orbit itself determines closed-loop stability when designing controllers on the basis of virtual constraints. The next section discusses work in [2] that shows how to systematically search through a family of virtual constraints to find stabilizing solutions, with no restrictions of the degree of underactuation. In parallel to this work, the selection of virtual constraints and switching conditions in order to obtain “self-synchronization” of frontal and sagittal plane motions of an underactuated 3D biped is explored in [36], along with regulation of the “pseudo-energy” (66).

## 7 Design of the Virtual Constraints

This section addresses how to make a concrete choice of the virtual constraints so as to achieve a periodic walking gait that satisfies important physical conditions.

### 7.1 Starting with an Existing Periodic Trajectory

*t*<

*T*is a periodic solution of (3) that has already been computed and verified to meet required conditions on ground reaction forces, actuators bounds, etc. Let

*θ*is called a

*gait timing variable or a gait phasing variable*. Without loss of generality, assume that \(\dot {\theta }(t) >0\), and thus

*θ*is strictly increasing along the periodic solution. Let (

*q*

_{c},

*q*

_{f}) be a choice of regulated and free variables such that (

*q*

_{c},

*θ*) are independent functions in the sense that their Jacobian has full rank. Then [48, Thm. 6.2, pp. 163] gives an explicit construction of a virtual constraint of the form

### 7.2 Design via Parameter Optimization

*h*

_{ d }is constructed from Bézier polynomials, which in turn introduces free parameters

*α*into the hybrid zero dynamics (37):

*J*(

*α*) subject to various equality and inequality constraints to prescribe walking at a desired average speed; with the unilateral forces on the support leg lying in the allowed friction cone, bounds on actuator torques are respected, a minimum swing foot clearance is achieved, and the solution is periodic [12, 35, 48].

*α*

^{∗}, and a periodic orbit of the zero dynamics. In addition, through (27), a controller is produced for the full-dimensional model. Figure 4 shows a typical stable limit cycle of the closed-loop hybrid system.

References [13, 18] show how to include common gait perturbations, such as terrain variations, into the optimization. The cost function (77) on the periodic orbit is augmented with terms that account for additional solutions of the model responding to perturbations. In this way, a controller is designed that not only creates a periodic solution on level ground, for example, but also responds appropriately to terrain height changes, slopes, and other perturbations.

### 7.3 Systematic Search of Virtual Constraints that Guarantee Stability

In the case of one degree of underactuation , one can impose the stability constraint, \(0 < \delta _{\mathrm {zero}}^2 < 1\), on the optimization problem (77), though experience shows that it is rarely necessary to do so because the pendulum-like evolution of a biped’s center of mass usually results in its velocity pointing downward at the end of the step. On the other hand, when there are two or more degrees of underactuation, the periodic solution resulting from the optimization problem may not be stabilized by the particular set of virtual constraints used in the optimization problem, though the same periodic solution may be stabilized by some other choice of virtual constraints [12]. The method of [2, 6] for systematically searching for stabilizing virtual constraints is briefly outlined.

*t*<

*T*is a periodic solution of (3), and that a choice of gait phasing variable (73) has been made. Let \(q_{\mathrm {c}}^\ast (\theta )\) and \(q_{\mathrm {f}}^\ast (\theta )\) be the values of the regulated and free variables, respectively, along the periodic orbit, and let

*H*(

*ξ*) be a

*k*× (

*N*−

*k*) matrix, depending smoothly on a vector of parameters \(\xi \in \mathbb {R}^p\). The parameterized virtual constraint

*ξ*. The matrix

*H*(

*ξ*) forms linear combinations of the free variables, such as roll or yaw, for example. When

*H*(

*ξ*

^{∗}) = 0, (78) reduces to the nominal virtual constraint resulting from the optimization problem (77), for example. More generally,

*h*(

*q*,

*ξ*) could be any smooth function that vanishes on the periodic orbit for all allowed values of the parameters: the linear combinations suggested in (78) are just one straightforward way to build a family of such functions [12].

*ξ*, say

*ξ*

^{∗}, the resulting Poincaré map will have a fixed point that is independent of the parameters, that is

*ξ*

^{∗}as

*ξ*near

*ξ*

^{∗}resulting in the sum of matrices on the right having eigenvalues in the unit circle. Reference [6] shows how the same analysis can be performed on the restricted Poincaré map, which is perhaps philosophically more satisfying as it is based on the low-dimensional pendulum-like dynamics of the underactuated portion of the model.

### 7.4 Event-Based Control

*β*into the virtual constraints:

*β*do modify the periodic orbit, as opposed to the approach of the previous subsection. The Poincaré map results in a discrete-time control system:

*x*

^{∗}=

*P*(

*x*

^{∗},

*β*

^{∗}) is a fixed point, then an

*event-based control action*of the form

A potential limiting factor in using event-based control is that the updates required for stability are only made when the solution crosses a Poincaré section, though this can be partly mitigated by using a Poincaré section at mid stance, for example, instead of (7). The event-based nature of the updates can induce delays in dealing with perturbations. Because velocity estimates are often “noisy,” smoothing must be considered when sampling the state vector, which can also induce phase lag.

## 8 Implementations on Real Robots

To date, virtual constraints and hybrid zero dynamics have been implemented on at least nine bipedal robots and three lower-limb prostheses. Figure 4 shows an example limit cycle. Videos of experiments for the robots Rabbit, MABEL, and MARLO are available at [19]. Videos of experiments for the robots AMBER-1, AMBER-2, AMBER-3, NAO, and DURUS are available at [4]. Videos of experiments for the robot ERNIE are available at [38]. Videos of experiments for the lower-limb prostheses AMPRO and the Vanderbilt Leg are available at [4] and [14], respectively. Publications associated with the above robots and prostheses are straightforward to find; they discuss practical aspects of implementing control laws based on virtual constraints.

## 9 Further Results and Open Questions

Many results have not been adequately covered in this overview. The hybrid models of bipedal robots typically have multiple continuous domains [21]. The use of virtual constraints and hybrid zero dynamics in this context can be found in [26, 43, 48] and references therein. Emphasis on nontrivial double support phases is found in [23]. Fully actuated robots are treated in [26, 46, 48]. Humanlike motion with foot rolling and double support phases has recently been demonstrated on DURUS [4]. The shape of the foot and its deformation can be very important for the walking properties as shown in passive walking. The rolling of a convex foot induces a model of the contact between the ground and the foot that is different from a point-foot contact, but it still involves underactuation [28].

The analysis procedures and control designs presented in the chapter have focused on periodic locomotion, a form of steady-state behavior in a hybrid model of walking. It is important to move beyond this assumption and address aperiodic or transient motions. There is room for improved notions of stability of aperiodic walking gaits. Two cases where aperiodic gaits arise naturally are walking on uneven ground and maneuvering a biped around obstacles. Composition of motion primitives as a means to handle aperiodic (uneven) terrain is featured in [31, 34, 50]. The introduction of elasticity in legged robots can improve robustness and adaptation to uneven terrain. Series elastic actuators are treated in [32, 33, 35, 42, 43].

How to choose the virtual constraints is an important question. In this chapter, parametric functions (or splines) were suggested, with the unknown parameters selected to optimize a given criterion. The choice of what to control has been partially addressed in [22]. The optimization criterion can be selected to account for uncertainty in the hybrid model. Terrain variation is addressed in [18]. Nonholonomic virtual constraints have just been introduced in [17]; they allow swing foot placement to be planned as a function of velocity. A systematic means to get started with virtual constraints is provided in Appendix A of [48].

Characterizing the domain of attraction of a periodic walking gait for a bipedal robot model is still in its infancy. Analysis via SOS (sums of squares) is investigated in [27].

Many other interesting questions arise, ranging from reflex actions to enhance stability under large perturbations, to bipedal robot safety when operating around humans, manipulation of objects, navigation, etc.

## Notes

### Acknowledgements

The work of J.W. Grizzle has been generously supported by NSF grants EECS-1525006, ECCS-1343720, and CNS-1239037. The work of C. Chevallereau is supported by ANR Equipex Robotex project.

## References

- 1.K. Akbari Hamed, J.W Grizzle, Event-based stabilization of periodic orbits for underactuated 3-d bipedal robots with left-right symmetry. IEEE Trans. Robot.
**30**(2), 365–381 (2014)Google Scholar - 2.K. Akbari Hamed, B.G. Buss, J.W. Grizzle, Continuous-time controllers for stabilizing periodic orbits of hybrid systems: application to an underactuated 3D bipedal robot, in
*2014 IEEE 53rd Annual Conference on Decision and Control (CDC)*, Dec 2014, pp. 1507–1513Google Scholar - 3.A.D. Ames, Human-inspired control of bipedal walking robots. IEEE Trans. Autom. Control,
**59**(5), 1115–1130 (2014)MathSciNetCrossRefzbMATHGoogle Scholar - 4.A.D. Ames, AMBER-Lab (2016). https://www.youtube.com/user/ProfAmes Google Scholar
- 5.A.D. Ames, K. Galloway, K. Sreenath, J.W. Grizzle, Rapidly exponentially stabilizing control lyapunov functions and hybrid zero dynamics. IEEE Trans. Autom. Control
**59**(4), 876–891 (2014)MathSciNetCrossRefzbMATHGoogle Scholar - 6.B.G. Buss,
*Systematic Controller Design for Dynamic 3D Bipedal Robot Walking*. PhD thesis, University of Michigan, Ann Arbor, May 2015Google Scholar - 7.B.G. Buss, A. Ramezani, K. Akbari Hamed, B.A. Griffin, K.S. Galloway, J.W. Grizzle, Preliminary walking experiments with underactuated 3D bipedal robot MARLO, in
*2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014)*, Sept 2014, pp. 2529–2536Google Scholar - 8.C. Byrnes, A. Isidori, Asymptotic stabilization of nonlinear minimum phase systems. IEEE Trans. Autom. Control
**376**, 1122–37 (1991)CrossRefzbMATHGoogle Scholar - 9.C. Chevallereau. Time scaling control for an underactuated biped robot. IEEE Trans. Robot. Autom.
**19**(2), 362–368 (2003)CrossRefGoogle Scholar - 10.C. Chevallereau, G. Abba, Y. Aoustin, F. Plestan, E.R. Westervelt, C. Canudas, J.W. Grizzle, RABBIT: a testbed for advanced control theory. IEEE Control Syst. Mag.
**23**(5), 57–79 (2003)CrossRefGoogle Scholar - 11.C. Chevallereau, D. Djoudi, and J.W. Grizzle, Stable bipedal walking with foot rotation through direct regulation of the Zero Moment Point. TRO
**25**(2), 390–401 (2008)Google Scholar - 12.C. Chevallereau, J.W. Grizzle, C.-L. Shih, Asymptotically stable walking of a five-link underactuated 3D bipedal robot. IEEE Trans. Robot.
**25**(1), 37–50 (2009)CrossRefGoogle Scholar - 13.H. Dai, R. Tedrake, Optimizing robust limit cycles for legged locomotion on unknown terrain, in
*2012 IEEE 51st Annual Conference on Decision and Control (CDC)*, 2012, pp. 1207–1213Google Scholar - 14.R.D. Gregg, High-performance control of a powered transfemoral prosthesis with amputee subjects (2016). https://www.youtube.com/watch?v=sl1IXs0j4Ww Google Scholar
- 15.R.D. Gregg, T. Lenzi, L.J. Hargrove, J.W. Sensinger, Virtual constraint control of a powered prosthetic leg: from simulation to experiments with transfemoral amputees. IEEE Trans. Robot.
**30**, 1455–1471 (2014)CrossRefGoogle Scholar - 16.R.D. Gregg, E.J. Rouse, L.J. Hargrove, J.W. Sensinger, Evidence for a time-invariant phase variable in human ankle control. PLoS One
**9**(2), e89163 (2014)Google Scholar - 17.B. Griffin, J.W. Grizzle, Nonholonomic virtual constraints for dynamic walking, in
*Preprint Submitted to IEEE Conference on Decision and Control*, 2015Google Scholar - 18.B. Griffin, J.W. Grizzle, Walking gait optimization for accommodation of unknown terrain height variations, in
*American Control Conference*, 2015Google Scholar - 19.J.W. Grizzle, Dynamic Leg Locomotion (2016) www.youtube.com/user/DynamicLegLocomotion Google Scholar
- 20.J.W. Grizzle, G. Abba, F. Plestan, Asymptotically stable walking for biped robots: analysis via systems with impulse effects. IEEE Trans. Autom. Control
**46**, 51–64 (2001)MathSciNetCrossRefzbMATHGoogle Scholar - 21.J.W. Grizzle, C. Chevallereau, A. Ames, R. Sinnet, Models, feedback control, and open problems of 3D bipedal robotic walking. Automatica
**50**(8), 1955–1988 (2014)MathSciNetCrossRefzbMATHGoogle Scholar - 22.K.A. Hamed, B.G. Buss, J.W. Grizzle, Exponentially stabilizing continuous-time controllers for periodic orbits of hybrid systems: application to bipedal locomotion with ground height variations. Int. J. Robot. Res.
**35**(8), 977–999 (2016)CrossRefGoogle Scholar - 23.A. Hereid, S. Kolathaya, M.S. Jones, J. Van Why, J.W. Hurst, A.D. Ames, Dynamic multi-domain bipedal walking with ATRIAS through SLIP based human-inspired control, in
*Proceedings of the 17th International Conference on Hybrid Systems: Computation and Control*(ACM, 2014), pp. 263–272Google Scholar - 24.J. Horn, J. Reher, Zhao, V. Paredes, A.D. Ames, AMPRO: translating robotic locomotion to a powered transfemoral prosthesis, in
*International Conference on Robotics and Automation, ICRA*, May 2015Google Scholar - 25.Y. Hürmüzlü, D.B. Marghitu, Rigid body collisions of planar kinematic chains with multiple contact points. Int. J. Robot. Res.
**13**(1), 82–92 (1994)CrossRefGoogle Scholar - 26.J. Lack, M.J. Powell, A.D. Ames, Planar multi-contact bipedal walking using hybrid zero dynamics, in
*2014 IEEE International Conference on Robotics and Automation (ICRA)*(IEEE, 2014), pp. 2582–2588Google Scholar - 27.I.R Manchester, Transverse dynamics and regions of stability for nonlinear hybrid limit cycles. IFAC Proc.
**44**(1), 6285–6290 (2011)Google Scholar - 28.A.E. Martin, D.C. Post, J.P. Schmiedeler, The effects of foot geometric properties on the gait of planar bipeds walking under HZD-based control. Int. J. Robot. Res.
**33**(12), 1530–1543 (2014)CrossRefGoogle Scholar - 29.B. Morris, J. W. Grizzle, A restricted Poincaré map for determining exponentially stable periodic orbits in systems with impulse effects: application to bipedal robots, in
*Proceedings of the 2005 IEEE International Conference on Decision and Control European Control Conference, Seville*, 2005, pp. 4199–206Google Scholar - 30.B. Morris, J. W. Grizzle, Hybrid invariant manifolds in systems with impulse effects with application to periodic locomotion in bipedal robots. IEEE Trans. Autom. Control
**54**(8), 1751–1764 (2009)MathSciNetCrossRefzbMATHGoogle Scholar - 31.H.-W. Park, A. Ramezani, J.W. Grizzle, A finite-state machine for accommodating unexpected large ground height variations in bipedal robot walking. IEEE Trans. Robot.
**29**(29), 331–345 (2013)CrossRefGoogle Scholar - 32.I. Poulakakis, J.W. Grizzle, The spring loaded inverted pendulum as the hybrid zero dynamics of an asymmetric hopper. IEEE Trans. Autom. Control
**54**(8), 1779–1793 (2009)MathSciNetCrossRefzbMATHGoogle Scholar - 33.M.J. Powell, A.D. Ames, Hierarchical control of series elastic actuators through control lyapunov functions. Decision and Control (CDC), 2014 IEEE 53rd Annual Conference on, 2986–2992Google Scholar
- 34.M.J. Powell, H. Zhao, A.D. Ames, Motion primitives for human-inspired bipedal robotic locomotion: walking and stair climbing, in
*2012 IEEE International Conference on Robotics and Automation (ICRA)*, 2012, pp. 543–549Google Scholar - 35.A Ramezani, J.W. Hurst, K. Akbari Hamed, J.W. Grizzle, Performance analysis and feedback control of ATRIAS, a three-dimensional bipedal robot. J. Dyn. Syst. Meas. Control
**136**(2), 0210112-1–0210112-12 (2014)Google Scholar - 36.H. Razavi, A.M. Bloch, C. Chevallereau, J.W. Grizzle, Restricted discrete invariance and self-synchronization for stable walking of bipedal robots, in
*American Control Conference*, 2015Google Scholar - 37.N. Sadati, G.A. Dumont, K.A. Hamed, W.A. Gruver,
*Hybrid Control and Motion Planning of Dynamical Legged Locomotion*. In: MengChu Zhou (ed.), IEEE Press Series on Systems Science and Engineering. (Wiley, 2012), 272 pages. ISBN: 978-1-118-31707-5Google Scholar - 38.J. Schmiedeler, ERNIE robot walking with different feet (2016). https://www.youtube.com/watch?v=T2x3VvPaacA Google Scholar
- 39.C.-L. Shih, J.W. Grizzle, C. Chevallereau, From stable walking to steering of a 3D bipedal robot with passive point feet. Robotica
**30**(7), 1119–1130 (2012)CrossRefGoogle Scholar - 40.A.S. Shiriaev, L.B. Freidovich, I.R. Manchester, Can we make a robot ballerina perform a pirouette? orbital stabilization of periodic motions of underactuated mechanical systems. Ann. Rev. Control
**32**(2), 200–211 (2008)CrossRefGoogle Scholar - 41.A.S. Shiriaev, L.B. Freidovich, S.V. Gusev, Transverse linearization for controlled mechanical systems with several passive degrees of freedom. IEEE Trans. Autom. Control
**55**(4), 893–906 (2010)MathSciNetCrossRefzbMATHGoogle Scholar - 42.K. Sreenath, H.-W. Park, I. Poulakakis, J.W. Grizzle, A compliant hybrid zero dynamics controller for stable, efficient and fast bipedal walking on MABEL. Int. J. Robot. Res.
**30**(9), 1170–1193 (2011)CrossRefGoogle Scholar - 43.K. Sreenath, H.-W. Park, I. Poulakakis, J.W. Grizzle, Embedding active force control within the compliant hybrid zero dynamics to achieve stable, fast running on MABEL. Int. J. Robot. Res.
**33**, 988–1005 (2014)CrossRefGoogle Scholar - 44.P. van Zutven,
*Control and Identification of Bipedal Humanoid Robots: Stability Analysis and Experiments*. PhD thesis, University of Technology, Eindhoven, 2014Google Scholar - 45.M. Vukobratović, B. Borovac, D. Surla, D. Stokic,
*Biped Locomotion*(Springer, Berlin, 1990)CrossRefzbMATHGoogle Scholar - 46.T. Wang, C. Chevallereau, D. Tlalolini, Stable walking control of a 3D biped robot with foot rotation. Robotica
**FirstView**:1–20, (2014)Google Scholar - 47.E.R. Westervelt, G. Buche, J.W. Grizzle, Experimental validation of a framework for the design of controllers that induce stable walking in planar bipeds. Int. J. Robot. Res.
**24**(6), 559–582 (2004)CrossRefGoogle Scholar - 48.E.R. Westervelt, J.W. Grizzle, C. Chevallereau, J.-H. Choi, B. Morris,
*Feedback Control of Dynamic Bipedal Robot Locomotion*. (CRC Press, Boca Raton, 2007)Google Scholar - 49.E.R. Westervelt, J.W. Grizzle, D.E. Koditschek, Hybrid zero dynamics of planar biped walkers. IEEE Trans. Autom. Control
**48**(1), 42–56 (2003)MathSciNetCrossRefzbMATHGoogle Scholar - 50.T. Yang, E.R. Westervelt, A. Serrani, J.P. Schmiedeler, A framework for the control of stable aperiodic walking in underactuated planar bipeds. Auton. Robot.
**27**(3), 277–290 (2009)CrossRefGoogle Scholar - 51.H.-H. Zhao, W.-L. Ma, A.D. Ames, M.B. Zeagler, Human-inspired multi-contact locomotion with AMBER2, in
*2014 ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS)*, 2014, pp. 199–210Google Scholar