GoCo: planning expressive commitment protocols
Abstract
This article addresses the challenge of planning coordinated activities for a set of autonomous agents, who coordinate according to social commitments among themselves. We develop a multiagent plan in the form of a commitment protocol that allows the agents to coordinate in a flexible manner, retaining their autonomy in terms of the goals they adopt so long as their actions adhere to the commitments they have made. We consider an expressive firstorder setting with probabilistic uncertainty over action outcomes. We contribute the first practical means to derive protocol enactments which maximise expected utility from the point of view of one agent. Our work makes two main contributions. First, we show how Hierarchical Task Network planning can be used to enact a previous semantics for commitment and goal alignment, and we extend that semantics in order to enact firstorder commitment protocols. Second, supposing a cooperative setting, we introduce uncertainty in order to capture the reality that an agent does not know for certain that its partners will successfully act on their part of the commitment protocol. Altogether, we employ hierarchical planning techniques to check whether a commitment protocol can be enacted efficiently, and generate protocol enactments under a variety of conditions. The resulting protocol enactments can be optimised either for the expected reward or the probability of a successful execution of the protocol. We illustrate our approach on a realworld healthcare scenario.
Keywords
Commitment protocols Intelligent agents Uncertainty Goal reasoning HTN planning Nondeterminism1 Introduction
Modern information technology (IT) applications in a variety of domains involve interactions between autonomous parties such as people and businesses. For example, IT serves a pivotal role for the patients, staff, departments, and stakeholders in a modern healthcare centre [40]. The field of multiagent systems provides constructs to deal with such settings through the notions of autonomous agents and their protocols of interaction. However, many challenges remain to building a realistic multiagent society.
In particular, agents in a system, although autonomous, may be interdependent in subtle ways. The physical, social, or organisational environment in which they interact can be complex. We need ways to accommodate the environment while supporting decoupling of the agents’ internals from their interaction, thus facilitating the composition of multiagent systems. The notion of a sociotechnical system (STS) [23, 58] provides a basis for representing such interactions between agents in the context of an organisation, such as a healthcare centre, and respecting technical artefacts required for the effective operation of the organisation. Here, activities in an STS are characterised by a combination of the goals sought by and the specified interactions between the agents.
Along these lines, the notion of (social) commitments [54, 55, 56] has been adopted to describe interactions among agents in a highlevel implementationindependent manner. A particularly important feature of commitments is that they are a public construct in that they define a relationship between the concerned parties. Of course, any party may represent a commitment internally but the meaning and significance of commitments derives from their relational nature. Over the years, there has been progress on structuring interactions in terms of commitment protocols [7, 8, 19, 20, 25, 35, 69]. Commitment protocols offer a noted advantage in that they enable participating agents to coordinate in a flexible manner, retaining their autonomy. An agent would comply with a protocol as long as it does not violate any of its commitments—thus, in general, an agent may act in a variety of compliant ways to satisfy its goals. Once stakeholders design a commitment protocol, it is then up to the individual agents to instantiate commitments operationally in order to achieve their individual goals [23].

Where do the protocols come from? Collaborative organisational design and redesign involve stakeholders jointly exploring the specification of a sociotechnical system in terms of the goals and commitments of the individual agents participating in the STS. Previous work examined this problem qualitatively, for example, through Protos [18], an abstract design process for capturing requirements of multiple stakeholders through an STS specification constructed in terms of commitments (with its associated formal assumptions).

How can an agent gain assurance that its goals are indeed achievable when the STS is instantiated? An important challenge is to specify an STS so that autonomous agents with their individual goals would want to participate in it—that is, to provide an assurance to the agents that their participation would lead to their goals being satisfied.
Our planningbased approach provides a computational mechanism to reason about a number of properties of commitment protocols and their enactments. In this article, we consider an enactment to be an instantiation of a protocol defined in terms of goals and commitments that corresponds to the full hierarchical decomposition of an Hierarchical Task Network (HTN) task using a method library [29], simply put, an enactment is a sequence of executable actions that fulfills agent goals and satisfies their commitments. An enactment is optimal if it maximises the expected utility [12] across all alternative enactments. First, we can identify whether or not a commitment protocol is compatible with the agent’s goals, i.e., whether there is at least one enactment of the protocol that achieves one or more individual goals. Second, our approach can quickly generate a suboptimal enactment to prove a protocol is feasible as well as generate all possible enactments, if needed. Third, we can provide a quantitative assessment of the utilities of each possible enactment of the commitment protocol. Fourth, we can use the exhaustive generation of enactments to select among them regarding their utility to one or more of the participating agents.
Our previous work provided the initial steps to automate the verification of the realisability of instances of commitment protocols using planning formalisms [50, 62] as well as quantifying the utility of such instances [51]. This article consolidates these contributions and provides five novel contributions. First, we formalise a typical sociotechnical system, namely a realworld healthcare scenario (introduced in Sect. 2 and formalised in Sect. 5), that is used throughout the article to illustrate the approach, and which can be used to test the scalability of approaches such as ours. Second, in Sect. 3 (augmented by Sect. 4.1) we provide a complete account of the planning formalism used to reason about commitment protocols and how they relate to individual agent goals. This formalisation extends our previous work by providing a probabilistic view of the environment and the utilities of states, allowing the algorithms that generate possible enactments to reason about their expected utilities. Third, in Sect. 4 we provide the complete extended formalisation of the commitment dynamics and reasoning patterns [63] underlying our verification system. Fourth, in Sect. 6 we develop a depthfirst search algorithm, NDPyHop, to explore commitment protocols in stochastic environments and generate realisable protocol enactments. We show that using NDPyHop we can generate protocol enactments that satisfy a minimal expected utility criterion in exponential time and linear memory. Finally, in Sect. 7 we evaluate the efficiency of the resulting approach for increasingly complex instantiations of the healthcare scenario. We conclude the article with a discussion of related research and future directions.
2 Healthcare scenario
The scenario is drawn from a realworld healthcare application domain. Figure 1 shows a breast cancer diagnosis process adapted from a report produced by a U.S. governmental committee [3]. We omit the tumour board, which serves as an authority to resolve any disagreements among the participants. For readability, we associate feminine pronouns with the patient, radiologist, and registrar, and masculine pronouns with the physician and pathologist.^{1}
The process begins when the patient (not shown in the figure) visits a primary care physician, who detects a suspicious mass in her breast. He sends the patient over to a radiologist for mammography. If the radiologist notices suspicious calcifications, she sends a report to the physician recommending a biopsy. The physician requests the radiologist to perform a biopsy, who collects a tissue specimen from the patient, and sends it to the pathologist. The pathologist analyses the specimen, and performs ancillary studies. If necessary, the pathologist and radiologist confer to reconcile their findings and produce a consensus report. The physician reviews the integrated report with the patient to create a treatment plan. The pathologist forwards his report to the registrar, who adds the patient to a statewide cancer registry that she maintains.
We formalise this scenario in the next sections and use it as an example in the rest of the article.
3 Formal background
Here we introduce the fundamental background upon which we build our formalisation of planning for commitment protocols. Section 3.1 defines the firstorder logic language we use to formally represent agent goals and commitments, and the planning operators with which the agents reason about the realisation of the commitment protocols. We review the propositional formalisation of commitments from Telang et al. [63] in Sect. 3.2, from which (in Sect. 4) we will build a firstorder operationalisation from Meneguzzi et al. [50] that can handle different instances of the same commitment. Finally, in Sect. 3.3 we introduce the planning formalism we subsequently extend in Sect. 4.1 and employ thereafter.
3.1 Formal language and logic
Our formal language is based on firstorder logic and consists of an infinite set of symbols for predicates, constants, functions, and variables. It obeys the usual formation rules of firstorder logic and follows its usual semantics when describing planning domains [32].
Definition 1
(Term) A term, denoted generically as \(\tau \), is a variable following the prolog convention of an uppercase starting letter \(A,B, \dots , Z, \dots \) (with or without subscripts); a constant a, b, c (with or without subscripts); or a function term \(f(\tau _0,\ldots ,\tau _n)\), where f is a nary function symbol applied to (possibly nested) terms \(\tau _0,\ldots ,\tau _n\).
Definition 2
(Atoms and formulas) A firstorder atomic formula (or atom), denoted as \(\varphi \), is a construct of the form \(p(\tau _0,\ldots ,\tau _n)\), where p is an nary predicate symbol and \(\tau _0,\ldots ,\tau _n\) are terms. A firstorder formula \(\varPhi \) is recursively defined as \(\varPhi ::= \varPhi \wedge \varPhi '  \lnot \varPhi  \varphi \). A formula is said to be ground, if it contains no variables or if all the variables in it are bound to a constant symbol.
We assume the usual abbreviations: \(\varPhi \vee \varPhi '\) stands for \(\lnot (\lnot \varPhi \wedge \lnot \varPhi ')\); \(\varPhi \rightarrow \varPhi '\) stands for \(\lnot \varPhi \vee \varPhi '\) and \(\varPhi \leftrightarrow \varPhi '\) stands for \((\varPhi \rightarrow \varPhi ') \wedge (\varPhi '\rightarrow \varPhi )\). Additionally, we also adopt the equivalence \(\{\varPhi _1,\ldots ,\varPhi _n\}\equiv (\varPhi _1\wedge \cdots \wedge \varPhi _n)\) and use these interchangeably. Our mechanisms use firstorder unification [2], which is based on the concept of substitutions.
Definition 3
(Substitution) A substitution \(\sigma \) is a finite and possibly empty set of pairs \(\{x_1/\tau _1,\ldots , x_n/\tau _n\}\), where \(x_1,\ldots ,x_n\) are distinct variables and each \(\tau _i\) is a term such that \(\tau _i \ne x_i\).
Given an expression E and a substitution \(\sigma = \{x_1/\tau _1, \ldots ,x_n/\tau _n\}\), we use \(E\sigma \) to denote the expression obtained from E by simultaneously replacing each occurrence of \(x_i\) in E with \(\tau _i\), for all \(i \in \{1,\ldots ,n\}\).
Substitutions can be composed; that is, for any substitutions \(\sigma _1=\{x_1/\tau _1,\ldots ,x_n/\tau _n\}\) and \(\sigma _2=\{y_1/\tau '_1,\ldots ,y_k/\tau '_k\}\), their composition, denoted as \(\sigma _1 \cdot \sigma _2\), is defined as \(\{x_1/(\tau _1\cdot \sigma _2),\ldots ,x_n/(\tau _n\cdot \sigma _2), z_1/ (z_1 \cdot \sigma _2), \ldots , z_m/ (z_m \cdot \sigma _2)\}\), where \(\{z_1,\ldots ,z_m\}\) are those variables in \(\{y_1,\ldots ,y_k\}\) that are not in \(\{x_1,\ldots ,x_n\}\). A substitution \(\sigma \) is a unifier of two terms \(\tau _1,\tau _2\), if and only if \(\tau _1\cdot \sigma = \tau _2\cdot \sigma \).
Definition 4
(Unify Relation) Relation \( unify (\tau _1,\tau _2,\sigma )\) holds iff \(\tau _1\cdot \sigma = \tau _2\cdot \sigma \). Moreover, \( unify (p(\tau _0, \ldots , \tau _n),p(\tau '_0,\ldots ,\tau '_n),\sigma )\) holds iff \( unify (\tau _i,\tau '_i,\sigma ),\) for all \(0\le i\le n\).
Thus, two terms \(\tau _1,\tau _2\) are related through the \( unify \) relation if there is a substitution \(\sigma \) that makes the terms syntactically equal. The logic language is used to define a state within a planning domain, as follows:
Definition 5
(State) A state is a finite set of ground atoms (facts) that represent logical values according to some interpretation. Facts are divided into two types: positive and negated facts, as well as constants for truth (\(\top \)) and falsehood (\(\bot \)).
3.2 Goal and commitment operational semantics
We adopt the notion of a (social) commitment, which describes an element of the social relationships between two agents in highlevel terms. A commitment in this article is not to be confused with a ‘psychological’ commitment expressing an agent’s entrenchment with its intentions [16, 54, 57]. Commitments are extensively studied in multiagent systems [27, 31, 65] and are traditionally defined exclusively in terms of propositional logic constructs. Recent commitmentquery languages, e.g., [21, 22], go beyond propositional constructs but do not address the challenges studied in this article.
We make a distinction between commitment templates (which describe commitments in general) and commitment instances, which allow for variable bindings that differentiate commitments adopted by specific parties and referring to specific objects in the domain. Although we elaborate on the formalisation of commitment instances in Sect. 4.2, we represent commitment template tuples exactly as the commitment formalisation of Telang et al. [63] and define commitment instances later in the article. Thus, in this section, we explain the commitment formalism in a simplified manner before extending it to handle multiple commitment instances and the additional formalism required to reason with them.

\(\mathsf {C}_1\): physician commits to patient to providing the diagnosis (represented by the predicate diagnosisProvided) if patient requests it (represented by the predicate diagnosisRequested), and does not violate commitments \(\mathsf {C}_2\) and \(\mathsf {C}_3\) (represented by the vio(X) predicate). To not violate \(\mathsf {C}_2\) and \(\mathsf {C}_3\), patient needs to keep her imaging and biopsy appointments.
\(\mathsf {C}_1 = \mathsf {C}\)(physician, patient, diagnosisRequested \(\wedge \) \(\lnot \) vio(\(\mathsf {C}_2\)) \(\wedge \) \(\lnot \) vio(\(\mathsf {C}_3\)), diagnosisProvided)

\(\mathsf {C}_2\): patient commits to physician to keep the imaging appointment (represented by the iAppointmentRequested predicate) upon physician’s request (represented by the iAppointmentKept predicate).
\(\mathsf {C}_2 = \mathsf {C}\)(patient, physician, iAppointmentRequested, iAppointmentKept)

\(\mathsf {G}_{1}\): physician has a goal to have a diagnosis requested.
\(\mathsf {G}_1 = \mathsf {G}\)(physician, \(\top \), diagnosisRequested, \(\bot \))

\(\mathsf {G}_{2}\): patient has a goal to have a diagnosis provided.
\(\mathsf {G}_2 = \mathsf {G}\)(patient, \(\top \), diagnosisProvided, \(\bot \))
3.3 Classical and HTN planning
Although classical planning has ExpSpaceComplete complexity in its most general formulation, approaches such as Hierarchical Task Network (HTN) planning [30] use suitable domain knowledge to solve many types of domains highly efficiently. A Hierarchical Task Network (HTN) planner [32] considers tasks to be either primitive (equivalent to classical operators) or compound (abstract highlevel tasks). It generates a plan by refinement from a top level goal: the planner recursively decomposes compound tasks by applying a set of methods until only primitive tasks remain. Methods in HTNs represent domain knowledge, which, when efficiently encoded enables HTN planners to be substantially more efficient than other classical planning approaches. Many HTN planners, such as JSHOP2 [39], also allow the encoding of domain knowledge in terms of hornclause type conditional formulas to encode simple inferences on the belief state, which we use to encode the dynamics of goal and commitment states.
We leverage the efficient solution algorithms of regular HTN planning to plan commitment–goal protocols in Sect. 4, and extend HTN planning to account for stochastic actions and develop a new algorithm, to accommodate uncertainty within the protocols, in Sect. 6. In between, Sect. 5 formalises the healthcare scenario in the HTN planning language of the next section.
4 Planning formalisation for the operational semantics
We now develop the logical rules, operators, and methods in the HTN formalism, which together operationalise the goal and commitment dynamics introduced above. Note that, although the planning model includes rewards, we assume that rewards are only accrued by actions of an agent in the environment (i.e., actions disjoint from the operational semantics of commitment and goal manipulation), consequently, operators in our operational semantics define zero rewards (and zero cost). Existing techniques show that it is straightforward to convert models of processes into HTN domain specifications [53], as well as to convert formalised process description languages, such as business process languages, into planning operators [38]. Granted these techniques, we assume that a large part of the domainspecific knowledge used in HTN encoding can be generated from the processes being validated.
In order to reason about multiple instances of the same type of commitment, we need to use firstorder predicates to represent the parts of commitments and goals. Consequently, our formalisation of the operational semantics departs from the propositional definitions of Sect. 3.2 [63] in two ways. First, in order to be able to identify the logical rules that refer to a specific type of commitment and goal, we extend the tuples with a type. Second, once an agent creates a specific instance of a type of commitment (alternatively goals), we introduce the variables required to identify such instances of commitments and goals as part of its tuple. It follows that our semantics presented below is in an expressive firstorder setting. First, in Sect. 4.1, we formalize HTN planning in our context. Then, in Sects. 4.2 and 4.3 we formalise respectively commitments and goals within the planning framework. Finally, in Sect. 4.4 we describe how to operationalize our semantics using a HTN planner.
4.1 Formalisation of HTN planning in a multiagent system
We first introduce a formalisation of HTN planning geared to multiagent systems. Let \(\langle \mathcal {I}, \mathcal {D}, \mathcal {A}\rangle \) be a multiagent system (MAS) composed of an initial state \(\mathcal {I}\), a shared plan library (also called domain knowledge), and a set of agents \(\mathcal {A}\) where each agent \(a = \langle \mathcal {G}, \mathcal {C} \rangle \in \mathcal {A}\) has a set of individual goals \(\mathcal {G}\) and commitments \(\mathcal {C}\). In this formalism, we assume each agent to have a known set of individual goals \(\mathcal {G}\) (each agent may have zero or more goals) and commitments \(\mathcal {C}\) (representing, for example, known workrelations or cooperation networks). Agent goals are not necessarily shared between multiple agents and commitments may not necessarily connect all agents in the multiagent system.
The shared domain knowledge \(\mathcal {D} = \langle \mathcal {M}, \mathcal {O}, \mathcal {R}\rangle \) consists of an HTN planning domain, which comprises a set of methods \(\mathcal {M}\), a set of operators \(\mathcal {O}\), and a reward function \(\mathcal {R}\). Operators are divided into strict mutually disjoint subsets of domain operators \(\mathcal {O}_{d}\) (e.g., operators modelling medical and administrative procedures) and social dynamics operators \(\mathcal {O}_{s}\) (i.e., the operators referring to the reasoning about goals and commitments). \(\mathcal {O}_{d}\) is specified by the designer of the multiagent system (and varies with each application) whereas \(\mathcal {O}_{s}\) is domainindependent and specified in this section.
In order to achieve their goals \(\mathcal {G}\), agents try to accomplish tasks by decomposition using methods \(\mathcal {M}\), which decompose higherlevel tasks t into more refined tasks until they can be decomposed into primitive tasks corresponding to plans of executable operators \(\omega _1, \dots , \omega _n\). Tasks in an HTN are divided into a set of primitive tasks \(t \in \mathcal {O}\). In our formalisation \(\mathcal {O} = \mathcal {O}_{d} \cup \mathcal {O}_{s}\) and a set nonprimitive tasks \(t \in \mathcal {T}\) defined implicitly as all tasks symbols mentioned in m that are not in \(\mathcal {O}\). Formally, a method \(m = \langle t, \phi , (t'_{0}, \dots , t'_{n})\rangle \) decomposes task t in a state \(s\models \phi \) (Definition 5) by replacing it by \(t_{0}, \dots , t_{n}\) in a task network. Thus, applying method m above to a task network \((t_0, \dots , t, \dots , t_{m})\) generates a new task network \((t_0, \dots , t'_{0}, \dots , t'_{n}, \dots , t_{m})\). Solving an HTN planning problem consists of decomposing an initial task network \(t_0\) from an initial state \(\mathcal {I}\) using methods in m. A solution is a task network T such that all task symbols in T are elements of \(\mathcal {O}\) and they are sequentially applicable from \(\mathcal {I}\) using the \(\gamma \) function of Eq. (2). The key objective of our work is to generate protocol instances or enactments and ensure that for a given MAS \(\langle \mathcal {I}, \mathcal {D}, \mathcal {A}\rangle \), we can generate valid HTN plan for any toplevel task \(t_0 \in \mathcal {D}\). A toplevel task in \(\mathcal {D}\) is a nonprimitive task that is not part of any task network resulting from a method. We assume that a domain designer always includes such tasks in the domaindependent part of the method library. We now develop a plan library (operators and methods) corresponding to the domainindependent commitment dynamics, which we refer to as \(\mathcal {I}_s\).
We assume, without loss of generality, that the commitment dynamics operators \(\mathcal {O}_{s}\) are all deterministic. Hence there is no uncertainty regarding an agent’s own commitment and goal state. Crucially, by contrast, the domain operators \(\mathcal {O}_{d}\) may or may not be stochastic.
The reward of individual atoms and the probabilities are domainspecific information that need to be modelled by the designer of the commitment protocol. As an example, consider a state in which a patient named alice has an imaging appointment but has not attended it yet (and thus a literal iAppointmentKept(alice) is not true). If we define a reward function whereby iAppointmentKept(X) has a value of 10, then the state resulting from the execution of an action attendImaging(Patient, Physician, Radiologist) that has in its positive effect iAppointmentKept(alice), the resulting state \(s'\) will accrue value of 10 in its utility.
4.2 Commitment dynamics

Ct is the commitment type

De is the debtor of the commitment

Cr is the creditor of the commitment

P is the antecedent, a universally quantified firstorder formula

Q is the consequent, an existentially quantified firstorder formula

\(\overrightarrow{Cv}\) is a list \([v_{1}, \dots , v_{n}]\) of variables identifying a specific instance of Ct.
The first challenge in encoding commitment instances^{2} in a firstorder setting is ensuring that the components of a commitment are connected through their shared variables. To this end, we model the entire set of variables of a particular commitment within one predicate. Here, the number of variables n for a commitment is equivalent to the sum of arities of all firstorder predicates in P, and Q, so if \(P = p_{0}(\overrightarrow{t_{0}}) \dots p_{a}(\overrightarrow{t_{k}})\) and \(Q = p_{a+1}(\overrightarrow{t_{k+1}}) \dots p_{b}(\overrightarrow{t_{m}})\), then \(n = \sum _{i = 0}^{i = m}\overrightarrow{t_{i}}\).
For example, consider a radiologist X who commits to reporting the imaging results of patient Y to physician Z if physician Z requests an imaging scan of patient Y. This is formalised as C(X, Z, \(\textsf {imagingRequested(Z,Y)},\) \(\textsf {imagingResultsReported(X,Z,Y)})\). This commitment has two predicates and three unique variables; however, with no loss of generality, we model the variable vector [X, Z, Z, Y, X, Z, Y] as having seven variables. Notice that no information about the implicit quantification of the variables induced by the commitment semantics is lost, since the logical rules referring to these variables remain the same, and the variable vector is simply used to identify unique bindings of commitment and goal instances.
A commitment is active if it is not null, terminal, pending, or satisfied [Eq. (9)]. An active commitment is conditional if its antecedent (p) is false [Eq. (7)], and is detached otherwise [Eq. (8)]. Note that terminal is a shortcut for being in any of the transitions cancelled, released, or expired [Eq. (13)]. A commitment is terminated if it is released or it is cancelled when its antecedent is false [Eq. (10)]. A commitment is violated if it is cancelled when its antecedent is true [Eq. (11)]. A commitment is satisfied if it is not null and not terminal, and its consequent (q) is true [Eq. (12)].
4.3 Goal dynamics

Gt is the goal type;

X is the agent that has the goal;

Pg is the goal precondition;

S is the success condition;

F is the failure condition; and

\(\overrightarrow{Gv}\) is a list of variables identifying specific instances of Gt.
For the reader’s convenience, Fig. 3 is reproduced as Fig. 5 with annotations corresponding to the numbers of the equations.^{5}
We use Eqs. (23)–(29) to logically represent the dynamics of an agent’s goal. As with commitments, goal states is instance dependent and ceases to be null for a particular instance whenever a predicate describing its instance number and variables is true, as encoded in Eq. (23). Although the state transition diagram of Fig. 3 contains only a Terminated state, we use an auxiliary axiom to identify terminal states (i.e., Failed and Satisfied) and ensure that once a goal reaches this state, it can never transition back to any other state, enforced by Eq. (29).
4.4 Reasoning patterns using hierarchical plans
The reasoning patterns extended from Telang et al. [63] in earlier sections can now be directly implemented using HTN methods relating the commitment and goal operators to domaindependent operators and predicates. This implementation as an HTN allows one to directly verify whether a specific commitment protocol is enactable using a number of reasoning patterns that allows individual agents to achieve their goals either directly or by adopting commitments towards other agents.
For instance, Telang et al. [63] employ the notions of end goal, commitment, means goal, and discharge goal. An end goal of an agent is a goal that the agent desires to achieve. Suppose \(G_{end} = \mathsf {G}(x, pg, s, f)\) is an end goal. If agent x lacks the necessary capabilities to satisfy G (or for some other reason), x may create a commitment \(C = \mathsf {C}(x, y, s, u)\) toward another agent y. Agent y may create a means goal \(G_{means} = \mathsf {G}(y, pg', s, f')\) to detach C, and agent x may create a discharge goal \(G_{discharge} = G(x, pg'', u, f'')\) to satisfy C.
We provide the full formalisation of all HTN rules encoding reasoning patterns from Telang et al. [63] in Appendix A and online [47]. Bringing it all together, in the next section we can now define a commitment protocol that implements the breast cancer diagnosis process from Sect. 2, and we can use an HTN planner to check for realisability. In Sect. 6 we describe our HTN planning algorithm which, further, also accommodates uncertainty in the agents’ excecution environment.
5 Healthcare scenario formalisation
5.1 Goals
Goals from the healthcare scenario
ID  Goal 

\(G_1\)  \(\mathsf {G}\)(physician, \(\top \), diagnosisRequested(patient,physician), \(\bot \)) 
\(G_2\)  \(\mathsf {G}\)(patient, \(\top \), diagnosisRequested(patient,physician), \(\bot \)) 
\(G_3\)  \(\mathsf {G}\)(radiologist, \(\top \), imagingRequested(physician,patient) \(\wedge \) iAppointmentRequested(patient, radiologist), \(\bot \)) 
\(G_4\)  \(\mathsf {G}\)(physician, \(\top \), imagingRequested(physician,patient) \(\wedge \) iAppointmentRequested(patient, radiologist), \(\bot \)) 
\(G_5\)  \(\mathsf {G}\)(patient, \(\top \), iAppointmentKept(patient, radiologist), \(\bot \)) 
\(G_6\)  \(\mathsf {G}\)(radiologist, \(\top \), imagingResultsReported(radiologist, physician, patient), \(\bot \)) 
\(G_7\)  \(\mathsf {G}\)(radiologist, \(\top \), biopsyRequested(physician, patient) \(\wedge \) bAppointmentRequested(patient, pathologist), \(\bot \)) 
\(G_8\)  \(\mathsf {G}\)(physician, \(\top \), biopsyRequested(physician, patient) \(\wedge \) bAppointmentRequested(patient, pathologist), \(\bot \)) 
\(G_9\)  \(\mathsf {G}\)(patient, \(\top \), bAppointmentKept(patient, pathologist), \(\bot \)) 
\(G_{10}\)  \(\mathsf {G}\)(pathologist, \(\top \), pathologyRequested(physician, pathologist, patient) \(\wedge \) tissueProvided(patient), \(\bot \)) 
\(G_{11}\)  \(\mathsf {G}\)(radiologist, \(\top \), pathologyRequested(physician, pathologist, patient) \(\wedge \) tissueProvided(patient), \(\bot \)) 
\(G_{12}\)  \(\mathsf {G}\)(pathologist, \(\top \), pathResultsReported(pathologist, physician, patient), \(\bot \)) 
\(G_{13}\)  \(\mathsf {G}\)(radiologist, \(\top \), integratedReport(patient, physician), \(\bot \)) 
\(G_{14}\)  \(\mathsf {G}\)(registrar, \(\top \), patientReportedToRegistrar(patient, registrar), \(\bot \)) 
\(G_{15}\)  \(\mathsf {G}\)(pathologist, \(\top \), patientReportedToRegistrar(patient, registrar), \(\bot \)) 
\(G_{16}\)  \(\mathsf {G}\)(registrar, \(\top \), inRegistry(patient), \(\bot \)) 
5.2 Commitments
Commitments from the healthcare scenario
ID  Commitment 

\(C_1\)  \(\mathsf {C}\)(physician, patient, diagnosisRequested(patient, physician) \(\wedge \lnot \) vio(C2) \(\wedge \lnot \) vio(C3), diagnosisProvided(physician, patient)) 
\(C_2\)  \(\mathsf {C}\)(patient, physician, iAppointmentRequested(physician, radiologist), iAppointmentKept(physician, radiologist)) 
\(C_3\)  \(\mathsf {C}\)(patient, physician, bAppointmentRequested(physician, pathologist), bAppointmentKept(physician, pathologist)) 
\(C_4\)  \(\mathsf {C}\)(radiologist, physician, imagingRequested(physician, patient) \(\wedge \) iAppointmentKept(patient, radiologist), imagingResultsReported(radiologist, physician, patient) 
\(C_5\)  \(\mathsf {C}\)(radiologist, physician, biopsyRequested(physician, patient) \(\wedge \) bAppointmentKept(patient, radiologist), radPathResultsReported(radiologist, physician, patient) 
\(C_6\)  \(\mathsf {C}\)(pathologist, radiologist, pathologyRequested(physician, pathologist, patient) \(\wedge \) tissueProvided(patient), pathResultsReported(radiologist, physician, patient)) 
\(C_7\)  \(\mathsf {C}\)(registrar, pathologist, patientReportedToRegistrar(patient, registrar), inRegistry(patient)) 
Domain operators from the healthcare scenario
ID  Domain operator 

\(O_1\)  requestAssesment(patient, physician) 
\(O_2\)  requestImaging(patient, physician, radiologist) 
\(O_3\)  performImaging(patient, physician, radiologist) 
\(O_4\)  requestBiopsy(patient, physician, radiologist) 
\(O_5\)  performBiopsy(patient, physician, radiologist) 
\(O_6\)  requestPathologyReport(patient, physician, radiologist, pathologist) 
\(O_7\)  sendPathologyReport(patient, physician, radiologist, pathologist) 
\(O_{8}\)  sendRadiologyReport(patient, physician, radiologist) 
\(O_{9}\)  sendIntegratedReport(patient, physician, radiologist) 
\(O_{10}\)  generateTreatmentPlan(patient, physician) 
\(O_{11}\)  reportPatient(patient, pathologist, registrar) 
\(O_{12}\)  addPatientToRegistry(patient, registrar) 
5.3 Domain operators
Table 3 lists the domainspecific operators. In \(O_1\), a patient requests diagnosis from a physician. In \(O_2\), a physician requests a radiologist for imaging, and requests an imaging appointment for the patient. In \(O_3\), the radiologist performs imaging scan on the patient upon the request from the physician, and when the patient keeps the imaging appointment. In \(O_4\), the physician requests the radiologist for a biopsy, and requests a biopsy appointment for the patient. In \(O_5\), the radiologist performs biopsy on the patient upon the request from the physician and when the patient keeps the biopsy appointment. In \(O_6\), the radiologist requests the pathologist for pathology report. In \(O_7\), the pathologist provides the pathology report to the radiologist. In \(O_{8}\), the radiologist sends the radiology report to the physician. In \(O_{9}\), the radiologist sends the integrated radiology and pathology report to the physician. In \(O_{10}\), the physician generates a treatment plan after receiving the radiology report or the integrated radiology pathology report. In \(O_{11}\), the pathologist reports a patient with cancer to the registrar. In \(O_{12}\), the registrar adds a patient with cancer to the cancer registry.
6 Dealing with uncertainty
So far we have described an approach for planning commitment–goal protocols, leveraging the efficient solution methods of HTN planning. We have allowed firstorder operators, but we have made the assumption that the outcomes of operators are observable and deterministic, i.e., there is no uncertainty. In this section, we extend our approach to accommodate uncertainty. We present a nondeterministic HTN planning procedure whose solutions encode commitment protocol enactments that take into account environmental uncertainty. Finally, we analyse properties of these protocol enactments and evaluate the expressivity and complexity of the underlying planning problem.
6.1 NDPyHop algorithm
Recall that the shared domain knowledge \(\mathcal {D}\) of the MAS consists of an HTN planning domain comprising a set of methods \(\mathcal {M}\), a set of operators \(\mathcal {O}= \mathcal {O}_{d} \cup \mathcal {O}_{s}\), and a reward function \(\mathcal {R}\). We continue to assume, without loss of generality, that dynamics operators \(\mathcal {O}_{s}\) are all deterministic , but now allow domain operators \(\mathcal {O}_{d}\) to be deterministic or stochastic. Recall also that we can compute the expected utility of each generated plan using the probability information for each operator outcome and the reward function \(\mathcal {R}(s,s')\), which returns the reward for transitioning from state s to state \(s'\).
In order to generate an optimal and feasible plan for achieving the goals of all participants within a MAS \(\langle \mathcal {I}, \mathcal {A} \rangle \), we employ a nondeterministic HTN planning algorithm adapted from earlier work [43] and implemented as an extension of the PyHop planner^{7} in the Python language. Specifically, instead of searching for a single socalled strongcyclic policy for the problem in a nondeterministic domain, our algorithm quickly searches for any plan with nonzero probability (proving that a protocol is enactable), and then continuing the search for higher utility plans (to achieve a desired minimal utility). In Sect. 7 we report on a reimplementation in the Ruby language; the algorithm remains the same.
The algorithm, illustrated in Fig. 7, is composed of two functions. First, it begins with the NDPyHop function, which takes an initial state \(\mathbf {I}\), an initial task network t, and domain knowledge \(\mathcal {D}\), and returns the utility of the best plan found. The initial state \(\mathbf {I}\) for the problem to be computed is generated by combining the initial state \(\mathcal {I}\) from the MAS, with the rules \(\mathcal {I}_s\) for goal and commitment dynamics, as defined in Sect. 4 [50], and predicates to uniquely identify and handle the dynamics of each goal and commitment throughout the planning process. Specifically, \(\mathcal {I}_s\) comprises the logic rules from Eqs. (4)–(13) and (20)–(29).
Second, at the core of the algorithm, we use the \(\textsc {ForwardSearch}\) function that searches forward (in the state space as operators in \(\mathcal {O}\) are executed) and downward (in the task decomposition space as methods in \(\mathcal {M}\) are selected to refine tasks), much like traditional HTN forward decomposition algorithms [32].
The \(\textsc {ForwardSearch}\) algorithm takes as input an initial state s, a task network t for decomposition, a partial plan \(\pi \) with the operators selected so far in the search process (and annotated with its probability of success and its expected utility), and an HTN domain \(\mathcal {D}\). In order to decompose t it first checks if t is fully decomposed, i.e., no task remains to be decomposed, and if so yields the full path in the contingency plan (Line 3). Instead, if further decomposition is possible, the algorithm takes the first task \(t_0\) in the HTN (Line 5). If \(t_0\) is a primitive task, the algorithm simulates the execution of all possible operator instantiations corresponding to the task, and decomposes every possible outcome of each operator (Lines 7–13). If \(t_0\) is a compound task, the algorithm tries all possible applicable methods, exactly like a traditional HTN planning algorithm (Lines 14–17). In either case, the algorithm recurses to perform the decomposition (Lines 13 or 17).
6.2 Expressiveness and complexity
In Sect. 7 we report on the empirical performance of our implementation of \(\textsc {NDPyHop}\). In this section, we examine our approach in conceptual terms.
Our intended problem setting is the specification of sociotechnical systems. Accordingly, we apply STS as a qualitative basis for evaluating our approach. Following Chopra and colleagues [18, 23], we understand an STS in terms of the commitments arising between the roles in that STS, specifically as a multiagent protocol.
It helps to think of an STS as a microsociety in which the protocol characterizes legitimate interactions. Therefore, the problem of specifying an STS is none other than the one of creating a microsociety that accommodates the needs of its participants. Indeed, for an STS to be successful, it must attract participation by autonomous parties. For this reason, it is important for the designer of an STS to take into account the goals of the prospective participants, that is, stakeholders in the STS.
Even though an STS would have two or more participants, it could be specified by one stakeholder, and often is. Marketplaces, such as eBay, are STSs that are specified by one stakeholder, in this case, eBay Inc. Our approach most directly reflects this case in which one stakeholder brings together the requirements and searches for proposed designs for consideration. That is, there is one locus of planning although the plan itself, viewed as a protocol, captures the actions of multiple participants. Potentially, the proposed designs could be voted upon or other otherwise negotiated upon—although negotiation is not in our present scope.
Accordingly, we proceed by assuming that a single mediating agent \(m \in \mathcal {A}\) is concerned to plan a MAS commitment protocol. The problem then is to validate a MAS including \(\mathcal {A}\) regarding its achievability, as we formally define below.
Definition 6
(Realisable MAS) A MAS \(\langle \mathcal {I}, \mathcal {A} \rangle \) is said to be realisable if the contingency plan generated by \(\textsc {ForwardSearch}{(\mathbf {I}, t , [], \mathcal {D})}\) contains at least one path with nonzero probability.
Informally, if the HTN formalisation of the domaindependent actions (e.g., a sociotechnical system specification), goals and commitments, together with the domainindependent HTN formalisation of Sects. 4.4 and 6.1, generate a realisable MAS from Definition 6—as proven by the algorithm of Fig. 7—then there exists at least one feasible joint plan representing a protocol enactment that achieves the goals of the system. In addition, each plan generated by NDPyHop measures its probability of success and its expected utility, allowing a system designer to choose the minimal quality required of the resulting enactment, i.e., which MAS is acceptable, in Definition 7.
Definition 7
(Acceptable MAS) A MAS \(\langle \mathcal {I}, \mathcal {A} \rangle \) is said to be acceptable w.r.t. an established utility \(\mathbf {U}\) if it is realisable and if \(\pi ^{*} = \textsc {NDPyHop}{(\mathbf {I}, t ,\mathcal {D})}\) is such that \(\pi ^{*}\cdot u \ge \mathbf {U}\).
Informally, an acceptable MAS has a certain expected utility on average, while, as the time allowed for NDPyHop to run reaches infinity, we have a guarantee to eventually generate the optimal plan.
Using these definitions, we can design multiple applications for the anytime algorithm of \(\textsc {ForwardSearch}\). For example, if there are time pressures on the agents to generate a commitment protocol in a short period of time (for example, for negotiation), \(\textsc {NDPyHop}\) can be modified to return a commitment protocol that proves a MAS (Definition 6) is realisable quickly while waiting for the algorithm to verify the existence of a protocol that proves an MAS is acceptable (Definition 7). Proving the former is fairly quick, since it implies generating only any one decomposition with nonzero utility, while proving the latter may, in the worst case, requires the algorithm to explore the entire possible set of plans.
Since our encoding requires logic variables in the HTN due to the firstorder logicstyle formalisation, as well as arbitrary recursions—which are a possibility from the formal encoding of a user’s application into a planning domain—the type of HTN problem we need to solve can fall into the hardest class of HTN planning [1]. Generating all possible plans for an arbitrarily recursive HTN with variables is semidecidable in the worst case [30]. Whereas, if we restrict the underlying planning domains to have only totally ordered tasks (as our domainindependent methods are), then the complexity of finding an acceptable plan is ExpSpace [1]. Hence assuming the domain follows what Erol et al. [29] define as ‘regular’ HTN methods (at most one nonprimitive, right recursive task), our algorithm has to solve an ExpSpacecomplete problem. This complexity, however, only refers to the language of HTN planning itself, so in practice, as is illustrated by our empirical evaluation, problems can be solved quite efficiently.
7 Implementation and experiments
We now exhibit our approach to planning expressive commitment protocol enactments using HTN planning. This section demonstrates the output of our approach on the healthcare scenario, and provides empirical benchmarking of the scaling performance.
Table 4 illustrates the first decomposition generated by Algorithm 7 for our formalisation of the healthcare scenario of Sect. 2. For each step in the generated plan, we provide a brief explanation of its meaning within the scenario.
Figure 8 shows a partial decomposition tree of the plans that our approach generates for the healthcare scenario. The nodes in the tree represent nonprimitive tasks (and their corresponding decomposition methods) or operators (when prefixed by the exclamation mark). The solid edges represent the decomposition of a task into other tasks or operators. When two or more edges come out of a box, they are interpreted as alternatives, indicating an OR decomposition. The dashed edge indicates the computed probability and utility of the predecessor node.
The algorithm of Fig. 7 decomposes the toplevel domainspecific task protocol into (generic) tasks such as entice, detach, and satisfy. These tasks can be further decomposed (e.g., entice(G1 (alice) C1 (alice) bob alice) is decomposed into create(C1 bob alice (alice)) and detach(G2 (alice) C1 (alice) bob alice), and to consider(G2 G2 alice (alice))). Operator performImaging(patient, physician, radiologist) has two possible outcomes: (1) success, if radiologist successfully carries out the imaging test with probability 0.7 and utility 10, and (2) failure, if radiologist does not generate a definite imaging exam, with probability 0.3 and utility 0.
Formalisation of the commitment protocol of Sect. 2 to be validated in the experiments
Plan step  Description 

!consider(G1 G1 bob (alice))  Physician considers and activates G1 
!activate(G1 G1 bob (alice))  
entice(G1 G1 (alice) C1 C1 (alice) bob alice)  Physician employs ENTICE rule to create C1 
detach(G2 G2 (alice) C1 C1 (alice) bob alice)  Patient employs DETACH for C1 which results in considering and activating of G2 
!requestAssessment(alice bob)  Patient requests assessment to physician 
!consider(G3 G3 clyde (alice))  Radiologist considers and activates G3 
!activate(G3 G3 clyde (alice))  
entice(G3 G3 (alice) C4 C4 (alice) clyde bob)  Radiologist employs ENTICE for G3 to create C4 
detach(G4 G4 (alice) C4 C4 (alice) clyde bob)  Physician employs DETACH for C4 to create G4 
!requestImaging(bob alice clyde)  Physician requests imaging and books appointment 
deliver(G6 G6 (alice) C2 C2 (clyde) alice bob)  Patient employs DELIVER for C2 to consider and activate goal G6 
attendTest(alice)  Patient brings about iAppointmentKept 
deliver(G7 G7 (alice) C4 C4 (alice) clyde bob)  Radiologist employs DELIVER for C4 to consider and activate goal G7 
!requestRadiologyReport(bob clyde alice)  Physician requests radiology report 
!sendRadiologyReport(clyde bob alice)  Radiologist brings about imagingResultsReported 
!consider(G8 G8 clyde (alice) )  Radiologist considers and activates goal G8 
!activate(G8 G8 clyde (alice) )  
entice(G8 G8 (alice) C5 C5 (alice) clyde bob)  Radiologist employs ENTICE for G8 to create commitment C5 
detach(G9 G9 (alice) C5 C5 (alice) clyde bob)  Physician employs DETACH for C5 to consider and activate goal G9 
!requestBiopsy(bob alice clyde)  Physician requests biopsy and books appointment 
deliver(G11 G11 (alice) C3 C3 (clyde) alice bob)  Patient employs DELIVER for C3 to consider and activate goal G11 
!performBiopsy(clyde alice bob)  Patient keeps the appointment 
!consider(G12 G12 doug (alice) )  Pathologist considers and activates goal G12 
!activate(G12 G12 doug (alice) )  
entice(G12 G12 (alice) C6 C6 (alice) doug clyde)  Pathologist employs ENTICE for G12 to create commitment C6 
detach(G13 G13 (alice) C6 C6 (alice) doug clyde)  Radiologist employs DETACH for C6 to consider and activate goal G13 
!performBiopsy(clyde alice bob)  Radiologist provides a tissue sample 
!requestPathologyReport(bob clyde doug alice)  and requests a pathology assessment 
deliver(G15 G15 (alice) C6 C6 (alice) doug clyde)  Pathologist employs DELIVER for C6 to consider and activate goal G15 
!sendPathologyReport(clyde bob doug alice)  Pathologist reports in results 
deliver(G16 G16 (alice) C5 C5 (alice) clyde bob)  Radiologist employs DELIVER for C5 to consider and activate goal G16 
!sendRadiologyReport(clyde bob alice)  Radiologist reports results 
!sendIntegratedReport(clyde doug alice bob)  
!consider(G17 G17 evelyn (alice) )  Registrar considers and activates goal 
!activate(G17 G17 evelyn (alice) )  
entice(G17 G17 (alice) C7 C7 (alice) evelyn doug)  Registrar employs ENTICE for G17 to create commitment C7 
detach(G18 G18 (doug) C7 C7 (alice) evelyn doug)  Pathologist employs DETACH rule if patient has cancer to consider and activate goal G18 
!reportPatient(alice doug evelyn)  Pathologist reports patient to registrar 
deliver(G19 G19 (alice) C7 C7 (alice) evelyn doug)  Registrar employs DELIVER rule for C7 to consider and activate goal G19 
!addPatientToRegistry(alice evelyn)  Registrar adds patient to registry 
We first implemented the algorithm of Fig. 7 in Python and benchmarked it on a 2.4 GHz MacBook Pro with 16 GB of memory running Mac OSX 10.11. This implementation solves to optimality a planning problem in a simplification of the healthcare scenario in 30 ms. This simplified version of the scenario has one patient (Alice), one doctor (Bob), and one insurance company (Ins). In order to empirically evaluate the complexity of our implementation in comparison to the abstract algorithm, we have run our implementation of NDPyHop on a scalable and scaled up version of the healthcare scenario. In this version, we generate larger scenarios with several patients, physicians, and medical staff.
The results of our scalability test are summarised as Fig. 9. The runtimes and memory usage shown are averages over 10 runs (with negligible variance) for each factor of domain growth. As our complexity analysis predicted, our algorithm runs in exponential time taking up exponential memory as the search tree expands. Nevertheless, runtime to decide realisability is substantially lower, even though memory usage is asymptotically the same (assuming that HTN expansion is depth first). Our empirical experimentation shows an exponential runtime growth: this growth is on the order of \(2^{0.79n}\) for acceptability and on the order of \(2^{1.03n}\) for memory usage, where n is the number of sets of three agents for our empirical scenario. Thus, although the runtime is exponential, the scenario we use for testing at the largest end of the scale has a substantial number of commitments and goals—totalling 144 goals and commitments for 13 agents—running in 15 min while consuming less than 10 GB of main memory.
8 Related work

Centralised planning one mediating planning agent plans the realisation of an entire commitment protocol for all agents. There are two subcases: (1) Cooperative setting: Other agents must or have already agreed to accept the plan (e.g., the members of a police team have agreed to accept the plan of the team leader); (2) Semi/noncooperative setting: Other agents can negotiate about and accept, reject, or ignore the plan (e.g., friends arranging a holiday). Even in the cooperative, centralised case, the planning agent needs to account for uncertainty over agents’ behaviours. The planning agent attempts to find common plans that, presumably, maximise social utility in some fashion [24, 66].

(Fully) decentralised planning each agent plans for itself (e.g., owners of restaurant franchises). A common (joint) overall plan might be sought or not [9, 10]. The multiple individual planning agents need to account for the uncertainty on thirdparty agents fulfilling a commitment.

Joint planning (a subset of) the agents plan together to make a common (partial) plan (e.g., husband and wife coordinating childcare). For a common plan, there must be some set of common goals, at least pairwise between some agents. SharedPlans [33] is an example of a structured joint planning protocol.
Crosby et al. [24] address multiagent planning with concurrent actions. As in our approach, Crosby et al. propose to transform the planning problem into a singleagent problem. Our work does not explicitly plan for concurrent actions and their constraints, but treats coordination through commitments.
Our planning approach is hierarchical. A number of authors consider what can be characterised as probabilistic HTN planning, i.e., HTN planning with uncertain action success. Prominent among them are Kuter et al. [42], who use probabilities directly in their HTN planner YoYo; Kuter and Nau [43], whose HTN planner NDSHOP2 accounts for nondeterministic action outcomes; and, Bouguerra and Karlsson [14], whose planner CSHOP extends the classical HTN planner SHOP with stochastic action outcomes and also belief states.
At first we attempted to employ NDSHOP2 directly for commitment protocol planning and found that its representation of stochastic action outcomes, which assumes a uniform probability distribution, would be difficult to extend. Instead, our algorithm NDPyHop adapts the NDSHOP2 algorithm to reason about probabilities and utilities. We also considered CSHOP due to its explicit handling of outcome probabilities and partial observability—even though our domain assumptions support perfect observability. However, CSHOP does not handle utility functions and decisiontheoretic optimisation. Thus, although our algorithm is not based on CSHOP, it can be seen, from the conceptual point of view, as a decisiontheoretic extension of CSHOP.
Other approaches rely upon decisiontheoretic models such as Markov Decision Processes (MDPs) to add probabilistic reasoning capabilities to their planning process. Kun et al. [41] use an MDP policy to guide the search for possible task decompositions in HTN planning. More specifically, the approach generates an MDP from the possible HTN decompositions, and the solution to this MDP guides the search to an optimal HTN plan. However, the primitive operators (i.e., the underlying planning problem) remain deterministic.
Meneguzzi et al. [49] and Tang et al. [61] explore the use of HTN representations to bound the search space for probabilistic planning domains. Whereas Meneguzzi et al. [49] use an HTN representation to generate a reducedsize MDP containing only the states that are reachable from the possible task decompositions, Tang et al. [61] use the grammar structure of the HTN to plan using a variation of the Earley parsing algorithm generating a result that is analogous to solving the MDP induced by Meneguzzi et al. [49].
In another work, Kuter and Nau [44] use the search control of a classical HTN planner (SHOP2, a predecessor of NDSHOP2 that has no uncertainty) in MDP planning, whereas in an orthogonal direction, Sohrabi et al. [59] add preferences to SHOP2.
None of these works consider social commitments in their planning, and, importantly for our purpose, none of these works enable evaluating commitment protocol enactments individually to optimise over multiple criteria. By contrast, the algorithm of Fig. 7 can easily be used to optimise over any criteria stored in a plan simply by changing the optimisation criteria used in the main algorithm in Lines 19–22.
Decentralized and joint planning Maliah et al. [46] are concerned with multiagent planning when privacy is important. The authors present a planning algorithm designed to preserve the privacy of certain private information. The agents collaboratively generate an abstract, approximate global coordination plan, and then individually extend the global plan to executable individual plans. Commitments are not considered.
The multiagent planning approach of Mouaddib et al. [52] is characteristic of decisiontheoretic approaches based on MDPs for multiagent planning under uncertainty. Each agent plans individually, with its objective function modified to account in some way for the constraints, preferences, or objectives of other agents. Specifically, Mouaddib et al. adopt the concept of social welfare orderings from economics, and use Egalitarian Social Welfare orderings so that in each agent’s local optimisation (its MDP), consideration is given to the satisfaction of all criteria and minimisation of differences among them.
The TÆMS framework [45] models structural and quantitative aspects of interdependent tasks among a set of agents. It features a notion of commitment. TÆMS provides a means of decentralised planning and coordination among a group of agents. Similar to the decomposition tree of HTN formalism, the task structure in TÆMS is hierarchical. At the highest level of its hierarchical task structure are task groups that represent an agent’s goals. A task group is decomposed into a set of tasks (representing subgoals) and methods that cannot be decomposed any further. Tasks and methods have various annotations that specify aspects such as conjunctive or disjunctive decomposition, and temporal dependence with respect to other tasks and methods, and quantitative attributes such as expected execution time distribution, quality distribution, and resource usage. In particular, TÆMS supports probability distributions over the quality, duration, and cost of methods.
The notion of interagent ‘commitment’ in TÆMS is of a promise by one agent to another that it will complete a task within a given time and quality distribution. Unlike our definition, there is no concept of antecedent; TÆMS rather uses commitments primarily to “represent the quantitative aspects of negotiation and coordination” [45]. Also, TÆMS is not geared toward autonomy and it is possible for one agent to send a commitment to another, making the recipient responsible for it. The expressive power of the full TÆMS framework has led to simplifications for practical applications [13, 45, 67]. Applications have not included commitment protocol planning in an expressive setting, such as ours.
Xuan and Lesser [68] introduce a rich model of uncertainty into TÆMS commitments. They assume cooperative, utilitymaximising agents. Like us, these authors develop a contingency planning approach; they consider distributions over commitment outcomes, contingency analysis, and a marginal cost/loss calculation. Although not accommodating goalcommitment convergence or firstorder tasks, the authors present a negotiation framework.
Witwicki and Durfee [67] aim to use commitments to approximate the multiagent policy coordination problem. In a Dec(PO)MDP context, these authors decompose the coordination problem into subproblems, and connect the subproblems with commitments. Our work differs from MDPbased approaches in aiming for a contingent plan (a commitment protocol) rather than an MDP policy. We chose to use an HTN planner and represent the resulting protocols as contingency plans because policies in MDPs need to inherently follow the Markov assumption, which does not allow for sequences of operators from any given state to be represented unless time is explicitly accounted for in the state representation. Adding explicit state variables to the state representation, however, quickly results in the curse of dimensionality that afflicts MDP solvers. Thus, even the most advanced factored MDP solvers cannot deal with temporal variables since they rely on twostep dynamic Bayesian networks, which maintain the Markov assumption [34]. In further comparison with Witwicki and Durfee [67], our work accommodates firstorder commitments and a semantics of goals and commitments.
Commitment protocols Baldoni et al. [9] study the problem of distributed multiagent planning with interagent coordination via commitments. In contrast to our position, they consider the case of decentralised planning. In addition, they consider an open system in which agents can leave or enter the system dynamically.
Günay et al. [36] also consider open systems but seek a protocol rather than a fullyspecified plan. These authors develop a framework to allow a group of agents to create a commitment protocol dynamically. The first phase has a centralised planning agent generate candidate protocols. The second phase has all agents rank the protocols. The third phase involves negotiation over the protocols to select one. In contrast to our work, Günay et al. assume the planning agent does not necessarily know the preferences of other agents, do not include an explicit model of uncertainty, and do not have the foundation of the commitmentgoal semantics.
Chopra et al. [18] present a requirements engineering approach to the specification of sociotechnical systems, founded on commitments to describe agent interactions and goals to describe agent objectives. The methodology, Protos, pursues refinements that seek to satisfy stakeholder requirements by incrementally expanding specification and assumption sets, and reducing requirements until all requirements are accommodated. Our approach and tools can be readily used to validate the executability of the resulting protocols in a computational platform.
Recent commitmentquery languages by these authors—Cupid [21] and Custard [22]—go beyond propositional constructs in commitments but do not address the planning challenges addressed in this article.
Baldoni et al. [7], similarly, are interested in the specification of commitmentbased interaction protocols. These authors propose a definition which decouples the constitutive and regulative specifications, with the latter being explicitly represented based on constraints among commitments.
Günay et al. [35] share a similar spirit to our work, in that they too are motivated to analyse agents behaviours in commitment protocols. Their ProMoca framework models agent beliefs, goals, and commitments. It models uncertainty over action outcomes, as we do, through probability distributions over (agents’ beliefs about) action outcomes. Günay et al. adopt probabilistic model checking to analyse a commitment protocol for compliance and for goal satisfaction. Although the work of Günay et al. features a more expressive language for stating commitments, our work differs in that our approach can develop (new) commitment protocols, not just analyse existing protocols. We adopt probabilistic HTN planning rather than probabilistic model checking.
Like Günay et al., Venkatraman and Singh [64] and subsequent works also use model checking to perform verification of commitment protocols, but for static verification of properties; we go beyond checking to protocol generation, and we accommodate goals and uncertainty. Bataineh et al. [11] present a recent approach for specification and verification of service composition contracts, again using model checking to verify commitmentbased properties but, in this case, properties that are derived automatically from the service compositions. Uncertainty is not addressed.
Sultan et al. [60] use model checking to verifying probabilistic commitments. These authors develop a logicbased specification of probabilistic commitments, in which the degree of agent’s commitment (“how much an agent is confident about its commitment”) and the degree of fulfillment (“how much the agent is confident about fulfilling a commitment”) can be expressed. This contrasts with our probabilistic planning approach which also accommodates an explicit representation of goals and firstorder operators.
Chesani et al. [17] study a different problem to commitmentbased plan or protocol generation: monitoring commitments. Their framework captures commitments and their execution using a firstorder Event Calculus representation.
As with our work, Baldoni et al. [4, 5, 6] use the commitmentgoal semantics of Telang et al. [63] as their starting point. These authors present an extension to the normand artefactaware agent programming language, JaCaMo, in which Jason agents can interact while preserving their deliberative capabilities by exploiting commitmentbased protocols.
Lastly, we explain how this article builds on our previous work. First, we established a framework for the coherence between an agent’s commitments and goals [63].^{10} Second, we used HTN planning to enact protocols over this framework [62]. Third, we extended the representation and planning approach to operate over first order commitments and goals [50]. Fourth, in a brief paper [51] we allowed uncertainty in action outcomes. In this article we draw together and extended this line of work. We retain the semantics of Telang et al. [63] as our basis, and, as has been discussed, exploit an HTN planner to develop protocols, as in Telang et al. [62]. We present a formalism that accommodates both a firstorder representation and uncertainty, and a planning algorithm in the case of a centralised planning agent.
In our other previous work, as detailed above, Meneguzzi et al. [49] and Tang et al. [61] use HTN representations to bound the search space for probabilistic planning domains; they do not consider commitments. More broadly, Meneguzzi and de Silva [48] survey how planning algorithms are used in agent reasoning.
9 Conclusion
We addressed the challenge of planning coordinated activities for a set of autonomous agents, in the form of flexible commitment protocols. We considered an expressive firstorder setting with probabilistic uncertainty over action outcomes; these probabilities can originate from historical data, be learned dynamically, or drawn from trust and reputation. We contributed the first practical means to derive protocol enactments which maximise expected utility from the point of view of one agent. First, we showed how Hierarchical Task Network (HTN) planning can be used to enact a previous semantics for commitment and goal alignment. Second, we extended that semantics in order to enact firstorder commitment protocols. Third, supposing a cooperative setting, we introduced uncertainty in order to capture the reality that an agent does not know for sure that its partners will successfully act on their parts of the commitment protocol.
We illustrated our approach on a realworld healthcare scenario, producing protocols, where possible, with a desired level of robustness in terms of the probability of success. We empirically show that these protocols can be generated efficiently for a number of scenarios using different implementations of the same basic algorithm.
Distributed coordinated multiagent planning is a broad topic and we have not touched on many interesting aspects [15]; we mention a few here. First, we support modelling different outcome probabilities for different agents executing the same action. Doing so opens up the possibility of delegating tasks to agents that are more likely to succeed. Similarly, agents may have different opinions of the utility of actions and plans—we have not touched on the negotiation process to arrive on common utilities for the planning problem. Second, we consider a single configuration of the agent system: the single mediating planning agent. Third, we consider commitment protocols founded on goals of achievement to the exclusion of goals of maintenance.
The future scenario to be considered, ultimately, involves an agent reasoning about commitments in which it participates with multiple other agents; this draws towards multiagent HTN planning [13, 45]. The relevant technical challenges include some studied in the literature (negotiation, plan fragment merging and coordination, and information exchange) and some that arise because of the commitments. An example challenge is representing temporal gaps between state changes of commitments, i.e., an agent might satisfy the antecedent of a commitment, but may need to wait some unspecified time for the consequent to be fulfilled.
Footnotes
 1.
This assignment is arbitrary; female and male can be swapped among the medical professionals.
 2.
Throughout the remainder, we refer to commitment instances simply as commitments.
 3.
The pending state is achieved by adding a single predicate via an action.
 4.
Throughout the remainder, we refer to goal instances simply as goals.
 5.
Note that, like for the commitment dynamics, the suspension of a goal is the result of adding a single predicate via an action that fully encodes this state.
 6.
We provide the full implementation of this formalisation for reference at www.github.com/meneguzzi/htngoco/, specifically in the healthcare.jshop file. We also provide the formalisation that underpins that implementation in Appendixes B and C.
 7.
 8.
 9.
 10.
An extended version of this commitment formalism is under review at the time of writing.
 11.
Note that eqGSCP is a logical rule to ensure that a goal’s success condition equals a commitment’s antecedent.
Notes
Acknowledgements
We gratefully thank those who shared their code with us. Special thanks to Ugur Kuter. We thank the anonymous reviewers, and also acknowledge with gratitude the reviewers at ProMAS’11, AAMAS’13, AAAI’13, and AAMAS’15, where preliminary parts of this work appeared. FM thanks the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) for the support within process numbers 306864/20134 under the PQ fellowship and 482156/20139 under the Universal project programs. NYS acknowledges support of the AUB University Research Board Grant Number 102853 and the OSB Grant OFFER_C1_2013_2014.
References
 1.Alford, R., Bercher, P., & Aha, D. W. (2015). Tight bounds for HTN planning. In Proceedings of ICAPS’15 (pp. 7–15).Google Scholar
 2.Apt, K. R. (1997). From logic programming to Prolog. Upper Saddle River, NJ: PrenticeHall.Google Scholar
 3.ASPE: The importance of radiology and pathology communication in the diagnosis and staging of cancer: Mammography as a case study (2010). Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services. http://aspe.hhs.gov/sp/reports/2010/PathRad/index.shtml.
 4.Baldoni, M., Baroglio, C., Capuzzimati, F. & Micalizio, R. (2015). Exploiting social commitments in programming agent interaction. In Proceedings of PRIMA’15 (pp. 566–574).Google Scholar
 5.Baldoni, M., Baroglio, C., Capuzzimati, F., & Micalizio, R. (2015). Leveraging commitments and goals in agent interaction. In Proceedings of 30th Italian conference on computational logic (CILC’15) (pp. 85–100).Google Scholar
 6.Baldoni, M., Baroglio, C., Capuzzimati, F., & Micalizio, R. (2015). Programming with commitments and goals in JaCaMo+. In Proceedings of AAMAS’15 (pp. 1705–1706).Google Scholar
 7.Baldoni, M., Baroglio, C., Chopra, A. K., & Singh, M. P. (2015). Composing and verifying commitmentbased multiagent protocols. In Proceedings of IJCAI’15 (pp. 10–17).Google Scholar
 8.Baldoni, M., Baroglio, C., Marengo, E., Patti, V., & Capuzzimati, F. (2014). Engineering commitmentbased business protocols with the 2CL methodology. Autonomous Agents and MultiAgent Systems, 28(4), 519–557.CrossRefGoogle Scholar
 9.Baldoni, M., Baroglio, C., & Micalizio, R. (2015). Social continual planning in open multiagent systems: A first study. In Proceedings of PRIMA’15 (pp. 575–584).Google Scholar
 10.Barbulescu, L., Rubinstein, Z. B., Smith, S. F., & Zimmerman, T. L. (2010). Distributed coordination of mobile agent teams: The advantage of planning ahead. In Proceedings of AAMAS’10 (pp. 1331–1338) .Google Scholar
 11.Bataineh, A. S., Bentahar, J., ElMenshawy, M., & Dssouli, R. (2017). Specifying and verifying contractdriven service compositions using commitments and model checking. Expert Systems with Applications, 74, 151–184.CrossRefGoogle Scholar
 12.Bellman, R. (1957). A markov decision process. Journal of Mathematical Mechanics, 6, 679–684.zbMATHGoogle Scholar
 13.Boddy, M., Horling, B., Phelps, J., Goldman, R., Vincent, R., Long, A., et al. (2007). cTÆMS language specification. Technical report, Adventium Labs.Google Scholar
 14.Bouguerra, A., & Karlsson, L. (2004). Hierarchical task planning under uncertainty. In 3rd Italian workshop on planning and scheduling (AI*IA’04), Perugia, Italy.Google Scholar
 15.Brafman, R. I., & Domshlak, C. (2008). From one to many: Planning for loosely coupled multiagent systems. In Proceedings of ICAPS’08 (pp. 28–35).Google Scholar
 16.Castelfranchi, C. (1995). Commitments: From individual intentions to groups and organizations. In Proceedings of ICMAS’95 (pp. 41–48).Google Scholar
 17.Chesani, F., Mello, P., Montali, M., & Torroni, P. (2013). Representing and monitoring social commitments using the event calculus. Autonomous Agents and MultiAgent Systems, 27(1), 85–130.CrossRefGoogle Scholar
 18.Chopra, A. K., Dalpiaz, F., Aydemir, F. B., Giorgini, P., Mylopoulos, J., & Singh, M. P. (2014). Protos: Foundations for engineering innovative sociotechnical systems. In Proceedings of 22nd international requirements engineering conference (RE’14) (pp. 53–62).Google Scholar
 19.Chopra, A. K., Dalpiaz, F., Giorgini, P., & Mylopoulos, J. (2010). Modeling and reasoning about serviceoriented applications via goals and commitments. In Proceedings of 22nd international conference on advanced information systems engineering (CAiSE’10) (pp. 417–421).Google Scholar
 20.Chopra, A. K., Dalpiaz, F., Giorgini, P., & Mylopoulos, J. (2010). Reasoning about agents and protocols via goals and commitments. In Proceedings of AAMAS’10 (pp. 457–464).Google Scholar
 21.Chopra, A. K., & Singh, M. P. (2015). Cupid: Commitments in relational algebra. In Proceedings of 29th conference on artificial intelligence (AAAI’15) (pp. 2052–2059).Google Scholar
 22.Chopra, A. K., & Singh, M. P. (2016). Custard: Computing norm states over information stores. In Proceedings of AAMAS’16 (pp. 1096–1105).Google Scholar
 23.Chopra, A. K., & Singh, M. P. (2016). From social machines to social protocols: Software engineering foundations for sociotechnical systems. In Proceedings of WWW’16 (pp. 903–914).Google Scholar
 24.Crosby, M., Jonsson, A., & Rovatsos, M. (2014). A singleagent approach to multiagent planning. In Proceedings of ECAI’14 (pp. 237–242).Google Scholar
 25.Dastani, M., van der Torre, L., & YorkeSmith, N. (2017). Commitments and interaction norms in organisations. Autonomous Agents and MultiAgent Systems, 31(2), 207–249.CrossRefGoogle Scholar
 26.Della Penna, G., Intrigila, B., & Magazzeni, D. (2015). UPMurphi Released: PDDL+ Planning for hybrid systems. In Proceedings of 2nd ICAPS workshop on model checking and automated planning (pp. 35–39).Google Scholar
 27.Desai, N., Chopra, A. K., & Singh, M. P. (2009). Amoeba: A methodology for modeling and evolution of crossorganizational business processes. ACM Transactions on Software Engineering and Methodology (TOSEM), 19(2), 6:1–6:45.CrossRefGoogle Scholar
 28.Durfee, E. (2001). Distributed problem solving and planning. In Multiagent systems and applications, LNCS (Vol. 2086, pp. 118–149). Springer, New York, NY.Google Scholar
 29.Erol, K., Hendler, J., & Nau, D. S. (1994). HTN planning: Complexity and expressivity. In Proceedings of AAAI’94 (Vol. 2, pp. 1123–1128).Google Scholar
 30.Erol, K., Nau, D. S., & Subrahmanian, V. S. (1995). Complexity, decidability and undecidability results for domainindependent planning. Artificial Intelligence, 76(1–2), 75–88.MathSciNetCrossRefzbMATHGoogle Scholar
 31.Fornara, N., & Colombetti, M. (2009). Ontology and time evolution of obligations and prohibitions using semantic web technology. In Declarative agent languages and technologies VII, LNCS (Vol. 5948, pp. 101–118). Springer, New York, NY.Google Scholar
 32.Ghallab, M., Nau, D., & Traverso, P. (2004). Automated planning: Theory and practice. Burlington, MA: Elsevier.zbMATHGoogle Scholar
 33.Grosz, B. J., & Hunsberger, L. (2006). The dynamics of intention in collaborative activity. Cognitive Systems Research, 7(2–3), 259–272.CrossRefGoogle Scholar
 34.Guestrin, C., Koller, D., Parr, R., & Venkataraman, S. (2003). Efficient solution algorithms for factored MDPs. Journal of Artificial Intelligence Research, 19(10), 399–468.MathSciNetzbMATHGoogle Scholar
 35.Günay, A., Liu, Y., & Zhang, J. (2016). Promoca: Probabilistic modeling and analysis of agents in commitment protocols. Journal of Artificial Intelligence Research, 57, 465–508.MathSciNetzbMATHGoogle Scholar
 36.Günay, A., Winikoff, M., & Yolum, P. (2015). Dynamically generated commitment protocols in open systems. Autonomous Agents and MultiAgent Systems, 29(2), 192–229.CrossRefGoogle Scholar
 37.Harland, J., Morley, D. N., Thangarajah, J., & YorkeSmith, N. (2017). Aborting, suspending, and resuming goals and plans in BDI agents. Autonomous Agents and MultiAgent Systems, 31(2), 288–331.CrossRefzbMATHGoogle Scholar
 38.Hoffmann, J., Weber, I., & Kraft, F. M. (2010). SAP speaks PDDL. In Proceedings of AAAI’10.Google Scholar
 39.Ilghami, O., & Nau, D. S. (2003). A general approach to synthesize problemspecific planners. Technical report, University of Maryland.Google Scholar
 40.King, N. (2014). Expanding the boundaries of clinical informatics for interdisciplinary systems research. Health Systems, 3(1), 1–11.CrossRefGoogle Scholar
 41.Kun, C., Xu, J., & ReiffMarganiec, S. (2009). MarkovHTN planning approach to enhance flexibility of automatic web service composition. In Proceedings of international conference on web services (ICWS’09) (pp. 9–16).Google Scholar
 42.Kuter, U., Nau, D., Pistore, M., & Traverso, P. (2009). Task decomposition on abstract states, for planning under nondeterminism. Artificial Intelligence, 173(5–6), 669–695.MathSciNetCrossRefzbMATHGoogle Scholar
 43.Kuter, U., & Nau, D. S. (2004). Forwardchaining planning in nondeterministic domains. In Proceedings of AAAI’04 (pp. 513–518).Google Scholar
 44.Kuter, U., & Nau, D. S. (2005). Using domainconfigurable search control for probabilistic planning. In Proceedings of AAAI’05 (pp. 1169–1174).Google Scholar
 45.Lesser, V. R., Decker, K., Wagner, T., Carver, N., Garvey, A., Horling, B., et al. (2004). Evolution of the GPGP/TÆMS domainindependent coordination framework. Autonomous Agents and MultiAgent Systems, 9(1–2), 87–143.CrossRefGoogle Scholar
 46.Maliah, S., Shani, G., & Stern, R. (2017). Collaborative privacy preserving multiagent planning—Planners and heuristics. Autonomous Agents and MultiAgent Systems, 31(3), 493–530.CrossRefGoogle Scholar
 47.Meneguzzi, F., Magnaguagno, M., Telang, P., Singh, M. P., & YorkeSmith, N. (2017). meneguzzi/htngoco: HTN goal commitment dynamics. https://doi.org/10.5281/zenodo.845435.
 48.Meneguzzi, F., & de Silva, L. (2015). Planning in BDI agents: A survey of the integration of planning algorithms and agent reasoning. Knowledge Engineering Review, 30(1), 1–44.CrossRefGoogle Scholar
 49.Meneguzzi, F., Tang, Y., Sycara, K., & Parsons, S. (2011). An approach to generate MDPs using HTN representations. In Proceedings of IJCAI’11 workshop on decision making in partially observable, uncertain worlds.Google Scholar
 50.Meneguzzi, F., Telang, P. R., & Singh, M. P. (2013). A firstorder formalization of commitments and goals for planning. In Proceedings of AAAI’13 (pp. 1–8).Google Scholar
 51.Meneguzzi, F., Telang, P. R., & YorkeSmith, N. (2015). Towards planning uncertain commitment protocols. In Proceedings of AAMAS’15 (pp. 1681–1682).Google Scholar
 52.Mouaddib, A. I., Boussard, M., & Bouzid, M. (2007). Towards a formal framework for multiobjective multiagent planning. In Proceedings of AAMAS’07.Google Scholar
 53.Pistore, M., Marconi, A., Bertoli, P., & Traverso, P. (2005). Automated composition of web services by planning at the knowledge level. In Proceedings of IJCAI’05 (pp. 1252–1259).Google Scholar
 54.Singh, M. P. (1991). Social and psychological commitments in multiagent systems. In AAAI fall symposium on knowledge and action at social and organizational levels (pp. 104–106).Google Scholar
 55.Singh, M. P. (1999). An ontology for commitments in multiagent systems. AI and Law, 7, 97–113.Google Scholar
 56.Singh, M. P. (2008). Semantical considerations on dialectical and practical commitments. In Proceedings of AAAI’08 (pp. 176–181).Google Scholar
 57.Singh, M. P. (2012). Commitments in multiagent systems. In The goals of cognition: Essays in Honor of Cristiano Castelfranchi (pp. 601–626). College Publications, London, UK.Google Scholar
 58.Singh, M. P. (2012). Norms as a basis for governing sociotechnical systems. ACM Transactions on Intelligent Systems and Technology (TIST), 5(1), 21:1–21:23.Google Scholar
 59.Sohrabi, S., Baier, J. A., & McIlraith, S. A. (2009). HTN planning with preferences. In Proceedings of IJCAI’09 (pp. 1790 –1797).Google Scholar
 60.Sultan, K., Bentahar, J., & ElMenshawy, M. (2014). Model checking probabilistic social commitments for intelligent agent communication. Applied Soft Computing, 22, 397–409.CrossRefGoogle Scholar
 61.Tang, Y., Meneguzzi, F., Sycara, K., & Parsons, S. (2011). Probabilistic hierarchical planning over MDPs. In Proceedings of AAMAS’11 (pp. 1143–1144).Google Scholar
 62.Telang, P. R., Meneguzzi, F., & Singh, M. P. (2013). Hierarchical planning about goals and commitments. In Proceedings of AAMAS’13 (pp. 761–768).Google Scholar
 63.Telang, P. R., YorkeSmith, N., & Singh, M. P. (2012). Relating goal and commitment semantics. In Proceedings of 9th international workshop on programming multiagent systems, LNCS (Vol. 7217, pp. 22–37). Springer, New York, NY.Google Scholar
 64.Venkatraman, M., & Singh, M. P. (1999). Verifying compliance with commitment protocols. Autonomous Agents and MultiAgent Systems, 2(3), 217–236.CrossRefGoogle Scholar
 65.Verdicchio, M., & Colombetti, M. (2002). Commitments for agentbased supply chain management. SIGecom Exchanges, 3(1), 13–23.CrossRefGoogle Scholar
 66.de Weerdt, M., & Clement, B. (2009). Introduction to planning in multiagent systems. Multiagent and Grid Systems, 5, 345–355.CrossRefGoogle Scholar
 67.Witwicki, S. J., & Durfee, E. H. (2007). Commitmentdriven distributed joint policy search. In Proceedings of AAMAS’07 (pp. 492–499).Google Scholar
 68.Xuan, P., & Lesser, V. R. (2000). Incorporating uncertainty in agent commitments. In Intelligent agents VI: Agent theories, architectures, and languages, LNCS (Vol. 1757, pp. 57–70). Springer, New York, NY.Google Scholar
 69.Yolum, P., & Singh, M. P. (2002). Flexible protocol specification and execution: Applying event calculus planning using commitments. In Proceedings of AAMAS’02 (pp. 527–534).Google Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.