The OMLP family of optimal multiprocessor real-time locking protocols

Abstract

This paper presents the first suspension-based multiprocessor real-time locking protocols with asymptotically optimal blocking bounds (under certain analysis assumptions). These protocols can be applied under any global, clustered, or partitioned job-level fixed-priority scheduler and support mutual exclusion, reader-writer exclusion, and k-exclusion constraints. Notably, the reader-writer and k-exclusion protocols are the first analytically-sound suspension-based multiprocessor real-time locking protocols of their kind. To formalize a notion of “optimal blocking,” precise definitions of what constitutes “blocking” in a multiprocessor real-time system are given and a simple complexity metric for real-time locking protocols, called maximum priority-inversion blocking (pi-blocking), is introduced. It is shown that, in a system with m processors, Ω(m) maximum pi-blocking is unavoidable. This bound is shown to be asymptotically tight with the introduction of the O(m) multiprocessor locking protocol (OMLP) family presented herein, which includes protocols that ensure an upper bound on maximum pi-blocking that is approximately within a factor of two of the lower bound. In addition to the coarse-grained asymptotic bounds, detailed blocking bounds suitable for schedulability analysis are derived using holistic blocking analysis. Based on the detailed bounds, the proposed locking protocols are compared with each other and with previously-proposed protocols in an empirical schedulability study involving more than one billion task sets. In this study, the OMLP was found to perform better than two variants of the classic (but non-optimal) multiprocessor priority-ceiling protocol (MPCP).

Appendix: Schedulability analysis

In this appendix, we first introduce a generic framework for expressing bounds on pi-blocking and then apply it to bound pi-blocking under each of the locking protocols presented in this paper. The blocking analysis presented in the following is essential for deriving safe blocking bounds suitable for schedulability analysis. However, such bounds tend to be somewhat technical in nature and are primarily required only for implementing schedulability tests (as used in the schedulability experiments presented in Sect. 5); the casual reader may safely skip this appendix and consult the overview presented in Sect. 4 instead.

The framework presented in the following is generic in the sense that it is not tied to any particular locking protocol. It serves two purposes. For one, it avoids redundancy in the subsequent analysis of the locking protocols, which have structurally similar blocking bounds. Second, the presented analysis takes a holistic analysis approach to reduce the pessimism inherent in analyzing requests individually. That is, it is intended to be applied to each job as a whole and bounds blocking across all requests that a job issues, instead of bounding delays on a request-by-request basis.

The presented holistic analysis approach was first used to analyze the FMLP under P-FP scheduling [16], and subsequently further developed to analyze RW spinlocks [18]. The version presented herein has been somewhat simplified compared to the previous variants. We next explain the intuition underlying the approach, which we then formalize in Sect. A.2 below.

1.1 A.1 Holistic blocking analysis

In the following, let J _i denote an arbitrary job of the task T _i for which a bound on maximum blocking is being derived. The main idea of the holistic approach is to avoid accounting for any individual possibly-blocking request more than once, and to avoid accounting for requests that cannot possibly interfere with J _i ’s requests. In particular, when a job request the same resource more than once, the holistic approach can avoid substantial pessimism compared to analyzing each resource request in isolation, and especially so if long requests occur much less frequently than short requests (i.e., if there are large differences among the tasks’ L _i,q , N _i,q , and p _i parameters).

Example 9

To illustrate possible pessimism when analyzing requests individually, consider the following scenario (in this and the following examples, the use of the clustered OMLP’s mutex variant is assumed). Suppose a task T _i shares a serially-reusable resource ℓ _q with another task T _x . Further, suppose J _i requests ℓ _q up to N _i,q =20 times and that jobs of T _x hold ℓ _q for at most L _x,q =10 time units. Finally, suppose jobs of T _x require ℓ _q at most once while any J _i is pending. When analyzing each of J _i ’s many requests individually (i.e., when bounding the maximum pi-blocking incurred by a single request), T _x ’s sole interfering request is effectively considered to block each of J _i ’s requests since T _x ’s request might delay any of the requests (but not all at once). Consequently, J _i ’s overall bound on pi-blocking due to requests for ℓ _q would be N _i,q ⋅L _x,q =200 time units, whereas the actual maximum possible delay is only L _x,q =10 time units since, when considering J _i ’s entire execution, J _x obviously delays J _i with at most only one blocking request for ℓ _q , and not with up to 20.

This example demonstrates that maximum contention should be analyzed as a whole across all of J _i ’s requests for a particular resource. (Since we assume that requests for resources are not nested, blocking bounds for individual resources are independent from each other and can be derived individually.) The extent to which J _i is blocked due to requests for a resource ℓ _q in the worst case is limited by the following constraints:

• 1. Maximum number of requests issued by other jobs. As discussed above in Example 9, if jobs of T _x issue at most k requests while any J _i is pending, then J _i will be blocked by at most k requests of jobs of T _x , regardless of the number of requests issued by J _i .

• 2. Maximum number of interfering requests per request issued by J _i . Suppose J _i requests a serially-reusable resource ℓ _q only once, that m=4, and that ℓ _q is requested by other jobs up to k=100 times while J _i is pending. In this case, J _i is delayed by at most m−1=3 competing requests, irrespective of the total number of requests k for ℓ _q since priority donation limits the maximum queue length to m jobs.

• 3. Maximum number of interfering requests per task. For example, suppose ℓ _q is shared among three tasks T _i , T _x , and T _y . If J _i issues only one request, then it is blocked by at most one request from T _x and one request from T _y , irrespective of the total number of requests issued by these tasks, and irrespective of the number of processors. Due to the FIFO ordering in the wait queue FQ _q , each task can precede J _i at most once per request.

• 4. Task locality. For example, suppose T _i shares a resource with tasks T _x and T _y under partitioned scheduling (c=1), and that T _i and T _x are assigned to processor 1, whereas T _y is assigned to processor 2. Jobs of T _y can cause J _i to incur acquisition delay because they can issue conflicting requests while J _i is scheduled. In contrast, jobs of T _x cannot cause J _i to incur acquisition delay because jobs of T _x are not scheduled while J _i is executing; however, a job J _x can cause J _i to incur pi-blocking if J _i must serve as J _x ’s priority donor upon release.

We formalize these four constraints next.

1.2 A.2 Interference sets

We begin with Constraint 1 by bounding the maximum resource requirements of competing tasks. In the task model assumed in this paper, a task T _i ’s resource requirements are characterized by the parameters N _i,q and L _i,q . The main advantages of this model are that it is general enough to reflect many possible job behaviors (e.g., no particular request order or minimum separation of requests is assumed) and that the required information can be obtained as part of worst-case execution time analysis (or empirically bounded if such analysis is not available). However, it is possible that more detailed knowledge is available for specific applications.

For example, it could be the case that jobs of a task T _i access a resource ℓ _q twice, and that the second access is always much shorter than the first access. In this case, using a single upper bound L _i,q for both requests is needlessly pessimistic. A similar concern arises with resources that are not accessed by every job of T _i . For, example to reduce overheads, an application T _a could be programmed to record status information in a shared log ℓ _l only once every five jobs. Assuming that each J _a requests access to ℓ _l would needlessly overestimate contention for ℓ _l . However, explicitly incorporating all such considerations yields a task model that is overly complicated for our goal (which is to study the underlying algorithmic properties of the protocols).

We instead use an abstraction called task interference bound to achieve a separation of concerns between the modeling of resource requirements and the actual analysis of locking protocols, which is structurally independent from model considerations. A task’s interference bound (for non-processor resources) is similar to a demand bound function (for processor time) in that it “upper bounds” a task’s worst-case resource requirement during some interval. The actual blocking analysis is expressed in terms of task interference, which can be defined to take advantage of detailed application-specific resource usage information. The primary benefit of this approach is that derived blocking terms can be reused to derive less pessimistic bounds when additional information in form of a more-detailed task model is available.

In the following, to achieve the desired separation of concerns, we formalize a task’s “interference bound” as a set of requests that safely approximates a task’s “actual contention” for a resource. Recall from Sect. 2.2 that $\mathcal {R}_{i,q,v}$ denotes the vth request for resource ℓ _q issued by any J _i , and that ${\mathcal{L}}_{i,q,v}$ denotes the request length of $\mathcal {R}_{i,q,v}$. This allows us to formalize the concept of a task’s “contention for a resource.”

Definition 6

Suppose jobs of a task T _i execute k resource requests for a resource ℓ _q during an interval [t ₀,t ₁). In a concrete, fixed schedule, the contention due to T _i during [t ₀,t ₁) is the set of requests

$$C_{i,q}(t_0, t_1) \triangleq \{ \mathcal {R}_{i,q,v}, \mathcal {R}_{i,q,(v+1)}, \ldots, \mathcal {R}_{x,q,(v+k-1)} \} $$

such that $\mathcal {R}_{i,q,v}$ is the first request and $\mathcal {R}_{i,q,(v+k-1)}$ is last request issued by any J _i during [t ₀,t ₁).

In general, v and k are unknown prior to the execution of T _i , as is the length of each request in C _i,q (t ₀,t ₁). To enable a priori analysis, a generic notion of worst-case contention is required. The purpose of T _i ’s request interference bound, given next, is to define a set of generic requests (i.e., virtual requests defined for analysis purposes) that upper-bound the worst-case contention during any interval of length t ₁−t ₀. That is, the interference bound for an interval of length t ₁−t ₀ contains at least as many requests as T _i issues in any interval of length t ₁−t ₀ in any actual schedule, and each generic request is at least as long as a corresponding actual one. This can be formalized as follows.

Definition 7

The task interference bound for an interval of length t, denoted tif(T _i ,ℓ _q ,t), is a set of generic requests that satisfies the following two properties.

• 1. For any C _i,q (t ₀,t ₁) with regard to some actual schedule, one can choose a set of corresponding generic requests $C'_{i,q} \subseteq\mathit{tif}(T_{i}, \ell _{q}, t_{1} - t_{0})$ that satisfies

  $$\bigl \vert C'_{i,q} \bigr \vert = \bigl \vert C_{i,q}(t_0, t_1) \bigr \vert \quad \text{and} \quad \sum_{\mathcal {R}_{i,q,v} \in C_{i,q}(t_0, t_1)} {\mathcal{L}}_{i,q,v} \leq\sum _{\mathcal {R}_{i,q,w} \in C'_{i,q}} {\mathcal{L}}_{i,q,w}. $$

• 2. Interference bounds are inclusive:

  $$t \leq t' \quad \Rightarrow\quad \mathit{tif}(T_i, \ell _q, t) \subseteq\mathit {tif}\bigl(T_i, \ell _q, t'\bigr). $$

Property 1 ensures that the task interference bound does not underestimate the number and length of requests in any actual execution of T _i , and Property 2 ensures that a derived bound remains valid when analyzing a larger-than-necessary interval (i.e., when over-estimating a job’s response time). In the case of RW constraints, we analogously define task T _i ’s read interference bound, denoted as rif(T _i ,ℓ _q ,t), with respect to read requests for ℓ _q , and T _i ’s write interference bound, denoted as wif(T _i ,ℓ _q ,t) with respect to write requests for ℓ _q .

These definitions serve as an interface that allows the analysis of specific lock types presented in the following sections to be seamlessly integrated with more-refined task and resource models. Next, we provide a suitable definition of tif(T _i ,ℓ _q ,t) for the model assumed in this paper. To this end, we require the following well-known bound on the maximum number of jobs that can execute requests in a given interval. Recall from Sect. 2 that p _i denotes T _i ’s period and r _i denotes T _i ’s maximum response time.

Lemma 16

At most $\lceil \frac{t + r_{i}}{p_{i}} \rceil$ distinct jobs of a task T _i can execute in any interval of length t (without proof, see e.g. [14, 18]).

It follows from Lemma 16 and the definition of N _i,q that jobs of T _i issue at most ⌈(t+r _i )/p _i ⌉⋅N _i,q requests for ℓ _q over any interval of length t. In the worst case, each request for ℓ _q is of length L _i,q . This yields the following interference bound for the task model assumed herein.

Definition 8

The request interference bound for task T _i with respect to resource ℓ _q over any interval of length t is the set of requests

$$\mathit{tif}(T_i, \ell _q, t) \triangleq \biggl\{ \mathcal {R}_{i,q,v}\ |\ 1 \leq v \leq N_{i,q} \cdot \biggl\lceil \frac{t + r_i}{p_i} \biggr\rceil \biggr\}, $$

where ${\mathcal{L}}_{i,q,v} = L_{i,q}$ for each $\mathcal {R}_{i,q,v}$. If T _i does not access a given resource ℓ _q , then tif(T _i ,ℓ _q ,t)=∅ for all t. We analogously define task T _i ’s read and write interference bounds as rif(T _i ,ℓ _q ,t) and wif(T _i ,ℓ _q ,t), respectively.

Based on per-task interference bounds, we next introduce a generic, parametrized “aggregate interference bound” for use in the subsequent analysis of specific locking protocols. We first define three convenience functions over sets of requests, which serve to simplify the expression of “aggregate interference” and protocol-specific bounds on blocking.

Definition 9

Given a set of requests S, we let S _k denote the kth longest request in S, where 1≤k≤|S| (with ties broken arbitrarily but consistently). Formally, if 1≤k≤l≤|S| and $S_{k} = \mathcal {R}_{a,b,c}$ and $S_{l} = \mathcal {R}_{x,y,z}$, then ${\mathcal{L}}_{a,b,c} \geq {\mathcal{L}}_{x,y,z}$.

Definition 10

Given a set of requests S, we denote the set of the l longest requests in S as top(l,S)≜{S _k | 1≤k≤min(l,|S|)} and their total duration as $\mathit{total}(l, S) \triangleq\sum_{\mathcal {R}_{i,q,v} \in\ \mathit{top}(l, S)} {\mathcal{L}}_{i,q,v}$. If l=0 or S=∅, then total(l,S)=0.

A task interference bound limits the maximum contention from jobs of a single task. Using the above definitions, we can formalize the notion of contention from a set of tasks. Recall Constraint 3 from Sect. A.1 above, namely that the number of requests per task that can possibly cause J _i to incur acquisition delay is limited if jobs wait in FIFO order. If a task T _x can delay J _i with at most l requests, then it is sufficient to consider only the l longest requests in T _x ’s interference bound. We therefore define the aggregate interference bound with a per-task “interference limit” parameter.

Definition 11

The aggregate interference bound of a set of tasks τ with respect to a resource ℓ _q over any interval of length t and subject to an interference limit l is given by

A task set’s aggregate read interference and aggregate write interference, denoted as rifs(τ,ℓ _q ,t,l) and wifs(τ,ℓ _q ,t,l), respectively, are defined analogously with respect to read and write interference.

Given an interference limit l, tifs(τ,ℓ _q ,t,l) contains the l longest requests in each task’s interference bound for ℓ _q and t. In the task model assumed in this paper, each request in tif(T _x ,ℓ _q ,t) is in fact of the same length ${\mathcal{L}}_{x,q,v} = L_{x,q}$ (see Definition 8). We define tifs(τ,ℓ _q ,t,l) with additional generality to accomodate more-expressive task models for which tif(T _x ,ℓ _q ,t) may contain non-uniform request lengths.

The holistic blocking analysis framework incorporates Constraints 1 and 3 from Sect. A.1 in a generic fashion. The remaining Constraints 2 and 4 are easier to incorporate on a protocol-by-protocol basis, which we do next to derive concrete, non-asymptotic bounds for the locking protocols presented in this paper.

1.3 A.3 The global OMLP for mutual exclusion

We begin with the global OMLP for mutex constraints under s-oblivious schedulability analysis. Since the global OMLP uses a hybrid queue that consists of a FIFO queue FQ _q (which holds at most m jobs) and of a priority queue PQ _q (which is only used if at least m+1 jobs are queued), maximum s-oblivious pi-blocking under the global OMLP depends on how many tasks share a given resource.

Definition 12

In the following, let A _q ≜|{T _i | T _i ∈τ∧N _i,q >0}| denote the number of tasks that access resource ℓ _q .

If A _q ≤m+1, then at most m jobs are waiting to acquire ℓ _q at any time, which implies that at most one job is queued in PQ _q . In this case, the global OMLP reduces to a simple FIFO protocol.

Lemma 17

Under the global OMLP, if A _q ≤m+1, then a job J _i incurs at most

s-oblivious pi-blocking due to requests for resource ℓ _q .

Proof

J _i ’s response time r _i upper-bounds the duration of the interval during which other jobs can issue conflicting requests; that is, the aggregate task interference bound tifs(τ∖{T _i },r _i ,N _i,q ) for any interval of length r _i is a sufficient approximation of the resource demands of competing tasks. If J _i is never enqueued in PQ _q , then the lemma follows trivially.

Otherwise, if J _i is enqueued in PQ _q , then m jobs are already enqueued in FQ _q at the time of J _i ’s request. Since A _q ≤m+1, this implies that no other job is enqueued in PQ _q . As soon as the head of FQ _q releases ℓ _q , J _i is moved to FQ _q . Hence there is at most one job in PQ _q at any time, and the ordering of PQ _q is irrelevant.

The FIFO ordering of FQ _q implies that each of J _i ’s requests is preceded by at most one request from each other task that accesses ℓ _q . The per-task interference limit is hence N _i,q . Since ℓ _q is shared among only A _q ≤m+1 tasks, one of which is T _i , no more than (A _q −1)⋅N _i,q requests pi-block J _i in total. Priority inheritance ensures that the resource-holding job is scheduled whenever J _i incurs s-oblivious pi-blocking; the cumulative duration of the (A _q −1)⋅N _i,q longest requests for ℓ _q by tasks other than J _i thus bounds maximum s-oblivious pi-blocking. □

In the case of A _q >m+1, higher-priority jobs of some other task T _x may “skip ahead” of J _i repeatedly while J _i waits in PQ _q . However, the per-task interference limit is still limited to 2⋅N _i,q , that is, the per-task interference limit is only doubled even if jobs of T _x “skip ahead” an arbitrary number of times.

Lemma 18

Let T _x denote some task other than T _i that accesses ℓ _q (i.e., T _i ≠T _x and N _x,q >1). Under the global OMLP, jobs of T _x cause J _i to incur s-oblivious pi-blocking for at most the duration of two requests each time that J _i requests ℓ _q .

Proof

In order to pi-block J _i , a request issued by some J _x must precede J _i ’s request in FQ _q (i.e., J _x enters FQ _q before J _i does). If A _q ≤m+1, the bound follows analogously to Lemma 17 since FQ _q is FIFO-ordered.

Hence assume A _q >m+1. In this case, jobs of T _x may enter FQ _q repeatedly while J _i waits in PQ _q . Let t _a denote the first time that a job of T _x , denoted J _x,a , enters FQ _q , and let t _b denote the second time that a job of T _x , denoted J _x,b , enters FQ _q while J _i is continuously waiting in PQ _q . Since tasks are sequential, J _x,b necessarily issued its request after J _i issued its request (this is not necessarily the case with J _x,a ).

Further, let t ₁ denote the time that J _i enters FQ _q (as indicated in Fig. 10). If t ₁ does not exist (i.e., if J _i never enters FQ _q ), then either FQ _q is continuously populated with higher-priority jobs and J _i does not incur s-oblivious pi-blocking, or some requests fails to complete (which is not possible since each L _i,q is presumed finite). Therefore assume t ₁ exists.

J _i does not incur s-oblivious pi-blocking during [t _b ,t ₁). Since J _i is waiting in PQ _q at time t _a , J _x,a is necessarily preceded by m−1 other jobs in FQ _q , which must complete before J _x,a ’s request is satisfied. Since tasks are sequential, J _x,a has completed its request before J _x,b enters FQ _q at time t _b . Therefore, at least m higher-priority jobs must have entered FQ _q during [t _a ,t _b ); otherwise, J _i would no longer be waiting in PQ _q at time t _b . The presence of m higher-priority pending jobs rules out s-oblivious pi-blocking after t _b (until J _i enters FQ _q at time t ₁).

Therefore, at most one of the requests issued by jobs of T _x after J _i issued its request pi-blocks J _i . Since sporadic tasks are sequential, at most one request of T _x that was issued prior to J _i ’s request is incomplete when J _i issues its request. Hence, at most two requests of T _x cause J _i to incur pi-blocking. □

As a result, the per-task interference limit in the case of A _q >m+1 is 2⋅N _i,q . This yields the following bound.

Lemma 19

Under the global OMLP, if A _q >m+1, then a job J _i incurs at most

s-oblivious pi-blocking due to requests for resource ℓ _q .

Proof

By Lemma 14, J _i incurs s-oblivious pi-blocking for the combined duration of at most 2⋅m−1 requests each time that it requests ℓ _q , which implies that J _i is delayed by at most (2⋅m−1)⋅N _i,q requests in total. Lemma 18 implies an interference limit of 2⋅N _i,q . Priority inheritance ensures that the resource-holding job is scheduled whenever J _i incurs pi-blocking. The bound follows. □

This yields the following overall bound on maximum s-oblivious pi-blocking.

Theorem 4

Under the global OMLP, a job J _i incurs s-oblivious pi-blocking for at most

time units, where x _q ≜A _q and l _q ≜1 if A _q ≤m+1, and x _q ≜2⋅m and l _q ≜2 if A _q >m+1.

Proof

Follows from Lemmas 17 and 19, since resource requests are not nested, and since J _i does not incur s-oblivious pi-blocking under the global OMLP while not requesting resources. □

This concludes the analysis of the global OMLP. Next, we consider the clustered OMLP from Sects. 4.2–4.4, which uses priority donation instead of priority inheritance.

1.4 A.4 The clustered OMLP for mutual exclusion

A job J _i is subject to two sources of s-oblivious pi-blocking under the clustered OMLP. J _i can be delayed each time it issues requests for shared resources, and additionally once upon release if it serves as a priority donor. We begin with the mutex variant of the clustered OMLP, which is the simplest of the three protocols based on priority donation. Recall from Sect. 4.2 that each resource ℓ _q is protected by a simple FIFO queue FQ _q .

Lemma 20

Under the clustered OMLP ’s mutex protocol, a job J _i incurs at most

pi-blocking due to requests for resource ℓ _q issued by jobs of tasks assigned to the jth cluster.

Proof

By Lemma 4, priority donation ensures that at most c requests are incomplete at any time in each cluster; therefore, at most c requests from each cluster C _j precede J _i in FQ _q each time that it issues a request. The strict FIFO ordering in FQ _q ensures a per-task interference limit of N _i,q . Due to priority donation, resource-holding jobs are always scheduled (Lemma 2). In the case of J _i ’s local cluster (i.e., if j=P _i ), only c−1 requests can interfere since J _i ’s own request counts towards the limit of c concurrent requests imposed by priority donation. Since jobs and tasks are sequential, J _i is not delayed by requests of (other) jobs of T _i . □

When bounding the maximum pi-blocking due to priority donation, we only need to consider the set of tasks that could have released a lower-priority job prior to J _i ’s arrival since priorities are only donated to jobs with lower base priority. This set of tasks necessarily depends on the specific scheduling policy.

Definition 13

We let lower(T _i ) denote the set of local tasks that could potentially require one of T _i ’s jobs to serve as a priority donor upon release. Under EDF-based schedulers, lower(T _i ) includes only tasks with longer relative deadlines. Under FP-based schedulers, lower(T _i ) includes tasks with lower priorities.

Lemma 21

Under the clustered OMLP ’s mutex protocol, a job J _i incurs at most $b_{i}^{D}$ s-oblivious pi-blocking upon release while serving as a priority donor, where

Proof

By Lemma 3, maximum s-oblivious pi-blocking due to priority donation is limited to one request span. Analogously to Lemma 20, $b_{i}^{D}$ bounds the maximum request span of any local, potentially lower-priority job J _x by considering the c longest requests in each remote cluster that could cause J _x to incur acquisition delay, and the c−1 longest requests in J _x ’s local cluster. □

Theorem 5

Under the clustered OMLP ’s mutex protocol, a job J _i incurs at most

s-oblivious pi-blocking due to requests for shared resources, where b _i,q,j and $b_{i}^{D}$ are defined as in Lemmas 20 and 21, respectively.

Proof

Follows from Lemmas 20 and 21, and the assumptions that resource requests are not nested and that tasks do not migrate across cluster boundaries. □

1.5 A.5 The clustered OMLP for RW exclusion

The bounds on maximum pi-blocking under the OMLP’s RW protocol are structurally similar to the bounds on maximum spin-blocking and pi-blocking under non-preemptive phase-fair RW spinlocks that we previously presented in [18]. This is because the OMLP implements phase-fairness, and because priority donation allows at most c concurrent requests in each cluster, which has an effect that is equivalent to non-preemptive execution.

We begin by considering the set of potentially blocking write requests. Since write requests are satisfied in FIFO order with respect to other write requests, maximum acquisition delay incurred by a writer due to earlier-issued write requests is the same under the OMLP’s mutex and RW variants. However, since reader and writer phases alternate, the maximum acquisition delay incurred by a reader due to earlier-issued write requests is limited to one critical section. That is, at most $N^{W}_{i,q} \cdot c + N^{R}_{i,q}$ write requests issued by jobs of a remote cluster can block J _i under the clustered OMLP’s RW variant. In the case of J _i ’s local cluster, if c>1, then the same reasoning applies and no more than $N^{W}_{i,q} \cdot(c - 1) + N^{R}_{i,q}$ write requests block J _i . In the special case of c=1, local jobs cannot cause J _i to incur acquisition delay since they are not scheduled while J _i waits. These considerations lead to the following definition of the set of possibly-interfering write requests.

Definition 14

In the following, let $x^{\mathit{rem}} = N^{W}_{i,q} \cdot c + N^{R}_{i,q}$ and $x^{\mathit{loc}} =N^{W}_{i,q} \cdot(c - 1) + N^{R}_{i,q}$, and define the sets of possibly-interfering write requests from jobs in the jth cluster, denoted as W(T _i ,j,ℓ _q ), as follows.

Further, let W _i,q denote the union of all possibly-interfering write requests across all clusters, and let w _i,q denote the maximum number of blocking write requests.

$$W_{i,q} = \bigcup_{j = 1}^{m/c} W(T_i, j, \ell _q) \quad w_{i,q} = \vert W_{i,q} \vert $$

Next, we consider the set of potentially blocking read requests. The defining property of an RW lock is that readers do not directly block other readers. That is, in the absence of any writers, a reader is not delayed in RW locks regardless of the number of concurrent read requests. Intuitively, a reader phase can only transitively block another read request if said phase is “assisted” by an also-blocking, interspersed write request. This intuition can be formalized to characterize acquisition delay due to interfering read requests in terms of the number of interfering write requests.

Lemma 22

(From [14, 18])

Let J _i denote a job that issues at most $N_{i,q}^{W}$ write requests for a resource ℓ _q , let w denote the number of write requests that cause J _i ’s write requests for ℓ _q to incur acquisition delay, and let r denote the number of reader phases that cause J _i ’s write requests for ℓ _q to incur acquisition delay. If ℓ _q is protected by a phase-fair RW lock, then $r \leq w + N_{i,q}^{W}$.

Similarly, a writer that is not delayed by other writers incurs acquisition delay for the duration of at most one read request regardless of the number of blocking readers. For example, if m−1 readers hold a resource ℓ _q when J _i issues a write request for ℓ _q , then all m−1 readers proceed in parallel and J _i incurs acquisition delay only for the duration of the longest earlier-issued read request. Therefore, J _i incurs acquisition delay due to interfering read requests for the combined duration of at most $N^{R}_{i,q} + (m-1) \cdot N^{W}_{i,q}$ read requests (recall Lemmas 8 and 9). Taken together, this leads to the following definition.

Definition 15

Let $r_{i,q} = \min(w_{i,q} + N^{W}_{i,q},\ N^{R}_{i,q} + (m-1) \cdot N^{W}_{i,q})$, and define the sets of possibly-interfering read requests from jobs in the jth cluster, denoted as R(T _i ,j,ℓ _q ), as follows.

Analogously to W _i,q , let R _i,q denote the set of all possibly interfering read requests across all clusters.

$$R_{i,q} = \bigcup_{j = 1}^{m/c} R(T_i, j, \ell _q). $$

With these definitions in place, we can state the following bound on pi-blocking due to requests for a given resource.

Lemma 23

Under the clustered OMLP ’s RW protocol, a job J _i incurs pi-blocking due to its read and write requests for resource ℓ _q for at most b _i,q =total(w _i,q , W _i,q )+total(r _i,q , R _i,q ) time units.

Proof

Analogously to Lemma 20. Each time that J _i issues a write request, it can be preceded by up to c other write requests in each cluster since the writer queue WQ _q is FIFO ordered, and because priority donation allows at most c concurrent requests per cluster. Also due to the FIFO order, each other task can block each of J _i ’s write requests with at most one request. Each time that J _i issues a read request, it is blocked by at most one write request since the OMLP implements phase-fairness. Therefore, the per-task interference with regard to write requests is $N^{W}_{i,q}+N^{R}_{i,q}$, and in total J _i ’s $N^{R}_{i,q}$ read requests and $N^{W}_{i,q}$ write requests are blocked by at most $N^{R}_{i,q}+N^{W}_{i,q} \cdot c$ write requests in the case of a remote cluster, and by at most $N^{R}_{i,q}+N^{W}_{i,q} \cdot(c-1)$ requests in the case of J _i ’s local cluster. The definitions of W(T _i ,j,ℓ _i,q ) and W _i,q follow.

By Lemma 22, the upper bound on the total number of blocking writes w _i,q implies an upper bound of $w_{i,q} + N^{W}_{i,q}$ on the number of blocking reader phases. The total number of blocking reader phases is also limited to $N^{R}_{i,q} + (m-1) \cdot N^{W}_{i,q}$: due to priority donation and because reader and writer phases alternate in a phase-fair RW lock, each of J _i ’s read requests is transitively blocked by at most one reader phase, and each of J _i ’s write requests is blocked by at most m−1 interspersed reader phases (since at most m−1 write requests block each of J _i ’s write requests). The lesser of the two bounds limits the total number of blocking reader phases r _i,q . The definitions of R(T _i ,j,ℓ _q ) and R _i,q follow.

Since J _i is blocked by at most w _i,q writer phases and r _i,q reader phases, total s-oblivious pi-blocking is bounded by the w _i,q longest requests in W _i,q and the r _i,q longest request in R _i,q . □

Since the clustered OMLP uses priority donation, a job may also incur s-oblivious pi-blocking when serving as a priority donor. The duration of priority donation depends on the request span of the priority recipient’s request, which may be either a write or a read. The maximum acquisition delay of a single write request for resource ℓ _q issued by job J _i can be bounded by instantiating Definitions 14 and 15 assuming $N^{R}_{i,q}=0$ and $N^{W}_{i,q}=1$. Similarly, the maximum acquisition delay of a single read request for ℓ _q can be bounded by instantiating said definitions assuming $N^{R}_{i,q}=1$ and $N^{W}_{i,q}=0$. To avoid needless repetition, we use the following definitions to denote these two special cases.

Definition 16

Let $W'_{i,q}$ and $w'_{i,q}$ denote the values of W _i,q and w _i,q , respectively, that result when assuming $N^{R}_{i,q}=0$ and $N^{W}_{i,q}=1$ in Definition 14 above. Similarly, let $W''_{i,q}$ and $w''_{i,q}$ denote the values of W _i,q and w _i,q , respectively, that result when assuming $N^{R}_{i,q}=1$ and $N^{W}_{i,q}=0$ in Definition 14 above.

Definition 17

Let $R'_{i,q}$ and $r'_{i,q}$ denote the values of R _i,q and r _i,q , respectively, that result when assuming $N^{R}_{i,q}=0$ and $N^{W}_{i,q}=1$ in Definition 15 above. Similarly, let $R''_{i,q}$ and $R''_{i,q}$ denote the values of R _i,q and R _i,q , respectively, that result when assuming $N^{R}_{i,q}=1$ and $N^{W}_{i,q}=0$ in Definition 15 above.

With these special cases in place, we can express the maximum request span. Recall from Definition 13 that we let lower(T _i ) denote the set of tasks local to T _i that could potentially cause J _i to incur pi-blocking upon release.

Lemma 24

Under the clustered OMLP ’s RW protocol, a job J _i incurs at most $b_{i}^{D} = \max(b_{i}', b_{i}'')$ s-oblivious pi-blocking upon release while serving as a priority donor, where

bounds the case of a writing priority recipient, and where

bounds the case of a reading priority recipient.

Proof

Follows analogously to Lemma 21 since J _i serves as a priority donor at most once and at most for the duration of one request span. The maximum request span of a lower-priority write request is bounded by $b'_{i}$; the maximum request span of a lower-priority read request is bounded by $b''_{i}$. The maximum of either scenario bounds maximum s-oblivious pi-blocking due to priority donation under the clustered OMLP for RW exclusion. □

This yields the following bound on s-oblivious pi-blocking.

Theorem 6

Under the clustered OMLP ’s mutex protocol, a job J _i incurs at most

s-oblivious pi-blocking due to read and write requests for shared resources, where b _i,q and $b_{i}^{D}$ are defined as in Lemmas 23 and 24, respectively.

Proof

Follows from Lemmas 23 and 24, and the assumptions that resource requests are not nested and that tasks do not migrate across cluster boundaries. □

1.6 A.6 The clustered OMLP for k-exclusion

In this section, we establish a bound on s-oblivious pi-blocking under the clustered OMLP for k-exclusion, which is presented in Sect. 4.4. The presented analysis is reasonably tight if blocking requests are relatively uniform in duration. However, if request lengths are heavily skewed (i.e., if there are some infrequent, long-running requests, but most requests are short), then a more accurate bound could be obtained by applying multiprocessor response-time analysis for non-preemptive global FIFO scheduling to each resource. In the following simpler analysis, which suffices for our purposes, some pessimism arises because Lemma 11, which implicitly lower-bounds the request completion rate, does not take non-uniform request lengths into account.

Lemma 25

Under the clustered OMLP ’s k-exclusion protocol, a job J _i incurs at most b _i,q s-oblivious pi-blocking due to requests for resource ℓ _q , where

Proof

By Lemma 4, priority donation ensures that at most c requests are incomplete at any time in each cluster; therefore, at most c requests in each cluster precede J _i in KQ _q or hold a replica of ℓ _q at the time that J _i issues a request. The FIFO ordering of jobs in KQ _q ensures a per-task interference limit of N _i,q . Therefore, the set of the N _i,q ⋅c longest requests issued by jobs in the jth cluster, denoted b _i,q,j , bounds the worst-case interference from jobs in that cluster. In the case of J _i ’s local cluster, only c−1 requests can interfere since J _i ’s own request counts towards the limit of c concurrent requests imposed by priority donation.

Lemma 11 implies that J _i holds a replica of ℓ _q after at most ⌈(m−k _q )/k _q ⌉ prior requests for ℓ _q complete. Therefore, across all N _i,q requests, J _i is pi-blocked at most for the cumulative duration of the N _i,q ⋅⌈(m−k _q )/k _q ⌉ longest requests issued by jobs in any cluster. □

To bound maximum s-oblivious pi-blocking due to priority donation, we again require a bound for a single request. Such a bound can be obtained by applying Lemma 25 above to a single request.

Definition 18

Let $b'_{i,q}$ denote the value of b _i,q computed assuming N _i,q =1 in Lemma 25 above.

Recall from Definition 13 that we let lower(T _i ) denote the set of tasks local to T _i that could potentially cause J _i to incur pi-blocking upon release.

Lemma 26

Under the clustered OMLP ’s k-exclusion protocol, a job J _i incurs at most

s-oblivious pi-blocking upon release while serving as a priority donor.

Proof

Follows analogously to Lemma 21 and Lemma 24. □

Theorem 7

Under the clustered OMLP ’s k-exclusion protocol, a job J _i incurs at most

s-oblivious pi-blocking due to requests for shared resources, where b _i,q and $b_{i}^{D}$ are defined as in Lemmas 25 and 26, respectively.

Proof

Follows from Lemmas 25 and 26, and since resource requests are not nested. □

1.7 A.7 Schedulability test

Having derived bounds on maximum s-oblivious pi-blocking, any sustainable [6, 8] locking-unaware schedulability test can be used to establish schedulability under the OMLP. In short, we require a sustainable schedulability test because each task’s parameter b _i is only an upper bound (i.e., it is not exact); therefore the employed schedulability test must be resilient to execution cost decreases at runtime.

Recall that b _i was derived assuming that suspended higher-priority jobs are accounted for as demand. Thus, each per-job execution time must be inflated by b _i before applying existing schedulability tests that assume tasks to be independent.

Theorem 8

Let $\mathcal{T}$ denote a sustainable schedulability test for independent tasks for the employed JLFP scheduling policy. A task set τ is schedulable under the OMLP if $\tau' \triangleq \{ T'_{i}(e_{i} + b_{i}, p_{i}) \ \vert \ T_{i} \in\tau\}$ is deemed schedulable by $\mathcal{T}$.

Note that the derivation of b _i itself does not depend on the actual scheduling policy or $\mathcal{T}$; the OMLP can thus be applied to any JLFP scheduling policy and any corresponding sustainable schedulability test.