Many-server scaling of the N-system under FCFS–ALIS

The N-system with independent Poisson arrivals and exponential server-dependent service times under the first come first served and assign to the longest idle server policy has an explicit steady-state distribution. We scale the arrival rate and the number of servers simultaneously, and obtain the fluid and central limit approximation for the steady state. This is the first step toward exploring the many-server scaling limit behavior of general parallel service systems.


Introduction
In this paper we study the many server N-System shown in Figure 1, with Poisson arrivals and exponential service times, under first come first served and assign to longest idle server policy (FCFS-ALIS), as the number of servers becomes large.Before describing the model in detail, we will first discuss our motivation for studying this system.The N-System is one of the simplest special cases of the so called parallel server systems, as defined in [9,16] and further studied in [15,22,7,18,6,21,13,14,1]. The general model has customers of types i = 1, . . ., I, servers of types j = 1, . . ., J, and a bipartite compatibility graph G where (i, j) ∈ G if customer type i can be served by server j.Arrivals are renewal with rate λ, where successive customer types are i.i.d. with probabilities α i , there is a total of n servers, of which nθ j are of type j, and service times are generally distributed with rates µ i,j .Assume the system is operated under the FCFS-ALIS policy, that is servers take on the longest waiting compatible customer, and arriving customers are assigned to the longest idle compatible server.For this general system necessary and sufficient conditions for stability (positive Harris recurrence for given λ), or for complete resource pooling (there exists critical λ 0 such that the system is stable for λ < λ 0 , and the queues of all customer types diverge for λ > λ 0 ) cannot be determined by 1st moment information alone (as shown by an example of Foss and Chernova [9]).In particular, under FCFS-ALIS calculation of the matching rates r i,j , which are long term average fractions of services performed by servers of type j on customers of type i, is intractable.
In the special case that service rates depend only on the server type, and not on customer type, with Poisson arrivals and exponential service times, the system has a product form stationary distribution, as given in [5].In that case matching rates can be computed from the stationary distribution.
The following conjecture was made in [1]: if the system is stable and has complete resource pooling for given λ, n, and we let both become large together, the behavior of the system simplifies: there will exist β j such that servers of type j perform a fraction β j of the services, and the matching rates r i,j will converge to the rates for the FCFS infinite matching model with G, α, β, as calculated in [4] (see also [2]).The conjecture is based on the following heuristic argument: in steady state the times that each server becomes available form a stationary process which is only mildly correlated with the other servers and so servers become available approximately as a superposition of almost independent stationary processes which in the many server limit becomes a Poisson process, and server types are then i.i.d. with probabilities β j , while customer types arrive as an i.i.d.sequence with probabilities α i , which corresponds exactly to the model of FCFS infinite matching.
In our current study of the many server N-System we shall verify the conjectured many server behavior for this simple parallel server system.To do so we start from the known stationary distribution of the N-System with many servers, as derived from [5], and study its behavior as n → ∞.As it turns out, the product form stationary distribution even for this simple case is far from simple, and the derivations of limits, which use summations over server permutations and asymptotic expansions of various expressions are quite laborious.We feel that this emphasizes the difficulty of verifying the conjectured behavior of the general system, which remains intractable at this time.
We mention that the N-System with just two servers, has been the subject of several papers, [12,3,11,19].In this paper, our focus is on the N-System with many servers under FCFS-ALIS policy, and its limit property.
The rest of the paper is structured as follows: In Section 2 we describe the model, in Section 3 we use some heuristic arguments to obtain a guess at the limiting behavior.In Section 4 we obtain the stationary behavior under many server scaling.In Section 5 we illustrate our results with some numerical examples.To improve the readability of the paper we have put all the proofs for Section 4 in the appendix.

The Model
In our N-System customers of types c 1 and c 2 arrive as independent Poisson streams, with rates λ 1 , λ 2 .There are skill based parallel servers, n 1 servers of type s 1 which are flexible and can serve both types, and n 2 servers of type s 2 which can only serve type c 1 customers.We assume service times are all independent exponential, with server dependent rates.The service rate of an s 1 server is µ 1 , the service rate of an s 2 server is µ Service policy is FCFS-ALIS.
The system is obviously Markovian.In [3,20,5] the following state description for the server dependent Poisson exponential system with J server types and I customer types was used: imagine the customers arranged in a single queue by order of arrivals, and servers are attached to customers which they serve, and the remaining idle servers are arranged by increasing idle time, see Figure ?? The state is then s = (S 1 , q 1 , S 2 , q 2 , . . ., S n−i , q n−i , S n−i+1 , . . ., S n ), where S 1 , . . ., S n is a permutation of the n servers, the first n − i servers are the ordered busy servers, and the last i servers are the ordered idle servers, and where q j , j = 1, . . ., n − i are the queue lengths of the customers waiting for one of the servers S 1 , . . ., S j , and skipped (cannot be served) by servers S j+1 , . . ., S n .
For the special case of the N-System, the following three random quantities are important: i 1 = I 1 (s) the number of idle servers of type s 1 , i 2 = I 2 (s) the number of idle servers of type s 2 , and k = K(s) ≥ 0 the number of servers of type s 2 which follow the last server of type s 1 in the sequence S 1 , . . ., S n .We let i = I(s) be the total number of idle servers.Because of the structure of the N-System, and the FCFS-ALIS policy the following properties hold for i = 0, . . ., n and k = 0, . . ., n 2 : (i) There are no customers waiting for any server which precedes the last s 1 server in the permutation.In other words, for all j < min(n − k, n − i) we have q j = 0.In particular, if there is an idle type s 1 server, in other words if i > k, then there are no waiting customers at all.
(ii) If there are any idle servers, then there are no type c 1 customers waiting for service, in other words, if i > 0 then all the waiting customers are of type c 2 .
(iii) If there are no idle servers, then only the last queue can contain type c 1 customers, in other words, if i = 0 then the last queue may contain customers of both types, but all the other waiting customers are of type c 2 .

Denote
Then a necessary and sufficient condition for stability is We shall require a stronger condition of complete resource pooling, defined by where β equals the long run fraction of services performed by s 1 servers.The value of β will be calculated in the next Section.Using the results of [4,5] we can then write the exact stationary distribution of this system.We wish to show that as the arrival rates and the number of servers increase the system simplifies, and we get very precise many server scaling limits.We will investigate the behavior of the system when α, θ, ρ are fixed, and n → ∞.To be precise, we shall then have n, λ = ρn, λ 1 = αλ, λ 2 = (1 − α)λ, n 1 = θn , n 2 = n − n 1 , all of which go to ∞.

Fluid Calculations
We perform the following heuristic calculation: As long as the system is underloaded (ρ < 1), each server of type s 1 will have a cycle of service of mean length 1/µ 1 , followed by an idle period, and similarly each server of type s 2 will have service of mean length 1/µ 2 followed by an idle period.
The key idea now is that when n → ∞, the idle periods should have the same length for both types, because of ALIS.Let T be the average length of the idle time.The average cycle times will be: 1/µ 1 + T and 1/µ 2 + T .Denote by β the long run fraction of services performed by s 1 servers, and 1 − β for type s 2 .The flow rate out of one type s 1 server is 1/(1/µ 1 + T ), the flow rate out of all type s 1 servers should equal λβ.Similarly the flow rate out of all type s 2 servers should equal λ(1 − β).That is, Now we solve for T and β: we rewrite and eliminate β: to get a quadratic equation for T : Here g(0) < 0 by ρ < 1, so the equation has one positive and one negative root.Solving for positive T we get: Note: For the case of µ 1 = µ 2 = µ we get T = 1−ρ ρ 1 µ .From T and little's law we can obtain the average number of idle servers in pool 1 and pool 2, denoted by m 1 and m 2 respectively.
The value of α does not come into the equation for T , or the calculation of β.Hence, once we solve and obtain β, the property of complete resource pooling will consist of checking that α > 1 − β.
We will show that the following holds for the stationary queue, as n → ∞: • K(s) is distributed as a geometric random variable, taking values 0, 1, 2, . . .with probability of success 1 and correlation .
• Successive idle servers except for the last K + 1 are i.i.d. of type s 1 with probability β and of type s 2 with probability 1 − β.
4 Many server limit of the stationary distribution

Exact Stationary Distributions
We first obtain the stationary distribution for each state s.We note that the stationary probabilities depend mainly on the values of K(s), I 1 (s), I 2 (s).Let µ(S j ) denote the service rate of the server at position j.
Theorem 1.The stationary distribution of the state s of the FCFS-ALIS many server N-system is given by: where B is a normalizing constant.
Before we manipulate equation (3), we introduce a lemma to facilitate the calculation.Lemma 1.Let A 1 , . . ., A m denote a permutation of m given positive real numbers a 1 , . . ., a m , we have (A1,...,Am)∈P(a1,...,am) where P(a 1 , . . ., a m ) denotes the set of all the permutations of a 1 , . . ., a m .Now we can get the joint stationary distribution of K(s), I 1 (s), I 2 (s).We denote by π(k, i 1 , i 2 ) the stationary probability of Theorem 2. The steady state joint distribution of K(s), I 1 (s), I 2 (s) is given by: (4) where B 1 is a normalizing constant.

The Distribution of
In this section we obtain the asymptotic distribution of (I 1 (s), I 2 (s)) conditional on K(s) = k, as n → ∞.We first show that as n → ∞, the probability of no idle servers of type s 1 goes to zero, and so the probability that customers need not wait goes to 1. Next we condition on K(s) = k and show where T is given in (1).Finally, we condition on K(s) = k and show that the scaled and centered values of (I 1 (s), I 2 (s)) converge in distribution to a bivariate normal distribution.
Theorem 3. When n → ∞, as long as ρ < 1, δ < 1, From this theorem we see that when n → ∞, n converge to (f 1 , f 2 ) in probability for any k ≥ 0. That is, for any > 0, when n → ∞, we have After showing the fluid limit result, we are now ready to show the central limit result.Theorem 5.For any k ≥ 0, when n → ∞, we have .

4.3
The Distribution of K(s), the Location of the First Type s 1 Server.

Summary of stationary distribution
Theorem 6 shows that K(s) converges in distribution to a geometric distribution, so P (K(s) < ∞) = 1.Therefore, we can extend Theorem 4 and Theorem 5 into unconditional versions.
Consider a special case when µ 1 = µ 2 = µ, we have θ = β.m 1 and m 2 can be easily solved: converges in distribution to a bivariate Normal distribution with mean (0, 0), variance and correlation ρ θ(1 − θ) The total idleness has mean of (1 − ρ)n and variance of

Comparison to the bipartite FCFS infinite matching model
In the infinite matching model corresponding to the N-System there is an infinite sequence of customers, of types c 1 , c 2 , where the customer types are i.i.d., type is c 1 with probability α and c 2 with probability 1 − α, and an independent sequence of servers, of types s 1 , s 2 , where the server types are i.i.d., type is s 1 with probability β and s 2 with probability 1−β, and compatibility graph with arcs {(c 1 , s 1 ), (c 1 , s 2 ), (c 2 , s 1 )}.Successive customers and servers are matched according to FCFS: each server is matched to the first compatible customer that was not matched to a previous server, and each customer is matched to the first compatible server that was not matched to a previous customer.
After n of the customers have been matched, consider the sequence of remaining servers.Let K n be the number of servers of type s 2 that are first in this sequence, preceding the first server of type s 1 .The (K n ) ∞ n=0 is a Markov chain.The steady state distribution for this Markov chain is that

Numerical Examples
We test our results by investigating an N-system.λ = 100, n 1 = n 2 = 100, µ 1 = µ 2 = 1, ρ = 0.5.From our approximation results, as long as α + θ > 1, or α > 0.5, both pools should have similar utilization.So the average number of idle servers in each pool is close to 50, with variance of We use exact stationary distribution to verify this.Now we can calculate the expectation and variance of idle number in each pool exactly, listed in the following table.We can see that when α > 0.5, the approximation is very good.When α < 0.5, the approximation does not work.In fact, when α + θ < 1 and system is large, complete resource pooling disappears, and server pool 1 seldom serves type s1 customers.The N-system operates like 2 separate queues: pool 1 serves type s2 and pool 2 serves type s1.The utilization of pool 1 is (1−α)λ n1 and the utilization of pool 2 is αλ n2 .From previous results, the number of idles When α goes down to 0.5, the approximation is getting worse.Note that E[K(s)] ≈ α α+θ−1 is large when α is close to 0.5, making it not negligible.We have a better approximation for α close to 0.5.When The approximated average number of busy servers in pool 1 is θ (k)ρn.
To make it consistent, we need θ = E[X] ρn .Solving this equation gives θ, and E[X].In this example, when α = 0.6, the solution to even when α = 0.5, the improved approximation is not bad.

Acknowledgment
We are grateful to Ivo Adan for helpful discussion of this paper.
A Appendix: Proofs for Section 4 A.1 Proof of Lemma 1 and Theorem 2 Proof of Lemma 1.We prove this lemma by induction.Define the left-hand-side as C m .
Step 2: Step m: Proof of Theorm 2. Summation over the geometric terms q j = 0, . . ., ∞ in (3) gives Next we see that in this expression, permutations of S 1 , . . ., S n with the same (k, i 1 , i 2 ) have a similar structure.We now sum over all the permutations of the appropriate S j , 1 ≤ j ≤ n − max{k + 1, i 1 + i 2 }.By Lemma 1 we obtain Each permutation of the remaining servers, S j , n − max{k + 1, i 1 + i 2 } < j ≤ n has the same stationary probability.It remains to count the number of permutations.When i 1 = 0 we have i 2 ≤ k.For each permutation we choose 1 type s 1 server and k out of n 2 type s 2 servers to form the last k + 1 servers.The number of permutations is When i 1 > 0, we have i 2 ≥ k.For each permutation, we choose i 1 out of n 1 type s 1 servers and i 2 out of n 2 type s 2 servers.We then choose 1 from the i 1 idle servers of type s 1 , and k from the i 2 idle servers of type s 2 to obtain the last k + 1 servers.The number of permutations is Multiplying the terms in (7) by the appropriate number of permutations and defining gives (4).
(ii) We show that where m 1 and m 2 are defined in (2).
(iii) We show that as n → ∞ which proves the proposition.
The details of the proofs of these three steps are as follows: Proof of (i): First we calculate We use induction to calculate Therefore, the induction is valid and we have Next we calculate Similar to the induction calculating U m above, we can obtain , where X is a Poisson random variable with parameter λ1 µ2 .Using Stirling's approximation, Recall that κ = λ1 µ2n2 and note that log κ + 1 − κ ≤ 0. Note also that when n → ∞, X can be approximated by a Normal distribution with mean λ1 µ2 and variance λ1 µ2 .Next we analyze P (X<n2) in 3 cases depending on κ.
• When 0 < κ < 1, from the Normal distribution approximation, when n → ∞, P (X < n 2 ) → 1.Therefore, • When κ > 1, when n → ∞, the Normal distribution approximation gives P (X < n 2 ) → 0. We need more care to treat this case.For any 1 ≤ j ≤ n 2 , In fact, for any fixed j, when n → ∞, For any > 0, let J := − log log κ .We have ≥ κ −J .There exists an N such that when n > N , for any 1 ≤ j ≤ J, P (X = n 2 − j) Therefore, In summary, when κ ≤ 1, P (I 1 (s) = 0, I 2 (s) = 0) is negligible compared with P (I 1 (s) = 0, I 2 (s) > 0) when n → ∞.We have Proof of (ii): From equation ( 4) we have The second equality is due to is of the order of n −1/2 .Therefore, When κ < 1, We have that Therefore, when n → ∞, This completes the proof that when n → ∞, Proof of Theorem 5. First we show that the weak convergence is valid given K(s) = 0. Then we show that the same holds when K(s) = k, for any fixed k.When K(s) = 0, we prove the convergence in probability in 2 steps.
(i) We show that for all states |I 1 (s)−m 1 | ≥ n or |I 2 (s)−m 2 | ≥ n, the conditional probability is dominated by a bounded constant multiple of the conditional probability of some point on the boundary of the rectangle (ii) When n → ∞, we approximate the conditional probability of the points in the rectangle We then show that the probability of points on the boundary is negligible compared with the conditional probability at ( m 1 , m 2 ).
Eventually the movement stops at the boundary which are n away from (m 1 , m 2 ).Therefore, the probability of any state (i 1 , i 2 ) satisfying i 1 > i * 1 or i 2 > i * 2 would be dominated by the probability of some point at the boundary.
For any (i 1 , i 2 ) satisfying i 1 ≤ i * 1 and i 2 ≤ i * 2 , since We have εn εn Figure 3: The dominance of steady steady probability and P (I 1 (s ) is dominated by the probability of some point at the boundary.
Proof of (ii): and n grows large, we can use Stirling's approximation.
We define The first order derivatives on x 1 and x 2 : solve the first order conditions.Look at the second order derivatives: The Hessian matrix is negative definite.Therefore, F (x 1 , x 2 ) is strictly concave on (0, θ)×(0, 1−θ) and reaches its unique global maximum at (f 1 , f 2 ).Since F (x 1 , x 2 ) is strictly concave and reaches its unique global maximum at (f 1 , f 2 ).The maximum of Since the boundary is a compact set, the maximum is attainable, denoted by F (f 1 , f 2 ) − η, where η > 0. Note that changes slowly when x 1 and x 2 change, compared with exp (nF (x 1 , x 2 )).We have exp(ηn) It converges to 0 when n → ∞.When K(s) = k > 0, and n → ∞, similarly, We can use the similar 2-step argument to show that (I 1 (s)/n, Proof of Theorem 5. To obtain the asymptotic distribution of I 1 , I 2 as n → ∞ we need to consider, by Theorem 4, only values i 1 , i 2 for which (i 1 − m 1 )/n → 0 and (i 2 − m 2 )/n → 0. We write i − m 2 are of the same order of magnitude as n, n 1 , n 2 , and we only consider i 1 , i 2 of the same order of magnitude.
, where the use of Stirling's approximation is justified for large n.Here B 2 = B 1 /P (K(s) = 0) and We clearly have: Therefore, We are left with We again write i 1 = m 1 + z 1 √ n, i 2 = m 2 + z 2 √ n, with z 1 / √ n → 0, z 2 / √ n → 0. We then have We can now use the same approximation as for k = 0 to show that I1(s)−m1 √ n , I2(s)−m2 √ n converge to the same bivariate Normal distribution.

Figure 1 :
Figure 1: The multi-server N-System

Table 2
5093.E[I 1 ] = 100 − θρn = 49.07,which is closer to the true value 49.18.The following table lists the comparison of the improved approximation and the true value: We can see that