Using efficiency frontier to inform operational decision-making

In this study, we introduce Efficiency Frontier, an economic assessment framework for business operations. The proposed framework emphasizes efficiency and sustainability. It aims to help organizations translate a complex metric system into a value index, conduct performance evaluation, and determine efficient resource planning strategies. Unlike the traditional methods that majorly focus on the achievement of performance targets, the proposed framework also accounts for the amount of resource that is invested in the achievement, which encourages managers to implement sustainable strategies that aim for long-term success. To validate the new framework, we systematically study the case of the City of Chicago’s 311 contact center with a base case analysis and a sensitivity analysis. In the base case, an Efficiency Frontier architected by a set of efficient solutions from 2013 to 2016 is used to identify outperforming scenarios given resource constraints. In the sensitivity analysis, we demonstrate the application of Efficiency Frontier in determining optimal resource planning strategies in conservative, moderate, and aggressive business investment scenarios. The base case Efficiency Frontier reveals that the marginal return on investment in improving the performance may diminish as investment increases. The sensitivity study suggests that optimal resource planning strategies should be adjusted according to value propositions. This work builds a framework for business leaders to economically assess their operational decision-making. Efficient strategies can be identified by exploring the maximum value increase per unit of resource given cost scenarios The fund of a strategy should be reduced when the marginal return of the strategy begins to diminish. The framework’s ability to suggest cost-saving strategies is especially informative in resource-limited settings. Efficient strategies can be identified by exploring the maximum value increase per unit of resource given cost scenarios The fund of a strategy should be reduced when the marginal return of the strategy begins to diminish. The framework’s ability to suggest cost-saving strategies is especially informative in resource-limited settings.


Introduction
An organization's performance measurement system may consist of a wide spectrum of measures that quantify time, cost, quality, and flexibility [1]. Among multiple means of correlating a broad range of measures to corporate strategies and integrating them as a system, perhaps the most prevalent method is Kaplan and Norton's Balanced Scorecard (BSC) [2][3][4]. According to a Bain & Company survey, it has a higher implementation rate than other widely used management approaches such as Activity-Based Management and Total Quality Management [2]. Although the previously published evidence indicates that BSC enables decision makers to oversee performance from financial, customer, business process, and learning and growth perspectives [3], a growing number of implementation limitations have been reported [7,8]. Particularly, the literature indicates that overemphasizing BSC goals would lead to resource underutilization [4]. For example, in service settings, a common Key Performance Indicator (KPI) is Abandon Rate, which is defined as a percentage of customers who give up on the services they intended to receive after a target response time has elapsed [5]. Although the BCS could measure the achievement of an Abandon Rate target, it neither accounts for the amount of resource invested in the achievement of the target nor considers the resource allocation relative to other KPIs, which may lead to operational inefficiency.
In order to fill the gap, we introduce Efficiency Frontier, an economic assessment framework to improve operational efficiency at a system level. Unlike BSC, which tracks a set of measures independently, Efficiency Frontier composes a set of KPIs as an index, and the index components or the selected KPIs are weighted against one another in order to balance the competing interests among them and ultimately maximize the overall value. Additionally, in contrast to the traditional way of measuring performance by comparing the changes of performance metrics with their corresponding pre-identified targets, the economic assessment approach enables managers to systematically measure how economically an organization's resources are distributed for a given level of customer satisfaction [1]. In other words, this article argues that managers should take efficiency factors into account when they evaluate production, marketing, sales, service, and operational performances. The Efficient Frontier theory was introduced by Harry Markowitz [6]. It was initially used to determine optimal investment portfolios given value and risk estimates [6]. Later, its popularity saw its use extended to other industries. For example, the National Bank of Belgium uses Efficient Frontier to assess the performance of education systems [7]. Efficiency Frontier identifies an education system as efficient if the system yields higher production value with lower costs. Shui et al. elevates Efficiency Frontier to improve manufacturing productivity and energy efficiency [8]. In healthcare, it is widely applied to evaluate the cost-effectiveness of new health interventions [9]. To our best knowledge, this is the first endeavor to introduce an Efficiency Frontier-based economic assessment framework into the operations domain. This framework allows researchers to conduct value-investment analysis on new business policies and strategies that aim for long-term success. It also encourages organizations to take value alignment and resource prioritization into account when they implement performance evaluation and resource allocation strategies.
The remainder of this article is organized into the following topics: in the Method section, we introduce Efficiency Frontier, an economic assessment framework, with a case study based on data from the City of Chicago's 311 contact center. We further illustrate the application of the Efficiency Frontier method in performance measurement and resource planning in the Results section. Study findings and insights are discussed and summarized in the Discussion and Conclusion sections respectively.

Method
An economic assessment framework relates the improvement of the value index to the resources that are invested in the improvement process. Therefore, the framework consists of two elements: value index and investment. Ideally, the hope is that the value derived from a strategy could be worth the investment, and the strategy is considered as optimal if its value-investment ratio is maximized. An Efficiency Frontier is formed by a series of optimal strategies given investment scenarios [10].

Forming an efficiency frontier-integrating KPIs as a value index
A value index, one of the two elements in Efficiency Frontier formation, could contain a set of KPIs. We define the KPIs that are included in a value index as the components of the index. Before introducing the method that integrates a set of KPIs or index components into an index, we will first provide notations that are used in the index formation process. In this article, we use V t to denote a value index at time t, m to represent an index component, and M to indicate the total number of index components that are included in the index. Additionally, the components are weighted differently to reflect an organization's strategic preferences. The weight of index component m is denoted as w m . Last, Y t,m is the value of component m at time t, and Ŷ t,m is the normalized value of Y t,m . The value index V t is constructed as a combination of all normalized and weighted components (Eq. 1). The normalization and weighting processes are described below.

Forming an efficiency frontier-normalizing index components
In order to integrate index components on different scales to a singular system, all types of measures Y t,m have to be normalized and converted to Ŷ t,m on a standard scale of 0-1, and the normalized value describes the degree to which a maximum value is achieved [11] (Eq. 2). A higher

Forming an efficiency frontier-determining weightings of index components
The selection of the component measure m should be aligned with organization's value proposition, and the corresponding weighting should reflect the priority level of the index component. The most prevalent method to determine the weighting is Thomas Saaty's Analytic Hierarchy Process (AHP) [12]. In this article, the weights vary between 0 and 1 ( 0 < w m < 1 ), and the sum of all weights should be equal to 1(Eq. 3).
For instance, in a scenario where w 1 = 0.4 , w 2 = 0.4 , and w 3 = 0.2 , component 1 is of equal importance to M ∑ m=1 w m = 1 component 2 and is two times more important than component 3. In a perfect scenario where ∀Ŷ t,m = 1 , An organization delivers the highest value when V t = 1 . In contrast, when ∀Ŷ t,m = 0, V t = 0.

Forming an efficiency frontier-calculating costs
The cost function of Efficiency Frontier consists of two elements: fixed costs and variable costs [13]. Fixed costs may include annual facility and machinery rental and maintenance costs, insurance, and utilities. Variable costs may vary with staffing strategies, production/service volumes, and the adoption of new programs/technologies.

Input data-City of Chicago 311 contact center
We use a case from the City of Chicago's 311 contact center to illustrate the benefit of Efficiency Frontier [14]. According to the center's official webpage: "the city of Chicago's 311 contact center is a comprehensive one-stop web and phone portal to provide non-emergency government services to Chicago residents, visitors, and business owners" [15]. In this study, we help the city of Chicago determine optimal performance evaluation and resource planning strategies. Table 1 summarizes the call center's Agent Efficiency, Utilization, Abandon Rate, and Average Wait Time information from 2013 to 2016. The definitions of the metrics are provided in Table 1. The metrics in Table 1 are used to architect a value index as index components. Note that the selection of index components for the City of Chicago is beyond the scope of this case study. However, the authors provide a concise suggestion on this topic in the Discussion section. In addition to the value index, investment is the other element of the efficiency analysis in this article. In this case study, investment is quantified as the number of agents per ten thousand calls within a certain timeframe (agent and call volume information is summarized in Table 1), and an expected cost can be derived from it once the information on fixed costs and variable costs (e.g. average salary information) is known for a given time interval of interest. In this case, weekly data was reported by the City of Chicago [15]. Note that expected cost per ten thousand calls is a more interesting measure than total cost in this case study because this is an efficiency measure that eliminates the scenarios where lower costs are simply caused by reduced call volume.

Predicting index component outcomes
This research work establishes a framework to help decision makers (1) evaluate past performance and (2) optimize future resource planning strategies. Usually, the outcomes of index components have already been available when decision makers use the metrics to evaluate past performance. However, when they need to optimize resource planning strategies for the next fiscal cycle, the future response of the index component to a strategy change may not be known at the current moment. In that case, we need to build a mathematical model to predict the component outcomes for the next planning cycle.
In order to predict the change of the index component given different resource planning strategies, we construct a nonlinear regression model using the Chicago 311 call center's 2016 data, the most recent data that is publicly available at the time of this work. Although Erlang-A and Erlang-C are considered to be the most prevalent models to conduct call center analytics [16], a nonlinear regression model has been chosen to predict the index outcomes because it is a more appropriate method given the data structure. Specifically, the center's information is inadequate to estimate call arrival and handling processes that are required in Erlang analytics. In this work, the nonlinear model is constructed as y � = a + b x + c ln x , where y ′ is an outcome variable, x is a predictor, and a, b, and c are coefficients. The coefficients can be obtained by minimizing the sum of squared residuals (SSR): where i represents a data point in a sample, I is a total number of data points, m denotes an index component of interest, Y i m indicates an actual outcome of component m among all I outcomes, and X i is a predictor for Y i m . The model was solved as an unconstrained nonlinear programming model. More details on solving an unconstrained nonlinear model can be found in the literature [17]. Additionally, the model was calibrated to the Chicago 311 call center's 2016 setting, and the calibration results are presented in Fig. 1.
With an optimized and calibrated model, we are able to test the impact of investment strategies on the performance measures, or the index components, and ultimately quantify their influence on the value index given weighting scenarios. In addition to a base case, we investigate six strategies for the Chicago 311 contact center's 2017 fiscal year (assuming we are currently in 2016): conservatively increase the investment by 5% from the baseline (Strategy 1 in Table 2), moderately increase the investment by 10% (Strategy 2), aggressively increase the investment by 20% (Strategy 3), conservatively decrease the investment by 5% (Strategy 4), moderately decrease the investment by 10% (Strategy 5), and aggressively decrease the investment by 20% (Strategy 6). In the base case, we assume the index components are equally important ( w 1 = w 2 = w 3 = w 4 = 0.25).

Performance evaluation-base case analysis
Let us assume that decision makers want to use historical data (2013-2015 data in this study) to oversee and evaluate weekly performance throughout 2016. By following the calculation procedures in the Method section of this article, we first obtain the aggregated investment and index outcomes in Fig. 2.
With the investment and index information, we use the center's 2013-2015 data to architect an Efficiency Frontier. Compared with the performance outcome in 2015 (orange dot in Fig. 2), the Chicago 311 center underperformed and overspent in 2013 (yellow dot in Fig. 2). In this case, we say that 2013′s performance was dominated by 2015′s performance. Any dominated dots should not be included in an Efficiency Frontier. Therefore, an Efficiency Frontier is linearly interpolated using 2014 and 2015 data (red solid line in Fig. 2).
Once the Efficiency Frontier is identified, it can be elevated to assess the 2016 weekly performance (blue dots in Fig. 2). Specifically, the frontier (red solid line in Fig. 2), its extrapolations (red dash lines in Fig. 2), minimum expected return to stay efficient if 2016′s investment level is greater than 2015′s (black dash line in Fig. 2), and maximum required investment to stay efficient if decision makers decide to maintain the index at the 2014 level (purple dash line in Fig. 2) divide the investment-value plane into four regions: NW, NE, SE, and SW (Fig. 2). As demonstrated in the frontier formation process, any performance outcomes that fall into SE would be dominated by the frontier, which means they are inefficient outcomes. In contrast, any performance outcomes that fall into NW become more economically efficient outcomes than the ones on the frontier, as a result of which they would define a new frontier. Last, the outcomes in NE and SW cannot be measured by the frontier because the most efficient outcomes in those regions have not been identified based on the historical data, or 2013-2015 data in this particular case.
Given the performance evaluation criteria described above, let us evaluate the Chicago 311 call center's weekly performance in 2016. In Fig. 2, each blue dot represents a weekly outcome in 2016. Note that the time interval of interest is selected based on the call center's data structure, and a total of 26 weekly outcomes are reported in the database [14]. In the SE region of the investment-value plane: no weekly outcomes in 2016 fall into this region, which indicates that no weekly performance results in 2016 can be identified as inefficient outcomes. In NE and SW, the Efficiency Frontier created based on 2014 and 2015 data is insufficient to measure the performance outcomes for week 3, 4, 7, and 8 in SW and week 15, 21, 23, 26, 27, and 28 in NE. For the weeks in NW, the call center performed efficiently, as the center created more value with less investment in those weeks compared with the historical standard. Additionally, a weekly-level Efficiency Frontier can be formed to compare 2016 outcomes among themselves. In Fig. 2, nodes 2, 9, 11, and 22 constitute an Efficiency Frontier (gray solid line). Compared with other 2016 weekly nodes, the nodes on the weekly-level Efficiency Frontier Moving on to the next performance evaluation cycle in 2017, the Efficiency Frontier can be updated by aggregating and incorporating 2016 data (Fig. 3). Similar analytical procedures apply if decision makers use the updated Efficiency Frontier to evaluate and monitor 2017 performance. Additionally, decision makers should be aware that in most business cases, return on investment diminishes over time (Fig. 3). In this case study, the incremental costeffectiveness ratio (ICER) of improving the value index from the 2014 level to the 2015 level is 7 AWCA (Table 3), which means it would cost the organization around 7 AWCA to have a full unit of value index improvement for every ten thousand calls. However, if decision makers intend to improve the performance from the 2015 level to the 2016 level, the marginal cost would be around 16 AWCA, more than double the 2015 level. Detailed calculations are included in Table 3.

Resource planning optimization-sensitivity analysis
Efficiency Frontier and its evaluation criteria can be used to help decision makers select the most efficient strategies. In the Method section of this paper, we identified six resource planning strategies. Each strategy is intended to either increase or decrease a certain amount of investment from the 2016 baseline for the next planning cycle, which is the 2017 cycle in this case study (assuming we are currently in the 2016 fiscal cycle). The investment of each strategy is reflected on the X-axis of the investment-value plane in Fig. 4. The return on investment of each strategy is reflected on the Y-axis of the plane. The outcomes of index components in the strategy scenarios are predicted through the nonlinear model described in the Method section.
Additionally, according to the calibration results in Fig. 1, Utilization is the only index component whose contribution to the value index is negatively associated with investment: more investment would increase the likelihood of generating more idle time, which would eventually impact the value index negatively. All other components are positively associated with investment. For example, more investment would lower the Abandon Rate, which would eventually improve the value index. A sensitivity analysis is conducted in this study to inform optimal resource planning strategies given value proposition scenarios where Utilization and the other components are weighted differently. In the base case (Fig. 4a), we assume decision makers value the index components equally ( w 1 = w 2 = w 3 = w 4 = 0.25 ). As shown in Fig. 4a, Strategy 1, 2, and 3 are favorable choices over Strategy 4, 5, and 6, since Strategy 1, 2, and 3 are in the NW region of the investment-value plane. Therefore, our results suggest that decision makers in the Chicago 311 call center should increase their investment in 2017. It is no surprise that the base case results recommend an investment increase because the baseline weighting results in w Utilization < w Agent Efficiency + w Abandon Rate + w Average Wait Time , which would eventually favor the decisions on an investment spike. The specific selection of Strategy 1, 2, or 3 would depend on the cost scenarios. In this case, Strategy 1 should be selected if the decision maker's  willingness-to-pay for a unit of value gain is low, whereas Strategy 3 should be selected if the willingness-to-pay is high.
In an extreme scenario where the weight on Utilization is significantly lower than the sum of weights on other index components (Fig. 4b), the benefit of investing more is expanded (the strategies are further deviated from the Efficient Frontier).
Moving from the scenario in Fig. 4b to a scenario where w Utilization = w Agent Efficiency + w Abandon Rate + w Average Wait Time , Fig.  4c suggests that the aggressive investment increase strategy (Strategy 3) becomes inefficient, whereas scaling back the investment conservatively (Strategy 4) would start to become an efficient strategy.
Finally, if the decision makers in the Chicago 311 service center decide to prioritize improving Utilization over other components in 2017 (Fig. 4-D), all investment increase strategies (Strategy 1, 2, and 3) would become inefficient, whereas a conservative or moderate investment scaling back (Strategy 4 and 5) would benefit the center from the perspective of operational efficiency.

Selection of component measures
There are multiple means of selecting index components. At the highest level, an organization could choose the component measures that promote the achievement of its strategic goals [1]. Kaplan and Norton's Balanced Scorecard framework is one of the strategic ways of determining key components from financial, customer, internal business, and innovation and learning perspectives [18]. At an operative level, decision makers should select the Fig. 4 Using Efficient Frontier to evaluate resource planning strategies given weighting scenarios measures that are directly related to the high-level components, which would align the local goals with the organization's strategic goals. More importantly, regardless of the use levels, the index components at the same level should not be highly correlated, which would create duplications.

Diminishing return on investment
In practice, the slope of Efficiency Frontier diminishes when investment increases (Figs. 3 and 4). In other words, the same amount of investment may result in a smaller value return when the frontier moves toward NE. When the index approaches the perfect score of 1, technology barriers and systematic noise could become the major causes of diminishing marginal return. Introducing evolutional technologies into the system and carrying out organizational transformations are common approaches to remove technology and system barriers. In a competitive environment, companies prefer to continue increasing investment even with a slower value return in order to phase the competitors out of the market. In a monopoly environment, companies favor an optimal balance between value improvement rate and investment. More information on contributing factors to diminishing marginal return can be found in the literature [19].

More on investment strategies
For certain investment strategies or initiatives, program costs should be accounted for when decision makers assess the economic impact of those strategies or initiatives. For example, evidence from previous studies has shown that implementing Lean Management programs could help call centers improve efficiency and reduce call volume [23,24]. When we use Efficiency Frontier to assess the impact of Lean Management programs, the program costs need to be taken into consideration in the evaluation process. Lastly, investment strategies could be further refined by introducing budget constraints. After incorporating fixed cost and variable cost information, decision makers can compare the total expected cost of an investment strategy with its budget, and the strategy should not be adopted when the budget is exceeded.

Conclusion
In this work, we introduce Efficiency Frontier, an economic assessment framework, to assist decision makers with integrating a complex metric system into a centralized value index. The composition of the value index balances the competing interests among the Key Performance Indicators by considering the strategic priority of an organization. Given the same amount of value gain, the framework favors sustainable decisions and strategies that cost less. The marginal benefit of the decision or strategy might diminish when the cost increases. Alternative decisions or strategies should be considered when the marginal cost exceeds the willingness-to-pay threshold.
To illustrate the practical application of the framework, we have systematically studied the case of the City of Chicago's 311 contact center with a base case analysis and a sensitivity analysis. In the base case, the center underperformed and overspent in the year 2013, as a result of which its performance outcome is excluded from the Efficiency Frontier. The center's performance has improved ever since. However, the return on investment in improving the performance may be diminishing over time. In the sensitivity analysis, we have determined the optimal resource planning strategies given conservative, moderate, and aggressive investment scenarios and quantified the value of new resource planning strategies for decision makers.
Finally, since the yearly-level Efficiency Frontier is constructed based on the mean value of lower level data points, a limitation of this study exists around the variation of the Efficiency Frontier. In other words, the current Efficiency Frontier method might not be feasible for recommending optimal solutions when the variation of the measurement system is high. In future studies, a Value of Information method can be used to incorporate the variation of information into the Efficiency Frontier construction process [25,26]. Similarly, the variation may exist around the effectiveness of the resource planning strategies. Stochastic features could be introduced into the effectiveness prediction process to account for the variation [8].
Availability of data and material The availability of data and material is specified in the Reference section of this article.

Conflict of interest
The authors have no conflicts of interest to declare that are relevant to the content of this article.
Employment All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.