1 Introduction

NHS Direct Wales (NHS DW) provides 24 h telephone access for answering health queries in Wales. Members of the public can ring in and seek medical support and/or advice on a whole range of problems. The service is provided for all the Health Boards in Wales by a single virtual call centre run from a main location in Swansea with satellite locations in Aberystwyth and Bangor. In 2014 (January 6th 2014 – January 4th, 2015) NHS DW call volume data indicated that 257,190 calls had been received from patients across Wales. Each call to NHS DW is given a priority code according to the issue being addressed and its urgency. It is therefore important to consider the distribution of incoming calls by their priority, as the priority allocated to the call dictates each call’s path through the system and the staff needed to process the call. The outcomes associated with the calls to NHS DW are wide-ranging, and can include transfer to the ambulance service, referral to a General Practitioner or Dentist, advice on self-care, provision of healthcare information and signposting to a particular health service.

During Out of Hours (OoH) times (evenings and at weekends) patients in Wales should be able to access their local General Practitioner by phoning their GP Out of Hours (GPOoH) service. Each health board in Wales is responsible for their own GPOoH service and these services vary across health boards. Typically, a call is triaged (by a nurse or GP) who then prioritises the call and arranges an appointment accordingly. The patient may be offered advice over the phone, see an Out of Hours GP at a local treatment centre, or receive a home visit. Whilst, the main outcomes associated with GPOoH calls are to receive phone advice, attend a treatment centre or have a home visit, there are others, and some patients may require an ambulance or be directed to a hospital. The call volume to the Out of Hours services in Wales, collected by the Advance Computer Software Group, during 2014 was more than double the call volume experienced by NHS DW, with 566,066 calls received in the same time period. In some cases, patients have historically contacted both services for the same episode of care. In the future, patients will only be able to access GPOoH services via a single phone number.

In April 2013, the Welsh Minister for Health and Social Services, committed to the roll out of a service that would deliver the functions of both NHS DW and local Out of Hours arrangements in Wales [1]. The service would be accessed by a single non-emergency number, ‘111’. A similar initiative had already been implemented in England and Scotland. The Welsh Minister for Health and Social Services recognised that similar initiatives in England had met with varied success, some achieving the intended results, others adding to the workload of the local healthcare community rather than reducing it [1]. Further commitment to the proposed ‘111’ service was reiterated in the Plan for Primary Care in Wales [2] and during a Ministerial Advisory Committee in November 2015.

The introduction of a ‘111’ service in Wales is intended to combine NHS DW and the GPOoH services on a pan-Wales basis using the existing NHS DW call centre and staff in conjunction with the existing OoH ‘hubs’ in each Health Board location. Considering the call volume for 2014, the proposed service would need to be able to answer over 800,000 calls a year (which is enough to keep seven or eight people busy on a 24/7 basis – based on the unrealistic assumption that there are no peak times and that a call lasts no longer than five minutes).

The main aim of this paper was to highlight how mathematical modelling (in particular Discrete Event Simulation) and data analysis of the current NHS DW and GPOoH call volume could be used to explain how a proposed rollout of ‘111’ could affect the call centre workforce planning needs of two of the seven health board areas in Wales: Cwm Taf and Hywel Dda University Health Boards which when combined serve a total population of approximately 660,000 people (20% of the population of Wales).

The purpose of our case study is to support the merger of the two telephony systems from both an organisational and service delivery perspective, by developing a computer based simulation to model the impact on service levels and staffing at each stage of the transition (merger), thus avoiding some of the difficulties experienced in other parts of the United Kingdom. A review of English and Scottish experiences of implementing ‘111’ services, quickly highlights the importance of understanding the impact of the merger both on total call volumes, and the seasonality of those volumes when considering staffing levels. In the English experience significant service level issues were encountered as a consequence of failing to fully consider the seasonality of the call volume and as a consequence the staffing needed to address that seasonality was under-estimated [3]. It is therefore essential to understand the call volume data ahead of developing the models that aim to replicate the call volume behaviour. Developing accurate models relies on good quality data and in-depth statistical analysis. The case study discussed in this paper is reliant on the data analysis conducted on over 75,000 calls to NHS DW and GPOoH in Cwm Taf and 95,000 calls in Hywel Dda.

The project management team behind the implementation of the Welsh ‘111’ service specifically requested the development of a simulation model that could be used to explore a range of possible scenarios relevant to the merger of the OoH services with NHS DW. The ‘111’ team recognised the benefits that a simulation model could bring in helping to forecast future call patterns and examining the effect of changes, such as in call handling times, on call flow and workforce assumptions. They also recognised that the visualisation of the current and proposed systems would be beneficial when they communicated the proposal to other stakeholders such as government ministers and policy makers.

The ‘111’ Programme Team wanted to know how a combined telephony system might work for patients trying to access Out of Hours health care services and advice, and what staffing needs would be required to provide the services during the transition to the new ‘111’ service. As the calls to both of the current services vary by priority, call length and staff involved, this was no mean task and led to using simulation techniques to answer the problem posed.

The simulation model was utilised to examine three different ‘What-if’ scenarios dependent on whether the two existing services were fully integrated (merged) or partially integrated during specific busier times in the week (7 pm – 11 pm on weekday evenings and 8 am – 6 pm at weekends):

  • What would happen to the demand on staffing resources if a fully integrated service was provided?

  • What would happen in terms of staffing resource if a partially integrated service was provided?

  • What would happen if the call volume to a partially integrated service increased by 20% on current levels?

The third scenario was to examine the effect on workforce planning from the proposed merger if the call volume increased by 20% as had been experienced in Scotland when their ‘111’ service was introduced [4]. The ‘111’ Programme Team in Wales were keen to undertake a robust analysis that would prevent the same happening to them as had happened in some areas of England, i.e. staff struggling to cope with the extra demand.

The call centre will be providing a combination of service levels during the transition to ‘111’: (i) the current NHS Direct Wales service for the health boards awaiting implementation of the new service and (ii) the ‘111’ service for the health boards that have transitioned. The merger of the two services is intended to improve the overall performance of both services in terms of service level improvements (call back times and outcomes) and cost efficiencies. The percentage change in staffing levels for the key staffing groups (nurses, dental nurses, health information workers, call handlers and Out of Hours staff) was determined by comparing the staff levels from each scenario with the baseline levels from the simulation model.

The paper is organised as follows. In Section 2, the background to the project is discussed. A literature review centred around modelling call centres is presented in Section 3. In Section 4, the statistical analysis of the call volume data is described. The current services are described in Section 5 prior to the model’s introduction in Section 6, which is then validated and verified. In Section 7, the results from the ‘What-if’ scenarios are discussed before the assumptions and limitations to the model are discussed in Section 8. The paper concludes, in Section 9, with a discussion on the overall findings from the case study and the possible future direction of further research.

2 Background

The project described in this paper focuses on the planned introduction of ‘111’ and was conducted within the Aneurin Bevan Continuous Improvement (ABCi) Modelling Unit, Aneurin Bevan University Health Board (ABUHB). ABUHB is one of seven local health boards in Wales (see Fig. 1) and serves an estimated population of over 639,000, approximately 21% of the total Welsh population. The project, however focuses on the Out of Hours (OoH) call volume in two of the other health boards, Cwm Taf University Health Board (serving Merthyr Tydfil and Rhondda Cynon Taf) and Hywel Dda University Health Board (serving Carmarthenshire, Ceredigion and Pembrokeshire), with estimated populations of 289,400 and 372,320 respectively. The project, also relates to an earlier Pathfinder Project which centred on the call volume experienced by the GP Out of Hours (GPOoH) team and NHS DW in Abertawe Bro Morgannwg University (ABMU) Health Board (serving an estimated population of approximately 520,000 in Swansea, Neath, Port Talbot and Bridgend).

Fig. 1
figure 1

Health boards in Wales

Full size image

A detailed data analysis of the call volume data for NHS DW and GPOoH in Cwm Taf and Hywel Dda University Health Boards has been undertaken and combined with the outcomes from discussions with members of both service providers and the ‘111’ Programme Team. These have informed the development of a discrete event simulation model for each health board area. The simulation models for the two areas have a very similar structure, the difference is in the call volume experienced and how the services are currently staffed. The key outputs of interest from the initial data analysis, the simulation model and a further data analysis were used to examine the workforce structure needed to satisfy different levels of service provision. The ‘111’ Programme Team were keen to understand the workforce needs in general and how the current workforce might need to be adapted in the future when the merger takes place.

3 Literature review

The intended service provision for the ‘111’ system builds on the current models for NHS Direct Wales and the Out of Hours functions seeming to take elements from both approaches as described by Collin-Jacques [5] and from the experiences of the Out of Hours system in Denmark which was set up in 1992, and uses a triage system operated by their General Practitioners [6].

The process envisaged for the proposed ‘111’ service in Wales presents the traditional challenges described by Aksin et al. [7] in terms of call centre operations such as call forecasting on a daily, weekly and monthly basis, workforce planning, rostering and scheduling etc. which may present particular challenges when modelling given the multi-skill nature of the current and proposed services [8]. Considering all of these factors, suggests that understanding both the current service provision and the proposed ‘111’ service is essential and that using traditional analytical approaches would be difficult.

The existing Operational Research (OR) literature surrounding call centres focuses mainly on waiting times and bottlenecks, therefore utilising queuing theory models, typically M/M/s which assumes: A steady state environment; arrivals conform to a Poisson process; a constant Poisson rate; service times are exponentially distributed and customers and servers are statistically identical, and act independently. However, as Koole and Mandelbaum [8], comment that when applied to modern call centres such as that utilised by NHS DW, M/M/s models may be an oversimplification as the incoming calls vary substantially by the type of caller and by the way the service request is dealt with. For example, compared to some other call centre operations, calls to NHS DW and GPOoH are allocated a priority (18 in total) and often have to be handled by more than one type of staffing resource. For example, it is not uncommon for a patient calling in to speak to a call handler who takes their demographic details before passing the call on to a nurse, a GP, a dental nurse or other healthcare professional. Another issue is that the call volume to both NHS DW and the Out of Hours Service shows a strong weekday influence and is affected by Bank Holiday periods and both of these factors need to be considered, especially when considering workforce planning issues. For example, the call volume to GPOoH is typically 8 times more on the weekend than during a weekday evening. Therefore the staffing schedule needs to ensure extra staff are in place for the busier weekend shifts.

Call centres can be thought of as stochastic systems with multiple queues and multiple customer types which leads to a number of challenges when managing these highly variable systems [9], particularly when trying to optimise their performance (e.g. reducing queue lengths and cost). The authors [9] comment on the difficulties associated with understanding the dynamics of these complex systems as the stochastic behaviour of the system needs to be modelled. In modelling the current NHS DW and GPOoH call centres ahead of the proposed ‘111’ service, we are aiming to model eighteen different incoming queues and five different skill sets currently based in 2 call centres. Avramadis et al. [10] suggest that systems that involve many different types of skill mix are best addressed through simulation. In this case study there are call handlers who answer all the calls but may only spend a short amount of time on each call through to GPs who spend longer giving the patient advice over the phone.

The requirement to understand how the two existing telephony systems operate and how they might be combined in the future lends itself to simulation modelling techniques such as Discrete Event Simulation (DES). Pidd [11] suggests that computer simulation is one of three options open to operations managers when seeking to understand the impact of making changes to an existing process or setting up a new one. The other two options are direct experimentation and using mathematical models. As one of the requirements of this case study was to project the number of staff needed in the proposed merged system, a mathematical model could have been used to determine the average staff requirements by staff type and by time of the day. However, as the clients were also interested in the data visualisation offered by DES as well as obtaining information on the queues and waiting times experienced by the callers under different scenarios, the mathematical model approach was not considered in this situation.

In this case study, the current and proposed call centre systems are a series of queues and services, and with the stochasticity involved, benefit from being modelled as a Discrete Event Simulation. In particular, DES enables us to include time dependent arrivals and the variation associated with the service times as well as many different staff group shift patterns. In this case study, the time dependent nature of the arrivals is very important due to the strong weekday and hour of the day pattern in the call volume.

Pidd also describes the benefit of being able to use “What-if” scenarios to examine the possible outcome associated with certain choices. Pidd [11] also comments on the flexibility that simulation can offer in comparison to the more traditional approaches. For example, SIMUL8, a company that specialises in simulation, provided support to NHS24 in Scotland to test the benefit of certain proposals in advance of implementation, thus avoiding potential costs. In this case study, we aim to evaluate different staffing strategies ahead of the merger of NHS DW and GPOoH.

Akhtar & Latif [9] suggest that Discrete Event Simulation (DES) is a viable option for accurate performance modelling and subsequent decision making. In the absence of these models, managers find it difficult to explore “What-if” situations on a daily basis and visualise the consequence of specific process changes before they are implemented. DES offers more immediate clarity and confidence by the end users and the ‘111’ Project Team were very enthusiastic from the offset to see the problem visually as they needed to convey the complex nature of the existing systems to other stakeholders including government ministers and policy makers. In particular, they were keen to be able to use the tool to examine where bottlenecks in the current operations occur and how they might be exasperated in the proposed merger.

Avramidis & Ecuyer [12] suggest that simulation modelling could have a central role in the decision making associated with managing call centres as the approach can examine the complexity and uncertainty in a system. Mehrotra et al. [13] also discuss how call centres are becoming more complex and that the simulation models developed need to consider both multiple-queues and multi-customer systems as we have in this case study.

Whilst simulation modelling related to call centres has progressed over the years to consider agent-based modelling [14], the models discussed in previous literature often concentrate on a single type of incoming queue. In this case study, we consider both multiple queues associated with the different level of priorities attached to the incoming calls from both NHS DW and GPOoH and a multi-skill workforce answering the two sets of calls. Ibrahim et al. [15] highlight that stochastic simulation provides a more realistic evaluation tool when applied to multi-skill centres as in our case study where we have five different staff types: GPs, nurse advisors, health information advisors, call handlers and dental nurses.

Discrete Event Simulation was considered as an appropriate modelling approach in this case study because it enabled the call pathway of each call, whether to NHS DW or OoH to be represented as a sequence of events, resourced by different staffing groups. The visual representation of the processes involved allows workforce planners, clinicians and the ‘111’ Programme Team to visualise how the current service is delivered in each setting and how the proposed service may operate. This was of particular importance to the ‘111’ team. Building such a qualitative representation of the process(es) was the first objective of our approach. Another benefit of simulation in this case study is that ‘What-if’ scenarios could be undertaken to assess the particular workforce needs given a transition from two separate healthcare telephony systems into one combined service. In this case study, the discussions around the “What-if” scenarios were beneficial as it enabled the team a means by which to investigate how a new service might look and what staff would be needed to ensure that it runs effectively. Using modelling to assess several possibilities of operating the combined NHS DW and OoH services rather than doing multiple test programmes or Plan Do Study Act (PDSA) cycles in the real world was the second objective in our research. A PDSA cycle is a tried and tested improvement methodology, often used in manufacturing industry and more recently healthcare, where a team plans a chosen intervention, tests it on a small scale and then reviews it before deciding how to proceed. PDSAs can be thought of as the improvement equivalent of testing “What-if” scenarios. The ‘111’ team were keen to consider different “What-if” scenarios to enable them to examine what might happen under different levels of service provision (fully integrated vs. partially integrated service) and efficiency (different call handling times).

The case study presented here aims to supplement the existing call centre literature with a description of the data analysis and simulation modelling undertaken to reflect the call volume experienced by two separate call centres prior to their proposed merger into a single service. Whilst the approach is not necessarily new, it provides an illustration of the scope of simulation modelling in relation to multi-queue (e.g. 18 different call priorities) and multi-skill (e.g. 5 different staff groups) call centres.

4 Statistical analysis of the call volume experienced by NHS Direct Wales and GPOoH

The purpose of this paper, is to understand the impact of merging two telephony systems, NHS Direct Wales and the Out of Hours service, on staffing levels in two of the seven health boards, Cwm Taf University Health Board and Hywel Dda University Health Board.

The data sets were provided by the ‘111’ Programme. The NHS DW datasets contained data on the calls that came into NHS DW for Cwm Taf and Hywel Dda as well as the remaining health boards. A total of 257,190 calls came through to NHS DW between 6th January 2014 and the 4th January 2015. Of these, our case study focuses on the 26,369 calls from Cwm Taf and 28,252 from Hywel Dda University Health Board areas. The calls associated with the remaining health boards also featured in the preliminary analysis but are not discussed in this paper.

The data provided, included: the date and time of the call, the call priority, the type(s) of staff member(s) that handled the call and the time the staff member spent on the call. The data, whilst non-identifiable, also included patient details such as age, postcode and the symptom that the patient phoned up with. This enabled demographic analysis to be undertaken, and highlighted differences between the health board areas in terms of the age profile of the callers that use the service and the services they required. For example, in one health board, almost 30% of calls were concerned with dental enquiries whilst in others, the percentage was much lower at 5%.

The GPOoH data sets for Cwm Taf and Hywel Dda University Health Boards covered the same time period. In comparison to the NHS DW call volume, a total of 50,981 calls came through to the GPOoH service for Cwm Taf University Health Board; almost double. In Hywel Dda, there are two OoH service providers, one covering Carmarthenshire, and the other, Ceredigion and Pembrokeshire. However, in this paper, the combined call volume from the two service providers was analysed and used to inform the model for Hywel Dda University Health Board. A total of 67,582 calls came through to the GPOoH services in Hywel Dda. In each of the seven health boards, the call volume to the Out of Hours Services was much higher than to NHS DW.

As with the NHS DW call data, the non-identifiable patient data from the OoH service providers also contained details such as age and the condition that the caller was enquiring about. Preliminary analysis, also included the age profile of callers using the service, which has subsequently informed a further examination of the OoH services needed for particular age groups. For example, the analysis showed a large number of calls related to children under 5 years of age, adults in their twenties and adults over 70.

Weekly, daily and hourly time-dependency analysis was completed in order to establish patterns and trends in the call volume experienced by NHS DW and the GPOoH services throughout the day/week. This was particularly important as the current services have different operating times. NHS DW is available 24 h a day, every day, whilst the Out of Hours service runs from 6.30 pm each weekday evening through to 8 am the next morning and runs throughout the whole of the weekend period. Both services saw much higher call volumes during the weekends compared to the weekday evenings with Saturday mornings being particularly busy. The arrival patterns for each call priority were analysed and implemented in the simulation model with the distributions updated hourly to reflect the time-dependent nature of the call volume experienced by both service providers.

Further examination of the call volume data and the types of service offered under the current OoH provision was conducted. Table 1 describes the services associated with each of the two health board areas. For example, in both health board areas, the majority of the calls were associated with outcomes requiring an appointment at a treatment centre, or advice from a doctor with smaller numbers resulting in a home visit or nurse advice. In Cwm Taf University Health Board, approximately 26,000 calls resulted in an appointment at a treatment centre, and 14,300 received advice from a doctor.

Table 1 Case types utilised in Cwm Taf and Hywel Dda out of hours services
Full size table

The daily call volume to NHS DW and the Out of Hours Service from patients within Cwm Taf University Health Board is presented in Fig. 2. The daily call volume ranges between 39 and 111 for NHS DW and between 41 and 490 for the Out of Hours Service. The median call volumes for both services are similar, 72 for NHS DW and 75 for GPOoH. However, the average call volume is much higher in the GPOoH service, 139 compared to 72 in NHS DW as it is affected by the high weekend and Bank Holiday call volumes. There is a strong weekday pattern in the call volume data with the data dominated by the weekend usage. There is also an increasing trend in the amount of calls received in the latter part of the year, particularly around the Christmas Period with the maximum OoH volume recorded on 27/12/2014.

Fig. 2
figure 2

Daily call volume for Cwm Taf University health board (6/1/14–4/1/15)

Full size image

The daily call volume exhibits a similar behaviour in Hywel Dda University Health Board with a strong weekly pattern which is dominated by the weekend Out of Hours calls. In Hywel Dda, the Out of Hours Service is provided by two separate call providers; one covers Carmarthenshire and the other Ceredigion and Pembrokeshire. The call volumes across the two service providers are similar with an annual call volume between 31,000 and 36,000 calls. Carmarthenshire has a daily call volume to NHS DW of between 39 and 111 calls and an average of 72. The OoH service for the area sees between 21 and 345 calls a day, and an average of 87. Ceredigion and Pembrokeshire see between 22 and 70 NHS DW calls and between 19 and 379 OoH calls a day.

As well as considering the daily call volume to each service, further analysis was conducted which considered the priority assigned to each call as each priority of call has a specified service time target associated with it (see Table 2). Knowing both elements was crucial in the development of the call handling simulation model.

Table 2 Call priorities associated with NHS Direct Wales and their expected service standards
Full size table

5 Development of a call handling DES model – the current services

In order to develop the required Discrete Event Simulation (DES) model, it was essential to understand how both of the service providers operate at present. To do this an examination of the service provision of NHS Direct Wales and GPOoH in Cwm Taf and Hywel Dda University Health Boards was undertaken.

5.1 Service provision under NHS Direct Wales (NHS DW)

Under the current service provision, patients may phone either NHS Direct Wales, their GP Out of Hours service, or in some cases both. If the patient chooses to call NHS DW, their call is answered by a trained call handler who takes contact details of the caller, assesses the priority using a set algorithm (if necessary supported by a qualified nurse) and any other relevant information. The call should be answered within 2 min. On average, it takes the call handler 450 s to take the patient’s contact details and assess the call priority. The call handler then routes the call accordingly. If the call is a high priority then it would be transferred directly to a nurse or dental nurse as appropriate; if less so, then it would be placed in a queue for call back from either a nurse, dental nurse or health information advisor as required.

The prioritisation codes used within NHS DW consist of a letter signifying the type of call and a number signifying the urgency (the lower the number, the higher the priority). The prioritisation codes, their description and the expected service standards are described in Table 2. For example, a call transferred from the ambulance service, labelled P0, should be answered within 10 min, whereas the expected call back time for a routine first advice call, labelled P5 is up to 24 h. As the call priority for each call to NHS DW was present in the data, the call volume for each priority group could be analysed and included in the simulation model.

It is important to consider the distribution of incoming calls by their priority, as the call volume for each is very different as can be seen in Table 3. For example, in 2014, there were 12,550 P0 calls and only two P4 calls. It is interesting to see the variety in the calls to NHS DW in terms of call priority, particularly in relation to the dental calls; Hywel Dda University Health Board experiences a far higher call volume than Cwm Taf, approximately 10,000 compared with 700.

Table 3 Annual NHS DW call volume for Cwm Taf and Hywel Dda University Health Boards, by priority
Full size table

The priority allocated to the call also dictates each call’s path through the system. Hence, the distribution of arrivals for each priority directly affects the need for different staff types at any one time. However, the low volumes received in some of the priority groups over the year proved a concern when considering their inclusion in the simulation model, and estimating the staffing resource needed to deal with them. In particular, the inter-arrival time between successive calls of a given priority could be very large and using a simulation model with hourly time slots could lead to an under-representation of these calls in the simulations. Also, trying to evaluate the staffing implications of servicing a small number of calls each week is problematic.

5.2 Current service provision with GPOoH

Out of Hours provision is commissioned by each Health Board, with sometimes more than one provider supporting the area. Cwm Taf University Health Board relies on a single provider whereas Hywel Dda University Health Board uses two, one covering Carmarthenshire and the other Ceredigion and Pembrokeshire.

If a patient chooses to phone their Out of Hours service, the calls are directed to a call centre, manned by nurses and one or more experienced GPs. As with NHS Direct Wales, the calls are prioritised. However, the priorities are only partially aligned with those of NHS Direct Wales. The prioritisation levels used in each of the GPOoH services considered are given in Table 4. The different priority codes for the Out of Hours Services in each health board highlights the need to look at each health board area separately when developing the simulation model, particularly in the data analysis that informs the model parameters. As with the NHS DW call priorities, there are different targets assigned to each call priority.

Table 4 Out of hours call priorities used in Cwm Taf, Carmarthenshire, Ceredigion and Pembrokeshire
Full size table

5.3 Proposed ‘111’ service infrastructure

The service provision envisaged for the proposed ‘111’ system in Wales uses the current NHS Direct Wales operation’s infrastructure in Wales. If the patient calls ‘111’, their call is answered by a non-clinical call handler and prioritised using the current NHS Direct algorithm before being routed to the appropriate health professional. If the caller is symptomatic, the ‘triage’ element of the call will re-route the call to either: the Ambulance Service, a healthcare professional, or an appointment with a GP, Dentist or specialist practitioner.

For calls that come into the proposed service during the evenings and weekends, the triage function could reside at either the NHS DW call centre or at the local OoH centre, or possibly move between them, depending on the workload at each site. The current intention is that during peak periods (e.g. Saturday mornings), a senior practitioner and additional nurse resources could be located at the NHS DW centre to provide an initial prioritisation process, prior to passing some of the calls direct to the local hub of the OoH, whilst carrying out the triage and queue management functions for the remainder. The initial streaming, could identify those calls which as a matter of course will require local GP input. For the other calls, it is anticipated, that after the initial prioritisation, urgent calls will be handed over directly to a nurse (or dental nurse) for triage and non-urgent cases will be dealt with in a subsequent scheduled call-back and either triaged and assessed, or passed to the relevant health care resource as appropriate. The intention is that there will be clear service levels associated with each of the prioritisation codes (time targets for the call back) in the proposed ‘111’ service.

The routes that a call with a given priority progress through are conceptualised in the pathway map presented in Fig. 3. Each box in the pathway represents a process that the call with a given priority goes through. The colour of the box coincides with the boxes at the bottom of the figure which represent the staffing resource allocated to that part of the pathway. For example, the P1 calls are initially handled by a call handler before they are triaged and assessed by a nurse and result in the nurse providing the outcome / disposition of the call. The solid arrows in Fig. 3 indicate a standard / default pathway that would usually be followed. The dashed arrows indicate pathways that would be followed if resources permit it. For example, a P2-P3 call would usually be transferred into a CallBackQ to wait for a nurse to become available. However, if there is no-one ahead in the queue, the call will go direct to a nurse.

Fig. 3
figure 3

Pathway map of the calls that will come in to the proposed ‘111’ service

Full size image

A double-ended arrow indicates possible pathways in both directions. For example, at the triage stage a call can progress to be clinically assessed or returned to a place on the call back queue. The pathway map was discussed and validated with the ‘111’ Programme Team and staff within NHS Direct Wales.

For the highest priority calls that currently come into NHS DW, labelled P0, the symptom is assessed by a nurse and a disposition given. This is illustrated by the top three boxes in Fig. 3. The boxes are coloured white to represent the nursing staff resource used for these calls. The dispositions usually recorded for this priority of call typically relate to: referral to an urgent or same day GP appointment, or referral to the ambulance service.

For all the other priority calls P1; P2 and P3; the dental calls D1, D2 and D3; the quick calls 4QC; H1 through to H5 (health information advice calls); and the Out of Hours calls, the call is answered and registered by a call handler (grey resource box) before being prioritised. The P1 calls are triaged and assessed by a nurse before a disposition is given. The P2 and P3 calls follow a similar pattern in that they are triaged by a nurse and may receive a call back before the symptom assessment is carried out and a disposition achieved. The reason for the call back is that priority is given to the P1 calls which need to be answered within a shorter time window than the P2 and P3 calls.

The dental calls, D1 through to D3 follow a similar pathway to the P2 and P3 calls; the only difference being that they are carried out by dental nurses (dark grey resource boxes). The very quick 4QC calls often only need information provided from the call handler. The health information calls, H1 through to H5 are handled by health information advisors (white, dotted resource box) and may require a call back if the system is busy when the call comes in.

The call pathway for the Out of Hours calls during off-peak hours involves triage by staff in the OoH hub (striped resource box) prior to any call backs and symptom assessments. During peak hours, there would be a nurse triage and call back prior to the symptom assessment element of the pathway (represented by the marble effect process and resource boxes).

A review of the pathway map identified that, in practice, the call centre ran a number of similar parallel operations some with shared resources, but all broadly similar in their function. The ‘Core Loop’, as it became labelled is presented in Fig. 4; the calls have been prioritised and are then streamed, either to a ‘call back queue’, or if urgent, passed to a health professional for Triage and Assessment.

Fig. 4
figure 4

The ‘Core loop’

Full size image

The ‘111’ Programme Team were specifically interested in the call volume and staffing levels associated with a ‘standardised week’, that is, a week that did not include one or more bank holidays. The call volume over Bank Holidays is significantly higher than other times of the year, often higher than normal weekend volumes. Therefore, in order to model a typical week, the call volumes were filtered to remove the call volumes associated with bank holidays, and the days immediately before or after them.

6 The simulation model

A discrete event simulation of the pathway map described in Fig. 3, was produced in Simul8.

6.1 Inter-arrival distributions

The initial phase of the model development considered the arrival of calls into the system, or more precisely, the inter-arrival times between successive calls of a given priority. The inter-arrival time distributions are updated at the beginning of every hour of simulated time in the model to capture the variation in call volume and associated inter-arrival times identified in the data analysis. For example, with the call data from Cwm Taf University Health Board, the inter-arrival time of the highest priority NHS DW calls, P0, varied between a call every 1.83 min on a busy Saturday to one call every 14.45 min on a quieter time. In comparison, the inter-arrival times for the calls to the GP Out of Hours Service for the same health board varied between a call every 1.6 min up to one every 218 min, with the shorter inter-arrival times experienced during the busier weekend periods.

Each inter-arrival time distribution was also reset to prevent the issue of under-representing the low call volumes associated with some of the priority groups such as the health information and medical advice calls. The inter-arrival distributions for both the NHS DW and GPOoH are held within an Excel spreadsheet that can be imported into the Simul8 model. Whilst the structure of the simulation model for each health board is the same, the Excel spreadsheet containing the inter-arrival times is specific to each health board and was determined from the data analysis conducted on each health board’s call volume data. The exact values are not presented in this paper as there are too many to discuss – an inter-arrival time for each hour of each day for every call priority (24 * 7 * 18). The Excel spreadsheets with the inter-arrival times can be made available from the authors on request.

6.2 Service times

Further analysis of the NHS DW call volume data provided estimates of the current service times associated with given call priorities and these varied from 0 min for ‘999’ and 4QC calls up to 47 min to triage and assess P1 calls in the Cwm Taf region and 51 min in Hywel Dda (for the call types see Table 4). Each of the non-zero service time distribution is represented by an average distribution (in minutes) with the mean given in Table 5. The routing of each call is governed by the call priority label assigned to the call.

Table 5 Mean service time distributions (in minutes) used in Cwm Taf and Hywel Dda University Health Board calls
Full size table

6.3 Staffing resource

The staffing resource needed to answer, triage and assess the calls is included in the simulation model and considers how nurses, dental nurses, health information advisors, call handlers and Out of Hours staff (typically GPs and nurses) will be affected under each of the scenarios considered. Shift patterns were applied within the model to drive the behaviour of the system according to the hour of the day and the day of the week. This allowed the workforce planning to be considered. To address the complex nature of incorporating the shift patterns of different staff groups from two organisations, a 24 h dummy shift pattern was utilised and updated every 30 min, to reflect the numbers of staff on shift at any one time.

6.4 Model screenshots

A screenshot of the final Simul8 simulation model is presented in Fig. 5. The left-hand-side captures the call priority groups associated with the current NHS DW service provision and two levels of Out of Hours (OoH) service provision (considering full or partial integration with the NHS DW service). The middle section of Fig. 5 demonstrates the call routing through call prioritisation and subsequent streaming for triage and assessment. The right-hand-side validates the model, ensuring that the call volume associated with each priority group agrees with the real-life values. The final box at the bottom of Fig. 5 captures the clock, and the resources associated with each group of calls. Each section of the simulation model is presented in higher resolution in Figs. 6 (call generation), 7 (the call process), 8 (Validation data) and 9 (Resources).

Fig. 5
figure 5

Screenshot of the simulation model for the proposed rollout of ‘111’ in Wales

Full size image
Fig. 6
figure 6

Screenshot of the call generation section of the simulation model

Full size image

Each phone icon in Fig. 6 represents a call stream of a given priority. There are eighteen in total with sixteen relating to NHS DW call priorities (P0, through to M3) and 2 from GP Out of Hours. The two GP Out of Hours relate to whether the fully or partially integrated service is in operation.

In Fig. 6, each call profile apart from the highest priority P0 calls are routed to a central operating facility where an automated message is played to the caller. This typically lasts one minute and has been modelled in the simulation model with a Fixed Distribution (1 min long). Each call then waits to be prioritised before it is routed to the correct healthcare professional. The prioritisation process takes on average seven and a half minutes and this has been included in the simulation model as an average distribution.

Figure 7 shows the routing of each call from prioritisation through to the healthcare professional that deals with their call. The top row deals with the health information calls H1, H2,.., H5 and are dealt with by a health information advisor. Each of the P1, P2 and P3 calls are routed through to a nurse for first advice. The dental calls, D1, D2 and D3 are routed through to a dental nurse and appear as the bottom row of icons in Fig. 7. The final group of calls that originate from the Out of Hours Service are triaged and if they need further assessment are passed on to the Out of Hours Assessment team in the Out of Hours Hub. The model assumes that 25% of Out of Hours calls will be dealt with easily whilst 75% will need further assistance from the Hub.

Fig. 7
figure 7

Screenshot of the call process section within the simulation model

Full size image

To ensure that the simulation model doesn’t lose any calls, the model includes a validation section as presented in Fig. 8. Each of the eighteen call streams are validated to ensure that the expected number of calls for a given time period are processed in the model.

Fig. 8
figure 8

Screenshot of the validation data section of the simulation model

Full size image

The staffing resources are also captured within the simulation model and presented in Fig. 9 along with the Clock. There are six different staffing resources: call handlers, nurses, dental nurses, health information workers, Out of Hours nurses and Out of Hours Staff (typically GPs). Whilst there are 5 different staff types, the nurses have been divided according to whether they are an NHS DW nurse or one from the Out of Hours team.

Fig. 9
figure 9

Screenshot of the resources section of the simulation model

Full size image

6.5 Running the model

The simulation model was run to simulate the behaviour of a typical week. The reason behind this was that both the clinicians and the ‘111’ Programme Team were keen to see how the staffing patterns altered in each of the scenarios over a typical week before considering times within the year affected by higher call volumes, e.g. Easter and Christmas. The model was run until the utilisation percentage for each of the staffing groups remained stable over consecutive iterations and the associated confidence interval was 95% (100 runs).

6.6 Verification and validation

Verification is the process of ensuring that the simulation model performs in the way that it was intended. The model was developed using a step-wise approach initially considering the core loop (Fig. 4) and then extending it for each call priority. The processes associated with each call priority were checked with experts from NHS DW and GPOoH.

The simulation model also includes an in-built check on the number of calls that have arrived and been processed to ensure that no calls are lost in the system and match the initial data analysis of the call volume data.

Validation is the process of ensuring that the simulation model accurately represents the system under study. The validity of the model was assessed, throughout its development, by members of the ‘111’ Programme Team and specialists within the current service providers, NHS DW and the GPOoH professionals. Detailed discussions with the ‘111’ Programme Team, and staff within the main call centre for NHS DW and the GPOoH services confirmed that the conceptual model accurately reflected how the calls come into both services, how they are processed and how long it takes to process a given call.

The workforce estimates from the baseline and scenario model runs were sense-checked against the estimated workforce derived using an existing forecasting tool within NHS DW. The senior scheduler within NHS DW was satisfied with the estimates provided by the simulation model in each of the model runs for both a typical week and busier Bank Holiday weekend call volume traffic.

Due to time constraints at the time of the model development and data availability issues, a sensitivity analysis was not conducted but this could be considered. One area that was identified as a possible source of sensitivity was the effect of the call handling times on the efficiency of the system. Another area that could also be considered is the availability of the staff during their shift. The ‘111’ team suggested that examining the effect of annual leave and staff breaks on the workforce needed in the new service would be extremely useful.

7 Model investigation & associated discussion

The primary aim of this paper was to investigate a range of ways of providing the necessary workforce to support the proposed roll-out of the ‘111’ service in Cwm Taf and Hywel Dda University Health Boards. The ‘111’ Programme Team were particularly interested in knowing whether extra staffing resource was needed under a given “What-if” scenario. The workforce needs for a combined service could then be estimated and costed ahead of the proposed ‘111’ rollout. The aim was to examine the effect of the different levels of service provision on various aspects of the system and evaluate the benefit, or otherwise, of each level of service provision. Various levels of service provision were explored through a series of ‘What-if?’ scenario analyses. In the first scenario, the extra workforce needed to staff a fully integrated service was assessed. In this scenario, all the call handling and clinical assessment would be carried out in one location. The ‘111’ team described this as the model that they would most like to implement if costs allowed. The next scenario considered the workforce needed if cost wouldn’t allow the fully integrated service; with an integrated service at busy times during the week and a non-integrated service during the quieter times. The third scenario considered the effect on workforce needs if the call volume increased by 20% on current levels and the staff were limited to providing the partially integrated approach.

Initially, the aim was to develop a simulation model that could estimate the exact number of staff of each staff type (call handlers, nurses, dental nurses, health information workers and GPs) needed in each scenario. However, as the simulation was developed it became clear that data on staffing levels for the OoH services was complex and in some areas not available due to its sensitive nature. To address this, the output from each scenario was presented in terms of a % increase in workload for each staff group compared with the baseline model which utilised a dummy 24 h shift pattern for each group of staff. The scenarios were applied separately to each of the two health boards as the current service operation was observed to vary across health board areas in Wales.

The key performance indicators collected for the baseline and “what-if” scenarios were:

  • Nurse utilisation (%);

  • Dental nurse utilisation (%);

  • Health information worker utilisation (%);

  • Call handler utilisation (%);

  • OOH Staff utilisation (%);

  • % of P0 calls queued less than time limit;

  • % of P1 calls queued less than time limit;

  • % of P2 calls queued less than time limit;

  • % of D1 calls queued less than time limit;

  • % of D2 calls queued less than time limit;

  • % of D3 calls queued less than time limit;

  • % of OoH triage calls queued less than time limit.

However, the indicators of most use to the ‘111’ team were the staff utilisation figures.

7.1 What-if 1: the proposed ‘111’ service is fully integrated

The first aspect considered was what might happen if the two services are combined and the proposed ‘111’ service provides all the call handling and the clinical assessment for the entire out of hour’s period (6:30 pm – 8:00 am each weekday, and all weekend). This was described as a fully integrated service and expected to be the most expensive to provide in terms of staffing resource especially given the current high weekend call volume. The results for Cwm Taf and Hywel Dda University Health Boards are presented in Table 6. The scenario considers the fully integrated service operating under the current call volume. The results highlight the percentage increase or decrease in the staffing resource needed to operate the fully integrated service in comparison to the current baseline approach in each health board area.

Table 6 Comparison of model output from ‘What-if 1’ with baseline for Cwm Taf and Hywel Dda University Health Boards
Full size table

In both health board areas there would be an additional staffing resource needed in terms of the nurses and call handlers required to operate the fully integrated service; approximately 24% (95% CI: 21.29% - 25.86%) more nurses for Cwm Taf and 28% for Hywel Dda and up to 51% more call handlers in Hywel Dda. In each health board area, the simulation models highlighted that the extra staffing resource would especially be required during the weekends. However, the extra requirement for call handlers and nurses would be compensated by a saving in terms of the number of OoH staff, typically GPs, needed, as nurses fulfil those roles (a reduction of 65% in Cwm Taf, with 95% CI 62.28% - 67.88%). Under the fully integrated service there would also be a slight saving in terms of the dental nurse and health information worker provision. However, the confidence intervals associated with the dental nurse are +/− 2.8% in the case of Cwm Taf University Health Board and those for the Health Information Workers larger again due to the small numbers of staff used in the original staffing resource pattern.

7.2 What-if 2: the proposed service is partially integrated during weekday evenings (7 pm -11 pm) and weekends (8 am – 6 pm)

The second aspect considered a partially integrated service in that the proposed ‘111’ service would provide call taking and some triage / clinical assessment of calls at specified times during the weekday evenings (7 pm – 11 pm) and at weekends (8 am – 6 pm). The specified time periods were carefully selected after a review of the patterns of call over a 2 year period that confirmed that these were the peak times when patients contact the service so they were selected to maximise the use of professionals during this time. The ‘111’ project team suggested that the workforce planners would require staffing estimates for a combined service during these periods ahead of those for the quieter out of hours’ periods. During the other times, the proposed ‘111′ service would function as if it were the two separate services with the workload divided based on the urgency and severity of the caller’s condition. For example, some groups of patients (e.g. symptomatic children under 5) are likely to require a face to face assessment so it would be important to route these calls to a treatment centre appointment to minimise the number of professionals they come into contact with during the initial phone call and booking process.

The results for the partial integration of the two services for Cwm Taf and Hywel Dda are presented in Table 7.

Table 7 Comparison of model output from ‘What-if 2’ with baseline for Cwm Taf and Hywel Dda University Health Boards
Full size table

As with the fully integrated service, the partially integrated service would require extra staffing resource in terms of the nurses and call handlers with Hywel Dda requiring a higher percentage, particularly in terms of call handlers. As with the fully integrated service, there would be a reduction in the OoH staff requirements as the roles are taken up by the extra nursing resource. Once again, the extra staffing resource would be required to cope with the call volume on weekends.

7.3 What if 3: the call volume increases by 20%

The final area considered was how the service provision might be affected if the call volume to the service increased by 20% as had been experienced in Scotland when they introduced their ‘111’ service [4]. The results for this scenario on a partially integrated service for Cwm Taf and Hywel Dda University Health Boards are presented in Table 8.

Table 8 Comparison of model output from ‘What-if 3’ with baseline for Cwm Taf and Hywel Dda University Health Boards
Full size table

If the call volume coming into a partially integrated service increased by 20% on current volumes, both health boards would need extra staffing resource in terms of the nurses, dental nurses, health information workers and call handlers. This would have a smaller effect on the OoH staffing reduction compared to previous scenarios. This scenario emphasised the importance of understanding the call volume coming into the service and the effect an increase in call volume can have on workforce planning issues.

In each of the three scenarios, extra call handling and nursing resources would be needed to answer and process the calls coming in. The reason being that at the time when the model was developed, it was suggested that the proposed ‘111’ service would be coordinated by the NHS DW call centres and the proposed staffing pattern needed to be compared against the NHS DW shift patterns. In the proposed service the call volume now includes the Out of Hours calls which have higher volumes and tend to favour the weekends. Therefore each scenario sees an increase in the number of staff (particularly call handlers and nurses) needed to answer the extra call volume when compared to the original NHS DW staffing resource.

8 Discussion

In this section we consider the assumptions within the simulation model and some of the limiting factors we experienced, as well as some of the other key learning points highlighted through this case study.

8.1 Key assumptions

The current model assumes that staff are available to answer calls, 100% of their time at work. Future updates could consider what % of time the staff are actually available, thus making allowance for breaks, annual leave, training etc. Initial investigations with the simulation model outlined in this paper suggest that the workforce required would need to increase if the percentage of time a member of staff is processing calls is reduced. As there are several different staff groups, the % of available time for each staffing group would need to be considered. Annual leave and sick leave could also be considered by altering the percentage of time staff are available.

Further investigation, is also needed to examine how staff resources may be shared at different times in the day. For example, with nurses helping call handlers to answer the phones, or calls from the OoH service being streamed into the NHS DW hub.

8.2 Limiting factors

One of the key limiting factors in developing the model relates to the shift patterns used by the GP Out of Hours service. At the time of the project, the service was looking to renew its staff contracts and did not want to provide sensitive information on their current rotas. Having the staff shift patterns would have improved the accuracy and the reliability of the workforce results in the baseline model and each of the three scenarios.

A further limiting factor was that the ‘111’ team didn’t know how the future telephony algorithms for prioritising the calls would work and how long they would take to process a call compared to the current NHS DW system. Therefore, the service times in the simulation model might not adequately reflect those for the new service. Similarly, the time taken to process calls with a new system may be very different from the current NHS DW and GPOoH approach. Both of these will have a knock-on effect in terms of the workforce needed to answer the out of hours’ calls in Cwm Taf and Hywel Dda University Health Boards.

In the current service provision, there is a target to restrict the abandonment rate to less than 5%. However, this is currently not incorporated in the simulation model. This could be considered, but special attention would need to be paid to the proportion of callers who abandon their calls, and try again later, and those that default to another service, such as phoning 999, or attending Accident and Emergency.

A final limiting factor is that the case study described in this paper does not include the financial costs associated with the current call volumes to NHS DW or GPOoH. This could be considered in the future. During the project, the ‘111’ team used the call volume data analysis in conjunction with an existing tool to estimate the financial costs associated with any extra workforce needs.

8.3 What else was highlighted during the course of the project?

The project highlighted the need for good quality data, particularly in terms of the length of the calls and the staff responsible for processing the calls. Having good knowledge from the current service providers was essential given the complexity and magnitude of the call volume data set.

9 Conclusions

The main aim of this paper was to highlight how mathematical modelling and data analysis of the current NHS DW and GPOoH call volume could be used to explain how a proposed rollout of ‘111’ could affect the call centre workforce planning needs of two of the seven health board areas in Wales: Cwm Taf and Hywel Dda University Health Boards which when combined serve a total population of approximately 660,000 people (20% of the population of Wales).

Whilst a mathematical model could have been developed to estimate the average number of staff at a given time of the day, Discrete Event Simulation (DES) was preferred as it could provide staffing requirements and an excellent visualisation of the current and proposed call centre systems.

A Discrete Event Simulation model describing the proposed call handling of the ‘111’ service was designed and then used to consider a number of alternative levels of service provision. This case study describes how simulation modelling was used to model the call volume traffic of two separate services, NHS DW and GP Out of Hours, and the likely effect of their merger into one service, ‘111’. Whilst the approach and techniques described are not necessarily new, this paper aims to supplement the existing call centre literature with a description of how simulation modelling was extremely useful in conveying the complexity of the two original services and how the workforce would be affected by the proposed merger.

The case study concentrated on the percentage change in the workforce needed to operate under each level of service provision. Three scenarios have been presented: partial and full integration of the two current services under current call volumes, and partial integration with an extra 20% of calls. The scenarios were chosen because the ‘111’ Programme Team wanted to examine how the workforce would need to change if the two current services were combined and possibly located in one central location. Each of the three scenarios considered for both health board areas, highlighted the need for extra nursing and call handling staffing resource whilst seeing a reduction in the OoH staffing resource. The final scenario which considered the effect of an increased call volume highlighted the importance of understanding demand and how it can impact on workforce planning needs.

The simulation model provided an important role in aiding the visualisation of two already complex services. The ‘111’ Programme Team commented on how the simulation model helped them understand the complexity involved in bringing two services together and the limitations associated with their current data sets. The simulation model has been used to consider the nature of different types of calls and the workforce who answer those calls. The model was used alongside the ‘111’ teams existing resource planning tools and helped the team make decisions about the workforce needed to operate the proposed service. Using the models to consider different ‘What-if’ scenarios has enabled the ‘111’ Programme Team to estimate the workforce needed under different levels of service provision, such as a fully or partially integrated service.

Undertaking the data analysis alongside the model development has enabled the ‘111’ Programme Team to conduct a further analysis into the call volume data held for NHS DW and GPOoH services in Wales and assist in planning their services going forward. The analysis has also helped the ‘111’ team understand their patterns of demand as well as some of the patient characteristics of those currently using the two existing services. For example, the age profile analysis has highlighted the large numbers of parents of children under 5 using the service, and the symptom analysis highlighted the high percentage of calls relating to dental care. These results have been particularly useful in identifying patient groups that use the service more than others. Understanding the needs of the patients is essential for effective provision of services and the call volume data representing this need has enabled the ‘111’ Programme Team to present their findings at both local service and Ministerial level.

The results of the data analysis and the development of the simulation model have provided the ‘111’ Programme Team in Wales with a confidence in their understanding of the current services and how they need to progress at each stage of the proposed rollout of ‘111’ in Wales. The ‘111’ team have also commented on how the analysis has given them the confidence to explain their assumptions and future plans to other stakeholders connected to the proposed roll out of the new service.

The data analysis of large, call volume datasets and the development of Discrete Event Simulation models can be used to consider the rollout of ‘111’ in other areas of Wales. Initial data analysis has highlighted the difference in both the call volume and the types of calls experienced in other areas of Wales. Therefore, each area should be considered separately rather than a generic model applied to all areas. This area by area approach provides an insight into the call volume in each geographical area and the types of patients that use the current service and may use the proposed ‘111’ service in specific areas of Wales. This depth of knowledge may assist in providing the right types of service in the right demographic areas.

The data analysis and development of a Discrete Event Simulation model has allowed policy makers and planners a means of interpreting the data and information that describe their systems and plan accordingly. They can use the analysis to estimate whether the current systems are adequately staffed to meet the call demand experienced and plan for the future rollout of ‘111’. Throughout the project, the planners and policy makers commented on how the data analysis and model development enabled them to get an understanding of how complex the current services are and what needed to be considered going forward.