Monitoring changes in individual surgeon’s workloads using anesthesia data
- First Online:
- 1.6k Downloads
We investigated whether changes in the number of cases performed by surgeons can be used as an appropriate surrogate for anesthesia departments’ billed units.
We used both number of cases performed and the American Society of Anesthesiologists’ Relative Value Guide™ (ASA RVG) units to assess all operating room anesthetics of an anesthesia group for two sets of 13 four-week periods. The units correspond to Canadian basic units and time units.
Although the number of ASA RVG units is an economically important variable that quantifies perioperative workload, the number of cases is a suitable surrogate for ASA RVG units when used to monitor individual surgeons. The pooled mean Pearson correlation coefficient between the two variables was r = 0.95, with 95% confidence interval 0.94 to 0.96. In addition, there were essentially none to very weak pairwise correlations among surgeons.
Informal hospital analyses of relative changes in a surgeon’s caseload over one year using anesthesia workload data or anesthesia billing data will generally give equivalent results. The principal importance of our findings is that they can be used by anesthesiologists, specifically department heads, in their role as part of operating room committees. Such committees institute plans to revise the caseload of one or a few surgeons, and they then evaluate the results of those plans. The findings of this study are applicable to all anesthesia groups and may be especially valuable to the heads of anesthesiology departments who do not have the data to repeat our analyses.
Suivi des modifications de la charge de travail des chirurgiens à l’aide des données d’anesthésie
Nous avons cherché à savoir si les changements dans le nombre de patients opérés par les chirurgiens pouvaient se refléter fidèlement dans les unités de facturation des départements d’anesthésie.
Nous avons utilisé à la fois le nombre de cas réalisés et les unités du guide de valeur relative de l’ASA (ASA RVG, American Society of Anesthesiologists’ Relative Value Guide™) pour évaluer toutes les anesthésies en salle d’opération pour un groupe d’anesthésie pendant deux périodes de 13 semaines chacune. Les unités correspondent aux unités canadiennes de base et aux unités de temps.
Bien que le nombre d’unités ASA RVG soit une variable économique importante qui quantifie la charge de travail périopératoire, le nombre de cas reflète le nombre d’unités ASA RVG quand il sert à suivre l’activité de chaque chirurgien. Le coefficient moyen de corrélation regroupé de Pearson entre les deux variables était r = 0,95, avec un intervalle de confiance à 95 % compris entre 0,94 et 0,96. De plus, les corrélations pairées pour chaque chirurgien étaient rarement nulles ou très faibles.
Des analyses hospitalières informelles des variations relatives du nombre de cas à la charge d’un chirurgien sur une année donneront des résultats équivalents à ceux que produisent les données sur la charge de travail d’anesthésie ou sur la facturation d’anesthésie. Ces résultats sont surtout importants en ce qu’ils peuvent être utilisés par les anesthésiologistes, essentiellement les chefs de départements, dans leurs rôles de membres des comités de salles d’opération. De tels comités établissent des plans pour réviser la charge de travail d’un ou de quelques chirurgiens et ils évaluent ensuite les résultats de ces plans. Les constatations de cette étude sont applicables à tous les groupes d’anesthésie et peuvent être particulièrement intéressantes pour les responsables des départements d’anesthésie qui ne disposeraient pas des données pour reproduire nos analyses.
Anesthesiologists serve on operating room (OR) committees, and often heads of anesthesia departments are chosen to represent the interests of their departments at these meetings. The focus of many OR committee meetings is on initiatives to increase caseload, and it could take six months to one year to implement these changes. The decisions could involve moving a surgeon’s cases from one facility to another, the purchase of equipment, or hiring additional personnel. These efforts are often carried out incrementally (e.g., initially renting a capital item on a trial basis). The head of the anesthesiology department needs to be engaged because decisions could affect the anesthesia department’s long-term plans for providing adequate personnel1 and the consumption of anesthesia drugs and supplies.2 However, the anesthesia department often has difficulty interpreting the decisions of OR committees. Goals are often described in terms of the percentage or number of expected changes in a surgeon’s cases, while the influence on the anesthesia workforce and drugs/supplies is related to units of work.1 If cases increase by 30%, the same increase may not occur for basic and time units.
It makes economic sense for OR committees to be making these decisions. Small changes to the OR master surgical schedule can balance OR time successfully among surgical specialties.3 Small changes can predict and balance use of surgical equipment (e.g., a robot) and workload in surgical wards and intensive care units.4,5 Small changes can also influence resident education.6 Additional block time (OR resources) can be planned appropriately for certain individual surgeons to increase the number of cases of targeted procedures.7–9 For example, if the goal is to perform more total hip replacements, then a larger budget would be needed for implants, and unless their current OR utilization is low, more block time would be required for joint replacement surgeons.
It makes statistical sense for OR committees to consider changes in caseloads. Provided analysis is performed using combinations of four-week periods and/or quarters of the year (see Appendix), such statements can include standard errors and/or P values (e.g., by way of a test of trend over time using a rank correlation coefficient).10
The objective of our paper is to test whether cases can be substituted for anesthesia units when considering changes in surgeon workloads over a one-year period. Since OR committees typically consider caseloads of one surgeon at a time, we also chose to evaluate whether changes in most surgeons’ caseloads are statistically independent over a one-year period.
The remuneration of anesthesiologists in the United States for clinical OR care is based on two factors, the number of base units of a procedure that reflects the intensity of work and the number of time units that reflects the actual hours of work.2 Base units are determined by the American Society of Anesthesiologists’ Relative Value Guide™ (ASA RVG). The ASA RVG data studied were obtained from one anesthesia group involved with all of the non-obstetrical ORs. The anesthesia group provided services for all cases performed by the surgeons in which there was an anesthesia provider present. There was one time unit for each 15 min period.
A one-year period was studied as one year is a typical assessment period within hospitals. We studied a “recent” year, Sunday July 5, 2009 through Saturday July 3, 2010, and to assure that our results were not simply an aberration of one period, we also studied a “distant” year, Sunday July 16, 2000 through Saturday July 14, 2001. Each year has 13 four-week periods. In the Appendix, we review the reasons for using four-week periods. Analyses were completed using the anesthesia group’s billing data for all surgeons performing at least two cases during each of the 13 four-week periods in one or both years. There were 93 surgeons studied for the recent year and 77 surgeons studied for the distant year. There were 18,644 cases of 337,736 units for the recent year and 14,646 cases of 270,239 units for the distant year.
Analysis of cases vs workload for individual surgeons
Each surgical case is assigned a certain number of ASA RVG base units depending on the physiological complexity of the procedure(s). For example, anesthesia for an adenoidectomy and tonsillectomy has five base units while anesthesia for a coronary artery bypass graft has 20 base units. In addition, each case was considered to have 0.0667 additional units per 1.0 minute, where 0.0667 = (1 unit/15 min). The economic value of the case to the anesthesia group is proportional to the total number of units, base plus time. Drug and supply usage and other costs are proportional to ASA RVG units.2
The number of ASA RVG units during each four-week period is the end point of relevance to the anesthesia group and a good measure of the hospital’s perioperative workload. On the other hand, the number of cases performed by a surgeon during each four-week period is easy to track and easy for non-anesthesiologists to interpret.
Changes in caseloads of individual surgeons are an accurate surrogate for changes in the surgeon’s ASA RVG units
Prior analysis of the workloads of groups of surgeons provides support for the hypothesis. At one hospital, proportional increases in the total number of cases among all surgeons closely predicted proportional increases in total ASA RVG units over a period of 26 years.1 At multiple hospitals, annual caseloads of representative procedures predicted total specialty workloads (e.g., lung resection predicted all of general thoracic surgery), whether workloads were quantified by ASA RVG units, relative value units, OR hours, or charges.11,12
The variable influencing tactical (once a year) decisions involving surgeons without changes in caseload over time is the variability among periods in the surgeon’s workload or caseload. The coefficient of variation (standard deviation/mean) was used as a measure of the variability of workload because it combines information about the signal (the mean) and the noise (the standard deviation). For each combination of surgeon and year, the coefficient of variation was calculated for the 13 counts of cases (13 four-week periods in each year) and for the 13 sums of ASA RVG units.
For most surgeons and years, coefficients of variation for caseloads are similar to coefficients of variation for ASA RVG units among four-week periods
The difference of the coefficients of variation of caseloads and ASA RVG units was calculated for each surgeon. A random effects model, with surgeon as the random factor, was used to pool the differences in the coefficients of variation among surgeons for each of the two years studied.13,3
Correlation analysis can be used as a second method to assess whether these two variables, caseload and ASA RVG units, convey the same information.
For most surgeons and years, caseload and ASA RVG units are highly correlated (Pearson r > 0.80) over multiple four-week periods
Statistical independence of surgeons’ caseloads
An example from national surgical rate data suggests that the caseloads of different surgeons can be correlated over relevant (one year) time periods.16 If workloads of all pediatric otolaryngologists varied seasonally (e.g., because young children develop more ear infections during the winter months),16 a change in the caseload of one surgeon would predict changes in the caseloads of the other pediatric otolaryngologists. Nevertheless, prior analyses from two hospitals suggested that the preceding example16 using national data does not apply to individual hospitals.1,10
During relevant (one year) time periods, caseloads of surgeons are uncorrelated or correlated so weakly as to be managerially irrelevant for predicting the caseload of an individual surgeon.
To interpret the prior studies,1,10 consider the decision to staff another OR (e.g., hire another anesthesiologist) when an upper percentile of the workload of one or multiple specialties exceeds a threshold, such as eight hours of cases, including turnovers, per OR per work day. Calculation of that upper percentile is accurate and valid.10 The method assumes that the workloads of specialties over 12 four-week time periods (i.e., slightly less than one year) are the summations of the workloads of multiple surgeons with statistically independent workloads.1,10,17
Over relevant (one year) time periods, very few surgeons’ caseloads are significantly correlated with caseloads of other surgeons
An omnibus test for pairwise correlation was carried out.E In addition, we tested whether the absolute value of each Pearson correlation coefficient differed from zero.
During relevant (one year) time periods, very few surgeons would have at least one other surgeon whose caseload predicts the surgeon’s caseload
The coefficient of variation was calculated for each surgeon’s number of cases and total ASA RVG units. During the recent year, the mean pairwise difference (% coefficient of variation of cases − % coefficient of variation of ASA RVG units) was 0%; 95% confidence interval (CI) −2% to 2%; P = 0.36 versus 0%. During the distant year, the difference was −1%; 95% CI −3% to 1%; P = 0.28. Thus, analysis by cases does not result in a significant increase in proportional variability relative to analysis by ASA RVG units. Hypothesis #1a is supported.
The Pearson correlation coefficient was calculated between each surgeon’s number of cases and total ASA RVG units. Over the recent year, the pooled mean Pearson correlation coefficient was r = 0.95; 95% CI 0.94 to 0.96. The results for the distant year were identical. Thus, analysis by cases and workloads are interchangeable for most surgeons, supporting Hypothesis #1b. The results for Hypotheses #1a and #1b support Hypothesis #1.
For the recent year, the null hypothesis of an overall lack of pairwise correlations among surgeons’ caseloads was rejected (P = 0.010).6 However, given the sample size, a strong effect would have resulted in a tiny, much smaller P value (e.g., P < 10−12).F Hypothesis #2a is equivocal for that year. None of the individual correlation coefficients was statistically significant at P < 0.05.F Hypothesis #2b is supported for that year.
During the distant year, the Hypothesis #2a of an overall lack of pairwise correlations among surgeons’ caseloads is supported (P = 0.099).7 In addition, none of the individual correlation coefficients was statistically significant at P < 0.05. Hypothesis #2b is supported.
Our results show that informal hospital analyses of relative changes in a surgeon’s caseload over a period of one year will generally give equivalent results using anesthesia workload data or anesthesia billing data. Number of cases can be substituted for ASA RVG units. The finding is not only useful for anesthesiologists but it is also useful for hospitals, since increased units are proportional to increased drug costs, postoperative care, etc.2 The implication of our results is that this holds for cases too. The findings are predicated on three important limitations. First, the hospital’s workload should be relatively stable during the year studied, e.g., not closing half of the ORs (See Results, Footnote F). Second, four-week time periods should be used in the analyses (e.g., assessment of trend over time uses n = 12 four-week periods).1,10 Third, the study should occur over relevant (e.g., ≤ 1 year) time periods. Over many years, there can be strong correlations among surgeons based on technological change (e.g., declines in cases of coronary artery bypass graft surgery).1 Month to month, staffing needs are determined and then staff scheduling is performed.20–22 Staffing needs depend on the combinations of surgeons who will be working, the need for anesthesia personnel at non-OR locations, etc.
Our results are applicable and likely especially valuable to those anesthesiology departments without the data to repeat our analyses. For example, a department may not have all OR anesthesia billing processed via one accounting office. The anesthesia department may not provide care for all of its surgeons’ cases. Billing software may not differentiate between OR and non-OR locations. We have shown that the heads of anesthesia departments can reasonably assume that plans for modifying caseload as well as committee reports evaluating changes in caseload over a one-year time period for individual surgeons both reflect corresponding changes in anesthesia units.
Survey* of anesthesiologist, surgeon, and nursing operating room managers about using the results of the analysis
A. “Is there a sense of community, or a sense of familiarity, or a comfort level in the environment where you work at Upstate† that makes it typical for co-workers and colleagues to share, or be aware of, at least some details about each other’s personal situation and circumstances?”
Most‡ participants answered Yes (18/19, P < 0.001).
“Would you feel comfortable approaching a surgeon whose cases have declined at Upstate,(†) or would you be concerned that you might be overstepping their boundaries or their expectations about your role and their role?”
Most participants answered Yes (16/19, P = 0.002).
“Would you feel comfortable approaching a surgeon’s colleagues or other staff that work in the OR, such as nurses or technicians, if that surgeon’s cases have declined at Upstate,† or would you feel that was overstepping understood boundaries or expectations about your role and their role?”
Most participants answered Yes (15/19, P = 0.010).
B. “If the highest volume surgeon did no cases for one week, you would probably think nothing of it, or maybe assume that they were on vacation. On the other hand, if they did no cases for 12 weeks, you would want to know about it. With that in mind, what threshold would you want to be used for a monitoring report? In other words, within how many weeks would you want to know that a surgeon’s cases had declined to zero?”
Most participants wanting to receive the monitoring report§ on a regular basis wanted to know within one to three weeks (8/8, P = 0.004).
C. “What do you think are the three most common reasons for a sudden decline in the number of cases a surgeon is doing at Upstate?†”
Most participants selected vacation (e.g., “going out of the country”) (17/19, P < 0.001).
Most participants also selected family leave or sick time (15/19, P = 0.010).
D. “Would you want to know if a surgeon’s cases per month had declined to zero? One?”
Most participants wanting to receive the monitoring report§ on a regular basis wanted to know of a decline both to zero cases (7/8, P = 0.035) and to one case (7/8, P = 0.035).
The terminology in the United States is “base units”. “Basic” is the Canadian term. Pp GP70-71 of the General Preamble to the Schedule of Benefits for Physician Services September 1, 2011, Available from URL: www.health.gov.on.ca/english/providers/program/ohip/sob/physserv/genre.pdf (accessed December 28, 2011).
In Canada, the value of time units depends on the duration of the case. Reimbursement is also increased if the case is performed during the evening or on a weekend or holiday. Rules in the United States, which differ from those in Canada, have been used as the basis for this study.
The analysis was analogous to a meta-analysis with each surgeon providing a point estimate (difference of the coefficients of variation) and a standard error. Each standard error was the difference in the coefficients of variation. The standard errors were calculated asymptotically based on a normal distribution.13 The variance of the difference of the coefficients of variation among surgeons (τ2) was estimated by Restricted Maximum Likelihood Estimation. Since the estimated τ2 equalled zero, a fixed effect meta-analysis was effectively performed. The 95% confidence interval was calculated with (i) the mean being the weighted mean difference between coefficients of variation, (ii) the weights being proportional to the inverse of the standard error of those differences, (iii) the standard deviation being the weighted standard error of the differences, and (iv) the corresponding Student’s t-statistic considering degrees of freedom being equal to the number of surgeons meeting the inclusion criterion minus one.
The Fisher transformation of the Pearson correlation coefficient between each surgeon’s 13 counts of cases and 13 sums of ASA RVG units was calculated. The standard error of the Fisher transformation was calculated asymptotically.14 The Fisher transformation14 was used because the random-effects meta-analysis assumes a normal distribution for the probability distribution of the end point among surgeons. Restricted Maximum Likelihood Estimation found a statistically significant but modest heterogeneity among surgeons’ correlation coefficients (τ2 = 0.40). Analysis of τ2 was therefore repeated using DerSimonian and Laird’s method of moments as a sensitivity analysis.15 The two methods gave point estimates and confidence intervals (CI) that matched within 0.01.
Analyses were repeated using different inclusion criteria. The second analysis included those surgeons who performed at least 26 cases at any time during the recent year, where 26 = 2 cases × 13 four-week periods. There were 140 such surgeons. The sample Pearson correlation matrix was 140 by 140. The matrix was symmetric above and below the main diagonal. All correlations along the main diagonal equalled 1.00. There were 9,730 non-redundant correlations where 9,730 = 140 × (140 − 1)/2, the number of elements below the main diagonal. The data were tested against the null hypothesis that the observations were random samples of 13 observations from a 140-dimensioned multivariate normal distribution with all 140 variables independent of one another. In other words, the sample Pearson correlation matrix was compared with the 140 by 140 identity matrix (i.e., ones along the main diagonal, otherwise zeros). This test was performed by comparing the sum of the squares of the 9,730 standardized Fisher transformations of the Pearson correlation coefficients with a corresponding Chi square distribution.18 There can be bias in the estimated P value because there are far fewer observations (n = 13) than groups (k = 140). Therefore, calculations were repeated using an alternative formula that relies on the second- and fourth-order cumulants of Hotelling's series expansion for the Fisher transformation.14
With n = 93 surgeons, there were 4,278 correlation coefficients, where 4,278 = n × (n − 1)/2 comparisons. The P value = 0.008 for the omnibus test with the use of Hotelling’s series expansion (see Methods, Footnote E). The analysis was repeated with a weaker inclusion criterion to confirm that significant correlations could be detected, if present.E The results of the omnibus test were: P = 1 × 10−12 without the use of Hotelling’s series expansion and P = 5 × 10−13 with its use. The correlation coefficient was significant, r = 0.94. The first surgeon of the pair had four to 14 cases during each of the first seven of the 13 four-week periods of the recent year, then zero or one case during each of the last six periods of the year. The other surgeon had six to 28 cases during each of the first seven periods, then zero cases during the last six periods. Thus, the behaviour detected was simply two of 140 surgeons leaving the hospital at nearly the same time.
With n = 77 surgeons, there were 2,926 correlation coefficients, where 2,926 = n × (n − 1)/2 comparisons. The P value = 0.087 for the omnibus test with the use of Hotelling’s series expansion.E
- 18.Steiger JH. Testing pattern hypotheses on correlation matrices: alternative statistics and some empirical results. Multivar Behav Res 15:335-52.Google Scholar
- 19.Samiuddin M. On a test for an assigned value of correlation in a bivariate normal distribution. Biometrika 1970; 57: 461-4.Google Scholar
- 25.Vasilakis C, Sobolev BG, Kuramoto L, Levy AR. A simulation study of scheduling clinic appointments in surgical care: individual surgeon versus pooled lists. J Oper Res Soc 2007; 58: 202-11.Google Scholar