Social ties between team members affect patient satisfaction: a data-driven approach to handling complex network analyses

Abstract

Research from outside the medical field suggests that social ties between team-members influence knowledge sharing, improve coordination, and facilitate task completion. However, the relative importance of social ties among team-members for patient satisfaction remains unknown. In this study, we explored the association between social ties within emergency teams performing simulated caesarean sections (CS) and patient-actor satisfaction. Two hundred seventy-two participants were allocated to 33 teams performing two emergency CSs in a simulated setting. We collected data on social ties between team-members, measured as affective, personal and professional ties. Ties were rated on 5-point Likert scales. In addition, participants’ clinical experience, demographic data and their knowledge about team members’ roles were surveyed. Perceived patient satisfaction was measured on a 5-point Likert scale. Data was analysed with a linear regression model using elastic net regularization. In total, 109 predictor variables were analysed including 84 related to social ties and 25 related to clinical experience, demographics and knowledge test scores. Of the 84 variables reflecting social ties, 34 (41%) had significant association with patient satisfaction, p < 0.01. By contrast, a significant association with patient satisfaction was found for only one (4%) of the 25 variables reflecting clinical experience, demographics and knowledge of team roles. Affective ties and personal ties were found to be far more important predictors in the statistical model than professional ties and predictors relating to clinical experience. Social ties between emergency team members may be important predictors of patient satisfaction. The results from this study help to enhance our conceptual understanding of social ties and their implications for team-dynamics. Our study challenges existing views of team-performance by placing emphasis on achieving collective competence through affective and personal social ties, rather than focusing on traditional measures of expertise.

Statistical appendix

Appendix 1: Operationalizing the network

The emergency cesarean section team in the operating room can be described as a network. This network is a fully connected graph with every node connected to every other node in the network. In the case of n = 10 nodes, we have n(n − 1) = 90 directed links in the network. In our analysis, the network nodes therefore, stayed roughly constant (there were small changes, since not all roles were filled in every operation), and what changes from team to team were the set of relationships (links) that represent the relationship between each pair of professional roles.

In understanding how the network affected outcomes, it was not feasible to model the behavior of every single link—we simply did not have enough observations (in these case, operations) to model the significance of individual links. Therefore, we used combined scores for connections within and between groups (Fig. 4). This averaging was also necessary, because the groups varied in size between different emergency cesarean teams. Thus, it was not desirable to draw conclusions related to a specific member of a group. Stated differently, we needed to summarize information from the network using knowledge of the operating workflow.

Fig. 4

A sketch of the team structure of the operating room network. See main text for full details

To avoid visual clutter in Fig. 4, we did not draw all 90 links in the network. Instead, we divided the network into three groups characterized by a high level of physical interactions during each section. These groups were: The obstetrics group consisting of midwives and obstetrics specialist shown in blue, the anesthesiology group consisting of doctors of anesthesiology as well as a nurse specializing in anesthesiology shown in green, and finally a group of scrub nurses.

Within each of the three groups, we drew all links. The fact that all links had arrows at both ends signifies that all ties were bidirectional, meaning that relationship A → B is different from B → A. Between the three groups, we summarized all connections with a gray, thicker arrow. Thus, the bidirectional gray arrow connecting the obstetric group with the anesthesiology group represented the 15 link from obstetric nodes to anesthesiology nodes as well as the 15 links from anesthesiology nodes to obstetric nodes. Similarly, the two other gray links represented multiple links between nurse nodes and individuals in the obstetrics and anesthesiology groups respectively.

We needed both intra- and inter-group measures to compare and differentiate between group performances. There is no formal or gold standard on what should be measures when investigating group performance. Thus we measured;

• (1) Different of descriptive predictors (e.g. average age, average cesarean section experience, etc.).

• (2) Average social ties (Affective, Personal, and Professional) ties within each group (on a team).

• (3) Average social ties (Affective, Personal, and Professional) towards each group (from all other groups).

• (4) Average social ties (Affective, Personal, and Professional) from one group to another. Below, we explain 1–4 in more detail.

Descriptive predictors The first set of predictors was independent of the network, but nevertheless aggregated within each group. These predictors included sex, average age, months of experience since graduation and number of cesarean sections performed, within each group. This resulted in a total of 25 descriptive predictors.

Within-group ties This is the average value of ties within each group. We did this for all three categories of ties (Affective, Personal, and Professional). There was also an indicator (yes/no) predictor for presence/non-presence of an affective tie. We also measured a proxy for asymmetry in pairwise ratings; here, we toke the average value of the absolute difference in pairwise ratings. This results in 4 [tie types]···2 [mean value, asymmetry measure]···3 [no groups] = 24 within-group predictors.

Group-level incoming ties. We also considered the aggregated relationships between groups. For example, what was the average value of ties from all obstetricians to all anesthesiologists? There were six such relationships resulting in 6···4 = 24 ties, as well as 12 asymmetry predictors, resulting in 36 group–group relationship predictors when all four link types were taken into account.

Full-network incoming and outgoing ties. Finally, we looked at the entire network’s relation to a single group, calculating the average value of all incoming and outgoing links impending of each of the three groups. This resulted in 4 [tie types]···2[incoming, outgoing]···3 [no groups] = 24 predictors.All 109 predictors are summarized in the table below (Table 3).

Table 3 To quantify team and group performance, the following predictors are used, which are invariant to the number group members

Appendix 2: Statistical methods; a detailed description

This appendix provides a short description of the penalized regression model (ElasticNet), cross validation, jackknife resampling, handling missing values, and the Percent Significant predictor Weight (PSPW) calculation. The goal of this appendix is to provide a more detailed description of the analysis outline in the main text, as well as the general idea behind these methods.

We note that all of these statistical methods are well established; as such, this appendix gives a non-comprehensive overview of the methods and the general idea behind them. We refer the reader to standard statistical and machine learning textbooks for a comprehensive review of penalized regression models, cross validation, jackknife resampling and handling missing values.

The dataset contained N = 33 observations (CS teams) and p = 109 predictor variables, so to learn from this large predictor set (N < p), we used a linear regression where predictor variables that were not supported by the data were pruned (assigned weight = 0) using elastic net regularisation (Pollack and Matous 2019). Further, in order to determine the most relevant variables, we used the regularized regression in combination with leave-one-out cross validation (Arlot and Celisse 2010) and Jackknife resampling (Afanador et al. 2014) (See “Appendix 2.2” for details). The ElasticNet mixing parameter r = [0,1] was estimated using cross validation. In order to identify significant variables, we used Jackknife resampling and then applied a two-sided T test at the 0.01 level (with Bonferroni correction (Weisstein 2004)) to the Jackknife-generated samples. This allowed us to associate a P-value with each non-zero predictor-weight. In interpreting the model, we use only variables that were significant according to the T-test and calculated the Percent Significant Predictor Weight (PSPW).

Appendix 2.1: Multiple regression using elastic net

The model assumed the outcome $y_{n}$ of the n’th cesarean-section had a linear relation to the predictor variables $\varvec{x}_{\varvec{n}} = \left[ {x_{n,1} ,x_{n,2} , \ldots ,x_{n,p} } \right],$ which can be described via linear regression,

$$y_{n} = \mathop \sum \limits_{i = 1}^{p} x_{n,i} \theta_{i} + \in_{n} = \varvec{x}_{n}\varvec{\theta}+ \in_{n}$$

where $\varvec{\theta}= \left[ {\theta_{1} ,\theta_{2} , \ldots ,\theta_{p} } \right]^{T}$ are the parameters, i.e. weights on each predictor variable. The variable $\in_{n}$ is the residual error (or noise) for the n’th observation. The parameters are the same for all caesarean sections and the linear relation is,

$$\varvec{y} = \left[ {\varvec{x}_{1} ,\varvec{ x}_{2} ,\varvec{ } \ldots ,\varvec{ x}_{N} } \right]\varvec{\theta}+ \in_{n} = \varvec{X\theta } + \in ,$$

where $\varvec{y}^{N \times 1}$ is a vector of outcomes for the N cesarean sections. $\varvec{X}^{N \times p}$ is a matrix representing all sectios and their associated p predictor variables, denoted the data matrix. The residual errors $\in$ are assumed to be independent and identically distributed white noise.

Linear regression is a widely used model that provides an unbiased estimate of the model parameters. The degrees-of-freedom (DoF) of the linear regression model is given by DoF = N − p. When N > p, linear regression has a rich theoretical background for characterizing uncertainties in the estimated parameters and for predicting new values.

In this study, the simulated data consists of N = 33 caesarean sections each described by p = 109 predictors, i.e. N < p. Applying the standard linear regression approach was not valid, as trying to estimate a 109-dimensional hyperplane based on 33 data points results in overfitting, since the model can describe the relation between outcomes and predictors perfectly using one parameter to describe each observation. The remaining 76 parameters can then arbitrarily rotate the solution. Intuitively, this type of overfitting can be thought of as trying to fit a cube (p = 3) using only two points (N = 2), where the free parameter can be used to rotate the cube along one axis.

Fitting an over-complete model (N < p) is possible by regularizing the model, i.e. penalizing model complexity (Friedman et al. 2001). Two widely used regularizations are Ridge (Hoerl and Kennard 1970) and LASSO (Tibshirani 1996) regularization, which can be combined into the so-called ElasticNet regularization.

ElasticNet is a linear regression model with penalty on the parameter coefficients $\varvec{\theta}$. The inclusion of a penalty term can help regularize the model fitting and perform predictor selection. The ElasticNet cost function can be written as,

$$\parallel \varvec{y} - \varvec{X\theta }\parallel_{F}^{2} + r\parallel\varvec{\theta}\parallel_{1} + \left( {1 - r} \right)\parallel\varvec{\theta}\parallel_{2}^{2}$$

where $\parallel *\parallel_{F}^{2}$ is the Frobenius norm (Euclidean norm for matrices). The penalty terms $\parallel\varvec{\theta}\parallel_{1}$ and $\parallel\varvec{\theta}\parallel_{2}^{2}$ are the LASSO and Ridge penalty, respectively. The LASSO term is based on the Manhattan distance (l1-norm) and seeks to sink elements of $\varvec{\theta}$ to zero. The Ridge term is based on the Euclidean vector-norm and seeks to spread out the influence of individual elements of $\varvec{\theta}$, such that the elements have similar magnitude.

In ElasticNet, the r parameter controls which penalty should be enforced by a ratio term $0 \le r \le 1$. For r = 0 only the Ridge term were used and for r = 1 only the LASSO term were used. A mixture of the two was used when 0 < r < 1.

We cross validated for an optimal setting of the parameter (Fig. 5a). The optimal l1-l2 ratio for the outcome (patient satisfaction) was identified as 0.0, pointing to an ElasticNet model with only the Ridge penalty, i.e. a Ridge regression model.

Fig. 5

a Parameter selection for 11–l2 ratio, identified as r = 0.0 with RMSE = 0.442. The predictive performance is evaluated on the training and test data, b and c, respectively

Regularization is necessary to obtain a solution when p > N, but the resulting parameters may be biased. For two highly correlated predictors, LASSO regularization will only select one and prune the other. In contrast, Ridge regularization will include both predictors, but their coefficients will shrink (go towards zero) together leading to biased estimates. The ElasticNet regularization mixes these two properties and may still result in biased estimates of the coefficients.

In our analysis, we use the Jackknife correction to reduce biased in the coefficients before testing their significance. While the Jackknife can theoretically remove the biased entirely, this is seldom the case in practice.

Appendix 2.2: Cross-validation and Jackknife resampling

Given a dataset {X,y} where X is the observed data and y is the corresponding outcomes we can to estimate the model parameters $\theta$. We use leave-one-outcross-validation (CV) to investigate model performance and select a set of model parameters that generalize to new data. That is, given a new data point $x_{new}$ we want to be able predict the outcome $y_{new} .$

Using the entire dataset for a single parameter estimate is likely to result in overfitting. Overfitting occurs when the parameter estimates are biased towards the dataset used for estimation. Therefore, overfitting reduces the model’s generalizability, i.e. the model cannot accurately predict the outcome for new data points.

Cross-validation is a common practice to avoid or minimize overfitting by holding out part of the available dataset as a test set $(X_{test} , y_{test} )$ not used for estimating the model. The training set $(X_{train} , y_{train} )$ is then the data not included in the test set. We estimate the model parameters from the training set and assess their validity by predicting the outcome on the test set.

In our analysis, we used leave-one-out CV. Leave-one-out (LOO) cross-validation splits the dataset into N training and test sets, also known as folds, where N is the number of caesarean sections (observations). In each fold one cesarean section (observation) was left out of training set and used to estimate the generalization of the model. In the first fold the first cesarean section was left out, in the second the second section was left out and so on until all N folds were processed. After performing LOO CV, a left-out sample estimate was available for each observation, which was used when calculating the generalization error or other relevant statistics. Figure 5b and c show the predictive performance of the model on the training and test set, respectively.

Appendix 2.3: Missing values

A challenge here was that the data set was characterized by a significant fraction of missing values. Importantly, these values are structured in the sense that participants preferentially chose not to answer certain questions. Further, team size may fluctuate between caesarean sections, varying from 6 to 10 team members.

Our approach to constructing the predictor variables from the demographic and network data use the average of present ties or values. This helps in reducing the amount of missing values, as well as accounting for varying team sizes, but approximately 3 percent of the values are still missing.

The regression model we used, however, was not able to handle missing data, which means that data must be imputed before modelling. We note that whichever method for imputation we choose may introduce biases; further, there is no formal method for validating imputation choices, given only a single dataset.

We experimented with a number of imputation strategies. Imputing by mean or median creates biases, since the missing data were structured. Another approach is k-nearest neighbor imputation, but this choice is also problematic since the neighborhoods often have the same missing values.

For tie-related predictors, we have chosen to set missing values to 0. This choice has the interpretation that not answering a subject is indicating a poor relation to the person being evaluated, especially when every other relation has been filled out. In the case of descriptive predictors (age, experience, etc.), we imputed by the mean value. Finally, missing elements were only imputed for the data matrix X introducing a bias in the explanatory variables/predictors. For missing outcomes (elements in Y) the outcome (and team) was removed from the analysis.

Appendix 2.4: Jackknife

The ElasticNet model reduced the variance of the model estimates by introducing a bias, i.e. a biased estimator. Our goal was to identify important predictors, and we used Jackknife resampling to calculate confidence intervals and p-values for the predictor weights. Jackknife resampling helps to minimize the bias in the estimates (Zou and Hastie 2005). The Jackknife resampling process was similar to LOO CV (and can be calculated simultaneously), but statistics were computed from in-sample estimates, i.e., training data.

Let the estimates of the model parameters in the ith CV fold be denoted $\varvec{\theta}_{i}$. Then the jackknife estimate of $\varvec{\theta}_{\left( * \right)}$ across all jackknife/folds is,

$$\varvec{\theta}_{\left( * \right)} = \frac{1}{N}\mathop \sum \limits_{i = 1}^{N} \varvec{\theta}_{i}$$

where N is the number of CV folds. The variance of the model parameter estimates is,

$$\varvec{\sigma}_{\left( * \right)}^{2} = \frac{N - 1}{N}\mathop \sum \limits_{i = 1}^{N} \left( { \varvec{\theta}_{i} -\varvec{\theta}_{\left( * \right)} } \right)^{2}$$

For brevity, we let the elementwise square of a vector v be denoted by v². The estimated mean and variance are used to calculate an uncorrected jackknife confidence interval for the estimates,

$$CI =\varvec{\theta}\pm t_{{\left( {1 - \frac{\alpha }{2} , N - 1} \right)}}\varvec{\sigma}_{\left( * \right)}$$

where $\varvec{\theta}$ was parameters estimated on the entire dataset, $t_{{\left( {1 - \frac{\alpha }{2} , N - 1} \right)}}$ is the $1 - \frac{\alpha }{2}$ quantile of the student-t distribution with N − 1 degrees of freedom. A one-sample t-test can be used to calculate a p-value to determine statistical significance.

The Jackknife procedure also allows an estimate of the bias introduced by the model. This bias correction $B_{\left( * \right)}$ can be used to obtain a bias-corrected confidence interval,

$$CI =\varvec{\theta}- \varvec{b}_{\left( * \right)} \pm t_{{\left( {1 - \frac{\alpha }{2} , N - 1} \right)}}\varvec{\sigma}_{\left( * \right)}$$

where $\varvec{b}_{\left( * \right)} = \left( {N - 1} \right)\left( { \varvec{\theta}_{\left( * \right)} -\varvec{\theta}} \right)$. A one-sample t-test can again be used to determine statistical significance. In our analysis, we assessed the statistical significance of 109 predictors. To account for multiple comparison, the p-values were Bonferroni corrected.

Appendix 2.5: Percent significant predictor weight

Given a vector of significant predictor weights $\varvec{\theta}_{{}}^{{\left( {sig.} \right)}}$ as estimated by the model, we define the percentage significant predictor weight (PSPW) as,

$$\theta_{i}^{{\left( {PSFW} \right)}} = \theta_{i}^{{\left( {sig.} \right)}} \cdot 100 \cdot \left( {\mathop \sum \limits_{{i \in {\mathcal{M}}}} \left| { \theta_{i}^{{\left( {sig.} \right)}} } \right|} \right)^{ - 1} ,\quad \forall i \in {\mathcal{M}}$$

where ${\mathcal{M}}$ is a set containing the indices of all significant predictors. A non-significant predictor has 0 PSPW, per definition. The PSPW is a linear scaling of the predictor weights (estimated by the model) and the sum over all absolute PSPW values is 100. The PSPW indicates how influential a (significant) predictor is in the model, i.e., the higher the (absolute) PSPW, the higher influence and vice versa.

Appendix 2.6: Determining R-squared

The coefficient of determination, R², is often reported in statistical texts to indicate how much of the variance in the dependent variables is predictable by the independent variables. There is, unfortunately, more than one definition of R² used in the literature. We calculate the R² based on the prediction error, which is a widely used definition and has the benefit of relating to the root-mean-square-error.

For N observations, let $\varvec{y} = \left[ {y_{1} , y_{2} , \ldots , y_{N} } \right]$ be the actual (i.e. true) value of the dependent variable and $\hat{\varvec{y}} = \left[ {\hat{y}_{1} ,\hat{y}_{2} , \ldots ,\hat{y}_{N} } \right]$ be the predicted (e.g. by the model) value of the dependent variable. The root-mean-square-error (RMSE) of the prediction is then,

$${\text{RMSE}}\left( {\varvec{y},\hat{\varvec{y}}} \right) = \frac{1}{N}\mathop \sum \limits_{i = 1}^{N} \left( { y_{i} - \hat{y}_{i} } \right)^{2} .$$

The definition of R² based on the prediction error is,

$$R_{err}^{2} = 1 - \frac{{\mathop \sum \nolimits_{i = 1}^{N} \left( { y_{i} - \hat{y}_{i} } \right)^{2} }}{{\mathop \sum \nolimits_{i = 1}^{N} \left( { y_{i} - {\text{mean}}\left( \varvec{y} \right)} \right)^{2} }} = 1 - \frac{{N \cdot {\text{RMSE}}\left( {\varvec{y},\hat{\varvec{y}}} \right)}}{{\mathop \sum \nolimits_{i = 1}^{N} \left( { y_{i} - {\text{mean}}\left( \varvec{y} \right)} \right)^{2} }} ,$$

where ${\text{mean}}\left( \varvec{y} \right)$ is the mean value of the actual value of the dependent variable. The $R_{err}^{2}$ takes, by definition, a value between $- \infty$ and 1, where 1 indicates no prediction error and 0 indicates the predicted values are as good at predicting the mean of the observed values. There is no lower bound on this definition, as ${\text{RMSE}}\left( {\varvec{y},\hat{\varvec{y}}} \right)$ can be between 0 and $\infty$, thus making negative $R_{err}^{2} nnnn$ values possible.