Social ties between team members affect patient satisfaction: a data-driven approach to handling complex network analyses

Abstract

Research from outside the medical field suggests that social ties between team-members influence knowledge sharing, improve coordination, and facilitate task completion. However, the relative importance of social ties among team-members for patient satisfaction remains unknown. In this study, we explored the association between social ties within emergency teams performing simulated caesarean sections (CS) and patient-actor satisfaction. Two hundred seventy-two participants were allocated to 33 teams performing two emergency CSs in a simulated setting. We collected data on social ties between team-members, measured as affective, personal and professional ties. Ties were rated on 5-point Likert scales. In addition, participants’ clinical experience, demographic data and their knowledge about team members’ roles were surveyed. Perceived patient satisfaction was measured on a 5-point Likert scale. Data was analysed with a linear regression model using elastic net regularization. In total, 109 predictor variables were analysed including 84 related to social ties and 25 related to clinical experience, demographics and knowledge test scores. Of the 84 variables reflecting social ties, 34 (41%) had significant association with patient satisfaction, p < 0.01. By contrast, a significant association with patient satisfaction was found for only one (4%) of the 25 variables reflecting clinical experience, demographics and knowledge of team roles. Affective ties and personal ties were found to be far more important predictors in the statistical model than professional ties and predictors relating to clinical experience. Social ties between emergency team members may be important predictors of patient satisfaction. The results from this study help to enhance our conceptual understanding of social ties and their implications for team-dynamics. Our study challenges existing views of team-performance by placing emphasis on achieving collective competence through affective and personal social ties, rather than focusing on traditional measures of expertise.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3

References

  1. Afanador, N. L., Tran, T. N., & Buydens, L. M. C. (2014). An assessment of the jackknife and bootstrap procedures on uncertainty estimation in the variable importance in the projection metric. Chemometrics and Intelligent Laboratory Systems,137, 162–172.

    Article  Google Scholar 

  2. Agoritsas, T., Bovier, P. A., & Perneger, T. V. (2005). Patient reports of undesirable events during hospitalization. Journal of General Internal Medicine,20(10), 922–928.

    Article  Google Scholar 

  3. Arlot, S., & Celisse, A. (2010). A survey of cross-validation procedures for model selection. Statistical Surveys,4, 40–79.

    Article  Google Scholar 

  4. Balkundi, P., & Harrison, D. (2006). Ties, leaders, and time in teams: Strong inference about network structure’s effects on team viability and performance. Academy of Management Journal,49(1), 49–68.

    Article  Google Scholar 

  5. Blomquist, J. L., Quiroz, L. H., Macmillan, D., McCullough, A., & Handa, V. L. (2011). Mothers’ satisfaction with planned vaginal and planned cesarean birth. American Journal of Perinatology,28(5), 383–388.

    Article  Google Scholar 

  6. Brown, L. D., Permezel, M., Holberton, J. R., & Whitehead, C. L. (2016). Neonatal outcomes after introduction of a national intrapartum fetal surveillance education program: A retrospective cohort study. The Journal of Maternal-Fetal and Neonatal Medicine,30(15), 1–5.

    Google Scholar 

  7. Cornthwaite, K., Edwards, S., & Siassakos, D. (2013). Reducing risk in maternity by optimising teamwork and leadership: An evidence-based approach to save mothers and babies. Best Practice and Research Clinical Obstetrics and Gynaecology,27(4), 571–581.

    Article  Google Scholar 

  8. Curtis, J. R., Back, A. L., Ford, D. W., et al. (2013). Effect of communication skills training for residents and nurse practitioners on quality of communication with patients with serious illness: A randomized trial. JAMA,310(21), 2271–2281.

    Article  Google Scholar 

  9. de Montjoye, Y. A., Stopczynski, A., Shmueli, E., Pentland, A., & Lehmann, S. (2014). The strength of the strongest ties in collaborative problem solving. Scientific Reports,4, 5277.

    Article  Google Scholar 

  10. Friedman, J., Hastie, T., & Tibshirani, R. (2001). The elements of statistical learning (Vol. 1, pp. 241–249). New York: Springer.

    Google Scholar 

  11. Gude, T., Grimstad, H., Holen, A., et al. (2015). Can we rely on simulated patients’ satisfaction with their consultation for assessing medical students’ communication skills? A cross-sectional study. BMC Medical Education,15, 225.

    Article  Google Scholar 

  12. Harrison, D., Mohammed, S., McGrath, J., Florey, A., & Vanderstoep, S. (2003). Time matters in team performance: Effects of member familiarity, entrainment, and task discontinuity on speed and quality. Personnel Psychology,56(3), 633–669.

    Article  Google Scholar 

  13. Henry, B. W., McCarthy, D. M., Nannicelli, A. P., Seivert, N. P., & Vozenilek, J. A. (2016). Patients’ views of teamwork in the emergency department offer insights about team performance. Health Expectations,19(3), 702–715.

    Article  Google Scholar 

  14. Hodges, B. D., & Lingard, L. (Eds.). (2012). The question of competence: Reconsidering medical education in the twenty-first century. Cornell: Cornell University.

    Google Scholar 

  15. Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems.,12(1), 55–67.

    Google Scholar 

  16. Ilgen, J. S., Ma, I. W., Hatala, R., & Cook, D. A. (2015). A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment. Medical Education,49(2), 161–173.

    Article  Google Scholar 

  17. Institute of Medicine (US) Committee on Quality of Health Care in America. (2001).

  18. Joint Commission on Accreditation of Healthcare Organizations. Preventing infant death and injury during delivery (sentinel event alert issue #30); 21 July 2004. Assessed 28 Oct 2008.

  19. Kurmann, A., Keller, S., Tschan-Semmer, F., et al. (2014). Impact of team familiarity in the operating room on surgical complications. World Journal of Surgery,38(12), 3047–3052.

    Article  Google Scholar 

  20. Liberati, E. G., Gorli, M., & Scaratti, G. (2016). Invisible walls within multidisciplinary teams: Disciplinary boundaries and their effects on integrated care. Social Science and Medicine,150, 31–39.

    Article  Google Scholar 

  21. Lingard, L. (2016). Paradoxical truths and persistent myths: Reframing the team competence conversation. Journal of Continuing Education in the Health Professions,36(Suppl 1), S19–S21.

    Article  Google Scholar 

  22. National Clinical Guideline Centre (UK). (2012).

  23. O’Neil, C. (2016). Weapons of math destruction. United States: Crown Books.

    Google Scholar 

  24. Pollack, J., & Matous, P. (2019). Testing the impact of targeted team building on project team communication using social network analysis. Int J Project Management,37, 473–484.

    Article  Google Scholar 

  25. Powell, A. E., & Davies, H. T. (2012). The struggle to improve patient care in the face of professional boundaries. Social Science and Medicine,75(5), 807–814.

    Article  Google Scholar 

  26. Rasmussen, M. B., Dieckmann, P., Barry Issenberg, S., Ostergaard, D., Soreide, E., & Ringsted, C. V. (2013). Long-term intended and unintended experiences after advanced life support training. Resuscitation,84(3), 373–377.

    Article  Google Scholar 

  27. Ren, Y., & Argote, L. (2011). Transactive memory systems 1985–2010: An integrative framework of key dimensions, antecedents, and consequences. Academy of Management Annals,5, 189–229.

    Article  Google Scholar 

  28. Riskind, P., Fossey, L., & Brill, K. (2011). Why measure patient satisfaction? The Journal of Medical Practice Management,26(4), 217–220.

    Google Scholar 

  29. Roberts, N. K., Williams, R. G., Schwind, C. J., et al. (2014). The impact of brief team communication, leadership and team behavior training on ad hoc team performance in trauma care settings. American Journal of Surgery,207(2), 170–178.

    Article  Google Scholar 

  30. Robinson, P., Salmon, P., & Yentis, S. (1998). Maternal satisfaction. International Journal of Obstetric Anesthesia,7(1), 32–37.

    Article  Google Scholar 

  31. Schlesinger, M., Grob, R., & Shaller, D. (2015). Using patient-reported information to improve clinical practice. Health Services Research,50(Suppl 2), 2116–2154.

    Article  Google Scholar 

  32. Siems, A., Cartron, A., Watson, A., McCarter, R., Jr., & Levin, A. (2017). Improving pediatric rapid response team performance through crew resource management training of team leaders. Hosp Pediatr.,7(2), 88–95.

    Google Scholar 

  33. Singh, S. C., Sheth, R. D., Burrows, J. F., & Rosen, P. (2016). Factors influencing patient experience in pediatric neurology. Pediatric Neurology,60, 37–41.

    Article  Google Scholar 

  34. Smirnova, A., Ravelli, A. C. J., Stalmeijer, R. E., et al. (2017). The association between learning climate and adverse obstetrical outcomes in 16 nontertiary obstetrics–gynecology departments in the Netherlands. Academic Medicine,92(12), 1740–1748.

    Article  Google Scholar 

  35. Stepaniak, P. S., Heij, C., Buise, M. P., Mannaerts, G. H., Smulders, J. F., & Nienhuijs, S. W. (2012). Bariatric surgery with operating room teams that stayed fixed during the day: A multicenter study analyzing the effects on patient outcomes, teamwork and safety climate, and procedure duration. Anesthesia and Analgesia,115(6), 1384–1392.

    Article  Google Scholar 

  36. Thye, S. R., Vincent, A., Lawler, E. J., & Yoon, J. (2014). Relational cohesion, social commitments, and person-to-group ties: Twenty-five years of a theoretical research program. Advances in Group Processes,31, 99–138.

    Article  Google Scholar 

  37. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society.,58, 267–288.

    Google Scholar 

  38. Tsai, T. C., Orav, E. J., & Jha, A. K. (2015). Patient satisfaction and quality of surgical care in US hospitals. Annals of Surgery,261(1), 2–8.

    Article  Google Scholar 

  39. Tucker, A. L. (2007). An empirical study of system improvement by frontline employees in hospital units. M&Som-Manufacturing and Service Operations Management.,9(4), 492–505.

    Article  Google Scholar 

  40. Wegner, D. (1986). A contemporary analysis of the group mind. In B. Mullen & G. Goethals (Eds.), Theories of group behavior (pp. 185–208). Springer: New York.

    Google Scholar 

  41. Weisstein EW (2004) Booktitle: Bonferroni correction. Mathworld. Wolfram.com.

  42. Rasmussen B, Tolsgaard M, Dieckmann P, et al. (xxxx) The complex influence of social relations on teamwork in managing clinical emergencies: A constructivis grounded theory of intensive care staff. Not published. In review.

  43. Zhong, X., Huang, Q., Davison, R. M., Yang, X., & Chen, H. (2012). Empowering teams through social network ties. International Journal of Information Management,32(3), 209–220.

    Article  Google Scholar 

  44. Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology),67, 301–320.

    Article  Google Scholar 

Download references

Acknowledgements

We thank the staff at the department of Gynaecology and Obstetrics and the Department of Anaesthetics/Operating room for their participation. Especially thank to the steering group: Midwives Mette Lise Buchwald, anaesthetic specialist Agnieszka Huscher, Andrea Kisling, Bjørn Dehlie, Pernille Bækgaard and scrub nurse Margrethe Klaaborg.

Funding

The project was funded by the Danish foundation TrygFonden.

Author information

Affiliations

Authors

Contributions

BRA contributed to the conception and design of the work, and to data acquisition and interpretation, data analysis and drafted the paper. MT and MBR contributed to the conception and design of the work and to data acquisition and interpretation, data analysis and assisted in drafting the paper. EL contributed to the conception and design of the work and to data acquisition and interpretation and assisted in drafting the paper. JLH and SL contributed to data analysis and interpretation and assisted in drafting of the paper. CR contributed to the conception and design of the work and assisted in drafting the paper. All authors contributed to the critical revision of the paper and approved the final manuscript for publication. All authors have agreed to be accountable for all aspects of the work.

Corresponding author

Correspondence to Betina Ristorp Andersen.

Ethics declarations

Conflict of interest

All authors have declare that they have no conflict of interest.

Ethics approval

The study was reported to the Danish Patient Safety Authority (3-3013-1732/1) and approved. Data was reported to the Danish data protection agency (NOH-2015-040. I-Suite: 04287).

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Statistical appendix

Statistical appendix

Appendix 1: Operationalizing the network

The emergency cesarean section team in the operating room can be described as a network. This network is a fully connected graph with every node connected to every other node in the network. In the case of n = 10 nodes, we have n(n − 1) = 90 directed links in the network. In our analysis, the network nodes therefore, stayed roughly constant (there were small changes, since not all roles were filled in every operation), and what changes from team to team were the set of relationships (links) that represent the relationship between each pair of professional roles.

In understanding how the network affected outcomes, it was not feasible to model the behavior of every single link—we simply did not have enough observations (in these case, operations) to model the significance of individual links. Therefore, we used combined scores for connections within and between groups (Fig. 4). This averaging was also necessary, because the groups varied in size between different emergency cesarean teams. Thus, it was not desirable to draw conclusions related to a specific member of a group. Stated differently, we needed to summarize information from the network using knowledge of the operating workflow.

Fig. 4
figure4

A sketch of the team structure of the operating room network. See main text for full details

To avoid visual clutter in Fig. 4, we did not draw all 90 links in the network. Instead, we divided the network into three groups characterized by a high level of physical interactions during each section. These groups were: The obstetrics group consisting of midwives and obstetrics specialist shown in blue, the anesthesiology group consisting of doctors of anesthesiology as well as a nurse specializing in anesthesiology shown in green, and finally a group of scrub nurses.

Within each of the three groups, we drew all links. The fact that all links had arrows at both ends signifies that all ties were bidirectional, meaning that relationship A → B is different from B → A. Between the three groups, we summarized all connections with a gray, thicker arrow. Thus, the bidirectional gray arrow connecting the obstetric group with the anesthesiology group represented the 15 link from obstetric nodes to anesthesiology nodes as well as the 15 links from anesthesiology nodes to obstetric nodes. Similarly, the two other gray links represented multiple links between nurse nodes and individuals in the obstetrics and anesthesiology groups respectively.

We needed both intra- and inter-group measures to compare and differentiate between group performances. There is no formal or gold standard on what should be measures when investigating group performance. Thus we measured;

  1. (1)

    Different of descriptive predictors (e.g. average age, average cesarean section experience, etc.).

  2. (2)

    Average social ties (Affective, Personal, and Professional) ties within each group (on a team).

  3. (3)

    Average social ties (Affective, Personal, and Professional) towards each group (from all other groups).

  4. (4)

    Average social ties (Affective, Personal, and Professional) from one group to another. Below, we explain 1–4 in more detail.

Descriptive predictors The first set of predictors was independent of the network, but nevertheless aggregated within each group. These predictors included sex, average age, months of experience since graduation and number of cesarean sections performed, within each group. This resulted in a total of 25 descriptive predictors.

Within-group ties This is the average value of ties within each group. We did this for all three categories of ties (Affective, Personal, and Professional). There was also an indicator (yes/no) predictor for presence/non-presence of an affective tie. We also measured a proxy for asymmetry in pairwise ratings; here, we toke the average value of the absolute difference in pairwise ratings. This results in 4 [tie types]···2 [mean value, asymmetry measure]···3 [no groups] = 24 within-group predictors.

Group-level incoming ties. We also considered the aggregated relationships between groups. For example, what was the average value of ties from all obstetricians to all anesthesiologists? There were six such relationships resulting in 6···4 = 24 ties, as well as 12 asymmetry predictors, resulting in 36 group–group relationship predictors when all four link types were taken into account.

Full-network incoming and outgoing ties. Finally, we looked at the entire network’s relation to a single group, calculating the average value of all incoming and outgoing links impending of each of the three groups. This resulted in 4 [tie types]···2[incoming, outgoing]···3 [no groups] = 24 predictors.All 109 predictors are summarized in the table below (Table 3).

Table 3 To quantify team and group performance, the following predictors are used, which are invariant to the number group members

Appendix 2: Statistical methods; a detailed description

This appendix provides a short description of the penalized regression model (ElasticNet), cross validation, jackknife resampling, handling missing values, and the Percent Significant predictor Weight (PSPW) calculation. The goal of this appendix is to provide a more detailed description of the analysis outline in the main text, as well as the general idea behind these methods.

We note that all of these statistical methods are well established; as such, this appendix gives a non-comprehensive overview of the methods and the general idea behind them. We refer the reader to standard statistical and machine learning textbooks for a comprehensive review of penalized regression models, cross validation, jackknife resampling and handling missing values.

The dataset contained N = 33 observations (CS teams) and p = 109 predictor variables, so to learn from this large predictor set (N < p), we used a linear regression where predictor variables that were not supported by the data were pruned (assigned weight = 0) using elastic net regularisation (Pollack and Matous 2019). Further, in order to determine the most relevant variables, we used the regularized regression in combination with leave-one-out cross validation (Arlot and Celisse 2010) and Jackknife resampling (Afanador et al. 2014) (See “Appendix 2.2” for details). The ElasticNet mixing parameter r = [0,1] was estimated using cross validation. In order to identify significant variables, we used Jackknife resampling and then applied a two-sided T test at the 0.01 level (with Bonferroni correction (Weisstein 2004)) to the Jackknife-generated samples. This allowed us to associate a P-value with each non-zero predictor-weight. In interpreting the model, we use only variables that were significant according to the T-test and calculated the Percent Significant Predictor Weight (PSPW).

Appendix 2.1: Multiple regression using elastic net

The model assumed the outcome \(y_{n}\) of the n’th cesarean-section had a linear relation to the predictor variables \(\varvec{x}_{\varvec{n}} = \left[ {x_{n,1} ,x_{n,2} , \ldots ,x_{n,p} } \right],\) which can be described via linear regression,

$$y_{n} = \mathop \sum \limits_{i = 1}^{p} x_{n,i} \theta_{i} + \in_{n} = \varvec{x}_{n}\varvec{\theta}+ \in_{n}$$

where \(\varvec{\theta}= \left[ {\theta_{1} ,\theta_{2} , \ldots ,\theta_{p} } \right]^{T}\) are the parameters, i.e. weights on each predictor variable. The variable \(\in_{n}\) is the residual error (or noise) for the n’th observation. The parameters are the same for all caesarean sections and the linear relation is,

$$\varvec{y} = \left[ {\varvec{x}_{1} ,\varvec{ x}_{2} ,\varvec{ } \ldots ,\varvec{ x}_{N} } \right]\varvec{\theta}+ \in_{n} = \varvec{X\theta } + \in ,$$

where \(\varvec{y}^{N \times 1}\) is a vector of outcomes for the N cesarean sections. \(\varvec{X}^{N \times p}\) is a matrix representing all sectios and their associated p predictor variables, denoted the data matrix. The residual errors \(\in\) are assumed to be independent and identically distributed white noise.

Linear regression is a widely used model that provides an unbiased estimate of the model parameters. The degrees-of-freedom (DoF) of the linear regression model is given by DoF = N − p. When N > p, linear regression has a rich theoretical background for characterizing uncertainties in the estimated parameters and for predicting new values.

In this study, the simulated data consists of N = 33 caesarean sections each described by p = 109 predictors, i.e. N < p. Applying the standard linear regression approach was not valid, as trying to estimate a 109-dimensional hyperplane based on 33 data points results in overfitting, since the model can describe the relation between outcomes and predictors perfectly using one parameter to describe each observation. The remaining 76 parameters can then arbitrarily rotate the solution. Intuitively, this type of overfitting can be thought of as trying to fit a cube (p = 3) using only two points (N = 2), where the free parameter can be used to rotate the cube along one axis.

Fitting an over-complete model (N < p) is possible by regularizing the model, i.e. penalizing model complexity (Friedman et al. 2001). Two widely used regularizations are Ridge (Hoerl and Kennard 1970) and LASSO (Tibshirani 1996) regularization, which can be combined into the so-called ElasticNet regularization.

ElasticNet is a linear regression model with penalty on the parameter coefficients \(\varvec{\theta}\). The inclusion of a penalty term can help regularize the model fitting and perform predictor selection. The ElasticNet cost function can be written as,

$$\parallel \varvec{y} - \varvec{X\theta }\parallel_{F}^{2} + r\parallel\varvec{\theta}\parallel_{1} + \left( {1 - r} \right)\parallel\varvec{\theta}\parallel_{2}^{2}$$

where \(\parallel *\parallel_{F}^{2}\) is the Frobenius norm (Euclidean norm for matrices). The penalty terms \(\parallel\varvec{\theta}\parallel_{1}\) and \(\parallel\varvec{\theta}\parallel_{2}^{2}\) are the LASSO and Ridge penalty, respectively. The LASSO term is based on the Manhattan distance (l1-norm) and seeks to sink elements of \(\varvec{\theta}\) to zero. The Ridge term is based on the Euclidean vector-norm and seeks to spread out the influence of individual elements of \(\varvec{\theta}\), such that the elements have similar magnitude.

In ElasticNet, the r parameter controls which penalty should be enforced by a ratio term \(0 \le r \le 1\). For r = 0 only the Ridge term were used and for r = 1 only the LASSO term were used. A mixture of the two was used when 0 < r < 1.

We cross validated for an optimal setting of the parameter (Fig. 5a). The optimal l1-l2 ratio for the outcome (patient satisfaction) was identified as 0.0, pointing to an ElasticNet model with only the Ridge penalty, i.e. a Ridge regression model.

Fig. 5
figure5

a Parameter selection for 11–l2 ratio, identified as r = 0.0 with RMSE = 0.442. The predictive performance is evaluated on the training and test data, b and c, respectively

Regularization is necessary to obtain a solution when p > N, but the resulting parameters may be biased. For two highly correlated predictors, LASSO regularization will only select one and prune the other. In contrast, Ridge regularization will include both predictors, but their coefficients will shrink (go towards zero) together leading to biased estimates. The ElasticNet regularization mixes these two properties and may still result in biased estimates of the coefficients.

In our analysis, we use the Jackknife correction to reduce biased in the coefficients before testing their significance. While the Jackknife can theoretically remove the biased entirely, this is seldom the case in practice.

Appendix 2.2: Cross-validation and Jackknife resampling

Given a dataset {X,y} where X is the observed data and y is the corresponding outcomes we can to estimate the model parameters \(\theta\). We use leave-one-outcross-validation (CV) to investigate model performance and select a set of model parameters that generalize to new data. That is, given a new data point \(x_{new}\) we want to be able predict the outcome \(y_{new} .\)

Using the entire dataset for a single parameter estimate is likely to result in overfitting. Overfitting occurs when the parameter estimates are biased towards the dataset used for estimation. Therefore, overfitting reduces the model’s generalizability, i.e. the model cannot accurately predict the outcome for new data points.

Cross-validation is a common practice to avoid or minimize overfitting by holding out part of the available dataset as a test set \((X_{test} , y_{test} )\) not used for estimating the model. The training set \((X_{train} , y_{train} )\) is then the data not included in the test set. We estimate the model parameters from the training set and assess their validity by predicting the outcome on the test set.

In our analysis, we used leave-one-out CV. Leave-one-out (LOO) cross-validation splits the dataset into N training and test sets, also known as folds, where N is the number of caesarean sections (observations). In each fold one cesarean section (observation) was left out of training set and used to estimate the generalization of the model. In the first fold the first cesarean section was left out, in the second the second section was left out and so on until all N folds were processed. After performing LOO CV, a left-out sample estimate was available for each observation, which was used when calculating the generalization error or other relevant statistics. Figure 5b and c show the predictive performance of the model on the training and test set, respectively.

Appendix 2.3: Missing values

A challenge here was that the data set was characterized by a significant fraction of missing values. Importantly, these values are structured in the sense that participants preferentially chose not to answer certain questions. Further, team size may fluctuate between caesarean sections, varying from 6 to 10 team members.

Our approach to constructing the predictor variables from the demographic and network data use the average of present ties or values. This helps in reducing the amount of missing values, as well as accounting for varying team sizes, but approximately 3 percent of the values are still missing.

The regression model we used, however, was not able to handle missing data, which means that data must be imputed before modelling. We note that whichever method for imputation we choose may introduce biases; further, there is no formal method for validating imputation choices, given only a single dataset.

We experimented with a number of imputation strategies. Imputing by mean or median creates biases, since the missing data were structured. Another approach is k-nearest neighbor imputation, but this choice is also problematic since the neighborhoods often have the same missing values.

For tie-related predictors, we have chosen to set missing values to 0. This choice has the interpretation that not answering a subject is indicating a poor relation to the person being evaluated, especially when every other relation has been filled out. In the case of descriptive predictors (age, experience, etc.), we imputed by the mean value. Finally, missing elements were only imputed for the data matrix X introducing a bias in the explanatory variables/predictors. For missing outcomes (elements in Y) the outcome (and team) was removed from the analysis.

Appendix 2.4: Jackknife

The ElasticNet model reduced the variance of the model estimates by introducing a bias, i.e. a biased estimator. Our goal was to identify important predictors, and we used Jackknife resampling to calculate confidence intervals and p-values for the predictor weights. Jackknife resampling helps to minimize the bias in the estimates (Zou and Hastie 2005). The Jackknife resampling process was similar to LOO CV (and can be calculated simultaneously), but statistics were computed from in-sample estimates, i.e., training data.

Let the estimates of the model parameters in the ith CV fold be denoted \(\varvec{\theta}_{i}\). Then the jackknife estimate of \(\varvec{\theta}_{\left( * \right)}\) across all jackknife/folds is,

$$\varvec{\theta}_{\left( * \right)} = \frac{1}{N}\mathop \sum \limits_{i = 1}^{N} \varvec{\theta}_{i}$$

where N is the number of CV folds. The variance of the model parameter estimates is,

$$\varvec{\sigma}_{\left( * \right)}^{2} = \frac{N - 1}{N}\mathop \sum \limits_{i = 1}^{N} \left( { \varvec{\theta}_{i} -\varvec{\theta}_{\left( * \right)} } \right)^{2}$$

For brevity, we let the elementwise square of a vector v be denoted by v2. The estimated mean and variance are used to calculate an uncorrected jackknife confidence interval for the estimates,

$$CI =\varvec{\theta}\pm t_{{\left( {1 - \frac{\alpha }{2} , N - 1} \right)}}\varvec{\sigma}_{\left( * \right)}$$

where \(\varvec{\theta}\) was parameters estimated on the entire dataset, \(t_{{\left( {1 - \frac{\alpha }{2} , N - 1} \right)}}\) is the \(1 - \frac{\alpha }{2}\) quantile of the student-t distribution with N − 1 degrees of freedom. A one-sample t-test can be used to calculate a p-value to determine statistical significance.

The Jackknife procedure also allows an estimate of the bias introduced by the model. This bias correction \(B_{\left( * \right)}\) can be used to obtain a bias-corrected confidence interval,

$$CI =\varvec{\theta}- \varvec{b}_{\left( * \right)} \pm t_{{\left( {1 - \frac{\alpha }{2} , N - 1} \right)}}\varvec{\sigma}_{\left( * \right)}$$

where \(\varvec{b}_{\left( * \right)} = \left( {N - 1} \right)\left( { \varvec{\theta}_{\left( * \right)} -\varvec{\theta}} \right)\). A one-sample t-test can again be used to determine statistical significance. In our analysis, we assessed the statistical significance of 109 predictors. To account for multiple comparison, the p-values were Bonferroni corrected.

Appendix 2.5: Percent significant predictor weight

Given a vector of significant predictor weights \(\varvec{\theta}_{{}}^{{\left( {sig.} \right)}}\) as estimated by the model, we define the percentage significant predictor weight (PSPW) as,

$$\theta_{i}^{{\left( {PSFW} \right)}} = \theta_{i}^{{\left( {sig.} \right)}} \cdot 100 \cdot \left( {\mathop \sum \limits_{{i \in {\mathcal{M}}}} \left| { \theta_{i}^{{\left( {sig.} \right)}} } \right|} \right)^{ - 1} ,\quad \forall i \in {\mathcal{M}}$$

where \({\mathcal{M}}\) is a set containing the indices of all significant predictors. A non-significant predictor has 0 PSPW, per definition. The PSPW is a linear scaling of the predictor weights (estimated by the model) and the sum over all absolute PSPW values is 100. The PSPW indicates how influential a (significant) predictor is in the model, i.e., the higher the (absolute) PSPW, the higher influence and vice versa.

Appendix 2.6: Determining R-squared

The coefficient of determination, R2, is often reported in statistical texts to indicate how much of the variance in the dependent variables is predictable by the independent variables. There is, unfortunately, more than one definition of R2 used in the literature. We calculate the R2 based on the prediction error, which is a widely used definition and has the benefit of relating to the root-mean-square-error.

For N observations, let \(\varvec{y} = \left[ {y_{1} , y_{2} , \ldots , y_{N} } \right]\) be the actual (i.e. true) value of the dependent variable and \(\hat{\varvec{y}} = \left[ {\hat{y}_{1} ,\hat{y}_{2} , \ldots ,\hat{y}_{N} } \right]\) be the predicted (e.g. by the model) value of the dependent variable. The root-mean-square-error (RMSE) of the prediction is then,

$${\text{RMSE}}\left( {\varvec{y},\hat{\varvec{y}}} \right) = \frac{1}{N}\mathop \sum \limits_{i = 1}^{N} \left( { y_{i} - \hat{y}_{i} } \right)^{2} .$$

The definition of R2 based on the prediction error is,

$$R_{err}^{2} = 1 - \frac{{\mathop \sum \nolimits_{i = 1}^{N} \left( { y_{i} - \hat{y}_{i} } \right)^{2} }}{{\mathop \sum \nolimits_{i = 1}^{N} \left( { y_{i} - {\text{mean}}\left( \varvec{y} \right)} \right)^{2} }} = 1 - \frac{{N \cdot {\text{RMSE}}\left( {\varvec{y},\hat{\varvec{y}}} \right)}}{{\mathop \sum \nolimits_{i = 1}^{N} \left( { y_{i} - {\text{mean}}\left( \varvec{y} \right)} \right)^{2} }} ,$$

where \({\text{mean}}\left( \varvec{y} \right)\) is the mean value of the actual value of the dependent variable. The \(R_{err}^{2}\) takes, by definition, a value between \(- \infty\) and 1, where 1 indicates no prediction error and 0 indicates the predicted values are as good at predicting the mean of the observed values. There is no lower bound on this definition, as \({\text{RMSE}}\left( {\varvec{y},\hat{\varvec{y}}} \right)\) can be between 0 and \(\infty\), thus making negative \(R_{err}^{2} nnnn\) values possible.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Andersen, B.R., Hinrich, J.L., Rasmussen, M.B. et al. Social ties between team members affect patient satisfaction: a data-driven approach to handling complex network analyses. Adv in Health Sci Educ 25, 581–606 (2020). https://doi.org/10.1007/s10459-019-09941-1

Download citation

Keywords

  • Social ties
  • Patient satisfaction
  • Education
  • Simulation