The purpose of this study was to garner multiple perspectives on fostering international collaboration in implementation science (FICIS). This study employed concept mapping (CM), a mixed-methods (qualitative and quantitative) approach for data collection and analysis that incorporates input from all participants in order to identify dimensions of productive collaboration and assess their importance and feasibility for FICIS.
Ten implementation researchers participated in a 3-day retreat to foster productive international collaborations in IS. Participants were selected to represent different implementation contexts (e.g., US, Europe) and health issues (e.g., mental health, cancer, occupational health, social care). The participant countries were as follows: France, Germany, the United Kingdom, Australia, Sweden, and the US. All participants had experience in implementation research in those countries as well as collaborations in Spain, Norway, Switzerland, Belgium, Colombia, Mexico, Nigeria, and Sierra Leone. Academic disciplines represented included epidemiology, anthropology, occupational health, social work, pharmacy, business management, organizational psychology, and clinical psychology. Participants represented several domains in health and human services including behavioral health, school-based care, social services, occupational health, nursing, pharmacy, and medicine. The mean number of years of experience in IS was 8.1 years (range = 1.5–18 years), and the mean number of years in international experience was 2.9 years (range = 0.3–5.5 years). Of the 10 participants, eight were professors/faculty and two were post-doctoral scientists.
Concept Mapping consists of six phases: (1) preparation, identify stakeholder participants and collaboratively develop a focus question; (2) generation, participants brainstorm responses to the focus question; (3) structuring, participants sort statements based on similarity and rate statements on a priori dimensions (e.g., importance, feasibility); (4) representation, researchers conduct multidimensional scaling (MDS) and cluster analyses to create a “concept map”; (5) interpretation, researchers/participants collaboratively develop cluster labels and interpretations; and (6) utilization, researchers/participants use results to identify action-items and next-steps. In CM small samples may be adequate and data from at least 10 participants can produce reliable results [17, 18].
Three primary outputs were used: (1) Cluster map identifying the most important dimensions or clusters representing each concept, (2) Cluster ranking based on ratings of importance and feasibility, (3) Pattern matching that shows and correlates the relative ranking for cluster importance and feasibility.
Participants jointly and iteratively developed a single focal question: “What are the ways to foster international collaboration in implementation science in health and social care?” In the generation phase, brainstorming occurred in-person collectively through a group process. Multiple responses from each participant were elicited. Ten participants contributed 61 unique statements regarding FICIS. In the structuring phase, participants used online software and individually sorted the statements into separate groups (or “clusters”) in a manner that was meaningful to them. Finally, participants individually rated each statement on importance: “How important is this factor for fostering international collaboration in implementation science (FICIS)?” (0 = not at all; 5 = to a very great extent), and feasibility: “How feasible is this factor for FICIS?” (0 = not at all; 5 = to a very great extent).
Statement sorting data were analyzed using MDS and hierarchical cluster analysis. These procedures resulted in the visual representations (i.e., concept maps) for how statements were typically clustered across all participants. Multiple CM outcomes were considered based on acceptable overall “stress” fit statistic and interpretability of each potential solution. The “stress value” of the point map is a measure of how well the MDS solution maps the original data. The stress value is derived from normalized residual variance for a perfect relationship of a regression of the distance of dissimilarity or similarity. The range for CM has been reported as 0.21–0.37, with lower stress values reflecting better fit of the MDS point map to the original data . The ideal model would include the fewest number of clusters that also retained distinct themes. This process considered a larger number of potential thematic clusters (e.g., 14) and then, in a stepwise fashion, consolidating clusters that were thematically similar based on participant responses. These models were reviewed by the study team with the final concept map (11 thematic clusters) approved by consensus of all authors. Each cluster in the final model was collaboratively named to reflect the content contained in each cluster. All participants reviewed the model and gave feedback on the final results. We also examined “Go Zone” maps that place each statement in a two-dimensional space with the Y axis indicating importance rating and the X axis representing feasibility.