CFA
In the first step of the data analysis, we tested the reliability of the latent variable (construct): the journal internationalisation.
We tested whether the four variables (the internationalisation of authors, the internationalisation of reviewers, the internationalisation of publication language, and the internationalisation of the editorial advisory board) constitute a defining part of the construct (i.e., we tested if there are correlations between the variables and the construct). According to Tabachnick and Fidell (2007), we have assumed that a variable is relevant for the particular construct when a minimal value of the standardised factor loading is.32. Table 6 presents the values of the standardised factor loadings and the standard errors. All four variables related to the internationalisation are significantly related to the construct (i.e., the journal internationalisation). This result confirms that including these variables in the next step of our analysis is valid.
Table 6 Standardised factor loadings for the factor confirmatory model of the Journal Internationalisation according to the fields of science
The construct reliability is high: Cronbach’s alfa coefficient is α = .813 for the H, α = .813 for the SS, and α = .803 for the ENM. We have conducted CFA to test construct reliability. The analysis has confirmed that the construct structures are identified in three fields of science: H: χ
2 = 2.196, df = 2, p > .05, CFI = 1, RMSEA = .012; SS: χ
2 = 2.018, df = 1, p > .05, CFI = .999, RMSEA = .04; ENM: χ
2 = 2.176, df = 1, p > .05, CFI = .999, and RMSEA = .041. The high CFI indicates that the construct structure is a valid one.
Evaluation of the models
In the second step, we have investigated how the theoretical (hypothetical) models designed for each field of science fit the data. We have tested the full model and the reduced model (without the expert-based evaluation) for each field of science. Next, we have compared the full model with the reduced one.
Model H
Figure 1 presents Model H that is specified for the humanities (H). Although the value of the general test for the fit was significant, χ
2 = 109.387, df = 18, p < .05, the values of other indicators, which were used for estimating the goodness of the model fit, show that the fit of Model H to the data is acceptable and that RMSEA = .086, CFI = .96, and SRMR = .075. The analysis of the significance of path coefficients has revealed that all associations in the theoretical model were significant (p < .001). The journal internationalisation (b = .156, SE = .021, p < .001), the expert-based evaluation (b = .771, SE = .016, p < .001), the age (b = .178, SE = .021, p < .001), and the electronic version (b = .231, SE = .02, p < .001) were predictors of the points. The age was a predictor of the expert-based evaluation (b = .317, SE = .035, p < .001). Model H explains 76% of the point variability (R
2 = .766).
Figure 2 presents model H′. As in Model H, a value of the general test for the fit was significant, χ
2 = 65.241, df = 13, p < .05. The other indicators show that the fit of Model H′ to the data is acceptable: RMSEA = .077, CFI = .962, and SRMR = .057. Analysis of the significance of path coefficients revealed that all associations in the theoretical model were significant (p < .001). The journal internationalisation (b = .316, SE = .033, p < .001), the age (b = .433, SE = .029, p < .001), and the electronic version (b = .248, SE = .032, p < .001) were predictors of the points. Model H′ explains 32% of the point variability (R
2 = .321).
The χ
2 difference test for the comparison of Model H with Model H′ is significant, χ
2difference
= 44.146, df
difference = 5, and critical value for a 5 df is 11.07 (p < .05). Model H (i.e., the full model) is more acceptable than the reduced model (i.e., Model H′). Moreover, Model H better explains the points variability (76%) than Model H′ (32%). Comparison of the models with and without the expert-based evaluation shows that the expert-based evaluation significantly influenced the results of the multidimensional evaluation in the H.
Model SS
Figure 3 presents Model SS, which is specified for the social sciences (SS). As in Model H, a value of the general test for the fit was significant, χ
2 = 103.681, df = 17, p < .05; the values of other indicators, which were used for estimating the goodness of the model fit, show that the fit of Model SS to the data is acceptable: RMSEA = .089, CFI = .962, and SRMR = .068. The analysis of the significance of path coefficients has revealed that all associations in the theoretical model were significant (p < .001). The journal internationalisation (b = .216, SE = .021, p < .001), the expert-based evaluation (b = .753, SE = .018, p < .001), the age (b = .121, SE = .02, p < .001), and the electronic version (b = .129, SE = .018, p < .001) were predictors of the points. The age was a predictor of the expert-based evaluation (b = .417, SE = .031, p < .001). The journal internationalisation was associated with the expert-based evaluation (b = .317, SE = .039, p < .001). Model SS explains 81% of the points variability (R
2 = .81).
Figure 4 presents Model SS’. As in Model SS, a value of the general test for the fit was significant, χ
2 = 93.812, df = 13, p < .05. The values of indicators RMSEA = .098, CFI = .94, and SRMR = .071 show that the fit of Model SS’ to the data is acceptable, but the value of indicator CFI = .94 is slightly below the acceptable limit. Thus, the model should not be accepted. Analysis of the significance of path coefficients revealed that all associations in the theoretical model were significant (p < .001). The journal internationalisation (b = .425, SE = .032, p < .001), the age (b = .435, SE = .029, p < .001), and the electronic version (b = .162, SE = .032, p < .001) were predictors of the points. Model SS’ explains 39% of the points variability (R
2 = .39).
The χ
2 Chi square difference test for the comparison of Model S with Model SS’ is significant, χ
2difference
= 9.869, df
difference = 4, and the critical value for a 4 df is 9.488 (p < .05). Model SS (i.e., the full model) is more acceptable than the reduced model (i.e., Model SS’). Moreover, Model SS better explains the points variability (81%) than Model SS’ (39%). Comparison of the models with and without the expert-based evaluation shows that the expert-based evaluation significantly influenced the results of the multidimensional evaluation in the SS.
Model ENM
Figure 5 presents the Model ENM that is specified for the engineering, natural, and medical sciences (ENM). As in Models H and SS, a value of the general test for the fit was significant, χ
2 = 59.293, df = 15, p < .05, and the values of other indicators, which were used for estimating the goodness of the model fit, show that the fit of Model ENM to the data is acceptable: RMSEA = .065, CFI = .983, and SRMR = .061. The analysis of the significance of path coefficients has revealed that all associations in the theoretical model were significant (p < .001). The journal internationalisation (b = .283, SE = .021, p < .001), the expert-based evaluation (b = .736, SE = .017, p < .001), the PIF (b = .045, SE = .016, p < .01), and the electronic version (b = .111, SE = .016, p < .001) were predictors of the points. The journal internationalisation was associated with the PIF (b = .154, SE = .039, p < .001) and the expert-based evaluation (b = .0434, SE = .034, p < .001). The PIF was associated with the expert-based evaluation (b = .273, SE = .035, p < .001). Model ENM explains 84% of the points variability (R
2 = .839).
Figure 6 presents the Model ENM′. A value of the general test for the fit was significant, χ
2 = 53.098, df = 11, p < .05. The other indicators show that the fit of Model ENM’ to the data is acceptable, RMSEA = .073, CFI = .973, and SRMR = .063. Analysis of the significance of path coefficients revealed that all associations in the theoretical model were significant (p < .001). The journal internationalisation (b = .577, SE = .03, p < .001), the PIF (b = .198, SE = .03, p < .001), and the electronic version (b = .134, SE = .03, p < .001) were predictors of the points. The journal internationalisation was associated with the PIF (b = .158, SE = .039, p < .001). The model ENM′ explains 43% of the points variability (R
2 = .427).
The χ2 Chi square difference test for the comparison of Model ENM with Model ENM’ is not significant, χ
2difference
= 6.195, df
difference = 4, critical value for a 4 df is 9.488 (p < .05). Thus, Model ENM does not significantly differ from Model ENM’, and it is fit to the data in the same way as Model ENM′. However, Model ENM better explains the points variability (84%) than Model ENM’ (43%). Comparison of the model with and without the expert-based evaluation shows that—contrary to the results in the H and the SS—the expert-based evaluation did not significantly influence the results of the multidimensional evaluation in the ENM. However, we have included the PIF only in the ENM and ENM’ models. This decision follows the assumption that we construct models in line with the previous empirical works. It is noteworthy that the correlation coefficients between the PIF and the expert-based evaluation are also very similar in all fields. Hence, including the PIF in the ENM model might provide an alternative explanation as to why there is no effect for the expert-based evaluation in ENM. However, such an explanation cannot be based on the models built according the SEM’s methodological and theoretical assumptions.
In our analysis, we have designed three pairs of theoretical models for three fields of science: the H, the SS, and the ENM. Each pair consists of the full model (e.g., Model H for the H) and the reduced model; that is, the model without the expert-based evaluation (e.g., Model H′ for the H). We have confirmed that our propositions are significant. At the same time, we have presented no results of any alternative models that could be built on the basis of relations identified by the descriptive statistics. According to the SEM, we have worked on the basis of propositions constructed in line with the previous empirical works.
Furthermore, the model comparison has shown that the full model significantly differs from the nested one in the H and the SS. Model H and Model SS are better fit to the data than Model H′ and Model SS′.
Moreover, the full model does not significantly differ from the nested one in the ENM. Therefore, if we decide to eliminate the expert-based dimension in the multidimensional evaluation of national journals in the ENM, both the full and nested models are acceptable. It means that Model ENM and Model ENM′ are well fit to the data.
However, the analysis of R2 shows that the full models explain the points variability in a better way in all three fields of science: H, SS, and ENM. It shows that the expert-based dimension of the multidimensional evaluation of national journals is significant, regardless of the field of science.