Application of Exclusion Criteria
In Section 4.9, we presented the in- and exclusion criteria of the study. Table 2 summarises the results of the application of these criteria to all submitted cases. The first column shows the group to which the respective filter/measure was applied and the second column presents the original number of cases for each group.
Columns three to six show the number of participants who opted out of more than seven questions for Athos, for JSprit, for either of the two, and for both. As was explained, a case was excluded if a participants opted out of more than seven questions for either approach so that the fifth column is the one that gives the actual number of cases excluded due to the number of unanswered questions. The sixth column is included in the table to show that most cases that were excluded from the study exceeded the number of seven non-attempts for both approaches.
The seventh and eighth column show the number of cases that were excluded due to an unusually high deviation (in relation to the observed deviation of the sample population) between the times spent and the scores achieved with the approaches, respectively. The ninth column shows the number of cases that were removed because participants spent less than 15 minutes on one (or both) of the approaches without achieving at least 38 points. Finally, the last column shows the number of included cases for the respective group.
Overall, the data in the table show two important facts: firstly, the limit to allowed non-attempts was the one on which most exclusions were based. This means, that the majority of participants either just skimmed through the questions without answering them, or they put in the required effort for both approaches. Secondly, it can also be clearly seen that in the Friedberg study there were substantially more cases excluded than in the Wetzlar study course. Given that the only compensation was that participation was announced to be a perfect training towards the final exam of the respective module, this was to be expected. Students in Friedberg have more freedom in defining their individual curriculum than students enrolled in the co-operative study-program offered in Wetzlar. Thus, only a certain proportion of students attending a module actually sit the final exam. This fact is reflected in the data presented in the table and we do not consider the observed number of drop-outs a threat to the validity of our study.
Prior Programming Knowledge
In order to be able to put the language evaluation results into context, we presented participants with questions on their programming background. Figure 8 compares the answers given by the two groups of the Friedberg participants, Fig. 9 does the same for the two groups from the Wetzlar study. Overall, the results show that a) participants from the respective groups of both studies (Athos first and JSprit first) possessed comparable prior programming skills and knowledge, and b) participants from Friedberg had less programming experience than those from Wetzlar.
In terms of years of programming experience, Fig. 8 shows that in Friedberg the JSprit first group (FbJf) was slightly more experienced than the Athos first group (FbAf). Only seven members of the JSprit first group (≈ 21% of all group members) stated to have less than a year of programming experience while twice as many members of the Athos first group (40%) claimed to have less than 12 months of programming experience. However, in both groups the vast majority of participants did not have more than two years of programming experience (more than 70% in both groups). Programmers with more than two years of experience were rare in both groups: only eight members of the Athos first group and ten members of the JSprit-first group stated to have more than two years of experience in the usage of computer languages. Based on these data, it can be concluded that most participants in the study conducted in Friedberg had only just started their programming careers.
Regarding the other questions, results for both groups were generally very similar. In both groups, most members had experience with two or three computer languages and there were exactly seven participants in each group who had only worked with one language. The two groups also had a similar number of participants that had used four or even more languages, though the Athos first group had slightly more participants in that category. Asked for a brief self assessment, in both groups the majority stated to consider themselves fair programmers. In both groups there was a strikingly high number of participants who expressed doubt in their own programming skills: As many as 13 participants from the FbAf and 16 from the FbJf group thought of themselves as poor or even very poor programmers. The majority of both groups was either interested or highly interested. While both groups also had a surprisingly high number of participants with only moderate interest, only a negligible number of participants stated their disinterest in the subject.
Figure 9 illustrates the results of the groups from the Wetzlar study. As regards the years of experience, neither group appears to have had a definitive advantage over the other. Although the JSprit first group consisted of more participants who had been programmers for more than five years, the Athos first group on the other hand had more members with four to five years of experience. While there were a few more members with 2–3 years of experience in group WzJf, it is also important to consider that the group consisted of four more members than the WzAf group.
Looking at the answers to the other questions, the results for both groups are also rather similar. The distinct difference in the number of group members who had used 4–5 programming languages can be assumed to result from the different group sizes. In any case, both groups nearly completely consisted of members who had experience with at least four languages. Only a minority of two participants in each group stated to have used only 2–3 different programming languages.
Members of both groups expressed similar confidence in their programming skills. The majority of both groups stated to be good programmers. However, in both groups there were five participants who only considered themselves to be fair programmers. In both groups, nearly all participants were either interested or even highly interest in the subject of programming.
A comparison of the groups from Friedberg and Wetzlar clearly shows that participants from Wetzlar were more experienced, active and confident in the usage of programming languages. The mode and median value for three of the prior knowledge questions are reported in Table 3. Overall, for each of the three questions, both the mode and mean value of the combined Wetzlar study group were one level above the mode and mean of the combined Friedberg study group. Not reported in the table are the mode and median of the question on participants’ interest in programming. For both the Friedberg and the Wetzlar study, the most frequent answer in both groups was that participants were “interested” in the subject of programming, which was also the median value.
These results allow two important conclusions: firstly, the validity of a between-subjects comparison, i.e. a comparison between the results of two groups of the same study, is not threatened by a lopsided group assignment (see Section 6) since members of the two respective groups in both studies had comparable programming knowledge. Secondly, the different knowledge levels observed for participants of the two studies from Friedberg and Wetzlar allow the assumption that participants from Friedberg can be considered to represent domain experts with only limited programming experience, whereas participants from Wetzlar are more suitable to represent junior software developers who possess a well-grounded background in programming. This provides some more context for the results presented in the following sections.
As was mentioned in Section 4.2, we intended to find out whether Athos has the potential to facilitate comprehension of VRP models. For this reason, we formulated the hypothesis that the application of Athos had no effect on the test results (scores) achieved by study participants. In this section, we report the scores achieved by the groups of the two conducted studies. In addition, we will present the outcome of various statistical tests performed on the obtained data and discuss their implications on our hypothesis.
Figure 10 presents an overview of the results obtained in the Friedberg study. What is particularly noticeable is the marked difference in the scores for both languages used as a first approach: the median value for Athos as a first approach surpassed the median of JSprit by more than 24 points. Even the lower quartile for Athos exceeds the median of JSprit by 7.5 points. It thus appears that for users with little programming experience, Athos had a substantially positive effect on the observed scores. A comparison of the results for both languages used as a second approach also shows that on average users scored higher when using Athos. However, the differences in the achieved scores do not appear to be as compelling as those observed when both languages were used as first approaches.
The group that started with Athos exhibited declining scores when JSprit was used as a second approach. The median, for example, dropped from 92 points to a mere 74 points. Even the lower quartile with Athos first is above the median for JSprit as a second approach. The top result with Athos first was a remarkable 146 points – with JSprit second the best result dropped by 12 points to a score of 136. Conversely, when participants started with JSprit and used Athos as a second approach, there was a remarkable increase in points. The median drastically rose from 67.5 points to 94 points and the upper quartile increased from 94.25 points to an impressive 121.25 points. With Athos second the top result was considerably higher than the best result achieved with JSprit first: with JSprit, the best participant scored 117 points which was 23 points less than the best score achieved with Athos second.
Figure 11 gives an overview on the scores achieved in the Wetzlar study. What is immediately obvious is the fact that participants in this study achieved higher scores than participants from the Friedberg study. This was to be expected due to the fact that participants from Wetzlar had distinctly more experience in the field of programming (see Section 5.2). Looking at the respective interquartile ranges, it is also apparent that the results were less dispersed than those in the Friedberg study. While there were exceptionally high scoring and exceptionally low scoring participants in both groups, the majority of participants scored in the range of 100–140 points.
Juxtaposing the results for both languages as the first approach, the results for JSprit were somewhat better than those for Athos. With both approaches the best score was 148 points and with Athos no participant scored less than 100 points which is 26 points above the lowest score achieved with JSprit. However, the median score with JSprit is 10 points above the median score achieved with Athos and both the upper and lower quartile comparison are in favour of JSprit.
On the other hand, if the languages were used as the second approach, the results for Athos were more favourable than those observed for JSprit. With Athos as the second approach, one participant even scored a perfect 150 points whereas the highest score with JSprit as the second approach was 139 points. The median value with Athos as the second approach was slightly above the median value for JSprit.
A within-subjects comparison of both approaches shows that participants who started with Athos were able to moderately improve their results with JSprit as a second approach. The median score in that group rose by 11.5 points from 116 to 127.5 points. However, the top result dropped from 148.0 to 139.0 points and the lower quartile also dropped by 2.5 points. On the other hand, a similar improvement with the second approach could also be observed when Athos was used second. Here, the median improved by 8 points and the other quartiles as well as both whiskers are also higher than those observed for JSprit as the first approach.
Statistical Significance Tests of the Scores
For the within-subjects comparison, we used the Wilcoxon signed-rank test. Table 4 reports on the results obtained. In the Friedberg study, both groups achieved significantly better scores with Athos than with JSprit. The p-values of the tests for both groups are well below the 5% significance level. In Wetzlar, the group that started with Athos performed better when using JSprit, however, the result was not statistically significant. By contrast, the group that started with JSprit performed significantly better with Athos as the second approach with a p-value of only 0.007 and thus high statistical certainty.
To statistically test the results of the between-subjects score comparison, we used the Mann-Whitney U test for which the results are displayed in Table 5. In Friedberg, Athos as a first approach resulted in significantly higher scores than JSprit as a first approach. As a second approach, Athos was also the approach for which – on average – higher scores were observed, though not at a statistically significant level. In Wetzlar, notwithstanding whether they were used as a first or second approach, neither the use of Athos nor the use of JSprit resulted in significantly superior scores.
These results show that Athos has the potential to facilitate users’ effectiveness in program comprehension. Three out of four within-subjects comparisons show statistically significant improvements in the achieved scores for the Athos approach (FbAf, FbJf, WzJf). Only one comparison was moderately in favour of JSprit (WzAf), i.e. not at a statistically significant level. Of the between-subject comparisons, the only one with a statistically significant result (Friedberg, first approach) clearly favoured Athos as the more effective approach. Two more comparisons also were in favour of Athos (Friedberg, second approach; Wetzlar, second approach), though not at a statistically significant level. Only one comparison (Wetzlar, first approach) was in favour of Athos but also not at a level of statistical significance.
In three out of four tests for the Friedberg study group, the application of Athos elicited a significant increase of achieved scores. Though there is no definite explanation as to why the positive effects on users’ effectiveness somewhat diminished when both Athos and JSprit were compared as second approaches, a reasonable assumption is that learning and exhaustion effects may have had a strong influence on the outcome. This is especially plausible given that participation in the study was completely voluntary and there was no special incentive for participants to keep the focus on the highest level for the entire study duration of up to three hours.
The results of the Wetzlar study group show a significant increase in scores when JSprit was the first and Athos the second approach. This provides reliable evidence for the claim that Athos has the potential to improve the effectiveness of more experienced programmers in comparison to traditional approaches. When both languages were used as a second approach by the more experienced programmers from Wetzlar, the observed scores with Athos were also moderately better than those achieved with JSprit, but the improvement was not enough to be statistically significant.
In case both languages were used as a first approach, the results for Athos were to some extent inferior compared to those achieved with JSprit. The same holds true for a within-subjects comparison that favours JSprit as the second approach over Athos as the first approach. These results show that despite its general potential to enhance users’ benefits, there are certain situations in which more experienced users might not benefit from the usage of Athos. But here it is important to note that it was to be expected that the positive effects of Athos might be less pronounced among a group of more experienced programmers. As Barišić et al. (2014) noted, the positive effects of a DSL application are generally less distinct among participants with sophisticated programming knowledge. Two additional factors that might have contributed to these results are the fact that participants in the Wetzlar study had to learn and apply Athos at the very same day (see Section 4.5) which might have been too short of a time to catch up with mostly 2–3 years of GPL experience. Secondly, it is possible that the presented questions were not complex enough for this group, since we used the same questions for the comparably inexperienced participants from Friedberg. As is reported by Kosar et al. (2018), improvement of results related to the application of a DSL are more distinguishable for tasks of high complexity.
Overview on Results Related to Participants’ Efficiency
The second research question asked whether Athos enhanced modellers’ efficiency in model comprehension and creation. In Section 4.10, we defined the achieved PPM as a measure for model comprehension efficiency. The next sections will present the observed efficiency results of both studies.
Overview on Efficiency Results from Friedberg
Figure 12 illustrates the amount of time participants from the two groups of the Friedberg study spent at the given language section as well as the final score they achieved. Each of the four plots is divided into six parts. The part on the bottom left, for example, contains data points representing those participants, that spent between 0 and 30 minutes trying to solve the tasks with the respective approach and achieved less than 75 points. Data points in the upper right part represent those participants who scored more than 75 points within a time frame of 60 to 90 minutes. Hence, the ideal approach would have the majority of data points located in the upper left section, i.e. participants would have been awarded more than 75 points in less than 30 minutes.
In the group that started with Athos as a first approach, efficiency dropped to some extent when JSprit was used as a second approach: with Athos, 24 participants scored above 75 points within 60 to 90 minutes. With JSprit, the number of participants who scored over half the points in 60 to 90 minutes reduced to only 15. However, with JSprit as their second approach, there were six participants who missed the 75 points mark by only six or less points.
Conversely, the group that started with JSprit as their first approach exhibited a massive efficiency gain with Athos as the second approach: with JSprit, only one single participant recorded 75 points in less than 60 minutes. In stark contrast, with Athos as the second approach, as many as 17 participants scored 75 points or above in less than 60 minutes.
Between Subjects Comparison
In order to compare how participants performed using either Athos or JSprit as a first approach, the two upper plots have to be analysed. A contrasting juxtaposition of the two plots suggests that participants scored higher in less time when using Athos as the first approach: with Athos, as many as 27 of the 35 participants (i.e. upwards of 75% of all participants from that group) scored more than 75 points. Three participants who started with Athos managed to do so in a time window of 30 to 60 minutes. With JSprit first, only 15 of all 34 participants (a mere 44.1% of that group) achieved more than half of all points and only one single participant achieved the score in a 30 to 60 minutes time window. Looking at the numbers of those who did not score at least half of all available points, with Athos only 8 (22.9%) participants fell short of achieving 75 points. With JSprit, on the other hand, as many as 19 participants (55.9%) lost more than half of all points. With Athos as the first method only six participants (17.1%) spent more than an hour on the tasks without achieving at least 75 points. By contrast, with JSprit first 14 participants (41.2%) spent over 60 minutes at the tasks and achieved less than 75 points. Used as a first approach, neither Athos nor JSprit saw a participant scoring close to 75 points in less than 30 minutes.
A comparison of the results for both approaches used as a second approach also leads to the conclusion that participants of the Friedberg study were more efficient when using Athos: with JSprit, only 17 out of 35 participants scored higher than 75 points (48.6%); with Athos, 21 out of 34 participants achieved more than 75 points (61.8%). From these participants, merely two JSprit users achieved their score within 60 minutes compared to 17 Athos users who achieved more than half of the points within a time frame of up to 60 minutes. With Athos as the second approach, there was even one participant who scored more than 75 points in less than 30 minutes (78 points in 19 minutes and 21 seconds). With JSprit second, the closest a participant came to achieving 75 points in under 30 minutes was a participant who scored 54 points in 15 minutes and 44 seconds.
Efficiency Results from Wetzlar
Having compared the efficiency results for Athos and JSprit when used by participants with comparably little programming experience, it is also interesting to see the results for both approaches applied by more experienced participants. Figure 13 illustrates the score and survey time of participants who took part in the Wetzlar study.
The two diagrams on the left-hand side of Fig. 13 illustrate that the group which started with Athos was noticeably more efficient with JSprit as their second approach. While the recorded scores seemed somewhat similar, the required time to achieve these scores was markedly reduced with JSprit. With Athos, all 14 participants scored more than 75 points, but only 9 (64.3%) did so in less than an hour. With JSprit, still all 14 participants scored more than half of the achievable points; with JSprit as the second approach, however, 12 participants (85.7%) finished in under 60 minutes.
Considering the right-hand side of Fig. 13, it is obvious that the group that used JSprit first and Athos second also exhibited a substantial increase of efficiency from the first to the second approach. Here, the increase was even more distinct: with JSprit all but one participant, i.e. 17 out of 18 (94.4%), scored above 75 points but not a single participant managed to do so in under 60 minutes. With Athos as the second approach, all 18 participants passed the 75 points mark and as many as 16 of these participants did so in under an hour. One participant even recorded 127 points in less than 30 minutes.
The plots in the upper half of Fig. 13 clearly show that a considerable number of participants solved the Athos tasks in less time than the respective JSprit tasks and still achieved a score of 75 or higher: With Athos, all 14 participants were awarded more than 75 points and 9 of these participants (64.3% of the entire sample) even managed to achieve their respective score in 30 to 60 minutes. With JSprit, 17 of 18 participants (94.4%) passed the 75 points mark but none of these participants finished in less than 60 minutes.
When used as a second approach, the time participants spent solving the tasks with the respective approach appear to be rather similar: With Athos second, 16 out of 18 participants (88.9%) scored more than 75 points in less than 60 minutes; with JSprit, 12 out of 14 participants (85.7%) collected more than 75 points in under 60 minutes. Though the survey times for the two approaches used as a second approach are not as far apart as when used as a first approach, participants who used Athos still spent less time than those who used JSprit. One of the participants who used Athos second even achieved more than 75 points in less than 30 minutes (127 points in 29 minutes and 56 seconds). With JSprit second, the closest a participant with over 75 points came to the half-hour mark was a participant who scored 128 points in 38 minutes and 18 seconds. Looking at the scores, it seems that Athos caught up with JSprit, especially in the number of high-scoring participants: With Athos second, 8 participants (44.4%) scored 135 or more points; With JSprit only 4 participants (28.6%) accomplished the same feat.
Test Results for Participants’ Efficiency
The results of the conducted significance tests confirm the observations made in the previous section. Table 6 summarises the results of the within-subjects significance tests for which we applied the Wilcoxon signed-rank test. In Friedberg, there was a significant reduction in achieved PPM when participants used Athos first and JSprit second. The other Friedberg group that had JSprit as the first approach displayed a significant increase in achieved PPM with Athos. In the Wetzlar study, there was a significant growth for either language when used as a second approach. There is however, a difference in the observed p-values for both within-subjects comparisons: the growth observed with JSprit as the second approach barely remains below the 0.05 threshold for statistical significance; in the other group where Athos was used as the second approach, the observed p-value of 0.000 leaves no doubt that Athos as a second approach substantially enhanced users efficiency in model comprehension and creation.
The results of the between-subjects significance tests are provided in Table 7. In the Friedberg study, both groups were significantly more efficient when using Athos: with our DSL – both as a first and as a second approach – the achieved PPM proved to be significantly higher than those obtained with JSprit.
In the Wetzlar study, results also hint at an increased efficiency of participants’ program comprehension and creation when using Athos. While the previous section showed, that if only the achieved scores were considered, the results of the group that used JSprit first were moderately superior to those who used Athos as a first approach. Putting these scores in relation to the time required to achieve them, however, shows that Athos was the significantly more efficient approach. Comparing both languages when applied as a second approach, also shows significantly superior efficiency results for Athos.
The presented results of participants’ efficiency (PPM) show that Athos substantially enhances the efficiency of end users. This is especially true for end users with limited knowledge in software development as the results from the Friedberg study group suggest. In all four conducted tests participants exhibited significantly increased efficiency results when using Athos. This clearly supports the hypothesis that Athos can positively affect programmers’ efficiency in model comprehension and creation.
The results of the Wetzlar study provide evidence in support of the claim that Athos has the potential to increase the efficiency of programmers who are experienced in the usage of GPLs and application libraries. Even though the group that started out with Athos and then used JSprit as a second approach exhibited significantly improved results with JSprit second, the three remaining tests were clearly in favour of Athos as the more efficient approach.
The result from the within-subjects comparison suggest that learning effects seem to be an important factor in the achieved efficiency. In both groups the second approach yielded superior results in terms of PPM. Performing a between-subjects comparison, however, shows that Athos significantly increases participants efficiency.
Results on Observed User Satisfaction
End user and/or domain-expert appreciation is a crucial factor that determines the success or failure of any DSL (Kahlaoui et al. 2008). Hence, we wanted to gain some insight on how participants perceived the usage of both languages. In order to be able to answer the question of whether participants felt more satisfied with Athos than with a traditional approach for modelling VRPTWs, we designed the final part of the survey so that they could express their opinion on both languages. More precisely, for both languages participants were presented five statements and asked to express their degree of agreement to the respective statement using a five-point Likert scale.
The answers obtained from both studies leave no doubt that participants preferred working with Athos over using JSprit to model/program VRPTWs. In both studies, participants expressed a similarly high appreciation for our DSL. By comparison, participants expressed distinctly less satisfaction with the JSprit approach. As was to be expected, the participants from Friedberg with less Java experience were even more skeptical towards JSprit than participants from the Wetzlar study.
Figure 14 illustrates the results obtained from the Friedberg study group. From 68 participants who answered the question whether they agreed that Athos was easy to learn, 81% gave their consent to this statement. For JSprit the same question was answered by 67 participants but only 22% felt that JSprit was easy to learn. A considerable 82% stated that Athos models were easy to read and understand whereas only 16% stated the same for JSprit. Nearly two out of three participants would affirm that modelling with Athos caused them little effort. With JSprit, however, not even one out of five participants was under the impression that model creation with JSprit was easy. A similarly high percentage was positive towards Athos’ support of fast and efficient modelling (79%) as well as its potential in helping to avoid mistakes (69%). With JSprit participants appeared distinctively more skeptical to support these claims with only around 19% attesting to a potential for efficient modelling and an even lower percentage (13%) feeling that the usage of JSprit could help to avoid mistakes.
The results from the study group in Wetzlar are illustrated in Fig. 15. As can be seen, the results obtained in this study are similarly positive towards Athos with agreement percentages ranging from 78% (avoidance of mistakes) to 97% (easy program/model creation). Compared to participants from Friedberg, participants from the Wetzlar study appeared to be slightly more positive towards JSprit, though agreement percentages ranging from 16% (easy program/model creation) to 38% (easy to learn) remained low in comparison to the results obtained for Athos.
Statistical Tests on Observed User Satisfaction
To test the obtained results for statistical significance, we applied the Wilcoxon signed-rank test on the answers for each statement. The test results are summarised in Table 8. The test results unequivocally prove that users were more satisfied with Athos than they were with JSprit. For both the Friedberg and the Wetzlar study, Athos received a significantly higher level of agreement for every single positive statement on a given language characteristic.
These results provide substantial evidence that Athos increases satisfaction levels of both language users with little programming experience as well as more experienced users at a junior developer level. Both type of users recognise and appreciate the effects that Athos has on the comprehension and creation processes required to understand and write models for VRPTWs.
Not only did participants from both studies express their perception that Athos was easier to be learned, they also left no doubt that they felt that Athos models were easier to be read and understood. What is more, participants did not only prefer Athos in order to read and understand already existing models, they also clearly expressed their preference for Athos when it comes to active creation of VRPTW models: participants stated that they considered model creation with Athos comparatively easy and effortless as well as fast and efficient. Finally, participants of both studies also stated their trust in the concise language syntax in supporting them in the creation of correct models.
As was already mentioned, a high level of user acceptance is a necessary requirement for the success of any DSL. Even though the presented levels of agreement are subjective perceptions of several language aspects rather than objective measurements of the respective language characteristics, they are no less important. If users experience the usage of a language in a negative way, it is likely that this will affect the results they produce with the repudiated language. On the other hand, if users feel that a given language is a helpful tool for a certain task, it is likely to positively affect the produced outcomes. This might even be reinforced by an easier and faster learning process that is not hindered by an inner distaste.
Quality of Teaching
In order to address deviations in the quality of the training material and the time granted for learning the approaches as possible threats to the internal validity of the study, we took care to not bias the results through either of the two means (see Section 6.2). To further ensure that our efforts were successful, we wanted to gain insight on whether they were recognised by participants of the two controlled experiments. To this end, we asked participants whether they considered the learning material provided for both approaches to be of similar quality and whether they had the impression that they were granted a comparable amount of time to learn both approaches. The results are presented in Table 9.
Asked whether they would agree that the learning material was of nearly identical quality, 60 of 69 participants (87.0%) from Friedberg stated their approval of this claim. Four participants (5.8%) deemed the Athos material superior and no participant considered the JSprit material superior. Five participants (7.2%) refused to answer the question. In the Wetzlar study group, 31 of 32 participants (96.9%) agreed that the learning material was of nearly identical quality. One participant (3.1%) preferred the Athos learning material. No participant opted out of the question.
Regarding the time granted to learn each approach, 58 out of 69 participants (84.1%) from the Friedberg study group would agree that the time for learning the approaches was nearly identical. Six participants (8.7%) felt that they were granted more time for learning JSprit, two participants had the impression that they were granted more time for Athos (2.9%). Three participants (4.3%) did not answer the question. In the Wetzlar study group, 30 out of 32 participants (93.8%) stated, that the time they had for learning both approaches was nearly identical. Two participants (6.3%) felt that they had more time to learn JSprit than they had for Athos. No participant felt that more time for learning Athos was granted and no participant chose not to answer the question.