Subsequently it is described how quality assurance in each area, i.e. in the USPECH workshops and projects, was carried out. The first part will focus on the projects’ quality assurance, and in the second part, the workshops are considered.
Ideally an evaluation must be planned from the beginning of each project when the objectives for the project are defined. By then it is possible to derive the content of the questionnaires, and each objective which is to be verified should be addressed by at least two questions in the questionnaire. All questions were uniquely formulated, and when the questionnaire was developed, it was tested first. In the test phase the questionnaires will be issued to persons who are not involved in the projects. Thus it is possible to detect some difficulties which may occur when the subjects are requested to complete the questionnaire, i.e. have to understand the respective questions. Once these difficulties have been corrected, the survey may start (cp. Moosbrugger 2012, 36ff). In the case of this study, standardised and introduced questions were used and adapted for the special case. The complete questionnaire was sent to test persons. They answered the questions from their point of view. After this they gave feedback where they had difficulty in understanding the questions. Based on this feedback, the questionnaires were adapted.
2.1 Elements of the Quality Assurance for the USPECH Projects
As first step of the quality assurance for the USPECH projects, a call for submission of project ideas was sent to all potential candidates who previously did participate in projects. The call included the following conditions: (1) the project must be innovative in the field of TVET, (2) the project idea must be performed for the first time, and (3) the entire project must be completed within 1 year (this period was later extended to 18 months).
From the proposed projects, only those projects which included the best ideas as estimated by GIZ (Gesellschaft für Internationale Zusammenarbeit GmbH - German International Collaboration Association) and seemed to be realisable were selected. Some more extensive project ideas were then discussed with the applicants in order to reduce the extent to a 1-year duration of the project.
To ensure the quality of projects and also to reduce the risk of failure of projects, different questionnaires were used (one for the start, two for intermediate results, and one for the final results) which were completed by all projects.
At the start of the project, all project teams received a questionnaire covering the following contents:
-
Objectives of the project
-
Milestones and the planned date of realisation
-
Project content
-
Support needed for successful implementation.
This first questionnaire was completed once by each project team, and based on the answers, the contents of the accompanying workshops were planned. Throughout the period of 12 months, a total of four accompanying workshops for all project teams were conducted by a group of international experts with the project managers as participants.
Each workshop covered a part of the project management as a topic. Thus, participants had received information about each impending project phase step by step. For example, in the first workshop, the participants were informed about important aspects of the project start. Furthermore, each workshop dealt with a specialised topic which was helpful for the implementation of the projects. The topics of the workshops were:
In addition, the status of the implementation process of the projects was discussed, evaluated and recorded through questionnaires in each workshop.
2.2 Findings of the Project Evaluation
In the questionnaire completed at the start of the projects, the respondents were asked specifically about the project objectives and planned milestones as well as the necessary support for each project. This was checked and analysed by the GIZ.
Great attention was given to the requirement that the objectives corresponded to the SMARTFootnote 1 model (Bernecker and Eckrich 2010, p. 226). In some cases the evaluator suggested a discussion with the project group to adapt the objectives, while the theory of the SMART model was a part of the content during the first workshops. A second part of the questionnaire covered the planning of the milestones. It came out that majority of the projects had a very realistic milestone planning and very few projects specified one milestone only. In these cases more important milestones were discussed in cooperation with the project group.
Based on this questionnaire, the actual planning of the workshops was then carried out, and the best possible support was given to all projects and all according implementation activities. Later it was shown that this was also an essential element to the success for all projects.
During the project phase, another questionnaire was used for observing the course of the projects. One aspect of this questionnaire constituted the motivation and satisfaction with the interim results of the projects. It was seen that it was satisfying as one may see that the motivation of project staff did increase over the period of the programme in all the projects. The satisfaction with the implementation of their own projects also steadily increased within all projects (see Fig. 17.1).
Due to the fact that the projects are close to completion at the time of reporting, there is limited opportunity to comment about the completion of the projects. However it was explained to respondents (usually the heads of the project teams) of all projects that they had to finish their endeavour in the planned time. The fact that none of the projects failed has shown that, among other things, continuous monitoring and controlling were helpful (cp. Fiedler 2016, 6ff).
2.3 Ensuring the Quality of the Workshops
To ensure the quality of the workshops, each workshop used a standardised evaluation form which consisted of two parts. In the first part, all participants wrote down their individual expectations at the beginning of the workshop. Only at the end of the workshop had all participants completed the second part of the evaluation form where they were asked about the workshops’ quality. This part included a set of questions about:
-
The quality of the execution of the workshop (eight multiple choice questions)
-
The quality of the results of the workshop (six multiple choice questions)
-
The possibility of transferring the knowledge acquired into the workshop towards the projects (two open-answer questions)
The participants received the questionnaire in paper form. In principle, it is also possible to offer questionnaires in digital form, for example, as a form on the Internet. However, to ensure that each participant has the opportunity to complete the questionnaire, the paper format was preferred.
All the evaluation forms were analysed using the statistical software SPSS.Footnote 2 Then the data was descriptive and variance analysis evaluated.
In order to ensure the influence of the findings, the experts who participated in the workshop were informed about the results of the evaluation by using digital documentations. In addition, the participants were informed about the results during the next workshop. With this procedure it could as well be demonstrated that the management of quality is an important part of each projects’ activities.
2.4 Results of the Workshop Evaluation
Throughout the USPECH programme, several workshops were planned in order to support the implementation of the projects. It turned out that participants’ expectation clearly confirmed the expectation of project-wise support. Indeed all respondents chose the respective item in the evaluation form from the second workshop onward.
At the beginning, the expectation was high that participants would refresh already known knowledge (cp. Fig. 17.2). This expectation even increased for the second workshop from 9% to 24% and to the fourth workshop to 54%.
All workshops have been performed a competence orientation which demanded awareness of the specific objectives for each workshop among all participants. In the evaluation this has been addressed in detail. Data shows that from workshop to workshop, more participants confirmed the statement: “The objectives of the workshop were presented”. At the first workshop, 70% of all participants did “totally agree”, while at the last workshop, 92% of the respondents answered that way. The author suggests that this is connected with the fact that all workshop leaders did receive the results of the evaluation.
Another item focused on participants’ satisfaction with the relationship between theory and practice. Along with this item, an increase in the number of participants who responded “totally agree” has been clearly observed.
In the last part of the evaluation form, factors including the growth of competence in teaching were measured. Results showed that the work focused by the lecturers also led to an increase in skills among participants. Here it was measured that in the third workshop, 58% of the participants did show a considerable increase in competence for teaching, while at the fourth workshop, 100% of the participants showed such growth.