Introduction

Thirty years after establishing the concept of sustainable development (Harlem, 1987), the United Nations (UN) presented, in 2015, the 17 Sustainable Development Goals (SDGs), within the framework of the 2030 Agenda for Sustainable Development. It is a plan of action in favour of the people, the planet, and prosperity. It also seeks to strengthen universal peace and access to justice (United Nations, 2015). This organisation is aware of the importance of education for all of us to get involved in trying to achieve these goals and reiterates this in the establishment of SDG 4 Quality Education (Ferrer-Estévez & Chalmeta, 2021). It states that, before 2030, it is necessary for students to acquire skills and knowledge to promote sustainable development, including: introducing education for development and sustainable lifestyles; human rights; gender equality; promoting a culture of peace; disseminating the concept of global citizenship; acceptance of cultural diversity; and identifying how culture can contribute to sustainable development. It is a matter of acquiring, through education, skills, attitudes and behaviours that lead to a sustainable future.

Therefore, it seems necessary to incorporate SDGs into educational models, reorienting curricula, programmes, practices, and policies (Ferrer-Estévez & Chalmeta, 2021). Educational change is needed at different levels involving educational governance, educational institution managers, educators, educational content, and pedagogies. This change is not occurring at the right speed and currently few universities are taking it on (Albareda-Tiana et al., 2018; Leal Filho et al., 2019). Barriers to this delay include lack of awareness or knowledge of sustainable development principles; lack of support from higher managers; resistance to change; the perceived irrelevance of sustainability to some disciplines; a sense of threat to academic freedom or integrity; time or funding available; and an oversaturated curriculum (Ferrer-Estévez & Chalmeta, 2021).

However, while these more structural changes are being introduced, action can be taken at other levels. For example, niches of action can be sought in specific subjects by adopting appropriate teaching methods and pedagogical approaches for the introduction of SDG-related topics. Although the number of initiatives in this direction is growing, there is still little literature on the subject (Ferrer-Estévez & Chalmeta, 2021). In Albareda-Tiana et al’s (2018) study, the potential of holistic strategies to connect theory with practice is highlighted. To enhance the necessary reflection, it is important to use active teaching–learning strategies where real problems related to SDGs are worked on, which allow going beyond theoretical knowledge (Albareda-Tiana et al., 2018). It is important to engage students and motivate them to do research on their own. In this context, teaching–learning methods related to student-generated content (SGC) are aligned with these premises and they seem quite promising (Ribosa & Duran, 2022; Snowball & McKenna, 2017). The activities associated with SGC can be performed individually or in teams, inside or outside the classroom and the results obtained are self-made content that allow students to demonstrate the acquired learning. The content can be performed in different formats including audio-visuals, Q&A’s, lessons learned, etc. (Arruabarrena et al., 2019; Jaime et al., 2022; Ribosa & Duran, 2022). In addition to the learning achieved during the creation of the content, the generated materials (e.g., test-type questions or video tutorials) constitute in themselves additional study material from which the rest of the class can benefit (Bakla, 2018; Snowball & McKenna, 2017). SGCs can be developed in any subject and, therefore, can be useful for incorporating SDGs following Albareda-Tiana et al’s (2018) aforementioned suggestions. Despite the potential of this teaching methodology for the incorporation of the SDGs in education, there are hardly any works that explore this line of research.

To our knowledge, there is only one study with a similar purpose, where SGC (blogs) are used to complement learning in an introductory course on climate change (Schuenemann & Wagner, 2014). In that study, they detected improvements in attitudes and beliefs regarding climate change and observed a perception of the problem and an interest in learning that was higher than in the general North American population. The authors acknowledge that students who choose to study their subject may be more aware of the problem and more interested in it than the rest of the population (Schuenemann & Wagner, 2014). Our study attempts to contrast the validity of the methods associated with the SGCs in the incorporation of the SDGs into a curriculum (beyond SDG 13, climate action). In addition, most of the studies on the introduction of SGCs are carried out with small groups of students from a single university. It seems necessary to test a method that includes various groups of students from different universities. On the other hand, the analysis of suitability of different SGC formats does not seem to be covered in previous works. In this way, we intend to assess the possibilities of this method and fill this gap in the literature.

This paper describes the inclusion of the SDGs in Project Management subjects of the Degree in Computer Science of two Spanish universities working in a coordinated manner. In particular, it arises from the collaboration between Universidad del País Vasco/Euskal Herriko Unibertsitatea and Universidad de La Rioja. This experience was launched in the 2019–20 academic year and has been replicated, introducing adjustments and improvements, for the 2020–21 and 2021–22 academic years. The work on the SDGs is integrated within the framework of its relationship with the ethical and deontological aspects that are part of the competencies of Project Management. In particular, which is the impact that Information and Communication Technologies (ICTs) can have on the SDGs. The experience is based on the application of the project-based learning (PBL) strategy, where students carry out, in teams, three successive projects where different student-generated content is obtained. In the third project, each team creates a website that integrates contents elaborated in the previous projects, together with newly created materials. The projects follow a common model of teamwork that encourages the exchange of information and the sharing of opinions. The generation of reusable resources brings some advantages observed in the literature, such as greater student engagement; improved learning with respect to the use of teacher contents; a positive opinion about the activities related to the generation and use of materials; and a good level of satisfaction with the experience (Bakla, 2018; Ribosa & Duran, 2022). In addition, student contributions foster self-recognition, collaboration and reflection, and have greater dissemination and impact on peer learning (Bakla, 2018; Ribosa & Duran, 2022). The main objective of this research is to analyse the interest of implementing learning projects, where websites that integrate different types of student-generated content are built, as a methodology to introduce the SDGs in the university curricula. Through this research, we aim to answer the following research questions about the series of three learning projects:

  1. 1.

    Does subjective knowledge about the SDGs improve?

  2. 2.

    What SGC generation or use activities are perceived as most effective for learning?

  3. 3.

    What is the awareness of the influence of ICT on the SDGs?

  4. 4.

    Are improvements in learning and awareness independent from the learning context (instructor, university…)?

  5. 5.

    Is the perceived learning, by generating and using SGC, related to awareness of the SDGs?

  6. 6.

    Is satisfaction with the SDG themes related to the learning, the awareness of the SDGs and the satisfaction with the SGC?

In this paper, firstly, a study of the background that supports this research is carried out. On the one hand, we explain the SGC and, on the other hand, the existing approaches to introduce SDGs in university education (“Background” section). This is followed by a description of the SGC methodology used, which takes the form of the development of three projects (“A series of three projects for learning about SDGs” section). Subsequently, the research method is described (“Research Method” section), the results obtained are detailed (“Results” section), and these results are related to those obtained by other authors (“Discussion” section). The paper ends with a section on conclusions.

Background

Student-generated content

In the traditional learning model, students are merely consumers of content, but the SGC model places them as the role of creators. A basic idea of the SGC is that students' creative activity fosters learning while introducing a change in the role of teachers, de-emphasising their role as knowledge transmitters (Doubleday & Wille, 2014). In addition, student-generated products can be used for other activities such as peer review or formative assessment, for example in the case of student-generated questions (Doubleday & Wille, 2014; Ribosa & Duran, 2022). Authors have tried to situate SGCs in different learning paradigms and pedagogies. From the point of view of content creation, one of the most mentioned is active learning, although it is also related to other types of learning such as learning by teaching, project-based learning, experiential, inverted, question-based (creation), generative, or collaborative learning (Bakla, 2018; Doubleday & Wille, 2014; Persada et al., 2020; Ribosa & Duran, 2022; Ryan, 2013; Yang et al., 2016). From the point of view of the use of content produced by other students, it relates to e-learning, peer learning, or question-based learning (use). It is also related to the pedagogy of the student contributor, based on constructivism and sociocultural constructivism (Ribosa & Duran, 2022). On the other hand, SGC is compared to the concept of user-generated content, related to the way in which knowledge spreads on the internet, since SGC usually involves the publication of content and, therefore, shares some advantages such as speed of expansion, collaboration and diversity of content (Persada et al., 2020).

In relation to learning, different experiments have been published that identify how students are adequately trained with SGC activities and are generally satisfied (Bakla, 2018; Persada et al., 2020; Yang et al., 2016). Students' opinions about the methods are generally positive regarding their benefits for learning (Snowball & McKenna, 2017) and the acquisition of other types of skills, such as digital skills (Bakla, 2018) or cross-curricular skills (Doubleday & Wille, 2014; Ribosa & Duran, 2022). More positive opinions about SGC are detected in students who achieve better grades (Bakla, 2018).

The literature proposes different strategies for the generation and use of student-generated content. Since it is about content produced for learning, the activity should not be limited to the generation of materials (Ribosa & Duran, 2022; Ryan, 2013). In the cycle presented in Fig. 1, an attempt has been made to cover all the stages, where the three basic ones are highlighted in grey background. For example, the cycle can begin with the generation of contents, which are then published in a repository accessible to students (or to any Internet user). In some cases, instructors may intervene to verify the correctness of the generated materials (stage marked in dashed line), who add extra reliability to the published content. Obviously, it is possible to publish the generated contents without verification, only the verified ones, or to mark which of the published ones have been verified. Students can then search for interesting materials in the repository, although in some cases instructors can prepare a selection and present it to the students. These learning materials are used by students to learn, either by viewing them, reading them, using them to study for an exam, or initiating a process of reflection and discussion.

Fig. 1
figure 1

SGC cycle

Full size image

The cycle could also begin by using (consulting) materials made by other students, for example, in previous courses. This use of materials prior to the generation of new content can be useful to consider different alternatives and encourage creativity. The use of materials made by other students gives rise to the possibility of assessing materials (likes, Likert scales…) or providing feedback or improvements, for example, in the form of comments. The content generated may be original or the creation of derivative works, that expand or improve pre-existing materials, may be requested. Within the derivation, the generation of new versions that rectify errors or deficits pointed out by other students in their comments and improvement proposals can be considered. It is also possible to elaborate content that are the result of compiling a selection of good pre-existing materials, for example, to create a quiz with test questions made by peers or a channel with videos covering the different parts of a topic.

With respect to content generation, different studies find that students are able to generate good-quality content by working collaboratively and with effective guidance from their teachers (Bakla, 2018; Galloway & Burns, 2015; Yang et al., 2016). Students begin their learning by studying a set of materials provided by instructors or through a research activity (Ryan, 2013). In the latter case, students often find enough quality resources on the Internet to tackle this task (Persada et al., 2020). The content produced by students is an assessable and valuable element, but so is the process followed, which, in some cases, is also observable and assessable (Doubleday & Wille, 2014). Both final contents and intermediate inputs provide instructors with valuable information about how students understand concepts (Doubleday & Wille, 2014) and allow them to identify problems in concept interpretation. Emotions associated with the generation process have also been studied. Ryan (2013) identifies some initial fear and discouragement, followed by enthusiasm and enjoyment, and ending with an overall positive feeling and satisfaction with the product achieved.

The content generated by students can be of different types. The first would be textual explanations, for example, with tutorials, narratives, lessons learned or documented software code, which are sometimes reinforced with figures, internet links or third-party audio-visuals embedded in the text (Jaime et al., 2022; Lambert et al., 2017). Other more specific textual content follows the question–answer format and are generally aimed at formative assessment (Bakla, 2018; Galloway & Burns, 2015). The questions can be multi-choice or of other types. The third type is audio-visuals, either photographs, audios or videos (Arruabarrena et al., 2019; Doubleday & Wille, 2014; Snowball & McKenna, 2017). The availability of free and simple computer tools for creating or editing audio-visuals makes this type of content a very interesting option. Some tools make it possible to create simple educational games (Ribosa & Duran, 2022; Schina et al., 2020). Finally, we highlight the containers of third-party materials, whether made by students or not. Containers usually contain homogeneous materials, such as a set of textual explanations, a set of audio-visuals, questions and answers, etc. In this case the task of the students is to elaborate a selection of quality materials (Arruabarrena et al., 2019). Although some types of content are certainly more suitable for acquiring certain types of competencies than others, we did not find studies that comparatively analyse their suitability.

Although students are able to generate good content, there are also materials with errors, deficiencies or are of low-quality. Students show their distrust of the materials when they identify problems and this influences their use of the materials for learning or subject preparation (Bakla, 2018; Ribosa & Duran, 2022). For this reason, many authors perceive the need to introduce a verification process, on the part of instructors, that filters or highlights quality products and encourages students to use the products created by their peers (Snowball & McKenna, 2017). However, in some cases students appreciate some advantages of identifying errors in materials made by peers (Bakla, 2018). The verification process allows instructors to identify the content that involves greater difficulties or even the presence of misinterpretations of certain concepts (Doubleday & Wille, 2014).

Publishing materials, either directly on the Internet or by restricting access to them to students in the course, leads to the so-called audience expectation effect. This means that students understand that their work will be exposed to criticism (positive or negative) from other people. This effect is found to have a positive influence on students' engagement during the generation of materials and on learning itself (Bakla, 2018; Ribosa & Duran, 2022; Snowball & McKenna, 2017). All types of repositories available on the internet are used to publish content. The most common are blogs, video channels, tools specialised in questions (such as PeerWise), wikis, websites, game development tools or gitHub to publish programme code (Bakla, 2018; Jaime et al., 2022; Ribosa & Duran, 2022).

The use of content is preceded by consulting public repositories or reviewing specific elements chosen by instructors. In some studies, students report learning more by generating content than by using peer materials (Doubleday & Wille, 2014; Ryan, 2013), while in others they understand that both experiences are equally beneficial (Snowball & McKenna, 2017). Published contents can receive contributions and evaluations from peers that make it possible to improve their quality, turning learning into a social process (Ribosa & Duran, 2022; Snowball & McKenna, 2017). In this way, students have the possibility of actively participating in the improvement and refinement of content, although students do not always have enough time for this activity (Bakla, 2018). Some comparisons have been made of the use of content elaborated by teachers and students in favour of the latter, observing in some cases a higher level of engagement, effort made and comprehension (Lambert et al., 2017) and in the case of test questions, better academic results (Sanchez-Elez et al., 2014).

SDGs in university education

Society's concern about SDGs has been transferred over to the university system. Some authors point out that SDGs should be part of university education (Leal Filho et al., 2019) and that universities should promote a culture of sustainability (Albareda-Tiana et al., 2018). Although a growing number of initiatives propose the inclusion of SDGs in education (Ferrer-Estévez & Chalmeta, 2021), there are still few proposals mentioned in the literature (Ferrer-Estévez & Chalmeta, 2021; Maharjan et al., 2022). In Albareda-Tiana et al’s (2018) study, the coverage of SDG-related concepts in a university curriculum is analysed to identify the different ways of including them. The study observes the generalised lack of this content in the curriculum despite the fact that educators agree on the essential role of the university in the transmission of these topics. The same study (Albareda-Tiana et al., 2018) highlights the potential of holistic strategies to connect theory and practice. Moreover, in order to enhance reflection, it is important to use active teaching–learning strategies and to consider real problems related to SDGs, which allow going beyond theoretical knowledge (Albareda-Tiana et al., 2018).

Some recent proposals for the inclusion of SDGs can be found in the literature, which follow the suggestions just explained, of which we highlight three. In Schina et al’s (2020) study they use PBL to promote SDGs from the field of teacher training. They apply it to a group of 21 students starting with an instructional phase on SDGs. Students are then asked to come up with, in teams, a game for a child-oriented educational Blue-bot. Topics are pre-assigned and include poverty and hunger awareness, marine pollution, social, financial and political inclusion, wildlife protection and saving clean water. Students analyse the project topic, develop learning materials for children, implement the project and self-assess it. This type of project allows for the improvement of students’ digital competence and to reflect on the work carried out. The research question of this work is: to what extent does the experience help to develop skills to design educational projects that integrate SDGs? The answer is obtained through three opinion questions, a self-evaluation question, a peer evaluation question and a teacher evaluation question. The answers are numbers on a scale from 0 (inadequate) to 3 (very good) regarding the SDG content of the projects. The mean of the five projects carried out is 2.8 for self-evaluation, 2.35 for peer evaluation and 2.4 as teacher evaluation. These results conclude that all five projects were useful in addressing an SDG (Schina et al., 2020).

A different proposal is found in Tsai’s (2018) study, where classroom discussions are organised in order to reflect on sustainability concepts. The experience was carried out with 68 undergraduate students from general educational classes. Through a quasi-experimental study, a group provided with debate topics is compared with other groups that do not debate and, instead, use textbooks, following a more traditional type of teaching. They measure some aspects such as students' environmental, social and economic attitudes. They find differences in attitudes towards sustainability, concluding that the introduction of debates in the classroom facilitates the development of such attitudes. Although differences are observed in environmental and social issues, they were not found in economic issues (Tsai, 2018).

The third proposal highlighted is detailed in Janakiraman et al (2021) study where digital games are used, again with 52 students from general educational classes. The games used were designed to create sustainability environments, e.g., environmentally friendly, sustainable and profitable cities. The aim of the game is to examine, in an immediate way, what the consequences of changes in people's behaviours are. The results of the study were measured by means of a questionnaire 1 week later and show that the use of the game for 1 week has a significant influence on affective, behavioural and social learning. However, it had no influence on cognitive learning. Five weeks after the game experience, the differences remained only in the social learning aspect. They attribute the lack of differences in cognitive learning to the fact that students are constantly exposed to knowledge about the environment through news, seminars, or social campaigns. They also argue that the social aspect can be maintained because players try to discuss and share a winning strategy, even weeks after they have played. Players remark the emotional impact and that in-game behaviours are directly transferable to their real life, which could be maintained for long periods of time (Janakiraman et al., 2021).

As previously noted, to the best of our knowledge, there is only one study where a goal (SDG 13, climate action) is introduced through the completion of SGCs in a special subject on climate change where 67 students generate blogs (Schuenemann & Wagner, 2014). The study by Schima et al. (2020) just explained follows a similar path but focused on generating games for children rather than useful learning contents for degree-level peers. In our case, following the guidelines of (Albareda-Tiana et al., 2018), students follow active strategies to create SDG-related contents but, in addition to being assessed by teachers, some content is assessed, selected and reused by peers (from the same course or later) to create more ambitious contents. In addition, the experiments we have just explained have been carried out with small groups of students from a single university. In our case we considered six groups of students, three from one year and three from another year, and of the six groups, two are from one university and the rest from another. On the other hand, we intend to analyse comparatively the suitability of different SGC formats, an aspect that, as we have mentioned, does not seem to be covered in the literature.

A series of three projects for learning about SDGs

The project management discipline is commonly part of computer science degrees (PMI, 2013). Project management courses introduce ways to organise and manage resources, both human and material, to accomplish project tasks within predefined scope, time and cost constraints (Nembhard et al., 2009). In this work, we propose a series of three learning projects whose outcome is content addressing the computer engineering profession and its relationship to SDGs. Although the projects’ main goal is to acquire project management skills, both reflection on the SDGs and the profession is also promoted while generating new content or selecting or making derivative works from content made by other peers. SDG-related content is not assessed in exams or other tests, and is used purely for formative assessment.

The content generated by the students in the first project is a video; in the second project a channel containing two new versions of existing videos; and in the third project a website that hosts content created in the previous projects (including content developed by peers from the same or previous years, or even from another university) plus other content created in the website project itself.

The video project

This project lasts 3 weeks and is carried out by student teams of three or four people. The overall theme of the video is the influence of ICTs professional practice on SDGs, but the video has to be limited to a specific SDG. For example, in 2021, teams chose one SDG from among the following five: (4) quality education, (8) decent work and economic growth, (12) responsible production and consumption, (13) climate action, and (16) peace, justice and strong institutions. Exceptionally, a team is allowed to select a different SDG, if previously agreed with the instructor. The video must be published on the Internet and its content must be oriented to computer science students. The audio-visual part must also include original elements and that, when third party elements were used, the appropriate permissions or licences from their authors should be available and their authorship acknowledged in the video itself. The video must include a creative commons licence in its credits.

The project presentation document, provided to the students, suggests some materials on each SDG. It also provides some examples of videos, made in previous courses on other SDGs, with different video formats such as dramatised, with animations or with annotated images (Arruabarrena et al., 2021). Both the materials and the example videos are available online.

The examination of these materials leads to a discussion process in each team to select the video's theme, scope, and style. Then, each team elaborates the outline and records scenes or generates them with a specialised programme. In many cases, appropriate audio-visual materials are found online (videos, photographs, music…). Afterwards, the video post-production is tackled, where the scenes, third party materials, voice-overs and, in many cases, background music are assembled. Students perform this task with any video editing programme of their choice. The process culminates with the publication of the video in a video channel provided by the instructor.

All these tasks involve documentation, reflection and sharing of various aspects related to the SDGs. They also encourage initiative and creativity, both individually and as a group. On the other hand, the product must be completed within 3 weeks and with a personal dedication time, outside of class sessions, of around eight hours. Before initiating each project, participants engage in a reflective review of a presentation document. Additionally, reading relevant reference materials is expected to aid in product development. The six hours of classroom time allocated to the project primarily focus on task comprehension and work organisation. This involves coordinating shared resources, as well as formulating and monitoring a comprehensive plan. Consequently, each team of four members is allotted approximately 32 h for individual work outside of scheduled class sessions. These limits both the exposure of points of view and approaches as well as discussions on the impact of ICTs on SDGs.

The channel project

The objective of the project is to make a new version of a couple of videos and host them in a channel that meets minimum configuration requirements. The new version of each video should include substantial changes on the content of the original work (outline and audio-visuals) or be complete remakes. This project has the same duration, team composition, and general theme as the previous one.

The list of videos that will be used as a source to create new derivative videos is proposed at the beginning of the project. These are videos created in the same or previous years that meet the specifications of the video project and are of good quality. Therefore, the list can include videos that address some SDGs different from those of the previous project. Each team selects four videos from the list, which should be from different SDGs, and proposes a set of interventions for each one. The teacher selects the two most interesting proposals from each team, with the intention to ensure that the greatest variety of topics is covered amongst all teams.

The resulting videos should be subtitled on the channel, at least in the original language. In addition, whenever possible, the needs of people with hearing impairments should be considered. All videos in the channel must include the same creative commons licence. As in the previous project, the activities to be carried out promote the contrast of perspectives within the team; the exchange of opinions; and constructive critical reflection, aimed at achieving interesting results. To encourage debate and reflection, the client figure is introduced, who is in charge of emphasising some priority requirements, such as highlighting in the videos the influence of computer science. The deadline for the product and the hours of personal dedication are to the same as those of the previous product, again limiting discussions and the contrast of viewpoints on the impact of ICTs on SDGs.

The website project

The goal of the last project is to build a multilingual public website that presents visitors with three sections: videos, web articles, and self-assessment. All sections should cover the SDGs considered in the project presentation document. Normally, the SDGs worked on in the channel project are included, although others covered in previous years can be added. The materials should be grouped, in each section, according to the SDGs considered. The project lasts 8 weeks; is carried out by teams of six to eight people; and shares the same general theme as previous projects, i.e., the influence of the professional practice of ICTs on SDGs.

The video section includes between ten and twenty videos generated by student teams in projects of the same or previous years. Therefore, the completion of this section requires the review and analysis of videos of the SDGs considered in the project. The selected videos must be hosted in a channel compatible with the specifications given by the client in the project channel. In addition, it is requested that the channel has a short presentation video (between 20 and 30 s) where the content and orientation of the channel are introduced. The creation of a playlist for each SDG of the project is also requested. Each video on the channel will be linked to the playlist corresponding to its SDG. In addition, the videos must be subtitled in the channel in the languages considered in the multilingual website.

The web articles section will contain links to web pages that present articles, with complementary information related to ICTs, that allow the reader to read more in depth on the topics included in the website. The information displayed for each article consists of its title, which contains a link to the article's web page; a brief description of the article; the estimated reading time; and its level of complexity. This task involves the search and selection of interesting sources and encourages team discussion on the information found on all the SDGs considered in relation to ICTs.

In the self-assessment section, visitors will be able to answer a set of multiple-choice questions that will allow them to find out their level of knowledge or awareness of the issues raised. The answers to the questions should be found in a video or in an article accessible from the website. The website will have a collection of questions from which several questionnaires will be created, organised according to the project’s client indications. When visitors answer one of these questionnaires they will receive some feedback related to his/her level of knowledge or motivation with respect to the issues raised. Question generation is not a simple task and involves analysis and discussion of content generated by other students.

The entire project is expected to be finalised within an 8 week timeframe, requiring an individual time commitment of approximately twenty hours outside of class sessions (a total of 140 h for teams of seven members.). As in previous projects, students are required to complete a preliminary task of understanding the project presentation document. In addition, two hours of class time per week (16 h in total) are dedicated to comprehending the specifications of the product to be developed (including client interaction), formulating and monitoring a plan, and reviewing suggested materials.

Figure 2 includes a screenshot of a website project developed by a student group and Table 1 summarises some details of the project and includes some links to student generated examples.

Fig. 2
figure 2

Screenshot of the final appearance of the video section of a website

Full size image
Table 1 Some details about the projects and the tasks to be carried out
Full size table

Research method

The series of three projects presented in the previous section has been applied over two years in two third-year Project Management courses in a Degree in Computer Science of two Spanish universities: Universidad del País Vasco/Euskal Herriko Unibertsitatea (UPV/EHU) and University of La Rioja (UR). The same academics from both universities have been involved in both courses, working closely together to organise the projects in the same way and to ask for equivalent and interrelated products. At the end of the course, in the years 2021 and 2022, students answered several questions about the impact that the activities carried out had on their knowledge of SDGs; their interest in learning about these issues; the satisfaction and effectiveness of the activities carried out; and SDGs with highest priority related to ICTs. The specific questions are listed in Table 2. Question 12 is open-ended and is designed to allow students to suggest changes or improvements. Almost all questions use a scale between 1 and 5 where 1 means very few/little and 5 means very many/much.

Table 2 Final questionnaire on learning from the projects, interest in the topics, activities carried out and priority of SDGs with respect to ICTs
Full size table

We include a statistical evaluation in the study. We use the Student’s t-test to check whether two sets of data are significantly different from each other, and the Pearson’s correlation coefficient to test the correlation between two variables. We use the ANOVA test to verify whether there are differences on three or more sets of data, and in that case, we compare each pair of these datasets using the Bonferroni correction. Parametric conditions are verified previously to use these tests. When parametric conditions are not verified, the corresponding nonparametric tests (i.e. Mann–Whitney U-test, Spearman Rho correlation test, or Kruskal–Wallis test) are also acceptable. The K-means procedure will be used to group students into clusters. Finally, effect size is interpreted following the Cohen’s effect size benchmarks (Ellis, 2010), which can help to determine whether the observed differences or correlations are meaningful.

Results

A total of 301 students, from the 2021 and 2022 academic years and from two universities, did the series of projects proposed in this research. All of them fall within the age range of 20 to 22 years, with 85% being male. Regarding their origin, 95.9% of the students are of European descent, 1.7% are from South America, 1.7% from Asia, and another 0.7% from Africa.

Firstly, we studied both reliability and validity of the questionnaire (11 items, because we excluded the item designed to allow students to suggest changes or improvements in the subject). Reliability assesses the internal consistency of the items (Hair et al., 2009), whereas validity explores whether a set of items can confidently be combined to represent an underlying construct (Knekta et al., 2019). The instrument has a high reliability with a Cronbach’s alpha of 0.802. Regarding the second facet, we studied the factorial structure of the items. Both, the Barlett’s sphericity test obtained a chi-square of 937.94 (df = 55, p < .001) and the Kaiser–Meyer–Olkin measurement of 0.802 indicates the adequacy for a factor analysis to proceed (Hair et al., 2009). Employing a principal components analysis as the extraction technique, and varimax as the orthogonal rotation method. A minimum eigenvalue of 1 was used as the cut-off value for extraction. Table 3 summarises the variables’ factor loadings. It contains three factors which explain 59.71% of the total variance. All the variables have a factor loading bigger than 0.5 in the selected factor, usually considered excellent. Furthermore, there are no factor loadings bigger than 0.5 in the other factors. This fact demonstrates the absence of multifactorial items (Hair et al., 2009). The table also includes the internal reliability of each factor. Only the last factor (which has only two items) is lower than 0.6, the minimum standard suggested for basic research and usually admitted for exploratory factor analysis (Hair et al., 2009).

Table 3 Factor analysis results
Full size table

The first two questions in Table 2 were compared for the 301 students of the 2021 and 2022 academic years from the two universities. That is to say, we compared the subjective knowledge which they reported having before and after completing the series of three projects. These questions are answered with a value between 1 and 5. The mean (standard deviation) of what they knew before is 1.74 (0.79), and of what they know after carrying out the projects is 3.83 (0.64). Significant differences are observed between what they knew before and after the course (Student's test t = − 41.658***, p < .001 (Cohen’ d = − 2.91). In addition, the mean (standard deviation) of students' opinion on the impact that ICTs have on SDGs is 4.05 (0.71), reflecting a high awareness of the importance of technologies on SDGs; with 81.7% of the students report that they are quite important or very important, with only 2% indicating that they have little or very little importance. Students believe that it is interesting to include SDG content in Computer Science studies with a mean (standard deviation) opinion of 3.30 (1.05); with 45.9% stating that it is quite or very interesting, compared to 20.9% reporting that it is little or very little interesting. Finally, students are satisfied with the inclusion of SDGs as a theme of videos, channel, and websites developed in the course with a mean (standard deviation) opinion of 3.50 (0.89); with 54.8% showing a high or very high level of satisfaction, compared to 13% reporting a low or very low level of satisfaction.

Secondly, we compared the declared learning (between 1 and 5) among the different activities carried out in the projects by the 301 third-year project management students. The means (standard deviations) are shown in Table 4 and significant differences were observed among all the activities. After performing the Bonferroni test, between each pair of variables in Table 4, it is observed that some differences remain. In particular, Activity E (search for complementary web articles) remains above all the others. Activities A (generate the video) and F (generate self-assessment questions) remain above the rest (B, C and D).

Table 4 Comparison between the different activities developed in the project series
Full size table

Thirdly, Table 5 shows the comparison of means of the first five items in Table 2 between the different groups of students who took the project management course at the two universities in the 2021 and 2022 academic years. Significant differences are detected among the groups in their perceived knowledge of SDGs, both before and after carrying out the projects. But, despite this, no significant differences are found in the final opinion on the impact of ICTs on SDGs, nor in the interest in introducing SDGs in CS studies, nor in their level of satisfaction with the choice of SDGs as a subject matter. In the table, EHU refers to Universidad del País Vasco/Euskal Herriko Unibertsitatea and UR to Universidad de La Rioja. There are differences in the preferences for certain learning methods among student groups across different years, but no clear pattern emerges.

Table 5 Knowledge and Computer Science items among the groups of students considered (2021 and 2022 academic years in Universidad del País Vasco/Euskal Herriko Unibertsitatea, EHU; and Universidad de La Rioja, UR)
Full size table

Fourthly, a K-means classification of students was performed using the levels of knowledge declared both before and after the completion of the series of projects (the first two items in Table 2). The result of this classification with a clearer interpretation had three clusters. The option with two and four clusters was discarded because the first one grouped two clusters quite differently and the second one produced two clusters with a very similar interpretation. This classification is shown in Table 6 and we have called the first cluster from very little, I learn something; the second from almost nothing, I learn a lot, which is the largest; and the third from the intermediate, I learn something.

Table 6 Categorisation of students into three clusters according to their knowledge of SDGs before and after completing the project series
Full size table

The distribution of students in each cluster by university and year is included in Tables 7 and 8, respectively. Notably, students from different groups and universities are unevenly distributed across the clusters. The significant differences highlighted in the first two items of Table 5 persist in this distribution. For instance, UR students' limited knowledge of SDGs before enrolling in the Project Management course is evident in their scarce representation in the third cluster.

Table 7 Cluster student distribution of the groups of students considered (2021 and 2022 academic years in Universidad del País Vasco/Euskal Herriko Unibertsitatea, EHU; and Universidad de La Rioja, UR)
Full size table
Table 8 Cluster student distribution of the groups of students considered (2021 and 2022 academic years in Universidad del País Vasco/Euskal Herriko Unibertsitatea, EHU; and Universidad de La Rioja, UR)
Full size table

Fifthly, a comparative analysis was done, shown in Table 9, which shows, from the third to the eleventh item, a comparison of the means obtained by the students of the three clusters. Significant differences are observed in almost all items. The exceptions are the seventh and ninth items, on the video review activities, and the tenth item on the web article search activity. After the Bonferroni review, differences remain between the first cluster and the second, and sometimes the third cluster intervenes.

Table 9 Comparison of means between the three student clusters
Full size table

The results in Table 9 complement the results shown in Table 4 on the effectiveness of the different SGC activities. Here, no differences are found in the activities of reviewing and selecting videos (items 7 and 9) or in the search for web articles (item 10) among the clusters, denoting a certain consensus among them. In the rest of the activities, differences are observed between the clusters which, after Bonferroni, are maintained between the largest group (the one that learns the most) and the one that starts from little knowledge and has managed to learn something, in favour of the first group. Therefore, there is a clear order in the effectiveness of the activities with a fairly uniform opinion in the three clusters.

Sixthly, Table 10 shows the correlations between the responses to the questions in Table 2. For clarity, the comparisons between the questions on activities (those in Table 4) have been removed, where all the correlations are significant with p < .001. The first item, which asks about prior knowledge of SDGs, only correlates significantly with the second item, on knowledge of SDGs after carrying out the projects, and with the fourth item on interest in including SDGs in the university curriculum. The remaining items appear independent of this first item. The other comparisons are significant with two exceptions. The first exception is the comparison between the second item, on the knowledge of SDGs after carrying out the projects, and the ninth item, on the video review activity of the project website. The second exception is the comparison between the third item, on the impact of ICTs on SDGs, and the sixth item, on the video generation activity of the first project.

Table 10 Spearman’s Rho correlation coefficients between the items of the questionnaire of Table 2
Full size table

The correlations in Table 10 complement the results observed in Tables 4, 5, and 9. Significant correlations are observed between all SGC activities and questions 2 and 3 on SDG awareness (ICTs and SDGs and inclusion in studies), indicating that SGC activities are related to SDG awareness. They also correlate with each other for all the SGC activities studied in Table 4, indicating cohesion among all the types of activities conducted. In Table 9, we have seen the relationship between learning and SDG awareness and the correlations show that knowledge after SGC activities correlates with SDG awareness questions 3 and 4, while prior knowledge only correlates with question 4 (SDGs and CS studies), reinforcing the relationship between learning and awareness. Finally, with respect to satisfaction with SDG topics, Table 9 showed differences with respect to learning achieved and, in Table 10, we see that satisfaction with SDGs correlates with all SGC activities, with prior and subsequent knowledge, and with the questions on SDG awareness.

Finally, Table 11 summarises the suggestions and comments in the open question of the survey. Contributions are categorised by theme, and it's worth noting that many responses included multiple contributions. Most of the answers focus on work-related topics, where students propose specific areas for exploration within computer science and various SDGs. Quite a few responses suggest expanding the number of SDGs or the number of specific topics to be worked on in the same course. Others recommend increasing examples showcasing the impact of computer science on SDGs. The second block with the highest number of responses pertains to students who refrain from providing specific suggestions. A notable group of students have no suggestions, and there are a considerable number who recommend the continuation of the same activities carried out. The third category centres around video development. Here, responses show less alignment in each suggestion, with some advocating for a reduction in the number of generated or utilised videos, while others propose the elimination of derived videos. The next block gathers suggestions for new content development, highlighting the possibility of writing reports or articles, or even creating a blog to host them. Furthermore, some creative proposals have been suggested, such as developing an infographic or curating a set of video channels related to the targeted SDGs. The subsequent section aims to engage the instructor more in learning about the SDGs. It suggests that the instructor organises an initial exposure to the working topics or selects materials for the same purpose. The remaining sections contain additional suggestions, with some presented separately as they pertain to specific sections of the website.

Table 11 Summary of the suggestions and feedback gathered in response to the open-ended question in the survey
Full size table

Discussion

The first research question is whether the realisation of the project series “improves subjective knowledge about SDGs”. We found significant differences in perceived subjective knowledge in students who did projects, before and after doing them. Different authors detect the influence of SGCs in the improvement of subjective learning (Bakla, 2018; Ribosa & Duran, 2022; Sanchez-Elez et al., 2014; Yang et al., 2016) despite including a certain degree of incorrectness (Bakla, 2018). In a study of the use of digital games to learn about the environment, they identify affective and behavioural learning of respect for the environment that is retained for 1 week and social learning that is retained for 5 weeks. However, in the aforementioned experience, no improvement in cognitive learning was observed (Janakiraman et al., 2021). Also, in Schuenemann and Wagner’s (2014) study, they observe that the SGC method is adequate to obtain subjective learning about climate change, although they apply it in an introductory subject on climate change.

As students have generated different types of contents related to SDGs, the second question seeks to identify “which activities of generating or using SGC are perceived as most effective for learning”. The results obtained indicate that there are significant differences among the six activities considered (Table 4). The two-by-two comparison, using the Bonferroni correction, allows us to order the activities starting with the search for web articles with complementary information as the most significantly effective (a few individuals even suggest extending this type of activity). This is followed by two activities: the generation of a video and the generation of self-learning questions. The previous analysis places, as the least effective activities, the generation of new versions of existing videos and the review and selection of videos made by peers. The creation and use of videos is the type of content that receives the highest number of suggestions from students in the open-ended question of the questionnaire. The feedback received reinforces the idea that making derivative videos is not as useful for learning as generating new videos from scratch. The comments also reinforce the idea of reducing the review and selection of videos made by peers. However, reviewing third-party videos is akin to searching for web articles, which is perceived as the best option for learning. Perhaps the apparent contradiction arises from the average quality of the reviewed videos, and seeking more professional videos could potentially change this perception.

We can observe that students perceive greater learning with content generation activities than with those that focus on the use of peer contents. This coincides with the findings of other authors (Doubleday & Wille, 2014; Ryan, 2013). Doubleday and Wille (2014) argue that this difference is caused by the time consumed in reviewing content and by the existence of poor-quality peer materials. It is remarkable that students highlight the search for web articles as the most effective activity for learning, which is the one that involves the greatest research load. We did not find studies that perform a similar activity or comparisons between the generation of different types of content. Our proposal follows the recommendation of Albareda-Tiana et al. (2018) to include SDGs in university studies using active methods that enhance reflection on their importance by including real scenarios. We have tried to enhance this reflection with activities of evaluation and selection of content from other colleagues and with the selection of materials from the internet.

The third research question asks about “what is the students’ awareness of the influence of ICT on the SDGs”. We have found that students exhibit a strong awareness of the pivotal role of technology in advancing SDGs, with over 80% acknowledging its substantial impact on SDGs, while only 2% state a minor influence. Moreover, they express a keen interest in incorporating SDG content into Computer Science studies through SGQ. Nearly 55% of students indicate a high or very high level of satisfaction, while only 13% report a low or very low level of satisfaction. Other authors also identify the influence of SGCs on students’ awareness of environmental issues and SDGs (China et al., 2020; Schuenemann & Wagner, 2014).

The fourth question tries to discern whether “improvements in learning and awareness are independent of the learning context (instructor, university…)”. Table 5 shows that the question on the impact of ICTs on SDGs has averages close to four points and that no differences are found between the groups of students. In the question on the interest of SDGs in CS studies, the averages are lower than the previous ones although they all exceed three points and no significant differences are detected between the six groups of students of the two years and the two universities. Similarly, no differences are detected in the question on satisfaction with SDG topics, with averages close to three and a half points. All this despite the fact that there are differences in the subjective knowledge about SDGs, before and after, between these groups. Therefore, the influence of the context in which the projects are carried out, i.e., the instructor or the university, on these opinions is not observed. It is not easy to find studies that allow us to compare this result. One study compared two groups of students who had used student-generated questions to study. One group had generated questions and the other had not. Despite the diversity of context, no significant differences in learning were detected (Sanchez-Elez et al., 2014), so the effect of the use of SGCs also seems to be independent of context.

The fifth research question seeks to identify whether “perceived learning from generating and using SGC is related to awareness of SDGs”. Three types of students have been identified according to the declared learning, with significant differences regarding knowledge before and after the projects (Table 6). The three groups contain students who (1) from very little, learn something, (2) from almost nothing, learn a lot, and (3) from something, learn something more. Differences are observed in items 3 and 4, influence of ICTs on SDGs and interest in SDGs in CS studies (Table 9), between the group that started with less knowledge and has learned something and the other two, in favour of the latter two. Thus, there is a certain tendency that the more knowledge achieved, the better the opinions on the relationship between SDGs and the degree course. Some reviews on the implementation of SGCs also note the relationship between learning and academic emotions, which include thematic and social emotions (Ribosa & Duran, 2022). Some studies observe differentiated opinions according to the learning achieved (Bakla, 2018), or positive emotions related to perceived learning (Ryan, 2013). The improvement of attitudes towards SDGs is also perceived with other types of activities, such as debates carried out in class (Tsai, 2018).

The last research question seeks to identify whether “satisfaction with SDG themes is related to learning, awareness of SDGs and satisfaction with SGCs”. On the one hand, Table 5 shows a fairly uniform degree of satisfaction (no significant differences) between the groups of students from both universities in the courses studied. Table 9 shows differences in satisfaction depending on the learning achieved, with satisfaction clearly lower in those students who perceive a lower level of learning. Table 10 also shows a positive correlation between satisfaction and perceived learning after completing the activities. In this same table, positive correlations are observed between satisfaction and the impact of the degree on SDGs and the interest in introducing the subject in the degree course. Positive correlations are also observed between satisfaction and all the activities carried out in the course. Therefore, the results allow us to affirm that there is a clear relationship between satisfaction with the way of introducing SDGs, with respect to learning, perceived interest and opinion on the activities carried out during the course. Several authors identify student satisfaction with the realisation and use of SGC (Bakla, 2018; Yang et al., 2016) and in some cases they also observe that the more learning achieved, the greater the satisfaction expressed (Orús et al., 2016). However, we did not find any studies that found a relationship between the perceived interest in the subject matter worked on and satisfaction.

Some aspects remain for future work. Firstly, we could conduct interviews or focus groups to complement our survey, aiming to obtain a deeper understanding of the process followed by students during the SGC and its relation to the knowledge and awareness about SDGs acquired. Secondly, a subgroup of students, randomly chosen, can be used as a control group to compare the objective knowledge and awareness acquired about SDGs through SGC. For example, this group can use the same SGC method but focus on a theme different from SDGs. Finally, the study could be expanded to include additional universities or countries to determine the generalizability of the results to different environments. Additionally, it would be interesting to investigate whether the findings are applicable to other engineering disciplines.

Conclusions

This work has shown and analysed an experience of inclusion of the 2030 Agenda and the SDGs in Project Management subjects of a Computer Science degree of two Spanish universities working in a coordinated manner. The proposal is not prescriptive in relation to SDGs and uses content generated by the students; encouraging learners’ reflection, proactivity and creativity. The different projects carried out by the students follow a common model of teamwork that encourages the exchange of information and opinions within the team, in the classes and between the universities involved. Through these projects, we have been able to complete the content of the subject itself, and also subjective learning of SDGs, increasing the understanding of the impact that ICTs can have on SDGs and students’ awareness on these issues. We have also found that more learning is perceived with information search and content generation activities than with those that prioritise the use of peer contents. Finally, it is observed that the method has similar results in different contexts and with different groups of students.

We believe that these results can inspire other institutions and degrees to incorporate the 2030 Agenda without waiting for a change in the curricula, which, if it comes, will not take effect until the graduating classes finish their studies before 2025. The generation of reusable resources gives additional value to the result achieved so far and will allow, in the future, a collaborative improvement and greater dissemination and impact of student contributions.

Finally, we should not forget that SDGs are part of a global social project, where educators are just one more agent, who does not have the endorsement of an academic authority based on prior training or certified accreditation. Sharing concerns, challenges and work in this area with students not only helps students to develop their skills, but also allows educators to reflect and learn about an issue that engages us all.