Introduction

Advances in digital technologies are transforming society and the way we live and work, with digital innovation being a major driver of these changes (Bogers et al., 2022). Digital innovation refers to the creation of new market offerings, business processes, or models resulting from the use of digital technology (Nambisan et al., 2017). This shift towards a digital future requires a new type of citizen who can function in more unstructured and unpredictable circumstances. Higher education institutions (HEIs) must adapt by offering effective, innovative, and high-quality learning experiences that equip students with the necessary skills for a changing labour market (Alexander et al., 2019).

As we move towards a more digital world, the demand for higher-order general cognitive skills such as problem-solving, critical thinking, innovation, creativity, and collaboration is increasing (Djankov et al., 2019). Additionally, socio-emotional skills such as collaboration, teamwork, resilience, and adaptability are becoming increasingly important (ibid, 2019). In the field of Information Systems (IS), the general skills needed are moving beyond technical expertise to include higher-level integration and the role of cognitive skills (van den Berg, 2019; Goulart & Liboni, 2022).

This paper focuses on defining the skills requirements of IS students to become competent digital innovators and to develop design principles to teach digital innovation, ensuring industry-informed curriculum design that is future-proof within a digital economy. Validated design principles that can be adopted by future developers to design learning environments that enhance digital innovation capabilities are proposed. These design principles were developed and refined via a four-phased design-based research (DBR) approach as proposed by Reeves (2006). This paper contributes to DBR literature by providing insight into the process followed to articulate validated design principles while also sharing reflections. The paper commences with a review of the development of digital innovation skills. Subsequently, the application of DBR and the methodology applied are presented. The results of the three iterations are then discussed by focussing on a portion of the analysis applied in each iteration. The paper concludes with the recommended design principles that can be applied to teach digital innovation skills.

Developing digital innovation skills

The focus of the study was on digital innovation and not digital transformation due to the ability to implement digital innovations in individual areas of an organisation whereas digital transformation refers to systemwide, long-term change. Digital innovation foregrounds new digital products and services, whilst digital transformation is the application of digital technologies to transform an organisation's operations and culture.

A digital innovator can see new possibilities created by advances in technology that meet organisational or societal needs (Fichman et al., 2014). Digital innovation is the recombination of something that already exists with new technology and follows a nonlinear pattern of innovation diffusion to change the ways products and services are developed, produced and used (Bogers et al., 2022).

In the study, the first objective was to identify the skills required by IS students to develop the competence to be digital innovators. The umbrella term used to describe the type of skills that students will need in the digital economy is “21st-century skills” (21st CS). Many different frameworks identify 21st CS, but the one deemed to be the most comprehensive was a study by Kereluik et al. (2013). They compared more than 15 frameworks to identify the types of knowledge claimed to be integral to 21st CS. This framework was used to test different skill sets and to identify the most prominent skills required to develop digital innovation capability. Figure 1 illustrates this framework.

Fig. 1
figure 1

Synthesis of 15 different 21st-century learning frameworks into one visual image (Kereluik et al., 2013)

Full size image

The development of certain 21st-century skills to teach digital business innovation can be enhanced by an authentic learning environment in which students must be “engaged in an inventive and realistic task that provides opportunities for complex collaborative activities” (Herrington et al., 2010, p. 1). The principles of authentic learning as outlined in Herrington et al. (2010) are described in Table 1 with a practical application attached to each principle as applicable to the cultivation of digital innovation skills.

Table 1 Implementation of an authentic learning environment in course design
Full size table

Design-based research

The study is situated within the paradigm of design research. Design science research in the field of Information Systems (IS) is characterised by the use of human creativity to develop innovative artefacts that address problems in digital environments (Hevner & Chatterjee, 2010). This type of research employs design cycles to test and refine solutions, with an emphasis on the design and implementation of novel artefacts that contribute to the advancement of the field of digital innovation (Hevner et al., 2019).

Design research applied in an educational setting follows a similar process of iterative development of solutions to complex and practical educational issues (McKenney & Reeves, 2019). The fundamental premise of design research is to apply theory to ground the design process, with the ultimate goal of expanding scientific understanding. This approach is collaborative, with input from multiple stakeholders representing different disciplines within iterative cycles of design, development, testing, and revision (McKenney & Reeves, 2019).

Design-based research (DBR) is an iterative approach that involves continuous design cycles within authentic learning settings to test and refine theories and advance practice. DBR studies use a mixed-methods design and involve multiple parties such as designers, researchers, and practitioners with diverse expertise to guide the design, conduct, and reporting of the research (McKenney & Reeves, 2021). This approach allows for the development of practical and effective solutions to real-world problems within educational settings and is an effective methodology for advancing educational practice (McKenney & Reeves, 2019; Reeves, 2006).

The four phases applied to this study are illustrated in Fig. 2

Fig. 2
figure 2

Adapted from Design-based research approaches in educational technology research (Reeves, 2006)

Full size image

Method

Phase 1: stakeholder consultation and a review of the literature

In a DBR study, the design principles are collaboratively developed among various stakeholders and underpinned by a review of the literature. The problems are explored by parties who deal with them on a day-to-day basis via a consultative process such as participant observation and conversation, interviews, focus groups or reflective journals and blogs (Herrington & Reeves, 2011; Herrington et al., 2010). In this study, consultations with industry participants, students and higher education practitioners in IS were conducted via interviews and focus groups see Table 1, Phase 1.

Phase 2: development of solutions

During this phase, the literature review was extended to find additional theories and existing design principles that address similar problems. This was further expanded to create the draft design principles.

Phase 3: iterative cycles of testing and refinement

According to Herrington and Reeves (2011), a single implementation cannot gather enough evidence about the success of the intervention prompting which prompted three iterations over three consecutive years. After each iteration, changes were made to improve the design to better address the problem in the subsequent iteration. Phase 1 and 2 required empirical research to be conducted and the data collection and analysis are depicted in Tables 2, 3.

Table 2 Methodology for data collection and analysis during the three iterations
Full size table
Table 3 Codes for testing skills sets and teaching and learning environment
Full size table

Phase 4: design principles

Once a learning design or intervention had been implemented, evaluated and refined in cycles, the last phase was to reflect on the entire process to produce design principles that could inform future development and implementation decisions. The aim is to provide at least three useful outcomes namely the design principles, a representation of the learning environment and societal outputs, such as professional development and learning (Herrington & Reeves, 2011).

Results

Phase 1: analysis of practical problems

In the first phase, consultation with industry partners who participated in the student project during previous years took place. The purpose was to understand their perception regarding the impact of digital innovation on organisations in South Africa, and further the type of skills required by IS students. The aspects highlighted included the development of social intelligence, creative thinking, and an innovative approach to problem-solving. Industry participants valued students who have been exposed to a “real work” environment via, for example, internships or projects in collaboration with the industry.

Six lecturers in the IS department at the University of the Western Cape (UWC) were consulted. They were concerned about the exact meaning and extent of digital skills requirements and how to prepare students for a digital economy. Limitations in the current IS curriculum were also highlighted as an issue to address. The practitioners further expressed their concerns about their practice and how this meets the requirements of a changing landscape.

A review of student perception related to skills requirements for a digital society took place before the first iteration. The students expressed the need to become more technically flexible. They also expressed the requirement to have more exposure to the industry and work on projects that deal with real-world problems.

The challenges faced by the different stakeholders are depicted in Fig. 3.

Fig. 3
figure 3

Challenges identified by students, teachers and industry

Full size image

Phase 2: draft design principles

The draft principles informed the development of the proposed solution, and the technological affordances identified also formed part of the process for drafting design principles. A mapping of the curriculum design principles to the actual learning environment, including the skill sets and authentic learning elements required (see Table 2), is depicted in Table 4

Table 4 Initial set of draft design principles with authentic learning elements and skills sets
Full size table

Phase 3: iterative cycles of testing and refinement

A Design-Based Research (DBR) approach was adopted to enhance the intervention through iterative cycles of data gathering, testing, and verification. After each iteration, the design was refined based on the findings. A review of each draft design principle was conducted by examining the authentic learning elements and skill sets associated with each principle (refer to Table 4) to analyse the outcomes. The analytical approach involved applying quantitative analysis first to test the skill sets acquired by learners (S1 to S8), followed by qualitative analysis to assess the presence of authentic learning elements in the course (A1 to A9). Subsequently, the content presented and the technology applied were analysed to determine the overall outcome of each draft design principle. The principles were updated and refined after each iteration to improve the intervention's overall effectiveness and refine the framework.

Given the extensive data analysis conducted over three years, it is not possible to present the results for each iteration for each section, such as quantitative, qualitative, project artefacts, industry feedback, and facilitator reflections. Instead, the focus will be on the quantitative analysis in iteration 1, the qualitative analysis in iteration 2, and project artefacts and industry feedback in iteration 3. The reflections on how the design principles were reviewed and updated are visually presented for the first two iterations.

Iteration 1

The first iteration took place during a first-semester course with a group of 40 postgraduate IS students. The objectives of the course were the identification, creation and implementation of digital innovation within a client’s business (industry partner). Groups could choose clients within the creative industries sector in Cape Town. As stipulated, the quantitative analysis is portrayed for the first iteration followed by the qualitative analysis in iteration 2 and project artefacts and industry feedback in iteration 3.

Quantitative results iteration 1

A regression analysis was conducted to determine the relationships between the variables (skills as indicated in Table 3 coded S1–S9) obtained in the survey results. The regression analysis applied student assessment scores as the dependent variable to test the importance of the different skills. The analysis tested the reliance on certain skills during an initial assessment and again at the end of the course. The analysis helped to identify the skills that were statistically significant using the p-value to test the null hypothesis. The collected data were analysed using Excel and the Statistical Package for Social Science (SPSS). The quality of results was verified with a hypothesis test where the null hypothesis was all the slope coefficients of the model equalling zero and the attentive hypnosis was that at least one of the slope coefficients is not equal to zero. The hypothesis is rejected if at least one of the independent variables explains the value of the dependent variable by reviewing the p-value. If the p-value is less than the level of significance, the null hypothesis that the coefficient equals zero is rejected; the variable is therefore statistically significant (Anderson, 2014). Typically you would like to produce a high R-value, thus a low p-value with a high R will indicate that the results explain the response variability. However, when one predicts human behaviour, lower R2 values are acceptable because humans are harder to predict (Anderson, 2014). For this analysis, the p-value, therefore, was examined more closely.

The second statistical test applied was a one-way ANOVA to test the differences between the students’ scores for their initial skills survey and the scores obtained for the second survey upon completion of the module. The purpose of one-way ANOVA is to test whether the means of different groups are common or different. The quantitative results are depicted in Table 5 below.

Table 5 Quantitative results iteration 1
Full size table

The dependent variable in the regression analysis was student assessment scores using the initial assessments (first blog post, initial presentations and peer reviews) for the first (pre) survey and the final assessments (final blog posts, industry presentations and reports) for the final (post) survey. Table 6 depicts the summary of findings applied to each iteration to review the overall results about skills development.

Table 6 Summary of skills data obtained iteration 1
Full size table

This process was repeated in each iteration and the results were analysed to review the skill sets that showed an improvement and to highlight areas where further interventions were required. See Table 7 for an example of the review of the multiple regression over the three iterations to show the progress.

Table 7 Multiple regression summary over three iterations
Full size table

The overall findings after each iteration were reviewed and the areas that were deemed to be satisfactory were highlighted in green, the areas that required improvement in amber, and the areas that needed intervention and new strategies in the following iterations were highlighted in red. These reflections are summarised in Fig. 4 below.

Fig. 4
figure 4

Outcome of iteration 1 and additional design principle added

Full size image

Figure 4 depicts the dashboard applied to summarise the overall outcomes, as seen several areas required further interventions in the following iteration. Only two of the draft design principles were successfully integrated and the following iteration required changes in the design of the projects with industry, the group formation, the assessment of students, coaching and scaffolding and the quality of feedback. After iteration 1, a new principle was added to allow all tasks to be funnelled into a comprehensive capstone project that applies agile methods. The application of Agile allows changes to group formation, regular feedback and the assessment of the overall process and not merely the end solution.

Iteration 2

The second iteration took place during a second-semester course with a group of 42 Information Systems students in their final year. The design of the project changed to incorporate an Agile methodology with clear team roles and regular opportunities for feedback by the facilitator and other teams. The assessments were designed to measure the outcomes per week to allow teams to make changes and improvements. A similar process of data collection was applied. As stipulated, the qualitative analysis is discussed in the second iteration but a similar process was applied to the analysis of the skills as described in iteration 1.

Qualitative results iteration 2

The steps prescribed by Miles and Huberman (1994) to systematically organise the data were applied in the qualitative data analysis. Firstly, data was organised and emerging patterns were identified from the different sources. Thereafter, data was coded about the key pedagogical and design principles identified in the literature and sorted into potential themes (see Table 2). The data analysis phases indicated in the DBR approach were followed through the iterative cycles.

During each iteration, students were tasked to subscribe to a blog and submit three blogs during the semester. The blogs were reviewed to find evidence of authentic learning elements to which the students responded positively in their learning.

The learning environment needs to enhance the ability of students to apply critical thinking to develop the capabilities to become digital innovators. This requires an authentic context (A1) to enable students to apply their knowledge as they would in real life to find their own solutions in the implementation of digital innovations. It also encourages interdisciplinary skills development in the implementation of the capstone project (updated principle). For example:

The course was very phenomenal because it teaches about the current issues facing the technological sector and how to improve business processes for an organisation using advance technology. (LJ2)

However, students also felt that the engagement with industry partners was not sufficient and that their projects were not “real” enough. An area that needed to be redesigned in the next iteration was the type of industry partners with whom students engage. Students ought to be partnered with industry partners that are active in the community for them to see a real change in terms of their digital innovations for example:

I would like to suggest that in the next group, the lecturer must identify companies to work with and actually make sure that relations are built beforehand because companies are disinterested in projects that are consultative. (St23)

Authentic tasks (A2) stimulate collaboration and consist of ill-defined activities that create a polished product with real-world relevance. They are complex and performed over some time to promote competing solutions and a diversity of outcomes. For example, as quoted by a student:

I found the incremental steps in developing a product interesting. It made me realise that all the small parts come together to form a product or final solution. (PG2)

Peer reviews and online feedback were utilised to achieve access to expert performance (A3). During iteration 1, students felt uncomfortable with this and more coaching was done to encourage participation. Students needed guidance on how to give and receive feedback, and this was an aspect that had to be built into the rubric to test the peer review process. Positive outcomes were achieved for example:

What I learnt from the exercise was that one does not see their mistakes but quickly notices them in someone else. What I mean by this is that the groups spotted what the other groups did not do or did wrong in their assignments but in actual fact, they also did not do the same mistake but they did not take note of it. (BN2)

An area in which students felt that they lacked expert performance was particularly their technical ability. Students will need more assistance from experts in the rollout of their projects, particularly in areas in which they are not that comfortable. As remarked by students:

More practical sessions to train students more about how to create a website from scratch or through using platforms such as WIX. (St24)

The provision of multiple roles and perspectives (A4) was explored more during the second iteration to try to encourage students to explore different avenues. More time was spent in class during which students had to work in their teams and analyse their chosen business from different perspectives. They were given a set of questions to answer and present to their peers regarding the industry forces that have an impact on their business, as well as the market forces and key trends that they envisage.

In the second iteration, the creation of a collaborative learning environment (A5) was expanded through the use of Google Drive. This was expressed by a student:

What I have noticed and learnt is that this module is presented digitally

figure a

well I guess it has to be because, after all, it is digital business innovation. (BN2)

However, group work and collaboration are a challenge and this needs to be monitored throughout to facilitate conflict resolution and teach students the necessary skills to cope in a group environment. As remarked:

Put more exercises which focuses [sic] on the individual because I don’t think personal development occurs much in group assignments. (ST2)

The importance of incorporating individual reflection (A6) was stressed in the literature and incorporated into the course from the first iteration. During the first and second iterations, this area was challenging for students as they were not familiar with reflective exercises, but the usefulness thereof was grasped by some students towards the end. This is expressed in the following quotes:

I did not enjoy being marked by my peers. (LJ7)

I didn't enjoy doing many self-evaluations. (St38)

I must say, blogging has really made me look deeper into topics and buzzwords in the world of science and technology. And this has for the first time challenged me as a Technology student t think outside the box and share my ideas about the coming future. (KW2)

Articulation to enable tacit knowledge to be made explicit (A7) was emphasised via regular presentations in class as remarked by a student:

The effective presentation and scrum session in class were very fundamental to help me to grow my understanding in the course. I have strong understanding now of how operations of the business function especially how to apply the knowledge from the course. (LJ18)

The principle of coaching and scaffolding (A8) can be achieved through assessment tasks that facilitate student engagement over time, with feedback generated by various sources. The facilitator needs to carefully coach the teams and put just enough scaffolding in place to enable teams to construct their understanding. It is always a careful balance though as some students need more and others less for example:

I wish we had more time with the lecturer, and that she made reference to projects she has done and part-took in, in the past and what she did when she was faced with hurdles and what happens when things do not go according to plan. (ST2)

Lecturer was somewhat repetitive when relaying learning material during group discussions which were distracting. (BM2)

The utilisation of authentic assessment (A9) is recommended to be integrated throughout the entire assessment process. In the second iteration, assessments were updated with more detailed rubrics to evaluate students on various aspects, such as their collaboration, communication, and content knowledge, at the end of the semester.

Figure 5 presents a summary of the review of iteration 2, following a similar process to that of iteration 1. It was observed that a new design principle was required, emphasizing the cultivation of “social change-makers” who implement digital innovations that benefit both businesses and society, as this was felt to be lacking.

Fig. 5
figure 5

Outcome of iteration 2 and additional design principle added

Full size image

Iteration 3

The third iteration took place during a second-semester course with a group of 31 students enrolled in a third-year IS course. After the first two iterations, it became apparent that the students did not learn enough from their industry partners and a different approach to partnering with entrepreneurs in the start-up phase of their business was tried. A start-up phase is more open to change and the opinion was that students would be able to propose initiatives to support digital innovations in the business.

The organisations that the teams worked with included a hair salon that produces natural hair products, a fashion designer, a recruitment agency, a guest house, a quantity surveyor and a winemaker. Table 8 highlights the overview and the results are depicted in (Table 9).

Table 8 Project team outputs iteration 3
Full size table
Table 9 Industry feedback questionnaire
Full size table

There was an overall improvement in the results obtained from the industry in the third iteration on the conduct of teams. In their engagement with entrepreneurs, the teams were better able to sell the benefits of the digital economy, according to the findings from the last section.

Phase 4: updated design principles

In the fourth phase, the draft design principles developed and updated during the three iterations were updated as portrayed in Table 10. It is important to note that the iterative process that culminated in the final design principles occurred over three years. As noted, the comprehensive data analysis that took place cannot be discussed in a single paper. The purpose of this article is to provide a view of the overall process that can be applied during a DBR study noting that “communicating the processes and outcomes of EDR studies can be challenging because these studies are typically large and complex and because their value to non-stakeholders is not always articulated” (McKenney & Reeves, 2021. p. 89).

Table 10 Course design
Full size table

Within the study, the students were active participants or co-creators of the research. Ethical approval was obtained and participation in the study was voluntary. The identities of students were protected and there were no risks to them for their participation in the project.

This was a collaborative effort, and much of the insights obtained came from continuous engagement among all the parties involved. Also, the engagement with industry participants required continuous interventions and tweaking of project results. It was not possible to pre-empt any interpersonal issues that occurred in the teams or requirement limitations experienced with the industry partners. The actions taken were different for the different groups and resulted in different reactions. However, it is difficult to gauge whether the positive and negative outcomes were a result of the interventions, or other, external factors.

The updated course design is depicted in Table 10 with more detail about the design principles and a summary of the updated design principles in Table 11.

Table 11 Updated design principles
Full size table

The updated design principles were implemented in the curriculum design of exit-level IS courses at the university. The principles continue to be tested and refined within the courses. Student projects are now fully interdisciplinary with groups from different universities and different disciplines working collaboratively to implement digital innovations. The student projects also include a strong emphasis on sustainable development goals (SDGs) during the scoping. The application of technology within the learning environment is also continuously updated and refined with a strong emphasis on the different technology tools to enhance interdisciplinary learning in a blended environment (van den Berg & Verster, 2022). It is expected that the model will continue to evolve as more practitioners implement it within their teaching and learning environments. One of the purposes of DBR is to circulate information to the broader educational community to inform both theory and practice.

Conclusion

The digitalisation of society has brought about significant change, with digital innovation being a major driver of transformation. As such, higher education institutions must provide effective, innovative, and high-quality learning experiences that equip students with the necessary skills for a changing labour market. This paper aimed to define the skills requirements of IS students to become competent digital innovators and to develop design principles to teach digital innovation, ensuring industry-informed curriculum design that is future-proof within a digital economy. To achieve this, a four-phased design-based research approach was utilised, and the study was conducted in a South African university, involving students, industry practitioners, and researchers.

The study aimed to answer the research question of how digital business innovation skills should be taught to South African Information Systems students. The findings yielded nine design principles that ensure a future-oriented, industry-informed curriculum design that is relevant to the digital economy. These principles include collaboration, experimentation, the application of design thinking, interdisciplinary problem-solving, regular reflection, project-based learning and industry participation.

This study contributes to the literature by providing valuable insights into the process of articulating validated design principles. However, the study is not without limitations. It was conducted in a single university, limiting the generalisability of the findings, and the study’s design and approach may not apply to other contexts or disciplines. Future research should explore the effectiveness of the design principles in other educational settings and examine their transferability across different domains. Ultimately, the study's findings have implications for the development of future-oriented, industry-informed curricula, ensuring that students are equipped with the skills necessary to become competent digital innovators in a rapidly changing digital economy.