Introduction

Today’s world is complicated, layered, interconnected, unpredictable, and with much variation (Braithwaite et al. 2018). Research to address such complex issues can be too narrow-minded and reductionist, inhibiting more meaningful evaluation (Reynolds et al. 2012). To ensure that a research problem is thoroughly investigated, a research design – involving the overall strategy and analytical approach – should be carefully chosen (Vaus 2001). It should warrant the logical integration of the different components of the study.

By making use of studies in which both quantitative and qualitative methods are used, a more comprehensive basis for analysis can be achieved that may assist in answering research questions (González-Díaz and Bustamante-Cabrera 2021). Different mixed-method research designs are used – for example, exploratory sequential designs, explanatory sequential designs, and convergent designs (Harrison et al. 2020). Although mixed-methods research can contribute to a better understanding of complex problems, it generally provides insufficient guidance on how the researcher should decide which sub-studies to perform to gain an understanding of a complex problem. The focus of mixed methods is on data collection, analysis and interpretation. Even if excellent data are gathered and analysed, decisions based on limited perspectives may lead to serious shortcomings. For instance, if only academia is asked what to include in a curriculum while the industry’s perspective is ignored, the resulting teaching course may have very little practical value.

Critical systems thinking addresses the need for multiple perspectives, since one of its characteristics is that the guiding assumptions of multiple perspectives should be investigated to gain a holistic view of a complex problem (Ulrich 1994). Individual parts of the system should not be viewed in a siloed manner but the interactions among the interconnected parts should be studied (Checkland 1981).

In this paper, we explain how a specific strand of critical systems thinking – critical systems heuristics (CSH) – in conjunction with action research (AR), can form a flexible framework for designing research studies of complex problems. We demonstrate it by designing a research study to develop instructional guidelines for improving the employability of data analytics students. We show how it can guide a researcher to determine which perspectives are needed and what sub-studies should be embarked upon to attain the overall goal. We start in Section "Theoretical Background" by presenting our understanding of key concepts of critical systems thinking in support of the proposed strategy for developing a research design as proposed in Section "Strategy for Designing Research". Section "Demonstration of the Research Design Strategy" demonstrates the proposed strategy in the context of developing guidelines to improve the employability of data analytics students. Here, we present how the framework is used to design the research. Finally, a summary of the proposed method and its potential benefits is presented in Section "Conclusion".

Theoretical Background

To understand how a strategy is developed for designing a research project, we first consider the topic of complexity, whereafter we explore systems thinking, critical systems heuristics, the three worlds of Habermas and action research.

The Management of Complexity

Managers often face dynamic situations that involve complex systems of interacting components, rather than well-defined independent problems (Rosenhead and Mingers 2001). Rittell and Weber (1973) referred to these complex issues as “wicked” or “messy” problems. Although individual problems within a complex issue may be solved, their solutions cannot simply be added together when they are components of a messy problem because they may interact and have unintended consequences (Rosenhead and Mingers 2001).

It can be challenging for operational practitioners to navigate high levels of complexity, dynamic systems, numerous stakeholders, and diverse perspectives. To address these challenges, systems thinking, which is based on understanding complexity and simple rules, can be a valuable approach (Cabrera et al. 2018).

Systems Thinking

A system, as defined by Churchman (1968), is a collection of organised parts working towards achieving specific goals. To be considered a system, five key aspects must be taken into account: (i) the overall objective that measures the performance of the system; (ii) the individual components that contribute to the objective; (iii) the resources used to reach the objective; (iv) the environment or constraints that have an impact on the system but are beyond its control; and (v) the management responsible for overseeing the system’s successful operation (Churchman 1968). Ackoff and Warfield (1977) highlighted the importance of recognizing the interrelatedness of components and studying subsystems in context.

Systems can refer to both physical entities and conceptual constructs, as noted by Ulrich and Reynolds (2010). Critical systems thinking involves five key commitments: being critically aware, socially aware, developing theories from various strands of systems thinking, being committed to human liberation, and embracing methodological pluralism (Jackson 1991).

Critical awareness refers to the intentional and active analysis of a belief or body of knowledge, taking into consideration the evidence on which it is anchored and its potential implications (Dewey, 1910). When practising critical thinking, assumptions are thoroughly scrutinized, context is evaluated, and values that impact systems and their designs are carefully examined. Critical systems planners must be transparent about these processes (Jackson 1991).

Social awareness involves understanding diverse perspectives and cultural influences (Ajao et al. 2023). According to Jackson (1991), social awareness in systems thinking involves two key aspects: first, examining the organizational and cultural factors that influence the popularity of different systems theories and approaches for addressing specific issues; and second, considering the potential consequences of these choices. It is important for planners to be aware of the social judgement of those affected by their decisions and to be transparent about their decision-making process (Jackson 1991).

CSH is a method for practising critical systems thinking that provides criteria for discussions among stakeholders – both those involved in the operations of the system and those affected but not involved (those not having control over the system).

According to Jackson (1991), critical systems thinking is focused on promoting human liberation and assisting individuals in achieving their full potential. This is accomplished by addressing each person’s technical, practical, and emancipatory interests, as described by Habermas (1987). Jackson argues that all individuals have an interest in understanding how organizations and society function, and a systems perspective that can support these diverse interests will greatly benefit people’s well-being and freedom. Systems thinking requires holistic, pluralistic, and participatory approaches, as well as intentional improvement through operational and action research (Reynolds et al. 2012).

Methodological pluralism suggests that it is feasible to address the problem of incompatible paradigms at the level of human interests (Flood and Jackson 1991), provided that the researcher is sensitive to the epistemological assumptions and justifies the use of a particular method. It is important to keep the initial choice of methodology in mind and be open to modifying it as the problem evolves, as noted by Jackson (2003).

Whiteman (2015) proposed that to support methodological pluralism, Habermas’s universal pragmatics, a comprehensive theory of truth, can be utilized. Whiteman argued that the combination of quantitative and qualitative methods is acceptable if the researcher clearly articulates their methodological decisions and links them to three ontological categories, while providing justification for their choices. It is important for the researcher to recognize that these assertions may be challenged and, therefore, research should be iterative and self-reflective.

In Section "Critical Systems Heuristics", Ulrich’s work, which is considered a separate strand of critical systems thinking (Jackson 2003), is explored. Ulrich (1983) recommended extending Churchman’s work by focusing on critical discourse to address deceptions and unfairness to establish a rational thinking base for a system.

Critical Systems Heuristics

Ulrich developed an approach termed CSH based on Kant’s view of knowledge and reasoning. Kant (1998) posed three questions in the quest for knowledge.

Kant’s first question (“What can I know?”) concerns the focus of theoretical reasoning that characterizes the planner’s interest in mapping social reality and understanding the conditioned nature of the maps. Ulrich links it to critical systems thinking: “the systems idea stands for the ideal of comprehensiveness on the side of conditions, and because such comprehensiveness is only an ideal, it implies a lack of comprehensiveness of all our maps” (Ulrich 1983 p. 260). Due to this deficiency, we need to critically reflect on the entire system’s judgment.

Kant’s second question (“What ought I to do?”) is practical and, according to Ulrich (1983), concerns the planner’s interest in designing for the improvement of the human condition, and comprehending the moral imperfection of his designs, which necessitates the reflection on the moral implications of the underlying whole systems judgements.

Kant’s third question (“What may I hope?”) emphasizes that even with careful planning, there is no guarantee of improvement. Therefore, the planner needs to design for guarantee by, for instance, involving a broad range of experts and witnesses and building a basis for consensus between those involved and those affected. Once again, there is a need to reflect on the lack of guarantee (Ulrich 1983).

A key feature of CSH is to consider different viewpoints of different groups to understand their underlying meanings to improve the understanding of a phenomenon. Since people view phenomena from their perspectives that are based on their prior experiences, their views are conditioned (Kant 1998). The more conditioned viewpoints we study and try to understand and interpret, the better our understanding of the phenomena will become (Ulrich 1983). CSH therefore strives towards what Ulrich calls “the totality of conditioned realities”. This makes CSH a framework that is ideal for reflective practice and criticism (Ulrich and Reynolds 2010).

Ulrich (2005) identified and developed four distinct boundary issues that comprehensively address values, power structures, knowledge basis and moral basis. Ulrich (2005) expanded upon each of these categories with three questions, leading to 12 questions (see Table 1) that can each be asked in two ways: what is, and what ought to be, and are aimed at self-reflection to determine boundary judgements. The purpose of these questions is to help the planner to become aware of boundary assumptions. These assumptions should be made transparent so that other stakeholders could challenge them. These questions form the basis of CSH and are used to obtain insight into a system from as many perspectives as possible: understand what its purpose is (or should be), who or what controls the system (or should control it), whose expertise and what expertise is used (or should be used), and how legitimate the system is (or should it be) (Ulrich 2005). These questions clarify boundaries that limit our understanding but guide our thinking (Ulrich and Reynolds 2010). Being transparent about boundaries and the values underlying them is an important aspect of CSH. We should always remember that “there is no objective solution but only a critical solution to the problem of boundary judgements” (Ulrich 1983 p. 230).

Ulrich (1983) distinguished between two interest groups that play a role in a system: the involved and the affected. The involved are those who play a role in making decisions (decision-makers) and performing roles (planners, experts, and witnesses) to enable the system to attain its goal, whereas the affected are those who are influenced by the system but do not have control over it. The affected are at risk of being exploited or oppressed and that is why Ulrich added the section regarding legitimacy – to look after the affected, providing them with a “voice” to ensure that their concerns are heard and considered.

Within CSH, a witness is defined as someone who represents the affected, and voices their experiences, feelings, suffering and concerns on their behalf (Ulrich 1983). CSH promotes polemical argumentation where the witness may deliver polemical views (that may be controversial and not necessarily rational), on behalf of the affected, to expose shortcomings in the experts’ case (Ulrich 1983). This is important to ensure that the system’s planner obtains a comprehensive view of the system, and not only the views of the so-called experts. This enables the planner to critically reflect on possible sources of deception (Ulrich 1983).

A crucial principle of CSH is boundary critique, which entails transparently identifying the system’s planner’s boundaries and prescriptive claims. Ulrich (1983) describes two boundary judgements that need to be made: (i) determine what belongs to the system, and what belongs to its environment and (ii) determine who is involved in the system, and who is affected by it (but is not involved in planning the system). Boundary judgements portray which empirical observations are significant, and which may be viewed as trivial (Ulrich 2005). Ulrich and Reynolds (2010) mentioned two additional boundary judgements: (iii) contrasting the ideal mode (constituted by the “ought to” questions) with the more realistic mode (constituted by the “is” questions), and (iv) contrasting judgements among the four stakeholder groups (intended beneficiaries, decision-makers, experts, and/or witnesses).

The purpose of the 12 boundary questions presented in Table 1 is to make the planner of the system aware of boundary assumptions, and to invite the planner to reflect on them (Ulrich and Reynolds 2010). It is needed because everything we think or say about a situation is influenced by selectivity, which can lead to partiality towards certain parties. CSH guides us to identify this selectivity, reflect on its implications, and be open to critical discourse on it (Ulrich and Reynolds 2010).

Table 1 Ulrich’s 12 questions for CSH (adapted from Ulrich 1987 p. 279).
Full size table

Ulrich and Reynolds (2010) proposed that the planner could begin with an ideal map (answering the “ought to” questions) to establish initial normative assumptions that can be compared to actual maps (answers to “is”).

Figure 1 shows how each of the questions addresses a different segment of the CSH matrix. Ulrich and Reynolds (2010) explain the role of the rows and columns: the four rows represent the sources of selectivity – motivation, control, knowledge, and legitimacy – that are essential to understand what it is all about, what we aim to achieve and what are the built-in limitations, whereas each column in Fig. 1 represents the three boundary categories – stakeholder (role), stake (role-specific concern) and stake-holding issue (key problem).

Fig. 1
figure 1

(adapted from Ulrich and Reynolds 2010 p. 259)

Proposed sequence for unfolding the boundary questions of CSH

Full size image

These 12 boundary categories highlight the issues we should consider to make a system meaningful and to justify or contest the claims we associate with it (Ulrich and Reynolds 2010). The fourth row (legitimacy) wants to remind the system’s planner about the importance of providing the affected with a voice and to enable them to be emancipated. Incorporating Habermas’s three worlds into the CSH framework can help gain a deeper understanding of the affected individuals by studying their worlds.

Habermas’s Three Worlds

Research is done with the purpose to understand, explain and improve the world we live in. Three worlds are relevant to research methods, as described by Habermas (1987) – the material world, the social world, and the personal world. The physical or material world exists independently of humanity and does not require our presence to continue. While our actions can affect it, we are limited by its natural laws. The social world is a space where humans engage in specific social systems, including language, meaning, and social norms, all of which are heavily influenced by power dynamics. One’s personal world is comprised of emotions, thoughts, experiences, and beliefs, which are unique to each individual and not accessible to others.

By incorporating the three worlds of Habermas into CSH, a deeper understanding of the affected individuals may be achieved. This framework (CSH-3W), would assist the researcher in identifying sub-studies that need to be conducted to address the complex research problem. However, it is important to have a strategy to ensure that insights obtained from each sub-study are incorporated into the larger study and critically reflected upon. If necessary, modifications should be made to the proposed sub-studies. Action research is a useful strategy for achieving this.

Action Research

Action research (AR) is a methodology, first proposed by Kurt Lewin in 1948, that enables individuals to achieve transformative change by systematically examining and taking action on their own professional practice while engaging in critical reflection (Frost 2002). Blum (1955) claimed that AR typically consists of two phases: (1) the diagnostic phase, where the problem is analysed and hypotheses are developed, and (2) the therapeutic phase, where an experiment that intentionally directs change, is used to test hypotheses.

A detailed cyclical process for AR is proposed by Denscombe (1998), consisting of five elements as illustrated in Fig. 2. The first step involves observing the current professional practice. Next, critical reflection is conducted. Third, research is performed on a problem that has been identified during the reflection. The research findings are then used to develop an action plan. Finally, the action plan is implemented, resulting in a modified state of professional practice. This cycle can be repeated once or several times.

Fig. 2
figure 2

(adapted from Denscombe 1998 p. 60)

Action research model

Full size image

Susman and Evered (1978) proposed slightly different steps for their AR model, depicted in Fig. 3. It consists of the diagnosis phase (identifying a problem), developing an action plan by considering alternative options, taking action, evaluating the results and specifying learning that has taken place.

Fig. 3
figure 3

(adapted from Susman and Evered 1978)

Susman and Evered’s AR model

Full size image

In Section "Strategy for designing research", we will present a strategy for designing research studies by combining the main ideas of CSH, AR and the three worlds of Habermas.

Strategy for Designing Research

A systems approach that acknowledges the interdependence of a complex system’s components and respects the various perspectives of different parties involved or affected by the system, presents a useful strategy for social researchers studying complex systems. CSH enriched with Habermas’s three worlds, combined with AR, presents a strategy for researchers to design their work and its sub-studies to ensure that the research objective will be met.

According to Checkland and Holwell (1997), any research involves three elements: a framework of ideas, a methodology, and an area of concern. Their FMA model advocates applying a framework (F) of ideas that is embodied in a methodology (M), to an area of concern (A), which will lead to an increased understanding of all three of these elements. Since the identified area of concern is often not simple but complex, we begin our discussion of concepts by illustrating our understanding of complexity. In terms of this model, our research question forms the “A”, the area of concern. The framework (“F”) we propose is CSH enriched with the three worlds of Habermas, and the methodology (“M”) is AR. The proposed steps for designing research with this strategy are presented in Fig. 4.

Fig. 4
figure 4

The CSH-3W framework within the AR methodology

Full size image

For the overall research project, Denscombe’s AR model is used, enhanced with CSH and Habermas’s three worlds. For each sub-study, Susman and Evered’s AR model is used and the learning specified in each sub-study serves to inform the overall project.

  1. 1.

    Observe the professional practice as a system: identify the area of concern and undertake an initial description of the system that should be researched: What is its intended purpose? What are the different components of the system that influence one another? Whose perspectives may need to be considered? Is there a party that is affected by the system but cannot influence the system? Are there things that influence how the system operates and performs but are beyond the control of the system?

  2. 2.

    Critically reflect using CSH: perform an ideal mapping by using the CSH’s “ought to” form of the 12 questions to reach an initial understanding of matters such as the purpose, whose perspectives should be considered (those involved and the affected parties) and what the environmental factors are that have an influence on the system but cannot be controlled by it.

  3. 3.

    Identify issues of the affected’s three worlds of Habermas: in terms of the affected, identify issues belonging to their material world (people have limited influence over it), social world (the space where people interact) and personal world (unique to each person, often inaccessible by others) that may have an impact on the area of concern.

  4. 4.

    Plan research in terms of sub-studies sensitive to underlying assumptions: Explicitly reflect on the expected outcomes of (2) and (3), and propose research sub-studies that hope to enhance the understanding of the system and enable the researcher to obtain a realistic map with greater clarity for contrasting with the ideal map, as well as exploring ways to improve aspects of the system to move the realistic map closer to the ideal map. For each of the studies, reflect on the three questions of Kant: “What can I know?”, “What ought I to do?”, and “What can I hope for?”

  5. 5.

    Perform research: for each sub-study, Susman and Evered’s AR model is used, as depicted in Fig. 2.

    1. a)

      Diagnose: decide on a research goal (it may be on how to explore the “is” of a certain aspect, or on how to get from the “is” to the “ought to be” in a specific part of the system).

    2. b)

      Action plan:

      1. i)

        Decide on the research method by taking the epistemological and ontological assumptions into account: Ontology is concerned about reality, depending on different perspectives and how the group whose perspectives will be studied should be clarified. Epistemology is concerned about what counts as knowledge and how such claims are justified. Here, objectivism (positivism) versus subjectivism (interpretivism) are important considerations.

      2. ii)

        The researcher should decide if a quantitative, qualitative or mixed method will be the most appropriate to reach the goal of each sub-study and explain why.

    3. c)

      Action: Collect and analyse the data in accordance with the assumptions of the research method.

    4. d)

      Evaluate: Draw conclusions from the data analyses.

    5. e)

      Specify learning: State what has been learned from the sub-study and how it influences the overall study. Critically reflect on how the new knowledge influences the broadening of the boundaries of the system.

  6. 6.

    Update research plan: (depicted by dotted line between plan research and research studies in Fig. 4) Based on the insights obtained from a sub-study, revisit the research design and the remaining proposed sub-studies. Adapt the design where necessary.

  7. 7.

    Strategic plan: Cycle through steps 5 and 6 until sufficient insight has been obtained to propose a strategic plan to improve the system (to address the area of concern).

  8. 8.

    Action: Implement the strategic plan.

After one full cycle has been completed, one can repeat the cycle by observing the effect of the action(s) taken on the area of concern. Next, reflect on the findings and their implications. How much progress has been made in moving from the original realistic map to the ideal map? Is it sufficient? Are there other issues in the three worlds of the affected that need to be studied?

Decide if more studies are needed. If so, repeat from step 4.

In Table 2 we show how each of these steps of the proposed strategy (as depicted in Fig. 4), map to critical systems theory.

Table 2 Mapping of steps to theory
Full size table

The proposed CSH-3W framework within the AR methodology provides a practical strategy that may prove beneficial to use. It is based on the theoretical underpinnings of various proven strategies and logically combines their ideas. In systems, it is said that the total is more than the sum of the individual parts. We believe that in this case, this proposed strategy may also prove to be more than merely the sum of its parts. CSH warrants critical practice and awareness, 3 W ensures that the three worlds of the oppressed are considered and AR facilitates continuous improvement through iteratively working towards a better understanding of the problem.

Demonstration of the Research Design Strategy

In this section, the design of a research study using the above-mentioned steps will be demonstrated. In view of the huge amount of data that has become available in recent years, the need for data professionals who can analyse the data, build models and turn them into actionable insight has grown immensely. These professionals need not only technical skills regarding such as statistics, mathematics and programming, but also business skills and various soft skills. Universities should therefore design curricula for these careers to align with employers’ requirements (Ohei and Brink 2019). According to Ilori and Ajagunna (2020), not only the curricula should be adapted, but also universities’ teaching and learning strategies to ensure that graduates can meet the challenges of the workplace of the 4th industrial revolution. The authors therefore decided to research this problem using the steps outlined above. It entailed the planning of sub-studies to direct the research on developing guidelines for the curriculum of data analytics students. Figure 5 shows how they defined this problem in terms of Checkland and Holwell’s FMA model.

Fig. 5
figure 5

Application of Checkland and Holwell’s FMA model in this study

Full size image

The area of concern to be studied (A) was defined as “the development of guidelines to enhance the employability of data analytics students”, CSH-3W is the framework (F), and AR is the methodology to be used (M). In the following paragraphs, each of the steps shown in Table 2 will be applied to this problem.

Observe Professional Practice as a System

The first step aimed to identify the system to be researched, which was “Holistic education of data analytics students in the context of their life world to facilitate their employability”.

Employability is defined as “an individual’s portfolio of previously acquired knowledge, skills, attitudes, competencies, experiences, and other qualifications that underpin their ability to be a reliable source of efficiency, innovation, and productivity to an employer” (Smaldone et al. 2022).

The reader is reminded of the definition of a system, which consists of multiple subsystems that interact and influence one another and where multiple perspectives need to be considered to obtain a better understanding of the system.

The first author, as the planner of this research, reflected on some of the subsystems that play a role in this system that may affect students and identified the following:

  1. (1)

    The various modules that data analytics students study, each having its own nature, lecturer, topics and assessments;

  2. (2)

    Extracurricular activities presented by the university such as academic societies, sports and cultural activities present opportunities for students to gain skills needed to enhance their employability;

  3. (3)

    Student finances; and

  4. (4)

    Students’ supplementary instruction.

However, this only presents the views of the planner. Córdoba and Midgley (2006) explains the importance of involving a wide range of stakeholders to ensure the “sweeping in” of a variety of different perspectives.

An initial reflection highlights the following perspectives that should be considered: lecturers of different modules, faculty management, industry and students. Preliminary identification of the different perspectives that may be relevant, provides guidance to which parties should be considered as participants of possible sub-studies. It is important to note that as the study proceeds, it is possible to (through gaining insight from other perspectives) identify more stakeholders that should be “swept in” that were not identified initially.

Critical Reflection Using CSH

After the planner developed an initial systems view of the research problem, the second step aimed to perform a more detailed analysis of the system identified. This was achieved by developing an ideal map based on the understanding of the system by answering the “ought to” questions.

The demonstration’s initial ideal map is presented in Table 3. Here, it is an initial exploration where the researchers indicate what they expect the different parties will say. It is used to determine where there is insufficient information that should be addressed by the sub-studies.

Table 3 Initial ideal map using CSH “ought to be” questions
Full size table

Initially, the planner answered the questions only from their perspective. In discussion with the second author, the planner became aware that more perspectives should be included. As a starting point, it was decided to make initial statements from three perspectives: that of students, the industry, and faculty. We realize that this will neither provide a comprehensive view of these stakeholders’ perspectives nor include all the stakeholders whose perspectives should be incorporated. However, it creates a starting point from which studies can be planned to increase understanding. As new perspectives emerge or new insights are gained, the table should be updated.

The involved are those that have a role in the decision-making process. They include the planner and witness, as well as the client, the expert, the decision-maker, and the guarantor. In contrast to the involved is the affected, who are influenced by the system but do not have control over it. To decide who fulfils each of these roles, one has to look through the lenses of different interest groups. For this study, we may consider the lenses of industry, lecturers, and students. Other lenses may be included at a later stage as required.

Identify Issues Regarding the Three Worlds of the Affected

CSH10 addressed who the witness should be to represent the affected. This can be enriched further by turning to Habermas’s’ three worlds to guide the witness on the aspects of the affected that should be studied. Figure 6 shows some of the aspects that may concern a data analytics student in South Africa.

Fig. 6
figure 6

The three worlds of the affected (data analytics student)

Full size image

When CSH and Habermas’s’ three worlds are used to guide a research project, the researcher should try to determine different parties’ perspectives. It will not be possible to gain all perspectives and understand all there is to know, but obtaining as many perspectives as possible will lead to a better understanding of students’ issues and may guide us to broaden the boundaries.

In Fig. 7, the three worlds of Habermas’s’ and Ulrich’s boundary questions are combined to show visually the aspects the planner of the system should consider.

Fig. 7
figure 7

The three worlds of Habermas and the 12 CSH questions to consider

Full size image

The CSH-3W framework guides the researcher to ascertain the sub-studies that are needed to create a better understanding of the system and its boundaries.

The initial understanding of the planner implies a certain boundary (the primary boundary). As new insights are obtained after the completion of each sub-study, additional issues may emerge, thus creating a wider, secondary boundary. Midgley (1992) explains that these additional issues form part of the marginal area (not part of the primary boundary but part of the secondary boundary) and may create conflict among stakeholders since some will deem them important while others may dismiss it as non-issues. This will then need dialogue between the stakeholders where the boundary is critiqued to either resolve or stabilise the conflict.

Plan Research sub-studies

Based on the preliminary “ought to be” answers to the CSH questions, combined with the three worlds of Habermas, we established that the studies proposed in Table 4 should be performed to judge how best to improve the education of data scientists.

Table 4 Proposed studies as directed by CSH and Habermas’s’ three worlds
Full size table

In the spirit of CSH, one would like to engage in dialogue with different stakeholders to tap into their views and create an opportunity for debating emerging issues as Córdoba and Midgley (2006) did in the planning of an information systems project at a Columbian university. However, it is not always easy to motivate people to participate in such a process, as noted by Córdoba and Midgley (2006) as well as Stephenson and Lawson (2013). In an arena where participatory decision-making is pursued, it “seems ironic that those with ‘weaker” views are effectively closed out of the decision-making loop” (Stephenson and Lawson 2013 p. 32) (if they choose not to partake).

We therefore decided to follow a different approach to provide those who are usually too shy to present their opinions publicly and defend them, an opportunity to make their voice heard. In each sub-study, different stakeholders’ views and perceptions will be studied through the use of different methods such as qualitative survey studies, quantitative survey studies and systematic literature reviews. Online job listings and learning management system data of students will also be utilized to gain additional insights.

The researchers would then combine all the insights obtained from the sub-studies and present them to a forum consisting of representatives of the various stakeholders. In this forum, the boundaries can be critiqued and decisions be taken on what should be addressed and how they should be addressed. In this way, the critique of the boundaries will be a rational process since all the parties involved in and affected by the design of the educational guidelines will be represented in the dialogue.

A good understanding of the data analytics students is crucial since they were identified as being both the affected and a client of the system. Their three worlds need to be understood more clearly. Studies 1 to 3 are proposed that should enhance insight into students’ needs.

Study 4 is proposed to improve insight into the needs of the industry (the ultimate client). Study 5 should consider the influence of one of the environmental factors that the university cannot control: the mathematical foundation of the student based on the command of mathematics with which they matriculated. Lecturers who are both witnesses to the impact of the system on their students and of the resources of the system, need ways to measure relevant indicators of their students’ progress. Study 6 is therefore proposed to determine which statistics will be useful for inclusion on a lecturer’s dashboard. This may provide lecturers with useful actionable insights. Finally, study 7 proposes that the expertise of psycho-social professionals should be obtained on how to address students’ struggles such as stress management.

Figure 8 clarifies how each of these studies fits into the system.

Fig. 8
figure 8

Schematic representation of how CSH can be used as a useful tool to plan a PhD study

Full size image

By positioning the sub-studies on the graph one can see which study addresses which area. As soon as the planner is satisfied with the research plan, the research can be performed.

Perform Research

To demonstrate how the process works, we will show how AR worked in study 4 to determine the needs of the industry by analysing online job adverts for data analytics practitioners.Footnote 1

  • Diagnose:

    The research goal for this study was to assess the needs of the industry in terms of the technical, hard and soft skills required. One must always check that the objective of each sub-study should be aligned with the overall objective of the research program to ensure that it is correctly directed.

  • Action plan:

    • Ontological assumptions: In this study, the perspective of industry was researched, as observed through online job advertisements. It was assumed that these formal documents were prepared in support of the company’s objectives according to approved guidelines. The data were treated as an “objective” source of information.

    • Epistemological assumptions: a mixed method was used. First, a qualitative method was used to find broad themes and identify initial skills, which were combined with those identified in the literature. Text mining was subsequently used to create a structured data set, which was analysed quantitatively to discover the most sought-after skills.

  • Action:

    • Collect data: online job advertisements relating to data analytics jobs were collected from Glassdoor and LinkedIn.

    • Analyse data: Mixed methods were used to analyse the data.

  • Evaluate:

    Some of the conclusions reached were that the most desirable hard skills relate to artificial intelligence, statistics, mathematics, and programming, whereas mastery of SQL, Python, Excel, and Power BI was the focus of the most popular technical skills. The most sought-after soft skills included inter- and intrapersonal relations, business, analytical and communication abilities, as well as creativity.

  • Specify learning:

    Universities should ensure that their programmes are aligned with these needs of industry. They should communicate the importance of soft skills to students and provide guidance on how they can be developed. Useful insights were gained by analysing the job advertisements. However, the data so obtained are relatively shallow since a deep explanation of the industry’s needs is not provided by this means.

Similarly, each of the studies performed will follow these stages.

Update Research plan

These insights obtained from study 4 were recorded and incorporated into the guidelines for improving the employability of data analytics students. However, the authors realized that the information obtained from online adverts lacked richness and specificity. It was therefore decided to revisit the proposed sub-studies and add a further investigation (study 4b) where practitioners will be asked more in-depth questions relating to the skills required, as well as to the nature and extent of stressors they experience in the workplace.

It should therefore be noted that new themes and ideas may emerge from sub-studies. The researcher should think carefully about how to deal with this eventuality by, for instance, changing planned studies slightly, or devising additional sub-studies to explore any emerging themes more fully.

Strategic plan

After each sub-study, knowledge gained should be incorporated into the preliminary guideline document being constructed and the strategic plan adapted where necessary. The preliminary guideline document will be presented in a forum of representatives of the various stakeholders where the proposed guidelines will be debated. Here, critical dialogue will take place and the need for further broadening of boundaries can be discussed. This is part of the execution of the plan, which does not form part of the scope of this article. After the words of Kant, we can hope for productive discussions where the proposed guidelines will be critically reviewed and decisions will be made on which guidelines to include, exclude, adapt or add to the final instructional design.

Implement the Action plan

Once consensus is reached, the instructional design of the curriculum can be implemented. Here, we can hope for a guarantor that will ensure that it is implemented as agreed in the forum of stakeholders.

Repeating the Cycle

Once the whole cycle is completed, it can be used again to determine if further improvements are necessary or desirable. One may evaluate the success of the improved curriculum by measuring student throughput and performance and asking their feedback (their experiences with the improved curriculum, its delivery, and assessment methods), as well as feedback from lecturers (their experiences and challenges with the new curriculum) and industry (their assessment of new graduates who have followed the latest curriculum).

Based on the evaluation of success, more research studies can be planned to improve the curriculum even further and ensure that it keeps up with changing times.

This demonstration shows that using CSH-3W with AR enables a researcher to design research dynamically. It may prove, as we hope, to be especially useful for young researchers to plan their research careers, ensuring that all their studies are aligned to a specific purpose. Teaching the CSH-3W with AR to students, may equip them with practical problem-solving tools and critical thinking skills to address their future employer’s objectives.

Conclusion

Using CSH enriched with the three worlds of Habermas, combined with the AR method, was useful in planning this research project. This approach assists researchers in developing an initial design that carefully considers various aspects of the system. The AR method helps researchers cycle between zooming in to a sub-component of the system and then zooming out to consider how the sub-system affects the overall system. This iterative process ensures that all components are properly integrated. The proposal advanced here is dynamic and can change based on the results obtained from the sub-studies, keeping in mind that these still support the overall objective of the main study. It challenges the researcher to reflect critically on the boundaries of the problem and opens opportunities for the researcher to widen the boundaries as more insight is gained as the study progresses. The method provides the researcher with the option to use qualitative, quantitative or mixed methods in each sub-study, depending on the character of the data needed to answer the corresponding research question.

In this case, we propose what studies should be conducted to determine how universities can improve the education of data analytics students, by challenging the researcher to consider different perspectives that should be investigated to get a broader overview. This ensures that the voices of students and industry should be included when planning a course of instruction. Moreover, CSH used in the context of research design enables researchers to devise emancipatory research with impact.

In summary, the steps of the proposed method are:

  1. 1)

    Observe Professional Practice as a System

  2. 2)

    Critically reflect on the system using CSH to create an ideal map. As more insight is obtained from step 5, revisit and update the table. Critically reflect on the lack of comprehensiveness of the map, the moral implications of the boundaries and the possible lack of guarantee.

  3. 3)

    Identify Issues of the Affected’s Three Worlds of Habermas

  4. 4)

    Plan research in terms of sub-studies sensitive to the underlying assumptions. Answer Kant’s three questions: “What can I know?”, “What ought I to do?”, and “What can I hope for?”

  5. 5)

    Perform Research (each sub-study), as Follows

    1. a.

      Diagnose: devise a research goal.

    2. b.

      Create an action plan.

    3. c.

      Execute the action plan.

    4. d.

      Evaluate the analyses and results.

    5. e.

      Specify learning by noting the insights gained.

  6. 6)

    Update the Research plan

  7. 7)

    Propose a Strategic plan

  8. 8)

    Implement the Strategic plan

This study’s proposed methodology was demonstrated to design a research study to gain insight into how to improve the education of data scientists. However, it can be used to guide the study of any academic research project. Donaires (2006) used CSH to establish a critical software development process, Venter and Goede (2018) showed CSH’s usefulness in improving the business requirements of an intelligence system, and Luckett (2006) claimed that the use of CSH in a policy development process contributed to a substantial improvement in the resulting policies.

CSH is a reflective practice that can assist in the identification of relevant questions and their exploration within a certain problem context. According to Venter and Goede (2018), it is therefore useful for ascertaining the scope of the problem.

Combining CSH with AR, as proposed in this study, places a valuable tool in the hands of someone designing a dynamic research project.

The authors plan to test the merits of the methods proposed in the article by applying them in a PhD study to develop guidelines for the design of a data analytics course to improve the students’ employability.