1 Introduction

The Industrial IoT (IIoT) and the Internet of Things (IoT) as a whole constitute a new frontier of industrialization, necessitating their separation. In the first scenario, which is a notion centered on the needs of the customer, the main focus is on people, homes, consumer electronics, automobiles, computers, and many more commonplace items. The IIoT, on the other hand, opens doors for businesses, factories, or complete sensor networks.

While both IoT and IIoT involve the interconnection of devices and the exchange of data, they differ in their scope and application. IoT is primarily consumer-centered, with a focus on smart consumer electronic devices that are interconnected to improve human awareness of the surrounding environment. In contrast, IIoT covers the domains of machine-to-machine (M2M) and industrial communication technologies with automation applications, enabling efficient and sustainable production (Sisinni et al., 2018). IIoT emphasizes the integration and interconnection of once isolated plants, working islands, or even machines, offering more efficient production and new services. Moreover, IIoT is characterized by huge amounts of data due to the nature of the environments it operates in (Jaidka et al., 2018). While most general requirements of the IoT and IIoT are similar, many requirements pertaining to timing, security, privacy, and reliability are specific to each domain and can be very different between the two. An example of this is the high reliability required for an IoT system collecting data about autonomous vehicles operating in a city, in contrast to an IoT system monitoring plants in a garden.

The IIoT idea should aid in maximizing operational efficiency and industrial output, spurring more growth, and enhancing the competitiveness of global businesses. The vast amount of data that can be gathered from the network of interconnected machines and sensors allows for the improvement of the entire production chain, detailed analysis of the system's flaws, and the use of the most effective technologies to keep businesses competitive.

More than 3.5 million jobs in manufacturing are anticipated to become available over the next ten years, yet fewer than half of these positions will be filled by qualified candidates (Turcu, 2018). Universities will be critical in preparing the next generation of workers since a highly skilled human resource will be more crucial than ever. Today's and tomorrow's students must possess the skills and information necessary to survive in a world that is highly technologically advanced and interconnected (Hernandez-de-Menendez et al., 2020). Competencies that need to be developed in I4.0 students include interdisciplinary thinking, decision making, problem-solving, cultural and intercultural skills, and commitment to lifelong learning (Coskun et al., 2019). Advanced analytics is a key hard skill that engineers must possess. Additionally, fundamental knowledge, abilities, and capabilities in creativity, analysis, and problem solving will be crucial. They will be able to use these to assist in real-time decision making, optimize production processes, and evaluate massive data sets from numerous sources (Sackey & Bester, 2016).

The goal of this work is to describe the layout of an Internet of Things (IoT) lab that enables students to (i) gain a basic understanding of creating an IoT platform, (ii) practice setting up a lightweight publish-subscribe messaging protocol, (iii) grasp the fundamentals of data analysis and data visualization to monitor industrial assets by processing data from sensors. The design of the entire educational activity using the definition of a Cyber-Physical System (CPS) in accordance with the formal definition (Song et al., 2016) that takes into account teachers, classrooms, and all other resources involved in the process is another objective of this work. With respect to this goal, this paper tries to address two open points identified in the literature introduced in the next chapter: the detailed description of how to structure an IoT course for science students, and the design of an evaluation methodology based on Bloom’s taxonomy to assess the didactical effectiveness of the course, considering a hybrid learning experience between remote and in-person.

This article is structured as follows: Sect. 2 presents the state of the art of didactical activities in this sector by analyzing the gaps. Section 3 introduces the evaluation methodology, describes all the technologies used to build the laboratory activity, and highlights how they are used together to build the didactical IoT system. The case study performed at an Italian higher education technical institute is presented in Sect. 4. Finally, Sect. 5 draws conclusions, including results in terms of students’ satisfaction and future work propositions.

2 State of the art

The ongoing revolution of Industry 4.0 has brought about a significant transformation in the manufacturing industry with the integration of cyber-physical systems and intelligent technologies such as simulation, augmented reality, robotics, and big data analytics. As a result, experts agree that teaching and learning related concepts, tools, and technologies are of vital importance for preparing the next generation of workforce with the required skills and competencies. Therefore, this literature review aims to explore the current state of research on teaching and learning Industry 4.0 technologies, concepts, and tools with a focus on the use of Bloom's taxonomy. Studies have shown that Industry 4.0 represents a significant transformation in practically every industry, and the smart concept emerges in autonomous decisions and cyber-physical systems based on production systems.

A first work regarding this sector aims to identify the key technologies of Industry 4.0 that have been ill-defined by previous researchers and to enumerate the skills required by Industry 4.0 (Bongomin et al., 2020). Another important work (Vilalta-Perdomo et al., 2022), with Vilalta-Perdomo as the first author, explores the integration of Industry 4.0 concepts into higher education, specifically through the application of challenge-based learning in the field of operations management. The authors found that this approach enhances students' understanding of Industry 4.0 technologies and their potential applications, promotes interdisciplinary collaboration, and encourages active engagement, which are essential skills in Industry 4.0-driven organizations. The work of Almalki et al. (Almalki & Durugbo, 2023) identifies and prioritizes the critical institutional enablers and barriers of industry 4.0 from the perspective of industry, academic, and government experts. The study found that mindsets that are opposed to reforming education to embrace Industry 4.0 are the top institutional enabler and barrier, respectively.

Several researchers have worked in identifying the necessary required skills for workers, managers, and engineers in various industrial sectors in the industry 4.0 context (Mishrif et al., 2023; Montesdeoca & Rivera, 2023; Kabasakal et al., 2022; Romero-Gazquez et al., 2021; Caratozzolo et al., 2022; Labanda-Jaramillo et al., 2022; Cazeri et al., 2022; Perini et al., 2022; Lupi et al., 2022). From the perspective of practical applications, the literature search also found several examples of laboratory activities carried out in order to develop specific industry 4.0 skills (Georgieva et al., 2022; Marcon et al., 2022; Mehrtash, 2023; Pajpach et al., 2022; Pereira et al., 2022; Ruppert et al., 2023).

The application of Bloom’s taxonomy for the classification of educational learning objectives and as an assessment tool for learning outcomes is highly prevalent in pedagogical literature; however, the usage of the taxonomy in the Industry 4.0 context is very scarce. The available literature mainly indicates the usage of Bloom’s taxonomy as an assessment framework for learning outcomes for specific I4.0 technologies. The research conducted by Churches (Churches, 2010) was the first work that started the discussion about using the taxonomy for online technologies, with the revised version named “Bloom’s Digital Taxonomy”. It provided practical recommendations for incorporating digital tools and online technologies into the learning process and associated rubrics for assessing student performance. The findings concluded that by employing this taxonomy, educators can enhance their instructional strategies and empower students to develop essential digital skills while achieving meaningful learning outcomes.

Other authors developed an IoT framework using pedagogy as defined in Bloom's taxonomy (Srivastava et al., 2019). Another inherent work proposed a gamification based cyber security curriculum where students are evaluated based on the various taxonomy levels (Debello et al., 2022). Furthermore, it has also been used in multidisciplinary courses to classify the different learning outcomes required by students belonging to different majors (Muci-Küchler et al., 2022). Blooms taxonomy has also been used in combination with other learning approaches and frameworks. In the work of Maffei et al. (2022), the authors use the constructive alignment method for learning as a replacement for the traditional transmission approaches. Moreover, the Technology-Enhanced Formative Assessment approach (TEFA) combined with Bloom’s taxonomy has been used to improve the quality of learning for machinery workers (Riadi et al., 2019). In the work of Litwin et al. (Litwin & Stadnicka, 2019), the authors propose a learning model for teaching modelling and simulation, consistent with the Conali ontology and Bloom’s taxonomy.

Finally, two review works in this field are interesting to highlight. The first (Cazeri et al., 2022) is a review work by Cazeri et al., where they identified and synthesized the main features explored by current research in training for Industry 4.0. Their findings indicated gaps in the training of managers who deal with I4.0. Moreover, there is a gap in ad-dressing content about the impact of I4.0 in business models and the relationship of Industry 4.0 learning methods with sustainability. The second review (Moraes et al., 2023) is published in the same year, 2022, with the scope of identifying the uses of Industry 4.0 technologies in education and how they contribute to learning, in addition to high lighting at what educational level they are used. Based on the results of the topic analysis performed by the authors, the most used 4.0 technologies in education are: AR (9 documents), Simulation (5), IoT (4), VR (3), Big Data (3), Cloud Computing (3), CPS (3) and Mobile (3). Artificial Intelligence, Machine Learning and Autonomous Robots present one document each. Other results of the same work are that the 4.0 technologies support the entire learning process, but they are not used as much as they should, and they are still largely restricted to universities and courses related to manufacturing.

Considering the open points reported by these two works, many case studies consider higher education courses on manufacturing, but none of them provides a holistic view of the complex 4.0 factory. This work does not include an innovative case study, but it tests Bloom’s taxonomy applied to the IoT concept. Furthermore, the work provides a description of the classroom based on the CPS concept in order to include this concept in the learning objectives. This work is regarding IoT, Cloud Computing and CPS, which are considered the least diffused 4.0 technologies found in scientific literature. Finally, also if the case study is regarding a manufacturing high education classroom, the audience of the presented laboratory is not limited to such classrooms only, but it is also suitable to all audiences of every age thanks to visual programming tools, like the drag-and-drop tool Node-RED used in this work. The sustainability gaps highlighted in literature are not addressed in this work, however a small part in the future works is dedicated to this.

3 Design of the course

The objective of the proposed laboratory activity is to teach students the concepts and methodologies needed to design and develop an IoT system with references to the manufacturing industry. The classroom and the course tools can be viewed as a single CPS (a detailed description is provided below) to give students a narrative of the course that attempts to follow the technological concepts exposed in the course itself. In fact, the group of agents and processes involved in the class—as well as the technologies employed—represent a CPS that is run through an IoT platform.

3.1 Educational objectives

As a framework for classifying statements of what we expect or intend students to learn as a result of instruction, a taxonomy of educational objectives is proposed. For this work, a revision of Bloom’s taxonomy proposed by (Krathwohl, 2002) is used. Any objective is represented in two dimensions shown in Table 1, called the taxonomy table, and representing two learning process dimensions: the Knowledge and the Cognitive ones.

Table 1 Taxonomy table about learning objectives of the IoT lab: the knowledge dimension is the horizontal axis of the table and cognitive process dimension is the vertical one
Full size table

The knowledge dimension consists of the following subcategories: (A) Factual Knowledge referring to basic elements, (B) Conceptual Knowledge referring to interrelationships among such elements, (C) Procedural Knowledge refers to how to do something, and (D) Metacognitive Knowledge that refers to self-awareness and independence in decision making (cognitive) processes, i.e., this knowledge helps learners to monitor their own progress and take charge of their learning experience as they read, write, and solve problems in the classroom.

The cognitive process dimension is made up of the following subcategories: (1) Remember or long-term memory, (2) Understand the meaning of instructional information, (3) Apply and use adequate procedures for each scenario, (4) Analyze and detect relationships between parts, (5) Evaluate based on criteria and standards, and, finally, (6) Create new solutions according to requirements. Therefore, Table 1 is a formalization of the aims of the proposed course according to Bloom’s structure, and it provides a description of the learning objectives for every element of the table.

Examples of HW components are sensors and Raspberry Pi; examples of software tools are DB systems or graphical programming tools like Node-RED; examples of analysis typologies are simulation and modelling techniques like machine learning; an example of communication technologies is the MQTT protocol.

In order to have a quantitative evaluation, variables must be defined for each cell of Bloom’s taxonomy table. Furthermore, for each single learner or group of learners, these variables are aggregated to provide an indicator of learning outcomes achieved.

3.2 The cyber-physical system of the course

The course is modeled as a CPS (for many aspects, it can be assumed to be a Digital Twin of it) (Colombo et al., 2014) in which students, professors, educators, technicians, laptops, servers, and all other human and machine resources of the course dynamically interact with each other to achieve both local and global goals. Following standards provided by ISA-95, Fig. 1 shows the organization of the CPS referring to the course.

Fig. 1
figure 1

CPS of the course based on the ISA-95 standard

Full size image

Referred to the professor is the level of planning, that is, the level of legal responsibility or delegated by an organization such as a university or academy, and thus the level with the highest degree of complexity, needing to interact tactically or strategically and, therefore, for a course of training, with frequencies on the order of magnitude of the week or month. The execution level is given by the operative standards and monitoring and control systems applied by the human organization that manages the course: a school, a university, an academy, or any other type of organization that manages the entire student lifecycle. The functional data flow model is given by mails, chats, in person or remote lectures (a datum is everything that can generate useful information regarding the CPS under analysis, i.e., the course), any material used during the lectures or provided for activities other than lectures, any material or repository used for student evaluations (exams paper sheets, digital tests, surveys, oral examinations, etc.), and, finally, any information resources provided by the overall organization (for example, a digital platform used by the university to manage students, professors and researchers). The class control level refers to lectures, including theory, exercise, evaluation, projects, and all the possible activities involved during the lectures, as well as the implied resources like physical components to be monitored, sensors, actuators, and edge-control tools.

Regarding this CPS, a required aspect is safety. No deep treatment of this aspect is analyzed in this work, but, during the course design, inadequate protection from electrical components and no proper preparation of human resources can cause accidents or problems. For this reason, all devices in the system have been selected to be low-power and low-voltage (5 V), and, furthermore, plastic elements are used in order to minimize the possibility of electrical faults.

An interesting course design aim is to create IT components of the CPS that are easy to install and configure, in order to make students available to do it on their own, and also to develop their procedural and metacognitive knowledge and the cognitive abilities of applying and understanding. New generation hardware solutions, like the Raspberry Pi, and drag-and-drop programming environments, like Node-RED, can be considered for their simplicity and in order to allow the student to focus on all the different knowledge levels. Another inherent aspect is the component’s availability. To leave students the freedom to develop their own personal physical systems to monitor and control, the components must be breadboard compatible and easy to get, i.e., commonly used and found in general electronics markets.

Finally, energy consumption requirements should be considered during the design of the course in order to build a system that does not significantly affect the electricity bill of the house, for general reasons and always considering that students can develop their personal physical system.

3.3 Bloom’s taxonomy evaluation of the course

In order to obtain a map of class progress that is based on Bloom's taxonomy, i.e., a quantitative mapping of how the course is progressing in terms of knowledge and cognitive skills transfer, a tabular form is used, where the columns refer to the different knowledges ordered from left to right, while the rows represent the cognitive dimension of learning, starting from remember, first row, to create, last row. In order to implement the table, it is necessary to define a few scores, marks, or evaluation metrics., \({x}_{is}\) to measure for each student: the approach is the same as building an exam divided into different exercises with relative maximum points, and \({x}_{is}\) is the \(i\)-th score referring to the \(s\)-th student. Assignments, homework, exams, deliverables, or generic tests to calculate scores can be submitted at the end, during, or at the beginning of the course in order to assess different aspects of learning or aspects of improvement and define, for example, scores based on the improvement achieved over the initial tests. Once all the necessary information has been gathered, the aggregate score \({x}_{im}\) can be calculated using a \(m\)-th statistics such as the mean, median, or standard deviation of individual student scores, i.e., \({x}_{im}={f}_{m}({x}_{is})\), in order to obtain a metric describing the performance of the whole class with respect to the course under study.

Each \({x}_{ik}\) indicator is assigned to one or more Bloom’s point (table cell) and each one Bloom’s point is linked or not to one or more indicators, where \({I}_{CK}\) is the set of marks \({x}_{im}\) assigned to the Bloom’s cell in the \(C\)-th cognitive row and in the \(K\)-th knowledge column: there are cases in which few Bloom’s cells are blank because of a choice of metrics focused only on certain aspects of knowledge and cognitive processes. Finally, different functions can be considered in order to obtain a single score for each Bloom's cell in cases where more than one score is associated with it: \({x}_{CK,nm}={f}_{nm}({x}_{im}\in {I}_{CK})\), where \({f}_{nm}\) is the \(n\)-th function used to summarize the set given by each \({x}_{im}\) score assigned to the same Bloom’s cell and that represents an aggregate class score calculated with the same \(m\)-th statistical function.

4 Case study

The fourth edition of the IoT lab provided by the authors of the paper in 2023 is considered a case study and refers to a class of about 89 students attending an Italian degree program of three years with the aim of providing a shorter educational path to introduce high school students to the manufacturing sector.

4.1 Introduction and objectives

The course presented as a case study was taught in 2022 for a class of bachelor’s degree students at an Italian higher education engineering institute, and the main arguments concern methods and technologies required by middle managers. The course consists of a laboratory (lab) with the aim of being an experience of seven hours and thirty minutes (7:30 h of experience composed of five lectures of 1:30 h) to introduce the use of IoT technologies in synergy with other 4.0 ones.

Our primary goal is to deal with the concepts of designing and developing an IoT platform for managing industrial resources, so in actuality an IIoT platform, and to do this by integrating theoretical and practical explanations according to the lecture program with ongoing explanation and analysis of the class's interaction with the IoT system designed for the course, which definitely does not fit manufacturing requirements but it nevertheless maintains strong parallels with IIoT systems. For this, the course itself is designed, explained, and executed using a CPS model consisting of an IoT system that can be looked at as an IIoT system because of the physical entities involved in such a system, i.e., teachers, students, and the system under-analysis (that represents a general production system) are highly integrated with the IoT system, like in an industrial scenario.

The assumption of an IIoT environment (from a didactical and scientific point of view) is based on the characteristics, typical of an IIoT system when a single information package (from a 4.0 milling machine for example) is a Big Data system itself, i.e., that each of one single entity exchanges with this IIoT system (specific and knowledge-oriented one) high-complex, precious and necessary data in order to guarantee the information to be process for achieving the targets of the wholes CPS (a good training course) and, consequently, the target of the IIoT system design to generate quantitative knowledge structured according the Bloom’s taxonomy to design, execute and evaluate a training path. The required learning outcomes of the students in this course are fundamental concepts for both IoT and IIoT. Therefore, in the context of this work, the authors use the terms synonymously and don’t distinguish between them.

4.2 Educational plan

Two teachers (in two seperate geographical locations) are involved in the course: an associate professor who is in charge of the introduction and an explanation of the industrial context, and a PhD student (designated as “the instructor”) who has had years of teaching experience and helps develop the parts of the system that are suggested to the students. In order to avoid the teacher wasting time and lowering the course's intensity, another tutor must be present while the practical tasks are being carried out. This tutor assists in monitoring the individual components of the class and ensures that the entire system functions. The essential elements of the course's IT infrastructure are the server hosting the VPN, the students' and instructors' laptops, which double as mini servers, and Node-RED, a program for building dashboards. This simplicity ensures easy system replication.

Table 2 describes in detail the course program. There are five lectures of one hour and half, where the first introduces the 4.0 context and the tools of the activity, the second introduces the CPS of the laboratory, the third includes concepts of connectivity and security, the fourth lecture is about data visualization and dashboarding, and, finally, the last presents the valuation methods and includes a clear description of the required deliverables for students.

Table 2 Program for the IoT lab divided into lessons
Full size table

For the evaluation of each student and the overall classroom, three deliverables are asked: (i) the first one assigns a maximum of two points to the students, and it consists of open written answers about theoretical and general concepts introduced during the lectures; (ii) the second delivery, assigning a maximum of 3 points, is regarding the practical activities, and it consists of delivering the Node-RED flows, i.e., delivering a software solution able to manage the required data flow, and, finally, (iii) the last deliverable is a facultative survey with the aim of providing the student's level of satisfaction with his or her personal experience. This last deliverable provides one point to the students in order to promote its compilation.

4.3 Description of the class CPS

IoT is not a specific technology; rather, we are discussing technologies that support an IoT lab. Table 3 lists the necessary technologies and categorizes them into three groups: devices, networks, and applications (app).

Table 3 Required technologies for the IoT lab are devices, network tools, and apps
Full size table

As mentioned in the previous sections, the IoT concepts introduced in the course and the classroom setup (including all agents involved in delivering the didactic experience, and the students who are the receivers of the experience) are viewed as a single CPS to give students a narrative of the course that attempts to follow the technological concepts exposed in the course itself. This is shown in the system architecture in Fig. 2, where the left side of the figure highlights the IoT system developed during the course and the right side refers to the classroom system. Both subsystems of the CPS follow the ISA95 layered model for industrial informatics.

Fig. 2
figure 2

System Architecture of overall classroom cyber physical system where: the left side refers to the IoT system developed during the course and the right side refers to the class CPS including all agents participating in the teaching and learning experience

Full size image

Starting from the lowest level (field layer) of the developed IoT system is the physical process to be monitored; this process is left to the discretion of the students, and it can be as simple as monitoring the climate and environment of a room in their house or a plant for example. The process is monitored by a set of sensors that are in layer 1 of the model, designated as the device layer. Components in this layer are publishers and consumers of data, and they represent the first interface between the physical and cyber realms (hence the term cyber-physical). In the usual industry 4.0 scenario, these components are usually sensors and effectors.

The data created by the sensors is aggregated in the controller layer, which in our case is the Raspberry Pi that is running the Node Red software. The software is used to process data coming from the sensors and to provide the communication functions necessary for IoT communication. Moreover, it provides the tools necessary for dashboard creation and data visualization. The dashboard created by Node-Red sits in the monitoring and supervision layer, and in accordance with the ISA95 hierarchy applied to industry 4.0 systems; this layer usually contains HMI’s (Human Machine Interface) for monitoring the production line.

The Raspberry Pi sends the data to the IoT platform in the Operations management layer via the MQTT protocol. MQTT was chosen due to its simplicity of implementation and its widespread adoption in industrial settings. The IoT platform contains a database to store the data, a VPN server to allow secure communication within the classroom eco-system, and an MQTT broker to manage communications between the various elements of the IoT system. The final Planning layer usually contains ERP systems (Enterprise resource planning) and deals with the business aspects of an industrial system, such as orders, vendors, scheduling, and several business analytics tools. These aspects are out of the scope of the course curriculum; however, we emulate this concept for the students in the form of the professor of the course, who is responsible for the high-level strategic and tactical decisions of the classroom.

The same ISA95 hierarchy is applied to the classroom agents of the CPS (shown on the right side of Fig. 2). The lowest level involves the students attending the course (remotely or in-person). Using the same analogy mentioned above, they are consumers and publishers of data (for example, a student can turn on a light and trigger the light sensor, or move near an ultrasonic sensor and change the distance read by the sensor). The device layer corresponds to the individual control level, which contains the student laptops or the computers in the laboratory in the case of students attending in-person. Each of these components contains a node-red instance to allow the students to receive, process, and visualize the data. The communication between this layer and the other parts of the IoT eco-system is established via the MQTT protocol, with the broker acting as the central hub.

From a conceptual point of view, the following two levels (Class control and Supervision) are filled by the instructor of the course, who also has a node red instance running on his computer in order to be able to replicate and verify the work done by the students. Another reason why the instructor is at this level is because of the role he plays in delivering the course content (the class level) and his role in supervising the systems developed by the students (the supervision level).

The execution level provides the operative standards and manages the student life cycle during the course (just as the operation management level manages the lifecycle of the processes and products involved in an industrial system). In this case, it is represented by the university information system, which is managed by an administrative technical support employee working at the institute. This allows the students to access various university resources, such as online lectures and remote servers (the university server hosting the IoT platform). Finally, there is the planning level that, as mentioned before, contains the course professor.

It is noteworthy to mention that the developed system allows for increased flexibility in terms of remote and in-person learning, in the sense that the architecture does not change in accordance with the geographical distribution of the agents involved in the system. For example, some students attend the course in-person and access data in the Raspberry Pi and sensor setup present in the lab, others can attend remotely and access the data in the same setup. Another scenario is that some students create their own setup at home and access the data in the same manner, or they can access sensor data provided by other student setups in their respective homes. This way, a sort of immersive learning is achieved where the hardware systems developed by the students represent different IoT nodes in the ecosystem. Furthermore, the more students join the course, the more the size and complexity of the IoT system increases, bringing the learning experience closer to the real-world scenario of an IoT deployment that contains hundreds of nodes communicating autonomously and collaborating to achieve higher-level goals.

4.4 Quantitative Bloom’s taxonomy

The laboratory assignment is to determine the mark of each student for a maximum of six points. The indicators used in the taxonomy table are unknown for the students, they are used only by the teachers to provide an overall evaluation of the course, and called \({x}_{ij}\) and described as follows: \({x}_{ij}\) is the \(j\)-th indicator referring to the \(i\)-th deliverable submitted by each student, where \({x}_{1j}\) is the \(j\)-th indicator provided by the first deliverable, \({x}_{2j}\) is the \(j\)-th indicator assigned through the second deliverable, and \({x}_{3j}\) is the \(j\)-th indicator associated with the third deliverable. Table 4 shows the aggregated results of all students in order to quantitatively describe the overall class achievements according to Bloom’s Taxonomy. In the table, to each Bloom’s point are associated three different types of information: (i) the list of \({x}_{ij}\) indicators considered, (ii) the average value of considered indicators, where each one is the average between students, and (iii) the average value of considered indicators, which is the standard deviation between students.

Table 4 Indicators used in each Bloom’s taxonomy point
Full size table

The first deliverable consists of the following open questions, where marks are assigned as expressed in brackets:

  1. 1.

    explain the concept of IoT and IIoT (\({x}_{11}\)) with a contextualization in Industry 4.0 (\({x}_{12}\));

  2. 2.

    describe the HW components you used in the IoT Lab (\({x}_{13}\));

  3. 3.

    describe the SW infrastructure, explaining in depth the role of the broker (\({x}_{14}\)), and other tools (\({x}_{15}\)) you used in the IoT Lab;

  4. 4.

    describe the process of reading data from sensors (\({x}_{16}\));

  5. 5.

    describe how to visualize such data for a human operator (\({x}_{17}\)).

The second deliverable consists of setting up an IoT system based on the MQTT protocol (\({x}_{21}\)), using a MQTT broker set by the teacher with a defined server address and port. The server is available from any network, like home Wi-Fi or a mobile hotspot. To demonstrate the results, students must present screenshots of their flows, the configurations of the nodes in the flow, and, finally, their dashboard. With the dashboard, supervisors can prove the proper functioning of Node-RED flows regarding sensors of temperature, humidity, and light (\({x}_{22}\)). Finally, if the dashboard is working and it shows the correct chart types for each sensor, the supervisors can prove the correct UI/UX development by the student (\({x}_{23}\)).

Finally, the last deliverable, that is optional, consists of a survey that helps course designers assess the overall learning experience. More than this, the survey includes a few questions to assess the knowledge acquired by the student, especially conceptual and metacognitive ones, as shown in the following.

  1. 1.

    Give a list of the costs of hardware and software resources needed to develop an IoT system, including a dashboard, to monitor the temperature of the house (\({x}_{31}\)).

  2. 2.

    To develop the system in the first question, is it enough to use what you learnt from the lab, or do you need a quick tutorial about new components? If yes, which one? (\({x}_{32}\))

  3. 3.

    Estimate the time that you need, as a junior software developer, to develop a dashboard with Node-RED for such a system. (\({x}_{33}\))

  4. 4.

    Can you describe a few interesting improvements to such a system? (\({x}_{34}\))

4.5 Analysis of the results

For the final class assessment, only 20 students participate in the analysis; where only 13 submitted all three deliverables, while 7 submitted only the third one. Figure 3 shows the result, referring to this sample representing the whole classroom. As expected, points “C1”, “D1”, “D2”, and “D6”, referring to proposal and metacognitive knowledge, are not considered. The top part shows the distribution of average values between indicators in each cell calculated as averages between students, while the bottom part refers to average values of indicators calculated as standard deviations of the students’ marks.

Fig. 3
figure 3

Class results resumed according to Bloom’s taxonomy

Full size image

The top part shows the distribution of average values between indicators in the cell calculated as averages between students, while the bottom part refers to average values of indicators calculated as standard deviations of the students’ marks.

Referring to average values, there are higher values on the right part, i.e., about metacognitive and procedural knowledge. This is in accordance with the structure of the laboratory: only a few hours are dedicated to transferring factual and conceptual knowledge, as shown by the total average value linked to these two dimensions. The main goal of the laboratory activity is to give students procedural knowledge (practical capabilities) and metacognitive knowledge regarding Industry 4.0 technologies. Moreover, in addition to application knowledge, the results show a very good overall average in terms of students' analytical skills.

Regarding result variability, the variability related to the ability to apply is relatively high, which denotes the need to structure the course in a way that standardizes in a more efficient manner each student's educational cognitive path. In contrast, good relative values of average variability were found in the class's acquisition of conceptual and metacognitive knowledge.

Summarizing, the average values of the class metrics indicate that, from the perspective of both the knowledge and cognitive process dimensions, a just sufficient outcome of the didactic course can be assumed, with a grade of six over ten (average of the different dimensions of knowledge and cognitive processes). This is not an excellent outcome from the didactic point of view, but nevertheless, according to the judgment of the professor and other training experts, it represents a correct view of the cyber-physical system managed by the IIoT infrastructure described with a positive reading of medium–low variability values for what concerns both dimensions of Bloom's taxonomy, demonstrating that a sufficient service was delivered homogeneously to the whole class with enough knowledge that has been transferred to students regarding all the Bloom’s typologies of cognitive processes.

5 Limitations, conclusions, and future improvements

5.1 General and specific limitations

The implementation of Bloom's Taxonomy in various educational settings presents several pedagogical challenges. Boles et al. and Ramirez (Boles et al., 2005; Ramirez, 2016) both emphasize the need for effective instructional design and the progression of educational objectives, respectively. Moreover, Horner et al. (Horner et al., 2005) found that the emphasis on lower-level cognitive skills in learning objectives was inconsistent with the course level, highlighting the challenge of aligning objectives with the taxonomy. Athanasiou et al. argued that the taxonomy neglects the emotional aspect of learning in favor of emphasizing the development of learners' cognitive abilities (Athanassiou, 2003). Moreover, Crowe et al. further underscore the need for discipline-specific implementation, suggesting that the taxonomy may not always be easily applicable across different subjects (Crowe et al., 2008).

The most recent work was a survey conducted about the usage of Bloom in computer science disciplines (Masapanta-Carrión & Velázquez-Iturbide, 2018). Their findings show that one of the most common difficulties is the classification of learning goals or assessment tasks into the taxonomy levels, which can be challenging even for experienced educators. These studies collectively point to the need for careful consideration and adaptation of Bloom's Taxonomy to address the diverse pedagogical challenges in its implementation.

The specific challenges faced by the authors during the design and implementation of the classroom experience can be summarized as follows:

  • Some difficulties were encountered when developing activities and evaluations that effectively target and align with each cognitive domain.

  • Designing assessments that accurately measure students' abilities at each level of the taxonomy was found to be challenging. For instance, evaluating creativity (a higher-order skill) might be more subjective than assessing the basic recall of facts.

  • Since Industry 4.0 is highly technology-driven, the need for cutting-technology in the lab experience is important. However, access to advanced technologies was difficult mainly due to their cost, therefore limiting the amount of hands-on experience the students receive.

  • The lack of standardized IoT curricula made it difficult to ensure consistency and quality across educational programs. Establishing common standards for IoT education is an ongoing consideration.

Bloom's Taxonomy, while a valuable framework for categorizing cognitive skills and learning objectives, may not inherently promote assessment diversity. The taxonomy primarily focuses on cognitive processes, ranging from simple recall of facts to higher-order thinking skills like analysis and creativity. However, it doesn't prescribe specific assessment methods, leaving the choice of assessments to the educators. Without explicit guidance on diverse assessment strategies, there is a risk that assessments may disproportionately emphasize traditional, knowledge-recall assessments, such as exams and quizzes, over a broader range of evaluation methods.

On the other hand, alternative tools and frameworks offer a more explicit and dynamic approach to assessment diversity. Anderson and Krathwohl's Revised Taxonomy (Anderson & Krathwohl, 2001), for instance, maintains the cognitive process framework but allows for greater flexibility in adapting assessments to different levels. Webb's Depth of Knowledge (DOK) goes further by categorizing tasks based on their cognitive demands (Hess et al., 2009), encouraging educators to design assessments that range from basic recall to complex application and analysis. Universal Design for Learning (UDL) places a strong emphasis on providing multiple means of representation, engagement, and expression, ensuring assessments are inclusive and accessible to diverse learners (Hall et al., 2012). These alternatives prioritize a broader range of assessment formats, such as projects, presentations, and real-world applications, fostering a more comprehensive understanding of student abilities across various learning styles and preferences.

5.2 Conclusion

This paper focuses on the design and implementation of an IoT laboratory for students to learn and practice various aspects of IoT development. The objective is to provide students with hands-on experience in developing an IoT platform, setting up messaging protocols, analyzing data, and developing chatbots. The paper also highlights the importance of incorporating a Cyber-Physical System (CPS) approach into teaching activities. The evaluation methodology of the course is based on Bloom's taxonomy, and the paper highlights the benefits of a hybrid learning experience. Overall, the paper presents initial results and lays the foundation for a remotely deliverable IoT training course. The required learning outcomes of the students in this course are fundamental concepts for both IoT and IIoT; therefore, in the context of this work, the authors use the terms synonymously and don’t distinguish between them.

One of the key strengths of this paper is the emphasis on incorporating a Cyber-Physical System (CPS) approach in the teaching activities. By considering the teachers, classroom, and all resources involved in the process, the authors ensure a holistic and integrated learning experience for the students. This approach not only enhances their understanding of IoT concepts but also prepares them for the real-world challenges of implementing IoT solutions in industry settings.

By incorporating Bloom's Taxonomy into IoT education, instructors can ensure a well-rounded approach that covers foundational knowledge, practical skills, critical thinking, and creativity in the context of IoT systems and applications. This framework helps in designing effective learning experiences and assessments that cater to the cognitive needs of learners in the IoT field specifically and in Industry 4.0 in general.

In summary, this paper contributes significantly to the field of IoT education by providing a comprehensive and practical approach to teaching IoT concepts and skills. By designing an IoT laboratory that encompasses various aspects of IoT development and incorporating a CPS approach, the authors ensure a well-rounded learning experience for students. The evaluation methodology based on Bloom's taxonomy and the inclusion of a hybrid learning experience further enhance the effectiveness and accessibility of the course.

5.3 Future improvements

Regarding future improvements, the most imminent one is to define a clear list of elements (tools, concepts, ideas, capacities) for each Bloom’s taxonomy point (table cell) that is customized for the related case study, i.e., customized for the course and for the audience to which it is referred (a single student, a class, or another learners’ group). Through this method, it is possible to associate each variable, indicator, or mark with such an element of the list and to assign a weight to each element in order to define a weight function that, from these variables, evaluates a quantitative level for such a point.

One main lack of this work is addressing contents about the impact of Industry 4.0 in business and manufacturing models and the relationship of 4.0 technologies on sustainability or other concepts structuring a company view. For such reasons, a second main future improvement is to test the proposed method to assess a more general and complete manufacturing course that includes contents referring to Enterprise Resource Planning (ERP), Product Lifecycle Management (PLM), Manufacturing Execution system (MES), lean organization, and other 4.0 concepts like Additive Manufacturing (AM), collaborative robots (cobots), and Augmented and Virtual Reality (AR and VR). Such concepts and technologies should be included as contents and as components of the CPS, referring to the classroom, always with the objective of knowledge transfer, and talking about the technologies used to deliver the course.

The last planned improvement is to introduce the Digital Twin (DT) concept as education content and as an applied technology for the course management activity. The scope of this application is to build a DT of a student or of a classroom and to apply Bloom’s taxonomy as a framework to describe the knowledge about a particular state of the student or the classroom.