1 Introduction

Control engineers play an essential role in the day-to-day design, operation, and maintenance of automation equipment and control systems in all process and manufacturing industries [10, 35]. For that, a control engineer undergoes four to five years of disciplinary and vocational training, often combined with an apprentice scholarship in a private enterprise. During this period, s/he must acquire a high understanding of control theory and automation (knowledge), as well as real-world skills and attitudes (experience). However, universities tend to focus more on teaching theoretical knowledge than on promoting practical skills and attitudes. Two of the main reasons for this imbalance are:

  • Actual industrial equipment is expensive and/or requires a considerable laboratory physical space. Accordingly, universities can typically afford humble small didactical plants that cannot prepare students for the complexity inherent to actual plants in the industry.

  • Academic-oriented simulation tools, such as Matlab/Simulink, LabVIEW, or online labs [3, 29] are frequently more used than industrial-oriented ones (InTouch, FastSuite, etc.).

As a consequence, although the curricula of these careers establish the formation of professional competencies in their graduates (including skills and attitudes), many times this is not fully achieved as declared [40]. On the other hand, the industry constantly puts pressure on educational institutions to produce control engineers with sufficient knowledge and experience so that in-company formal and/or non-formal learning is kept to a minimum [17, 47]. This way, there is a need to provide learning materials and innovative pedagogical strategies that are relevant to the industry and to continuously update them as technology evolves.

To reduce this gap, this paper proposes an approach based on the combined use of traditional academic-oriented simulations together with an innovative 3D tool, named Factory I/O [19], which has been specially conceived for training students in control and automation [46]. This tool is commercialized by the company Real Games [44], and it includes a 3D engine to compose typical industrial automation scenarios up to a remarkable extent. Likewise, any element in the deployed scenario can be either simulated or connected to an actual device through an I/O point, supporting this way the combination of virtual and real parts.

On the other hand, given that the new pedagogical methods should contribute not only to the theoretical understanding but also to acquire practical experience, their integration into official teaching models must be formalized [25]. Regarding the educational models, nowadays, there is a clear tendency to that academic institutions redirect their teaching methodologies towards a Competency-Based Education (CBE) [22, 28, 36]. The CBE paradigm goes beyond the simplistic evaluation of students based only on their knowledge, also considering their skills and attitudes to face real problems [51]. This paper contributes to CBE by providing a guide to structure all the pedagogical elements in a competency-based course, particularly focused on the assessment aspects. Our proposed methodology has been applied to an Industrial PID Control course given at the Pontifical Catholic University of Valparaíso (PUCV) in Chile [33]. First, an evidence-based competency assessment model for the course was implemented. In this sense, assessment rubrics were specially designed to evaluate how learning outcomes and competencies were developed along the course by students. The data collected from the assessment tools allowed a refined quantitative analysis of the students’ competency achievement levels. Finally, this analysis was complemented with the students’ opinions that were gathered through surveys and helped us to analyze qualitatively the perception they had regarding our pedagogical methodology and the used software tools.

According to the results, the educational benefits of Factory I/O are ample, as they allow developing many skills hard to be practiced in an academic context. For instance, students learned how to detect and fix typical faults that suddenly appear in industrial systems, and they got awareness of the importance of adhering to established operation protocols.

The remainder of this paper is organized as follows. Section 2 summarizes related work. Section 3 explains the course and the required pedagogical features. Section 4 presents the learning resources used in the course under analysis. Section 5 describes the context where the methodology was applied and reports our evaluation results. Section 6 discusses the findings, implications, and limitations of our empirical evaluation. Finally, Section 7 provides some concluding remarks.

2 Related work

The literature on simulations and remotely teleoperated equipment to teach control engineering is ample [4, 6, 20, 26, 48, 52]. Nevertheless, this section is devoted to this paper’s main topic, and thus our review will be constrained to immersive 3D simulations for educational purposes in engineering courses.

In 1995, Hollands and Mort [27] proposed a virtual reality framework to create immersive 3D simulations of manufacturing systems. The framework’s capabilities were tested with two simulations: a model car factory and a steel soaking pit plant. Following this seminal work, many other frameworks have appeared, mostly to support the rapid prototyping of manufacturing systems [2, 9, 13, 16, 21].

In contrast, some other frameworks have been born with an educational vocation. 3D techniques allow simulating an entire factory, and thus they provide students with a general perspective that integrates theoretical concepts and practical skills. Moreover, these simulations support exposing students to common problems and faults that arise in real contexts. Accordingly, 3D simulations have been used in a variety of educational domains and situations [7], such as engineering education [24], improving students’ manipulative skills in chemical production processes [53, 56], training technical personnel on power system operation and maintenance [54], or providing undergraduates and practitioners with a holistic understanding of smart factories [30].

This paper focuses on the domain of Programmable Logical Controllers (PLCs), where most 3D simulations are also oriented to support engineers in their work [8, 31, 32, 39]. Nevertheless, a new outstanding tool called Factory I/O (https://factoryio.com/) has been recently launched for learning PLC programming and other automation technologies. Factory I/O has been created and is commercialized by the Portuguese company Real Games. Besides, some academic organizations are joining efforts to further develop Factory I/O, for instance, the CReSTIC laboratory at the University of Reims Champagne Ardenne [42, 43, 45] and the University of Mid Sweden [34].

Factory I/O enables the fast construction of virtual 3D factories by selecting among more than eighty configurable parts. Also, Factory I/O comes with over twenty ready-to-use typical industrial applications (also called scenes) to practice real-world control tasks. Furthermore, Factory I/O supports blending virtual and real information by replacing data/control signals simulated by software with information coming from real devices; that is, Factory I/O provides many “I/O points” to plug actual PLCs, microcontrollers, data acquisition cards, FPGAs, etc. Finally, Factory I/O allows instructors to inject failures on parts and thus help students to practice troubleshooting routines (Fig. 1).

Fig. 1
figure 1

The virtual tank is controlled through a real PLC

Full size image

Recently, several conference papers have reported the successful application of Factory I/O to teach PLC programming in university courses all over the world, e.g., the Claude Bernard University (Lyon, France) [50], the Pan-American University (Aguascalientes, Mexico) [14, 15], the Western Kentucky University (USA) [1], and the University of Baghdad (Iraq) [23].

Our work reports a successful case study as well. Moreover, it contributes to the existing literature by deepening the pedagogical aspects that need to be considered to seize Factory I/O capacities for improving students’ motivation and learning.

3 Description of the course

This section describes the course on Industrial PID Control we teach as part of the master degree in Electrical Engineering at PUCV, which has the following contents: (1) PID control fundamentals, (2) Practical aspects of PID control, (3) PLC programming, and (4) Labs for training PID control.

It is important to point out this course is the continuation of another one named automatic control, which was described in [51]. The degree follows the CBE paradigm, encompassing a total of twenty competencies. In this course, students develop two of them:

C 1 :

Design, evaluation, and optimization of equipment operation and maintenance in industrial processes

C 2 :

Detection and diagnosis of equipment faults in industrial processes, establishing possible solutions.

These competencies define a set of Learning Outcomes (LOs) that students should demonstrate at the end of the course. For this course, four LOs were defined (see Table 1). Finally, for assessment purposes, these LOs were decomposed into low-level assessment indicators, which are obtained from Assessment Tools (ATs) applied to students. Table 2 summarizes the competency assessment model defined for the course which involves applying three assessment tools:

Table 1 Competencies and learning outcomes for a course on industrial PID controllers at PUCV
Full size table
Table 2 Competency assessment structure
Full size table
AT1:

Theoretical test of PID control fundamentals

AT2:

Practical experience using Matlab/Simulink

AT3:

Practical experience using Factory I/O

First, some lectures are provided to teach the first two course contents. The involved subjects are: (i) process modeling and identification, (ii) analysis and design of PID controllers by analytical and empirical methods, and (iii) practical aspects of implementing PID controllers. During lectures, Matlab/Simulink is used to reinforce the theoretical explanations. Subsequently, AT1 is applied to students for gathering partially the assessment indicators Ind1-Ind4 (see Fig. 2).

Fig. 2
figure 2

Low-level indicator percentage values obtained from each AT

Full size image

Second, students are required to perform simulation activities with Matlab/Simulink in order to get started on training implementation skills of PID controllers. Thus, they face AT2, where the analysis, design and implementation of a liquid level control system must be developed. As a consequence, a complementary part of Ind1-Ind4 is obtained.

Finally, some practical training sessions to teach about PLC programming and Factory I/O are given. After that, AT3 is applied to students by requiring them to undertake the commissioning of the level control system of Factory I/O and to perform a set of lab assignments on it. As a consequence, the remaining part of Ind1-Ind4 and Ind5- Ind10 are obtained.

Once that all low-level indicators are gathered, a numeric qualification for each LO is obtained and consequently, C1 and C2 are evaluated according to the weights in Table 2. Finally, students pass the course with a final grade FG obtained as the weighting sum up of competencies grades. All these qualifications will be later used to perform the final course learning analytics.

4 Learning resources

This section explains how Factory I/O, together with Matlab/Simulink, is being used to teach control at PUCV; in this particular case, to learn practical aspects of PID controllers for the Industry. The material we have created for our students includes an original Factory I/O application and a collection of Simulink templates, which are freely available at the following repository: https://github.com/rheradio/FactoryIO

4.1 Tank level control

The liquid level regulation inside containers is a common problem in the manufacturing industry of multiple productive sectors. Consequently, it is part of most control engineering curricula [18]. In its most basic form, the problem is simplified to a single tank that receives an input liquid flow, which is partially drained out through a discharge outlet. Figure 3 sketches this single tank control scheme.

Fig. 3
figure 3

Single tank control scheme

Full size image

Equations (1) and (2) represent, respectively, the non-linear and linearized differential equations of the tank system that model the conservation of the mass; h accounts for the level of the liquid inside the tank, A the area or section of the tank, a the area of the drain hole, g the acceleration of gravity and u the voltage applied to the valve that regulates the liquid input flow to the tank (V 1 in Fig. 3).

$$ \frac{dh}{dt}= -\frac{a}{A}\sqrt{2gh} + \frac{K_{m}}{A}u $$
(1)
$$ \frac{d{\overline{h}}}{dt}= -\frac{a\overline{h}}{A}\sqrt{\frac{g}{2\overline{h}}} + \frac{K_{m}}{A}\overline{u} $$
(2)

Finally, making \(d\overline {h}/dt=0\) in (2), an operation-point (h0, u0) can be established (3).

$$ {u_{o}}= \frac{a\sqrt{2g{h_{o}}}}{K_{m}} $$
(3)

Based on this simple scheme, several control concepts can be studied, such as linearization, operation-point, system identification, simple PID control design, control saturation, anti-windup strategies, bumpless problem, among others. In addition, advanced PID control problems can be considered by adding a draining valve (V 2 in Fig. 3).

First, we use Factory I/O to motivate our students, showing them the industrial importance and complexities of the tank problem. Then, they use Simulink to work with the single tank mathematical model. Finally, Factory I/O is used again to provide our students with a more holistic and realistic perspective and to practice PLC programming.

4.2 Simulink activities

We ask our students to obtain the mathematical model of the process, using a set of physical parameters different for each student (a, A, \(h_{\max \limits }\) and Km). Then, students have to choose an operation-point (h0, u0) on which they work for getting a linearized model of the tank. Subsequently, students carry out open-loop experiments on both non-linear and linear tank systems over the operation-point. Matlab/Simulink is used in these learning activities, being a primary objective that students verify that both non-linear and linear models work equally when these are validated around the operation-point. Thus, the correct understanding of modeling and linearization concepts is reaffirmed before going on.

Later, the control saturation effect and anti-windup mechanisms are analyzed. Figure 4 shows the Simulink base model we provide our students for learning these concepts.

Fig. 4
figure 4

Basic simulink model for tank level control

Full size image

First, the PID controller is configured in its most basic form; that is, without saturation nor anti-windup methods (neither TR nor Saturation are used in Fig. 4). Students must design a PID controller using analytical methods to fulfill a set of performance specifications. Then, students run several closed-loop simulations around the operation-point to check their design, considering both non-linear and linear models. For instance, the blue color signals in Fig. 5 show a typical closed-loop experiment with good results at this learning stage. As it can be seen, the liquid level raises the reference under the required specifications and with a clear control signal without saturation (i.e., the ideal response).

Fig. 5
figure 5

Checking the behavior of a single tank controller

Full size image

Second, students are required to introduce a control saturation into the Simulink model (now using the Saturation block in Fig. 4), observing its effects over the response and control signals. The orange signals in Fig. 5 show the tank dynamic behavior under these operating conditions. Students can see that the response does not fulfill the required specifications as a consequence of the control signal limitation, i.e., the control signal is saturated all the time, and thus the closed-loop is not working as the linear control theory postulates. This problem is known as windup effect.

Third, students must implement an anti-windup mechanism that counteracts the windup effect as much as possible. Figure 6 shows the block diagram of a PID controller that includes an anti-windup method known as integral tracking. At this point, students must (i) tune a new control parameter Tr, and (ii) include a tracking signal for the saturated control signal TR into the PID algorithm. The green signals in Fig. 5 show how the process response and control signal improve as a consequence of this change. Students can see how the difference between the ideal control signal (the one that the PID controller generates) and the saturated control signal (the one interpreted by the actuator) is used to compute the integral action that equals both signals.

Fig. 6
figure 6

Anti-windup method using an integral tracking mechanism

Full size image

Practical aspects about PID control implementation many times are not easily observed by recent graduates in industrial applications, thus playing the next section a key role.

4.3 Factory I/O activities

Figure 7(a) shows a real scenario, where the level control of chemical products for making foam mattresses is performed. This particular industrial application of the tank is used by instructors to motivate students at the beginning of this learning activity. Some videos of the system working are watched and analyzed in the first lecture as a previous step to start the final stage of the course.

Fig. 7
figure 7

Industrial application v/s simulation in Factory I/O

Full size image

As seen in Fig. 7(b), Factory I/O comes with a complete scene that simulates the single tank problem in a remarkable way and very similar to the previously analyzed.

4.3.1 Exploring the instrumentation

Students start inspecting the documentation available at https://docs.factoryio.com/to identify the tank’s physical parameters. Utilizing this information, they develop the specific non-linear and linear model of the tank system in Factory I/O as they did in the previous learning activities. At the same time, we encourage our students to move freely throughout the 3D scenario (see Fig. 8), inspecting every component and checking the technical properties of the instrumentation used in the automation scene which is listed in Table 3.

Fig. 8
figure 8

Single tank application fully provided by Factory I/O

Full size image
Table 3 Industrial instrumentation in the tank system
Full size table

This is a major advantage regarding the previously done with Matlab/Simulink, where sensors, actuators, and other practical equipment used in real industrial applications are somehow hidden. Once the instrumentation is verified, students can prove the tank operating limits and its dynamic effects by using the control panel as shown in Fig. 8. Likewise, a specific operation point can be obtained from (3) and then verified visually over the tank.

4.3.2 PLC programming and PID control

After the initial exploration, students practice PLC programming for controlling the tank. To do so, they work with a PLC Siemens S7-1200 connected to the computer running Factory I/O (as shown in Fig. 1). The PLC is programmed using the Siemens Totally Integrated Automation software TIA Portal [49].

For didactical purposes, the simpler tank control scheme performed by students is described hereafter. In this case, they firstly work operating only the filling valve, so the liquid discharge valve must remain open. To do that, students must force the discharge valve to totally open, which is achieved by defining its operating limit to the maximum in the control panel in Fig. 8. At this point, they should observe (and keep in mind) the operating limits of the rest of the components exposed in Table 3.

Subsequently, students proceed to configure the communication with the external controller by assigning a memory address in the PLC in order to identify the input and output variables of the simulated process by tags in Factory I/O (see Fig. 9). Later, these same variables (or tags) are used by students to program the control logic.

Fig. 9
figure 9

Tag configuration to identify I/O variables in Factory I/O

Full size image

By using the TIA Portal software, the control logic of the system is then programmed. For this purpose, students must identify and use the different functional tools offered by this software along with the well-known Ladder programming logic [5]. One of the main programming blocks studied in this course corresponds to the PID-Compact, which contains the structure of a non-interactive PID controller with an anti-windup filter (see Fig. 10). This block allows to select the regulator structure and the measurement unit of the controlled variable, to define the actuator saturation limits, and finally establish the PID control parameters (Kp, Ti and Td). For the latter, students must design these parameters for the Factory I/O tank model, considering the PID concepts previously studied and evaluated in AT1 and AT2.

Fig. 10
figure 10

PID compact instruction in ladder programming on TIA Portal

Full size image

To start the PID block configuration, students must assign the memory addresses in the PLC according to the type of data to be used (Word, INT, DINT, etc.), then the process signals must be normalized and scaled to work with the actual values of the system. For this case study, the input and output signals mentioned in Table 3 must be assigned to the PID Compact block. These, in turn, are associated with certain parts of the instrumentation shown in Fig. 8.

Once the connection between both software (Factory I/O and TIA Portal) is established, the control signals are displayed using the real-time plots offered by TIA Portal. Likewise, students check the process status by visualizing the 3D animations of Factory I/O such as shown in Fig. 11.

Fig. 11
figure 11

Visualizations of the system behavior in Factory I/O (bottom figure) and signal time evolution on the TIA Portal (top figures)

Full size image

By analyzing the signal graphs, students can evaluate the performance of the implemented regulator, comparing the results obtained with Matlab/Simulink and Factory I/O (with TIA Portal). One of the interesting activities that students performed during this learning activity was the comparison of the performance specifications required in the problem with those obtained by using Matlab/Simulink. Table 4 shows the percentage of error associated with the maximum overshoot (Mp) and settling-time (Ts) obtained by a student.

Table 4 Overshoot and settling time of the time response
Full size table
Fig. 12
figure 12

Flowchart representing the steps followed by students in the last lab assignment

Full size image

4.3.3 Fault detection, diagnosis and recovery

After the design and implementation of the PID controller, students are required to develop the final learning experience intended to training skills and attitudes associated with the detection, diagnosis and repair of equipment failures in industrial processes, corresponding to the competency C2 in Table 1. To do so, instructors first perform a lecture to teach students industrial maintenance protocols typically used for the detection of events and failures related to the components of a process and how to proceed in these situations. Based on this lecture, instructors present the most frequent failures that may occur on the tank system in order to promote skills to fault detection, diagnosis and recovery. Moreover, they describe a set of possible actions that could compose a maintenance protocol for this process, such as: (i) initial overview of the system; (ii) physical and logical connection examination; (iii) sensors, actuators, and reference values testing; (iv) PID control parameter checkup, and (v) PLC program testing. This way, work attitudes can be trained and evaluated based on whether the student is able to create an appropriate maintenance protocol and systematically applies it under these situations.

Finally, the latest lab assignments are applied to students for assessment purposes. In this case, instructors inject a set of fault events, either in the programming of the control logic or by contextualizing a problematic situation in the system instrumentation. The objective of these exercises is that the student can firstly prepare a maintenance protocol proposal and then apply it to detect and establish the possible causes of a failure in an industrial context and finally fix it. Figure 12 illustrates a flowchart that represents the steps followed by students in the final stage of AT3.

As shown in Fig. 12, students could bypass the dashed red-line box and go ahead with the fixing action without finding out the cause of failure. This is explained by instructors in the initial lecture given that, in a real industrial context, many times the operators know how to repair the fault without knowing its cause and they must fix it as soon as possible in order to avoid a prolonged stop of the process. Nevertheless, this is an important stage which should be done during or even after the repairing action. Table 5 describes some examples of fault events.

Table 5 Examples of fault events
Full size table

According to the maintenance protocol created, the students must first detect the fault and then ideally establish its possible cause to provide an efficient solution. When a limited period of time to complete the assignment is given, students could bypass finding out the fault cause. Nevertheless, students who repair the fault, also establishing its cause, are better evaluated than those who do not. Table 6 describes the possible causes of the fault events presented in Table 5.

Table 6 Possible causes of failures
Full size table

5 Experimental results

This section reports the results of the experimental evaluation we undertook on a master’s course on Industrial PID Control at Pontifical Catholic University of Valparaíso (PUCV) with the following Objectives:

  • O1: Evaluate if C2 is being taught adequately with the support of Factory I/O. Course 2020-21 incorporated for first the time Competency C2. To help students perform experiments related to detecting, diagnosing, and fixing faults in industrial environments, we decided to use Factory I/O. To evaluate if the new competency was taught consistently with the rest of the course, we were interested in answering the following Research Question:

  • O2: Evaluate the benefit of the students’ decompositional assessment we propose. Before the course 2020-21, students’ evaluation basically consisted of a global final grade. In 2020-21, we decided to decompose the evaluation into competencies, learning outcomes, and low-level indicators (see Section 3). We wanted to check if this extra work pays off. In particular, we were interested in answering the following question:

  • O3: Students’ opinion concerning Matlab/Simulink and Factory I/OA. Matlab/Simulink and Factory I/O are commercial and expensive tools. Consequently, we were particularly interested in knowing if the investment, especially in Factory I/O, pays off and thus justifies its acquisition. Accordingly, we were interested in answering the following question:

5.1 RQ1: Are students acquiring C 2 at an academic level comparable to the one they are learning C 1?

The master’s course on Industrial PID Control has been taught since 2017. Table 7 summarizes the number of students per course.

Table 7 Number of students per course
Full size table

The box plots in Fig. 13 show the grades students got on a scale from 0 to 100 (each course is represented with a different color). While the first three courses consisted exclusively of teaching C1, the last course introduced C2. No major difference is appreciated between C1 and C2 in the figure. To check this numerically, Table 8 summarizes the results of analyzing the course differences with Tukey’s Honestly Significant Difference (HSD) test. Of all comparisons, only two are statistically significant: the grades students got in the first course compared with the grades obtained in the last course (i.e., both C1 and C2 in 2020-21 are different than C1 in 2017-18).

Fig. 13
figure 13

Students’ grades obtained in each course

Full size image
Table 8 Tukey’s HSD multiple comparison between course grades (95% statically significant differences are highlighted with *)
Full size table

5.2 RQ2: Does students’ decompositional assessment help to identify course elements that need improvement?

The box plots in Fig. 14 summarize the grades that students obtained in Course 2020-21. Thanks to our decompositional methodology, grades can be analyzed at different abstraction levels. The higher level examination reveals that students achieved extraordinary marks. Looking at the low-level indicators, we detected the course should be improved to raise above all Ind3, and also Ind2 and Ind4, which got the worst results. Usually, the PID controller design (Ind2) and, even more, its validation (Ind3) are the stages where students present more doubts. Likewise, Ind4 is the point where students stop working with an ideal PID controller and incorporate the saturation limits of the actuator, adding an extra difficulty that is reflected in Fig. 14.

Fig. 14
figure 14

Students’ grades decomposed at different abstraction levels (Course 2020-21).

Full size image

5.3 RQ3: What do students think about the educational value of Matlab/Simulink and Factory I/O for learning control engineering?

Closed questions

We designed a questionnaire to gather students’ opinions about the educational value of Simulink and Factory I/O. Out of the 15 students in the course, 12 answered the voluntary questionnaire, which was divided into two parts: ten closed questions and four open-ended questions.

Highly structured questionnaires are useful in that they generate response frequencies amenable for comparing students’ opinions [11]. Figure 15 summarizes the obtained frequencies for ten questions that inquiry about the Simulink and Factory I/O benefits to acquiring control engineering competencies. Responses are rated on a five-point Likert scale.

Fig. 15
figure 15

Students’ response frequencies to the closed questions

Full size image

Open-ended questions

Open-ended questions are a complement to closed questions that invite honest and personal comments from students. They are especially useful when the number of respondents is reduced [11]. Following Oppenheim’s recommendations [38], the questions were formulated as sentence-completion items to center students’ responses. For example:

Table 9 summarizes students’ responses. Cohen et al.’s [11] notation is used. The left-hand column indicates the number of students that agree with the response in the right-hand column, followed by the students’ identifiers. For instance, according to the first row, eight students (with identifiers S02, S04, S05, etc.) agree that one of Simulink’s strengths is that it helps to simulate control systems, and thus to understand their behavior visually.

Table 9 Students’ responses to the open-ended questions
Full size table

6 Discussion

The following points summarize the key findings per research question:

  • RQ1: Are students acquiring C2 at an academic level comparable to the one they are learning C1?. Yes, they are. According to the results summarized in Fig. 14 and Table 8, students are acquiring C2 with the support of Factory I/O at a level similar to what they are acquiring C1 with Simulink. In other words, we are teaching C1 and C2 consistently.

  • RQ2: Does students’ decompositional assessment help to identify course elements that need improvement?. Yes, it does. Thanks to the decomposition of competencies into learning outcomes and low-level indicators, we detected that the design and validation of PID controllers are not being taught adequately. Thus we should reframe them in the next courses.

  • RQ3: What do students think about the educational value of Matlab/Simulink and Factory I/O for learning control engineering?. According to their responses, students think that both tools are valuable in general. Simulink seems to be better for understanding specific contents, such as the analysis and design of PID control loops. In contrast, Factory I/O is superior to arouse students’ interest to learn industrial automation. Also, it provides a more holistic vision of control systems that encompasses many different aspects, such as fault detection and repair, system’s set up and maintenance, and awareness of the importance that has to adhere to established protocols. Accordingly, students think that Simulink and Factory I/O complement each other.

The implications of our research are twofold:

  1. 1.

    Competency-based education has shown to provide important benefits, including deeper learning, increased student intrinsic motivation, autonomy and competence, etc. [12]. As a result, this paradigm is very extended nowadays. Our decompositional methodology can be applied to any discipline’s competency-based courses, thus helping instructors identify their teaching strengths and weaknesses for continuously improving their courses.

  2. 2.

    Our experimental validation shows that modern 3D simulation tools (Factory I/O) adequately complement classic tools (Simulink) to teach students how to face practical problems that arise in industrial automation and go beyond the design and implementation of PID controllers.

It is worth noting that our work has the following limitations:

  • Our empirical validation has been performed on a single institution: the Pontifical Catholic University of Valparaíso (PUCV) in Chile. The same experimental validation should be conducted in other institutions worldwide to confirm the results. It is worth remarking that an obstacle to undertaking these additional validations is the considerable cost of Simulink and Factory I/O, which may not be affordable by other institutions.

  • Our course is currently devoted to teaching PID control, and thus our validation is restricted to this particular type of control. We do not think this is a significant concern since (i) PID controllers are the most used ones in practice, and (ii) our setup can be easily extended to learn other types of control. Concerning the latter, our setup comprises Factory I/O and Siemens PLC S7-1200, on which a variety of controllers have been implemented, including Fuzzy Controllers [41], Model Predictive Controllers (MPC) [37], a Nonlinear Differential and Integral Sliding Mode Controller (NDISMC) [55], etc.

7 Conclusions

The potential benefits of making available a whole factory to engineering students from their first class are tremendous. Students can easily (i) understand the motivations of the course contents, (ii) appreciate the limitations of simplistic solutions and develop the interest in more sophisticated approaches, (iii) put into practice every content they learn, (iv) test their own creative solutions to novel problems, (v) acquire an integrated understanding of the entire system, (vi) practice fault detection and troubleshooting routines, etc.

Factory I/O brings closer this utopia, supporting hybrid factories that blend virtual reality with a variable number of actual devices. Nevertheless, technology by itself is not magical, and thus its integration into the classroom needs to be carefully planned. In this paper, we have described how Factory I/O was used in a master’s course on Industrial PID Control at Pontifical Catholic University of Valparaíso (PUCV) to train students on practical problems that arise in industrial environments and go beyond the mere implementation of controllers. Also, the paper includes the methodology we propose to design the course curriculum and evaluate students, which decomposes competencies into learning outcomes and low-level indicators. The application of our methodology revealed the following findings:

  • Factory I/O provides a satisfactory virtual environment to teach fault detection, diagnosis and fixing in automated engineering courses. This was confirmed both quantitatively with the grades students obtained, and qualitatively by surveying students.

  • Decomposing competencies’ assessment enables the identification of the strengths and weaknesses of a course and thus its continuous improvement.