Teaching automation with Factory I/O under a competency-based curriculum

Some of the most critical competencies students need to acquire to become control engineers require performing practices under actual industrial conditions. This means that they must not only master the theoretical aspects of the discipline but also acquire skills and attitudes to face unpredictable real-world situations. Software tools such as Matlab/Simulink are widely used to train the design and validation of controllers, but they fail to provide real industrial contexts. Nowadays, there are 3D simulation tools that support recreating industrial environments to a remarkable extent, making them very attractive for university courses. Nevertheless, their application in engineering courses is scarce yet. This paper presents a methodological framework for seizing into competency-based courses one of these simulation tools, called Factory I/O. Our approach was evaluated in a master’s course on Industrial PID Control at Pontifical Catholic University of Valparaíso (PUCV) in Chile. The evaluation comprised the qualitative analysis of students’ grades over four consecutive course editions and the qualitative study of students’ opinion on Factory I/O educational value. The objectives of our evaluation were (i) testing if Factory I/O helped students develop skills hard to practice in academic contexts, such as detecting faults or recognizing the importance of having well-defined operation protocols; (ii) validating our methodology for competency-based courses; and (iii) surveying our students about Matlab/Simulink and Factory I/O strengths/weaknesses to teach control engineering. According to the results, (a) Factory I/O complements Simulink by providing an adequate virtual environment to learn the aforementioned skills; and (b) our methodology supports courses’ continuous improvement through the statistical analysis of students’ achievements at different abstraction levels.


Introduction
Control engineers play an essential role in the day-to-day design, operation, and maintenance of automation equipment and control systems in all process and manufacturing industries [10,35]. For that, a control engineer undergoes four to five years of disciplinary and vocational training, often combined with an apprentice scholarship in a private enterprise. During this period, s/he must acquire a high understanding of control theory and automation (knowledge), as well as real-world skills and attitudes (experience). However, universities tend to focus more on teaching theoretical knowledge than on promoting practical skills and attitudes. Two of the main reasons for this imbalance are: • Actual industrial equipment is expensive and/or requires a considerable laboratory physical space. Accordingly, universities can typically afford humble small didactical plants that cannot prepare students for the complexity inherent to actual plants in the industry. • Academic-oriented simulation tools, such as Matlab/Simulink, LabVIEW, or online labs [3,29] are frequently more used than industrial-oriented ones (InTouch, FastSuite, etc.).
As a consequence, although the curricula of these careers establish the formation of professional competencies in their graduates (including skills and attitudes), many times this is not fully achieved as declared [40]. On the other hand, the industry constantly puts pressure on educational institutions to produce control engineers with sufficient knowledge and experience so that in-company formal and/or non-formal learning is kept to a minimum [17,47]. This way, there is a need to provide learning materials and innovative pedagogical strategies that are relevant to the industry and to continuously update them as technology evolves.
To reduce this gap, this paper proposes an approach based on the combined use of traditional academic-oriented simulations together with an innovative 3D tool, named Factory I/O [19], which has been specially conceived for training students in control and automation [46]. This tool is commercialized by the company Real Games [44], and it includes a 3D engine to compose typical industrial automation scenarios up to a remarkable extent. Likewise, any element in the deployed scenario can be either simulated or connected to an actual device through an I/O point, supporting this way the combination of virtual and real parts.
On the other hand, given that the new pedagogical methods should contribute not only to the theoretical understanding but also to acquire practical experience, their integration into official teaching models must be formalized [25]. Regarding the educational models, nowadays, there is a clear tendency to that academic institutions redirect their teaching methodologies towards a Competency-Based Education (CBE) [22,28,36]. The CBE paradigm goes beyond the simplistic evaluation of students based only on their knowledge, also considering their skills and attitudes to face real problems [51]. This paper contributes to CBE by providing a guide to structure all the pedagogical elements in a competencybased course, particularly focused on the assessment aspects. Our proposed methodology has been applied to an Industrial PID Control course given at the Pontifical Catholic University of Valparaíso (PUCV) in Chile [33]. First, an evidence-based competency assessment model for the course was implemented. In this sense, assessment rubrics were specially designed to evaluate how learning outcomes and competencies were developed along the course by students. The data collected from the assessment tools allowed a refined quantitative analysis of the students' competency achievement levels. Finally, this analysis was complemented with the students' opinions that were gathered through surveys and helped us to analyze qualitatively the perception they had regarding our pedagogical methodology and the used software tools.
According to the results, the educational benefits of Factory I/O are ample, as they allow developing many skills hard to be practiced in an academic context. For instance, students learned how to detect and fix typical faults that suddenly appear in industrial systems, and they got awareness of the importance of adhering to established operation protocols.
The remainder of this paper is organized as follows. Section 2 summarizes related work. Section 3 explains the course and the required pedagogical features. Section 4 presents the learning resources used in the course under analysis. Section 5 describes the context where the methodology was applied and reports our evaluation results. Section 6 discusses the findings, implications, and limitations of our empirical evaluation. Finally, Section 7 provides some concluding remarks.

Related work
The literature on simulations and remotely teleoperated equipment to teach control engineering is ample [4,6,20,26,48,52]. Nevertheless, this section is devoted to this paper's main topic, and thus our review will be constrained to immersive 3D simulations for educational purposes in engineering courses.
In 1995, Hollands and Mort [27] proposed a virtual reality framework to create immersive 3D simulations of manufacturing systems. The framework's capabilities were tested with two simulations: a model car factory and a steel soaking pit plant. Following this seminal work, many other frameworks have appeared, mostly to support the rapid prototyping of manufacturing systems [2,9,13,16,21].
In contrast, some other frameworks have been born with an educational vocation. 3D techniques allow simulating an entire factory, and thus they provide students with a general perspective that integrates theoretical concepts and practical skills. Moreover, these simulations support exposing students to common problems and faults that arise in real contexts. Accordingly, 3D simulations have been used in a variety of educational domains and situations [7], such as engineering education [24], improving students' manipulative skills in chemical production processes [53,56], training technical personnel on power system operation and maintenance [54], or providing undergraduates and practitioners with a holistic understanding of smart factories [30].
This paper focuses on the domain of Programmable Logical Controllers (PLCs), where most 3D simulations are also oriented to support engineers in their work [8,31,32,39]. Nevertheless, a new outstanding tool called Factory I/O (https://factoryio.com/) has been recently launched for learning PLC programming and other automation technologies. Factory I/O has been created and is commercialized by the Portuguese company Real Games. Besides, some academic organizations are joining efforts to further develop Factory I/O, for instance, the CReSTIC laboratory at the University of Reims Champagne Ardenne [42,43,45] and the University of Mid Sweden [34].
Factory I/O enables the fast construction of virtual 3D factories by selecting among more than eighty configurable parts. Also, Factory I/O comes with over twenty readyto-use typical industrial applications (also called scenes) to practice real-world control tasks. Furthermore, Factory I/O supports blending virtual and real information by replacing data/control signals simulated by software with information coming from real devices; that is, Factory I/O provides many "I/O points" to plug actual PLCs, microcontrollers, data acquisition cards, FPGAs, etc. Finally, Factory I/O allows instructors to inject failures on parts and thus help students to practice troubleshooting routines (Fig. 1).
Recently, several conference papers have reported the successful application of Factory I/O to teach PLC programming in university courses all over the world, e.g., the Claude Bernard University (Lyon, France) [50], the Pan-American University (Aguascalientes, Mexico) [14,15], the Western Kentucky University (USA) [1], and the University of Baghdad (Iraq) [23].
Our work reports a successful case study as well. Moreover, it contributes to the existing literature by deepening the pedagogical aspects that need to be considered to seize Factory I/O capacities for improving students' motivation and learning.

Description of the course
This section describes the course on Industrial PID Control we teach as part of the master degree in Electrical Engineering at PUCV, which has the following contents: (1) PID control fundamentals, (2) Practical aspects of PID control, (3) PLC programming, and (4) Labs for training PID control.
It is important to point out this course is the continuation of another one named automatic control, which was described in [51]. The degree follows the CBE paradigm, encompassing a total of twenty competencies. In this course, students develop two of them: C 1 Design, evaluation, and optimization of equipment operation and maintenance in industrial processes C 2 Detection and diagnosis of equipment faults in industrial processes, establishing possible solutions.
These competencies define a set of Learning Outcomes (LOs) that students should demonstrate at the end of the course. For this course, four LOs were defined (see Table 1).

Fig. 1
The virtual tank is controlled through a real PLC Table 1 Competencies and learning outcomes for a course on industrial PID controllers at PUCV Competency C 1 Design, evaluation, and optimization of equipment operation and maintenance in industrial processes Learning outcomes Assessment Indicators (content, skills, and longterm attitudes or values) LO 11 : students will be able to design PID control loops and evaluate their correct operation Ind 1 : The student identifies processes and their variables, modeling their behavior, either analytically or empirically Ind 2 : The student designs ideal PID controllers using both analytical and empirical methods Ind 3 : The student evaluates the performance of PID control loops, checking their correct operation LO 12 : students will be able to optimize PID control loops for improving their operation and maintenance Ind 4 : The student implements real PID control applications, taking into account the process working limits Finally, for assessment purposes, these LOs were decomposed into low-level assessment indicators, which are obtained from Assessment Tools (ATs) applied to students. Table 2 summarizes the competency assessment model defined for the course which involves applying three assessment tools: AT 1 Theoretical test of PID control fundamentals AT 2 Practical experience using Matlab/Simulink AT 3 Practical experience using Factory I/O First, some lectures are provided to teach the first two course contents. The involved subjects are: (i) process modeling and identification, (ii) analysis and design of PID controllers by analytical and empirical methods, and (iii) practical aspects of implementing PID controllers. During lectures, Matlab/Simulink is used to reinforce the theoretical explanations. Subsequently, AT 1 is applied to students for gathering partially the assessment indicators Ind 1 -Ind 4 (see Fig. 2). Second, students are required to perform simulation activities with Matlab/Simulink in order to get started on training implementation skills of PID controllers. Thus, they face AT 2 , where the analysis, design and implementation of a liquid level control system must be developed. As a consequence, a complementary part of Ind 1 -Ind 4 is obtained.
Finally, some practical training sessions to teach about PLC programming and Factory I/O are given. After that, AT 3 is applied to students by requiring them to undertake the commissioning of the level control system of Factory I/O and to perform a set of lab assignments on it. As a consequence, the remaining part of Ind 1 -Ind 4 and Ind 5 -Ind 10 are obtained.
Once that all low-level indicators are gathered, a numeric qualification for each LO is obtained and consequently, C 1 and C 2 are evaluated according to the weights in Table 2. Finally, students pass the course with a final grade FG obtained as the weighting sum up of competencies grades. All these qualifications will be later used to perform the final course learning analytics.

Learning resources
This section explains how Factory I/O, together with Matlab/Simulink, is being used to teach control at PUCV; in this particular case, to learn practical aspects of PID controllers for the Industry. The material we have created for our students includes an original Factory I/O application and a collection of Simulink templates, which are freely available at the following repository: https://github.com/rheradio/FactoryIO

Tank level control
The liquid level regulation inside containers is a common problem in the manufacturing industry of multiple productive sectors. Consequently, it is part of most control engineering curricula [18]. In its most basic form, the problem is simplified to a single tank that receives an input liquid flow, which is partially drained out through a discharge outlet. Figure 3 sketches this single tank control scheme.
Equations (1) and (2) represent, respectively, the non-linear and linearized differential equations of the tank system that model the conservation of the mass; h accounts for the level of the liquid inside the tank, A the area or section of the tank, a the area of the drain hole, g the acceleration of gravity and u the voltage applied to the valve that regulates the liquid input flow to the tank (V 1 in Fig. 3).
Based on this simple scheme, several control concepts can be studied, such as linearization, operation-point, system identification, simple PID control design, control saturation, Fig. 3 Single tank control scheme anti-windup strategies, bumpless problem, among others. In addition, advanced PID control problems can be considered by adding a draining valve (V 2 in Fig. 3).
First, we use Factory I/O to motivate our students, showing them the industrial importance and complexities of the tank problem. Then, they use Simulink to work with the single tank mathematical model. Finally, Factory I/O is used again to provide our students with a more holistic and realistic perspective and to practice PLC programming.

Simulink activities
We ask our students to obtain the mathematical model of the process, using a set of physical parameters different for each student (a, A, h max and K m ). Then, students have to choose an operation-point (h 0 , u 0 ) on which they work for getting a linearized model of the tank. Subsequently, students carry out open-loop experiments on both non-linear and linear tank systems over the operation-point. Matlab/Simulink is used in these learning activities, being a primary objective that students verify that both non-linear and linear models work equally when these are validated around the operation-point. Thus, the correct understanding of modeling and linearization concepts is reaffirmed before going on.
Later, the control saturation effect and anti-windup mechanisms are analyzed. Figure 4 shows the Simulink base model we provide our students for learning these concepts.
First, the PID controller is configured in its most basic form; that is, without saturation nor anti-windup methods (neither TR nor Saturation are used in Fig. 4). Students must design a PID controller using analytical methods to fulfill a set of performance specifications. Then, students run several closed-loop simulations around the operation-point to check their design, considering both non-linear and linear models. For instance, the blue color signals in Fig. 5 show a typical closed-loop experiment with good results at this learning stage. As it can be seen, the liquid level raises the reference under the required specifications and with a clear control signal without saturation (i.e., the ideal response).
Second, students are required to introduce a control saturation into the Simulink model (now using the Saturation block in Fig. 4), observing its effects over the response and control signals. The orange signals in Fig. 5 show the tank dynamic behavior under these operating conditions. Students can see that the response does not fulfill the required specifications as a consequence of the control signal limitation, i.e., the control signal is saturated all the time, and thus the closed-loop is not working as the linear control theory postulates. This problem is known as windup effect.
Third, students must implement an anti-windup mechanism that counteracts the windup effect as much as possible. Figure 6 shows the block diagram of a PID controller that includes an anti-windup method known as integral tracking. At this point, students must (i) tune a new control parameter T r , and (ii) include a tracking signal for the saturated control  signal TR into the PID algorithm. The green signals in Fig. 5 show how the process response and control signal improve as a consequence of this change. Students can see how the difference between the ideal control signal (the one that the PID controller generates) and the saturated control signal (the one interpreted by the actuator) is used to compute the integral action that equals both signals.
Practical aspects about PID control implementation many times are not easily observed by recent graduates in industrial applications, thus playing the next section a key role. Figure 7(a) shows a real scenario, where the level control of chemical products for making foam mattresses is performed. This particular industrial application of the tank is used by instructors to motivate students at the beginning of this learning activity. Some videos of the system working are watched and analyzed in the first lecture as a previous step to start the final stage of the course.

Factory I/O activities
As seen in Fig. 7(b), Factory I/O comes with a complete scene that simulates the single tank problem in a remarkable way and very similar to the previously analyzed.

Exploring the instrumentation
Students start inspecting the documentation available at https://docs.factoryio.com/ to identify the tank's physical parameters. Utilizing this information, they develop the specific non-linear and linear model of the tank system in Factory I/O as they did in the previous learning activities. At the same time, we encourage our students to move freely throughout the 3D scenario (see Fig. 8), inspecting every component and checking the technical properties of the instrumentation used in the automation scene which is listed in Table 3.
This is a major advantage regarding the previously done with Matlab/Simulink, where sensors, actuators, and other practical equipment used in real industrial applications are

PLC programming and PID control
After the initial exploration, students practice PLC programming for controlling the tank. To do so, they work with a PLC Siemens S7-1200 connected to the computer running Factory Table 3 Industrial instrumentation in the tank system

Equipment Description
Fill valve (analog output): Input flow valve driven by a pneumatic actuator that can be positioned with voltage signals between 0 and 10 V.
Discharge valve (analog output): Output flow valve driven by a pneumatic actuator that can be positioned with voltage signals between 0 and 10 V.

Level meter (analog input):
Capacitive level sensor used to detect liquid levels. The measurement is provided by a voltage signal between 0 and 10 V.

PLC (Siemens S7-1200):
Programmable Logic Controller used to control the system. TIA Portal software is used to program the PLC using its internal PID controller block.
Control Panel: Electric switchboard that contains the PLC and operators such as: one potentiometer (analog input), two displays (analog outputs), and three push buttons (digital inputs). Fig. 1). The PLC is programmed using the Siemens Totally Integrated Automation software TIA Portal [49]. For didactical purposes, the simpler tank control scheme performed by students is described hereafter. In this case, they firstly work operating only the filling valve, so the liquid discharge valve must remain open. To do that, students must force the discharge valve to totally open, which is achieved by defining its operating limit to the maximum in the control panel in Fig. 8. At this point, they should observe (and keep in mind) the operating limits of the rest of the components exposed in Table 3.

I/O (as shown in
Subsequently, students proceed to configure the communication with the external controller by assigning a memory address in the PLC in order to identify the input and output variables of the simulated process by tags in Factory I/O (see Fig. 9). Later, these same variables (or tags) are used by students to program the control logic.
By using the TIA Portal software, the control logic of the system is then programmed. For this purpose, students must identify and use the different functional tools offered by this software along with the well-known Ladder programming logic [5]. One of the main programming blocks studied in this course corresponds to the PID-Compact, which contains the structure of a non-interactive PID controller with an anti-windup filter (see Fig. 10). This block allows to select the regulator structure and the measurement unit of the controlled variable, to define the actuator saturation limits, and finally establish the PID control parameters (K p , T i and T d ). For the latter, students must design these parameters for the Factory I/O tank model, considering the PID concepts previously studied and evaluated in AT 1 and AT 2 .
To start the PID block configuration, students must assign the memory addresses in the PLC according to the type of data to be used (Word, INT, DINT, etc.), then the process signals must be normalized and scaled to work with the actual values of the system. For this case study, the input and output signals mentioned in Table 3 must be assigned to the PID Compact block. These, in turn, are associated with certain parts of the instrumentation shown in Fig. 8.
Once the connection between both software (Factory I/O and TIA Portal) is established, the control signals are displayed using the real-time plots offered by TIA Portal. Likewise,  Fig. 11.
By analyzing the signal graphs, students can evaluate the performance of the implemented regulator, comparing the results obtained with Matlab/Simulink and Factory I/O (with TIA Portal). One of the interesting activities that students performed during this learning activity was the comparison of the performance specifications required in the problem with those obtained by using Matlab/Simulink. Table 4 shows the percentage of error associated with the maximum overshoot (M p ) and settling-time (T s ) obtained by a student.

Fault detection, diagnosis and recovery
After the design and implementation of the PID controller, students are required to develop the final learning experience intended to training skills and attitudes associated with the detection, diagnosis and repair of equipment failures in industrial processes, corresponding to the competency C 2 in Table 1. To do so, instructors first perform a lecture to teach students industrial maintenance protocols typically used for the detection of events and failures related to the components of a process and how to proceed in these situations. Based on this lecture, instructors present the most frequent failures that may occur on the tank system in order to promote skills to fault detection, diagnosis and recovery. Moreover, they describe a set of possible actions that could compose a maintenance protocol for this process, such as: (i) initial overview of the system; (ii) physical and logical connection examination; (iii) sensors, actuators, and reference values testing; (iv) PID control parameter checkup, and (v) PLC program testing. This way, work attitudes can be trained and evaluated based on whether the student is able to create an appropriate maintenance protocol and systematically applies it under these situations.
Finally, the latest lab assignments are applied to students for assessment purposes. In this case, instructors inject a set of fault events, either in the programming of the control logic or by contextualizing a problematic situation in the system instrumentation. The objective of these exercises is that the student can firstly prepare a maintenance protocol proposal and then apply it to detect and establish the possible causes of a failure in an industrial context and finally fix it. Figure 12 illustrates a flowchart that represents the steps followed by students in the final stage of AT 3 . As shown in Fig. 12, students could bypass the dashed red-line box and go ahead with the fixing action without finding out the cause of failure. This is explained by instructors in the initial lecture given that, in a real industrial context, many times the operators know how to repair the fault without knowing its cause and they must fix it as soon as possible in order to avoid a prolonged stop of the process. Nevertheless, this is an important stage which should  Fig. 12 Flowchart representing the steps followed by students in the last lab assignment be done during or even after the repairing action. Table 5 describes some examples of fault events. According to the maintenance protocol created, the students must first detect the fault and then ideally establish its possible cause to provide an efficient solution. When a limited period of time to complete the assignment is given, students could bypass finding out the fault cause. Nevertheless, students who repair the fault, also establishing its cause, are better evaluated than those who do not. Table 6 describes the possible causes of the fault events presented in Table 5.

Experimental results
This section reports the results of the experimental evaluation we undertook on a master's course on Industrial PID Control at Pontifical Catholic University of Valparaíso (PUCV) with the following Objectives: • O1: Evaluate if C 2 is being taught adequately with the support of Factory I/O.
Course 2020-21 incorporated for first the time Competency C 2 . To help students perform experiments related to detecting, diagnosing, and fixing faults in industrial environments, we decided to use Factory I/O. To evaluate if the new competency was taught consistently with the rest of the course, we were interested in answering the following Research Question: • O2: Evaluate the benefit of the students' decompositional assessment we propose.
Before the course 2020-21, students' evaluation basically consisted of a global final grade. In 2020-21, we decided to decompose the evaluation into competencies, learning outcomes, and low-level indicators (see Section 3). We wanted to check if this extra work pays off. In particular, we were interested in answering the following question: • O3: Students' opinion concerning Matlab/Simulink and Factory I/OA. Matlab/Simulink and Factory I/O are commercial and expensive tools. Consequently, we Process instrumentation failure A fault is introduced in the initial configuration of the simulation in Factory I/O, specifically in the process instrumentation, which causes the controlled variable does not follow the setpoint correctly.
PID controller failure A fault is introduced in the parameterization of the PID controller block. This causes that the imposed performance specifications are not met. • The numerical format of the display is not congruent with the one configured in TIA Portal.
• The digital display is not programmatically connected to the potentiometer.
Process instrumentation failure • Inlet valve opening is forced, adjusting a constant tension.
• The discharge valve is not fully open.
• The liquid level sensor is forced, providing a constant voltage.
PID controller failure • Incorrect value of the control parameters (K p , T i and T d ).
• Incorrect configuration of the process limits or scaling values.
were particularly interested in knowing if the investment, especially in Factory I/O, pays off and thus justifies its acquisition. Accordingly, we were interested in answering the following question:

RQ1:
Are students acquiring C 2 at an academic level comparable to the one they are learning C 1 ?
The master's course on Industrial PID Control has been taught since 2017. Table 7 summarizes the number of students per course. The box plots in Fig. 13 show the grades students got on a scale from 0 to 100 (each course is represented with a different color). While the first three courses consisted exclusively of teaching C 1 , the last course introduced C 2 . No major difference is appreciated between C 1 and C 2 in the figure. To check this numerically, Table 8 summarizes the results of analyzing the course differences with Tukey's Honestly Significant Difference (HSD) test. Of all comparisons, only two are statistically significant: the grades students got in the first course compared with the grades obtained in the last course (i.e., both C 1 and C 2 in 2020-21 are different than C 1 in 2017-18).

RQ2: Does students' decompositional assessment help to identify course elements that need improvement?
The box plots in Fig. 14 summarize the grades that students obtained in Course 2020-21. Thanks to our decompositional methodology, grades can be analyzed at different abstraction levels. The higher level examination reveals that students achieved extraordinary marks. Looking at the low-level indicators, we detected the course should be improved to raise above all Ind 3 , and also Ind 2 and Ind 4 , which got the worst results. Usually, the PID controller design (Ind 2 ) and, even more, its validation (Ind 3 ) are the stages where students present more doubts. Likewise, Ind 4 is the point where students stop working with an ideal PID controller and incorporate the saturation limits of the actuator, adding an extra difficulty that is reflected in Fig. 14. Highly structured questionnaires are useful in that they generate response frequencies amenable for comparing students' opinions [11]. Figure 15 summarizes the obtained frequencies for ten questions that inquiry about the Simulink and Factory I/O benefits to 17% 25% 50% Q10 -learning how to perform the preventive maintenance of a control system Q09 -mastering how to set up an industrial control system Q08 -learning how to detect, diagnose, and repair common flaws that arise in industrial control systems Q07 -acknowledging the necessity of following action protocols for control systems Q06 -giving me the perception of being physically present in an industrial automation environment Q05 -acquiring a holistic view of a control system Q04 -recognizing the differences between the theoretical design of a PID controller and its actual implementation The following tools have contributed to...

Fig. 15
Students' response frequencies to the closed questions

Discussion
The following points summarize the key findings per research question: • RQ1: Are students acquiring C 2 at an academic level comparable to the one they are learning C 1 ?. Yes, they are. According to the results summarized in Fig. 14 and Table 8, students are acquiring C 2 with the support of Factory I/O at a level similar to what they are acquiring C 1 with Simulink. In other words, we are teaching C 1 and C 2 consistently. • RQ2: Does students' decompositional assessment help to identify course elements that need improvement?. Yes, it does. Thanks to the decomposition of competencies into learning outcomes and low-level indicators, we detected that the design and validation of PID controllers are not being taught adequately. Thus we should reframe them in the next courses. 3: S02, S11-S12 It is very versatile, and it comes with a wide variety of ready-to-use functionalities for control systems (i.e., toolboxes).
2: S03, S10 It is easy to use in a very intuitive way.
1: S01 It helps to model a control system starting from empirical data.

Q12 -Weaknesses of Simulink
6: S01-S03, S05, S09, S11 Too theoretical; its idealized behavior does not match reality; it is not able to take into account important factors that typically produce problems in real plants.
2: S04, S06 Excessively "block-oriented": it is challenging to define a functionality when no block supports it.
1: S07 Its learning curve is steep for students without programming expertise.

Q13 -Strengths of Factory I/O
11: S02-S12 It provides a realistic and immersive environment; it helps to understand how control systems really work by watching the behavior of actual physical elements; it gives students a real context. 5: S01, S06, S09, S11-S12 It supports controller implementation on actual PLCs.
1: S10 It includes a library of predefined plants.

Q14 -Weaknesses of Factory I/O
2: S01, S06 It has restricted compatibility with other software and hardware.
• RQ3: What do students think about the educational value of Matlab/Simulink and Factory I/O for learning control engineering?. According to their responses, students think that both tools are valuable in general. Simulink seems to be better for understanding specific contents, such as the analysis and design of PID control loops. In contrast, Factory I/O is superior to arouse students' interest to learn industrial automation. Also, it provides a more holistic vision of control systems that encompasses many different aspects, such as fault detection and repair, system's set up and maintenance, and awareness of the importance that has to adhere to established protocols. Accordingly, students think that Simulink and Factory I/O complement each other.
The implications of our research are twofold: 1. Competency-based education has shown to provide important benefits, including deeper learning, increased student intrinsic motivation, autonomy and competence, etc. [12]. As a result, this paradigm is very extended nowadays. Our decompositional methodology can be applied to any discipline's competency-based courses, thus helping instructors identify their teaching strengths and weaknesses for continuously improving their courses.
2. Our experimental validation shows that modern 3D simulation tools (Factory I/O) adequately complement classic tools (Simulink) to teach students how to face practical problems that arise in industrial automation and go beyond the design and implementation of PID controllers.
It is worth noting that our work has the following limitations: • Our empirical validation has been performed on a single institution: the Pontifical Catholic University of Valparaíso (PUCV) in Chile. The same experimental validation should be conducted in other institutions worldwide to confirm the results. It is worth remarking that an obstacle to undertaking these additional validations is the considerable cost of Simulink and Factory I/O, which may not be affordable by other institutions. • Our course is currently devoted to teaching PID control, and thus our validation is restricted to this particular type of control. We do not think this is a significant concern since (i) PID controllers are the most used ones in practice, and (ii) our setup can be easily extended to learn other types of control. Concerning the latter, our setup comprises Factory I/O and Siemens PLC S7-1200, on which a variety of controllers have been implemented, including Fuzzy Controllers [41], Model Predictive Controllers (MPC) [37], a Nonlinear Differential and Integral Sliding Mode Controller (NDISMC) [55], etc.

Conclusions
The potential benefits of making available a whole factory to engineering students from their first class are tremendous. Students can easily (i) understand the motivations of the course contents, (ii) appreciate the limitations of simplistic solutions and develop the interest in more sophisticated approaches, (iii) put into practice every content they learn, (iv) test their own creative solutions to novel problems, (v) acquire an integrated understanding of the entire system, (vi) practice fault detection and troubleshooting routines, etc. Factory I/O brings closer this utopia, supporting hybrid factories that blend virtual reality with a variable number of actual devices. Nevertheless, technology by itself is not magical, and thus its integration into the classroom needs to be carefully planned. In this paper, we have described how Factory I/O was used in a master's course on Industrial PID Control at Pontifical Catholic University of Valparaíso (PUCV) to train students on practical problems that arise in industrial environments and go beyond the mere implementation of controllers. Also, the paper includes the methodology we propose to design the course curriculum and evaluate students, which decomposes competencies into learning outcomes and low-level indicators. The application of our methodology revealed the following findings: • Factory I/O provides a satisfactory virtual environment to teach fault detection, diagnosis and fixing in automated engineering courses. This was confirmed both quantitatively with the grades students obtained, and qualitatively by surveying students. • Decomposing competencies' assessment enables the identification of the strengths and weaknesses of a course and thus its continuous improvement.