Keywords

1 Introduction

Safety-critical systems (SCS) [20] are the systems whose failure could result in loss of life, substantial economic loss, or damage to the environment. There are many well-known examples in different domains such as avionics, nuclear plants, transportation, and automotive. Formal verification is highly recommended by safety standards, e.g., DO-178C for the avionics domain, in order to ensure the safety of this kind of systems [17]. SCADE is both a formal language [9] and a model-based development environmentFootnote 1 widely used to build and verify the models of safety-critical system. SCADE provides three modeling styles, i.e., safety state machines, data flow and their combination.

Design Verifier (DV)Footnote 2, the formal verification module of SCADE, is a model checker based on a SAT-solver. However, DV cannot adequately express complex temporal specifications [24]. Temporal logics are popular methods for describing complex temporal properties, such as LTL [23], CTL [7] and TCTL [1], etc. There are several related works [12, 24] to enhance the verification capability of SCADE, which respectively transform SCADE models into UPPAAL and nuSmv to verify temporal properties. However, these works only consider safety state machine models. Additionally, DV may fail due to complexity problems such as floating numbers which are common in the aeronautics domain.

Moreover, it is always a challenge to manually translate natural language requirements into temporal logic formulae. As natural language is generally informal and ambiguous, this process is error-prone and time-consuming. Existing works on translating natural language requirements into temporal logics can be classified as several categories, such as rule-based [15, 29], deep learning [18], and Large Language Models (LLMs) [11]. However, these methods require either manual writing of formal specifications for atomic propositions, or utilizing plenty of patterns, or training with plenty of data. To the best of our knowledge, these works always consider the translation of English, but few consider Chinese, and there is little work focusing on the ejection seat control system domain.

To address the challenges above, we propose an AGVTS method, automatically generating and verifying temporal specifications for aeronautics SCADE models. The main contributions are summarized as follows:

  1. (1)

    A modular pattern language (MPL) to precisely express Chinese natural language requirements. Benefiting from the modular structure, users can write requirements in a restricted and composite way with less patterns.

  2. (2)

    A rule-based method augmented with BERT for automatically translating requirements expressed by MPL into LTL and CTL formulae.

  3. (3)

    An automated transformation from SCADE to nuXmv that provides SMT-based and SAT-based model checking techniques to verify LTL and CTL properties with floating numbers. Compared with existing works, our transformation covers more modeling styles, such as data flow, safety state machines and their combination.

  4. (4)

    We apply our method to an industrial ejection seat control system. It successfully translates 124 requirements and verifies the SCADE models of six modules of the system. The result convinces our industrial partners of the usefulness of formal methods to industrial systems.

2 Global View of the AGVTS Method

Figure 1 gives an overview of the AGVTS method. AGVTS has three modules shown as follows.

  • Modular Pattern Language (MPL): Define a pattern language organized in a modular structure. The syntax and semantics of each pattern guide users to write requirements in a restricted and composite manner.

  • Rule-based Requirements Translator Augmented with BERT: Parse requirements written in MPL and build pattern structure trees for them. Then generate LTL and CTL formulae through traversing the tree.

  • SCADE2nuXmv Model Transformation: Transform SCADE models into nuXmv. Subsequently, verify the nuXmv models, and show the verification results and the traceability between requirements and counterexamples.

In the following sections, we will introduce the three modules in detail.

Fig. 1.
figure 1

The Global View of the AGVTS Method

Full size image

3 Modular Pattern-Based Language

The modular pattern-based language (MPL) focuses on ejection seat control system which usually contains complex computation in the requirements. MPL has three segments: Atomic Proposition (AP), Relation, and Scope. Each segment has several patterns. This feature allows users to write requirements in a composite manner with fewer patterns. Additionally, we define the formal syntax and semantics of the patternsFootnote 3 As shown in Fig. 2, we first write three atomic statements in AP patterns. Then the Relation patterns are utilized to connect statements for constructing compound statements and Scope patterns are added to declare the effective extent of different statements. Finally we obtain the complete requirement. Please note that the Relation and Scope patterns can be nested in any way.

Fig. 2.
figure 2

Writing Requirements in a Composite Manner

Full size image

AP Patterns. AP patterns form the basis of MPL, serving to specify an atomic statement on an event or a state of the system. MPL has nine AP patterns. Each pattern consists of several Ingredient tags we define, such as Ne for variables and constants, Ope for operators, and Formula for complex computation. We have defined seven Ingredient tags in total. For Instance, in Fig. 2, “the indicator velocity equals \(abs(0.5*(Vi\_GD1)+0.5*(Vi\_GD1))\)” is written in “Ne Ope Formula” pattern, in which “rational flag of inertial navigation module 1”, “equals” and “\(abs(0.5*(Vi\_GD1)+0.5*(Vi\_GD1))\)” are labeled as Ne, Ope, and Formula respectively.

Relation Patterns. Relation patterns describe the temporal or logical relations between atomic or compound statements, which are characterized by different keywords. We have defined six Relation patterns: Simple, Response, Condition, Precedence, Conjunction and Disjunction, as shown in Table 1 where \(\phi _i\;(1\,\le \,i\,\le \,n)\) denotes a statement. The Simple pattern represents the atomic statements in the requirement, which are specified by the AP patterns. The Response and Precedence patterns are introduced from [2]. Conjunction and Disjunction patterns express the complex nesting relations of a series of statements, as the parentheses are seldom used in Chinese requirements. To support the nesting of Relation pattern, we define a priority for each of them.

For example, in Fig. 2, we use Conjunction pattern to connect two atomic statements. The compound statement “the rational flag of inertial navigation module 1 is true and the rational flag of inertial navigation module 2 is true” indicates that the rational flags of both inertial navigation modules must be true simultaneously. Moreover, the statement “\(\phi _1\) weak and \(\phi _2\), or \(\phi _3\) weak and \(\phi _4\)” can express “\((\phi _1\,\wedge \,\phi _2)\vee (\phi _3\,\wedge \,\phi _4)\)”, based on different priorities.

Table 1. Relation Patterns
Full size table

Scope Patterns. Scope patterns describe the effective extent of an atomic or compound statement. We have defined 11 Scope patterns, such as “Globally”, “In x state”, and “Every n cycles”, where x denotes a state name and n denotes a positive integer. As shown in Fig. 2, the scope “In the inertial navigation valid state” expresses that the property stated subsequently must hold when the system is in the “inertial navigation valid” state.

4 Rule-Based Translation Augmented with BERT

In this section, we will introduce the rule-based translation augmented with BERT method which translates requirements into LTL and CTL specifications. As shown in Fig. 3, to illustrate the workflow of the method, we consider the following requirement:

Example 1

Globally, if the state of the system is calculating angle by two inertial navigation modules, then globally the input angle always equals 0.5 times the sum of the angles in inertial modules 1 and 2.

In the following paragraphs, we will represent the details of each step.

Fig. 3.
figure 3

Overview of the Rule-Based Translation Augmented with BERT

Full size image

Rewriting & Pre-processing. Natural language are usually ambiguous, so we rewrite the requirement in MPL first. Additionally, the complex calculations expressed in natural language should be replaced with corresponding formal expression to simplify the translation. For example, the statement “0.5 times the sum of the angles in inertial modules 1 and 2” in Example 1 is replaced by “\((0.5*(angle\_GD1 + angle\_GD2))\)”. The Example 1 is rewritten as “Globally, if the state equals calculate angle by two inertial navigation modules state, then globally, the input angle equals \((0.5*(angle\_GD1 + angle\_GD2))\)”.

Extract Scopes & Natural Language Parsing. The accuracy of a NLP parser decreases as the length of the requirement increases, so the requirement should be as concise as possible. Since the expressions of Scope patterns in MPL are fixed, we extract them with regular expression before parsing the rewritten requirement. For example, we extract “Globally,” from Example 1. Then HanLPFootnote 4, a Chinese NLP toolkit, is leveraged to parse the requirement, including tokenization, POS tagging, and dependency relations analysis.

Extracting Terms by BERT. Compared with English, Chinese natural language lacks separators to distinguish words. Therefore, tokenization is the initial task when parsing Chinese with NLP techniques. However, incorrect tokenization may occur. Clearly, an incorrect tokenization will finally lead to a wrong parsing result. For example, “\(angle\_GD1 + angle\_GD2\)” in Example 1 is recognized as one token.

Fig. 4.
figure 4

The Structure of the BERT-Based Model

Full size image
Fig. 5.
figure 5

Examples of a Terms Map

Full size image

To solve this problem, we implement a Term Extractor to extract terms from requirements, which assists in improving the accuracy of word segmentation and correcting its results. Term extraction essentially classifies each token in the requirement into three categories: the beginning of the term, the body of the term, and non-term. Therefore, we build a deep learning model, as shown in Fig. 4, to complete this task. This model first utilizes BERT [13], a pre-trained language model, to extract text features from the requirements. Then a fully connected layer (FC Layer) is utilized to calculate the probability of each token belonging to different categories based on the text features. Finally, each token is labeled as the category with the highest probability.

The extracted terms serve two purposes. One is to expand the tokenization lexicon of HanLP, which reduces the probability of incorrect tokenization. The other is to rectify the potential errors in the parsing result of HanLP. In addition, we construct a terms map in order to match the extracted terms with their corresponding variables or constants in the SCADE model, as shown in Fig. 5.

Build Pattern Structure Tree & Rectify NLP Errors. After getting the NLP parsing result, we recursively construct a pattern structure tree for each statement in the requirement. The whole pattern structure tree describes the nesting relations among the Relation patterns present in the requirement. Each node of the tree corresponds to a statement in the requirement, including the Relation pattern of the statement, the keywords of the Relation pattern, and the Scope pattern of the statement. Please note that each leaf node of this tree corresponds to an atomic statement in the requirement.

Each Relation pattern recorded by the tree node represents the one with the highest priority in its corresponding statement. To determine the Relation pattern with the highest priority, we utilize tokenization and lexical matching to find the keywords of Relation patterns. Then we compare the priorities of the patterns they belong to, as shown in Table 2.

In cases where no keywords are identified, the pattern of the statement is the Simple pattern. Then we identify the AP pattern of the statement by determining the Ingredient tag of each token and combining the tags in order. The task is performed by tokenization, lexical matching and POS tagging. Additionally, we utilize domain lexicons which include verbs, adjectives, operators and functions in the requirement to filter useless tokens. The extracted terms are leveraged to enhance the NLP result in this process. That is, for tokens categorized as “noun”, we substitute the original token with the term that matches the longest consecutive characters in the requirement.

As shown in Fig. 3, the Condition pattern recorded by the root node of the pattern structure tree has the highest priority in Example 1. Both of its children are atomic statements, so the corresponding nodes of these statements record their AP and Scope patterns. In this process, the incorrect token “\(angle\_GD1+angle\_GD2\)” is rectified to “\(angle\_GD1\)”, “\(angle\_DD2\)” and the operator “+”.

Formula Generation. The last step is generating formal specifications by composing formal expressions of each node on the pattern structure tree in post order. The mapping of our Relation patterns to LTL and CTL are shown in Table 2, and the mapping rules of AP patterns and Scope patterns are illustrated in the appendix. These rules strictly follow the formal semantics of MPL.

Table 2. Mapping Rules of Relation Patterns (Except for Simple Pattern)
Full size table

Based on the mapping rules above, as shown in Fig. 3, we first translate the leaf nodes on the pattern structure tree into AP formulae. For instance, “state equals calculate angle by two inertial navigation modules state” is translated to “\(state\,=\,state\_CalA\_BothGD\_State\)”. Secondly, the generated formulae are connected by “\(\rightarrow \)” which is translated from the keywords recorded by the root node. During the translation, the Scope patterns ,“globally”, are translated into corresponding temporal operators and added to the corresponding formulae. The LTL and CTL formulae are shown as follows:

$$\begin{aligned} \begin{aligned} LTL: G((state\,&=\,state\_CalA\_BothGD\_State)\rightarrow \\ &G(angle\,=\,(0.5*(angle\_GD1\,+\,angle\_GD2))))\\ \end{aligned} \end{aligned}$$
(1)
$$\begin{aligned} \begin{aligned} CTL: AG((state\,&=\,state\_CalA\_BothGD\_State)\rightarrow \\ &AG(angle\,=\,(0.5*(angle\_GD1\,+\,angle\_GD2)))) \end{aligned} \end{aligned}$$
(2)

5 SCADE2nuXmv Model Transformation

In this section, we will introduce the automated transformation from SCADE to nuXmv. Figure 6 shows the overview of SCADE2nuXmv. We first extract the textual representation of the SCADE model through SCADE IDE. Subsequently, use ANTLR, a toolkit for lexical analysis and syntax analysis, to build a syntax tree for it. Then generate target nuXmv models based on the syntax tree. Finally, we employ the model checker of nuXmv to verify the generated model. The traceability between the execution trace of counterexamples, and corresponding formulae and requirements is also generated.

Fig. 6.
figure 6

SCADE Models Verification Achieved by SCADE2nuXmv

Full size image

The reasons why we choose nuXmv are the ability to express hierarchical models, support for both SMT-based and SAT-based verification, and the verification of both infinite-state and finite-state systems. For instance, the middle and right columns in Fig. 7 construct a nuXmv model that contains hierarchical state machine. The SM_EJ_Core is the top state machine of the model. It declares the sub-state machine SM_GDValid_SSM module, which selects strategy according to the different variables (e.g., Vi), in its VAR part.

Fig. 7.
figure 7

The Translation from SCADE to nuXmv

Full size image

The nuXmv models generated by our method contains four module types: Monitor, State Machine, Function and Main. The Monitor module implements the monitor mechanism. The State Machine module and Function module respectively represent safety state machine and data flow in SCADE models. The Main module is the entry of target models. In the following, we will introduce these modules in detail.

Fig. 8.
figure 8

The SCADE Model Example

Full size image

Monitor Module. The language supported by nuXmv forbids multiple assignments to a variable [4], which is common in safety state machine, so we created Monitor modules inspired by [8] to manipulate the assignment of output and local variables. Each Monitor module incorporates two “case” expressions with multiple execution branches, to assign the value of the monitored variable. One initializes the monitored variable, while the other updates it after the first cycle.

Each “case” expression branch corresponds to an assignment expression of the monitored variable in the original SCADE model. To determine the branch to execute, we create monitor variables similar to the ones in [8]. Each monitored variable triggers a branch when it becomes true. To distinguish between assignment expressions related to state transitions, we employ two types of monitor variables, \(reset\_x\_s\) and \(set\_x\_s\), where x denotes the monitored variable and s denotes the state assigning the value of x. When the state machine transitions to s, \(reset\_x\_s\) turns TRUE. If none of the transition conditions of state s are met, \(set\_x\_s\) becomes TRUE. We further replace variables prefixed with “_L”, which represent connection lines in the SCADE model, with their equivalent variables to reduce the state space. For example, the “\(Set\_Vi\)” in Fig. 7 is an example of such a module. The “case” expression assigns the value of “Vi” according to the monitor variables of each branch.

State Machine Module. Each State Machine module corresponds to one state machine in the original SCADE model. It records the state transition, assigns monitor variables and instantiates submachines. Such modules also implement the hierarchical structure in safety state machine utilizing active and default variables, which respectively denote the activation of the state machine and whether the state machine transitions from inactive to active in the cycle. For example, the “SM_GDValid_SSM” module in Fig. 7 is a State Machine module, representing the state machine “GDValid_SSM” in Fig. 8. The first “case” expression depicts the state transition, while subsequent one assigns monitor variables related to it.

Function Module. A Function module corresponds to a data flow operator in the original SCADE model. Each output variable of it is defined by two expressions in the corresponding Function module. One, the same as the expression in the SCADE model, assigns its initial value. The other, similar to the expression in the SCADE model but with each variable enclosed by a next operator, assigns the value of the variable after the first cycle. After the translation, we further replace the temporary variables “_Li” (where i is a positive integer), that represent connection lines in the SCADE model, with their equivalent variables to reduce the state space. For example, the “CalVi” module in Fig. 7 is a Function module. The temporary variable “_Li” is replaced with its equivalent variable, e.g., “_L2” is replaced with “V_HG”.

Main Module. Each nuXmv model has one Main module which instantiates the interface of the SCADE model, the local variables, all monitor variables and modules, and the top state machine. It also assigns the initial value of all monitoring variables. Additionally, a variable equalling the state of the top state machine is defined in the Main module, allowing users to verify specifications related to a specific state. This variable only exists when the original SCADE model contains safety state machines.

As mentioned in Sect. 1, SCADE provides three modeling styles, i.e., safety state machines, data flow and their combination. When the SCADE model is data flow style, its nuXmv model contains Main Module and Function Module. On the contrary, when the SCADE model is safety state machine style, we use State Machine and Main modules to construct the target nuXmv models. In addition, we use all the above four types of modules to construct the target nuXmv model for the SCADE model that combines data flow and safety state machine.

6 Industrial Case Studies and Evaluation

6.1 Ejection Seat Control System

Ejection seats must eject pilots out of the cockpit when the pilots pull the switch and open the parachute at an appropriate time to safely send pilots back to the ground. When the seat is ejected from the cockpit, the control system in the seat chooses different control strategies as the environment varies. During this process, it avoids rotation, excessive loads and wrong movement direction. When the seat is in a non-upright position, the control system adjusts the attitude of the seat.

As a typical safety-critical software, SCADE is suitable to model this control system. Moreover, the control system controls all subsystems of the ejection seat to faithfully implement the chosen strategy. Each step in the strategy has a strict execution order. This feature makes temporal logic suitable for describing the specifications of its requirements.

6.2 Implementation and Experiments

We have developed a tool chain to implement the AGVTS method in this paper. To verify the effectiveness of AGVTS, we utilize the tool chain to verify six modules in the ejection seat control system against a set of 124 requirements.

Fig. 9.
figure 9

Implementation of the Approach

Full size image

As shown in Fig. 9, the tool chain first uses an BERT-based terms extractor, which is fine-tuned on the requirements of the ejection seat control system, to extract terms from the requirements written in MPL. Industrial engineers subsequently confirm and rectify the errors in the extracted terms. Then they build a terms map based on the terms. Thirdly, the specification generator reads the terms map and converts the requirements written in MPL into LTL and CTL formulae. Finally a model transformer & verifier transforms the SCADE model into nuXmv and verifies the generated formulae against the nuXmv model. Additionally, it generates the traceability between the execution trace of counterexamples, and corresponding formulae and requirements to help rectify the original model.

To ensure the effectiveness of the tool chain, we invite several formal experts to confirm whether the generated LTL and CTL formulas accurately capture the intent of the requirements. Then the experts check the generated nuXmv models to determine whether these models faithfully implement the functions of the original models.

6.3 Evaluation

The major objective of this subsection is to evaluate the effectiveness of our method. We will explain the evaluation from three perspectives: Terms Extraction, Requirements Translation and Model Verification.

Table 3. The Statistics of Our BERT-Based Model
Full size table

Terms Extraction. As shown in Table 3, the BERT-based terms extractor achieves a precision of 93.94% and a recall of 95.88% on the fine-tuned test set. The 124 requirements used in our experiment contain 87 terms. The BERT-based terms extractor extracts 91 terms, 79 of which are correct. Error-extracted terms can be divided into two categories. One is common nouns (e.g., “cycle”) that do not belong to ejection seat control system. The other is the segment of larger terms, for example “Inertial Navigation Module” is a wrong term split from “Valid Inertial Navigation Module State”. However, these can be easily identified and rectified by engineers.

Table 4. The Statistics of the Verification Results
Full size table

Requirements Translation. Our method successfully translates all the pre-processed requirements written in the pattern based language into LTL and CTL formulae. Table 4 shows the translation statistics data. In the following we provide two requirements described in MPL to show the translation.

Example 2

“Globally, in the Inertial Navigation Valid State, the mode will be set to reverse mode, or the mode will be set to low velocity low altitude mode, or the mode will be set to high velocity mode, or the mode will be set to high altitude mode”.

This is a property used to limit the output of the system whose corresponding LTL and CTL formulae are shown as follows.

$$\begin{aligned} \begin{aligned} LTL: G(( state = state\_&GDValid\_State ) \rightarrow \\ F(&mode = 4\,|\,mode = 6\,|\,mode = 0\,|\,mode = 5)) \end{aligned} \end{aligned}$$
(3)
$$\begin{aligned} \begin{aligned} CTL: AG(( state = state&\_GDValid\_State ) \rightarrow \\ AF(&mode = 4\,|\,mode = 6\,|\,mode = 0\,|\,mode = 5)) \end{aligned} \end{aligned}$$
(4)

Please note that, the terms map has matched the reverse mode, low velocity low altitude mode, high velocity mode, and high altitude mode to the constants defined in the SCADE model. For instance “the mode will be set to reverse mode” is translated to “\(mode = 4\)”.

Example 3

“If the rational flag of inertial navigation module 1 is true, and the rational flag of inertial navigation module 2 is true, then in the next cycle, state will be set to calculate_angle_Both_Inertial_Valid state”.

As the statement does not have a Scope, it is only checked in the first cycle. Formula (5) and (6) represent the LTL and CTL formula of this requirement.

$$ \begin{aligned} \begin{aligned} LTL: (Validity\_GD1 = 1\,& \& \,Validity\_GD2 = 1 )\rightarrow \\ X(&state = state\_CalA\_BothGD\_State) \end{aligned} \end{aligned}$$
(5)
$$ \begin{aligned} \begin{aligned} CTL: (Validity\_GD1 = 1\,& \& \,Validity\_GD2 = 1 )\rightarrow \\ AX(&state = state\_CalA\_BothGD\_State) \end{aligned} \end{aligned}$$
(6)

Model Verification. The SCADE models of the six modules contain three types of structure: data flow, safety state machine and their combination. Our method successfully transforms them. Then we verify the nuXmv models with the generated formulae.

During the verification, two bugs caused by the “case” statement are found. The first one is that the front case condition covers the latter case condition. The second one is caused by two identical case condition, but their actions are different. This result shows that our method for verification is effective. Additionally, our method verifies the SCADE models of the six modules in a shorter time than DV.

Four requirements of the Velocity Control mode contain nonlinear calculations, such as square root, that cannot be verified by SCADE DV and our method. Note that, when the SCADE models contain unbounded integers and real numbers, nuXmv just verify the LTL specifications.

7 Related Work

Formal Specification Generation. As writing formal specifications is time-consuming and error-prone, [14, 19] propose a set of patterns corresponding to different scenarios and their formal semantics to guide users to write formal specifications. [15] develops a SpeAR tool which translates requirements written in pattern language into PLTL. [10, 16, 21] provide a framework to translate requirements written in the pattern language FRETISH into formal specifications. [2] proposes a framework combining existing classical patterns. [22, 26, 29] utilize NLP techniques, such as POS tagging and dependency parsing, to pre-process requirements. Then they define translation rules to generate formal specifications based on the pre-processed requirements. [18, 25] treat the translation from natural language to formal specifications as a machine translation task and utilize deep learning models to solve it. [5, 11] utilize Large Language Models (LLMs) to complete the translation. [11] decomposes the natural language input into sub-translations by utilizing LLMs. However, these methods require either manual writing of formal specifications for atomic propositions, or utilizing plenty of patterns, or training with plenty of data.

Verification of SCADE Models. As SCADE DV cannot adequately express complex temporal specifications and it may fail due to complexity problems such as floating numbers, there are several related works to enhance the verification capability of SCADE. [12] transforms SCADE models into UPPAAL and verify the liveness properties in TCTL. However, it may fail when the original SCADE model contains hierarchical structure. [24] transforms SCADE models into nuSmv [6] models for verification, but limited to safety state machines and incapable of verifying infinte-state system. [3] introduces LAMA as an intermediate language into which SCADE programs can be transformed and which easily can be transformed into SMT solver instances. However, the method performs worse than DV.

8 Conclusion and Future Work

We propose an AGVTS method for automatically generating temporal specifications and verifying aeronautics SCADE models. AGVTS begins by defining a modular pattern language (MPL) to express Chinese requirements precisely. Then it uses a rule-based method augmented with BERT to translate requirements in MPL into LTL and CTL formulae. Finally it verifies SCADE models by transforming them into nuXmv which supports SMT-based and SAT-based verification. The method is applied to an ejection seat control system.

We are currently incorporating LLMs to enhance the capability of terms extraction and the process of parsing requirements. The patterns will be refined to cover more domains and enable users to articulate requirements with greater flexibility. Additionally, we will consider theorem prover Coq to prove the correctness of formula generation and the transformation from SCADE to nuXmv, based on our previous researches [27, 28]. Improving the verification capability of nuXmv is also an interesting work.

The details of patterns, translation algorithms of SCADE2nuXmv and appendix are provided at https://github.com/yayi-mei/AGVTS.