1 Introduction

From 2015 EU Member States have to comply with the Regulation EU 376/2014, [1], that requires the publication of an Annual Safety Review (ASR), a report of the occurrences related to safety, at least once a year.

The ASR is a tool to comprehend and display not only the state of the art of safety reporting and analysis but also how they change with time. It also deeply connects to the relevance that every single country has in European aviation in terms of number of airports and passengers, economically and historically.

The EU regulation, while clearly stating that the ASR is to be published at least once a year, see Article 13(11) of EU 376/2014, [1], leaves each state free to decide how to organise and make such document available to the public, and what data include, allowing every ASR to be personalized to reflect the peculiarities and features of each country. While this gives a reliable portrait of the local scenario, it complicates the possibility to compare effectively the situation among different countries, and share valuable data with EASA, which is required by EU Regulation 376/2014 to publish the EASA Annual Safety Review report, where information about the general safety situation is presented, in accordance with Article 72(7) EU 1139/2018, [2]. In this sense, a common standardised approach could facilitate the exchange of the information included in every SRs.

This paper aims to suggest a possible standardised approach to the ASR where the most important features can be presented following a template.

The safety reports of a set of countries are studied both in terms of how many documents are available online and how they are organised, thus giving the state of the art of the problem, then the common parts of the ASRs are analysed both qualitatively and quantitatively to understand what features to display in a standardised approach.

The period of time taken into account for the ASRs considered for this study goes from 2015 to 2021, which means that the data are from 2014 to 2020.

In no way is this paper intended to question the various approaches that exist regarding ASR publications. It presents a comparative analysis of the state of the art for safety reporting where only public information retrievable online has been taken into account, and the single cases included are exclusively studied to create a template that may facilitate the organization of future ASRs.

2 Safety Review Analysis at European Level

The data concerning the collected Safety Reviews are here presented taking into account the time period from 2015 to 2021. In general the analysis of the data regarding a certain year is published in the following year report to have the correct count of occurrences of the twelve month period.

Due to the public nature of these documents, only the reports available on the websites of Civil Aviation Authorities were considered: no documents were obtained privately or through other sources.

2.1 Data Table

A representative set of countries is considered for the analysis of the ASR scenario in Europe. Several factors are taken into account, such as the importance that the chosen countries have in the European aviation, their geographical location, and their aviation cultural background. Table 1 lists the twenty countries that make up the statistical sample, the corresponding number of available reports regarding the time period from 2015 to 2021 and how many years of data are presented in such reports. Only members of both the European Union and the European Union Aviation Safety Agency were considered.

It is useful to introduce the difference between the number of published documents, i.e. how many ASRs can be found online, and of available data, that is the number of years of analysed data included into a SR. There is not necessarily a one-to-one correspondence between the number of Safety Reports and how many years of data on Safety are included in those documents, mostly due to the following reasons:

  1. 1.

    A single ASR collects the analyses of data over a period of time that is longer than one year;

  2. 2.

    There is no obligation for data to remain public for a certain time, countries subject to mandatory publication can update their site annually without maintaining a track record about past documents;

  3. 3.

    The Safety Review is a web site which is periodically updated.

Table 1 The number of Safety Reviews currently available for a chosen set of countries
Full size table

Out of the twenty countries that have been studied, eight countries have published at least four reviews online during the range of seven years that has been taken into consideration. The number of published documents is generally not homogeneously distributed in a specific geographical area, for example among the Scandinavian countries—Sweden, Finland and Norway.

Some countries have published a single summary Safety Review for a specific time period, for example a three-year or two-year period. In this case, the number of years for which data are available is necessarily greater than the number of published documents. For example, in the historical record of the website of the Italian competent authority for the drafting and publication of the Annual Safety Review, [3], three different documents can be identified, with the titles: “SR2016”, [4], “SR2019”, [5], and “SR2020”, [6], and a website. The first, which refers to the year 2016, contains data for the two-year period 2014–2015. Within the second one, that of 2019, the available data range from 2015 to 2018, while the last document has data for the entire period from 2015 to 2019. From 2019 there has been a yearly publication of the SR that contains data from 2015, and from 2021 a website has been created, [3].

In the Maltese CAA website, [7], only one Safety Review, the most recent one, [8], can be found. This is probably due to the lack of a collection of older editions, as the 2020 document contains data for the 4-year period from 2017 to 2020. In fact, the obligation introduced by Regulation (EU) No 376/2014 concerns only the publication of the Annual Safety Review and does not constrain the period of time for which these documents must remain public. It is then possible to publish an updated version of the SR document every year, replacing the previous one. Other similar cases are Finland, [9], which only keeps public the two most recent documents, each of them covering only the year of interest, and the Netherlands, [10], which updates its data every quarter: two ways to favour the dynamism of its own data at the cost of a disadvantage in the study of their long-term trend.

3 Safety Review Structure

The quantitative comments of the previous section are here followed by the analysis of the general organization of Safety Reviews. In particular, the aim of this section is to identify a common basic structure among the studied documents. This common framework essentially consists of two partsFootnote 1:

  1. 1.

    Main Indicators: these are defined by one or more indices for which a first statistical analysis is carried out. Their main task is to make an initial subdivision of safety events according to their severity, and give an overview of the situation in certain aviation segments (e.g. business travel, general aviation, commercial transport,...). The most frequently used Main Indicators are Accidents and Serious Incidents. All the considered documents have at least this first part of statistical analysis;

  2. 2.

    Secondary Indicators: these provide a further subdivision of the recorded occurrences according to their categories. Also called “Safety Performance Indicators” (SPIs), they were introduced to reduce the subjectivity in the reports and to provide a form of standardisation of the data, making them comparable. These indicators allow individual occurrences and potential safety problems to be addressed according to standardised indices, in principle facilitating the analysis of different scenarios among countries. The number of occurrences contained in the Secondary Indicators is larger than in the Main Indicators. This as Secondary Events do not require a safety event (i.e. accident or serious incident) to happen. SPIs are criteria for gathering knowledge and analyses about safety situations, intended as potential indicators of safety events.

Another important feature in the structure of SRs is the choice of the unit of measurement.

Accidents and serious incidents are rare events nowadays, and that is why they are represented, in the majority of the considered cases, as pure numbers, i.e. not normalised with respect to the number of flights or movements, since the resulting number would be small and hard to read.

Normalised indices, instead, offer a way of easy comparison among countries, and this is the way in which the Secondary Indicators are often presented. The study of Secondary Indicators is not always carried out in the considered Safety Reviews. If it is present, the SPIs are usually used, and each country can decide which ones to study. While the definition of these indicators is regulated by European legislation, the member states are given complete freedom regarding the choice of unit of measurement or normalization. The indicators included in the SR and their units are the two major factors that make the generalization and coherence of the structure of the Safety Reviews more problematic, even at European level. In the following section an example of how a single indicator can be presented in many different ways is shown.

3.1 Index Comparison

The different choices regarding the indicators included in the statistical analysis of the Safety Review and their unit of measurement inhibit an easy comparison among SRs of different countries. This is caused by a varied and faceted presentation of the data. Studying one particular example facilitates exploring the range of possibilities given in different SRs, to find out what may be a suggestion for a common representation of such a case.

A specific SPI has been chosen for this: Table 2 shows the Runway Incursion (RI) data for five different European countries in 2019.Footnote 2 The definition of this index can be found in the ICAO document regarding Occurrence Categories, [11], “Aviation Occurrence Categories—Definitions and Usage Notes”: a Runway Incursion is ’any occurrence at an aerodrome involving the incorrect presence of an aircraft vehicle or person on the protected area of a surface designated for the landing and take-off aircraft’.

Table 2 2019 RI data for five European countries
Full size table

According to 2019 data there were 79 such occurrences reported in Finland during the year and in the Netherlands 76 occurrences happened, which were divided and represented in the digital portal of the Dutch authority for each month. Although these numbers are similar, they are effected by the two countries’ differences in airport traffic: in 2019 Amsterdam-Schiphol airport was twelfth in the world for total number of passengers. A similar consideration can be made when noticing that in Italy there were more occurrences (187) than in Finland (79) and the Netherlands (76). The representation of this indicator in terms of number of events is not effective for comparing it with other realities, since it does not take into account the global situation of the aviation in each country (number of movements, flights, passengers, airports, the culture of reporting, etc.).

An interesting case is that of Italy and Belgium: they opted for a different way of representing their data, but both countries have chosen to normalise their indices according to the same factor, the number of movements. In particular, the Italian authorities show the rate of RI for every 10,000 movements, while the Belgian authorities have done it for every 1,000. The fact that these are both rates with respect to the number of movements allows a first preliminary analysis between the two countries. The 0.055 occurrences rate per 1,000 movements in Belgium can be compared to the 1.01 per 10,000 in Italy if we use a same denominator. In this case, opting for normalizing to 1,000 movements, the Italian rate is 0.101 and it can be now comparatively analysed with the data declared by the Belgian authorities.

Another example worth exploring is the representation of CFIT (Controlled Flight Into Terrain), which refers to an event that occurs when an airworthy aircraft under the complete control of the pilot is inadvertently flown into terrain, water, or an obstacle, [11]. The pilot is generally unaware of the danger until it is too late.

Table 3 2019 CFIT index data for some European countries
Full size table

This index is analysed in different ways within the European scenario, as shown in Table 3. CFIT events occur rarely, in general, but with high severity, as it is also underlined in the French SR where in 2019 CFIT events were about 5 per cent of the total representing 42 per cent of the severity[12].

The severity of this index is also illustrated in the document published by the Romanian authorities, [13] in 2019 where 1 fatal accident and 2 accidents with injuries were attributable to this cause. However, the analysis of CFIT events is not present in some of the documents under study.

4 Results and Discussion

The study carried above shows the major differences and similarities of the European SRs scenario and leads to the draft of a first template that is organized so that the minimum demands required by the legislation can be easily complied and the comparison among different SRs is encouraged, while still allowing the singular authorities to personalize the annual review whenever the analysis of local peculiarities is meaningful. The cultural and economic diversity within the Member States of the European Union and EASA has a wide spectrum, so each country can compose its own document by adding all the elements it considers necessary or of interest, still respecting some identified common features.

The minimum prerequisites that build the proposed template are similar to the aims presented by EASA and the European Union in Regulation 376/2014: “[...] to improve aviation safety in the EU and globally by ensuring that relevant safety information relating to civil aviation is reported, collected, stored, protected, exchanged, disseminated and analysed”. Their compliance following a standardised approach to the presentation of the data analysis would allow a greater and more efficient control by the agency as well as an easier distribution and comparison of data between countries, also making more straightforward for less experienced users to find and read these documents.

The building blocks of the following template are the main and secondary indicators, and within each part the specifics are shown and discussed.

4.1 Main Indicator and Measurement Type

After the introductory phase of their document, most of the examined countries carry out an analysis of accidents and incidents within a given time period. This reference time span can be longer or shorter, some countries choose longer and not fixed periods (e.g. Norway, which has been studying accidents since 1946 with a moving mean every 5 years) while others decide to set a fixed reference span.

Generally even the subject of such study can vary from one country to another, some consider only accidents, others include serious accidents as well, and some also include the respective fatalities, as shown in Table 4.

As already mentioned in Sec. 3, this analysis is present in all the reviewed documents, the only difference is the way it is presented. Due to this, it can be assumed that all countries regularly collect data on such events, meaning that they should have no problem in complying with a standardized request.

To have a more complete analysis it would be possible to retain the results from the preceding years, allowing for the identification of data trends.

Table 4 Safety Review main indicators
Full size table

Once the type of events to analyse have been identified, it is crucial to choose their unit of measurement properly.

As shown in Table 5, there are several ways in which the main indicators are presented in the SRs. Accidents and serious incidents are rare events nowadays, some years even zero. The low number means that it would be possible to present and analyse them as individual events. Another effect of this is that rates with respect to the total number of movements or flights do not give a valid representation of these data, since the results can be difficult to interpret (i.e. very small numbers). The most efficient way to describe accidents and serious incidents therefore seems to be reporting the total number of events.

Table 5 Safety Review main indicators measurement type
Full size table

4.2 Secondary Indicators and Measurement Type

In general, the Secondary Indicators are the Safety Performance Indicators (SPIs), but no standard choice has been proposed for the units of measurement of these indices or which ones and how many of them should be included in the SRs.

SPIs describe incidents, occurrences where there is not necessarily a corresponding safety event (accidents or serious incidents). This means that the number of events that they analyse is large with respect to that of the Main Indicators, so normalised indices can be considered as the bast option to describe SPIs’ results. For the sake of a standardised approach, an appropriate normalisation of the SPIs is needed, so that it can be easily used by any member state, and that it can facilitate the comparison among data of different countries. As already seen in Sec. 3, Table 2 shows how an SPI (in this case RI) can exemplify some of the different approaches used in the SRs for their data representation.

Regarding the normalization, a valid option is to use the number of movements or of flights of the country of interest and consider the rate for 10,000 movements/flights (the choice between movements or flights depends on the definition of the considered SPI). This numerical choice is justified by the need to make the results more readable for any user interested in the SRs, since it is important that the data in the reports are numbers that are easy to interpret and compare. In should be noticed that the factor 10,000 works for any country that publishes a SR, and for any type of operation included in the report, but it would not be appropriate in the case of analyses that include large numbers of occurrences. For instance, when EASA creates the statistics for its Annual Safety Review, it deals with numbers of occurrences, movements and flights of all the member states. This makes the total number of data to analyse way larger than that of a single country (even by some orders of magnitude); therefore, a different choice in the numerical factor when calculating the rates may be required.

Another fundamental step in the process of defining a standardised template is the number of SPIs to be analysed and which ones should be included in the SR. Different countries have different safety culture and different operating conditions, which result into a wide variety of situations that are difficult to be described in a unique way. So far, no indication of which SPIs should be present in the SRs has been given, and, considered the complexity of the subject, a proper study of the issue may be conducted in another paper or by a dedicated working group, but it lies outside of this document’s purpose. Rather than a list of indices set a priori without a reliable knowledge of each country’s background, a more useful criterion could be a recommended number of indices to be taken into account to adequately describe the current safety situation in each state. The analysis of the considered SRs suggests that an appropriate number of indices for a standardised template could be between 8 and 10. Then, if needed or considered suitable, the agency responsible for a SR can always opt for a richer set of indices to characterise the safety situation of the respective country even further.

This ends the description of the main features of a standardised template than can be summarized in Table 6, where the suggested time interval of 5 years of results to present in a SR refers to the possibility to identify any trends in the analysed data.

Table 6 Proposed template for a Safety Review
Full size table

5 Conclusions

In accordance with Regulation (EU) No 376/2014, [1], EU member states and EASA must comply with the publication of a Safety Review at least once a year. To the present day the differences in the European Safety Reviews are significant, both in the type of the presented statistical analyses and how the reports are published. The structure of the report and the presentation of the results are not standardised and some countries have developed online portals where they can present their data in a clear and dynamic way, while others prefer pdf documents.

In the present paper, a first comparison among different SRs chosen in a certain period of time has been performed, while looking for the common features that could lay the foundations of a standardised approach in such reports. Our proposal highlights the key elements that a Safety Review should include to meet regulatory requirements and to provide a useful tool for states to track and monitor their safety status through the constant exchange of information at the European level.