1 Introduction

Helicopter ship deck landings are some of the most demanding tasks due to various factors such as the complex aerodynamic environment, the ship motions and the adverse weather and sea conditions. Factors such as degraded visibility, ship airwake and turbulence as well as precipitation increase the difficulty level of flying for the pilots in such unforeseeable circumstances. In addition, the pilots are often under pressure to land, no matter what the weather is, as there might not be an alternative landing place nearby. Therefore, the assessment of such a complex manoeuvre becomes important to reduce pilot workload and to expand the operational boundaries of the rotorcraft. Additionally, there is already a whole set of well-developed domains of research focusing on the development of robust controllers and pilot assistance systems to reduce pilot workload and to further expand the operational boundaries [1, 2].

There exist many evaluation methods for the stability, control and flying characteristics of rotorcraft mostly using Handling Qualities metrics, but most of the scales do not apply specifically for ship deck landings. US Army Aeronautical Design Standard-33 (ADS-33E-PRF) provides both predicted levels of Handling Qualities where the rotorcraft’s flying qualities parameters are determined and compared with the criteria limits appropriate to the operational requirements as well as assigned levels of Handling Qualities where the pilots use the Cooper–Harper Handling Qualities Rating Scale to assess the workload and task performance required to perform the designated Mission-Task-Elements (MTE) [3]. This kind of assessment requires either real helicopter flight tests or at least highly equipped simulation provisions to perform detailed investigations by the pilots [4]. Pilot rating scales such as Cooper–Harper Handling Qualities Rating scale, Bedford Workload, Deck Interface Pilot Effort Scale (DIPES) and NASA Task Load Index, etc. can be used to assess the task [4,5,6,7].

Moreover, such assessments require excessive testing preferably in real flights. Therefore, such simulations are excellent tools for designing and evaluating the pilot assistance systems and flight controllers because they are cost efficient, they provide a safe environment to test and train different scenarios which might not be possible in real flight [8, 9]. Although the final evaluation has to be done in the flight test, initial simulations are highly useful for training, design, certification and pilot aids assessment.

SHOL testing is conducted to determine the handling and performance limits of helicopter-ship launch and recovery operations [9, 10]. In most of the assessments, a subjective assessment is performed using different pilot rating scales like DIPES which are used in operational clearances and Cooper–Harper Handling Qualities Ratings and NASA Task Load Index which are mostly used in research activities. However, this subjective assessment is qualitative, involves human factors and is not sufficient. Therefore, we need an objective assessment of the simulation data such as an evaluation of the task performance to compare the subjective assessment accurately [11, 12]. The paper aims to bridge the gap to use objective data along with subjective assessment in such evaluations.

The current study presents an overall evaluation toolchain and demonstrates a detailed overall assessment of helicopter ship deck landings focusing on the quantitative evaluation of the ship deck landings along with a subjective assessment by the pilots. The focus of this paper is the evaluation toolchain, which is used for control system design performing the assessment of different control configurations. The configurations that have been tested using the toolchain during the current study are Attitude Command Attitude Hold (ACAH) and Translational Rate Command (TRC). These modes can be incredibly useful in reducing the workload of the crew. Hence, it is considered useful to test the improvements in safety and efficiency of the landing operation when using the control modes. The toolchain evaluates various touchdown conditions such as helicopter position, velocity and attitude relative to the ship deck and the helicopter for the quantitative evaluation. In addition, for a subjective assessment of the ship deck landings, pilots rated each landing task using different rating scales. The toolchain is verified based on a preliminary pilot study with test pilots in the Air Vehicle Simulator (AVES) at DLR Braunschweig [13]. This time-efficient toolchain provides a well-structured and comparable quantitative and subjective analysis readily available, once the pilot study has been performed in a simulation facility or even in a real flight. The goal is to be able to define and assess objective as well as subjective parameters to determine if a landing was well performed or not.

Section 2 provides the details of the simulation facility and maritime simulation environment used for the study. In Sect. 3, the specifications about the objective and subjective decision criteria for the mission success are provided. Section 4 describes the test scenario which was considered for the assessment. In Sect. 5, results of the pilot study are presented and Sect. 6 provides the conclusions of the study.

2 Simulation environment

The overall assessment of helicopter ship deck landings was performed in the AVES, the flight simulation research facility at the German Aerospace Center (DLR), Braunschweig.

2.1 The research flight simulation facility

The AVES is capable of representing one of the DLR’s research helicopter Active Control Technology/ Flying Helicopter Simulator (ACT/FHS) (a highly modified EC 135), an Airbus A320 fixed-wing transport aircraft cockpit and a single aisle passenger cabin. The cockpits are interchangeable with either a hexapod motion platform or a fixed base platform using a roll-on/roll-off system.

For the visual projection system, a total of 15 60 Hz Digital Light Processing-based LED projectors with a resolution of 1920 × 1200 are used for the visual systems in both the fixed and the motion-based platforms. The helicopter simulation features a dome-shaped design with a vertical field of view (FoV) of −58° to 35° and a horizontal FoV of 240 degrees. A detailed description of AVES features can be found in [13].

For the software architecture, a distributed approach is used. Different software modules can run either on a single computer or together on multiple computers via an ethernet connection. Processes like flight models and motion cueing run on real-time operating systems since they feature time critical calculations.

An OpenGL-based render engine, the Real-Time Image Generator (RTIG), is used as the image generation system. The object-oriented software is adaptable to new functionalities. The platform has multiple visual features such as 3D dynamic objects and various environmental effects like fog, brownout and realistic wave and water effects for maritime scenarios. The visuals are fully accessible and can be modified anytime according to the needs of the user.

The simulation environment including a nonlinear helicopter flight model with a model-based flight control system, an F124 class like frigate ship dynamic model with a simulation of the turbulent air wake, realistic wave and water effects and various weather and sea conditions was considered as the basis for Helicopter Ship Dynamic Interface [14]. A detailed description of the airwake implementation within the helicopter model and validation is described in [15] and [16]. The control design has been described in [17] and [18] in detail.

2.2 Ship motion and dynamics

The maritime simulation environment consisted of a German Navy F124 'Sachsen class' frigate like ship located in the German Bight. The integrated German Navy frigate class 3D ship model (as shown in Fig. 1) has a deck landing spot at the stern. To provide the best visual references to the helicopter pilots, a photorealistic texture as shown in Fig. 2 was added to the rear superstructure of the ship.

Fig. 1
figure 1

Maritime simulation environment in AVES with good visibility in Sea State 6

Full size image
Fig. 2
figure 2

Rear view on the F124 “Sachsen” class frigate

Full size image

Ship motions were calculated using the software VehicleControl, which features a 6DOF hydrodynamic ship model [14]. For this study, the ship motions were mainly considered in pitch and roll axes which demonstrate peaks and quiescent periods in the angular motion. In general, the helicopter pilots wait for those quiescent periods to perform the landing on the ship deck. The heave motion was disabled due to a missing synchronization between the wave image generator and VehicleControl. This led to waves occasionally flooding the deck at higher sea states, which was often commented on by pilots as it disturbed their immersion.

The motion of the ship was calculated with the International Towing Tank Conference (ITTC) wave spectrum, which spectral density for a wave frequency \(S\left(\omega \right)\) is defined in Eq. 1 with the significant wave height \({H}_{{\text{S}}}\) and the period of the average frequency \({T}_{1}\) [19].

$$S\left( \omega \right) = \frac{A}{\omega^5 }\exp \left( {\frac{ - B}{{\omega^4 }}} \right)\, {\text{with}}\, A = \frac{{172.8H_{\text{S}}^2 }}{T_1^4 } \,{\text{and }}\,B = \frac{691.2}{{T_1^4 }}$$
(1)

It should be noted that the used EC135 is not suited for landing on moving ships due to its rigid skid landing gear. Therefore, no limits exist for the maximum allowable pitch, roll and heave movements. Nevertheless, the EC135 is commonly used for other offshore operations such as wind farm maintenance or to transport maritime pilots due to a certification in performance class 1, its CAT A capability and available emergency floatation devices. Those features enable the helicopter to safely operate in the offshore environment even during engine failures or other emergencies.

2.3 Sea waves and weather

A scenario representing the wave and water effects was integrated into the RTIG software. It is possible to choose between four spectra (JONSWAP, Pierson-Moskowitz, ITTC and Phillips) to generate the wave effects in the sea [19,20,21,22]. The wave effects assist the representation of various other effects like ship wakes and the helicopter rotor downwash in the maritime scenario which further aids the pilots with attitude and height visual cues to achieve good situational awareness.

It is common to have a degraded visual environment (DVE) in the maritime environment due to extreme weather conditions such as fog and rain. The visibility can be configured via a fog and particle simulation in the RTIG software. Other conditions like clouds and precipitation are simulated using the Sundog Silver-lining SDK [23]. The visual environment that is used for the current study with the water waves and good visibility can be seen in Fig. 1.

2.4 Flight model

A nonlinear helicopter flight model based on DLR’s research helicopter ACT/FHS is considered for the simulation study. This model is based on the real-time simulation model SIMH [24]. It is an EC135 rigid-body helicopter model comprising a main rotor, tail rotor, fuselage, vertical, horizontal stabilizer, etc. A fully articulated main rotor model consisting of rigid blades is used with a hinge offset and spring restraint. The aerodynamic forces and moments are computed using the blade element theory. The tail rotor is modelled as a simple disk and a simple engine model is considered for the given flight model.

The nonlinear helicopter model comprises an interface for simulating local aerodynamic effects for external wind fields such as turbulence and air wakes [15]. Due to its flexible design, external wind fields can be attached to any object within the simulation environment (rotating wind turbines [15], dynamic ships [16], air-to-air refueling [25]. High fidelity methods such as computational fluid dynamics (CFD) have been used to calculate these external wind fields in advance. For example, in [16], the time-accurate wind velocities in the structured block behind the ship were computed with Detached Eddy Simulation. The instantaneous velocities for each timestep and each 0.3 m cell were exported from the CFD solver as a 30-s (750 solution) loop into the simulator. Then, these external wind fields are superimposed on 43 distributed Airload Computation Points (ACPs) of the helicopter flight model as shown in Fig. 3. The tail rotor ACP is not currently available in the model structure. Also, the interaction between the wind field and the tail rotor of the helicopter is not considered in this study.

Fig. 3
figure 3

Distribution of Airload Computation Points (ACPs) for the ACT/FHS in helicopter flight model

Full size image

3 Evaluation toolchain

The test scenario involved the deck of a German Navy F124 “Sachsen class” frigate proceeding at 10 kt in a sea state 6 with the waves approaching at a wind speed of 40 kt at 30° (green wind) off the ship’s bow. After completing the scenario design, a preliminary pilot study was performed in AVES at DLR Braunschweig. Further details about the scenario are provided in Sect. 4. Various ship deck landings were performed and during these manoeuvres the helicopter and the ship motion variables were recorded against the pilot control inputs. The pilot ratings were recorded using different scales for different phases of flight and the landing manoeuvre for the subjective evaluation. To assess the mission success quantitatively, the task performance is evaluated during the flight and at the touchdown point. A schematic of all the above-mentioned steps of the toolchain for the objective assessment of the helicopter ship deck landings is shown in Fig. 4.

Fig. 4
figure 4

Schematic of the toolchain for ship deck landing objective assessment

Full size image

3.1 Quantitative evaluation

For the quantitative analysis (see Fig. 4), the developed toolchain evaluates the touchdown conditions specifically the helicopter position offset (\(\Delta x\), \(\Delta y\)), velocity offset (\(\Delta {V}_{x}\), \(\Delta {V}_{y}\), \(\Delta {V}_{z}\)) and attitude offset (\(\Delta \Phi , \Delta \theta , \Delta \psi\)) relative to ship deck. The desired and adequate values are defined for each of the touchdown conditions based on the pilot feedback and the size of the helicopter and the ship. Further information for the selection of the desired and adequate values is detailed in Sect. 4.1. The quantitative mission success is determined depending on if the pilot could achieve the desired or adequate limits during landing. It is very important for the pilot to remain within the desired and adequate landing limits, to avoid a bad (positionally inaccurate), inclined or a hard landing (with high touchdown velocity).

Along with the touchdown evaluation, the objective criteria to evaluate the entire manoeuvre are also investigated such as levels of control activity (amplitude and frequency), lateral track (distance about the ship axis), position and track error (\(x\) and \(y\)), and heading variations during the whole flight trajectory. In this study, the approach phase is only observed and not evaluated. Hence, the approach phase does not become a part of the performance requirement for evaluating the success of the deck landing mission.

3.2 Subjective evaluation

To conduct the subjective evaluation, pilots rated each landing task using three different rating scales. It is very important to consider the subjective assessment by the pilot for evaluating the task along with the objective assessment because if the workload is unacceptably high as the pilot attempts to control the vehicle, it is more difficult for the helicopter pilot to achieve a safe landing on the deck of the ship. The pilot workload could vary completely based on different factors such as the presence of a pilot assistance display, various control modes, DVE, etc.

For providing qualitative ratings, pilots were informed about the desired and adequate values before awarding the ratings. However, the pilots awarded the ratings based on observation and subjective assessment because there were no objective displays available to judge the parameters or to judge their adequate/desired values. Although, to evaluate landing position, the pilots had optical references which were the ship markings, e.g. they were asked to land in the circle marked on the ship deck. For velocity, they could check the vertical speed indicator and judge accordingly.

The Cooper–Harper HQ rating scale was used to assess the handling qualities of the helicopter during the final phase of the landing manoeuvre [6]. The 10-point rating defines the best handling qualities of an aircraft as 1 “excellent, highly desirable” with minimal pilot compensation and the worst handling qualities as 10 “major deficiencies” with a loss of control. Another rating scale called DIPES which was used to quantify pilot workload of an average fleet pilot is designed specifically for ship deck landings [4]. Using this 5-point scale, the pilot rates each landing manoeuvre based on workload, performance, accuracy and consistency. The cause of the growing workload can also be provided with letter suffixes along with each rating (e.g. ‘D’ for deck motion or ‘V’ for visual cues). The pilot workload was also evaluated using the NASA Task Load Index (TLX) which is an overall workload score based on a weighted average of ratings on six subscales: Mental Demands, Physical Demands, Temporal Demands, own Performance, Effort and Frustration [7].

4 Test scenario

During the pilot study, the pilots performed multiple ship deck landings on a frigate class ship in a good visual environment (GVE). To not have any effects of a limited visual environment during the trials, a GVE was chosen. The sea conditions corresponded to sea state 6 with waves at a wind speed of 40 kt at 30° (green wind) with a significant wave height of \({H}_{{\text{S}}}=6{\text{m}}\) and a peak frequency of \({f}_{{\text{P}}}=0.53{\text{rad}}/s\) (Table 1). The resulting maximum motion is \({\varphi }_{{\text{max}}}=3.8^\circ\),\({\theta }_{{\text{max}}}=2.9^\circ\) and \(z_{{\text{max}}} = 3.4\,{\text{m}}\) as shown in Fig. 5. The environmental parameters were chosen to provide a scenario which challenges the pilots due to the strong winds and ship motion. As the approaches were conducted close to the ship where low visibility was expected to have no effect on the results, no visual degradation was included in the scenario.

Table 1 Sea state code from the World Meteorological Organization (WMO) [26]
Full size table
Fig. 5
figure 5

Roll and pitch ship motion for sea state 6 configuration

Full size image

Two helicopter pilots participated in the piloted simulation study. Both the pilots are doing research in the area of scientific flight testing and both of them are experienced with subjective pilot rating methods. Pilot 1 is qualified as an experimental test pilot, however, as Pilot 2 is doing research his experience is considered highly valuable for the evaluation. Unfortunately, none of the two pilots is experienced in helicopter ship deck landings but have been participating in many of the ship deck landing pilot studies in the simulator. Hence, their feedback is considered valuable for the study. The focus of this paper is the evaluation toolchain, which is used for the assessment of different control configurations. During the current study, ACAH and TRC control configurations were tested using the toolchain. In fact, these modes can be incredibly useful in reducing the workload of the crew. Hence, their experience is considered useful for testing the improvements in the safety and efficiency of the control modes. Various weather and sea conditions were also tested during the preparation phase with the pilots and one setting was selected for the simulation study. The feedback of the pilots was used to define the complete scenario including but not limited to approach types, flight manoeuvres, weather conditions, evaluation scales and performance limits.

4.1 MTE description

The mission task element (MTE) was based on the standard port lateral approach as defined in the HOSTAC (as shown in Fig. 6) [27]. The pilot starts the manoeuvre at a 65 ft altitude with a forward speed of 20 kt approximately 600 ft behind the ship and establishes a relative hover alongside the deck followed by a lateral transition maintaining the same barometric altitude over the deck. During the quiescent period, the pilot performs a vertical descent ending with a touchdown of the helicopter at the center of the ship deck [28]. The touchdown conditions like helicopter position, velocity and attitude relative to the ship deck are evaluated to assess the accuracy of the ship deck landing. The desired and adequate limits used in the preliminary simulation trials were taken from [9] and [11].

Fig. 6
figure 6

Deck Landing Mission Task Element [29]

Full size image

These defined limits were adapted and narrowed down based on the size of helicopter, ship and the pilot feedback during the simulator trials to make the task more challenging for the pilot. The selected desired and adequate limits that were used in the final piloted simulation tests are provided in Table 2. The pilots were provided the information about desired and adequate limits in the units that they see on the displays such as the rate of descent in ft/min so they can assess the parameters properly. However, all the parameters are provided here in the Standard International units.

Table 2 Desired and Adequate tolerances for landing
Full size table

4.2 Test configuration

As the main goal of the piloted simulation trials was to present the assessment of ship deck landing manoeuvre using the developed evaluation tool chain, the test matrix was designed for very specific conditions. The frigate class F124 ship moving with a speed of 10 kt was considered in sea state 6 with a wind over the deck of 40 kt at 30° (Table 1). The ship airwake and turbulence were also modelled along with the deck motion simulation [14]. Test points based on two different flight control configurations namely ACAH and TRC were considered for a specific sea state and level of turbulence.

The ship deck landing modes are embedded in a model-following controller architecture. The employed control design is a complete model-following control system, which imposes the desired command model dynamics on the controlled helicopter. With the ACAH command mode, the longitudinal and lateral stick deflections are proportional to aircraft pitch and roll attitudes. The yaw rate is commanded with the pedals and is combined with a direction hold (DH) and vertical speed is commanded using the collective stick and is combined with a height hold (HH). With the TRC mode, the stick deflections are proportional to longitudinal and lateral aircraft velocities. The yaw rate and vertical speed are commanded in the pedals and collective axis, combined with a DH and a HH, respectively.

5 Simulation results

The piloted simulations performed in AVES were used for the quantitative evaluation as well as to record a subjective pilot assessment. Two test pilots from DLR performed the ship deck landings. Only one pilot performed the tests at a time. The pilots had free flight time to get comfortable with the scenario and the flight control configurations before performing the actual tests. Both pilots evaluated the task independently from each other.

5.1 Quantitative assessment

For the quantitative evaluation, the touchdown conditions are evaluated to assess the accuracy of the landing manoeuvre. Along with the touchdown conditions, the approach phase is also investigated. In this study, the approach phase is only observed as additional information. Hence, the approach phase does not become a part of the performance requirement for evaluating the success of the deck landing mission. The desired and adequate limits for the touchdown conditions are provided in Sect. 4.1.

5.1.1 Touchdown conditions

To evaluate the touchdown conditions, mainly three metrics were evaluated: (1) position offset, (2) velocity offset and (3) attitude offset between the helicopter and the ship deck at the touchdown point. Table 3 provides position, velocity and attitude offsets during touchdown for both ACAH and TRC by both pilots. It can be seen in Table 3 that the pilots could achieve mostly desired position tolerances (in green) or at least adequate position tolerances (in orange) for the landing task. The desired and adequate tolerances for landing are provided in Table 2. It can be observed from Table 3 that both the pilots were only able to achieve an 'adequate' velocity tolerance (below 1.5 m/s (~ 5 ft/s)) for \(\Delta {V}_{x}\) for ACAH configuration. However, for TRC, the pilots could achieve the ‘desired’ performance with a touchdown velocity of less than 0.76 m/s (~ 2.5 ft/s).

Table 3 Position, Velocity and Attitude offset during touchdown with desired tolerances in green and adequate tolerances in orange
Full size table

To achieve the desired accelerations, a helicopter needs to tilt its orientation which can often lead to large attitude offsets. However, it can be seen in Table 3 that the pilots were able to achieve the desired landing position, while maintaining significantly small attitude offsets (mostly within the desired attitude tolerance of less than \(\pm\) 5° or at least within ‘adequate’ tolerance (\(\pm\) 10°)).

5.1.2 Entire manoeuvre

The exemplar positions of the helicopter relative to the center of the deck throughout the approach and during the landing phase can be seen in Figs. 7, 8, 9 and 10 for ACAH and TRC by Pilot 1 and Pilot 2 respectively. It can be seen that for all the cases the pilot was able to achieve a landing position inside the desired limits or at least inside the adequate limits.

Fig. 7
figure 7

Helicopter position relative to landing position throughout ACAH Approach (by Pilot 1)

Full size image
Fig. 8
figure 8

Helicopter position relative to landing position throughout TRC Approach (by Pilot 1)

Full size image
Fig. 9
figure 9

Helicopter position relative to landing position throughout ACAH Approach (by Pilot 2)

Full size image
Fig. 10
figure 10

Helicopter position relative to landing position throughout TRC Approach (by Pilot 2)

Full size image

The level of control activity for performing the manoeuvre is also considered as one of the metrics in the evaluation. The pilot control inputs for the approach and landing phase for ACAH and TRC approach are shown in Figs. 11 and 12 for Pilot 1 and Pilot 2 respectively. It can be seen that the pilot had to do relatively more efforts during the landing phase (solid lines) in comparison to approach (dotted lines) which is typical as the pilot has to do adjustments against the drift caused by turbulence which is also expected as the current implementation of the control modes did not include the effects of unknown dynamics and external disturbances. Additionally, both pilots mentioned that it was more comfortable to do the final precisions for the landing maneuver using the TRC when compared to ACAH. However, ACAH was more agile and would be less suitable for the landing maneuver. The results also show that ACAH resulted in slight reductions in the pilot input power in specifically collective and lateral directions. An increment in the pilot control activity could indicate an increase in workload and a larger control deflection demands a larger control stick deflection force [30].

Fig. 11
figure 11

Time Histories of pilot control activity throughout ACAH and TRC Approach (by Pilot 1)

Full size image
Fig. 12
figure 12

Time Histories of pilot control activity throughout ACAH and TRC Approach (by Pilot 2)

Full size image

5.2 Subjective assessment

For the subjective assessment, three different rating scales were recorded during the pilot study in the simulator. The Cooper Harper Handling Qualities Ratings (HQR), DIPES and NASA TLX were awarded by two pilots for the landing manoeuvre for the recorded landing maneuvers for which the touchdown data has been provided in Table 3.

Both the pilots remarked that the presence of the deck motion (in a high sea state) and the airwake turbulence were contributing to the causes of task being difficult and challenging. This experience is also depicted in the handling quality ratings in Fig. 13. The pilots could achieve the desired or adequate performance as provided in Table 3 with moderate compensation. As the pilots awarded the ratings based on subjective assessment and observation based on optical references on the ship for position and based on vertical speed indicator readings because there were no objective displays available to judge the parameters or to judge their adequate/desired values, a slight mismatch is visible between the quantitative assessment shown in Table 3 and the HQRs shown in Fig. 13. Indeed, the final HQRs awarded by the pilot is the subjective assessment based on the perceived performance. The pilots commented that the workload was reduced with TRC and they had to do less compensation to match the speed and could perform the landing with less effort when compared to ACAH. Additionally, the pilots have the level of effort, workload and control activity in mind while providing the subjective ratings as well as they provided each rating after performing each landing with a specific control configuration. The DIPES ratings are shown in Fig. 14. The DIPES ratings were 'acceptable' for all the test points for both pilots.

Fig. 13
figure 13

Cooper–Harper HQR results for deck landing manoeuvre using ACAH and TRC

Full size image
Fig. 14
figure 14

DIPES results for ship approaches using ACAH and TRC

Full size image

The pilots can also give one or more suffixes along with a DIPES rating to describe the cause of the increased workload. For the test points with both the control configurations with ACAH and TRC, Deck Motion (D) was always the factor that was causing increased workload for both pilots. Apart from deck motion, turbulence (T), lateral positioning (L) and height control (H) were some other factors stated by the pilots, which were also the cause of increased workload.

Furthermore, the higher ratings by Pilot 2 could be influenced by not only the task being demanding but also the level of experience of the evaluation pilot. This also highlights the difference between the factors stressed in both the rating scales. In general, both scales focus on task performance and pilot compensation, but the DIPES scale focuses more on the safe workload limits of a junior fleet/operational pilot. The test pilots had to consider if an operational pilot is capable of performing a safe deck landing or not. The DIPES scale also obligates the test pilot to consider the possibility of adequate spare capacity for conducting ancillary tasks.

As shown in Figs. 13 and 14, ratings illustrate that the TRC configuration reduced the overall pilot workload during the landing task. Pilots also commented that it took overall relatively less effort with TRC when compared to ACAH to perform the landing task. The pilot assessment of all the six subscales of the NASA TLX score is shown in Fig. 15. It can also be seen in Fig. 15 that the temporal demand and frustration were lesser with TRC as it was more comfortable for the pilots to use this mode to perform final precisions for the landing maneuver. Additionally, the pilots mentioned that it was physically more challenging and more effortful during the approach because they had to give bigger stick inputs with TRC as it had a rather slow responsive nature in contrast to ACAH mode which is known to be agile. However, this could be because the task was defined in a way that the whole manoeuvre of approach and landing was performed using only one control configuration. However, the overall scores for both modes were not very far from each other.

Fig. 15
figure 15

NASA TLX Individual Score

Full size image

6 Conclusions

An evaluation toolchain framework that provides a structured approach for the assessment of the success of helicopter ship deck landings is proposed in this paper. A maritime simulation environment consisting of a non-linear helicopter flight model with a model-based flight control system, a ship model with motion and turbulent airwake based on ship attributes and sea states, wave and water effects and several weather and sea conditions is presented. The whole scenario is tested based on a pilot study with the test pilots for evaluating the success of helicopter ship deck landing. The mission success is determined by mainly two factors, the task performance and the pilot workload. For the objective assessment, touchdown conditions such as helicopter position, velocity and attitude relative to the ship deck are evaluated and for the subjective assessment, the pilot effort and workload are evaluated using different rating scales.

The presented example of the complete assessment of the simulated ship deck landing provides a comparable quantitative and subjective analysis based on well-structured methodology readily available, once the pilot study has been performed in a simulation facility or even in a real flight. The overall assessment by the toolchain would be useful to perform comparisons based on different weather conditions, various control configurations or in the presence of other pilot assistance systems.

The current evaluation was focused only on the landing maneuver, therefore, in the future it can be extended to different phases of flight including approach, lateral transition over the ship, hover in addition to touchdown. Even the HQRs can be awarded for different phases of flight. Moreover, in the current pilot study, the pilots did not use any objective displays for awarding HQRs, this can also be a possible improvement that can be included in future work.