Remote AFIS: development and validation of low-cost remote tower concepts for uncontrolled aerodromes

Abstract

Remote tower systems are widely established as a means to provide efficient air traffic control (ATC) from a remote location. However, even these cost-effective systems cause procurement, implementation, and maintenance costs, which make them unaffordable for non-ATC aerodromes with low revenues, often only offering an aerodrome flight information service (AFIS) or UNICOM information service. In this paper, two more inexpensive concepts enabling remote control at these aerodromes are presented. They are based on a simplified camera set-up comprising a pan-tilt-zoom-camera (PTZ-camera) and a simple panoramic camera. A virtual reality-headset (VR-headset) is used to display the video streams and to control the PTZ-camera. The results of a validation study with nine ATC and AFIS officers using live traffic at the Braunschweig-Wolfsburg aerodrome (BWE) are presented. They are discussed with respect to perceived usability, virtual reality induced cybersickness, and operator acceptance. The system’s cost is compared to that of a conventional remote tower set-up. Furthermore, measured objective data in the form of angular head rotation velocities are presented and requirements for the camera set-up are deduced. In conclusion, the developed concept utilizing both the panorama camera and PTZ-camera was found to be sufficiently usable and accepted by the validation participants. In contrast, the concept based only on the PTZ-camera suffered from jerky and delayed camera movements leading to considerable cybersickness and making it barely usable.

1 Introduction

Driven by the need for a more cost-efficient aerodrome air traffic control (ATC) and a shortage of air traffic controllers (ATCOs), optical camera systems that enable remote control of aerodromes were first proposed in 2002. These remote tower systems allow for the control of multiple aerodromes from one remote tower centre and an on-demand allocation of personnel. Furthermore, the replacement of the out-of-the-window (OTW) view by a video panorama enables sophisticated automations—such as object recognition—that support the ATCO and the addition of an infrared panorama raising the safety in adverse weather conditions.

While controlled aerodromes were the primary target of past remote tower research, the benefits also extend to non-ATC (uncontrolled) aerodromes offering an aerodrome flight information service (AFIS) or no certified air traffic control services (UNICOM). The possible use-cases include sharing the costs for providing AFIS or UNICOM or dynamically extending the opening hours without the need for on-site personnel. Furthermore, higher level ATC services could be provided on demand. However, the costs of implementing, running, and maintaining state of the art remote tower systems exceed the financial capabilities of non-ATC airports.

The aim of the study reported in this paper therefore is the development and validation of a low-cost Remote Tower solution specifically tailored towards the requirements of non-ATC airports.

Building on a simplified camera set-up consisting of a 180° panoramic camera and a pan-tilt-zoom (PTZ) camera, three remote AFIS concepts are developed. Two of these concepts build on a novel visualization and interaction method using a head-mounted display (HMD), also referred to as virtual reality (VR)-headset, tracking data of which are used to position the PTZ-camera.

In parallel, interaction concepts utilizing the controllers supplied with the headset and a Leap Motion hand tracker are developed and presented in [2].

2 Theoretical background

2.1 AFIS and UNICOM driven system performance requirements

In contrast to ATC services, AFIS and UNICOM are not designed to give the pilots concrete routing clearances nor to are in charge of traffic separation. Instead, their purpose is to provide the pilots with all necessary information so that they are in the position to safely navigate the airspace on their own and to self-separate from other traffic by see and avoid procedures. Further, the traffic load and its complexity might be lower in average compared to ATC operated aerodromes. Therefore, operationally driven performance requirements on optical surveillance systems for provision of AFIS and UNICOM are possibly lower than for provision of ATC.

Particular performance requirements which might be less strict in an AFIS or UNICOM environment are the detection range, tracking performance, automatic PTZ following function, use of the system in ambient light conditions, and mean time to repair. More generally, lower acceptance margins for the overall quality of the video in terms of display resolution, sensor resolution, field of view, contrast, video update rate, colour depth, video compression/CODEC, network latency, jitter, noise, packet loss, and image uniformity are possible.

This situation is also reflected by EUROCAE in its minimum aviation system performance standard for remote tower optical systems (ED-240A) [3]: apart from some fixed minimum performance requirements, most of the core requirements are user driven, aerodrome- and use case specific. They heavily depend on the type of service provision, airspace classification, and the use of visual observation as a part of decision-making. The user specifies what must be seen and where to provide the required service.

These operational performance requirements drive the technical performance required from the optical surveillance system, which in turn affects its design: the number and siting of the cameras, their resolution, sensitivity, noise and frame rate, the bandwidth and latency of the network, the quality of the operator’s displays etc. All of these are design considerations which directly influence the cost of the system so that lower operational performance requirements could in turn make it less expensive and, ultimately, affordable for non-ATC aerodromes.

2.2 Cybersickness

Virtual and augmented reality devices—such as the VR headset used in the Remote AFIS concept—are known to cause a special form of motion sickness, called “cybersickness”. The term describes ‘an unintended psychophysical response to exposure to the perceptual illusions of virtual environments (VEs)’ [4].

The typical symptoms of cybersickness include nausea, cold sweats, vomiting, dizziness, headache, and fatigue. The users of VR devices often also experience oculomotor symptoms—like eyestrain or difficulty focussing—due to the close proximity of the headset’s screens to the eyes [5].

Based on studies about simulator and motion sickness, cybersickness is believed to degrade the user’s performance. More severe symptoms also pose health and safety risks, especially since they can last long beyond the exposure itself [4,5,6].

In contrast to other forms of motion sickness, cybersickness is visually induced, meaning that subjects experience symptoms from what they see, not from the motion they feel. Therefore, the symptoms can be avoided entirely by the subject closing their eyes [4]. There currently is no agreed-on theory on the cause of cybersickness. However, the most common explanations are that of a “sensory mismatch” between the eye and the vestibular system [7] and a “postural instability” because the user is forced into an unstable posture by the VR-stimulus [8].

Since the exact cause of cybersickness is unknown and due to the polysymptomatic nature of it, there is no singular measure to reduce the severity of symptoms. Instead, different measures have proven to be effective for different VR systems and tasks. The measures suggested in literature include a precise calibration of the headset, seated use, limitation of the duration of exposure, prior training, more user control, and slowing down the displayed movements. Furthermore, properties of the used HMD were found to influence the induced cybersickness. As an example, reducing the field of view also reduced the cybersickness experienced by participants. In addition, users habituate to the VE, reducing the cybersickness after multiple exposures [4, 9,10,11,12].

There generally are three approaches to detect cybersickness: subjective questionnaires, observing the postural instability, and measuring the biometric and physiological state. As they are quick and easy to administer and require no specialized equipment, subjective questionnaires are used in the study described in this paper. The primary choice of questionnaire is the simulator sickness questionnaire (SSQ) [13] originally created for flight simulators. In recent years, the virtual reality sickness questionnaire (VRSQ) [14] and the cybersickness questionnaire (CSQ) [15] were proposed as questionnaires specifically designed to measure cybersickness. They aim to solve issues of the SSQ—like over-sensitivity and generalization—but are yet to be widely validated.

3 Design

In accordance with the principles laid out in Sect. 2.1, the operational requirements for the provision of AFIS or UNICOM services were determined at the start of the design phase by performing a site survey at the Schönhagen AFIS aerodrome and interviewing AFIS officers (AFISOs). During the site survey and interviews, it was discovered that the AFISO seems to be a very limited resource. Often they have secondary tasks that sometimes even require them to leave the tower, limiting their ability to maintain a continuous watch on the aerodrome and traffic and thus to provide a qualitatively and quantitatively high service level. The following three specific use-cases for the Remote AFIS concept were derived, based on which the camera set-up described in Sect. 3.1 was designed:

• 1. Continuous or extended provision of AFIS services from a centralised remote location, where several AFIS aerodromes are connected.

• 2. Ad-hoc provision of temporary AFIS services from the airport site in situations where the AFISO is tasked with secondary work (e.g. billing) and cannot be on the tower.

• 3. On-demand provision of temporary AFIS or even ATC services from a remote location on request by a pilot.

3.1 Camera set-up

Generally, every remote tower must provide two core optical surveillance functions: a panoramic view of the aerodrome’s surroundings (replacing the OTW view) and the ability to zoom into areas of interest (replacing binoculars). In the most basic form, both functions can be provided by a single PTZ-camera. Typical remote tower optical systems also include a static 360° panorama captured with multiple, circularly arranged cameras to improve on the core surveillance functions. The panorama allows the operator to instantly see every part of the field of view (FOV) without moving parts and enables compressing a wide FOV (e.g. 200°) into an easily glanceable area.

Based on the operational requirements collected during the site survey, the most basic set-up (i.e. only a PTZ-camera) could already be sufficient to provide AFIS or UNICOM and thus forms the basis of the concepts developed herein.

Mirroring the previously described remote tower system, the PTZ-camera is supplemented with a simple panorama that is captured by a low-cost, off-the-shelf 180° panoramic camera. This addition widens the design space for concepts so that both, a minimalistic remote surveillance solution and a simplified variant of the current set-up can be evaluated.

In preparation of the evaluation experiment, the specified camera system is erected at the Braunschweig-Wolfsburg aerodrome. As depicted in Fig. 1, the camera set-up is located on the roof of a six-story building next to the aerodrome’s perimeter that is approximately as high and far from the runway (about 380 m) as the tower is. The panoramic camera is oriented so that the runway, apron, and departure and arrival sectors are covered by its field of view.

Fig. 1

Location of camera set-up at BWE and horizontal field of view (HFOV) of the cameras

Due to the absence of specific requirements for AFIS or UNICOM optical surveillance systems, the cameras’ specifications are deduced from the standards for remote tower solutions and the operational requirements. The latter of which requires that the PTZ-camera is capable of rotating as fast as the user is able to turn their head as this is considered as method to position the camera (see Sect. 3.3). Concrete data on how fast operators usually turn their head is scarce, however. The values found for rotation speeds while using an HMD in general are up to 120°/s for heavy and above 300°/s for light VR headsets [16] opposed to 60°/s stated in the ED-240A [3]. These values are therefore used as guideline for the selection of the PTZ-camera.

As Head-Mounted Device, an HTC Vive Pro Eye was chosen, the angular resolution of which equals that of the panoramic camera ensuring that all captured image data is displayed. The resulting specifications of the two cameras and the VR headset used are listed in Table 1.

Table 1 Main specifications of the cameras and the HMD [17, 18]

3.2 Cost comparison with typical remote tower optical system

Compared to a typical remote tower optical system—multiple, circularly arranged TV-cameras and one PTZ-camera—the camera set-up presented in this paper enables considerable cost savings in implementing and operating the system. The components used in the proposed set-up are all commercial-off-the-shelf products and cost approximately 10,000 Euros for the PTZ & Panorama concept (i.e. one PTZ-camera, two panoramic cameras for 360° coverage, one HMD and the computer required for the processing) or about 7000 Euros for the PTZ only concept (excluding the panoramic cameras) opposed to at least 14 times as much for a comparable remote tower optical system. Further cost savings are possible due to the lower bandwidth requirements for the transmission of the video feeds. On the end of the controller working position, a big multi-monitor set-up is replaced by a less expensive HMD that can be integrated into confined spaces while also offering portability and flexibility.

3.3 Concepts

For the visualization of the video data recorded by the camera set-up, a baseline and two different concepts are developed. They are designed to cover a preferably large area of the design space utilizing both standard computer monitors and the HMD as well as the complete camera set-up (PTZ-camera and panorama camera) and the minimalistic set-up (PTZ-camera).

3.3.1 Baseline

Building on the state-of-the-art display solutions of current remote tower optical systems, two standard computer monitors are used in the first concept. On one monitor, the video stream of the panoramic camera is displayed. The other monitor shows the video stream of the PTZ-camera, which the operator is able to control with a standard computer mouse. Since this concept is an adaption of a remote tower for the reduced camera set-up, it is regarded as baseline in the context of this paper.

Because of the lower operational requirements of AFIS and UNICOM compared to ATC, novel visualization approaches that are tailored towards the work environment may also be acceptable. The second use-case (see Sect. 3) specifically requires the visualization of the video feeds on a compact and portable device also integrating all necessary controls, allowing the operator to carry it with them when not on the tower. As option to fulfil these requirements, two further concepts utilizing a Head-Mounted Device are developed.

3.3.2 Concept 1 (Panorama & PTZ)

The first concept utilizes both cameras (panorama & PTZ-camera) available in the set-up. In analogy to the view from an actual tower, the video stream of the panoramic camera is projected around the user onto a curved shape. This projection also corrects the skew that occurs when displaying the stream on a standard computer monitor (see Fig. 2).

Fig. 2

Frame of the panoramic camera’s video stream

Through the VR-headset, the part that matches its FOV and orientation is visible. As on the tower, head movements are required to view the entire panorama and are translated instantaneously since no mechanical rotations are necessary.

Replicating the use of binoculars, the video stream of the PTZ-camera is overlaid on the currently viewed area by a user action (compare Fig. 3). The camera is then automatically adjusted to the orientation of the HMD, but remains decoupled from the user’s head turns, allowing the operator to freely look around while the video stream is fixed in the panorama.

Fig. 3

Screenshot of concept 1

3.3.3 Concept 2 (PTZ only)

The second concept builds on the minimalistic camera set-up and uses only the video feed of the PTZ-camera. The camera is directly coupled with the VR-headset so that its orientation is constantly adjusted to match the HMD’s orientation. The video stream stretches the entire FOV of the operator as is pictured in Fig. 4.

Fig. 4

Screenshot of concept 2

4 Validation study

4.1 Research questions

In a first step, the suitability of the chosen camera set-up (compare Sect. 3.1) as basis for a low-cost remote tower solution is assessed. If the set-up is suitable, the second step then is to determine the acceptability of a head-mounted device as a way to display the video streams and to control the PTZ-camera. The study aims to answer three research questions. The research questions consequently are:

• 1. Is a low-cost optical remote tower system sufficient to provide AFIS or UNICOM for non-ATC airports?

• 2. Is an HMD in general, and one of the developed concepts (PTZ & Panorama or PTZ only) in particular, acceptable as a way to display the video streams and to control the PTZ-camera?. In this context, each concept is regarded as one unity as the visualization of the video streams and the PTZ-camera’s control are designed in conjunction with each other and cannot be separated.

• 3. What angular rotation velocity is required for the PTZ-camera to directly follow the operator’s head rotation at any zoom level?

4.2 Sample

The experiment took place between the 28th October and 5th November 2020 at the DLR Institute of Flight Guidance in Brunswick. In total, nine ATCOs and two AFISOs from five German aerodromes, two of which are non-ATC aerodromes targeted by the remote AFIS concept, participated in the study. Eight of the operators were male, one was female. Except for one ATCO, all held a valid license. While nine participants is a relatively small number of participants for human factors studies, the general shortage of controllers and work schedule restrictions considerably limit the number of participants that can be recruited for remote tower related studies.

4.3 Independent variables

The validation study is designed as within-subject experiment in which the independent variable (IV) “Visualization” is manipulated. The different factors of this variable are the baseline and the two developed concepts described in Sect. 3.3. They are denoted as A₁ through A₃ as follows:

IV A : Visualisation

IV A₁: Baseline (Displays)

IV A₂: Concept 1 (VR, PTZ & Panorama)

IV A₃: Concept 2 (VR, PTZ only)

4.4 Dependent variables

During the evaluation, performance data—including the framerate and angular acceleration of the VR headset—for IV A₂ and A₃ were collected. The participants’ subjective evaluation was furthermore assessed using standard questionnaires and tailor-made questions. In this paper, four subjective metrics—the system usability scale, the simulator sickness questionnaire, and two tailor-made questions—as well as the measured angular head rotation velocity are presented.

4.4.1 System usability scale

The system usability scale (SUS) [19] is a commonly used standard questionnaire to assess a system’s usability. It consists of 10 questions which are rated on a five-point Likert scale from 1 “strongly disagree” to 5 “strongly agree”. The participant’s rating of the items is converted to a score between 0 and 100. As basis for a classification, the rating scale developed by Bangor, Kortum and Miller [20] is used.

4.4.2 Simulator sickness questionnaire

The Simulator Sickness Questionnaire [13] is utilized to assess the severity of the cybersickness symptoms that result from using a VR-headset. It is comprised of 16 symptoms whose severity is rated on a four-point Likert scale from 0 “none” to 3 “severe”. From the evaluation, a “Total Score” (TS), providing an overall indication of sickness criticality, and three subscale scores—“Nausea” (N), “Oculomotor” (O), and “Disorientation” (D)—that enable a detailed analysis of the symptoms’ origin, are calculated. The symptoms associated with each subscale are listed in Table 2. According to literature, a score above 30 in any scale is critical for a virtual environment [21].

Table 2 SSQ subscales’ symptoms and multiplication factors [13]

4.4.3 Tailor-made questions

In addition to the standard questionnaires, specific questions to evaluate the operators’ acceptance of the solutions are designed. In this context, two aspects are considered. First, the operator must be able to comfortably work with the solution for at least as long as is required by their task. Considering the use cases outlined in Sect. 3, the simplest task is the temporary supervision of the traffic at an aerodrome in order to guide a single arrival or departure. It is assumed that this will require 15–20 min of worktime, including time to accommodate oneself with the traffic situation. To evaluate this, the participants were asked to estimate how long they could continuously work with each of the three concepts. The possible values were capped at eight hours, equalling the length of a full workday.

The second aspect is the preference of the individual solutions. Hence, the participants were asked to order the concepts according to which they most preferred.

4.4.4 Angular head rotation velocity

In order to aid in the future development of the Remote AFIS concept, the angular velocities in degrees per second measured by the build-in sensors of the HTC Vive Pro Eye were logged. Of the recorded values around all three movement axes, the pan (right–left movement) and tilt (up–down movement) velocities were evaluated and are presented in this paper.

4.5 Apparatus & testbed

The baseline and the two developed concepts were installed on the same workstation equipped with two 27-inch monitors with a resolution of 2560 by 1440 pixels, depicted in Fig. 5. A wireless computer mouse was provided to control the PTZ-camera’s movements in all three concepts. For the evaluation, the live streams of the panoramic camera and the PTZ-camera were used. During the entire study, there was no significant downfall and visibility of at least 6000 m. The radio communication with the aerodromes’ tower was played in real time.

Fig. 5

Set-up of testbed

4.6 Procedure

At the start of the study, all participants were briefed on the project’s motivation and the three developed concepts. They were also asked if they feel well enough to participate in the evaluation which is a precondition for the SSQ to produce valid results. The participants were given 10 min for each concept to accompany themselves with the PTZ-camera’s controls and the concept as a whole. Subsequently, they passively observed the live traffic at the Braunschweig-Wolfsburg aerodrome for 30 min with each concept one after another. The order in which the concepts were evaluated was randomized between participants to minimize learning and fatigue effects that systematically influence the results. As measures to reduce the experienced cybersickness, the participants were instructed to adjust the headset to their comfort and remained seated during the study. Both of these, as well as the prior training with the concepts, are suggested by literature [4, 9,10,11,12]. Some of the recommended measures would, however, have required changes to the concepts themselves and therefore were not implemented. It was assumed that, due to the lack of overview, the participants would experience the severest cybersickness while using the PTZ only concept and special attention was placed on their wellbeing while using the concept.

After each concept run, the participants filled out the Simulator Sickness Questionnaire and estimated the duration for which they could work with that visualisation condition. The remaining questionnaires (one SUS each for the baseline and the two concepts and the ranking of the concepts) were answered after all concepts have been used. At the end of the evaluation, the participants were debriefed and had time to make any comments or remarks that have not been covered previously.

All participants gave their explicit consent to participate in the study and received no compensation.

5 Results

Due to the small number of participants, inferential statistics, even non-parametric statistics, have only limited meaningfulness. In this paper, the data are therefore presented and discussed using descriptive statistics [22].

5.1 Usability

5.1.1 System usability scale

The SUS scores of the eight participants for the two concepts and the baseline are presented in Fig. 6. In the right part of the figure, the rating scale developed by Bangor, Kortum and Miller [20] is depicted.

Fig. 6

SUS score for n = 9 participants and rating scale from [20]

For the baseline, the median usability score is 90.0. This value lies between the “excellent” and “best imaginable” usability ratings. There is one outlier—defined as being more than 1.5 times the interquartile distance below the first quartile (i.e. the bottom end of the box)—at a score of 27.5. Overall, the participants’ SUS scores range from 22.5 to 97.5. The median rating for IV A₂ “VR PTZ & Panorama” is 75.0 with a range between 22.5 and 95.0. For IV A₃, the median is 65.0. The lowest evaluation is 25.0 and the highest evaluation 95.0. Overall, all scores above the first quartile are higher than 52.0 which is the limit to “OK” usability according to the rating scale.

It can be noticed that all the minimum SUS scores lie around the “worst imaginable” rating and differ greatly from the remaining scores. All of these ratings were given by the same participant who also expressed concern over remote tower optical systems in general during the study. Similar observations were made for the other subjective metrics. It was therefore concluded that it is likely that the questionnaire results are influenced by a personal bias and the subjective results subsequently presented are calculated for the remaining eight participants.

5.2 Cybersickness

5.2.1 Simulator sickness questionnaire

The aggregated scores of the Simulator Sickness Questionnaire are presented in Fig. 7 for the PTZ & Panorama and PTZ only concepts. Due to incomplete data, only the evaluation of five participants is shown. The critical SSQ score of 30 (compare Sect. 4.4) is marked. During the use of the baseline concept, the participants experienced no symptoms of cybersickness, resulting in a score of 0.00 in all four scales. For better legibility, the baseline is therefore not included in the figure. The marked outliers are defined as being more than 1.5 times the interquartile distance from the third quartile as postulated by Tukey [23].

Fig. 7

SSQ score for concepts 1 and 2 for n = 5 participants; median is emphasized; SSQ score for baseline is 0.00 in all subscales

In the nausea subscale, the majority of scores for both concepts lie between 0.00 and 9.54. There is one outlier at a score of 57.24 for the PTZ only concept. The median scores are 0.00 and 9.54 for IV A₂ and A₃ concept, respectively.

The SSQ scores in the oculomotor subscale are higher. For the PTZ & Panorama concept, they range from 7.58 to 15.16. The values for the other VR concept lie between 22.74 and 37.90 with an outlier at 7.58. The former concept’s median is at a score of 7.58 while that of the latter concept is 22.74.

In the disorientation subscale, the only non-zero scores are calculated for IV A₃. The spread is the largest of the three subscales and the total score, ranging from a score of 0.00 to a score of 41.76. The median is 27.84.

The total score for IV A₂ “VR PTZ & Panorama” is rated at 3.74 by one participant, while the others rate it at 7.48. For IV A₃ “PTZ only”, the total score ranges between 14.96 and 48.62 with the median being at 18.70.

In summary, the maximum SSQ score for the first VR concept is 15.16 in the oculomotor subscale, which is below the critical score of 30. The total score of the second VR concept and the evaluations in the oculomotor and disorientation subscales partially exceed this value.

5.3 Acceptance

5.3.1 Maximum possible worktime

In Fig. 8, the participants’ estimate of how long they would be able to continuously work with the three concepts is shown.

Fig. 8

Maximum possible worktime with concepts for n = 8 participants

For the baseline, the highest estimates were given. They range between 240 and 480 min with one participant stating a possible worktime of 30 min. The median value is 300 min. The variance for the PTZ & Panorama concept is smaller, stretching from 15 to 120 min. The median estimate is 75 min. Lastly, the PTZ only concept’s median value is 30 min with the lowest estimate being 7 min and the highest being 60 min.

5.3.2 Ranking

The participants’ ranking of the concepts is summarized in Fig. 9. The most preferred concept is displayed at the top of the figure, while the lowest ranked is at the bottom.

Fig. 9

Ranking of concepts for n = 8 participants

The baseline is the most preferred concept of four of the participants. For the other half, it is the second most preferred concept.

The PTZ & Panorama concept is favoured by three of the operators. For additional four of them, it was the second-best concept. It was rated the least preferred concept by one participant.

The PTZ only concept was rated the best by one study participant. The others rated it on the last rank.

5.4 Angular rotation velocity

For all nine participants, 180,281 velocity datapoints were recorded for IV A₂ “VR PTZ & Panorama” covering a combined 15,213 s (253.6 min) of usage. For the PTZ only concept, 733,561 datapoints over 16,316 s (271.9 min) were logged. The recorded angular velocities were divided into the pan and the tilt movement. Then, the duration during which the angular velocity was at or below thresholds set in 5°/s steps (5°/s, 10°/s, 15°/s, …) was determined for the pan and tilt movements separately. A linear interpolation was used between the data points. As the angular velocities below the first threshold (5°/s) are expected to mostly stem from unintentional movements, it was discarded for the evaluation. For the remaining thresholds, the determined duration’s proportion of the total duration was calculated.

Thereafter, the pan movements cover 3223 s (PTZ & Panorama concept) and 2405 s (PTZ only concept) with velocities above 5°/s. The tilt movements cover 1198 s and 948 s, respectively.

5.4.1 Pan movement

In Fig. 10, the calculated proportion of head rotations performed by the participants slower than the pan velocity thresholds are depicted as thin lightly coloured lines. The two thick lines denote the average proportion over all participants for the PTZ & Panorama and PTZ only concepts. As the HMD was not used for the baseline concept, no pan velocities were recorded for IV A₁ “Displays”. In the right part of the figure, a close-up of proportions for thresholds of 140°/s and higher is presented. Furthermore, the maximum rotation velocity of the PTZ-camera (140°/s) is marked.

Fig. 10

Proportion of horizontal head movements slower than pan velocity thresholds, close-up of pan velocities from 140°/s on the right

With the PTZ & Panorama concept, the majority of participants’ horizonal head movements (55.55%, SD = 4.9) are performed with angular velocities up to 10°/s. With higher pan velocities, the slope constantly deceases. The highest pan velocity for one participant was measured to be 415°/s, the maximum velocity of the other six participants did not exceed 295°/s. At the maximum angular velocity of the PTZ-camera, 99.41% (SD = 0.31) of the participants’ overall horizontal head rotations can be directly followed.

For head rotations with the PTZ only concept, the pan velocities are lower overall. At 10°/s, 66.90% (SD = 9.96) of horizontal head movements performed by the participants are covered. The general shape of the curve is equal to the first concept and the maximum velocity is reached at 295°/s. Compared to the first concept, a higher percentage of head movements, 99.80% (SD = 0.27), is performed with pan velocities up to the maximum angular velocity of the PTZ-camera.

5.4.2 Tilt movement

Identical to how the pan movements are depicted in Fig. 10 the proportions for the tilt movements are presented in Fig. 11.

Fig. 11

Proportion of vertical head movements slower than tilt velocity thresholds, close-up of tilt velocities from 140°/s on the right

Compared to the pan movements, the tilt movements are slower in general. For IV A₂ “PTZ & Panorama”, 80.02% (SD = 6.23) of movements are performed with up to 10°/s. The maximum tilt velocity measured is 175°/s. At the maximum velocity of the PTZ-camera, 99.95% (SD = 0.10) of tilt movements are covered.

With IV A₃, 74.57% (SD = 5.56) of tilt movements are performed with up to 10°/s, which is closer to the other concept that for the pan movements. In general, the curves for the two concepts are closer together and there also is more overlap between the curves for individual participants. The highest velocity measured is 200°/s. With the PTZ-camera currently included in the set-up, 99.91% (SD = 0.11) of head rotations can be directly followed.

5.5 Participants’ comments

In total, 120 comments made by the participants during the study and the debriefing thereafter were recorded by the experimenter and through the questionnaire. Two persons then independently of each other clustered the comments resulting in the five categories “control of the PTZ-camera is cumbersome”, “Virtual Environment (VE) or HMD not suited”, “limitations of the study”, “positive aspects of the concepts”, and “suggested improvements”. An additional five sub-categories were identified: “PTZ-control to sensitive”, “too high delay of PTZ-control”, “virtual environment generally not suited for remote surveillance”, “used HMD not suited for remote surveillance”, and “360° panorama preferred”. Some comments were added to more than one category.

The number of comments clustered into each of the categories and sub-categories is listed in Table 3 and separated according to when they were mentioned, i.e. while using the baseline, the PTZ & Panorama concept, the PTZ only concept, or during the debriefing. Examples for comments clustered into the first category (“control of the PTZ-camera is cumbersome”) are: “The camera hangs and then jumps to a new position.” (Original: “Die Kamera hängt und springt dann zu einer neuen Position”) and “It is difficult to precisely look at a specific object” (Original: “Es ist schwierig etwas zielgenau anzuschauen.”). In the second category (“VE or HMD not suited”), comments noting the isolation from the outside world and the HMD being uncomfortable are included. The noted limitations of the study are that the PTZ-camera not precisely aligned with the gaze direction of the user in the PTZ & Panorama concept and distorted images. Finally, in the last two categories, multiple different aspects (e.g. good quality of video feeds, alternative interaction concepts) are clustered.

Table 3 (Sub-) categories identified and number of associated comments divided by time of mention

6 Discussion

6.1 Usability

The median usability of the IV A₁ “Displays” condition is rated as above excellent by the participants indicating a sufficient usability. This result is to be expected as monitors are a well know visualization medium in the ATC domain. Furthermore, the concept is generally similar to current remote tower systems and was not implemented in a prototypic manner.

The usability of IV A₂ “VR PTZ & Panorama” is evaluated as lower than IV A₁ “Displays”. However, as the median usability is in the range between “good” and “excellent” usability, it is also interpreted as sufficient. Considering the feedback given during the study and in the debriefing, the lower usability may result from smaller issues like the PTZ-camera not precisely aligning with the gaze direction upon activation. Further issues reported by the participants are the PTZ-camera’s zoom and general movement being over-sensitive and unprecise as well as highly delayed. The majority of these problems stem from the prototypic implementation, not the visualisation concept itself, suggesting that the usability can be improved with a better prototype. This excludes the perceived delay between an input (i.e. a head rotation) and the result being visible to the operator, which is a technical limitation of the system.

The IV A₃ “VR PTZ only” is the least usable concept of the three evaluated. The majority of participants rate it below “good” and one as barely “OK”, initially pointing towards insufficient usability. Based on the comments of the participants (see Sect. 5.5), the low rating is primarily the result of two issues related to the PTZ-camera: First, the participants noticed that minor inputs resulted in a disproportionally large rotation of the camera. Second, the time between an input to the Pan-Tilt-Zoom camera and the reaction being visible in the video stream was perceived as too long. Both issues largely equal those described for IV A₂ “VR PTZ & Panorama” and made it difficult for the operators to precisely and comfortably adjust the camera’s position. Differing from IV A₂, comments related to the PTZ-camera make up a larger percentage of all comments for IV A₃—12% for the former compared to 60% for the latter—and the Pan-Tilt-Zoom camera is the only source of optical image information making a high level of usability more critical.

Considering these factors, IV A₃ “VR PTZ only” is overall judged as having insufficient usability. Since the only difference compared to IV A₂ “VR PTZ & Panorama”—which is sufficiently usable—is the omitted panorama and the resulting direct linking between the head rotations and camera movements, the main reason for this is the inability to overlook the entire surrounding without visible lag. As noted in the preceding paragraph, a certain amount of delay between an input and the result being visible cannot be avoided. Hence, even an improved prototype may not be sufficiently usable.

6.2 Cybersickness

While using the IV A₁ “Displays” condition, none of the participants experienced any Cybersickness symptoms. Hence, working with this system leads to no degradation of the operator’s performance and poses no additional health and safety risks (see Sect. 2.2). This result was anticipated as standard computer monitors are used to display the video streams which the operators are familiar with. Opposed to Virtual Environments, the monitors also allow the perception of the real environment providing reference points for movements.

For IV A₂ “VR PTZ & Panorama” non-zero cybersickness scores are calculated for all scales but the Disorientation subscale. Using the Total Score as indication of the overall symptom severity—as is suggested by Kennedy, et al. [13]—only sparse symptoms of low severity are experienced by the participants. Hence, it is expected that the operators’ performance is just slightly degraded and that no specific measures have to be taken to ensure their wellbeing. As noted in Sect. 2.2, the users also habituate to the VE suggesting that the number and severity of symptoms further decrease with repeated exposure.

Analysing the scores in the three subscales, the elevated total sickness score primarily results from the participants experiencing symptoms related to the oculomotor system that are commonly associated with wearing an HMD (i.e. screens close to one’s eyes) in general rather than the specific Virtual Environment presented [4]. Preventing these kinds of symptoms by deploying an improved system is therefore not entirely possible, but they are more likely to be experienced less after repeated exposure.

In summary, the cybersickness symptoms reported for IV A₂ “VR PTZ & Panorama” are very mild for a virtual reality-based system—far below the threshold of 30 for critical cybersickness—and of little concern to the productivity or wellbeing of the operator so that there is no immediate need for measures to reduce the severity. Nevertheless, any potential measures implemented, some of which were listed in Sect. 2.2, will further reduce the cybersickness and be of benefit for the concept’s usability, participant’s productivity and wellbeing and ultimately promote the adaption of the proposed VR-based solution.

In contrast to the other two factors, Total Scores close to and above the critical score are calculated for IV A₃ “VR PTZ only”. They also stretch a wider part of the scale indicating a diverse perception of the cybersickness’ severity. Three of the five participants whose ratings are considered for the SSQ scores experienced symptoms that equal a Total Score below 30¹ indicating only a slightly degraded performance, while the TS of the other two is at or above this mark² pointing towards additional health and safety risks.

The ambivalent Total Score mostly stems from the scores calculated for the Disorientation subscale: While one participant reports to have experienced no symptoms at all, equalling a score of 0.00, another experienced four different symptoms resulting in a score of 55.68. By comparison with IV A₂ “VR PTZ & Panorama”—for which no Disorientation symptoms were reported—this observation is explained with the participants coping differently well with the head’s movements being directly coupled with the PTZ-camera’s rotation and the absence of fixed reference points provided by the panorama: the operators experiencing only minor symptoms already adapted to the delayed movement of the camera discussed in Sect. 6.1 and the resulting temporary mismatch between the gaze and camera directions. For the operators reporting a critical score, this mismatch and no separate point of orientation results in symptoms such as dizziness and vertigo. This ambiguity is furthermore amplified by the over-sensitivity of the Disorientation subscale (compare Sect. 2.2) resulting from a scaling factor of 13.92—the highest among the three subscales—because of which a few low-severity symptoms already equal a critical score.

In addition to the Disorientation subscale, critical scores are also present in the Oculomotor subscale. The specific symptom profile reported by the participants is similar to that of IV A₂ “VR PTZ & Panorama”: alongside fatigue and eyestrain, it comprises general discomfort, difficulty focussing (each reported by three participants), and blurred vision (reported by one participant), partially experienced as moderately severe. Since more symptoms are reported than for IV A₂, it is hypothesised that they not only result from wearing the headset—as is the case for the PTZ & Panorama concept—but also from the specific content displayed (i.e. the jerky and delayed video feed of the PTZ-camera). Fully preventing the latter may be problematic, however, due to the technical limitations of the system.

Overall, the operators experience critical cybersickness when working with the PTZ only concept resulting in serious health and safety risks as well as potentially degraded performance. During the study, the former aspect was pointed out multiple times, although none of the participants had to drop out. While the primary reason causing the symptoms—the problems in the control of the PTZ-camera—can be addressed, the concept still suffers from the absence of fixed reference points (e.g. a panorama) so that symptoms of disorientation may not be fully preventable.

6.3 Acceptance

The usability and cybersickness discussed in the previous sections provide an initial indication which concept is suitable for the targeted use-cases. In the following, the estimated maximum possible worktime and ranking are used to further determine which concepts the operators are willing to use in their day-to-day work.

Due to its similarity with current remote tower optical systems and the operators’ familiarity with monitors in general, IV A₁ “Monitors” is expected to enable the highest possible worktime and to be preferred over the concepts utilizing the VR-headset.

This is confirmed by the results of the validation study: the estimated worktime is high enough not only for temporarily guiding single aircraft, but also for a continuous observation over multiple hours. The ranking is less definitive with the first and second rank split equally between the baseline and the other concepts. Nevertheless, IV A₁ is the only one not sorted on the last rank indicating a preference over the latter.

In conclusion, the baseline is highly accepted among the participants. Furthermore, no restrictions to the aircraft sequence have to be applied given the more than sufficient maximum possible worktime.

In comparison, IV A₂ “VR PTZ & Panorama” is estimated to enable a much shorter worktime but still enough to fulfil the 15–20 min requirement outlined in Sect. 4.4. It is hypothesised that the primary reason for the reduced maximum worktime is the immersion into an VE and the cybersickness that stems from it. As described by [24], the number and severity of symptoms increases over the length of exposure, therefore limiting the amount of time a user is able to comfortably use the virtual reality-headset. Since those symptoms can last beyond the exposure itself (see Sect. 2.2), multiple temporary exposures in close sequence, i.e. multiple 15–20 min usages without adequate time in between, may stress the operator the same way a continuous usage would and therefore exceed the maximum possible worktime stated. Improvements to the system that lower the experienced cybersickness and habituation are expected to prolong the possible worktime.

The ranking of IV A₂ shows a similar trend as IV A₁ and is mostly placed on the first and second rank. However, one participant places the concept on the last rank indicating a slightly lower acceptance than for the baseline.

Factoring in the conclusions for the worktime, the acceptance is determined to be sufficiently high for a day-to-day use of the concept. Nevertheless, special considerations may have to be applied to the sequence of aircraft movements as to avoid repeated exposures with little time in between.

Finally, the conclusions reached on the basis of the maximum possible worktime for IV A₂ similarly apply to IV A₃ “PTZ only”. While the worktime has again shortened, it is still longer than what is required for the considered use-case, i.e. 15–20 min. This correlates with an increase in the calculated cybersickness score supporting the previously discussed hypothesis that the possible worktime is primarily influenced by the severity of symptoms. Hence, the reasoning for the elevated cybersickness, that is visible delay in the PTZ-camera’s video feed, jerky movement, and missing orientation points, also apply to the possible worktime. In contrast to the PTZ & Panorama concept, the delta between the median worktime (30 min) and the required time (15–20 min) is small so that two consecutive exposures to the VE are unacceptable to the operator.

The ranking also stands in contrast to that of the other two concepts: the clear majority of participants place it on the last rank pointing towards a low acceptance. Opposing this, one operator prefers the PTZ only concept without stating a specific reason. Based on the participants’ comments during the study, it is assumed that the simplicity of the concept is favoured in this case.

In summary, however, the general acceptance of the PTZ only concept is not sufficiently high. Additionally, the maximum possible worktime is just marginally higher than what is required, placing restrictions on the sequence of aircraft movements.

6.4 Angular rotation velocity

In the two concepts utilizing the HMD, i.e. IV A₂ “PTZ & Panorama” and A₃ “PTZ only”, head rotations are the sole method for the operator to look around and follow arriving, taxiing, or departing aircraft. Hence, the participant’s angular rotation velocities are an important indicator of whether they faced any issues that would limit their movement.

From the pan and tilt velocities, two main conclusions can be drawn: the first conclusion is that the tilt (vertical) head movements performed by the participants are for both concepts slower than the pan (horizontal) head movements. This is explained by the fact that most of the operator’s surveillance tasks, e.g. following an arriving or departing aircraft, checking if the runway is clear of traffic, etc., predominantly require horizontal head movements rather than vertical movements. Consequently, higher velocities and longer timespans (see Sect. 5.4) are reached for the pan movement.

A further observation is that pan velocities measured for the PTZ & Panorama concept are higher overall than for the PTZ only concept which is evident in the visible separation of the curves in Fig. 10. This is not visible for the tilt movement, however. This may indicate an influence of the jerky and delayed movement of the PTZ-camera discussed previously, which could have led to the participants limiting how fast they turn their heads when working with the PTZ only concept in order to avoid issues. In the other concept, the panorama enabled a lag-free movement regardless of rotation velocity.

7 Conclusions regarding research questions

Accumulating the conclusions reached in the preceding sections for the usability, cybersickness, and acceptance of the three concepts, as well as the angular velocity measured for the two concepts utilizing the VR-headset, the three research questions formulated in Sect. 4.1 are discussed.

To answer the first research question,³ solely the findings for IV A₁ “Displays” are considered as this is evaluated as the best concept for the proposed low-cost optical remote tower system. Throughout all four subjective metrics presented in this paper, the concept using standard computer monitors is valued sufficiently high. It is therefore judged to be a highly usable solution that is accepted by the operators and causes no relevant side-effects degrading their performance. Considering that many of the requirements outlined in ED-240A (see Sect. 2.1) are user-driven, the baseline is a technology candidate to remotely provide a traffic information service at non-ATC operated airports and the research questions hence is positively answered.

With a low-cost optical remote tower system as presented in this paper found to be generally possible, the suitability of a VR-headset to display the video feeds and control the PTZ-camera is discussed under the second research question.⁴ From the preceding discussion, it is clear that the answer to this question depends on the considered concept:

The PTZ & Panorama concept was determined to be sufficiently usable and cause negligible symptoms of cybersickness that do not influence the performance or wellbeing of the operators. It furthermore is accepted by the participants so that the headset in this implementation is an acceptable visualization medium and control device.

The PTZ only concept in contrast is rated as barely usable and led to critical cybersickness. It can also be used only for a limited time making it inadequate for the use at non-ATC aerodromes in its current implementation.

Based on the conclusion drawn in Sect. 6.4, the final research question⁵ is discussed. As it was deduced that the pan movement is the most relevant in the remote surveillance context, the curves of the pan velocity are used to define the required angular velocity of the PTZ-camera. Furthermore, the PTZ & Panorama concept was found to allow the more unrestricted movement. Based on these two aspects, the currently used PTZ-camera is found to be sufficient to directly (i.e. without lag) follow 99.41% of the participants’ head rotations. Consequently, 0.59% of the time (19 s), the camera couldn’t keep up with how fast the operator turned their head. The maximum angular rotation velocity of the PTZ-camera also is not high enough to match the maximum pan velocity of all but one participant. While it is clear that the camera does not have to be as fast as the maximum measured pan velocity, it is reasonable to require angular rotation velocities of up to 230°/s, which would cover the maximum pan velocity of four of the seven participants.

Furthermore, while the time the used PTZ-camera was lagging behind represents a small percentage of total head rotations, high pan velocities may be required in emergency situations not explicitly studied during the evaluation. Due to the measurement of the rotational velocities within the VR-environment itself, the recorded data may also underestimate the head rotation velocities in the controllers' current work environment. The velocities presented in this paper are therefore only a general indication of the required angular velocity that can be used as a starting point for the design of future VR-based remote tower solutions. Future measurements during situations demanding faster movements, ideally at the operators’ current workplace, need to be conducted. However, especially for low-cost systems, a compromise between PTZ-camera speed and cost must be made based on the users’ operational requirements.

From the conclusions drawn in chapter 5.5 and the preceding discussion of the three research questions, the following list of basic requirements to be fulfilled by a concept using a virtual reality-headset in order to be acceptable is compiled:

• 1. Designed for temporary use.

• 2. Precise calibration of PTZ-camera’s direction to gaze direction of user.

• 3. Overview of aerodrome for orientation, if possible 360°.

• 4. PTZ-camera’s movement either decoupled from user’s head turns or sufficiently fast and precise.

• 5. PTZ-camera’s angular rotation velocity not lower than 140°/s.

While the IV A₂ “PTZ & Panorama” condition mostly conforms to all items, further improvements are necessary to IV A₃ “PTZ only”.

8 Advantages compared to conventional remote tower concept

The design of the deployed prototype is based on the existing remote tower concept. It consists of a combination of a single Pan-Tilt-Zoom camera and a simple panorama image of the airfield, whereas the PTZ is used as the core sensor and the panorama view more with a supporting purpose. The video streams generated require less bandwidth than conventional remote tower video streams. This saves network costs and also provides the advantage of a wireless transmission via the cellular network (4G or higher) to another air traffic control facility or even a remote tower centre (RTC). This is particularly helpful for rural aerodromes that are not (yet) offered with a wired broadband connection.

Furthermore, the video streams can be displayed in the classical way on screens or, as required, via virtual reality glasses, which open up additional options: with virtual reality glasses including video display a complete virtual working position could be realized. In this virtual environment the AFIS/UNICOM officer would interact in the same way s/he does today in their real tower environment. The working position could be superimposed (augmented) head down to the video stream and could be used to operate radio and telephone coordination with the pilots and other air traffic control units, to operate airport systems or to view and process electronic flight strips. Location fixed working positions, which require space and generate additional costs, could thus be avoided. Furthermore, it could make the AFIS/UNICOM officer on site independent of her/his tower working position, since the glasses can be operated wirelessly and completely independent of a fixed location. This can have advantages: In addition to providing flight information, flight managers often have other tasks at an airport and are often left on their own. In times of no announced or very sporadic traffic, the flight controller or air traffic service provider can carry out other work outside the tower cabin. If traffic is unexpectedly announced via the radio, the flight controller puts on the virtual reality glasses, which they use to control the PTZ camera and communicate with the pilot. The AFIS/UNICOM officer captures the aircraft with the PTZ camera and sees the corresponding video image. By moving his head, s/he intuitively changes the direction of the PTZ camera in order to monitor the airfield and the traffic, similar to binoculars, and to pass on the necessary flight information to the pilot. The PTZ camera image is best supplemented by a simple panorama image of the airfield so that the AFIS/UNICOM officer gains a better overview of its surroundings.

9 Summary and outlook

In this paper a low-cost remote tower system tailored towards non-ATC aerodromes was developed. The underlying camera set-up designed on the basis of AFIS operators’ operational performance requirements and consisting of a Pan-Tilt-Zoom-camera and a panoramic camera was presented. Furthermore, two concepts to visualize the camera feeds were developed and a baseline was defined. One of the concepts was regarded as baseline with standard computer monitors used to display the cameras’ video feeds. The others build on a novel approach utilizing a Head-Mounted Device presenting both cameras’ and only the PTZ-camera’s video feeds. Additionally, three potential use-cases were outlined.

In a validation study, the baseline and the two developed concepts were evaluated by nine participants—the results of one were found to deviate significantly—with respect to system usability, virtual reality induced cybersickness, and operators’ acceptance. In this course, the concept utilizing monitors proved to be highly usable and accepted with no induced cybersickness. The PTZ & Panorama concept was evaluated slightly worse than the baseline across all three categories, but still sufficiently high for the targeted use cases. The reason for this difference was found to be twofold: wearing the headset’s itself caused some symptoms of cybersickness, lowering the acceptance and the prototypic implementation of the PTZ-camera’s controls affected the perceived usability. These issues also persist in the PTZ only concept. However, due to the lack of fixed reference points (e.g. a panorama) and the focus on the PTZ-camera, they were more pronounced resulting in critical cybersickness.

In summary, the low-cost optical remote tower system building on a minimalistic camera set-up presented in this paper was found to be an acceptable technology candidate to provide remote control at non-ATC operated airports, as is evident in the high evaluation of the baseline concept. The acceptability of a VR-headset as display and control device was found to be highly dependent on the information provided: utilizing only the PTZ-camera left the participants disoriented, leading to critical cybersickness whereas the addition of a panorama largely prevented side-effects degrading the operators’ performance.

From the study’s results, a set of four basic requirements for VE-based visualization approaches were drafted.

Finally, the angular velocity with which the participants turned their head was measured during the study and used to assess the applicability of the PTZ-camera used in the set-up, as well as to deduce requirements for future camera set-ups. It was concluded that the camera is sufficiently fast to follow most, but not all of the operators’ head rotations without lagging behind. As high velocities are especially required in emergency situations, angular rotation velocities of at least 230°/s are advised when controlling a PTZ-camera via an HMD.

Future research in this topic should build on the outlined requirements to develop more sophisticated concepts with improved PTZ-camera controls. A specific focus should be placed on avoiding virtual reality induced cybersickness which was found to be the primary reason for a reduction of the perceived usability and operator acceptance, especially for IV A₃ “PTZ only”. Any future concept should include suitable measures to reduce the cybersickness, starting with the PTZ & Panorama concept as a basis.

It is planned to conduct further studies with more participants to validate the conclusions drawn in this paper for a limited number of controllers. Furthermore, it is intended to merge the visualization concepts presented in this paper with the independently developed interaction concepts [2] and subsequently set up a testbed at an AFIS aerodrome in order to validate the performance in day-to-day operations.