Abstract
The cockpit display system is the most important source for the pilot to obtain information, so its usability has a great significance. However, the relevant research is very rare and inappropriate for the moment. Therefore, in order to improve the cockpit display system, this paper proposes five evaluation factors and a series of evaluation indicators, and build a set of evaluation method. Firstly, this paper adopt Analytic Hierarchy Process to confirm the weight of each factor and its indicators. Secondly, this paper adopt expert scoring method to obtain all indicators’ usability score. Finally, this paper integrates the usability score and corresponding weight of every indicator to give a overall usability score of the cockpit display system. Besides, using the above method, we invite five graduate students to make a cockpit simulator display system’s usability evaluation, the evaluation result is between good and very good, demonstrating that the method of this paper indeed can make a quantitative evaluation for the usability of cockpit display system. Fortunately, this will be the first time in the history of the whole cockpit display system’s usability evaluation, undoubtedly, it will also accelerate the development of the whole cockpit display system’s usability rapidly. Similarly, we can also generalize this method to the whole cockpit or even the whole aircraft’s usability evaluation.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
The concept of usability arises from the internet industry in North America and some developed countries in Europe at the end of 1980s, now it has become one of the important research contents in the field of man-machine interaction by now. The international standards ISO9241 defines the usability as Extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.
The cockpit is the terminal for pilot and aircraft interaction directly. Just like other interactive devices, the cockpit’s usability is directly related to the flight safety. 80 % of the information in flight is provided by the display system, and the information on the display system is numerous and complicated, especially in emergency, various visual or auditory alarm signals appear at the same time. There is no doubt that the pilot will bear great psychological pressure, not only may slow down the effective operation, may also make errors in busy. So the usability design of cockpit display system should be effectiveness, efficiency and satisfaction.
Wang Haibo from Southeast University (Nanjing, P.R. China) made an eye movement experiment to evaluate the usability of the fighter cockpit digital interface. By analysing four kinds of quantitative data of the eye movement which include the pilot’s gaze point coordinate, the gaze point number, the pupil frequency and the event time of each layout, they confirmed the optimal layout since all the eye movement’s date of this one is better than all others.
Besides, Wang Haibo made another experiment to compare the digital interface’s usability of F18 and a new 4th-generation fighter. They offered the cockpit digital interface prototypes by computer simulating, and then noted down the task complete ratio, errors, task time and assists in the same task. As a result, all the four data of the new fourth-generation fighter were better than the F18’s. So they drawed a conclusion that the new fourth-generation fighter digital interface’s usability was higher.
There are at least two methods to layout the information on the display. One is Multi Criteria Decision Making (MCDM) that is employed as a quantitative tool. Firstly, experts give a importance’s sequence of all information, then according to the importance’s sequence of all positions on display, put each information onto the corresponding position. Another is Card Sorting that is used as a qualitative tool for the same problem. Firstly, each information is written on one of the cards, then users put all cards with information onto the positions of the display, explaining every choice’s reason, finally, confirm the information’s layout on the display on the basis of these reasons recorded. We can evaluate a certain layout’s usability of the information on the display according to the above methods’ result.
Wei Hengyang from Beihang University think the situation awareness (SA) is a very important factor for the aircraft cockpit display interface (CDA)’s design. Based on the simulation environment, they carry out a human-in-the-loop experiment to measure the SA by the situation awareness global assessment technique (SAGAT) and find the SA can serve as an objective way to evaluate the design of CDI. That is to say, the SA will affect the CDA’s usability.
Williams and Ball from Office of Aerospace Medicine conducted a study to assess the impact of advanced navigation displays on instrument flight procedures for general aviation, single pilot operations and found the advanced display in flight performance was advantaged under high-workload conditions. So the advanced display has a better usability under high-workload conditions.
The above researches just focus on someone aspect’s usability of the display system, such as the information layout, instead of the whole display system, in addition, the above evaluation methods have a same latent shortcoming: It can’t confirm which one is the best when there is no one group whose all date are better than others. So this paper is devoted to solve these problems, we focus on the usability evaluation of the whole cockpit display system, including information representation, information layout and information updating and so on. Differently in the method, this paper adopt Analytic Hierarchy Process to confirm the weight of each factor and its indicators, and then integrates the usability score and corresponding weight of every indicator to give a overall usability score of the cockpit display system, thus avoiding the above shortcoming.
2 Methodology
-
A.
Evaluation factor and its indicators
-
Factor1: The layout rationality of the whole display system
-
Its indicators: installation angle, importance principle, common use principle, function principle, sequential use principle
-
Factor2: The layout rationality of the information on display
-
Its indicators: importance principle, accordant motion principle, adjoin interrelated information principle
-
Factor3: Comprehensibility
-
Its indicators: icon simplicity, icon figurativeness, icon distinction
-
Factor4: Visibility
-
Its indicators: resolution, brightness, icon size, color settings, signal reaction time, visual angle
-
Factor5: Information updating rate
-
Its indicators: PFD, ND, ECAM/EICAS
-
Some annotations are as follows:
Importance principle: put the most important display in the most convenient location for the pilot.
Common use principle: put the display of highest use frequency in the most convenient location for the pilot.
Function principle: put displays which have relevant function together.
Sequential use principle: the layout of displays should be coincident with their usual use sequence in the task execution.
Accordant motion principle: when the pilot manipulates the aircraft during the flight, all the pilot, the aircraft and the aircraft’s icon on the display should be accordant in the direction of motion.
Adjoin interrelated information principle: the principle of proximity to the specific processing of the structure of the display problem. Generally should be related to the information displayed in space close to. Relevance or similarity of the display depends on the correlation of the task and the system. Therefore, the related display components should be placed together with the integration of the system.
Signal reaction time: it reflects pixel points’ response speed to input signals of the liquid crystal display, the smaller the signal reaction time is, the better the picture’s changing-over effect will be.
Visual angle: the maximum angle that one can clearly see all the contents of the screen from different directions.
-
B.
Evaluation procedures
-
(1).
Adopt Analytic Hierarchy Process to confirm the weight of each factor and its indicators. That is to say, every user fills in a judgment matrix of factors, and then we take the mean as a final factors’ weigh. In the same way, we obtain each factor’s indicators’ weighs.
-
(2).
Establish a set of numerical indicator evaluating system whose score is from 1 to 9 that corresponds to extremely bad, very bad, bad, slightly bad, neutral, slightly good, good, very good, extremely good respectively. And then users score the indicators according to their performance in use. This paper take the mean score as a final evaluation result.
-
(3).
Integrate some factor’s all indicators’ usability scores and corresponding weights to give a usability score of the factor. As such, we can obtain other factors’ usability scores. And then integrate all factors’ usability scores and corresponding weights to give a overall usability score of the cockpit display system.
-
(1).
3 The Usability Evaluation Experiment of a Cockpit Simulator’s Display System
-
A.
Participants
There are 5 participants including both undergraduate and graduate students from Northwestern Polytechnical University in this test. Among which there are one woman and four men students. They are 23.2 years old in average, ranging from 22 to 25. All the participants have plentiful experience of using computer, but didn’t familiar with the cockpit display system, so they can evaluate its usability by their experiences without mindset. That is to say, they are appropriate participants.
In this experiment, we only employ 5 users, because the key point of this article is to provide a method, of course, the more the user’s number is, the better.
-
B.
Apparatus
The test is provided with a cockpit simulator, which is a simplified version, but the basic functions on the display are complete (Fig. 1).
-
C.
Tasks
In order to exhibit the display system’s performance in work, we will manipulate the plane in the simulator take off, cruise and land. Experimenters look at the display carefully, fill the judgment matrix of factors and the ones of each factor‘s indicators, and give their usability scores to all the indicators.
-
D.
Results
The experimenters are marked (A), (B), (C), (D), (E) respectively. Due to the experimental data is massive, we only list some of the data as follows (Table 1):
We can work out A’s factors’ weights by using MATLAB after putting the above judgment matrix into computer, the result is as follows (Table 2):
We can obtain B, C, D and E’s factors’ weights in the same way, the results are not listed. The following is the average of all the participants’ factors’ weights (Table 3).
Similarly, we can obtain each factor’s average weights of corresponding indicators as follows (Tables 4, 5, 6, 7 and 8):
We can acquire each factor’s indicators’ scores with the same processing procedure, and this paper just take the factor(1) for a example (Table 9).
We can figure out the factor(1)’s usability score since its each indicator’s weight and each indicator’s score is known (Table 10).
Similarly, we can obtain all the factors’ usability scores. Finally we’ll work out the cockpit simulator display system’s usability score (Table 11).
Thus it can be seen the cockpit simulator display system’s usability score is 7.46 (the full mark is 9), which is between good and very good.
4 Conclusion
This paper proposes five evaluation factors and a series of evaluation indicators, and build a set of cockpit display system’s usability evaluation method. Moreover, we verify this method is feasible by a experiment. And our work will be the first time in the history of the whole cockpit display system’s usability evaluation, undoubtedly, it will also accelerate the development of the whole cockpit display system’s usability rapidly. Similarly, we can also generalize this method to the whole cockpit or even the whole aircraft’s usability evaluation.
We will continue to optimize this method by regulating the evaluation factors and indicators with the development of the cockpit display system’s technology.
References
McCall, J.A., Richards, P.K., Walters, G.F.: Factors in Software Quality: Concepts and Definitions of Software Quality, vol. 1. Information Service, Springfield, VA (1977)
Mo, C., Chengqi, X., Haiyan, W., Jing, L., Jiang, S.: The analysis of virtual product based on the usability. Ind. Eng. Manag. 03, 135–140 (2014)
Saffeh, A., Kececi, N., Donyaee, M.: QUIM: a framework for quantifying usability metrics in software quality models. In: Second Asia - Pacific Conference on Quality Software, APAQS 2001, Hong-Kong, pp. 311–318 (2001)
Van Welie, M., Van Der Veer, G.C., Elins, A.: Breaking down usability. In: Proceedings of INTERACT, Amsterdam, The Netherlands, pp. 613–620 (1999)
Tang, L., Yang, M., (Wuhan University of Technology Wuhan, 430070): Research of usability engineering based on the design process of product. The industrial design institution of Chinese Mechanical Engineering Society. In: Proceedings of the 2007 International Conference on Industrial Design, vol. 2/2. The Industrial Design Institution of Chinese Mechanical Engineering Society, May 2007
Wang, H., Xue, C., Liu, Q.: The eye movement experiment and the usability evaluation of the fighter cockpit digital interface. In: 2010 2nd International Conference on Information Engineering and Computer Science (ICIECS), pp. 1–4. IEEE (2010)
Wang, H., Xue, C., Wang, Z.: The quantitative usability evaluation of fighter cockpit digital interface. In: 2010 IEEE 11th International Conference on Computer-Aided Industrial Design and Conceptual Design (CAIDCD), pp. 384–387. IEEE (2010)
Senol, M.B., Dagdeviren, M., Kurt, M., et al.: Evaluation of cockpit design by using quantitative and qualitative tools. In: IEEE International Conference on Industrial Engineering and Engineering Management, IEEM 2009, pp. 847–851. IEEE (2009)
Wei, H., Zhuang, D., Wanyan, X., et al.: An experimental analysis of situation awareness for cockpit display interface evaluation based on flight simulation. Chin. J. Aeronaut. 26(4), 884–889 (2013)
Williams, K.W., Ball, J.D.: Usability and effectiveness of advanced general aviation cockpit displays for instrument flight procedures. Usability and Effectiveness of Advanced General Aviation Cockpit Displays for Instrument Flight Procedures (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Xue, Hj., Yu, Xb., Zhang, Xy. (2016). Usability Evaluation of the Cockpit Display System. In: Marcus, A. (eds) Design, User Experience, and Usability: Novel User Experiences . DUXU 2016. Lecture Notes in Computer Science(), vol 9747. Springer, Cham. https://doi.org/10.1007/978-3-319-40355-7_63
Download citation
DOI: https://doi.org/10.1007/978-3-319-40355-7_63
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-40354-0
Online ISBN: 978-3-319-40355-7
eBook Packages: Computer ScienceComputer Science (R0)