From Design Errors to Design Opportunities Using a Machine Learning Approach

  • Sanghee Kim
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4333)


Human Errors, e.g. a pilot mismanaged the fuel system causing engine failure and fuel starvation, are known to contribute to over 66% of aviation accidents. However, in some cases, the real sources of the errors are the design of aircraft, e.g.the pilot was confused with the different fuel systems across different models in the same manufacture. The failed collaboration between human operators and the systems therefore has been a major concern for aviation industries. Aviation accident reports are critical information sources to understand how to prevent or reduce such problematic collaboration. In particular, the portions of the reports describing how the behaviour of human operators deviated from an established norm and how the design of aircraft systems contributed to this deviation are particularly important. However, it is a time-consuming and error-prone task to manually extract such information from the reports. One reason is that current accident reports do not aim specifically at capturing the information in format easily accessible for aircraft designers. Therefore, an automatic approach that identifies the sentences describing Human Errors and Design Errors is needed. A preliminary test using hand-crafted cue phrases, i.e. a special word or phrases that are used to indicate the types of sentences, showed a limited identification performance. Therefore, a machine learning technique that uses a greater variety of the linguistic features of the cue phrases than the pre-defined ones and makes the identification decisions based on the combinations these features, looks promising. The examples of the features are active or passive sentence styles and the position of keywords in the sentence. This paper presents the results of developing such automastic identification approach.


Human and Design Errors natural language processing supervised learning approach 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abdalla, R.M., Teufel, S.: A Bootstrapping Approach to Unsupervised Detection of Cue Phrase Variants. In: Proc. on the Association for Computational Linguistics (ACL), Australia (2006)Google Scholar
  2. 2.
    Aitken, J.S.: Learning information extraction rules: An inductive logic programming approach. In: Proc. European Conference on Artificial Intelligence ECAI, France, pp. 335–359 (2002)Google Scholar
  3. 3.
    Ayan, N.F.: Using Information Gain as Feature Weight. In: Proc. 8th Turkish Symposium on Artificial Intelligence and Neural Networks (TAINN 1999), Turkey (1999)Google Scholar
  4. 4.
    Berninger, D.J.: Understanding the Role of Human Error in Aircraft Accidents. In Transportation Research Record No. 1298 (1990)Google Scholar
  5. 5.
    Bruseberg, A., Johnson, P.: Understanding Human Error in Context: Approaches to Support Interaction Design Using Air Accident Reports. In: Proc. 12th Int. Symposium on Aviation Psychology, U.S.A, pp. 166–171 (2003)Google Scholar
  6. 6.
    Forman, G.: An Extensive Empirical Study of Feature Selection Metrics for Text Classification. Technical Report, HPL-2002-147R1 (2002)Google Scholar
  7. 7.
    Grimaldii, M., Cunningham, P., Koraram, A.: An Evaluation of Alternative Feature Selection Strategies and Ensemble Techniques for Classifying Music. Technical Report, TCD-CS-2003-21, University if Dublin (2003)Google Scholar
  8. 8.
    Grishman, R.: Information Extraction: Techniques and Challenges. Lecture Notes in Artificial Intellgence, vol. 1299 (1997)Google Scholar
  9. 9.
    Hackey, B., Grover, C.: Sequence modeling for sentence classification in a legal Summarisation System. In: ACM symposium on Applied computing, U.S.A, pp. 292–296 (2005)Google Scholar
  10. 10.
    Hovy, E., Lin, C.Y.: Automated Text Summarisation in Summarist. In: Mani, I., Maybury, M. (eds.) Advances in Automated Text Summarisation (1999)Google Scholar
  11. 11.
    Joachims, T.: Making large-Scale SVM Learning Practical. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods - Support Vector Learning (1999)Google Scholar
  12. 12.
    Johnson, C.: A First Step Towards the Integration of Accident Reports and Constructive Design Documents. In: Felici, M., Kanoun, K., Pasquini, A. (eds.) SAFECOMP 1999. LNCS, vol. 1698, p. 286. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  13. 13.
    Knott, A., Dale, R.: Using linguistic phenomena to motivate a set of coherence relations. Discourse Processes 18(1), 35–62 (1995)CrossRefGoogle Scholar
  14. 14.
    Macus, M.P., Santorini, M.B., Marcinkiewicz, A.M.: Building a Large Annotated Corpus of English: The Penn Treebank. Computational Linguistics 19(2), 313–330 (1994)Google Scholar
  15. 15.
    Miller, G.A., Beckwith, R.W., Fellbaum, C., Gross, D., Miller, K.: Introduction to wordnet: An on-line lexical database. Int. Journal of Lexicography 3(4), 235–312 (1993)CrossRefGoogle Scholar
  16. 16.
    Mitchell, T.: Machine Learning. McGraw Hill, New York (1997)zbMATHGoogle Scholar
  17. 17.
    Muggleton, S.: Inverse entailment and progol. New Generation Computing 13, 245–286 (1995)CrossRefGoogle Scholar
  18. 18.
    Nasukawa, T., Yi, J.: Sentiment Analysis: Capturing Favorability Using Natural Language Processing. In: Int. Conf. on Knowledge Capture, U.S.A, pp. 70–77 (2003)Google Scholar
  19. 19.
    Parson, R., Muggleton, S.: An experiment with browsers that learn. In: Furukawa, K., Michie, D., Muggleton, S. (eds.) Machine Intelligence, vol. 15. Oxford University Press, Oxford (1998)Google Scholar
  20. 20.
    Salton, G.: Advanced Information-Retrieval Models. In: Salton, G. (ed.) Automatic Text Processing. Addison-Wesley Publishing Company, Reading (1989)Google Scholar
  21. 21.
    Sekine, S., Grishman, R.: A Corpus-Based Probabilistic Grammar with only two Non-Terminals. In: Fourth Int. Workshop on Parsing Technologies, pp. 216–223 (1995)Google Scholar
  22. 22.
    Shin, I., Kim, S., Busby, J.S., Hibberd, R.E., Mcmahon, C.A.: An application of semantic annotations to design error. In: IEEE International Conference on Hybrid Information Technology, Korea (in press, 2006)Google Scholar
  23. 23.
    Turney, P.D.: Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews. In: Proc. 40th Association of the Computational Linguistics (ACL), U.S.A, pp. 417–424 (2002)Google Scholar
  24. 24.
    Strzalkowski, T., Wang, J., Wise, B.: A Robust Practical Text Summarisation System. In: AAAI Intelligent Text Summarisation Workshop, U.S.A, pp. 26–30 (1998)Google Scholar
  25. 25.
    Wiebe, J., Riloff, E.: Creating Subjective and Objective Sentence Classifiers from Unannotated Texts. In: Gelbukh, A. (ed.) CICLing 2005. LNCS, vol. 3406, pp. 486–497. Springer, Heidelberg (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Sanghee Kim
    • 1
  1. 1.Engineering Design Centre, Department of EngineeringUniversity of CambridgeU.K.

Personalised recommendations