Abstract
Future military and civilian unmanned systems will be extensively used on land, under the sea, in air, and in space. Evaluation of their autonomy is critical to realize the autonomous operation ability of unmanned systems. This paper discussed the basic concepts of autonomy and the significance of autonomy evaluation, and then surveys and describes existing evaluating methods. Based on analyses of key technologies and these technologies’ technical readiness levels, a cobweb evaluation model as a universal evaluation model has been used to evaluate unmanned systems. If the technology warps (aspects of autonomy) and wefts (levels of those aspects) of unmanned systems can be determined for the cobweb model, the model can handle the mutual coupling, high dimensions, and diversity of warp lines. This model potentially has wide application in evaluating various unmanned systems.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Lin C Y, Zhang Y Q. Intelligent Unmanned Combat Systems (in Chinese). Changsha: National University of Defense Technology Press, 2008
Jiang X S. Underwater Robots (in Chinese). Shenyang: Liaoning Science and Technology Press, 2000
Antsaklis P J, Passino K M, Wang S J. An introduction to autonomous control systems. IEEE Contr Syst Mag, 1991, 11: 5–13
Research Council of the National Academies. Technology Development for Army Unmanned Ground Vehicles. Washington DC: The National Academic Press, 2002
ASTM F2541-06, Standard Guide for Unmanned Undersea Vehicles (UUV) Autonomy and Control, 2006
Huang H M, Albus J, Messinan E, et al. Specifying autonomy levels for unmanned systems: Interim report. In: Gerhart G R, Shoemaker C M, Gage D W, eds. Proceedings of the 2004 SPIE Defense and Security Symposium Conference. 2004 Apr 12–14. Orlando, Florida: Unmanned Ground Technology CI Press, 2004. 386–397
Huang H M, Pavek K, Novak B, et al. A framework for autonomy levels for unmanned systems (ALFUS). In: Association for Unmanned Vehicle Systems International, eds. Proceedings of the AUVSI’s Unmanned System North America Conference, 2005 Jun 28–30. Baltimore: Arlington Va Press, 2005. 849–863
Huang H M. Autonomy Levels for Unmanned Systems (ALFUS) framework: Safety and application issues. In: Messina E, Madhavan R, eds. Proceedings of the Performance Metrics for Intelligent Systems Workshop, 2007 Aug 28–30, Washington DC USA, ACM New York NY USA Press, 2007. 48–53
ASTM E2521-07, Standard Terminology for Urban Search and Rescue Robotic Operations, 2007
Office of the Secretary of Defense. Unmanned Aerial Vehicles Roadmap 2000–2025. Technical Report, Department of Defense, Washington DC, 2001
Office of the Secretary of Defense. Unmanned Aerial Vehicles Roadmap 2002–2027. Technical Report, Department of Defense, Washington DC, 2002
Office of the Secretary of Defense. Unmanned Aircraft System Roadmap 2005–2030. Technical Report, Department of Defense, Washington DC, 2005
Brent A, Rosemary D P, Robert J S. Mission planning system for vehicles with varying levels of autonomy. US Patent, US 7765038B2, 2010-7-27
Satoshi K, Koichi N, James K, et al. Low-level autonomy of the humanoid robots H6 & H7. In: Jarvis R A, Zelinsky A, eds. Robotics Research STAR 6. Heidelberg: Springer, 2003. 83–97
Jeremi G, Simon L. Embedding heterogeneous levels of decisional autonomy in multi-robot systems. In: Alami R, Chatila R, Asama H, eds. Distributed Autonomous Robotic Systems 6. Heidelberg: Springer, 2007. 263–272
Hervé A, Frederic D, Magali B, et al. Goal driven planning and adaptivity for AUVs. In: Proceedings of the First Workshop on Control Architectures of Robots, Montpellier, 2006 Apr 6–7, 191–197
Broten G S, Collier J A, Giesbrecht J L, et al. Architecture for autonomy: Implementation and usage on the raptor UGV. Technical report, Defence R&D Canada-Suffield, 2006
Protti M, Barzan R. UAV autonomy — which level is desirable? — which level is acceptable? alenia aeronautica viewpoint. In: Proceedings of the AVT-SCI Joint Symposium, France, 2007 Nov 19–21. 1–12
Fil M. The test and evaluation of unmanned and autonomous systems. Int Test Eval Assoc J, 2008, 29: 388–395
Suresh M, Ghose D. Role of information and communication in redefining unmanned aerial vehicle autonomous control levels. P I Mech Eng G-J Aer, 2010, 224: 171–197
Gao J S, Chen X D, Yang H. UAV autonomy research (in Chinese). In: Chinese Society of Aeronautics and Astronautics, eds. Proceedings of the second China-Russia Symposium on UAV, 2005 4 12–15, Beijing. Beijing University of Aeronautics & Astronautics Press, 2005. 97–102
Gao J S, Ding Q X, Zou Q Y. Preliminary analysis of the foreign quantitative assessment methods of UAV’s autonomy(in Chinese). In: Proceedings of the First China Navigation, Guidance and Control Conference, 2007 11 4–5, Beijing. 722–726
Chen H, Wang X M, Zhao K R. Hierarchical control architecture of autonomous control for unmanned combat aerial vehicles (in Chinese). Acta Aeronaut et Astronaut Sin, 2008, 28(Suppl): 224–228
Gao J S, Yu F, Ji X G. Current situation of studies on autonomous control level of UAVs (in Chinese). Electron Optics Contr, 2009, 16: 51–54
Yang Z, Zhang R B. Fuzzy evaluated method for the autonomy levels of unmanned systems (in Chinese). J Chin Comput Syst, 2009, 30: 2043–2047
Chen H, Wang X M, Li Y. A survey of autonomous control for UAV. In: Deng H P, Wang L Z, Wang F L, et al., eds. Proceedings of the International Conference on Artificial Intelligence and Computational Intelligence, 2009 Nov 7–8, Shanghai, China. Springer Press, 2009. 267–271
Gao J S, Wang Z Y, Chen X D. Study on UAV’s autonomous control level (in Chinese). Aeronaut Sci Tech, 2010, 2: 40–43
Chen Z J, Wei J Z, Wang Y X, et al. UAV autonomous control levels and system structure (in Chinese). Acta Aeronaut et Astronaut Sin, 2011, 32: 1075–1083
Zhou R, Li R F, Chen Z J. Autonomous control of unmanned tactical aircraft (in Chinese). Contr Decis, 2001, 16: 344–346
Wang Y X, Cai Z H. Autonomous flight control of unmanned aerial vehicle (in Chinese). Aeronaut Manuf Tech, 2009, 8: 26–31
Wang Y C, Qi J T, Han J D. Research on advanced control algorithms for flying robot with rotary wing (in Chinese). In: Chinese Association of Automation, eds. Proceedings of the China Automation Association Conference, 2009 Nov 1–3, Hangzhou, Zhejiang University, 2009. 88–95
Wang Y C, Liu J G, Li B. A-B autonomy of a shape-shifting robot ‘AMOEBA-I’ for USAR. In: Behnam M, ed. Climbing and Walking Robots. Croatia: InTech, 2010. 425–444
Zhu H Y, Niu T F, Shen L C, et al. State of the art and trends of autonomous control of UAV systems (in Chinese). J Nation U Defense Tech, 2010, 32: 115–120
Ren M, Huo X H. Asynchronous double-precision windows based unmanned aerial vehicle real-time path planning. Sci China Inf Sci, 2010, 53: 215–222
Zhang X Y, Duan H B, Yu Y X. Receding horizon control for multi-UAVs close formation control based on differential evolution. Sci China Inf Sci, 2010, 53: 223–235
Duan H B, Liu S Q. Unmanned air/ground vehicles heterogeneous cooperative techniques: Current status and prospects. Sci China Tech Sci, 2010, 53: 1349–1355
Zou Q J, Zhang R B. Research on adjustable autonomy of unmanned system (in Chinese). Mini-micro Syst, 2009, 30: 2254–2258
Wu L Z, Niu Y F, Zhu H Y, et al. Modeling and characterizing of unmanned aerial vehicles autonomy (in Chinese). In: Meng Q H, Max H K, Jia L, et al., eds. Proceedings of the 8th World Congress on Intelligent Control and Automation, 2010 Jul 7–9, Jinan, China. IEEEXplore Press, 2010. 2284–2288
Wang Y C, Liu J G. Bio-inspired energy autonomy for robot (in Chinese). In: Information science editorial board of the 10000 selected problems, eds. 10000 Selected Problems in Sciences (Information Science). Beijing: Science Press, 2011. 692–693
Huntsberger T, Aghazarian H, Castano A, et al. Intelligent autonomy for unmanned sea surface and underwater vehicles. In: Association for Unmanned Vehicle Systems International (AUVSI), eds. Proceedings of the AUVSI Unmanned Systems, 2008 Jun 10–12, San Diego, California. Curran Associates, Inc Press. 2009, 111–123
Nickles M, Rovatsos M, Weiss G, eds. Autonomy 2003. Heidelberg: Springer, 2004. 17–19
Reed N E. A user controlled approach to adjustable autonomy. In: Proceedings of the 38th International Conference on System Sciences, 2005 Jan 3–6, Hawaii, IEEE Computer Society Press, 2005, 295–304
Committee on Autonomous Vehicles in Support of Naval Operations, National Research Council. Autonomous Vehicles in Support of Naval Operations. Washington DC: the National Academy of Sciences Press, 2005
Sheridan T B. Automation, authority and angst — revisited. In: Human Factors Society, eds. Proceedings of the Human Factors Society 35th Annual Meeting, 1991 Sep 2–6, San Francisco, California. Human Factors & Ergonomics Society Press. 1991, 18–26
Parasoraman R, Sheridan T B, Wickens C D. A model for types and levels of human interaction with automation. IEEE T Syst Man Cy A, 2000, 30: 286–297
Larry A Y, Jeffrey A Y, Mark D G. System analysis applied to autonomy: Application to high-altitude long-endurance remotely operated aircraft. In: Proceedings of the AIAA Infotech Aerospace Conference, 2005 Sep 1–3, Seattle, 22–31
Cleary M, Abramson M, Adams M B, et al. Metrics for embedded collaborative intelligent systems. Technical report, Charles Stark Draper Laboratory, 2000
Curtin T, Crimmins D, Curcio J, et al. Autonomous underwater vehicles: Trends and transformations. Mar Technol Soc J, 2005, 39: 65–75
Fleming M, Cohen R. A utility-based theory of initiative in mixed-initiative systems. In: Proceedings of The IJCAI-01 Workshop on Autonomy, Delegation, and Control: Interacting with Autonomous Agents, 2001 Aug 6–8, Seattle, 58–65
Brainov S, Hexmoor H. Quantifying relative autonomy in multiagent interaction. In: Multiagent Systems, Artificial Societies, Simulated Organizations, eds. Proceedings of The IJCAI-01 Workshop on Autonomy, Delegation, and Control: Interacting with Autonomous Agents, 2001 Aug 6–8, Seattle, 2003, 7: 55–73
Barber K, Martin C, Reed N, et al. Dimensions of adjustable autonomy. In: Kowalczyk R, Loke S W, Reed N, et al, eds. Advances in Artificial Intelligence: PRICAI 2000 Workshop Reader. Berlin: Springer, 2011. 353–361
Goodrich M A, Crandall J W, Stimpson J R. Neglect tolerant teaming: Issues and dilemmas. In: Alto P, ed. Proceedings of the 2003 AAAI Spring Symposium on Human Interaction with Autonomous Systems in Complex Environments, 2003 Mar 24–26, Stanford, CA, AAAI Press. 2003, 111–119
Clough B T. Metrics, schmetrics! How the heck do you determine a UAV’s autonomy anyway? In: Proceedings of the Performance Metrics for Intelligent Systems Workshop, 2002 Aug 13–15, Gaithersburg, 12–20
Jochem T M, Pomerleau D A, Thorpe C E. Vision-based neural network road and intersection detection and traversal. In: Proceedings of the IEEE Conference on Intelligent Robots and Systems, 1995 Aug 5–9, Pittsburgh, 344–349
Doboli A, Curiac D, Pescaru D, et al. Cities of the future: Employing wireless sensor networks for efficient decision making in complex environments. Technical Report, State University of New York at Stony Brook, 2008
Zhuang Y, Chen D, Wang W, et al. Status and development of natural scene understanding for vision-based outdoor moblie robot (in Chinese). Acta Automat Sin, 2010, 36: 1–11
Jing X J, Wang Y C, Tan D L. Artificial coordinating field and its application to motion planning of robots in uncertain dynamic environments. Sci China Tech Sci, 2004, 47: 577–594
Liu J G, Ma S G, Wang Y C, et al. Network-based reconfiguration routes for a self-reconfigurable robot. Sci China Inf Sci, 2008, 51: 1532–1546
Zhuang Y, Gu M W, Wang W, et al. Multi-robot cooperative localization based on autonomous motion state estimation and laser data interaction. Sci China Inf Sci, 2010, 53: 2240–2250
Gu F, He Y Q, Han J D, et al. An active cooperative observation method for multi-robots in three dimensional environments (in Chinese). Acta Automat Sin, 2010, 36: 1443–1453
Li Z Q, Ma S G, Li B, et al. Analysis of the constraint relation between ground and selfadaptive mobile mechanism of a transformable wheel-track robot. Sci China Tech Sci, 2011, 54: 610–624
Lu Z L, Ma S G, Li B, et al. Gaits-transferable CPG controller for a snake-like robot. Sci China Inf Sci, 2008, 51: 293–305
Liu J G, Wang Y C, Li B, et al. Center-configuration selection technique for the reconfigurable modular robot. Sci China Inf Sci, 2007, 50: 697–910
Duan H B, Shao S, Su B W, et al. New development thoughts on the bio-inspired intelligence based control for unmanned combat aerial vehicle. Sci China Tech Sci, 2010, 53: 2025–2031
Song D L, Qi J T, Han J D, et al. Model identification and active modeling control for rotor fly-robot: Theory and experiment (in Chinese). Acta Automat Sin, 2011, 37: 480–495
Bowen D G, MacKenzie S C. Autonomous collaborative unmanned vehicles: Technological drivers and constraints. Technical Report, Defence R&D Canada, 2003
Xu Y R, Su Y M, Pang Y J. Expectation of the development in the technology on ocean space intelligent unmanned vehicles (in Chinese). Chin J Ship Res, 2006, 1: 1–4
John C M. Technology readiness levels: A white paper. Technical Report, Advanced Concepts Office, Office of Space Access and Technology, NASA, 1995
Hobson B. A technology maturity measurement system for the department of national defence: The TML System. Technical Report, Defence R&D Canada-Atlantic, 2006
Guo D Q. TRL Based Technology Maturity Model and Assessment Research (in Chinese). Master Dissertation. Changsha: National University of Defense Technology, 2010
James W B. A suite of tools for technology assessment. Proceedings of the AFRL Technology Maturity Conference, 2007 Sept 11–13, Virginia Beach, VA, 1123–1129
Qian D, Cui L, Xiao C M. Technology readiness level evaluation method and application to underwater weapon (in Chinese). Torpedo Tech, 2006, 14: 1–7
Chen Y B, Huang Y H. Missile technology maturity assessment using TRL method (in Chinese). Sci Tech Res, 2007, 8: 3–4
Zhu Y L. Develop aerospace technology maturity research (in Chinese). Aerosp Ind Manag, 2008, 5: 10–13
Caroline P G, Suzanne G, Jeannine S, et al. Using the technology readiness levels scale to support technology management in the DoD’s ATD/STO Environments. Technical Report, Carnegie Mellon University, 2002
Nolte W L, Kennedy B C, Dziegiel R J J. Technology readiness level calculator. In: NDIA Systems Engineering Conference, 2003 Oct 20-22. Presentation available from http://lincoln.gsfc.nasa.gov/trl/Nolte2003.pdf
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is published with open access at Springerlink.com
Rights and permissions
This article is published under an open access license. Please check the 'Copyright Information' section either on this page or in the PDF for details of this license and what re-use is permitted. If your intended use exceeds what is permitted by the license or if you are unable to locate the licence and re-use information, please contact the Rights and Permissions team.
About this article
Cite this article
Wang, Y., Liu, J. Evaluation methods for the autonomy of unmanned systems. Chin. Sci. Bull. 57, 3409–3418 (2012). https://doi.org/10.1007/s11434-012-5183-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11434-012-5183-2