Augmented Reality (AR) and Mixed Reality (MR) are new emerging technologies that are becoming increasingly popular. Because these technologies are so new, there is a lack of standards and consistency in application and hardware design. This can make it difficult to learn and frustrating for users. One way to standardized design and enhance the usability of a product is through the use of heuristic evaluations. General heuristics, such as Nielsen’s 10 or Schneiderman’s 8 usability heuristics have been used to evaluate these technologies. These heuristics are a useful starting point because they bring attention to many crucial aspects of the usability of a product. However, additional aspects that could alter the users’ experience may not be assessed due to the uniqueness of AR and MR. There are very few validated AR and MR heuristics in the literature for practitioners to use. The purpose of this study was to create and validate a heuristic checklist that can be used to assess and inform design changes that influence the user experience of an AR or MR application and/or device. We followed an established and comprehensive 8-stage methodology developed by Quiñones, Rusu, & Rusu to create and validate our AR and MR usability heuristic checklist . This included a search and summary of the current literature, formally defining heuristics based on this literature search, and validating through heuristic evaluations, expert reviews, and user testing. Our final revised heuristic checklist included 11 heuristics and 94 checklist items that encompasses usability aspects for AR and MR technologies.
- Human factors
- User experience
- Augmented Reality
- Mixed Reality
- Design guidelines
This is a preview of subscription content, access via your institution.
Milgram, P., Kishino, F.: A taxonomy of mixed reality displays. IEICE Trans. Inf. Syst. 77(12), 1321–1329 (1994)
Azuma, R.T.: A survey of augmented reality. Presence Teleoperators Virtual Environ. 6(4), 355–385 (1997)
Microsoft Docs: Mixed Reality Design Start designing and prototyping, https://docs.microsoft.com/en-us/windows/mixed-reality/design/design, Accessed 10 Feb 2022
Quiñones, D., Rusu, C., Rusu, V.: A methodology to develop usability/user experience heuristics. Comput. Stan. interfaces 59, 109–129 (2018)
Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 249–256, Association for computer machinery, New York, NY (1990)
Nielsen, J.: Ten Usability Heuristics (2005)
Schneiderman, B., Plaisant, C., Cohen, M.S., Jacobs, S., Elmqvist, N., Diakopoulous, N.: Designing the User Interface: Strategies for Effective Human-Computer Interaction. Pearson, Hoboken, NJ (2016)
Gonzalez-Holland, E., Whitmer, D., Moralez, L., Mouloua, M.: Examination of the user of Nielsen’s 10 usability heuristics & outlooks for the future. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp. 1472–1475 (2017)
De Paiva Guimarães, M., Martins, V.F., Corrêa, A.G.D.: Usability Evaluation of Learning Objectives with Augmented Reality for Smartphones: A Reinterpretation of Nielsen heuristics. Human-Computer Interaction, pp. 214–228 (2019)
Usability Heuristics Applied to Virtual Reality: https://www.nngroup.com/articles/usability-heuristics-virtual-reality/, Accessed 11 Feb 2022
Endsley, T., Sprehn, K., Brill, R., Ryan, K., Vincent, E., Martin, J.: Augmented reality design heuristics: designing for dynamic interactions. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp. 2100–2104 (2017)
Kalalahti, J., Porras, J.: Developing Usability Evaluation Heuristics for Augmented Reality Applications (2015)
De Paiva Guimarães, M., Martins, V.F.: A checklist to evaluate augmented reality applications. In: 2014 XVI Symposium of Virtual and Augmented Reality, pp. 45–52. IEEE (2014)
Aultman, A., Dowie, S., Hamid, N.: Design heuristics for mobile augmented game user interfaces. In: CHI EA ’18: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–5. Association for Computing Machinery, New York, NY (2018)
Wetzel, R., McCall, R., Braun, A., Broll, W.: Guidelines for designing augmented reality games. In: Proceedings of the 2008 Conference on Future Play: Research, Play, Share, pp. 173–180. Association for Computing Machinery, New York, NY (2008)
Santos, C., Miranda, B., Araujo, T., Carneiro, N., Marques, A., Mota, M., Morais, J., Meiguins, B.: Guidelines for graphical user interface design in mobile augmented reality applications. In: Lackey, S., Shumaker, R. (eds.) VAMR 2016, LNCS 9740, pp. 71–80 (2016)
Ko, S.M., Chang, W.S., Ji, Y.G.: Usability principles for augmented reality applications in a smartphone environment. In: International Journal of Human-Computer Interaction 29(8), 501–515 (2013)
Ganapathy, S.: Human Factors in Augmented Reality Environments. Springer Science + Business Media, New York, NY (2013)
Gale, N., Mirza-Babaei, P., Pedersen, I.: Heuristic guidelines for wearable augmented reality applications. In: Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play, CHI PLAY, Association for Computing Machinery, New York, NY (2015)
Franklin, F., Breyer, F., Kelner, J.: Usability heuristics for collaborative augmented reality remote systems. In: Proceedings of the 16th Symposium on Virtual and Augmented Reality, SVR, pp. 53–62. IEEE, Piata Salvador, Brazil (2014)
Tuli, N., Mantri, A.: Usability principles for augmented reality based kindergarten applications. Procedia Comput. Sci. 172, 679–687 (2020)
Masmuzidin, M., Aziz, N.: The adaptation of Shneiderman’s golden rules and Nielsen’s heuristics on motivational augmented reality technology design for young children. In: 2019 IEEE 9th International Conference on System Engineering and Technology, ICSET, pp. 62–67, IEEE, Shah Alam, Malaysia (2019)
Liang, S.: Design principles of augmented reality focusing on the aging population. In: Pr4oceedings of the 30th International BCS Human Computer Interaction Conference, pp. 1–7. BCS Learning and Development Ltd, Swindon, England (2016)
Gotsis, M., Lympouridis, V., Turpin, D., Tasse, A., Poulos, I.C., Tucker, D., Swider, M., Thin, A.G., Jordan-Marsh, M.: Mixed reality game prototypes for upper body exercise and rehabilitation. In: Coquillart, S., Feiner, S., Kiyokawa, K. (eds.) 2012 IEEE, Virtual Reality Workshops (VRW), pp. 181–182 (2012)
de Belen, R.A., Bednarz, T.: Mixed reality and the internet of things (MRIoT) Interface design guidelines for elderly people. In: Wyeld, T.G., Banissi, E., Ursyn, A., Bannatyne, M., Datia, N., Sarfraz, M. (eds.) International Conference In Information Visualization, Imaging and Visualization, CGIV, pp. 82–85. Adelaide, SA, Australia (2019)
Helle, S., Lehtonen, T., Woodward, C., Turunen, M., Salmi, H. (eds.): Miracle Handbook: Guidelines for Mixed Reality Applications for Culture and Learning Experiences. University of Turku, Turku, Finland (2017)
XR Association: XR primer 2.0: a starter guide for developers. https://xra.org/research/xr-primer-1-0-a-starter-guide-for-developers/, Accessed 11 Feb 2022
Apple: getting started with AR: https://developer.apple.com/news/?id=c6vr1ag2, Accessed 11 Feb 2022
Magic leap: design principles: https://developer.magicleap.com/en-us/learn/guides/design-principles, Accessed 11 Feb 2022
ISO 9241-11:2018: https://www.iso.org/standard/63500.html#:~:text=ISO%209241%2D11%3A2018%20provides,services%20(including%20technical%20and%20personal, Accessed 11 Feb 2022
Usability 101: Introduction to usability: https://www.nngroup.com/articles/usability-101-introduction-to-usability/, Accessed 11 Feb 2022
User Experience Basics: https://www.usability.gov/what-and-why/user-experience.html, Accessed 11 Feb 2022
Principles for Usable Design: https://www.usabilitybok.org/principles-for-usable-design, Accessed 11 Feb 2022
Bangor, A., Kortum, P.T., Miller, J.T.: An empirical evaluation of the system usability scale. Intl. J. Human-Comput. Interaction 24(6), 574–594 (2008)
Bangor, A., Kortum, P., Miller, J.: Determining what individual SUS scores mean: adding an adjective rating scale. J. Usability Stud. 4(3), 114–123 (2009)
Editors and Affiliations
Derby & Chaparro AR and MR Usability Heuristic Checklist
Unboxing & Set-Up
Is the unboxing process a positive experience?
When the user interacts with the device for the first time, are they introduced to the user interface, basic interaction methods, and basic features/content?
Is a quick start guide available with the device?
Is a call to action (QR code, instructions to use AR, etc.) clearly marked in the physical space?
Help & Documentation
Is there the option of a tutorial upon first use of the device and/or application?
Does the tutorial explain all of the necessary actions/mechanics to use the device and/or application?
Is the tutorial easy to understand?
Are required interactions easy to learn?
Is help or documentation easily accessible for the application?
Are instructions easy to understand?
Do instructions provide actionable feedback?
If auditory instructions are given, do these instructions match what the user is seeing in the application?
Are error messages easy to understand?
Do error messages provide actionable feedback?
Does the device’s user interface and/or application avoid irreversible errors?
Is there a way for the user to report errors or crashes to the developer?
Is the user eased into the virtual environment?
Does the device’s user interface and/or application avoid clutter, as appropriate?
Does the device’s user interface and/or application avoid large amounts of text?
Does the screen space focus on the virtual elements rather than controls or other non-AR features, as appropriate?
Is information organized in an understandable manner?
If the quantity of information is large, is it organized in a layered or hierarchical manner so it is easy to understand?
Does the device’s user interface and/or application avoid tasks that involve a large amount of steps to complete?
Does the application make use of all of its AR functions (including information that is visual, auditory, and involved other sensory modalities)?
Integration of Physical & Virtual Worlds
Are physical (real-world) elements easily distinguishable from virtual elements?
Is it clear which virtual elements can be interacted with and which cannot?
Does the device’s user interface and/or application avoid obstructing physical or virtual elements that are necessary for the users’ goals?
Does the device’s user interface and/or application avoid obstructing virtual navigation elements?
Consistency & Standards
Are virtual elements easy to delete or close out of?
Can the user pause the application at any point?
Are all aspects of the device’s user interface and/or application (virtual elements, controls, text, etc.) clear and readable?
Are virtual elements sized appropriately?
Are virtual elements rendered a reasonable distance away from the user’s targeted point?
For mobile devices, are the controls based on known interactions for mobile devices?
For mobile devices, are landscape and portrait mode supported?
For mobile devices, is the application responsive?
Do virtual elements act as the user would expect them to in the real world?
Are virtual elements accurately placed on the real environment?
Does the device and/or application avoid lag, delays, jitter, drift, and other forms of virtual element malfunctions?
Is the navigation consistent throughout the device and/or application?
Can the user navigate freely throughout aspects of the device and/or application?
Does the device adjust to the environment it is used in?
Are environmental requirements clearly defined?
Does the device and/or application remind users to be aware of their surroundings?
Does the device and/or application avoid jargon?
Are sans serif font types used, as appropriate, throughout the device and/or application?
Is the contrast between the background and text sufficient enough that the text can be read easily under a range of normal lighting conditions?
If the text background is transparent, is the text visible across different backgrounds and under a range of normal lighting conditions?
Is the volume adjustable so the user can hear audio, even in noisy environments?
Are auditory features understandable?
Are captions available for auditory features as appropriate?
Does the device and/or application allow for the user to control privacy-related content?
Is it clear what information is private or public content?
Does the application allow users to preserve virtual elements from others users’ changes?
Is it clear which virtual elements can be interacted with and which cannot for each person?
Is content consistent across all users, as it is appropriate?
Can the user experience the device and/or application without pain, discomfort, nausea, etc. DURING use?
Can the user experience the device and/or application without pain, discomfort, nausea, etc. AFTER use?
Is the device’s weight light enough to feel comfortable?
Does the device avoid overheating to the point that it is uncomfortable to use?
Are physical interactions with the application safe and comfortable?
Does the application avoid making the user walk backwards, pull their head back, or push their head downwards to see virtual elements?
Do interactions with the device and/or application avoid tiring the user?
Does the device and/or application avoid causing the user eye strain?
Are users reminded to take breaks to prevent eye strain and fatigue?
Does the device easily adjust its size for different users?
Does the device accommodate users with eyeglasses?
Does the device accommodate for personal protective equipment?
Does the device and/or application provide feedback on its status?
Does the device and its accessories provide feedback about battery levels and charging state?
Does the device and/or application provide feedback for user input?
Does the device and/or application respond quickly to user input?
Does the device and/or application provide the user feedback after automatic selections?
If an automatic selection occurs, does the device and/or application suggest what to do next?
Does the user feel in control?
Are user interactions simple and easy to understand?
Does the device and/or application include multiple forms of interaction so users can choose based on ability, preference, & skill?
Are the forms of interaction direct when it is appropriate to use this form of interaction?
Does the device and/or application avoid interactions that force the user to make large or sudden movements?
Does the device and/or application accommodate for the user to complete other necessary real-world tasks?
Does object manipulation work well in all instances?
Do virtual elements adapt to the users’ position appropriately?
Does the device and/or application avoid input overloading by assigning distinct functions to buttons or gestures?
Recognition Rather than Recall
Are virtual elements and icons self-explanatory (does their form communicate function)?
Are virtual elements and controls placed near objects they reference?
If a virtual element is related to an object that is in motion, is the virtual element tightly coupled with object in motion appropriately?
Are virtual elements that are outside of the field of view easy to find?
Are available user actions identifiable?
If voice commands are included, are text labels for voice commands given?
Is the device sturdy enough to withstand multiple uses?
Does the device have a sturdy storage case?
Is it easy to clean the lenses, cameras, and other components on the device?
Are device parts fixable and replaceable as needed?
Does the device’s battery life last long enough to perform necessary tasks of the application?
Rights and permissions
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Derby, J.L., Chaparro, B.S. (2022). The Development and Validation of an Augmented and Mixed Reality Usability Heuristic Checklist. In: Chen, J.Y.C., Fragomeni, G. (eds) Virtual, Augmented and Mixed Reality: Design and Development. HCII 2022. Lecture Notes in Computer Science, vol 13317. Springer, Cham. https://doi.org/10.1007/978-3-031-05939-1_11
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-05938-4
Online ISBN: 978-3-031-05939-1
eBook Packages: Computer ScienceComputer Science (R0)