Keywords

1 Background

Augmented Reality (AR) refers to a view of real or physical world in which certain elements of the environment are computer generated. These virtual elements could be a modification of a current element in the real world or could be an entirely new element. In an AR system a cue in the environment (such as picture, photograph, QR code, etc.) that when viewed through a screen is digitally replaced with a new element. The modified element could be many things such as changing the look of an object or adding completely new fictitious objects to the view.

AR has already been employed for useful applications in a number of different areas. One example of this is IKEA’s system (Fig. 1) which allows shoppers to virtually preview furniture from their catalog within a room [1]. Apart from retail uses, AR has been a useful tool in industrial maintenance and repair applications. In these cases the AR application can provide workers with relevant information or even instructions for complex and/or dangerous tasks. Instead of needing to rely on printed materials or other references, goggles with integrated video or other heads up display devices are used to highlight or add additional information to the user’s view of the environment (Fig. 2). Instructions, video or other information can be presented to assist with a task that is in progress. This ability to dynamically replace certain visual elements with new/different ones can be a potentially very helpful aid for product designers.

Fig. 1.
figure 1

IKEA’s system allowing different furniture to be virtually previewed within a room

Full size image
Fig. 2.
figure 2

A example of a user’s view of a physical environment overlaid with augmented information related to a maintenance task.

Full size image

Standard AR provides ease in changing the appearance or arrangement of an object, but it does not allow for natural interaction. There are efforts to develop systems to overcome this in the form of Tangible Augmented Reality (TAR). Tangible augmented reality combines visualization properties of AR with tangible interactions of physical objects [2]. Lee and Kim [3] talk about how developers and designers can create better target content using tangible augmented reality, which is, being able to do usability interactions before being used or felt by the target audience. The implementation process included making markers on cards which were placed in front of user. These markers are detected by an AR display such as a head mounted display and a 3d virtual object is placed on the markers. Users can then interact with the cards by moving them around in space as they would normally do with a physical object in real environment to interact with virtual objects. Markers might also be placed on a 3D printed model which is printed to resemble the form of the digital replacement [4]. The digital model is superimposed on the 3D printed model in the augmented display. Since the physical portion of the model is similar to the digital replacement viewed by the user, it allows the view to be manipulated while allowing natural physical interaction. This approach allows a user to interact naturally with products with physical interface elements (such as knobs or switches) which is not possible with a standard AR display.

This is important because the physical method of interaction will directly impact the results of a usability test. Every year companies devote significant resources to the development of new products [5, 6]. Companies like Google and Microsoft spend 5–6 billion US dollars every year on research and development [7].

A new product development (NPD) cycle involves five steps (Exploration and screening, Business analysis, Product development and Testing, Commercialization). Product development and Testing uses 54 % of the resources [8]. These products could be interactive or non-interactive. Before bringing a product to market, companies develop the entire product with all interactions integrated into the product and then release models for usability testing. If they find some glaring issues during testing, they have to revisit the development process. Once a product is in development, it costs 10 times more to fix a problem than during design and once the product has been released it costs 100 times more to fix a problem than in design [6]. The cost to fix design defects during design might be lowered if the cost of building an actual product can be removed and reliable usability testing can be done on a virtual product. In this case if there are issues with the usability of a product, all that needs to be done is changing the virtual rather than a physical model so that another round of testing can be performed.

Some research into the use of AR/TAR applications in product design have been undertaken. Woohun Lee and Jun Park introduced a TAR method which combined a physical artifact (Fig. 3) equipped with buttons which monitored/recorded button use [4].

Fig. 3.
figure 3

Physical artifact of a cleaning robot (left) equipped with a marker to enable a tangible augmented view (right) with natural interaction.

Full size image

Prototypes like this can be very helpful although one such as this example still requires significant engineering and programming skills to create. Such an approach may not always be suitable for early product concept testing. The method also does not give an indication of the reliability of any usability input gathered via the augmented view. If a fully functional version were produced, would the users’ evaluation of the augmented version match the evaluation of the produced version? If it does not, any design decisions based on usability tests from the augmented version may not lead to design decisions which reduce/eliminate problems in the final design.

A study was recently conducted comparing the validity of usability input gathered via AR compared to a finished product [9]. In this investigation, the touch screen interface of a digital music player was modeled in AR. Users were able to navigate and perform actions through the virtual interface (such as selecting and playing songs). Usability testing was performed in three groups.

For the first group, a simple card with an image was used by the AR display to show the virtual re-creation of the music player.

For the second group, a rectangular plastic block of the same size as the music player was used to show the music player interface in the AR display.

The third group used the music player itself. Subjects independently performed a usability evaluation of one of the interfaces.

When the results were compared, there was no significant difference between the results from the AR interfaces compared to the actual product. This is a promising result, but since the interface used was a touch screen, the user interaction was exactly the same in all cases. If the player featured a physical knob, there would be no way to turn it within the virtual interface in the same way that one would be able to do with a real knob.

This study intends to explore this gap through the application of tangible augmented reality. The objective will be to compare the usability test results obtained from a TAR representation of a product with physical interface components compared to the actual product. It intends to gather evidence of whether usability test results from a TAR product artifact are or are not equivalent to physical prototypes. If similar, it can indicate that TAR may be a useful tool for collecting highly accurate feedback on a product concept.

2 Method

80 subjects aged over 18 will be recruited. All participants will be randomly assigned to one of 4 groups:

  • Augmented reality

  • Tangible Augmented Reality with hand visibility correction

  • Tangible Augmented Reality without hand visibility correction and the actual product.

Subjects in the AR group will able to interact with the product via a display screen. They will still be able to operate the virtual product, although the interaction with physical elements will not be exactly the same. For example a knob can be turned by sliding a finger on a touch screen instead of actually being able to grasp the interface element. Subjects in the TAR with hand visibility correction group will able to interact with a physical model which is simple, but similar to the product. They will view the product through an augmented display which replaces the simple model that they are touching with a realistic representation of the product. With this group, AR display will be able to compensate for times when the subject’s hand passes in front of (and blocks) the image that is used by the AR software to render the augmented view. Normally this would cause the realistic display to disappear or not show the user’s hand realistically in the display. Users in the TAR without hand visibility correction group will also view the product through an augmented display which replaces the simple model that they are touching with a realistic representation of the product. In this case the view will not be corrected (meaning that the realistic view may disappear or not show the hand realistically).

All subjects in all groups will perform a series of basic tasks with the product. This will be followed by completing a standard usability survey to objectively measure usability.

The Sunbeam fan-forced heater (Fig. 4) will be modeled in AR and used for the study. It features a physical upper knob for adjusting the temperature. The lower knob is used to control the power level and other settings (on/off, fan-only, low heat, high heat). The product was chosen as it has a physical user interface and is simple to control as we want to minimize the potential influence of the operation of the device from the testing in order to focus on the differences introduced by the different interaction methods between the groups. The heater was modeled in Solidworks and a model was 3D printed (Fig. 5).

Fig. 4.
figure 4

Sunbeam Fan-forced heater

Full size image
Fig. 5.
figure 5

Basic 3D printed model of the sunbeam heater

Full size image

Each experimental group will perform the same set of tasks with their representation of the product before completing the usability survey:

Task 1::

Turn on the heater, shift it to “low” mode, adjust the temperature and turn off the heater

Task 2::

Turn on the heater, shift it to “high” mode, adjust the temperature and turn off the heater

Task 3::

Turn on the heater, shift it to “fan” mode and then turn it off

Each group will consist of 20 users. Each subject will perform the tasks independently, followed by the completion of the NASA-TLX survey and the USE questionnaire [10, 11].

3 Discussion

Tangible Augmented Reality presents a number of potential benefits. It is a relatively low cost solution. Product conceptualization generally involves digital modeling and rendering meaning that simplified, non-functional models may be easily printed using standard rapid prototyping techniques, such as 3D printing. If TAR is shown to give equivalent usability results to a fully functional prototype, it would allow very early and accurate testing of conceptual ideas. Problems may be identified early and easily corrected before other important design decisions are made. It also would allow for easy testing of multiple, different conceptual ideas with a level of accuracy that would otherwise not be possible.

There are technological issues that may need to be overcome. The issue of the augmented view disappearing or becoming unrealistic looking when the marker used to generate the augmented view is blocked by user interaction. This study will begin to provide some data to indicate how significant this issue actually is and thus how critical it is to always correct for.

This study is currently ongoing. It is expected that usability test results from a TAR representation of a product will provide results similar to those from a fully functional product similar to the results found for touch interfaces with a non tangible AR approach. As much as half of the resources dedicated by a company in the design of a new product occur in the early conceptual phases. A tool which would allow reliable testing/input to be collected on early conceptual ideas, without the need for functional prototyping, would allow the merits of many additional approaches to be tested. More reliable data would also allow design teams to make better decisions and fix design defects early in the design process where changes are easier and much less expensive to make.