1 Introduction

Immersive 3D interfaces offer potential advantages over 2D interfaces, including higher dimensionality input and output and more effective use of proprioception, kinesthesis, and spatial memory. Realizing those advantages, however, requires effective 3D interaction techniques. Researchers have created some techniques, but they represent just part of the possible design space. As such, more effective techniques may exist, but to find them we need to create new techniques.

Creating an effective 3D interaction technique typically entails five steps:

  1. 1.

    Identifying the task that users must perform.

  2. 2.

    Identifying the target user population.

  3. 3.

    Choosing input, output, and tracking hardware.

  4. 4.

    Generating an idea for a new technique that allows users to more effectively complete the task with the available hardware.

  5. 5.

    Evaluating the technique to verify its efficacy.

Completing most of these steps is straightforward. Steps one through three involve either choices or constraints, and designers can draw on well-known methods for step 5. Step 4, generating an idea, is more difficult because it requires inspiration. A designer can know exactly what the task, user population, hardware, and evaluation method will be, but without a good idea he cannot succeed.

Researchers have proposed approaches to help inspire ideas. Considering the task (Tan et al. 2001), user, and hardware characteristics may help shape potential interaction techniques. Alternately, designers can methodically combine existing components of interaction techniques in new ways (Bowman and Hodges 1999). These approaches are primarily evolutionary: they yield new techniques that are similar to existing techniques.

We propose generating ideas by identifying and breaking assumptions about the real world. Identifying our assumptions and imagining what is possible if we break them is an established technique for spurring creativity e.g., (Gentner and Nielsen 1996). Concentrating on assumptions about the real world can help designers focus on where virtual reality can productively diverge from, rather than emulate, the real world. This approach, when used as part of the larger process, could help designers create revolutionary new interaction techniques. In the following sections we describe the approach and present a case study illustrating its application.

2 Identifying and breaking assumptions

When creating 3D interaction techniques, many designers implicitly break assumptions. We propose instead explicitly identifying and breaking assumptions, particularly assumptions about the real world. Emphasizing differences from the real world may help designers to more fully explore how to leverage the total control that they possess over virtual worlds.

To apply this approach, designers walk through four steps: choosing a problem to solve, assembling a list of assumptions to break, repeatedly iterating through the list and breaking the assumptions, and finally selecting promising ideas.

Step 1: Choose a problem to solve. The problem can range from very general to very specific. Choosing a general problem facilitates more free-form brainstorming and can theoretically lead to particularly revolutionary techniques. However, we found in practice that it primarily generates impractical or useless ideas. During our initial experimentation with this approach, the primary author chose very general problems (e.g., manipulating a distant object, traversing a large virtual world) and, over 6 months, generated roughly 700 ideas worth recording (and many more not even worth that). However, none of them were sufficiently promising to warrant more than informal evaluation.

We had more success when we chose more specific problems shaped by characteristics of the task, users, or hardware. Breaking up general problems into multiple, specific problems led the primary author to generate roughly 200 smaller ideas worth recording in 4 months and yielded three publishable techniques. While we thus had significantly more success choosing specific problems, we note that unconstrained exploration does have some value. While not directly useful, ideas we generated to solve general problems did influence the ideas we later generated to solve specific problems.

Step 2: Assemble a list of assumptions to break. Because the set of potential assumptions is almost limitless, reducing the candidate assumptions to a manageable quantity is critical. The task, user, and hardware characteristics help constrain the assumptions to consider. For example, when trying to create a technique for efficient navigation in very large worlds, breaking assumptions about the properties of space may be effective.

Designers can also reduce the number of assumptions they consider by starting with assumptions that existing techniques break. For our work, we assembled an initial list of assumptions by reviewing the literature and identifying assumptions implicitly broken by existing techniques. We then adjusted our list over time based on our experiences using it.

Some of our initial assumptions carried through to the final list (e.g., space is linear and continuous or the visual and physical properties of a space are identical). We combined and generalized others that we found too specific to help generate a variety of techniques. For example, we initially had multiple assumptions about object properties: their size remains constant, their shape remains constant, they have mass etc. Rather than focusing too narrowly on a subset of properties, we combined the individual assumptions into the more general assumption that an object’s intrinsic and extrinsic properties are inviolate and distinct. Finally, we completely eliminated some assumptions. For example, we eliminated the assumption that the world is round because round worlds are already the exception, rather than the rule. By contrast, we kept the assumption that gravity exists as a reminder that simulating physics in virtual worlds, a subject of much research, is not necessarily a good idea.

Table 1 lists the final version of our list. We found three of the listed assumptions particularly useful: space is linear and continuous, objects are persistent, and appearance does not necessarily reflect reality.

Table 1 Potential assumptions drawn from previous work
Full size table

Step 3: Iterate through the list. Take each assumption and break it to generate as many ideas as possible. We found it useful to choose a quantifiable goal (e.g., generate 30 ideas in 30 min) and continue iterating through the list until we reached it. The key for this step is quantity, not quality. Focusing on producing only “good” ideas makes it harder to create techniques in new parts of the design space because it reinforces the tendency to rely on useful aspects of known techniques.

Step 4: Select promising ideas. We found it useful to take an initial pass through the generated ideas immediately after each brainstorming session and discard those that were impractical or that clearly violated established interface design principles. We then typically selected a small set (roughly 1 in 20) of the remaining ideas that, based on our experience and/or the research literature, seemed particularly promising. We built quick prototypes of them, and if they still seemed interesting we conducted informal evaluations. We only formally evaluated the small number of ideas that still seemed worthy of publication.

We successfully applied this approach to create three interaction techniques: Voodoo Dolls (Pierce and Pausch 2002), painting Interaction Surfaces (Pierce and Pausch 2003), and navigating with visible landmarks and place representations (Pierce et al. 2004). In the next section we present a case study describing how we applied this approach to create the latter technique.

3 Case study: visible landmarks and place representations

Visible landmarks and place representations facilitate navigation in very large virtual worlds. Visible landmarks keep landmarks visible, allowing image plane navigation (Pierce et al. 1997) relative to them. Actual and symbolic place representations enable travel to distant locations. Users travel using an actual representation (a scaled replica) of a place by holding it to achieve the desired view and teleporting. Users travel using a symbolic representation (an “artist’s conception”) of a large place by reaching into it, pulling out a representation for a smaller place, and then either repeating the step with the new representation (if it is symbolic) or teleporting (if it is actual). The world’s semantic place hierarchy determines which visible landmarks and place representations are visible to users in different places.

When we initially set out to create a technique for efficiently navigating large virtual worlds, we chose adults in their 20s and 30s who are proficient with virtual worlds as our target users and a head-mounted display and trackers for the user’s head and hands for our hardware. We chose as our initial problem helping users navigate to locations at intermediate distances: far enough that users could not currently see them, but close enough for users to know an approximate distance and direction.

Table 2 lists, in temporal order, some of the ideas that the primary author generated during this stage, their shortcomings, and the employed assumptions. For brevity this list is not exhaustive; we instead present a sample to illustrate the range of ideas we explored and to convey how our ideas converged over time.

Table 2 Technique ideas for navigation over intermediate distances
Full size table

While we informally evaluated some of those ideas, the idea that landmarks can serve an expanded role in virtual worlds stood out as both grounded in real world practice and yielding practical benefit. In the real world, landmarks help people locate their objective. When looking for a particular place, people often locate the nearest landmark and navigate relative to it (Lynch 1960). In the real world, the utility of landmarks is limited because at greater distances they get too small to see or other objects occlude them. If we break the assumption that space is linear and continuous, we can provide visible landmarks that always remain visible and are always available as targets for image plane navigation (Fig. 1).

Fig. 1
figure 1

Views of a farm before (top) and after (bottom) the addition of visible landmarks

Full size image

While visible landmarks had the potential to aid navigation over intermediate distances, they do not scale to worlds with many landmarks. We thus chose as our next problem how to support navigation over large distances in complex worlds and applied our process again. Table 3 lists the temporally ordered subset of ideas generated by the primary author.

Table 3 Technique ideas for navigation over large distances
Full size table

During brainstorming our emphasis slowly shifted to exploring mechanisms to allow users to work with large worlds at a tractable scale, particularly by dividing them up. A key insight was that, as people naturally organize their cognitive map of the world into a semantic hierarchy (e.g., neighborhood, city, state, region, country) (Chase 1983; Stevens and Coupe 1978; Gould and White 1986), we could break the assumption that not all abstractions are represented by physical objects and organize the world as a hierarchy of place objects. Subsequently generated ideas contributed other technique elements: showing representations (initially WIMs) rather than landmarks for distant places, providing both symbolic and actual place representations, using the hierarchy to determine which representations to show, embedding representations hierarchically, and teleporting to a place by achieving the desired view of its representation (Figs. 2; 3).

Fig. 2
figure 2

A view of a city with two symbolic place representations in the foreground

Full size image
Fig. 3
figure 3

To travel to the beach from the city, the user pulls away the symbolic representation of the east coast (top left) and then pulls out the symbolic representation of the coast itself (top right). The user pulls out the actual representation of the beach (bottom left), positions it to get the desired view, and then lets go to teleport (bottom right)

Full size image

Combining those elements with visible landmarks seemed likely to support efficient navigation over both intermediate and large distances in very large worlds. We built an initial implementation and refined it using informal evaluations, and a summative evaluation confirmed the technique’s effectiveness (Pierce et al. 2004).

4 Conclusion

Our case study illustrates that explicitly identifying and breaking assumptions about the real world can successfully generate new 3D interaction techniques. While we recognize that other individuals’ degree of success with this approach will vary, our work applying this approach and the large body of existing techniques that implicitly break assumptions suggests that it is promising. Even if the approach offers only a small benefit, exploring the design space for effective 3D interaction techniques is sufficiently difficult that even small benefits are worthwhile. We look forward to other designers and researchers successfully employing this approach, instead of or in addition to their current practices, to create new and unusual techniques.