How to Design Virtual Reality Experiences that Feel Natural

Learn how to make VR experiences intuitive, immersive, and human by designing for natural movement instead of menus and buttons.

---

This blog was co-written with Pip Turner, a Product Designer, Software Engineer & UX Prototyper

---

In home virtual reality (VR) gaming, hand tracking is a relatively new way of interacting within experiences that feel natural. But the same can’t be said for VR arcades and other location-based experiences who were early adopters of this technology. There are many design recommendations from VR arcades that can be applied by content developers designing and building hand tracking applications for VR home gaming.

1. Don’t underestimate the value of social interaction in VR

What's the first thing you do when you see a friend across the street?

This might be a controversial answer, but usually we find that we shout their name and wave our hands at them! It sounds obvious, but just as in real life, in location-based VR with hand tracking we find that people are naturally social with their hands. It’s how we communicate in real life, and something we instinctively bring with us to virtual reality.

Even seeing your hands in virtual reality is immediately delightful. Using them to communicate naturally with your friends takes this one step further.

“When people grab controllers, they are like, ‘Ah, okay, this is a videogame.' But when they see their hands in VR… they are like, ‘Wow, I’m actually inside the game right now.’ Shortly followed by the realization that their friends and family are there with them.”

Jorge Torales - CEO, Torch VR

Social gestures and interactions are powerful, and once you’ve got hand tracking in your VR game they’re almost a freebie – most of it will come naturally to players. That said, there are a few things you can do to help:

• Encourage moments that will be enhanced by social interaction: For example, try to design your VR game around teamwork. Have stages where two people need to be working in parallel, communicating with each other, giving each other instruction.

• Prompt your players to use social gestures immediately: Immersive Tech created a location-based VR experience called Uncontained. When they introduce players to hand tracking, they run them through a few easy social gestures. This engages them in the VR experience and gets them interacting early. Doing social gestures such as waving, thumbs up, or pointing can be a great prompt for players to hold on to and remember later.

• Social gestures can make NPCs feel more alive: Guiding the player through the VR experience using the blocks-bot. They show the player how to interact with the environment by performing actions the player will need. These natural gestures bring the warm, fuzzy feeling of social interaction to a single-player experience and make the NPC feel more alive.

To get the bot’s attention in our Blocks VR demo, the player needs to wave at them, and do a thumbs-up when they want to move to the next stage of the tutorial.

2. Signpost players towards interactable objects

When in VR, and especially when using hand tracking, users' expectation is often high. Their hands are realistic – why shouldn’t the world be too? To set expectations, use visual affordances and a strong visual design language. These indicate to a player naturally what can and can’t be interacted with.

Visual affordances

Affordances are the characteristics of an object that signal what actions are possible, and how they should be done. Physical objects convey information about their use. A doorknob affords turning, a chair affords sitting.

Well-designed virtual objects provide clues about what can be done, what’s happening, and what is about to happen. Putting grooves in the surface of a virtual ball guides users where to place their fingers to pick it up. Buttons can look like they need to be pushed, UI panel elements can look like they can be grabbed and moved.

We ran a design sprint focused on core hand tracking interactions for casual VR gaming. This tower defence game prototype included a big button to explode the towers. The button affords being pushed – it’s floating on top of the tower, and has a massive “EXPLODE” sign on it!

A strong design language

Pair your well-designed virtual objects with a strong visual design language to manage player expectations. For example, you could design your VR game so that interactable objects are always highlighted in a colour when users' hands are near, or only allow interactions with objects not attached to physical space. The main point here is consistency – whatever you choose, be consistent with it. Having no design language at all can lead to player disappointment.

In our Cat Explorer VR demo, controls are all in white, subtly guiding players towards interactable objects. This is further linked through the hands also being in white – creating an immediate connection.

3. Prioritise tangible and physical interactions in hand tracking experiences

A player’s main context is their lifetime of physical experience. With hand tracking, you can lean on this as an initial paradigm. If virtual objects behave similarly to real ones, interaction feels natural and can be almost effortless to learn. Pushing buttons, grabbing blocks, spinning turntables – using these as foundations of your VR game makes for a compelling experience.

It’s definitely occasionally useful to include a few abstract gestures, such as turning your hand over or pinching – for example, to teleport or open a menu. Keep them to a minimum though, teach users slowly, and user test extensively.

Here’s a prototype gesture we tested as a “return to launcher” action. The user is required to hold both hands together in a triangle. This interaction could be useful, but would be hard for a VR player to discover by themselves. It needs to be taught.

4. Mix and match hand tracking and VR controllers when it makes sense

It would be tempting for us to argue that hand tracking is the best input modality for all VR use cases. But actually, different ways of interacting all have their own strengths. For example, VR controllers tend to be better for high precision, whereas hand tracking wins out on social interaction. (There’s also voice control and eye tracking…)

Mixing and matching can create a multiplier effect where the whole becomes more than the sum of its parts. We also recently wrote a swapping algorithm that makes the hand-off between VR controllers and hand tracking robust and speedy.

Hyper-VR developer VEX used a VR controller in one hand and kept the other free. Interacting with your spaceship and crewmates? Use hand tracking. Shooting at enemies? Use physical controls.

5. Let players do things with their hands in VR that they can’t in real life

Just because players are using their hands doesn’t mean interactions have to be exclusively realistic. In fact, in location-based VR, we’ve found that players expect to be wowed and to have “an experience”.

The whole point of VR is to do things you’re unable to do in real life. So, see where your imagination can take players' hands to develop new VR game mechanics. Just remember that if the interactions used are not based on real life, keep them rooted in context to make them easy to learn.

A “finger gun” game prototype built during a design sprint on casual VR games. By contextualizing the interaction in a cowboy theme, the concept makes sense. It also utilizes player expectations – we’ve all made finger guns as children.

(Due to technical issues, the search service is temporarily unavailable.)

Conclusion: Designing Virtual Reality Experience that Feel Natural: Bridging the Gap Between VR Arcades and Home Gaming

Hand tracking in VR has the potential to revolutionize home gaming by borrowing lessons from its successful use in VR arcades. By prioritizing natural interactions, social engagement, and intuitive design, developers can create experiences that feel both magical and effortless. As the line between physical and virtual worlds blurs, these insights from location-based VR offer a roadmap for crafting immersive, memorable home gaming experiences.

Key Takeaways for Developers and Designers:

1. Leverage Social Interaction:

• Design cooperative gameplay that requires communication (e.g., teamwork challenges).

• Prompt players to use gestures like waving or pointing early to foster engagement.

• Use NPCs with lifelike gestures to enhance immersion, even in single-player games.

2. Clarify Interactivity with Visual Cues:

• Apply visual affordances (e.g., buttons that look pushable, grooves for gripping) to signal interactivity.

• Maintain a consistent visual language (e.g., color highlights near interactable objects) to set player expectations.

3. Anchor Interactions in Real-World Physics:

• Build interactions around tangible actions (e.g., grabbing, pushing) to reduce learning curves.

• Limit abstract gestures (e.g., teleportation shortcuts) and ensure they’re taught gradually.

4. Combine Input Methods Strategically:

• Use hand tracking for social/expressive actions and controllers for precision (e.g., shooting mechanics).

• Ensure seamless transitions between input modes to enhance versatility.

5. Embrace the Impossible:

• Introduce fantastical interactions (e.g., finger guns, energy blasts) but ground them in familiar themes (e.g., cowboy settings).

• Balance creativity with contextual clarity to keep actions intuitive.

By embracing these principles, developers can unlock the full potential of hand tracking, creating VR experiences that are as intuitive as they are extraordinary. The future of home VR lies not in replicating reality, but in enhancing it—letting players’ hands become portals to worlds where the unimaginable feels natural. Let’s build that future, one gesture at a time.

---

This post was originally published on the Ultraleap website & the Ultraleap documentation site