Beyond Air Guitar: How Mixed Reality Will Shape Our Future

Discovering Unplugged: A Gesture-Based VR Experience

Today I want to introduce you to an interesting air guitar app called "Unplugged." What makes this app unique is that it relies entirely on hand gesture control without requiring any physical controllers. This approach creates a fascinating and novel experience.

After entering the app, I started with the tutorial. The interface is controlled by simply moving your hand left or right, and using your index finger to press buttons. The tutorial features a humorous rock star mentor who guides you through the basics of air guitar in VR - everything from proper rock star posture to hand placement techniques.

The app's tracking isn't perfect - my right-hand pick sometimes failed to register, causing me to miss notes. However, what's truly interesting isn't the app's execution but the concepts it introduces.

The Inspiration: Beyond Just an App

This app inspired me to think about two important realizations:

1. Hand gesture interaction has numerous potential applications

2. The Oculus Quest 2 isn't just a VR machine - it's a mixed reality (MR) device

Recognizing this second point completely changes the perspective. I immediately began imagining countless possibilities for how this technology could be used, which is incredibly exciting.

Enabling Hand Tracking Features

The hand gesture functionality can be activated through your device settings. Navigate to the "Experimental Features" section where you'll find the hand tracking option. Once enabled, you can adjust related settings like automatic switching between controllers and hand gestures.

The basic gestures include:

• Pinching your thumb and index finger together for a click

• Long-pressing to scroll up and down pages

• Turning your palms toward yourself with fingers pointing up to bring up the menu

• Making a "flower admiration" gesture (pinching thumb and forefinger) to access settings

There's something beautifully zen about these gestures - they remind me of that lyric "Left hand picking flowers, Right hand dancing swords"

Imagining the Future of Gesture Controls in Games

Hand gesture controls open up exciting possibilities for gaming. Imagine an Avengers-themed game where you can snap your fingers like Thanos and watch half the world disappear.

Or a Spider-Man game where you can shoot webs with the iconic hand position (though not too quickly, or the tracking might fail).

You could even implement Naruto-style hand signs for jutsus, though I noticed that when I put my palms together, the system sometimes loses tracking of my hands.

Meta Quest's Mixed Reality Features

Beyond hand tracking, Meta Quest has introduced several mixed reality features:

• Bluetooth pairing with your mouse, trackpad, and keyboard, allowing you to see your physical keyboard in VR and type with your hands

• The ability to map your sofa and desk into VR, creating a tighter connection between virtual scenes and your physical space

• A "Space Sense" feature that reveals outlines of real-world objects and people, enhancing safety and environmental awareness

Applications Beyond Gaming

Hand tracking has applications beyond gaming. Consider creative apps like Pottery VR, which currently uses controllers to simulate hands in VR. Using your actual hands would provide better precision and a more natural experience that matches real-world interactions.

Of course, hand gestures aren't always superior to controllers. For instance, when holding a virtual gun, having something physical in your hand provides better tactile feedback than empty-handed gestures. The choice between hand tracking, gloves, or controllers should depend on which interaction method best suits the specific scenario.

The Potential of Mixed Reality

Mixed reality offers even more possibilities because the system can recognize your physical surroundings. Imagine pressing a virtual switch that's mapped to a real switch on your wall, providing tactile feedback for a more realistic experience.

For escape room games, developers could overlay virtual elements onto your actual room, transforming your living space into a completely different environment while maintaining physical interaction with real objects. This creates endless opportunities for innovation.

Conclusion

I believe the future belongs to MR, not VR. What are your ideas for these technologies? I'd love to discuss them together. If you see potential for collaboration, I'm open to that too! That's all for today's exploration - see you next week!