Responsiveness and interactivity are key to immersion

A few weeks ago, I attended my second Microsoft Holographic Academy session and was particularly inspired by the session Design 201: Humans. Led by Becky Haruyama and Amit Bapat of Microsoft Analog Design, the course started by discussing mixed reality as a framework of environments, objects and humans.

Environments set context

We react based on the environment we’re in. It’s not something we think about, it’s natural. Think about how people behave at a library vs. a concert. So much can be leveraged from the physical world and we should view it as our guide.

Objects affect behavior

Objects, on the other hand, support our intent. When we interact with objects we don’t distinguish between physical or digital; we treat them universally. We don’t walk through a hologram, and we avoid sharp corners because they could hurt us.

Humans are the reason why we create

Human need is the thing we're trying to solve or satisfy. Understanding what you’re trying to accomplish can guide you to better results .

In mixed reality you can have a physical environment, with virtual objects and humans, mixing and matching as needed.

One of my favorite statements from the course: In the physical world everything has to be physical, in virtual reality everything has to be virtual, in mixed reality you can have a physical environment, with virtual objects and humans, mixing and matching as needed. Mixed reality opens so many possibilities.

Perception plays a key role in mixed reality. What our eyes see our mind believes, and what our mind believes our body reacts to. The following video was shared, and is a great example of this learning - watch the guy reach for the fork!

The more senses you can activate in mixed reality, the more your body believes it and physically feels like you are there. Instead of being an observer you are in the presence of feeling. Human presence directly correlates with environment immersion.

To increase presence, we must increase immersion

  • Make holograms believable - dark, light, shadows, grounding, occlusion
  • Add depth
  • Provide environment to influence surroundings
  • Leverage sound
    • Humans have a natural response to sound
    • We can divert attention with environmental sound
    • However, sound and no visual confuses people, sound needs a visual touchpoint. If we can’t see it, we will look for it – which on the flip side can be a great tactic if you want to guide people to another location.
  • Haptics - we can’t touch holograms, but we can do things to enhance interactions. Haptic sound as you do things adds realism and grounding to the real world
  • Utilize avatars
  • Relate to expectations – an interesting insight is that people sometimes expect a blue wireframe look to holograms. We can add believability by starting with a blue wireframe hologram and then bursting into color.
 
Illustrations by Erik Kazuo Takara and Colour Block from the Noun Project.

Illustrations by Erik Kazuo Takara and Colour Block from the Noun Project.

 

Presence does have risks. Humans can consider items like holographic tables real, and place physical objects on them, only for the objects to fall to the floor. Beware of this and save some phone screens. Also remember that responsiveness and interactivity are key to immersion. By thinking about our expectations and leveraging real world cues, we can more successfully create great apps for our audiences.


Kat McCluskey (@mccluskeykat) is Senior Art Director for the Razorfish Emerging Experiences team, based out of our Atlanta office. Header image: thesocialjoystick

Comment