We can finally break free from traditional screen jail and communicate like humans
I had the opportunity to participate in Microsoft's Holographic Academy during its first week with two colleagues. It was an eye-opener to see creatives and techs with diverse backgrounds learning together. The HoloLens is a unique device that requires collaboration between different fields more than ever. Within these three days, we learned about the ingredients for designing a human experience that is unique only to the HoloLens.
Audio plays a big part in creating an immersive HoloLens experience. When we were designing a make-believe experience in the academy, we were taught to give audio an equal role as visual, for it carries an equivalent amount of information. Imagine trees springing up around you in an experience. Looks cool. Now add sounds of bird chirping, leaves rustling and rain drizzling. You will now believe that your room exists in the forest. It is also not just hearing, but communicating. One of the best HoloLens features is its voice input to speed up multitasking. We can concentrate on doing something without the UI interfering, simply by speaking the commands for a function.
Aside from seeing and hearing, we can make use of touch, smell and taste to elevate an experience. This can be done by taking factors outside of the HoloLens into consideration. Think about how physical objects can interact holograms. For our make-believe project my team designed Candy Concert, a conceptual candy eating experience. The experience involves all five senses. In short, the user would unwrap the candy that releases an aroma, taste it, then enjoy the visuals and sounds on the HoloLens.
Holograms should also obey our laws of physics, since they co-exist in our world. When you throw a holographic basketball up, it should drop down and bounce off the physical ground. If it continues flying up instead, the immersion is immediately broken, because the user is unfamiliar with that reaction. Some experiences may defy that, but bare in mind that your user does not exist in an abstracted dimension with different physics which he or she is unfamiliar with.
The environment where the experience takes place plays a critical role in the design process too. I had the chance to experience the Volvo car demo at Microsoft. I was brought into a dark enclosed room, with a circular table in the middle lit by a single spotlight above, with a black wall on one end of the room. After putting on the HoloLens, I knew immediately where the hologram would appear because of the harsh spotlight on the table. A miniature Volvo car appears on it. I then found myself walking around the table instinctively because of the its circular design, to check out features of the car. Then, it transformed into streams of light, shooting across the room towards the black wall, where I was already anticipating it. The light streams turned into a larger car model and drives through the city. All the while the hologram was floating, but I did not mind that because of the black wall which defined an infinite universe.
Designing for HoloLens requires us to think about spatial layouts, and this is especially important for UI design. Standard conventions no longer apply, and UI elements should be accessible in any direction—which in some cases makes sense for a circular design, or objectifying the button. The standard 2D buttons will feel cumbersome in 3D space, and having it tag along to the user's field of view obstructs vision. Most importantly, it disconnects the user from immersion. There are several other ways to get input aside from buttons and voice. Orientation can also be used. When I was walking around the table in the Volvo demo, the car highlighted its features that were relative to my direction.
Another way to make use of spatial information is data visualization. Data does not have to be text anymore, it can manifest as a hologram in the physical world. Think about navigation, where the path exists in your world as a beaming line on the floor instead of textual directions. Think of holographic barricades that block off paths you should not access on the way to your destination.
This three-day course taught me that the possibilities are endless on the HoloLens, as we can finally break free from traditional screen jail and communicate like humans—improving the way we collaborate in future.
Fyn Ng (@fyn.ng) is a Motion Designer for the Razorfish Emerging Experiences team, based out of our New York office.