Hands On With Microsoft HoloLens
REDMOND, Wash.—It is safe to say that no one in the audience at the Windows 10 launch event expected Microsoft to announce a Holographic platform.
When Alex Kipman, a technical fellow on Microsoft's OS team, first said the word holography, I thought I had misheard him. My mind flashed back to when holograms made their way onto charge cards as a security measure.
Microsoft's plans are much more ambitious. Redmond showed a video that demonstrated how holograms will be incorporated into our everyday lives, which looked a lot like the augmented-reality demos we have seen for years.
Then the company brought out a prototype headset, which looks more like Oculus Rift than Google Glass. HoloLens isn't quite ready for sale, but Microsoft let the assembled press into its development lab to take it for a spin anyway.
HoloLens is augmented-reality headset that allows you to mix the virtual world with the real world. Put on the headset and the glass screen can project a digital overlay on top of the physical word. It can be as simple as a Skype window or as complex as a 3D model of a jet engine. In addition to the holograms, you can also see through the lens into the world around you, unlike Oculus Rift. As you move around an object, it stays in place the way a physical object would. Interestingly, the HoloLens maps physical space using a scanning technology that is very similar to the one used in Kinect. Like any new interface, it is hard to explain but a lot easier to understand when you are using it.
My HoloLens tour came with some limitations, the biggest being that no recording equipment was allowed in the lab, not even a cell phone. (You don't know frustration until you are a journalist touring a research lab with no ability to capture photos or video.) The prototypes we used were also less refined than the slick kit that was demoed on stage, which was completely wireless, self-contained, and pretty elegant.
The demo units I used required a power cable, external Holographic Processor Unit (HPU) that hung around my neck, and a headset that needed to be screwed onto my head to secure it. The technician also manually measured and set my Inter-pupil distance (IPD)—essentially the distance between pupils. Again, this will be automatic in the final version, and this is exactly what you would expect from a technology still under development. But if you thought Google Glass made you look like a geek, this takes it to a whole nother level.
Once I was properly strapped in, I had to learn the unique controls for HoloLens, which come down to Gaze, Gesture, and Voice. Turn your head and the HoloLens will follow your gaze and place a curser or arrow wherever you look. To select an object, control, or anything else, look at it and then execute an "air click" with your finger. Again, this is exactly what it sounds like—just hold your finger up and click. Finally, for more complex controls, you can just speak commands—Copy, Call, Open, etc. For my first holographic experience, it was pretty easy to pick up.
Of course, with a brand-new interface like HoloLens, the most immediate question is what can you use it for? Microsoft didn't let the question hang for long. I tried the device in three very different applications, all under the careful and very scripted guidance of a Microsoft developer or engineer. Each had its own merits and issues, but I could appreciate that HoloLens brought something different to the experience. The order was random, but I will describe them in the same order I experienced them.
Building a 3D Model
To warm up, we watched an engineer use HoloLens to build a 3D model in real-time using HoloStudio, a 3D modeling tool. He stood in the middle of the room, tethered to the ceiling via a power cable and draped in gear. The 3D figure he was creating—a koala with a rocket pack—was in the middle of the room. I could see what he saw by checking two large HDTVs on the side of the room. He walked around the hologram, grabbing tools from a holographic control panel, and then used a combination of voice and gestures to build and shape the koala.
I don't know how many times he has done this, but he built a model in minutes. He said a relatively complex model of an X-Wing fighter took about an hour and a half. The model building was impressive, especially since the room was filled with models that were built with HoloStudio and then sent to a 3D printer for manufacturing. Koalas are nice, but if you imagine really complex 3D objects, like a car engine, this kind of prototyping and building gets really interesting. I didn't get to try the HoloLens for this demo, but the potential was pretty clear.
My first actual HoloLens experience was with a Minecraft-like building game. Once strapped in, the small living room I was in filled with blocky castles—on the coffee table and along the wall. I could walk around the structures, gaze upon individual blocks, and then make changes to them using the air click. Voice command let me change tools quickly. After drilling some holes in the castle I could look down through the virtual floor into the levels below. It was definitely immersive, but probably a little slower than it might be with a mouse and keyboard. With a different game, this could be a lot of fun, but this seemed like a pretty basic demo.
Installing a Light Switch
How many PCMag editors does it take to change a light switch? Just one, as long as they have a Microsoft staffer, probably MCP certified, walking them through the process via HoloLens.
For this demo, I tried a HoloLens-enabled version of Skype. At its core, it was a Skype video call. A small window appeared in my virtual field so I could video chat. But because I was wearing the HoloLens, that person could see what I was seeing—the exposed wires, my tools, my fear that I was going to fail to properly connect the switch and go down in infamy as the guy who couldn't make the light go on. Better still, she could annotate my view—drawing an arrow that shows exactly the wire I should connect. Think of it as illustrated technical support. Again, moving beyond the light switch to something more complex like a jet engine, you can see how HoloLens could make technical support a lot more technical.
Finally, in what was undeniably the coolest application of HoloLens, I went to Mars.
Microsoft has been working with NASA's Jet Propulsion Laboratory (JPL) to create a holographic version of Mars for research and maintenance purposes. I strapped on HoloLens and stood next to the Mars rover, surrounded by the Red Planet's vistas. Looking down at my feet, I could see the rocks just inches away. When the wonder passed, I had to ask how this was different than just looking at a high-resolution picture of Mars. After all, this entire environment had to be constructed from photos taken from the rover. In fact, the actual JPL control panel was open on a nearby screen, so I could see what JPL uses today. Turns out, HoloLens brings two added factors to the table, and I was prepared for neither.
First, HoloLens allows for virtual collaboration. With the click of a button, I was joined on my virtual Mars by a JPL project lead, or at least his avatar. He was able to describe the landscape, highlight portions of the terrain for more analysis, and generally explore the landscape with me. Collaborative exploration of the Mars landscape trumps swapping high-res 2D photos any day.
The second benefit is more profound. The 3D HoloLens version of Mars was made from the high-resolution photos the rover took—so there isn't any more information in the scene. In fact, if you look at the 2D pictures, they look sharper. But the hologram overlays multiple photos to create depth and enable the scene to shift naturally as you walk through it. This kind of immersion uses different parts of your brain than looking a photograph. You notice different patterns; the curve of a limestone shelf is more apparent than on a 2D photograph with no depth. I would need a lot more time with the HoloLens to say it is better, but it is definitely different.
All told, I spent about an hour trying out the HoloLens. It is undeniably cool, but will it really make us more productive, creative, and connected, as Microsoft intends? I'm paid to be skeptical, but I will say I'm really looking forward to trying it again.