Microsoft’s HoloLens: The good, the bad and the ugly

Microsoft HoloLens. (Image courtesy Eddie Offermann.)

Microsoft HoloLens. (Image courtesy Eddie Offermann.)

An overview for those who haven’t used one yet

So I was fortunate enough to have gotten on the developer program for the HoloLens.

Was it hard? I don’t know. I just filled out my application and got in line.

My number finally came up, and I’ve had my developer kit for about a week.

HoloLens. (Image courtesy Microsoft.)

HoloLens. (Image courtesy Microsoft.)

For my day job, I work in a constantly evolving tech company — Mirada Studio — a transmedia company that produces a range of creative storytelling approaches that took us through music videos, commercials, feature films, and now into the emerging worlds of interactive digital real-world experiential entertainment and interaction.

I, my coworkers, and most of my personal and professional associates are really tech-savvy people.

Many of us had an educated idea what to expect, but you really don’t know HoloLens until you actually use HoloLens. There were a number of somewhat startling revelations, some that I experienced, and many that were experienced by the dozens of people that have used mine for a few minutes here or there. This article will cover a few of them.

Spatial awareness

It really cannot be overstated how exceptional the spatial awareness of this device is. It’s quick, it remembers, and it’s so precise it surpasses all reasonable expectations.

From a hardware standpoint, the HoloLens contains what amounts to two Kinect-like sensors mounted to either side of the visor, an infrared depth sensor and a motion sensor, all of which are processed by what Microsoft calls the HPU or the Holographic Processing Unit.

The HPU processes the sensor and location data in hardware, instead of the software-based approach that the competing Google Tango platform uses.

What this means is that the mapping functions are rapid, regularly updated, and far more robust than anything you’ve experienced on another device.

If a hologram is located on a desk, for instance, you can’t move your head fast enough to make that object slide.

Some of the most savvy folks that I’ve shown my HoloLens to have been blown away by how astoundingly rock-solid the tracking is. Reflective and semi-reflective surfaces are, predictably, the enemies of this. Metallic surfaces, glass, and mirrors will cause hiccups in tracking or make areas hard to get geometry reconstruction from, which means holograms won’t “stick” there.

From a spatial memory perspective, what Microsoft has done in HoloLens is also unique versus other solutions that I’ve worked with.

An individual hologram lives within a small “sub-space” where it is understood in high resolution relationship to its environment. Many of these sub-spaces may exist, and as you travel around an area — say, an office building or your home — the entire area is mapped out including connections to the smaller sub-spaces. These maps are updated, and the individual relationships of the sub-spaces are updated to one another.

On a Tango device, if you place an object in a space, wander around for a while, and then return, you’ll typically find that the object has wandered slightly. It’ll be close — Tango does a great job of maintaining a good camera position solution — but it will often have drifted by a few inches. This is due to the space and movement being understood as a single space and one continuous movement.

I’ve seen no such drift on the HoloLens — their approach to spatial understanding is remarkable.

Additionally, thanks to understanding the geometry of your environment, renders are occluded by foreground objects. This is a really remarkable thing that you have to see to get just how powerful it is. Other platforms that I’ve built apps in provide this but this is so powerful, so fast, and so comparatively accurate that it’s quite compelling.

HoloLens remembers individual spaces in relation to what WiFi network you’re attached to. This way, it’s quick for it to find where it is in your home, your office, or a client’s office. I’m not sure that it’s actually doing this purely via WiFi – but the WiFi network name is used to help you identify which spaces it remembers, and it definitely helps it quickly acquire the relevant map.

The human user experience

First experiences with HoloLens present a unique challenge. It’s not surprising, really, since the whole platform is a new paradigm in interaction.

No-one will know what to do when they first put on the headset, so passing the headset around to various people for their first time HoloLens experience is an interesting exercise.

They don’t know whether to point at things or to wave their hands or what. The gestures are easy to learn, though, small variations in technique don’t affect recognition, and Windows Holographic does a truly remarkable job of efficiently recognizing them.

(Image courtesy Microsoft.)

(Image courtesy Microsoft.)

Gesture recognition, both in HoloLens and in general, is tricky, and the HoloLens limit of around two-and-a-half gestures is the state of the art at the moment.

I can’t really count the “ready” gesture — the hand with the index finger extended as if to say “one moment, please” as a gesture itself since it doesn’t trigger anything by itself.

Having developed another augmented reality gestural interface in the past, there are a couple things that become obvious:

Gesture recognition is only acceptable if it’s recognized about 99.5 percent of the time. If the UI frequently misses an interaction, like a “tap,” it rapidly becomes frustrating.

Only a small handful of gestures are predictably recognized. Gestures that can be confused with other gestures have to be avoided. You have to program for finger joint proximity and changes of finger joint proximity.

HoloLens handles both of these situations well, keeping the number of gestures to a minimum and doing a great job with dependably recognizing the small set that are available. There’s a big future in improving the number of available gestures that can be dependably recognized and lots of research to be done in that area.

Traditional applications

There are a few traditional applications that will run on HoloLens at the moment. The selection grows regularly since applications don’t need to be ported explicitly to HoloLens, only to the Universal Windows Platform, provided you’re OK using them as flat panels in space.

For items where I like keeping a window open somewhere, like my Twitter feed or a news site, I almost immediately preferred HoloLens since it doesn’t occupy any real estate on my laptop screen.

Speech recognition is superb though the logic of what Cortana does with that recognition is miles from being a consumer product.

Cortana feels grafted onto HoloLens from Windows 10, rather than being an integrated part of the experience. I regularly wanted to say, “Hey, Cortana, open Bluetooth settings,” for instance, only to have a browser window open with a Bing search for “Bluetooth settings.”

Once you know what things you can say, though, it’s really expedient to use. I can type faster, for instance, glancing around and saying “select” for each key than by air-tapping on each.

When it comes to typing — ugh. There’s nothing good here, but it’s not really HoloLens’ fault.

It doesn’t even make sense as a paradigm and yet we’re constrained to it anyway because there’s nothing else that’s replaced typing yet. Voice recognition is really good here and yet talking out loud to your glasses is still not a great way to send messages. A couple misinterpreted words take *forever* to fix.

Do yourself a favor and if you’re going to need to use a keyboard very often, just pair a Bluetooth keyboard. A few words here or there are fine. Using it to feed a search is fine. For typing in the future, we’re going to need a novel interface that’s neither a traditional keyboard nor does it require us to talk out loud to ourselves.

Visual quality

People complained about the field of view. They complained enough and with loud enough voices that I was very skeptical of getting on board with the developer kit.

Of the dozens of people that I’ve shown the device to, guess how many were blown away by the experience?

All of them.

People are typically jaw-droppingly amazed by the actual experience of seeing and doing things with the HoloLens. They’re amazed by it.

And how many mentioned the field of view?

One.

And his words were, roughly, “I’ve heard people talk about this field of view being small… This is amazing!”

A crew member aboard the International Space Station uses Microsoft's HoloLens . (Image courtesy NASA.)

Commander Scott Kelly uses Microsoft’s HoloLens aboard the International Space Station. (Image courtesy NASA.)

Because here’s the thing: You use the center of your vision to look at things. The fovea, where humans have their highest visual acuity, covers only 6 degrees of your total field of view. Visual acuity drops off steadily from there. You don’t use your peripheral vision to read text, you use it to detect movement and avoid predators.

Now, should the field of view be wider? Absolutely. I’d love it. If it was wider it’d be easier to find misplaced interfaces — yeah, you can totally lose windows — and when a window was a bit too close or too large, it’d be more graceful to take it all in at once.

Increasing the field of view will be a constant battle for augmented reality displays, just as it is for virtual reality.

Meanwhile, the resolution is good and crisp. The text is easy to read, the graphics easy to make out.

The nature of what you’re interacting with, though, makes a bare-bones comparison of HoloLens resolution versus, say, laptop resolution a very poor comparison.

If I said it was 720p, or 4k, neither would be a good way to evaluate the display quality.

The resolution is very adequate. Does that make sense? Comparing it to Vive or Rift is likewise useless — they’re very different devices aimed at very different use cases.

Headset comfort

The headset is highly adjustable. So the weight shouldn’t be on the bridge of your nose or pressed weirdly on your temples or whatever.

It’s balanced across your head, with an optional strap, presumably if you have a strangely shaped head that it wants to slide down on. The weight, then, is evenly distributed making it much more comfortable to wear.

I typically place the “halo” with the front just below my hairline and the back below the crown of my head and adjust the eyepieces so they hover just out in front of my eyes, rather than touching the bridge of my nose.

I got a lot of questions from people about how it worked without cables and how far it could be from the computer.

So it’s important to note that this thing isn’t tethered to anything, and it feels like it can’t possibly have a computer in it, even though it does. I’ve used mobile virtual reality headsets that felt heavier.

Another startling realization is that the HoloLens doesn’t get uncomfortably hot. It gets slightly warm during use, but I haven’t felt it being hot in a week of random usage testing.

I’m sure that a very aggressive program will probably challenge that — and I’ll be running it through some pretty grueling paces soon — but HoloLens isn’t heat-challenged.

A whole lot of this is likely because the heavy sensor processing that heats up a lot of other augmented reality devices has been offloaded to hardware instead of piling a major load to the main processor.

Column reprinted with permission from the Big Blue Ceiling blog

Eddie Offermann

Eddie Offermann is the founder of Big Blue Ceiling, an extended reality thinktank based in Marina Del Rey, Calif. He is also a software architect, multimedia developer and post-production computer graphics artist with a history in aerospace, defense and corporate media production

  • Excellent non-biased review! I appreciate the candidness of the “field of view”. That was my concern as well. I’m glad to hear that those fears are unfounded.

    • lovethetech

      Bigger FOV, you will get injuries, I am sure about that. After watching Holotour, I realized The holotour APP was completely occluding everything(walls, furniture ) behind it. If it had bigger FOV, it is recipe for Accidents.

      In my workplace, out of 30+ tried, only one mentioned about it. All the users were smart enough to move back or zoom out .Of course he is tech biased, Google glass owner. The kids were so easy at this.

      These are other reasons in addition to power/batterylife for FOV.

      • Headcase Games

        ha! you’re crazy dude. I have a Hololens as well for awhile now and really enjoy working with it, it’s a superior piece of kit overall. The FOV is absolutely noticeable and absolutely must be improved – BUT FOR NOW, especially as a dev model, it is definitely acceptable.
        It is inaccurate to state that the FOV is a non-issue overall.. some apps, like the Galaxy Explorer, completely lose their impact due to the limitation – but in many cases app design can alleviate the problems greatly. Full disclosure, I am working with separate wearable AR hardware as well (not Hololens) and it’s got a bit better FOV, still far from perfect but we’ve not come across issues yet with people being injured do to decreased “reality” visibility (hope not, we will see with more testing I’m sure!) Anyway it’s fine to project your opinion out there, I just wanted to point out that overall this is still early days and Hololens is still development hardware. It’s cool for it to not behave as a consumer release model as of yet, and from plenty of experience I can attest that the pros far outweigh the cons for the aspiring AR developer. No need to wave the fanboy flag.

        • lovethetech

          What I prefer is the “FOV” is controlled by the user with a allowed minimum size to the Max FOV size.

          Thanks. I have others “2”. I was not talking about smaller or slightly better FOV, but people want a larger peri-view. What I try to say was “The huge FOV” is not at all required.

          And the common people never noticed it at all in the current version. It is a good news.

          Other OEMS are going to bring a bigger FOV.

      • Vittorio Vaselli

        so with holotour you cannot see behind? it is like VR?

        • lovethetech

          it is one hologram extending beyond its FOV. Turning the head, the extension covered the view, I moved and tripped by the step stool. Battery life/power consumption, current mobile CPU and frame rate to avoid the nausea are more important reasons.

  • lovethetech

    Big FOV = Big Blinders. You are certain to have accidents and prone to injuries. The reason MS did not diss that point is to let other MFRs make bigger FOVs mistakes in their AR wearables.

    • walkergw

      I cant say that I agree with you. MS can increase transparency on objects in peripherial view to a void this while still maintaining object integrity. I believe the reason MS did this was the processors are not up to par to produce a better FOV. I expect later models to correct this in a couple years.

      • lovethetech

        increase transparency on objects in peripherial view = no point in having bigger FOV because the objects really become invisible. I have tried the Hololens.

        • walkergw

          Look, I get that you tried it and didn’t see a problem. But that doesn’t mean that it can’t be improved. I haven’t tried it, but I can envision it. And trust me, abrupt cut off as something leaves the “letterbox” will diminish the sense of reality. Of course that is better than nothing, so I am not condemning it. Again, I am just saying it can be improved. I am not 100 percent convinced that holograms in your periphery are a danger if you are walking. But this probably shouldn’t be used when driving anyway. No, You would avoid walking into holograms in your periphery so if one lies over a real object you still won’t walk into it. What could happen is that it covers a moving object that could come through the hologram and hit you such as a ball, and you didn’t see it in time. So I have given you the benefit of the doubt on this and came up with the fade in transparency idea. It protects your periphery if you want it and stops the abrupt disappearance of objects that leave the current letterbox. This expansion of the FOV will without doubt improve Hololens.

          • lovethetech

            I am sure, the OEMS will develop their Holodevices with the Windows Holographic OS with bigger FOVs.

            Unless used for longer hours and days, you will not understand it.
            I was not happy at first and felt uncomfortable with the current FOV.

            But after using Holotours which needs 0 transparency, I understood the reason. MS could have increased the FOV, with all the scientists, users testing in their house, I think they made a good decision.

          • Vittorio Vaselli

            they said the reason is because larger fov impact on battery and production cost.

  • amanieux

    sure it is light, comfortable and has a good tracking but i am surprised to hear a user say the hololens fov is acceptable, of course it is “usable” but the small fov totally breaks the magic of the immersion. we will see what happens when competitors AR headset with wider fov are available (meta2, sulonQ) we will see if the author of this article still enjoy his hololens 🙂

  • James Lloyd

    Great review, thank you Eddie and Maria. This answered so many of the questions I had.

  • I’d suggest that developers provide feedback to Microsoft requesting that the FOV be increased. We’ve done numerous demos with the device and have received more than one complaint about the FOV size. Having said that, the device is impressive no doubt.

  • Vittorio Vaselli

    what is the device weight?

    • About 1.25 pounds.