May 28, 2022

Why Microsoft Should Cover the VR Ground?

Since the announcement of HoloLens in Jan. 21st, augmented reality(AR) and virtual reality(VR) has been the talk in technology world. Since then many VR vendors have announced their plans. Many analysts believe the AR/VR is upon us starting this year, certainly next year.

Taking a closer look at the AR/VR play field, notice there are interesting patterns: 1. There are more players in VR. 2. VR products (or projects) are more mature, more ready for the market. They generally have real stuff to show the press, even some estimated release dates. AR are more quiet. There are fewer players, and most of them don’t have a solid time table for release. Some AR products, like the one from ODG, are already on sale, but those are very limited in functions, not full featured AR devices.

Many put HoloLens in AR category, I honestly hesitate to do that. HoloLens is certainly most advanced AR device to be released this year, but did Microsoft just missed the VR? Lets take a look.

Market wise, VR is more of niche. Because of nature of technology, the chance of the headset getting much smaller is low, I have wrote about this before. Its target users would likely be hardcore gamers, who value immersive-ness over the look and style. Unfortunately, that’s the main market for Xbox console. If Sony comes up with VR device that works with PS4. Valve/HTC come up with VR headset work with their game titles, Microsoft stands to loose hardcore gamers. I wouldn’t simply say, Microsoft don’t know or don’t understand this. I also wouldn’t say that Microsoft was caught by surprise, because Sony and other VR vendor’s VR plan are highly visible for a couple years now. The question remains: what is Microsoft’s plan for VR. Let me say it again, they can’t ignore it. They probably haven’t. So, what is it?

At the GDC 2015 early this month, Phil Spencer talked with Eurogamer.  When asked if Microsoft has locked itself out of VR, he replies,

I don’t think we’ve locked ourselves out, We’ve looked at a mixed reality space that we could do with Hololens and think about it as a unique set of features and technologies to enable, that doesn’t preclude us from doing anything in the VR space either from a first-party or partnership perspective. I’ve used Morpheus, I’ve used Oculus, I’m going to see more of the demos here.

I am trying to make sense out of his comments. This is probably the only time someone from Microsoft talks about VR and answers the exact question. Do I have an answer from that? Not really, but he did give us something to chew on. Microsoft develop a separate headset for VR is unlikely, and doesn’t make much sense. Partnership with VR headset makers is highly possible, but I wonder how that works with Windows Holographic framework. The third option is for HoloLens to cover VR functions instead of just AR. Microsoft have always called HoloLens and Windows Holographic mixed reality. I think it is possible that the strategic play lies in this. Technically, the only thing stop HoloLens from being a VR headset is its field of view. Field of view, from my understanding is mainly limited by processing power in the case of HoloLens, so that can change over time. I tend to think options 2 and 3 are both very likely. I think it’ll be clearer after the BUILD conference next month.


HoloLens Technology – Eye Tracking


One of the key technologies behind virtual reality and augmented reality is eye tracking. VR needs precise eye tracking to adjust the perception of the images when the viewer is looking at different position. Also it needs eye tracking to tell exactly what and where to show the images for each pupil to generate stereoscopic 3D illusions. HoloLens works differently than VR, but eye tracking is part of core technologies in Hololens. In this post, I’ll talk about eye tracking technology, and what role it plays in HoloLens.

Eye tracking is a method that detect the point of gaze for each individual pupil. Over the years, many techniques has been developed, including dynamic analysis of the eye images and dynamic analysis of the electrooculogram. Eye tracking has been recently used in human-computer interaction, like gazing to hover on the screen, moving pupil to scroll. Marketers user eye tracking to figure out their ads efficiency: marketing is all about eyeballs after all. However when we talk about VR or AR, eye tracking is in the front and center. Without eye tracking, without HoloLens.

Some Important Information About HoloLens

Now we know that HoloLens’ using natural user interface instead of keyboard and mouse, please check sources: here and here. Those user interfaces are: gestures, voice and gaze. Needless to say, without eye tracking, gaze wouldn’t work. The fact it’s part of user interface implies that eye tracking need to be swift and accurate.

More importantly, eye tracking is part of the HoloLens light engine. As we already know that the HoloLens light engine not only generates lights, but also projects them to the eyes. It needs to know the exact position of both eyes, and where they are looking at. A hologram can’t be created without knowing the point of view, and the light engine wouldn’t know where to send the lights if it doesn’t know where the eyes are.  Eye tracking can significantly reduce images that are generated at each moment. apparently it would be very inefficient to render the entire hologram at every moment, that’s a lot more information than a stereoscopic 3D image.

HoloLens: Release Date (Updated)
HoloLens: Price Estimation

Some Business Ideas with HoloLens – (15) Holo-Rock Star
Some Business Ideas with HoloLens – (14) Holo-Classroom, Holo-Conference
Some Business Ideas with HoloLens – (13) Holo-Home Design




Why HoloLens’ Most Remarkable Achievement Is Its Display


There are a few things that separate HoloLens from other AR/VR devices: Magic Leap, Sony Morpheus, HTC Vive, or Oculus Rift to name a few. The Kinect gesture recognition and 3D world mapping are far and away the best in class. Kinect has been in market for a few years. It has been improved in size and response time, especially coupling with HPU in HoloLens. People who have tried the new implementation on HoloLens are generally impressed. More Importantly, HoloLens is built on top of Windows Holographic platform, which will be available on all Windows 10 devices. There is a huge advantage with an established platform, I will get to that in another post. In this post, I’m talking about another technology in HoloLens that makes it stand out from the others: the display technology.

Yes, I am aware that we haven’t seen the product yet, and we don’t even know what is behind the display technology in HoloLens. I think it doesn’t really make a different whether we have seen the final result or not, I believe the final product can only be better. As to the technology details, we may never know even with a product in hand, but it doesn’t matter, I have enough information to say things that I am sure of.

HoloLens: Price Estimation

Display Technology: HoloLens and VR

First, I want to compare HoloLens’ display with the VR displays from Sony or Oculus. Almost all VR displays are using the same technology, with some variation of course. There is a rectangular screen in front of viewer’s eyes, LED or LCD. It displays high resolution images on the screen. Between the screen and viewer’s eyes, there are a pair of glass to make it feel like viewing from a distance. Because the viewer’s eyes are totally covered, there is a feeling that you are separated from the world, immersed into the virtual world. Immersive-ness is something that VR vendors are tooting all along. Unfortunately, in my opinion, immersive-ness has very little to do with its display technology itself. It has, however, a lot to do with the fact that the viewer’s eyes are entirely covered, which by itself is not much of a technology to talk about. If there is something to the VR technology is that it appears to have an impressive field of view, but I am not sure how much a difference between the two. Also the 3D effect from VR is far less realistic than the holographic 3D from HoloLens. With holographic 3D, you can move you head to see an object in different angle. With stereoscopic 3D, no matter what angle you look at the same object, you see the same image. Look at the picture above, with HoloLens it’s hard to tell which object on the table is real, which one is virtual.

The major gripe I have with VR headsets is how they look. I know they are not meant to be worn walking around, but wearing a box is too alien-ish to me. I am not talking about the bulkiness, which can be improved over time, but in its finest and smallest form possible, I still don’t see how it can look good. Sony’s headset is probably the best looking one. Lets make it 2 times thinner, would you say it looks fine? The problem lies here: 1. It has to have a distance from the eyes. 2. It needs a big field of view. That’ why it has to have this size, it can’t be much smaller. Because of technology difference, with AR, like HoloLens, the lens can be much closer to the eyes.

HoloLens: Release Date (Updated)

Display Technology: HoloLens and AR

Now, lets look at other AR display technologies. Microsoft calls its HoloLens mixed reality, but it is actually more AR than VR. When field of view is improved, it will eventually cover entire spectrum from AR to VR. First of all, in AR field no one else has a published demo yet. So we have no idea how good they are. One thing I know: They are all based on a variation of one technology, that is sending light directly to the eyes, whatever that is called (some call it Light Field). Another thing I know: This technology is very hard. Microsoft has worked on this for at least 4 years as far as I know(maybe longer), with many top scientists in the field, and spent tons of money to have demo units as some have seen in January. I have a feeling that this is not something everyone can fondle with, which explain why non of those other AR companies have a demo, or remote releasing date yet.

Some Important Information About HoloLens

The Big Picture

There is a bigger problem in AR and VR together other than technology: the platform problem. I will talk about that later.

Update: Phil Spencer, head of games in Microsoft, has just said in an interview that the technology under HoloLen is more challenging than VR.

In the holographic mixed-reality space, there are fewer players. In some ways, it’s a little more technically challenging — not to take away from any of the challenges of VR.

Some Business Ideas with HoloLens: (1) Holo-IKEA
What Is Mixed Reality, Really?
Some Business Ideas with HoloLens – (4) Holo-Repair and Holo-AAA
Some Business Ideas with HoloLens – (3) Holo-Support
Some Business Ideas with HoloLens: (2) Holo-Architect


Xbox + HoloLens: How Can They Work Together?


Yesterday(Mar 3, 2015), Phil Spencer, head of gaming of Microsoft had announced that games, along with applications will be cross all Windows 10 devices, which includes Xbox, PCs, phones and HoloLens. People are wondering, what’s the connection between Xbox and HoloLens. It’s a topic that is too tempting to pass.

Some Important Information About HoloLens

HoloLens, as we already know, is a stand alone PC. With its Cherry Trail CPU, it should play general mobile games just fine. It will run some light non-indie game as well. But for CPU and GPU intensive games, it has to be on Xbox. Now the question: For holographic games to work on Xbox, HoloLens has to be used. Just how do the two work together?

HoloLens: Release Date (Updated)

Phil Spencer had already hinted in January that Xbox could stream games to HoloLens: So basically HoloLens works as a display to replace the TV, because TVs can’t show holograms. Interesting question is: Can HoloLens also work as an input device to replace the Xbox controller? Lets look at it.

Based on materials from Microsoft, HoloLens only support gestures, voice and gaze. No mouse, no keyboard, no touch. Are those enough to replace the Xbox controller? Assume there is no latency to worry about. Things like grabbing the gun, shoving the dirt, throwing the torch would be hard to do. The idea of virtual Xbox controller probably won’t work because there is no haptic response to the fingers, it’s going to be too slow.

My guess is: HoloLens will work as a display and input device to interact with the holograms, also to provide 3D mapping for the room, but for many games, Xbox  controller is still required.


HoloLens: Price Estimation
Some Business Ideas with HoloLens – (8) Holo-Movie
Some Business Ideas with HoloLens – (7) Holo-Escort
Some Business Ideas with HoloLens – (6) Holo-Online Store



What Is Mixed Reality, Really?




Before we know about HoloLens and Windows Holographic, there are Virtual Reality(VR) and Augmented Reality(AR). VR targets at cutting the reality from the user’s view, and making sure user is totally immersed into the virtual world. That’s main source of thrill when you wear a VR headset. Yes, the content is important, but bungee jumping from 3D movie theatre is different from on a VR headset. AR, on the hand, focuses on overlaying virtual objects onto the real world. At the current state, most workable or demoed AR implementation are very light overlays on top of what the user see in the real world. Mostly numbers, text, diagrams, very light in image and video, let alone 3D. There are reports that some AR implementations, like CastAR’s or Magic Leap’s, put holograms on the real world, but those are far from workable products.

Some Important Information About HoloLens

HoloLens, or Windows Holographic in general is called Mixed Reality during their announcement in the Windows 10 event. Microsoft didn’t invent the term Mixed Reality, but it is probably the first time this term is brought in the spot light of tech scene. As I have mentioned in another post,  Mixed Reality is indeed used in Windows 10 Holographic APIs (in form of namespace Windows::Mirage::Media::MixedRealityCapture). This confirmed it’s formality. So, is this “Mixed Reality” thing (I am the first to use it in capital, but that doesn’t matter) a mere marketing trick or word game? Yes and no.

In my opinion, “Mixed Reality” fits HoloLens better than Augmented Reality. For one, conceptually, AR and VR are in two opposite of the spectrum. Of course, everyone agree that they can overlap with each other, but in reality, if you turn a VR into AR, it defeat a purpose of VR. On the hand, AR has no intention to bring VR effect to reality. HoloLens doesn’t care. Please look at the two images above: In the top image, the wearer sees very minimum amount of virtual objects. Over 90% of her field of view, or her mind is in real world. In the second image, however, the viewer sees 100% of virtual world. Should we call HoloLens VR, or should we call it AR? It doesn’t matter, since it can scale from 0% virtual to 100% virtual (my number may go extreme a little bit, but that doesn’t change the argument).

HoloLens: Release Date (Updated)

Important thing to the user is what problem it’s trying to solve, not how it is called. HoloLens tries to bring the virtual world to the real world, so that they don’t just look real, you can interact with them like real. To me, the term AR and VR don’t catch the essence of HoloLens.

HoloLens: Price Estimation
Some Business Ideas with HoloLens – (17) Holo-Advertising
Will There Be Low Cost HoloLens?
Some Business Ideas with HoloLens – (16) Holo-Map