November 19, 2018

Why Microsoft Should Cover the VR Ground?

game1
Since the announcement of HoloLens in Jan. 21st, augmented reality(AR) and virtual reality(VR) has been the talk in technology world. Since then many VR vendors have announced their plans. Many analysts believe the AR/VR is upon us starting this year, certainly next year.

Taking a closer look at the AR/VR play field, notice there are interesting patterns: 1. There are more players in VR. 2. VR products (or projects) are more mature, more ready for the market. They generally have real stuff to show the press, even some estimated release dates. AR are more quiet. There are fewer players, and most of them don’t have a solid time table for release. Some AR products, like the one from ODG, are already on sale, but those are very limited in functions, not full featured AR devices.

Many put HoloLens in AR category, I honestly hesitate to do that. HoloLens is certainly most advanced AR device to be released this year, but did Microsoft just missed the VR? Lets take a look.

Market wise, VR is more of niche. Because of nature of technology, the chance of the headset getting much smaller is low, I have wrote about this before. Its target users would likely be hardcore gamers, who value immersive-ness over the look and style. Unfortunately, that’s the main market for Xbox console. If Sony comes up with VR device that works with PS4. Valve/HTC come up with VR headset work with their game titles, Microsoft stands to loose hardcore gamers. I wouldn’t simply say, Microsoft don’t know or don’t understand this. I also wouldn’t say that Microsoft was caught by surprise, because Sony and other VR vendor’s VR plan are highly visible for a couple years now. The question remains: what is Microsoft’s plan for VR. Let me say it again, they can’t ignore it. They probably haven’t. So, what is it?

At the GDC 2015 early this month, Phil Spencer talked with Eurogamer.  When asked if Microsoft has locked itself out of VR, he replies,

I don’t think we’ve locked ourselves out, We’ve looked at a mixed reality space that we could do with Hololens and think about it as a unique set of features and technologies to enable, that doesn’t preclude us from doing anything in the VR space either from a first-party or partnership perspective. I’ve used Morpheus, I’ve used Oculus, I’m going to see more of the demos here.

I am trying to make sense out of his comments. This is probably the only time someone from Microsoft talks about VR and answers the exact question. Do I have an answer from that? Not really, but he did give us something to chew on. Microsoft develop a separate headset for VR is unlikely, and doesn’t make much sense. Partnership with VR headset makers is highly possible, but I wonder how that works with Windows Holographic framework. The third option is for HoloLens to cover VR functions instead of just AR. Microsoft have always called HoloLens and Windows Holographic mixed reality. I think it is possible that the strategic play lies in this. Technically, the only thing stop HoloLens from being a VR headset is its field of view. Field of view, from my understanding is mainly limited by processing power in the case of HoloLens, so that can change over time. I tend to think options 2 and 3 are both very likely. I think it’ll be clearer after the BUILD conference next month.

 

HoloLens Technology – Eye Tracking

 

??????????????????
One of the key technologies behind virtual reality and augmented reality is eye tracking. VR needs precise eye tracking to adjust the perception of the images when the viewer is looking at different position. Also it needs eye tracking to tell exactly what and where to show the images for each pupil to generate stereoscopic 3D illusions. HoloLens works differently than VR, but eye tracking is part of core technologies in Hololens. In this post, I’ll talk about eye tracking technology, and what role it plays in HoloLens.

Eye tracking is a method that detect the point of gaze for each individual pupil. Over the years, many techniques has been developed, including dynamic analysis of the eye images and dynamic analysis of the electrooculogram. Eye tracking has been recently used in human-computer interaction, like gazing to hover on the screen, moving pupil to scroll. Marketers user eye tracking to figure out their ads efficiency: marketing is all about eyeballs after all. However when we talk about VR or AR, eye tracking is in the front and center. Without eye tracking, without HoloLens.

Some Important Information About HoloLens

Now we know that HoloLens’ using natural user interface instead of keyboard and mouse, please check sources: here and here. Those user interfaces are: gestures, voice and gaze. Needless to say, without eye tracking, gaze wouldn’t work. The fact it’s part of user interface implies that eye tracking need to be swift and accurate.

More importantly, eye tracking is part of the HoloLens light engine. As we already know that the HoloLens light engine not only generates lights, but also projects them to the eyes. It needs to know the exact position of both eyes, and where they are looking at. A hologram can’t be created without knowing the point of view, and the light engine wouldn’t know where to send the lights if it doesn’t know where the eyes are.  Eye tracking can significantly reduce images that are generated at each moment. apparently it would be very inefficient to render the entire hologram at every moment, that’s a lot more information than a stereoscopic 3D image.

HoloLens: Release Date (Updated)
HoloLens: Price Estimation

Some Business Ideas with HoloLens – (15) Holo-Rock Star
Some Business Ideas with HoloLens – (14) Holo-Classroom, Holo-Conference
Some Business Ideas with HoloLens – (13) Holo-Home Design

 

 

 

Why HoloLens’ Most Remarkable Achievement Is Its Display

 

mr3
There are a few things that separate HoloLens from other AR/VR devices: Magic Leap, Sony Morpheus, HTC Vive, or Oculus Rift to name a few. The Kinect gesture recognition and 3D world mapping are far and away the best in class. Kinect has been in market for a few years. It has been improved in size and response time, especially coupling with HPU in HoloLens. People who have tried the new implementation on HoloLens are generally impressed. More Importantly, HoloLens is built on top of Windows Holographic platform, which will be available on all Windows 10 devices. There is a huge advantage with an established platform, I will get to that in another post. In this post, I’m talking about another technology in HoloLens that makes it stand out from the others: the display technology.

Yes, I am aware that we haven’t seen the product yet, and we don’t even know what is behind the display technology in HoloLens. I think it doesn’t really make a different whether we have seen the final result or not, I believe the final product can only be better. As to the technology details, we may never know even with a product in hand, but it doesn’t matter, I have enough information to say things that I am sure of.

HoloLens: Price Estimation

Display Technology: HoloLens and VR

First, I want to compare HoloLens’ display with the VR displays from Sony or Oculus. Almost all VR displays are using the same technology, with some variation of course. There is a rectangular screen in front of viewer’s eyes, LED or LCD. It displays high resolution images on the screen. Between the screen and viewer’s eyes, there are a pair of glass to make it feel like viewing from a distance. Because the viewer’s eyes are totally covered, there is a feeling that you are separated from the world, immersed into the virtual world. Immersive-ness is something that VR vendors are tooting all along. Unfortunately, in my opinion, immersive-ness has very little to do with its display technology itself. It has, however, a lot to do with the fact that the viewer’s eyes are entirely covered, which by itself is not much of a technology to talk about. If there is something to the VR technology is that it appears to have an impressive field of view, but I am not sure how much a difference between the two. Also the 3D effect from VR is far less realistic than the holographic 3D from HoloLens. With holographic 3D, you can move you head to see an object in different angle. With stereoscopic 3D, no matter what angle you look at the same object, you see the same image. Look at the picture above, with HoloLens it’s hard to tell which object on the table is real, which one is virtual.

The major gripe I have with VR headsets is how they look. I know they are not meant to be worn walking around, but wearing a box is too alien-ish to me. I am not talking about the bulkiness, which can be improved over time, but in its finest and smallest form possible, I still don’t see how it can look good. Sony’s headset is probably the best looking one. Lets make it 2 times thinner, would you say it looks fine? The problem lies here: 1. It has to have a distance from the eyes. 2. It needs a big field of view. That’ why it has to have this size, it can’t be much smaller. Because of technology difference, with AR, like HoloLens, the lens can be much closer to the eyes.

HoloLens: Release Date (Updated)

Display Technology: HoloLens and AR

Now, lets look at other AR display technologies. Microsoft calls its HoloLens mixed reality, but it is actually more AR than VR. When field of view is improved, it will eventually cover entire spectrum from AR to VR. First of all, in AR field no one else has a published demo yet. So we have no idea how good they are. One thing I know: They are all based on a variation of one technology, that is sending light directly to the eyes, whatever that is called (some call it Light Field). Another thing I know: This technology is very hard. Microsoft has worked on this for at least 4 years as far as I know(maybe longer), with many top scientists in the field, and spent tons of money to have demo units as some have seen in January. I have a feeling that this is not something everyone can fondle with, which explain why non of those other AR companies have a demo, or remote releasing date yet.

Some Important Information About HoloLens

The Big Picture

There is a bigger problem in AR and VR together other than technology: the platform problem. I will talk about that later.

Update: Phil Spencer, head of games in Microsoft, has just said in an interview that the technology under HoloLen is more challenging than VR.

In the holographic mixed-reality space, there are fewer players. In some ways, it’s a little more technically challenging — not to take away from any of the challenges of VR.

Some Business Ideas with HoloLens: (1) Holo-IKEA
What Is Mixed Reality, Really?
Some Business Ideas with HoloLens – (4) Holo-Repair and Holo-AAA
Some Business Ideas with HoloLens – (3) Holo-Support
Some Business Ideas with HoloLens: (2) Holo-Architect

 

Xbox + HoloLens: How Can They Work Together?

 

xbox2game1
Yesterday(Mar 3, 2015), Phil Spencer, head of gaming of Microsoft had announced that games, along with applications will be cross all Windows 10 devices, which includes Xbox, PCs, phones and HoloLens. People are wondering, what’s the connection between Xbox and HoloLens. It’s a topic that is too tempting to pass.

Some Important Information About HoloLens

HoloLens, as we already know, is a stand alone PC. With its Cherry Trail CPU, it should play general mobile games just fine. It will run some light non-indie game as well. But for CPU and GPU intensive games, it has to be on Xbox. Now the question: For holographic games to work on Xbox, HoloLens has to be used. Just how do the two work together?

HoloLens: Release Date (Updated)

Phil Spencer had already hinted in January that Xbox could stream games to HoloLens: So basically HoloLens works as a display to replace the TV, because TVs can’t show holograms. Interesting question is: Can HoloLens also work as an input device to replace the Xbox controller? Lets look at it.

Based on materials from Microsoft, HoloLens only support gestures, voice and gaze. No mouse, no keyboard, no touch. Are those enough to replace the Xbox controller? Assume there is no latency to worry about. Things like grabbing the gun, shoving the dirt, throwing the torch would be hard to do. The idea of virtual Xbox controller probably won’t work because there is no haptic response to the fingers, it’s going to be too slow.

My guess is: HoloLens will work as a display and input device to interact with the holograms, also to provide 3D mapping for the room, but for many games, Xbox  controller is still required.

 

HoloLens: Price Estimation
Some Business Ideas with HoloLens – (8) Holo-Movie
Some Business Ideas with HoloLens – (7) Holo-Escort
Some Business Ideas with HoloLens – (6) Holo-Online Store