November 13, 2019

The Possibility of Streaming from Xbox to HoloLens






Yesterday, I read some great article: Can Virtual Reality Cut the Cord? Very Informative. The article addresses a common unsolvable issue: a cord between PC and the VR HMD. A possible route to remove the wire is to use wireless streaming. One major issue with that is latency. From the article, 20ms or less is the ideal latency. Because of the nature of the technology, VR has to pursuit high definition (1440p), high refresh rate (90Hz), which generates gigabits of data every second, streaming under current technology wouldn’t be feasible. Gigabits/sec WiFi is still a couple of years away to be commonly available and economical.

This raises the same questions to my previous articles:

PC + HoloLens: Would It Work?
Xbox + HoloLens: How Can They Work Together?

So, I need to take another look.

Initially, I didn’t take latency into account. I based my writings on Phil Spencer’s comments about how Xbox can work with together. He said streaming is a possibility.  Also, I have found in Microsoft January 1st demo/promo video, a PC does work with HoloLens synchronously, if that demonstrates a real use case.

In doubt of my previous theories, I did a similar calculation to the article mentioned at the beginning of this post.

Remember HoloLens is very different from VR technology. VR uses traditional display, but it is very close to the eyes, so, high definition is required and high refresh rate is required. Wait a minute, 1440p is a massively high resolution for a little screen that the VR headset has. The reason is the screen is too close to the eyes. Even under this resolution, pixelization is still visible. Another thing is refresh rate, 90Hz is ridiculously high.

Lets pause for a second, 90Hz refresh rate, why? We all know human eyes can’t see faster than 24frame/sec. The movies are still shot at this rate. To understand that, notice there are two things into play: 1. TVs or monitors are digital devices, they draws on screen pixel by pixel, which are slower than light projection on to a screen in the theater, therefore choppiness appears in very fast motions, like video games. 2. TV and monitors are far closer to our eyes than the screen in the theatre. This the second factor is amplified in the VR headset. So, VR has no choice but pursuit a high fresh rate.

For Corporate IT, HoloLens is Another Mobile Device To Manage

Next time you look at some VR specs, you see, 1440p display, 90Hz refresh rate, don’t panic. It’s not as good as it sounds, that’s minimum requirement rather than super high specs.

HoloLens, on the other hand, doesn’t have the issue with VR. While, doesn’t HoloLens’ lenses are as close to the eyes as the VR does? Yes. But the lenses on HoloLens are close to the eyes doesn’t make the images closer to the eyes. As a matter of fact, there is no image generated outside the eyes. What human eyes see is the same light we see in the real world. The real world doesn’t need a fresh rate, right?

As a result, HoloLens doesn’t need to have super high resolution, and refresh rate to generate high quality videos/images to make it look good. HoloLens has claimed HD resolution. I would assume that’s 1080p, and lets also assume 30Hz refresh rate and also assume a 24bits of color depth, how much data would it need to transfer from Xbox (or a PC) to HoloLens? 1.5Gb/sec. For 720p resolution, it’s 660Mb/sec. According to the afore mentioned article, “Consumer WiFi routers can handle about 100 megabits/sec.  Even the highest-end industrial routers max out at about 800 megabits“.  It appears, if the data is uncompressed, it needs expensive wireless routers to handle the data transfer for even the low HD video to be streamed to HoloLens. So it looks like it will need some fast compression algorithm to handle it. I am not really sure.

Phil Spencer had said (from Windows News,

I think gaming will be important. Specific scenarios with the Xbox, we’re thinking hard about. People could ask about streaming solutions. Could I use it as a display for my Xbox? We don’t have answers to any of those things, but know it’s all part of the same organization.

Does he have an answer now? We will find out soon.


What Is Mixed Reality, Really?

Does HoloLens Have to Be Worn on the Head?

HoloLens Misconception – 3D