April 12, 2021

The Magic behind the E3 HoloLen Demo



I am still fascinated by Microsoft E3 HoloLens Minecraft demo,  shown on stage during the E3 event on Monday, check the video.  How was it done, and what do the two people play together? Here is my interpretation.

It is the Minecraft game designed specifically for HoloLens. The setting is like the above picture. Lydia is holding a Surface, and Saxs is wearing a HoloLens, with Xbox controller in hands. They are playing the same Minecraft game together, both see the same game, and both have control of the game. According to Phil Spencer, the game is running on HoloLens.

If the game is running on HoloLens, how do the Surface and HoloLens communicate with each other? Wirelessly, as it appeared, and likely just WiFi, which means the game that is running on HoloLens is streamed to the Surface. How else can it work together like this? This is significant if you think about it for a second: game streaming is extremely challenging on WiFi because of bandwidth limit, and gaming requires no delay. If that’s really happening, I have no reason to doubt, it’s really exciting. If streaming from HoloLens to PC works, then it should work the other way around, and Xbox to HoloLens too. Wirelessly, ladies and gentlemen. This is the first time I have seen it. We already know Xbox to PC streaming is already working, but that’s through wired connection, but this is WiFi. I am afraid I am missing something, I hope I am not.

A couple month ago I wrote about how PC and HoloLens work together, and how Xbox and HoloLens work together. See the following links: PC + HoloLens: How It Works, PC + HoloLens: Would It Work?, Xbox + HoloLens: How Can They Work Together? It appears this Minecraft demo is more evidence they can work together, wirelessly.

The next thing is Xbox controller works with HoloLens flawlessly, this is important because this was one of gamer’s main concern over HoloLens. This shouldn’t be surprise now, since if Xbox controller works with Oculus Rift, what’s the chance it won’t work with HoloLens? However the confirmation is important. I would like to extend this: a keyboard or a mouse shouldn’t be a problem, right?  If you remember my post: HoloLens: Where Is My Keyboard?

The third thing come out of this demo is: the same game can play as 2D and 3D. Why is this important? You ask. Because that means all current Xbox games can be streamed to HoloLens in 2D form, and they can be refactored for HoloLens to be played in 3D and 2D mode.

I like this demo, it revealed so much things I have guessed but couldn’t confirm.


HoloLens: Field of View (FOV) Collections