January 28, 2020

PC + HoloLens: How It Works

PC n HoloLens

In a previous post, I have done some research based on the Microsoft HoloLens Promo video, and concluded that HoloLens can and very likely will work with other PCs as an extra input and output device. I didn’t, however, answer this question: How would that work? In this post, I will try to address this question, basing partially on what I know, and partially on logically induction(aka educated guess).

Xbox + HoloLens: How Can They Work Together?

If HoloLens only works as a display, things get easier. HoloLens just need to implement standard protocols as regular monitor does, making sure it is discoverable as a normal display device, regardless how different a monitor and a HoloLens are. Two things makes this more complicated than first thought: One is the hologram display, there isn’t a ready made streaming format or protocol to stream a hologram image or video. So they have to be done from the ground up. And second, user input (hand gestures, voice, etc.) has to be fed back to the PC. This will be a totally different mechanism than traditional streaming.

HoloLens: Some Early Bits for Developers

Regardless, I believe strongly that this mechanism it already in Windows 10 under Windows Holographic subsystem. That’s the only way to explain everything.

Some Business Ideas with HoloLens – (8) Holo-Movie
Some Business Ideas with HoloLens – (7) Holo-Escort
Some Business Ideas with HoloLens – (6) Holo-Online Store

HoloLens: Where Is My Keyboard?



Many people had expressed concerns over text input on HoloLen. Microsoft has said (also please read the Wired article) the user interface for HoloLens are: gestures, voice and gaze. None of those do anything for text input. If all you do is playing games, browsing the web, I would assume the Windows on screen keyboard is still there. However if you want to do anything serious, like what you do in the office, that keyboard wouldn’t work. In the HoloLens promo video (below), office workers who are wearing HoloLens also have a real computer, with a keyboard. There are many cases where you just want to use HoloLens not only as a display, but also a full PC. HoloLens is a full Windows 10 PC after all. We want to have fun with it, we also want to be productive with it. A full featured physical keyboard is definitely a necessity.

There two ways this problem can be solved: One is to have a wireless keyboard, either WiFi or Bluetooth, paired with HoloLens. There really no need to do anything speciall, as HoloLens is a PC. One problem with that solution is you have to have a table to put the keyboard on. That is the case most of times. Still, there are times you may not have a table, like in the kitchen, you want to send someone an email. Luckily, there is a better solution: a virtual keyboard. A virtual keyboard floats in the air. It shows up when you need it, hides itself when you don’t. It looks exactly like a real keyboard, and it gives sound feedback. Only when you type, it feels typing in the air. I think it would work much better than the on-screen keyboard. Also, there can be a virtual pen that you can hold in hand, looking like the Surface Pen. If you want a virtual mouse, that can be done the same way.

I am not sure if things like virtual keyboard, virtual mouse or virtual pen will be implemented with HoloLens, if they won’t, that’s an opportunity for developers. There are plenty use cases.