April 12, 2021

HoloLens Variations?



Fig. 1

Fig. 1


When I read a VentureBeat’s article “Microsoft patent describes use of floating holograms to guide gesture control of mobile devices” yesterday, I was instantly attracted. Part of me thought: wow, something major is happening. So, I read through the patent “Holographic Interaction Device” submitted by Microsoft. My excitement has faded a lot, but still, if this is something related to upcoming products, that still makes a small revolution in computing. Read on to see if you agree.

The reason I set my expectation so high is, as you already know I am writing about HoloLens almost exclusively, HoloLens is probably one of the biggest breakthrough in modern technology (not just computing). And if you are aware, I have wrote these two pieces not long ago:

Does HoloLens Have to Be Worn on the Head?
In 2019, You May Not Need a Smartphone, Or a Tablet

Honestly, I was hoping HoloLens technology would propagate to other forms beyond head mounted, because there is no reason it wouldn’t. After reading the patent, I feel it is not really what I have hoped, but it can be a stripped down version of it, or a first step to that direction. You know, that’s amazing enough.

Before we get too carried away, let me just say, we are not talking about a product, it’s merely a patent. Sometimes a patent indicates a future product, sometimes it doesn’t. With that caveat in mind, let get to it.

The patent was filed on Oct. 15, 2013, and published on April 16, 2015. So it is very current.
HoloLens, Recommended Readings – Business Ideas Collection


What is it?

The patent is about using holograms to assist user interaction with computing devices. That’s the main claims there. Anything else is generic instead of specific to certain type of devices. That’s why they use very generic terms like computing device instead of a computer, holographic device, instead of specific type. Even though the drawings are obviously using Microsoft Surface as an example (Fig. 1).


Fig. 2

Fig. 2

The system concerning the patent has a few modules (Fig.2). Like all patent claims, they use generic terms to cover all bases. As it is stated in the patent, any of the those modules can have different implementations. You can think of this as a common laptop setup, with the Gesture Module and Holographic Device added. The trick obvious lies in the last two.

The problem it is trying to solve or the amazement it is trying to add to a regular computing system is: adding a layer holographic images (can be 2D or 3D) in the vicinity of where user interaction takes place. The holograms can act as a directional hint to guide user to what to do. Or extend functions of physical input devices, for example, add virtual function keys around a small physical keyboard.


Fig. 3

Fig. 3


Fig. 3 depicts examples of that idea. The holograms shown in Fig.3 are a virtual Xbox controller, a directional image that guides user’s actions, some hover points hinting what user to do next, and so forth. These are just a tiny set of what is possible. I will get to that later.


What is so interesting about that?

One thing I noticed reading through the patent is many things are very generic. Logically because the Holographic Interaction Device is NOT limited to one type of device. It is clearly stated in the patent that the computing device can be in many different forms: a laptop, tablet, phone, music player (yes Zune image is shown in the patent drawings), Xbox, even wearables. This is obviously very very interesting, I will tell you later if you don’t already know.

The patent also mentioned there can be many different setup. For example, the keyboard can be a physical one, or virtual one (I wrote about it before), or none. The input device can be a keyboard, a mouse, touch, acoustic (voice), or gesture. That covers almost every user interactions we have or will have now.

The patent doesn’t tell how the holograms are generated. It does mention holographic recording mechanism, and a light source in the forms of laser or LED. However I failed to make sense of it. But in any case, it outputs the holograms.

A new concept called Near-Field-Interface(NUI) is mentioned a least 8 times in the patent, which refers to “close but not touch the object”. Sounds familiar, right? Here is a quote:

a near-field user interface (NUI) may be supported in which gestures made proximal to but not touching either one of the computing device 102 and/or input device 104 may be used to initiate operations of the computing device 102 using a “legend” output as a hologram that indicates how this interaction is to be performed.


So, what does all these mean?

To anybody else, this patent is just another one among hundreds/thousands new applications from Microsoft, but to me this doesn’t seem to be regular one. I feel there is a strong connection to HoloLens, at least being part of Windows Holographic. As I said at the beginning, in the case when a full HoloLens like implement is not yet possible because of size of it, and most importantly because of limitation of small field of view hence the light source is too far away from the eyes, as oppose to head mounted system in HoloLens. Even with that, it is still massively interesting.

Imagine your next Windows Phone or Surface has this feature in. A virtual key board hologram can be floating out of your screen, well, do I have to say more? Holographic images can appear anywhere over/beside, on the screen or the keyboard. The holograms are not  as big and splendid as the ones on HoloLens, but enough to be an extended user interface. There are many amazing uses of it if that’s true, but I feel I need another post for it.

Before ending this post, I want to tell you why I want to make the connection between the patent and all these I have talked about in this post. As many of you already realized, Microsoft had skipped a flag ship Windows Phone for a year. This must be a major mistake if there isn’t something more than amazing to justify it. In the meantime, a normal Surface release cycle had been 6~8month. SP3 was released June 2014, it’s two month short of a whole year. I know Surface 3 will be on sale this may 5th, but we know the major attraction is still Surface Pro. The only reason I can imagine that is what I have wrote in this post. As a matter of fact, I have been thinking about this since HoloLens was revealed.

I have high hope for Lumia 940 and SP4.


UAP, HoloLens, Recommended Readings – Developer’s Collection

Some Business Ideas with HoloLens – (20) Holo-Golf Swing

Is HoloLens Going to Be a Commodity?