November 16, 2018

Why HoloLens Is Not Kinect

holo-kinect1g
Since HoloLens was accounced back in Jan 21st, many articles on the Internet have referred Kinect while reporting on HoloLens in various ways. Most of them tend to put a shade of skepticism over HoloLens from the comparison. In this post, I will try to give a little inside look at why this comparison comes so often and why it’s still invalid.

Sure, both HoloLens and Kinect use hand gesture for interaction, both use voice command for action control. Both are developed by the same group of people in Microsoft, and both happen to be amazing technology with no peer in the world. Indeed, HoloLens has a couple of Kinect sensors inside it, miniaturized of course. However, amazing technology doesn’t always lead to great consumer adoption. Despite great initial sales, Kinect’s momentum hasn’t been able sustained, which leads some people in the media draw comparison with HoloLens in the market department. I think people who draw comparison between the two are missing the point.

Kinect is a sensor, it does a lot of things, like body movement recognition, field depth mapping, etc., but it is still a sensor. HoloLens is a full featured computer, Kinect is just one of the its many sensors, but that’s not my point. HoloLens is a display device. As I have pointed out in another post, what distinguishes Hololens from anything else is its display technology. It breaks the traditional screen barrier: Contents are no longer contained in a fix-sized screen, not to mention its 3D holographic capability. But this is still not quite my point.

In essence, Kinect is an input device. HoloLens is a output device first, input device second. And that is my point. So, what does it mean? What difference does it make? Lets take a closer look (I know I have used this sentence too much, but I am not tired yet).

HoloLens Technology – 3D World Mapping

Kinect is not a great input device, neither is HoloLens. To understand this, please look at another post of mine: Natural User Interface: Is Natural Always Better? Sure, Natural User Interface (NUI) is great concept, however, no matter how we label it, in the presence of mouse, keyboard and game controller, every other ways of interactions look inferior, that includes hand gesture, voice, gaze and touch. Those NUIs are only valuable when you don’t have access to a mouse, keyboard or game controller, like the way you use a phone, a tablet on the go. Whenever you’re siting at a desk, a mouse is always the best way of human-computer interaction. I think every gamers or game developers agree with me that use gesture to play a hard core game is close to impossible. So, basically, Kinect is trying to use an inferior input method to replace great input methods. And we know the results.

HoloLens is an input device as is Kinect, but its main function is an output device: a display. It is its own display, it can also be a display for a PC, a display for Xbox. The key point here, it doesn’t exclude other input methods: mice, keyboards and game controllers. You can still use a PC the way you use it before, HoloLens just adds another great display. You can play games on Xbox the way you always do, with a game control in hand, the game can be streamed to a giant display on Hololens. Even you use HoloLens as a standalone machine, you can still use a physical keyboard, a mouse, or a game controller. What does it mean, really?

HoloLens Technology – Eye Tracking

It means every application that works on a PC should work with HoloLens, because you consider HoloLens as a mere display. Every Xbox games will be instantly available to HoloLens, I said every.

Yes, I heard your argument: Why do I use HoloLens only as a display? I buy HoloLens for 3D holograms. I believe there will be 3D holographic games, and 3D holographic applications. DirectX has been used for game development, I believe having those games appear as 3D holograms wouldn’t be a major problem, but my point is even without those 3D stuff, HoloLens gives you a giant display which is can be a huge boost to your PC or gaming experience.

In summary, Kinect is a input device, which is its weak point, in my opinion. HoloLens still has the same input device function, but it doesn’t exclude other input methods. HoloLens’ display is what makes it great: 3D or 2D.

 

PC + HoloLens: Would It Work?

Why HoloLens’ Most Remarkable Achievement Is Its Display

Xbox + HoloLens: How Can They Work Together?

In 2019, You May Not Need a Smartphone, Or a Tablet

vision

Imagine, there will be no screen.

Clearly, I have touched this topic more than once(here, here, here). I have wrote about using Hololens as a display for Xbox, PC and gone as far as fascinating a miniature HoloLens on the phone and a giant HoloLens on the wall, but I have never put things together to imagine what it is like when there will be no screen on a phone, a tablet, a laptop or a PC.

Some Important Information About HoloLens

Lets watch a video created by Microsoft Office Lab, called Microsoft Office Labs Vision 2019. There are different versions, this version was made in 2009, the original title states Vision 2019. No explanation required.

Now forget about whether this will be true in 2019, for a second, think about what it’s like if computer screens are gone. If there is no screen, then any surface can be a screen: the coffee table, the desktop, the cubical wall, the office wall, the teacup, the newspaper. No, that’s not good enough, that’s so 2009. Today, in the context of HoloLens, a screen can exists in the air, or even in the vacuum, the whole concept of screen is gone.

HoloLens: Release Date (Updated)

The essence of the HoloLens is NO SCREEN. If you still don’t get it. Lets continue. With HoloLens, the new law of existence is:

M{your world} = I{yourself} + R{the real world} + D{the digital world}

In the terms of Windows Holographic, M is called mixed reality, I is called Human, R is called World, D is called Holograms. And remember R and D are almost indistinguishable,  I didn’t make this up, this is real.

I’d like to quote Craig Mundie’s talk in 2011 from The Sydney Morning Herald,

When the user looks at the phone, instead of seeing a screen it can beam individual rays of light into your eyes right on your retina … [so] you can look at your phone and see HDTV. I don’t know whether the big screen tablet pad category is going to remain with us or not.

Standing in the spring of 2015, a few months before HoloLens launch, when we look back at what Mundie said in 2011, you know what he said is starting to be realized: The only difference between a phone, a tablet, a laptop is the screen size. When the screen no longer exists, what’s the difference between them? Nothing. Well, CPU, the computing power, you might argue. In the world of cloud computing, more and more computing power are consumed on the cloud, and you don’t have to carry it with you. For the local computing, I am sure HoloLens is a more powerful computer than any smartphones, and majority of tablets. For office computing or home computing, you can have a powerful PC on the desk, HoloLens works as a display.

What Is Mixed Reality, Really?

The only problem is the size of the HoloLens. But remember, HoloLens has two Kinects inside, an entire computer, a light engine, lenses, sound systems and a bunch of sensors, it weighs 400grams. Who can imagine this two years ago, when you can’t even wear one Kinect on the head? We heard an entire Intel computer can reside in a thumb sized stick. Could HoloLens turn to a mere eyewear in 2019? I am hopeful.

In 2019, you may not need a smartphone, or a tablet.

(Just for argument’s sake, some say, ODG already got AR glass as small as an eyewear. Lets me say this, they are miles away from HoloLens, I will talk about this later.)

Some Business Ideas with HoloLens – (15) Holo-Rock Star
Some Business Ideas with HoloLens – (14) Holo-Classroom, Holo-Conference
Some Business Ideas with HoloLens – (13) Holo-Home Design

 

 

PC + HoloLens: How It Works

PC n HoloLens

In a previous post, I have done some research based on the Microsoft HoloLens Promo video, and concluded that HoloLens can and very likely will work with other PCs as an extra input and output device. I didn’t, however, answer this question: How would that work? In this post, I will try to address this question, basing partially on what I know, and partially on logically induction(aka educated guess).

Xbox + HoloLens: How Can They Work Together?

If HoloLens only works as a display, things get easier. HoloLens just need to implement standard protocols as regular monitor does, making sure it is discoverable as a normal display device, regardless how different a monitor and a HoloLens are. Two things makes this more complicated than first thought: One is the hologram display, there isn’t a ready made streaming format or protocol to stream a hologram image or video. So they have to be done from the ground up. And second, user input (hand gestures, voice, etc.) has to be fed back to the PC. This will be a totally different mechanism than traditional streaming.

HoloLens: Some Early Bits for Developers

Regardless, I believe strongly that this mechanism it already in Windows 10 under Windows Holographic subsystem. That’s the only way to explain everything.

Some Business Ideas with HoloLens – (8) Holo-Movie
Some Business Ideas with HoloLens – (7) Holo-Escort
Some Business Ideas with HoloLens – (6) Holo-Online Store

PC + HoloLens: Would It Work?

 

We know that HoloLens is a full PC. It is supposed to be running on Intel Atom CPU(Cherry Trail to be specific). It should run any Windows universal app from Windows Store, and theoretically any traditional Windows software, for example AutoCAD. For one thing, Atom CPU is not powerful enough to run CPU intensive applications like AutoCAD. For another, it’s always better to leave heavy computing to a powerful CPU and let HoloLens work as input and output device. Would this work? And How?

Xbox + HoloLens: How Can They Work Together?

To answer these questions, I revisited Microsoft’s original promote video,  looked closely at the motorcycle design portion of the video, and find out the answer. The answer to the first question is yes and I will guess how. Well, I am assuming the video content about the motorcycle part is not pure illustrative, but captured by HoloLens. Or at least this video indicates how they can work together. I have great confident in the first scenario, and I have little doubt in my conclusions. Lets get to it.

PC n HoloLens

First I want you to look closer at this picture (above) from the promo video. I have marked three import points to watch for: The motorcycle AutoCAD(maybe another software) 3D view on the desktop PC monitor, the holographic motorcycle on the table and the designer’s right hand. The setup is this: The designer is sitting in front to desktop computer, using traditional keyboard and mouse. He is designing a motorcycle on his desktop, as the monitor is showing. In the mean time, he is also wearing a HoloLens, therefore, beside the monitor on the left, he can see a hologram of the same design in 3D. Important thing is: While watching the video, the designer’s right hand, the 3D view on monitor and the hologram all in sync. I have cut this portion from the original video, and make it playing at 50% of normal speed. Not really slow motion, but makes it easier to see the moment. Now watch this piece of the video below, pay attention to about 3 sec mark.

HoloLens: Release Date (Updated)

What the above video has shown is the PC and the HoloLens works in collaboration. The use case displayed in the video is HoloLens works as a second display. The designer still does what he normally does on a PC, and the keyboard, mouse, monitor is still there. The system that is running on is probably a powerful workstation. At the same time, the designer also see the hologram in 3D. Interesting thing is the hologram can also be manipulated by the mouse. Furthermore, HoloLens is also used as a input device, for example, the designer can type on the holographic console beside his left hand. I would assume his hand gesture works as well.

I think this will be the most common scenarios in an office setup with HoloLens. HoloLens works as an extra input/output device, instead of replacing keyboards, mice and monitors, enhance them. 2D or 3D is not really the point, the point is: the content that is traditionally contained in the monitors, now break free. You can have 10 screen scatters everywhere you see. You can put an Office window far way on the wall across your coworkers, only you can see it.

In next post, I will try to answer the question: How it works.

HoloLens: Price Estimation

Some Business Ideas with HoloLens – (7) Holo-Escort
Some Business Ideas with HoloLens – (6) Holo-Online Store
Some Business Ideas with HoloLens – (5) Holo-Tutor

 

Why HoloLens’ Most Remarkable Achievement Is Its Display

 

mr3
There are a few things that separate HoloLens from other AR/VR devices: Magic Leap, Sony Morpheus, HTC Vive, or Oculus Rift to name a few. The Kinect gesture recognition and 3D world mapping are far and away the best in class. Kinect has been in market for a few years. It has been improved in size and response time, especially coupling with HPU in HoloLens. People who have tried the new implementation on HoloLens are generally impressed. More Importantly, HoloLens is built on top of Windows Holographic platform, which will be available on all Windows 10 devices. There is a huge advantage with an established platform, I will get to that in another post. In this post, I’m talking about another technology in HoloLens that makes it stand out from the others: the display technology.

Yes, I am aware that we haven’t seen the product yet, and we don’t even know what is behind the display technology in HoloLens. I think it doesn’t really make a different whether we have seen the final result or not, I believe the final product can only be better. As to the technology details, we may never know even with a product in hand, but it doesn’t matter, I have enough information to say things that I am sure of.

HoloLens: Price Estimation

Display Technology: HoloLens and VR

First, I want to compare HoloLens’ display with the VR displays from Sony or Oculus. Almost all VR displays are using the same technology, with some variation of course. There is a rectangular screen in front of viewer’s eyes, LED or LCD. It displays high resolution images on the screen. Between the screen and viewer’s eyes, there are a pair of glass to make it feel like viewing from a distance. Because the viewer’s eyes are totally covered, there is a feeling that you are separated from the world, immersed into the virtual world. Immersive-ness is something that VR vendors are tooting all along. Unfortunately, in my opinion, immersive-ness has very little to do with its display technology itself. It has, however, a lot to do with the fact that the viewer’s eyes are entirely covered, which by itself is not much of a technology to talk about. If there is something to the VR technology is that it appears to have an impressive field of view, but I am not sure how much a difference between the two. Also the 3D effect from VR is far less realistic than the holographic 3D from HoloLens. With holographic 3D, you can move you head to see an object in different angle. With stereoscopic 3D, no matter what angle you look at the same object, you see the same image. Look at the picture above, with HoloLens it’s hard to tell which object on the table is real, which one is virtual.

The major gripe I have with VR headsets is how they look. I know they are not meant to be worn walking around, but wearing a box is too alien-ish to me. I am not talking about the bulkiness, which can be improved over time, but in its finest and smallest form possible, I still don’t see how it can look good. Sony’s headset is probably the best looking one. Lets make it 2 times thinner, would you say it looks fine? The problem lies here: 1. It has to have a distance from the eyes. 2. It needs a big field of view. That’ why it has to have this size, it can’t be much smaller. Because of technology difference, with AR, like HoloLens, the lens can be much closer to the eyes.

HoloLens: Release Date (Updated)

Display Technology: HoloLens and AR

Now, lets look at other AR display technologies. Microsoft calls its HoloLens mixed reality, but it is actually more AR than VR. When field of view is improved, it will eventually cover entire spectrum from AR to VR. First of all, in AR field no one else has a published demo yet. So we have no idea how good they are. One thing I know: They are all based on a variation of one technology, that is sending light directly to the eyes, whatever that is called (some call it Light Field). Another thing I know: This technology is very hard. Microsoft has worked on this for at least 4 years as far as I know(maybe longer), with many top scientists in the field, and spent tons of money to have demo units as some have seen in January. I have a feeling that this is not something everyone can fondle with, which explain why non of those other AR companies have a demo, or remote releasing date yet.

Some Important Information About HoloLens

The Big Picture

There is a bigger problem in AR and VR together other than technology: the platform problem. I will talk about that later.

Update: Phil Spencer, head of games in Microsoft, has just said in an interview that the technology under HoloLen is more challenging than VR.

In the holographic mixed-reality space, there are fewer players. In some ways, it’s a little more technically challenging — not to take away from any of the challenges of VR.

Some Business Ideas with HoloLens: (1) Holo-IKEA
What Is Mixed Reality, Really?
Some Business Ideas with HoloLens – (4) Holo-Repair and Holo-AAA
Some Business Ideas with HoloLens – (3) Holo-Support
Some Business Ideas with HoloLens: (2) Holo-Architect