June 27, 2017

HoloLens Technology – 3D World Mapping

map1 map2


In the other posts, I have discussed technologies behind HoloLens: here and here. I said the most remarkable technology behind the HoloLens is its display. In order for HoloLens display technology to work correctly, 3D mapping plays a key role. In this post, I will try to explain how 3D mapping gives the power for developers to create interactive holographic applications and games, also how 3D mapping helps to generate realistic mixed reality, rather than an overlay.

In addition to a pair of RGB cameras, there are also a pair of depth cameras on HoloLens. Those depth cameras receive infrared rays (originated from IR emitter on the front of HoloLens), and map the 3D space around the wearer. In Windows Holographic there three components: the human(HoloLens wearer), the world(everything surround the human, aka the real world), and the holograms(aka the virtual world). The function a of the depth sensor is to fill in the world, tell the human’s position, orientation in the world, recognize human hand gestures. The first one is what we are focusing now.

Some Important Information About HoloLens

With the 3D mapped “world”, developers can write world-aware applications and games. For example the Holo-Ikea idea I wrote in another post: When the user is moving around the furniture, the app is aware of the room dimensions, and what is already in the room. The app can also automatically rearrange things around to optimize the space. You won’t run into the problem where you got the furniture delivered to your home, only find you measured it wrong.  The Minecraft game is a typical use of the world 3D map to adjust the play setup in games.

HoloLens: Price Estimation

Another important use of 3D mapping in HoloLens is: It makes mixed reality possible. I have read articles on the Internet saying that augmented reality overlays virtual object on the real world, there are different algorithms, of course. That’s not HoloLens. Look at the second picture above: the plumbing demo. Notice the two circled 3D arrows surrounding the pipes, they don’t look overlaid on top of real world, they are mixed in the real world. Think about it for a second, how is that possible? The real world is just one layer of image, how do you insert a virtual object into it, not top of it? I am not sure if the arrows cast shadows on the wall, but this is enough. The answer to the question is 3D world mapping. Without 3D mapping, the girl’s hand wouldn’t be in front of the arrows, but behind or merged into the virtual objects, because the hands are part of the real world image, if the arrow is in front the pipe, it has to be in front of the hand. 3D world mapping allows application to insert virtual objects into the real world to create mixed reality.

Some Business Ideas with HoloLens – (5) Holo-Tutor
Some Business Ideas with HoloLens – (4) Holo-Repair and Holo-AAA
Some Business Ideas with HoloLens – (3) Holo-Support
Some Business Ideas with HoloLens: (2) Holo-Architect
Some Business Ideas with HoloLens: (1) Holo-IKEA