January 18, 2018

Jerry Nixon: “Why you must build a HoloLens app”

 

jerry1

 

Microsoft’s Developer Evangelist Jerry Nixon wrote a blog post: Why you must build a HoloLens app, on March 27th. I don’t know how, but I missed it. Today, I am reading his blog together with you. I will also try to read between his lines, hopefully we can get something we didn’t know before.

I haven’t watched Channel 9 videos often recently, so, I am not really familiar with Jerry. Only know him after watch his Developer’s Guide to Windows 10 Preview (with Andy Wigley). Andy has been a regular on Channel 9. Anyway, important thing is Jerry is from Microsoft, and he know pretty much everything regarding Microsoft’s plans in UAP and HoloLens development. It all depends on how much he is allowed to say. We know he can’t say too much before April 29th, but as long as he is talking, we can heard something from his tune, or guess some lines that are not printed.

clip_image003_thumb[2]

 

Regarding HoloLens programming, He said,

On top of all that black magic is HoloLens, introducing a new set of holographic APIs for app developers. Though it leverages Windows Core and though it leverages UAP, the device-specific APIs that enable the holographic experience are going to be different to what app developers do today. This means developers can use what they know, but will need to learn this new paradigm.

This means when we develop for HoloLens, it still follows UAP scheme. You create an UAP app (I don’t know if there is other choices in for Windows 10). You probably also add “HoloLens Extension”, if such thing exists. Basically there are things we already know: like coding for UAP, and things we don’t know: like HoloLens related APIs. HoloLens APIs is very much already in Windows 10 preview, but Windows 10 SDK doesn’t yet include HoloLens. I am sure we will see that during BUILD.

HoloLens is going to be a commodity. That means it is going to be cheap (which is relative, I know). When you include HoloLens in a solution, you aren’t including an investment in some strange, proprietary whatever. You are just including HoloLens, with a standardized version of Windows, a standard application model, with standard languages and APIs, and everything you can expect a development team to already know or easily learn.

Well, I don’t want to read too much. But when he put HoloLens and commodity in same sentence, I can’t stop speculating. Did he just said HoloLens is going to cheap or develop for HoloLens is going to cheap, or both? He did say: HoloLens is going to be a commodity. Take a note, please.

Don’t let HoloLens become a specialty skill. You are going to find the SDK just like every other Microsoft property. It will be standard and asynchronous, full of controls, methods, and classes.

Ok, it’s going to be asynchronous, like any WinRT APIs, but what those controls look like is interesting thing to find out. Obviously they are different from regular controls we do in XAML for PC and phone.

 

UAP, HoloLens, Recommended Readings – Developer’s Collection

 

How to Choose Programming Language for UAP? – Updated

 

Windows-10_Product-Family

 

This is a short post, because it is a simple question and there is a simple answer. The reason I want this post is: if you ask anyone, you will get a long answer, and that never answers your question. Actually, I have talked about this before (here), but I feel I left Visual Basic out, and something about DirectX needs clarification.

The typical answer you get is: It depends. Then you get a few answers for different situations, and you have no idea what your situation is.

The choices are:

  1. HTML, CSS and JavaScript.
  2. XAML and C#/VB
  3. XAML, DirectX and C++

The answer is: Forget about #1, because it is silly. If you develop apps, #2 is your best choice, because it is easy, time-saving comparing to C++. If you develop games, #3 is your best choice, because that’s pretty much your only choice.

Update: If you want to develop both apps and games, the answer is: it’s still the best to learn both C# and C++. Reasons are: a. It’s not hard to do, because they are very similar in syntax. b. Writing apps in C# is a lot easier than in C++.

You can stop here, because you’ve already got the answer. The following explanations are not part of the answer, and can be ignored.

HoloLens Technology – Eye Tracking

I don’t have problem with JavaScript, as a matter of fact, I am coding it every day, right this moment. But writing serious application in JS is really silly. Microsoft put this up, hoping someone only knows JavaScript won’t have to learn C# or C++. I am not sure if there is a programmer who only writes in JavaScript.

Visual Basic is still supported, that doesn’t mean you should stick to it. I feel Visual Basic is in legacy state, but it’s just how I feel, not that I have any proof. I love Visual Basic. Visual Basic is the start of modern programming.

WinRT programming in C++, without MFC, is not programmer friendly. You don’t need write COM code, but IUnknown, QueryInterface() really turn me off, I have a fair amount of experience in COM programming though. However, C++/CX adopts C# style syntax, which makes it easier. But if you are developing games, you can’t avoid C++ for the moment. The reason is you can’t avoid DirectX.

Xbox + HoloLens: How Can They Work Together?

There was a time, when games can be done in Silverlight and XNA. This has changed to DriectX. At some point, DirectX has interface for .NET, so C# with DirectX worked. Since DirectX 10, there is no official support to .NET. There is no word in DirectX 12, which is available in Windows 10. We will have to wait for BUILD to check this out. There are third party packages that make DirectX work with .NET.  As of now, if you plan to write games, it’s XAML + DirectX + C++.

Do I have to use DirectX to develop games? No, but most games required timely response and deals with a lot of graphics, DirectX is your best bet, even though you are not doing 3D. Can I write apps with XAML + DirectX + C++? Yes. DirectX can be used whenever it fits, but it is not required. DirectX works with XAML as ImageSource or SwapChain.

 

UAP, HoloLens, Recommended Readings – Developer’s Collection

HoloLens, Recommended Readings – Business Ideas Collection

 

UAP, HoloLens, Recommended Readings – Developer’s Collection

 

uap

 

BUILD Conference is less than one month away (29th, April). I’m collecting some of my posts relating to app development on Windows 10, some of my findings/thoughts about developing for HoloLens.  My intention is to give you one-place-for-all approach, but it’s obviously not. However, if you have done Windows or Windows Phone development before, the links I gave in the posts are pretty much it. For the moment, there isn’t much really, but remember to get a good understanding of the following terms: UAP, Mobile Extension, Adaptive UI, SplitView, RelativePanel, Metro Design Language (the term may have changed, but influence has not). There is a little bit of Windows Holograhic APIs, but nothing materialized. We are expecting to hear a lot more during BUILD conference.

I have a couple of other writings related to this topic. I will add them here when they come out.

By the way, it is a good idea to download and install Windows 10 Technical Preview and Windows 10 Technical Preview SDK. Before install the SDK, download and install Visual Studio 2015 CTP6 first, which can be downloaded on the same link with the SDK. I suggest you install them on a VM. There are some instructions on PCWorld. Also, please make sure you have about 50GB or more space on your hard drive for the whole thing.

 

HoloLens: Some Early Bits for Developers

Developers: Get Yourself Ready for HoloLens

Digest the Windows 10 Technical Preview SDK (Updated)

 

Optional Readings:

What Is Mixed Reality, Really?

How HoloLens Will Change Windows UI – (1) Break of Screens

How HoloLens Will Change Windows UI – (2) Another Dimension

Natural User Interface: Is Natural Always Better?

PC + HoloLens: Would It Work?

PC + HoloLens: How It Works

Xbox + HoloLens: How Can They Work Together?

HoloLens Misconception – 3D

From HoloLens to “True” Hologram: How?

 

 

Digest the Windows 10 Technical Preview SDK (Updated)

 

Fig. 1 Convergence

Fig. 1 Convergence

 

Windows 10 Technical Preview SDK was released two days ago. This is probably the biggest change in the history of Windows, in terms of impact on user experiences and businesses. However, from a developer’s point of view, it is not easy to say. On one hand, it is huge because one binary runs everywhere, on the other, the technology is not entirely new. Coding for Universal Application Platform(UAP) is not much different than coding for Windows or Windows Phone. So the transition should be very smooth if you’ve ever done anything on Windows or Window Phone, coded for Silverlight or WPF. Well, that’s when you have a good understanding of the new development tools.

Along with the release of Windows 10 Technical Preview SDK, 14 videos titled Developer’s Guide to Windows 10 Preview was posted on Microsoft’s Channel 9 site. It’s Jerry Nixon and Andy Wigley talk about the new SDK and what’s new in UAP. If you have any interest in developing for Windows 10, I strongly recommend you to watch those videos to get a big picture of what’s going to come in the summer. BUILD conference starts April 29th, the official SDK will likely come out then, but it is important to get some feelings now. There are a total of 14 videos, I don’t think anyone have the time or patience to watch them all. So I did it for you. No, I just watch some of them, but enough to give an overview what it is about, before you spend time on it.

The Convergence Story

They talked about the story of convergence in Windows 10(please see Fig.1). There are a few checkpoints: At the time of Windows 7, Windows Phone 7.5, and Xbox 360, those devices are running on totally different operating systems. The first step of convergence happens during Windows 8, Windows Phone 8 and Xbox One, when NT kernel was adopted by all devices, but leave the UI and user experience separated. Windows 8.1 and Windows Phone 8.1 moved things a little closer: Developer can create one project and target both Windows and Windows Phone, but you still build two binaries, and summit them to two different market places, user has to buy them separately. With Windows 10, it comes to full convergence.

 

UAP

Fig.2 UAP

Fig.2 UAP

UAP – Universal App Platform is the single most important concept brought in with Windows 10. Under Windows 10, you no longer target specific type of devices (PC, phone, Xbox, HoloLens, etc.), you develop for UAP. It a big change, but it is also great news. You create one code, build one binary, publish it to one app store. I don’t know how you feel, to me, it’s a dream to both developers and users.

Extension SDK

 

Fig. 3 Extension SDK

Fig. 3 Extension SDK

 

UAP is great, but there are many things cannot be unified: A PC doesn’t make calls, a phone doesn’t have game controllers. The solution is Extension SDK. You can reference those extension module from Visual Studio. Also the SDK provide a way to check device capabilities at run time. Fig. 4 shows an example of that.

 

Fig. 4 Check Capability at Runtime

Fig. 4 Check Capability at Runtime

Another thing is you don’t need #if anymore. Well, the purpose of #if is to build more than one binaries on one code base, since you need only one binary, there is no need for #if anymore. It is still supported though. We now check device capabilities at run time.

The SplitView

This is first of two important new controls introduced in Windows 10. What is SplitView? This is SplitView (see Fig. 5).

Fig. 5 SplitView

Fig. 5 SplitView

 

It is a new style of menu that is more adaptable to the screen size changes, and work well for both touch and mouse. It is a control that can be constructed in XAML like other controls. Interesting thing to mention is the menu button can be positioned at the bottom (for phones and tablets), or at top (for PCs).

The RelativePanel

This is another important new control in Windows 10. It’s more important than you think. RelatvePanel is a panel, you use it for layout. It solves the problem that programmer has to anticipate screen size and orientation, and reorganize controls accordingly. Instead, you put controls inside a RelativePanel, define relationship between them. You are done. It won’t solve all layout problems, but most of them. See Fig. 6.

 

Fig. 6 RelativePanel

Fig. 6 RelativePanel

 

Now you have it. You probably want to watch some details on these videos: #2 Hello World, #4 Extenstion SDK, #5 SplitView, #8 RelativePanel. As to the other ones, it depends on your time.

Remember this is tech preview, things may change a little at end of April, but I don’t see much change then. I would expect the SDK will include Windows Holographic and HoloLens though.

 

-Update: Just found this video also on Channel 9: A First Look at Building Windows 10 Universal Apps. This is Kevin Gallo from Microsoft talking about Windows 10 UAP at MWC 2015. It’s important because there is live demo of how to adjust UI to suit different screen sizes. Also there are demos of SplitView and RelativePanel which I found better than the afore mentioned videos. You can go right to 18:05~37:00 (the whole video is long):

18:05 – 20:15 UAP
20:16 – 25:40 Adaptive UX, How UI change to adapt to screen sizes
29:28 – 37:00 Live Demo of Adaptive UI, SplitView and RelativePanel.

I want also post an interesting screenshot here. The picture is blurry but the content is good.

 

Yes, HoloLens is there!

Yes, HoloLens is there!