June 23, 2017

Seattle Symphony to Perform with Kinect Instruments




I am not joking. It’s true, Kinect controlled instruments are to perform with human musicians in an orchestra.

From GeekWire,

On Friday, May 1, the Seattle Symphony will present the world premiere of a new work. It will be conducted by Music Director Ludovic Morlot and feature ten human musicians and three “kinetic instruments.” All will be directed, as is tradition, by gesture.

It will be in Benaroya Hall, downtown Seattle, which is Seattle Symphony’s home theatre. The report also said, after performance, the Kinect instruments will be left there on display for exhibition.

The concert will be held in the grant lobby, not the concert hall. There will be 11 human musicians: 9 instrumental musicians, 1 soprano, the conductor (Morlot), and three Kinect instruments: a 24-reedhorn sculpture, a set of concert chimes, and a customized grand piano. So, it is a small orchestra, more of chamber setup. The piece is to performed lasts 22 minutes.

Those three robotic instruments are controlled by Kinect, the one comes with Xbox One. It doesn’t say whether there are 3 or 1 Kinect. The challenge here is how the robots understand the conductor’s gestures. Think about this for a second, those are not predefined, limited set of gestures like that are used on Xbox One. Those are the gestures both human and Kinect understand in real time, for 22 minutes. I guess the conductor has to limit type of gestures but still, it is not an easy thing.

I am not sure if the conductor is using a baton, or just hand. Since if it is a baton, it would be more important, because that would mean object recognition other than hand recognition. I will write about object recognition(it is an excite topic) later.

Here is the big deal: how to keep the human musicians and robotic musicians in sync? If there is any small latency caused by Kinect, the performance would be out of sync. It won’t work. I think there must be a very small latency in Kinect. If there wasn’t a conductor, musicians can usually adjust speed to catch each other up, with the conductor, the robotic instruments have to respond in real time.

This gives me more confidence to HoloLens, and that’s the whole point I am writing this.


Some Business Ideas with HoloLens – (20) Holo-Golf Swing

Is HoloLens Going to Be a Commodity?

Review of Surface 3 Reviews, and My Thoughts

Jerry Nixon: “Why you must build a HoloLens app”