Kinect for Windows: A New Interface for Your Apps

Microsoft officially announced that a new Kinect for Windows device will be released on February 1st along with a commercial license. The device isn't identical to the Xbox version, nor is it the same price. Additionally, the new device shouldn't be seen as simply a toy any more than a computer mouse is a toy.

This new device will be an improvement over the Xbox 360 Kinect in that it will be optimized for shorter distance. If you've used the Kinect for Xbox 360, you likely know that one of (the few) frustrations is that you have to be a good distance from the device for it to fully recognize you. This makes sense if you consider that on the Xbox, you are often doing activities that require the Kinect to view your entire body. For a computer, it is less likely you need to know what the bottom half of your body does. As such, the Kinect for Windows is able to be placed at a distance similar to where people sit relative to their computer (about 50 cm).

The price for the Kinect for Windows is stated to be $249, which is basically $100 more than the list price of the Xbox 360 version. While this seems a bit high for a peripheral, time will tell if this price is too high. With 18 million Xbox 360 Kinect units sold, it is easy to speculate that the new device will sell well assuming the price doesn't get in the way.

A higher price for the Kinect for Windows should not be a surprise. Just as the Xbox console is cheaper than a standard PC even though the hardware is similar in many ways, so too is the Kinect priced higher. Microsoft points to the obvious reason for this difference in price, which is to state that there are software subsidies used to offset the cost of the Kinect for 360. In other words, Microsoft can sell Xbox hardware cheap — even at a loss — because they will make up the difference by selling games and in licensing fees from games that other sell. Because the Kinect for Windows device doesn't have corresponding software or licensing fees, it doesn't make fiscal sense to sell it at or below cost.

Using the Kinect in an Application

You can currently use an Xbox Kinect with your PC. This has allowed people to build applications today (prior to release of Kinect for Windows). If you download the beta version of the Kinect SDK, you'll find that it comes with several sample applications and code that you can play with. You'll be able to tap into its motion detection as well as work with its voice features. There are also a number of sample applications you can find publicly available.

I present monthly at a .NET meeting where I've talked about the potential of developing for the Kinect. For fun, I used the Kinect to control my PowerPoint presentation. Thanks to Joshua Blake, I was able to find an application that was already built that does this. What I learned was, the code to do so is exceedingly simple. You can find his application at http://kinectpowerpoint.codeplex.com/. This application actually allows you to send right and left arrow commands to the currently active application using a single command. In other words, you can run this Kinect application and then bring PowerPoint to the foreground to control a presentation. Using swipes of the hand (left and right), you can navigate through slides. The program also includes a few sample voice commands that you can use.

Note: I found that as a presenter, I move my hands a lot. As such, I changed the code to test when my hands were above my head. While this is a bit goofy, it was great for illustration the point of what the Kinect can do and how easy it can be done.

The code for this PowerPoint controlling application is relatively straightforward once you've installed the Kinect SDK. If you take a look at the code, you'll see that most of the code is in one method:

void runtime_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{
    SkeletonFrame skeletonSet = e.SkeletonFrame;

    SkeletonData data = (from s in skeletonSet.Skeletons
         where s.TrackingState == SkeletonTrackingState.Tracked
         select s).FirstOrDefault();

    var head = data.Joints[JointID.Head];
    var rightHand = data.Joints[JointID.HandRight];
    var leftHand = data.Joints[JointID.HandLeft];

    SetEllipsePosition(ellipseHead, head, false);
    SetEllipsePosition(ellipseLeftHand, leftHand, isBackGestureActive);
    SetEllipsePosition(ellipseRightHand, rightHand, isForwardGestureActive);

    ProcessForwardBackGesture(head, rightHand, leftHand);
}

As you can see in the code, the complex effort of figuring out where a person is in the Kinect capture is done for you. You simply use the runtime_SkeletonFrameReady event. This will let you create and use a SkeletonFrame. From the Frame you can get the data for hand and head positions.

var head = data.Joints[JointID.Head];
var rightHand = data.Joints[JointID.HandRight];
var leftHand = data.Joints[JointID.HandLeft];

Once you have the positions, you can then do whatever functionality you want. In this case, ellipses are drawn where the head, left hand, and right hand are located using a method within the code.

After drawing the ellipses where the body parts are located, a call is made to ProcessForwardBackGesture. This routine is used to determine if a hand is the appropriate distance from the head. The code is for checking the right hand is:

if (rightHand.Position.Y > head.Position.Y)
{
   if (!isBackGestureActive && !isForwardGestureActive)
   {
      isForwardGestureActive = true;
      System.Windows.Forms.SendKeys.SendWait("{Right}");
   }
}

In this code, if the hand is higher than the head, then a check is done to make sure that a gesture isn't already being handled (!IsBackGestureActive && !IsForwardGestureActive). If no Gesture is being handled, then the action happens. In this case the action is to send a key command to the System. Additionally, the code indicates that a gesture is now happening. You could just as easily change this code to do something else.

There is a bit more to the code, but not much. You can download it and a number of other examples. Obviously you'll need a Kinect to see how it works.

What's up with the Kinect Licensing?

In a day of patent lawsuits and licensing fees, it is worth a minute to look at Microsoft's licensing of the Kinect for Windows development software. The short answer is that Microsoft's licensing for Kinect is hardware-only based. This is very good in that you won't need to pay a licensing fee for selling any Kinect applications you build. You only need to buy a Kinect.

Of course, there is a "but...."

But, you will only be able to deploy your applications written with the Kinect SDK for Windows to Kinect for Windows devices. You cannot use a Kinect for Xbox 360 device with the applications you create for commercial use. You also cannot use the release version of the software for non-commercial use with a Kinect for Xbox 360. If you want to use an Xbox Kinect, you have to use the beta version of the SDK, and you can only do so for non-commercial uses. Because of this restriction regarding the use of Xbox Kinects, Microsoft has agreed to maintain the beta SDK until June 16th, 2016.

In Conclusion

Microsoft has seen huge success with the Kinect for Xbox 360. The potential exists for similar success with the Xbox for Windows device. It will be interesting to see the types of applications that are developed that can help justify the added cost of the peripheral for people.

While people chuckled at my presentation where I raised my hands above my head to transition slides, I received a different look on another possible use. How often do you ask a yes or no question on a screen such as "do you want to continue?" What if you used the Connect for Windows to detect a right and left movement of the head versus an up and down movement? All of the sudden, you don't need to wait on a mouse click or a keyboard press, the user already answered with a simple shake of their head. I expect one of the first add-in controls to be built will be a "test for yes/no" control. With it, suddenly there is a bit of interaction automation that makes sense.

It will also be interesting to watch how Microsoft licenses the hardware to others to build into other devices. For example, televisions are getting smarter. It would be interesting to see Kinect technology built into televisions.

Kinect for Xbox 360 has changed the game playing market. What will happen with Kinect for Windows or the Kinect technology in general? Time will tell.



About the Author

Bradley Jones

Bradley Jones, in addition to managing CodeGuru, Brad! oversees the Developer.com Newtwork of sites including Codeguru, Developer.com, DevX, VBForums, and over a dozen more with a focus on software development and database technologies. His experience includes development in C, C++, VB, some Java, C#, ASP, COBOL, and more as well as having been a developer, consultant, analyst, lead, and much more. His recent books include Teach Yourself the C# Language in 21 Days, Web 2.0 Heroes, and Windows Live Essentials and Services.
Google+ Profile | Linked-In Profile | Facebook Page

Comments

  • There are no comments yet. Be the first to comment!

Leave a Comment
  • Your email address will not be published. All fields are required.

Top White Papers and Webcasts

Most Popular Programming Stories

More for Developers

Latest Developer Headlines

RSS Feeds