Hands-On With Microsoft's Touchless SDK

snydeq writes "Fatal Exception's Neil McAllister takes Microsoft's recently released Touchless SDK for a test spin, controlling his Asus Eee PC 901 with a Roma tomato. The Touchless SDK is a set of .Net components that can be used to simulate the gestural interfaces of devices like the iPhone in thin air — using an ordinary USB Webcam. Although McAllister was able to draw, scroll, and play a rudimentary game with his tomato, the SDK still has some kinks to work out. 'For starters, its marker-location algorithm is very much keyed to color,' he writes. 'That's probably an efficient way to identify contrasting shapes, but color response varies by camera and is heavily influenced by ambient light conditions.' Moreover, the detection routine soaked up 64 percent of McAllister's 1.6GHz Atom CPU, with the video from the Webcam soon developing a few seconds' lag that made controlling onscreen cursors challenging. Project developer Mike Wasserman offers a video demo of the technology."

Re:I have an EEE PC

[Issildur03]

I tried it out, and the drawing demo seems to be the most promising application. In the absence of a touch-screen monitor, this could be a lot better than an external touchpad. And there's definitely something neat about using a tomato to play snake. Still a long way to go, though...

Re:It's Open Source

[PaintyThePirate]

If you're interested in a truely Open Source version of this, Pygame [pygame.org] has camera and computer vision functions in the SVN that let you do exactly this. I could track two different colored objects in realtime (30fps) with no lag, on a 433mhz OLPC XO.

It is Linux only at the moment, but Windows and OS X support is likely to be finished before the next release.

Re:I have an EEE PC

[sp332]

1. Any really cool technique will be patented by Microsoft and doomed to obscurity by their poor implementation of same; and

From the license:
"(B) Patent Grant- Subject to the terms of this license, including the license conditions and limitations in section 3, each contributor grants you a non-exclusive, worldwide, royalty-free license under its licensed patents to make, have made, use, sell, offer for sale, import, and/or otherwise dispose of its contribution in the software or derivative works of the contribution in the software."

OpenCV

[Anonymous Coward]

Intel made a very nice open source library for computer vision. It's called OpenCV [http://sourceforge.net/projects/opencvlibrary/] and can be used to track pixels (or hands, or heads...).

I first saw it on Pycon Brasil 2008, with EHCI python bindings [http://www.slideshare.net/dannyxyz22/ehci-interao-com-computador-atravs-de-webcam-presentation].

Microsoft library is not a big deal... I made a script to switch KDE desktops using face moviments with 45 lines of python script + ehci, including a lot of useless code and testing/debuging prints. :-)

Thiago F Pappacena

better cross platform alternatives

[nan0]

opencv [sourceforge.net] has nice python bindings, runs on mac, win & nix.
openframeworks [openframeworks.cc] wraps c++ like processing [processing.org] wraps java, also has opencv bindings.

MS appears to basically doing optical flow & color tracking. the above libs can do those, and more, and are great for programmers and nonprogrammers alike. tho if you really hate code, you may rather use max/msp/jitter or gem/pd [puredata.info].

The last man standing

[westlake]

Can it recognise that someone's about to pick up a chair?
.

In a financial crisis the prize goes to the last man standing

Microsoft is the first U.S. industrial corporation in ten years to earn a AAA bond rating from S&P and Moody's.

More than 70 percent of S&P ratings for U.S. nonfinancial companies are currently below investment grade and classified as "junk", or speculative-grade bonds. That's up from 32 percent in 1980. Microsoft wins top credit ratings from S&P, Moody's [reuters.com]