Wednesday, December 3, 2008

The Coming Multitouch Gesture IP Wars

It looks like PQ Labs is searching for ways to run around gesture patents by Microsoft/Bill Buxton/Alias Wavefront/Apple/Fingerworks/et al.

While the Cool Factor in this demo is undeniably impressive, it's actually difficult to learn because of the number of new, unique gestures.


Unless the multitouch community rapidly develops a standard gesture library, I fear that this demo will mark the end of NATURAL user interfaces and the beginning of a geometrically growing set of gestures that users have to learn when they buy new products.

Microsoft is the only player with the breadth and influence to lead a standardization charge. They need to quickly publish a standard gesture system that is freely distributed and supported by major PC and Mobile manufacturers like Dell, HP and HTC.

A common gesture set will have to be small, extremely flexible and usable at multiple scales, ranging from smartphones to video walls. Even Apple's current set of ~10 gestures seems beyond what consumers will bother with.

A shockingly tiny percentage of users consistently leverage the right click menus or function keys in GUI apps. There's little reason to expect that human beings (aside from Tom Cruise) will master a gesture library that resembles American Sign Language.

Microsoft, please, please, please be responsible. Publish a full gesture library ASAP, with a community commons license...before the coming IP wars leave us in usability hell and kill the multi-touch market.


If you, as a developer, like PQ Labs, are creating proprietary gestures, there are a couple of simple strategies that can make your UI easier to use:

A. Prime users through a subtle gesture demo, either while the application is being selected or loaded. If PQ Labs had started their demo this way, users would actually know how to use gestures like:
- the two handed camera-rotate
- the "4? finger swipe" to make objects go away
- the "5 finger squish" to turn the TV on.

B. Provide some sort of contextual markup menu that allows users to efficiently drill down to what they need when they don't remember the gesture.

On a Different Note

Augmented reality concepts have been around for a long time.

Images by Colin Bulthaup, Dan Goldman and Saul Griffith, colleagues at Squid Labs and inventors of Magic Mirror, the first fully functional wireless augmented reality tablet.

It's interesting to see Heads Up Augmented Reality Overlays starting to appear as a solution in multitouch user applications.


  1. The idea of a standard gesture set library is very cool. Your blog is a great resource for my works. Thank you.

  2. I totally agree that we need more standardized gestures from MS and I've been a little disappointed that there weren't more built into the Surface SDK. I assume we'll see something to coincide with Windows 7.

    And I agree that PQ Labs video is gesture overkill. Personally I don't see the point of how they've implemented navigating the space. It seems they have not learned from the hundreds of FPS game on the market. It doesn't ALL have to be touch-based. I think that's where they're shooting themselves in the foot. They should utilize touch for the commands that are intuitive.