Monday, February 2, 2009

Wanna See What Happens When Real People Use the "Minority Report" Interface?

A couple of months ago, I blogged about Oblong, the new venture of Minority Report UI guy John Underkoffler.

He showed up at sundance with his hell contraption and tried to use it on innocent spectators. See the results.

Here's 9 minutes of usage:

Here are the results:

What do you think? Is there a big idea in here somewhere?


  1. Wonderful example.

    Touchless gesture interaction is not natural. (Unless you want to stop a taxi)

    thanks for sharing,

    Xavier Vilar.


  2. @ Xavier,

    I actually disagree, there is a time and a place for freeform gestures ... When I was working with the Context Aware Computing group at MIT, we had eyetracking sensors and capacitive floor tiles that enabled contextual interactivity. They worked very well...

    The difference, I think was that we focused on passive interactivity ... If users were in the space, things would happen based on their natural behavior ... stare at something and it will be registered. Walk around the room and your media follows you.

    Oblong focuses on active interaction. It's easy to knock on Underkoffler, but frankly the speed to competance is pretty impressive.

    I've taught multiple complex 3D and image editing applications. This stuff is actually quite hard to pick up with a keyboard and mouse.

    btw. I've been watching your progress on vimeo. It would be great to learn more.


  3. @

    I have to admit that I had an unappropiate reaction to your post. I totally agree that touchless gestures, eyetracking, and other sensing techniques are great for passive interactivity. There's no discussion about that.

    However, I have my doubts about touchless gestures as an appropiate active interaction method. I'm sure it improves learning curves in complex applications involving 3D environments, but it remains unnatural for common tasks like a simple video manipulation.

    Since we started playing around with multitouch (just 1 year and 4 months ago) we've always been dealing with the Minority Report thing. This video will do some help to distance Tom Cruise from natural interaction.


    (sorry about my english)

  4. @Xavier,

    I completely agree with your last statement and I generally agree with you that with simple task paths, Underkoffler's solution is absurd ... and the sorts of gestures this demo uses are far too fatigue inducing for any productivity app. (I looked in the US Army Interrogation Manual. It doesn't actually list illegal pain positions, but Underkoffler was clearly inspired by research in this area :~P).

    While the solution may not be successful, I think it manages to run around the core problems related to multitouch.

    At arms length, you can only actively view an ~18-24" diameter circle of content at any time. This is VERY limiting, especially when issues of occlusion are taken into consideration.

    So far, multitouch has been used for relatively simple, highly contextual tasks.

    It hasn't proven anywhere near as flexible as a command line interface.

    The complexity of Underkoffler's solution has more in common with a keyboard than a touch wall ... and it's a first stab at bridging the benefits of both.