Saturday, November 29, 2008

Anand Argwala's Bumptop on a Multitouch Wall

After a night of late night scrounging, I found Anand demoing his very intriguing UI.

The video suggests several design concepts that would work very well in a table top, multiuser scenario...

1. 1-point perspective. This helps keep the illusion of depth when viewed from all sides.

2. Using a "room" to simulate depth.

3. Pinning content to a side wall and then scaling it on the z axis to make space on the table. An expanding tray/wall that rotates out to become become a temporary "roof" instead of moving the floor might look less cool. It would also be less disturbing to other users.

The Bumptop website claims that their video on tablet computing is the most watched video on youtube, ever. Not sure about that, but here it is...quite cool.

1. Many of the group selection/sorting metaphors seem highly relevant to nultitouch.

2. The cornucopia of display options seem like they could be modified for multitouch. In reality, you only want a couple in an app. Otherwise users get confused. The endless file display options in Vista come to mind.

3. Highly intuitive tools to sort content on the z axis as well the traditional xy plane.

Sign up to be a beta user at

Friday, November 28, 2008

Vectorform and BMW Finally Release Surface Car Configurator

It looks like BMW is putting their Surface PR push together. Frances at Vectorform in New York was kind enough to demo this for me recently.

This app confirmed my suspicion that Surface is an excellent solution for complex product sales that involve:
1. Group decision making
2. Complex product specification
3. High order accuracy
4. Highly granular feedback on product features from a large user group.

A couple of immediate thoughts:

1. There is what looks like a very nice use of prerendered video to simulate a 3D interior (see the last post on Adobe Labs research for some ideas on how you might do this even better)...

2. The application really shows the power and limitations of using Surface's object recognition capabilities.
- The magic of placing a paint chip on the car and then actually seeing it is amazing.

- Successful Surface developments require interaction designers who understand how to design tangible objects. Even with an additional tray to hold the artifacts, they seem difficult for a novice to find and use. It's a bit bizarre that Surface isn't designed to accomodate a tray for artifacts.

3. With the exception of Harrah's bar, every app developed outside of Microsoft has mounted Surface at Bar height. Customers should demand an optional stand to mount surface at this height. After all the bells and whistles, Surface is a $15,000 box. It should be usable without buying a $5,000 custom cabinet.

4. Scatter view is the right solution for almost nevery situation. This is a GERMAN DRIVING MACHINE. Why are the pictures spinning around all willy-nilly?

Very interesting stuff. More later.

Interesting Stuff. I'll post a more in depth review later.


Great Video on Direct Manipulation of Video

Here's what Adobe's cooking up for CS5...yummy.

Interactive Video Object Manipulation from Dan Goldman on Vimeo.

The techniques Dan describes in the video are a subset of an interaction strategy called "Direct Manipulation". How could this be used in multitouch apps? Leave a comment.

Wednesday, November 26, 2008

Starfire, the Inspiration for Surface

It's the day before Thanksgiving. You're not gonna be productive.

Click me to see the Starfire video

Bruce "Tog" Tognazzi posted this video about Starfire, a 1992 "Product of the Future" from Sun Microsystems...the great grandpappy of Microsoft Surface. It's must see TV for anyone in the multitouch space.

For my money, it's the first compelling "vision of the future" for multitouch and digital convergence. In a space where everyone is still awed by swirly paint programs, 16 years after the fact, Starfire is still ahead of the curve.

This video inspired a direction and significant research into the area of multitouch productivity at corporate research labs including Microsoft Center for Information Work and IBM's USER group, run by Ted Selker.

Bill Buxton, one of the early multitouch researchers (University of Toronto, Autodesk, EuroPARC, Microsoft) also cites Starfire as a significant project.

Sit back. Get some popcorn. Tell your boss it's research.

This posting was modified to provide more context about the Starfire Project in response to Richard Monson-Haefel's comments. You can check out his blog at:

Make yourself heard. Comment and join the party. What would you like to learn about?

Tuesday, November 25, 2008

Teemu Arina's Brilliant Talk On Social Media

Teemu Arina lit up the stage at Mobile Monday with a compelling, comprehensible talk linking Marshall McLuhan's The Medium is the Massage to social media. It's an explanation that even my father might understand.

Enjoy Dad 8~P

While he is generally dead on, his arguments about ignoring the wisdom of crotchety old managers, social norms and standard file formats seem naive. What do you think?

Best Practices for Touch Screen Mobile Devices

Thanks to our colleagues at Touch Usability for digging up this slideshow from UIQ on best practices for mobile touch interfaces. It closely matches our experience:

Monday, November 24, 2008

What would you do with a bunch of really big TVs?

In the next year or so, hi-res projectors will break the $500 barrier and 50" LCDs will be below $1500. This begs the question...what will we do with them?

Last week, John Underkoffler, the big brain behind the Minority Report User Interface, dropped a demo exploring key questions in this space:

1. How can users interact efficiently with large displays?
2. How can gestures work across both multiple table and wall displays?
3. How can users effectively work with z-depth?

Outside of movies, real people perform tasks on computers that require finer manipulation than Underkoffler's UI allows (modifying documents, speadsheets, websites, videos, etc.) .

While his demo is a partial solution at best, combining his insights with an already ubiquitous device has real potential. Replace the black gloves with an iPhone.

This would enable "Minority Report"-style 3D gestures for gross manipulations...and a touch interface for precise tasks. (The iphone is perfect for's a ubiquitous, battery powered, hand-sized, wireless peripheral with an accelerometer, a camera, and a small multitouch display).

Check out this tantalizing demo from MSG at Aachen University:

Underkoffler's ideas could enrich MSG's framework if it were expanded to leverage other hardware features of the iphone:


- The accelerometer could sense 3D gestures, similar to those seen in Underkoffler's video.

- Rocking the rounded back of the iphone on a table could create a mouse-like interaction, controlling x/y/orientation and velocity.

- The primary display could be chosen by pointing the iphone toward it (the built in camera would identify/confirm the position of the screens).


- The touchscreen could sense fine manipulations, provide contextual options and feedback.

What would you do with a bunch of really big TVs? ...and if you are interested in mocking up a demo like this, drop a line :~} Jonathan

Friday, November 21, 2008

Look mom, a big ass ipod...

Actually, this demo from Stimulant demonstrates how low cost sensors make NUI more interesting...I can think of half a dozen practical apps right here, right now.

from Stimulant on Vimeo.

Thursday, November 20, 2008

Scaling and organizing the Multitouch workspace

One of the core issues with most multitouch devices is that they are WAAAAY to small and low resolution to manage large sets of information all at once. They need a widget to conveniently contain and access information and applications on the fly...much like Spaces on the Macintosh or that Aero thing in Vista.

For a bunch of reasons, scaling the background (like pptPlex) is a convenient way to do this. As tasks requirements change, it might also be useful to reorganize background "plates".

Check out this early test from Infostrat. Might a controller, sort of like this like make an interesting solution?

What about if the background were thought of as a sphere so that the top of the screen seamlessly mapped to the bottom...

Any thoughts? Comments welcome.

Direct Manipulation of Characters

Watch this video from ~1:30 onward. This is the first example I have seen of direct manipulation of a character...It might be much less scary if it was SpongeBob or something :~}

Whoever did this, if you are tuning in, drop a line.

Would GM need a bail out if their designers had these?

Feast your eyes on this excellent video posted by mighty Dan Wigdor of the Surface SDK team was doing this as a student while the rest of us were out partying. Go Dan!

The Argument against R+D

After a long day of dreaming up new technologies, it's humbling to realize that it's often easier and more profitable to rejigger old ones.

Check out Rain Noe's great article on product innovation over at Core 77

There's at least two ways to design a new category of product:

1) Invent a new technology and incorporate it into your design (the Dyson Vacuum), or

2) Design something around a novel use of an existing technology (the iPod).

Both types can be exciting, but what we like about 2) is that you don't have to spend a dime on R&D vis-a-vis the technology...

Let's take vacuum technology, for instance.

More after the link:
Core 77

Monday, November 17, 2008

Interactive Wall Round Up

There's a new shop out there called Monster Media.

They've been tearing up the ad market for the past few months with some cool in window demos. They are definitely attention getting. I wonder how useful they are though...Monster claims a 40%+ unaided recall. VERY IMPRESSIVE. I'm sure they seared some eyeballs, but did prospects remember the message?

It's easy for design technologists to get caught up in the media and forget to communicate the message.

Friday, November 14, 2008

Critiquing James Bond's Multitouch Computer

OK, it's Friday, the new James Bond is out and it's gonna be great!! Quantum of Solace is must see TV. Go, go now.

But before you do, let's take a look at the incredibly cool multitouch UI eyecandy. It has a lot to teach about designing for multitouch.

It is filled with interesting ideas, yet it contains core usability issues that I see clients struggle with every day.

Comments are keyed to the numbers in the figure below:

1. Windows, Size and Orientation.
Windows- So, James Bond is working on the world's largest touchscreen, yet there isn't enough room for all of the content????? Why is the map covered up?

Size- It's really hard to use 12' interactive maps that are 3 feet away. They look cool, but IMAX-scale images do things like can cause epileptic seizures when they move around.

Orientation- Looking down all day causes a lot of stress on the neck. That's why old school drafting and writing tables are angled. Even Startrek got this one right.

2. Secondary content. It's a great idea to have a gridded secondary area for content that isn't actively being manipulated. This would be more useful if it was rotated 90 degrees so both M and Bond could read the text.

3. Is M scanning a piece of paper in here? Very cool. Someone, please do this with Surface. It doesn't have the resolution of a proper scanner, but if you can OCR a crappy fax, a 12 pt. laser print should work just fine.

4. It's great that there is a "communal content area", but it would work better for both users if it was nongridded, like a Surface scatterview. It looks like there is text next to the headshots. unfortunately, it's to small for both M and Bond to read. There should also be an orientation strategy that enables both users to read it.

5. What is this? How does it work? Why is it here? What if M wants to use it?

Interesting Apple Multitouch Patent

Mark over at Mark's Technology News has got his eagle eyes open. Check this out:

Can Surface Save Retail Banking?

Surface is at BAI Retail Delivery Conference in Orlando. I am looking forward to seeing what they have to say.

This morning's press release has me concerned:

"It's cool" isn't a B2B value proposition...especially when there's a big, topical opportunity that will line bankers up around the block:

National-level command and control of home loans.

The loan decision process is a muddled mess. 20% of initial loan applications contain a significant error. The approval of many of these bad loans, combined with over-leveraged national banks has caused of our current credit crisis.

Essentially, the generals failed to keep the soldiers in check. Credit ratings and income statements be damned. Home loans are based on some local loan committee's gut about inaccurate applications.

Hard boiled, quantitative validation is a fantasy because the people selling and approving these loans are compensated on volume, rather than institutionalized scheme that is unlikely to change.

The current system works well for the local sales team. It fails the bank and it leaves consumers frustrated by consistent human failure and lack of honest feedback.

Surface is the first interactive system I have seen that has potential to intuitively streamline the loan application process and increase order accuracy.

Inputting loan applications on Surface will enable the NATIONAL standardization of the loan validation process. Subtle, interactive techniques (like progressive disclosure, contextual prompting and cross referencing) can automate vetting.

This will:
1. Increase the accuracy and truthfulness of applications.
2. Provide a better, transparent metrics-based decision process.

Consumers get a better experience. Sales teams make better loans. National decision makers gain necessary control over local processes and improve the quality of debt packages by decreasing the unknown unknowns.

That's a value proposition banks can get behind.

If this tickles your fancy, email me for our white paper "Consumer Lending and Microsoft Surface"

Tuesday, November 11, 2008

How to Create Great Multitouch User Experiences, Part 2 of 3

Here's the second Key Experience Design Concept For Multitouch:

All visible content communicates how to interact with the software.

A few rules of thumb really improve usabilty:
1. Visual Cues should communicate what the underlying software is doing, how to affect it and when users have been successful.
2. All nonambient visuals communicate changes int he information structure.
3. All unnecessary elements should be eliminated to minimize the chance of confusion.

Check out the video for examples.

Wednesday, November 5, 2008

How to Create Great Multitouch User Experiences, Part 1 of 3

As promised, here's the first video on the first rule of thumb for designing Multitouch interfaces:

1. Visual and auditory responses should make up for the lack of tactile feedback.

Tuesday, November 4, 2008

How and Why Apple Will Make a Macbook Touch

A couple of weeks ago, the Steve Jobs Juggernaut released updated software for its touchpads including a number new gestures. While Apple continues to push its "Brick" design. The real brick is the gimmicky "interface improvements".

See how and why Apple will create the next generation Macbook Touch here:

The 3 Most Important Experience Design Concepts for Multitouch

Over the next several Surface posts, we will be talking about the 3 most important experience design concepts on Microsoft Surface.

1. Visual and auditory responses should make up for the lack of tactile feedback.

2. All visible content should communicate how to interact with Surface.

3. Keep the content and the interaction as tightly coupled as possible.

Applications that incorporate these concepts are easier to learn and use...No more looking for the right button.

That's the power of Point and Do.

Monday, November 3, 2008

What is Point and Do...and why should you do it?

Here are some great examples of Point and Do on Surface.

Over the past several years, academics have been playing around with a concept called direct manipulation...the illusion that when you touch content on a screen, it direcctly affects that content...instead of using checkboxes, buttons and the like. A very cool idea with avery dumb name.

Jeff Han's Perceptive Pixel is likely the best known example of this idea.

While Jeff's stuff is undoubtedly neat, it, like the term Direct Manipulation smack of academia.

As Microsoft Surface and similar products come on line, we suggest "Point and Do" as a simpler, more accessible term that includes the breadth of experiences that they will make possible.

Sunday, November 2, 2008

Interknowlogy's Portfolio Reel

Tim Huckaby at Internknowlogy just put this up. They have done several significant demos on Surface. Their Vitruview / Angiographer demo made us in the audience drop our collective jaws when they showed it at Microsoft. 3D manipulation on Surface has definitely got Sizzzzzle, but there are a bunch of usability issues. We'll be reviewing this content later and pointing out opportunities for improvement. In the mean time, GREAT work Interknowlogy. Enjoy!