Things To Keep In Mind When Designing A Multitouch Table

Otto Von Bismark once said,

"Some wise men learn from their mistakes. I prefer to learn from those of others."

This is a wise maxim that we designers often forget. Julie Meridian recently wrote a great assessment of the Surface applications at Harrah's RIO iBar.



She talks eloquently and insightfully about the UX problems. en I look at a failure of this magnitude, I wonder how it came to exist...so I don't end up in the same place.

The issues I experienced with this system scream of poor project management and a failure to ask for outside help. You wouldn't just pick up a Trumpet and try to play it. Why multitouch?
In the current economic climate, I know that asking for this sort of thing isn't popular, but the other alternative is that the project WILL either run out of control or there will be signifiant usability issues.

If you are starting planning on your first multitouch project, please for your own sake, reach out and ask for experienced help during the planning stage. It will pay back in spades.

Below, I've written out the top 13 issues I see when reviewing work for others:

Layout
- Occlusion (what content is blocked by hands or other people)
- Button size (Fitts' Law) - Text size create (legible from 4' away)
- Usable from multiple directions

Visuals
- Highly visual: The less words and the more specific the imagery, the easier it will be for a broad demogrphic to use the device.
- No irrelevant information (Anyone ordering olives in their Bailey's simply needs to be cut off)
- Transitions: When working on your state/transition diagram, Transitions should be planned carefully. The way content and controllers animate can do a huge amount to suggest what they do, how to use them, and whether touch events have been registered.

Interaction
- Strong progressive disclosure
- Modeless controls
- Privacy (what and when should various users have access to what information, private pin #'s, etc.
- Provide a visual cue that all touches have been registered...even when they don't trigger a state change.

Form Factor and Environment
- Appropriate height and orientation
- Infrared lighting (for vision-based systems like Surface)
- Orientation is appropriate for the length of the interaction.

Keeping Users Oriented in Complex Multitouch Interfaces

Last week, I started a series of posts about the greatest interface challenges for Multitouch applications.

People love multitouch because it feels natural and easy to use. There are a couple of reasons for this:
1. Users are able to directly manipulate content in a nonmodal interface

2. Users are able to use kinesthetic (touch) memory to remember where thing are and how to accomplish tasks, just like they would other physical tasks.

These requirements create almost insurmountable layout limitations for a complex system.

Let's start with the first limitation:

Fitts' Law

Fitts' law is a remarkably successful predictor of appropriate button size in multitouch systems:



If a common interface like Microsoft Word were modified for Surface using Fitts' Law, The screen would look something like this.

Ignoring the fact that data entry applications don't work well in multitouch, it's easy to see a much larger problem: There's no room for the actual content.

Modal Interfaces

Simply putting all of the controllers on screen (actually, less than 1/10 of Word's 1800 features are directly accessible in this view) makes the interface unusable.

The only way to enable a feature rich application is to HIDE A LOT OF CONTROLLERS.

This strategy is called a Modal Interface. If you Google this term, you will find nothing but vehement opposition for this strategy. HCI people have an almost religious hate for this stuff...because it's hard to learn, it provokes user error and is frustrating as hell when users don't know what mode they are in. Suddenly nothing works.

When interfaces are highly modal they are, by nature, less intuitive. For these interfaces to work, users need to know that modal and "quasi-modal" solutions (content and controllers that appear on command and disappear when released) exist, how to call them up and how to release them.

This can rapidly become overwhelming to new users...Even in a relatively simple, superbly designed application, like the tablet-enabled Alias Sketchbook Pro.


State Orientation

For modal interfaces to work, they require that the user always know the current application states and modifiers available to them. This can be as simple as changing the treatment of content and controllers to show which application elements are currently active: like these buttons from the Vista Application Bar


Often times, in multitouch applications, more is required. The application needs to show how multiple pieces of content relate. As seen in the Reactable demo below, this can be extremely "pixel inefficient".


When the users approach new interfaces that have numerous possible states, it is remarkably difficult to avoid confusion and keep the user oriented. Doing so requires the extensive use of affordances. Each of these affordances needs to be large for the user to actually notice state changes.

This leads us full circle to the maximum usable size of a multitouch surface: the comfortable breadth of an outstretched arm...or about 36". Bottom line: In multitouch, the there will never ever ever be enough pixels.

What do you think? What ideas do you have for running around these usability traps?

Joe Fletcher Talks About Designing for Multitouch

The Problem with Zooming Interfaces ZUI

Recently, I've been looking at using zooming interfaces in multitouch, like the much vaunted Seadragon and Google earth. The underlying problem with these sorts of interfaces is that, while direct manipulation of the content is extremely compelling, it's very difficult to index between content that have weak relationships to eachother.

It's easy to get from this:


To this:


...but how do users efficiently get from one part of the image map or datacloud to another without a structured hierarchy (zooming is effectively a way to move up and down a tree)?



To get to highly unrelated data over here:



This still strikes me as 1 of the top 3 unsolved interface issues in multitouch (more to come on the other 2). In GUI's this function would typically be handled by a search box or similar command line interface.

Google is a great example of an interface that uses this strategy to move efficiently from one link in a datacloud to another ... but multitouch interfaces are fantastically bad at character input.

How would you go about doing this in multitouch?

Infostrat Opensources Surface Virtual Earth Code

I guess I'm more influential than I thought. Check out the sweetness.
http://virtualearthwpf.codeplex.com/



Thanks to the might Joshes of Infostrat for this!

Texture Mapping and Image Manipulation Using Multitouch

Thomas Hansen put together this demo of a Python-based system that he is building out of the University of Iowa. Check out the 3D texture mapping at :03 and the image distortion at 2:39. Thomas proves once again that ideas don't have to be complex to be cool.


pymt demo reel from Thomas Hansen on Vimeo.

Excellent Job Opportunity In London

My colleagues are doing an insanely great project in London. I wish I could talk about it ... They need some immediate contract or fulltime help from someone with:

- Strong knowledge of Photoshop and Illustrator.
- Working knowledge of After Effects and ability to create motion graphics
- Able to work with a large variety multimedia formats
- Experience designing & developing for offline standalone flash applications
- In depth knowledge of Actionscript and the OOP Model

Minimum requirements:
You must be living in London, have the paper to work in the UK, preferably be British and have 3-5 years work experience.

Drop me a line (at the email to the left) if you know the right person and a CV if you are them. I promise you will want this job.

~Jonathan

Designing Better Multitouch Affordances

In multitouch interface design, visual hints (affordances)are keystones for usability. While the MS Surface demos provide one vision of the user experience, there is plenty of room to better articulate how to use controllers and what to do with content.

I believe that motion, lighting and color are keys to achieving this.

Matt Lambert and Favorite Color put together this beautiful piece that I have found inspiring in my work. I hope you feel the same.



How could better lighting and motion design make your software exceptional?


Check out Matt Lambert at:
http://dielamb.com/

Alksolutions Opensources a Cool Demo

Can Balioglu is open sourcing the code.

http://www.alksolutions.com/en/Blog/RecentPosts.aspx

This is the best thing that could happen for the Surface community. We need to start getting a bit more communal. Noone seems to be lining up to resell components, so I hope more developers get into the spirit of sharing!

I'll be glad to let the community know about any opensource distributions. Just let me know. Everybody could sure use a nice virtual earth solution. Hint, hint JoshB.

Beautiful Surface Retail Application from d'strict

I didn't know that Surface was distributing in Korea yet, but it looks like the badasses at d'strict got their paws on one.

I
UX SOLUTION, TOUCH I Commercialization from phoebe on Vimeo.

While this application is really just a bobble in a retail playground, it points to how interactive video and 3D manipulation can work together to create amazing content.

It clarifies that the Surface hardware can actually look quite snappy when when installed in a proper case.

Building an Effective Multitouch Keyboard

Apparently, Asus is dropping a new multitouch enhanced keyboard.



While I am completely in love with integrating hard and soft interfaces, this isn't the way to do it. Replacing a numeric keypad forces users to look down when they are using it and likely take both hands off the keyboard. This massively decreases user flow and productivity.

Alternately, placing an angled touch screen between the keyboard and monitor decreases lateral head motion and encourages users to keep one hand on the keyboard at all times. This both improves speed of task completion and keeps users focused on tasks instead of the interface.



We toyed with this a while back and got the entire interface of Photoshop to work on this form factor.

Why Microsoft's Future Vision Will Fail

Over the past year, Microsoft has produced a stunning series of “future vision” videos that express where new hardware technologies will take computing.

<a href="http://video.msn.com/?mkt=en-GB&playlist=videoByUuids:uuids:a517b260-bb6b-48b9-87ac-8e2743a28ec5&showPlaylist=true&from=shared" target="_new" title="Future Vision Montage">Video: Future Vision Montage</a>

At first glance, the underlying argument looks really sexy. The 20,000 view of the technology mix looks something like this:

Users will want unified communication between and more contextual interaction with devices.

This will be enabled by:
- Complex sensor networks
- Voice recognition
- Cross-device interactions
- Simple, idiot-proof security
- Cloud computing
- Constant geolocation of users

An Inconvenient Truth
Unfortunately, the amazing slickness factor of these videos (they are produced by Digital Kitchen), covers up an inconvenient truth: Microsoft believes we will be using tomorrow's high speed wireless networks, powerful batteries, sensing and display technologies to do what we do with computers now:

- Authoring, editing and organizing digital content
- Assembling customized solutions
- Obtaining and understanding information
- Communicating
- Meeting
- Working from anywhere
- Scheduling

The idea that businesses will spend on technology that enables workers to do the same basic tasks we do today is an insane argument.

The economics of hardware require a new platform every ~36 months, so we can rely on planned obsolescence on that timeframe. If all of these devices need to work together seamlessly, they will need to be replaced on that cycle or become outdated all at once.

Web 3.0 and Context Aware Computing need to create a MASSIVE increase in worker/material efficiency to jsutify the invesment. More of the same is just not a compelling argument to build the neccesary IT army to manage it. The cost and effort to constantly track and replace sensors, displays, routers and computers will be incredible.

The Scenario Makes 3 Wrong Assumptions
1. Tech manufacturers will play nice with each other
2. High-density/high-band width cloud computing will work.
3. Federated authentication and human tracking are good ideas.

Industrial Collaboration
The Microsoft Future Vision will require a closed hardware/software platform to work seamlessly.

The need for this goes beyond the performance bump of integrated technologies to the basic physics of the situation. IT managers simply won’t be able to cable all of this stuff. It will have to be wireless.

1-10k “Web 3.0” wireless devices in an office work group would create an RF density beyond anything you have experienced and beyond anything the FCC has ever considered approving. The sheer chaos of multiple vendors will scare off anyone who has any IT manager who has been burned by RF issues.

The resulting IP/System standards wars will turn off the few IT execs who aren’t scared of getting even more locked in to MS.

In any case, I’m unsure that systems lock-in is ever in anybody’s interest besides the vendor.

Limitations of Cloud Computing
At current growth curves, power available to datacenters in the US will run into a hard ceiling in 2012-13.

The energy usage required for constant cloud processing and delivery of realtime HD video for 300,000,000 consumers will be through the roof.

The amount of last mile wiring will require massive labor and egregious expense.

The materials used in optical switching limit the maximum speed of networks
Early adopters of digital video are choking the system today. 300,000,000 people downloading HD video at the same time would and will kill it..

The materials used in CPUs also have physical limitations.
Multicore computing is an essentially inefficient technology solution. Building 3D circuits with Silicon Germanium may help solve this problem, but this is an unproven technology and the tooling will be through the roof.

Federated Authentication and Human Tracking

Federated Authentication: The idea that one organization manages your security across many networks is a disaster waiting to happen.
When a hacker gets access to one ID, they can break all of the others. BAD. BAD. BAD ... but maybe good for Microsoft, because everyone has a hotmail account.

Human Tracking: Privacy Is Important
Noone wants their boss, employees and colleagues to know exactly where they are all the time, something 50% of men have cheated on their wives. Let me restate that. 50% of executives have or are cheating on their wives. They REALLY, REALLY, REALLY don't want to be trackable.