Ubiquitous sensors, networking, cheap computation and low cost displays are changing how we use and what we do with interactive devices. These new “augmented realities” will likely look much like Microsoft’s recent future vision videos (cameras, RFID and handheld devices as the primary interaction tools).
There are only so many new interfaces we can learn in a day. For this paradigm to be successful, designers will need to understand the dynamically changing models of user, task and system that these videos demonstrate. The interface design model will have to change from one in which users focus their attention on inputting commands to one in which devices use contextual data to identify intent.
While this isn't fully expressed in the following video, I believe it is where Microsoft's Dane Storussen and August De Los Reyes are attempting to go with this video:
This new mindset will go beyond rethinking existing peripherals to reimagining how we experience all of the objects and information in our everyday lives. While Microsoft would like you too think this is a new idea, it isn't.
I have been involved in this space for nearly a decade and in academia, this sort of thinking has gone by a number of names, including: XUI, Gestural Interfaces, Tangible Media, Context Aware Computing and Augmented Reality. Whichever flavor you prefer, it will require a fundamental rethinking of HCI.
For the purposes of simplicity, let’s call this field of design XUI. It seems that XUI has five basic tenets:
1. Technology should help users:
- Identify strengths, augment limitations
- Streamline social interactions
- Remember events as they actually happened
- Understand complex datasets.
- Learn how to achieve goals more easily and intuitively
2. Technology should focus on augmenting the experiences people have, not the hardware used to achieve that task.
- Systems should measure intent as well as action. Eye tracking measures what you are thinking about. Mouse tracking measures what you did.
- Interfaces should strive to improve complex behavior by recognizing contextual cues that signal a a desire for augmentation.
3. The first system priorities should be to identify:
- Whether it is safe and appropriate to provide feedback
- How to keep users focused on achieving their goals
- Discovering when users want to explore vs. when they need help achieving a specific goal.
4. The primary objective of interfaces design should be make users think the right thought, not to tell users what to do.
- Interfaces should always be useful, even if when they fail to achieve the user's goal.
- Interfaces should subtly coach users to use scheduled variable affirmations to improve user performance.
- When there is a high opportunity for error, an important task or destructive command, input events should be verified through at least two separate senses. This significantly decreases input error.
5. The interface should be "experiential".
- Wherever possible, animation, appropriate sounds and haptic feedback should be used instead of text. Users are likely to respond more quickly and enjoy the experience more.
If you are interested in designing XUI's, here’s a talk by one of my mentors, Ted Selker, formerly of IBM Research and the MIT Media Lab talking about the decade of research projects at the lab:
Your comments and help expanding this list would be deeply appreciated.