That was fast!

Wasn't it not only a few months back that we first heard of multi touch technology, the iPhone and Jeff Han? I recently gave a lecture on touch technology at the University of Salzburg, Austria. I wanted to start my talk by going back to the roots. So I (image-)googled the word "touch" expecting to find images of mothers with babies or perhaps pornography. Instead my search returned pictures of devices equipped with touch technology - over 58,000,000.00 of them!! Has the word "touch" become synonomous with "touch technology"?

I am surprised how fast it trickled into the public conciousness, especially since touch technology only uses a small fraction of our actual sense of touch. The abilities to sense temperature, shape, degrees of softness, texture, pain or the position of your muscles and joints are not playing any part in touch technology so far. A slick surface provides little haptic feedback which for example makes typing difficult.

At this year's Cebit convention, T-Mobile had an multi-touch installation, that was part Minority Report, part Jeff Han Screen and part vertical MS Surface "wall". In this video people seem so bored with the content itself. In fact they are not dealing with the data at all. How long will it take for the novelty of scaling and turning objects to wear off? What kind of interesting public application could this offer?

No Flash Player for iPhone Anytime Soon!

(Sound of expelled breath...)

At the shareholder meeting two days ago Steve Jobs spelled it out: no Flashplayer for the iPhone! The reasons? Flash Lite, Adobe's Flash Player for the mobile platform is too weak, the desktop version too slow on the iPhone. The iPhone, in other words, needs a new Flash Player!

(not holding my breath here...)

Since Safari on the iPhone is the real thing, a Flash Lite version of the player wouldn't help anybody. Most Flash sites for the internet simply can't be transported to the lite player. More then ever it is best practices to have an html version of your Flash sites to avoid user frustration.

Applelujah!

Physical Interfaces: BumbTop, the iPhone and Leopard

Recently I rewatched Anand Agarawala's demo BumbTop at TED 2007. BumbTop is a three dimension cubicle desktop. Documents on the desktop have physical qualities like weight, size, light reflection etc. They can be stacked, flipped through, bumbed into and organized in ways more similar to a real world desktop.



What struck me in the demo was the audience's reception. Partly due to Anand's humorous presention style, but largely because of the surprise effect the interface holds, the audience was clapping, laughing, even cheering. When the iPhone was introduce in January there were tons of moments like these. When Steve Jobs first scrolled through a song list with the flip of his fingers, the audience erupted. It is the RE-cognition factor. In that instance we identify a resemblance between real life physical behavior and the onscreen simulation of such physicality. It is a magical moment.

At the WWDC 2007 Steve Jobs presented the latest beta release of Leopard, Apple's upcoming OS for Macintosh. For a long time I imagined Apple's designers to arrive at their user interface solution by cheer genius. It never occured to me that the might take their "inspiration" from others, even purchase UI solution like Coverflow (first introduce to the general public with iTunes 7) from Steelskies . With the release of Leopard it will be a standard way to view information in its redesigned Finder.


"Stacks" in Leopard's Desktop

Leopard continues and broadens its use of animation as a general interface element. The Dock tags on the new "stack" feature, which may have been borrowed from BumbTops stack function, but lacks its physicality. Other Leopard features show me the direction Apple's interfaces are headed. Coverflow, Time Machine, Stacks and Spaces are all feature that rely heavily on animation. At the heart of them is Core Animation, a set of animation routines offered to developers to easily produce animated interfaces.

Leopard looks fun to use, but generally feels light, digital and spacey (quite literally in Time Machine). Animations in Leopard accelerate and decrease in speed which is a quality taken from a real world physicality. There is a new quality of light and shadow peeking through the new 3d-ish Dock design, but compared to BumbTop there is no sense of weight. The reason for that might be that we are still using a mouse pointing device.
In BumbTop is strikes me as odd to manipulate objects imbued with weight using a mouse pointer which cannot return physical feedback that resembles pushing weighted objects around. In makes sense that Leopard lacks the physical quality of weight. Leopard will also run on the iPhone, where the manipulation of information is accomplish with your fingers. While weighted pressure on its multi-touch surface will have no bearing, speed of movement does and thus represents one more step in the direction of physical interfaces.

Much To Do About Microsoft Surface

Microsoft's introduction of surface computing has caused a ruckus on the IxDa, which is the discussion board of the Interaction Design Association. There was a huge backlash on the product and I commented negatively as well. I was disappointed that already at the introduction of the product Microsoft had partnered with chain hotels and casinos to introduce the product to the public in the near future. It just shifted the focus from innovation toward selling the product. Many of the shown applications are lame like showing augmented information about the wine you are drinking right on the surface of the table...

On the other hand I really like the interaction with real objects. Using the table to share data between different devices. There is a lot of promise in that. I wonder if they could have solved the recognition problem with RFID tags instead of cameras. Cameras need distance, which makes the table bulky.

Ideas and innovation happens in synchronicity. All of a sudden a few people have the same ideas around the same time. Much of it has to do with technology moving to the next level opening up the doors for certain new ideas. 
This has happened for multi-touch. The iPhone made it popular and the time for multi touch computing is now. It's everywhere and we are going to see a lot more of it. Exciting times.

This is another example of multi-touch used for a collaboration instrument as shown at the Transmediale 2007 in Berlin



This is one of MS's cheesy ads for Surface:




http://www.microsoft.com/surface/
http://www.microsoft.com/presspass/presskits/surfacecomputing/default.mspx

Tactile Interaction with Mobile Devices

More and more our digital devices and computers are being equipped with sensors. Physical computing is here at last! Apple's labtops have a motion sensor, that picks up fast movement. If a labtop is being dropped, it will shut down the spinning hard drive instantly to prevent damage to it. The upcoming iPhone is going to be equipped with proximity sensors as well as a accelerometer that detect when the device is being rotated from portrait to landscape.

Beyond the commercial applications there is exiting research being done at the university level. Steven Brewster, Faraz Chohan and Lorna Brown from the University of Glasgow introduced and tested tactile vibrational feedback on touchscreen devices. Typically people would like to use their mobile devices to get work done on subways and busses. Entering data is prone to errors because of shaking vehicles and environmental noise. Auditory feedbacks as tested by Brewster improve usability, but add no benefit in noisy conditions. In this system successful and erroneous typing (double taps or slips) each produce a specific feedback that suggests a "smooth" (success) vs. a "rough"(error) sensation. The study showed significant usability improvement. It increased the amount of data entered by the user and more successful error correction.
More information can be found here.

Technologyreview.com reports today (05-16-07) that the "British Telecom tries to wed Nintendo Wii-style technology to a tablet PC". Tilting and rotating the tablet PC would let the user navigate the computer "Etch-A-Scetch" style. While studying for my Masters (1999) Han Gene Paik and I developed an email application prototype that used tilt sensors to navigate in a similar fashion. Our concept departed from the WIMP interface and suggested the on screen environment to be filled with water. Sorting, filtering and selecting the email objects felt like digging for gold nuggets.


"Liquid Mail" by Han Gene Paik and Dirk J. Platzek

Today these accelerometer sensors are a lot more sophisticated and very tiny. They are known as microelectromechanical systems (MEMS) and are used for example for the iPod Sport Kit , a joined effort by Apple and Nike. Accelerometers are placed into special running shoes. A wireless connection transmits the information to the iPod.

Another exciting idea in the area of physical computing was just introduced at the Proceedings of the SIGCHI conference on Human factors in computing systems 2007, in San Jose, California: "Shoogle", - excitatory multimodal interaction on mobile devices has been developed by John Williamson, Rod Murray-Smith and Stephen Hughes. It also uses accelerometers to pick up user gestures in exchange for vibrotactile and audio feedback. The system produces feedback that - for example - feels like balls of different weights bouncing inside a box. From the sound and vibrotactile sensation a user is then able to distinguish length or urgency of a message. It is not necessary for the user to look at the device, which is great in situations where that is not appropriate. The idea takes advantage "of user's familiarity with the dynamics of processes in the physical world to present information in a natural and non-irritating manner". The highly recommended article can be found here.

Apple's Patent for Back Side Multi Touch Interface

Just when I thought the iPhone's multi-touch-the-information was a breakthrough for a consumer product the next exciting interface appears at the horizon. Yesterday macdailynews.com reported that patents have been filed by Apple concerning their 6th G iPod. It looks like you can navigate the front face of the iPod with a touch panel located on the back side. Input and output will be on different sides of the iPod.

UPDATE 05-15-07: This post on Core77.com explains why a touch panel on the back is so smart: This way the user is no longer obscuring the very interface he/she is supposed to interact with...



The greatest innovation is often the timely implementation of a particular technology that often has been around for a while. This only happens after other technical hurdles, such as CPU speed, bandwidth or component size have been solved. The iPhone made me research multi-touch technology and I have written about it in previous posts. During my studies I discovered a touch table interface on youtube.com which utilizes the underside of the table for additional navigation as well.



Matthias Müller Prove recently reminded me of Alan Kay's famous quote: "The computer revolution hasn't happened yet". Physical computing finally makes it to the market place. Only the very best will catch our attention. Apple stock briefly climbed to an all time high yesterday again ....

The Emotional iPhone Experience

Mac owners always had a more emotional relationship to their machines. That it is a great accomplishment on the part of Apple's designers. I like looking at my MacBook Pro every day! Touching it, using it is a delight. To prove my gushiness isn't rare to find amoung Mac users I'd like you to take a look at this video by Jason Ponton, Editor of The MIT Technology Review. He explains what makes a beautiful machine and his enthusiasm is a great example for the kind of emotional bond I am talking about.



Cell phones are intimate machines. We carry them close to our bodies and use them primarily to communicate, they are connectors to the world around us. I have never built that emotional relationship with any of my cell phones ( I am on my 5th phone now) . The novelty of a new phone wears off after a couple of weeks. It becomes a mere tool I expect to work. Since its features a miserably integrated I don't bother to use them. I am still doing the same few things with my phone I have always done: calling, texting and entering data into the calendar and the address book. My personal feature wish list has not changed for years. I don't need more features just a few done the right way. Nothing but sad compromises so far....

I think the iPhone will create that emotional relationship.

Why?

The integration with my Mac will be perfect. The iPhone will behave like my computer. It will be its ultra portable extension.

It will feel great in my hand, the multi-touch interface will make it feel like a real extension of my body. I will be able to directly touch and manipulate the information!

The overall success of the iPhone may depend on how well it will integrate with the Windows machines. The exclusive provider deal with AT+T is also annoying. No matter how great the iPhone is, I will not change my current phone plan for it.

I guess...

ZUIs: Archy, Zenzui, iPhone and Deepfish

When I wrote the post about the Zenzui interface I wasn't aware of the fact that efforts have been made for a long time to develop ZUIs - Zooming User Interfaces. The idea is a desktop that is infinitely expandable in all directions, as well as zoomable to get an overview or to show detail. To my knowledge the Zooming User Interface is one part of "ARCHY", a proposed system for interacting with computers designed by Jef Raskin. He first described these ideas in his book The Humane Interface which was published in 2000.





Apple is making efforts in that direction with the introduction of virtual desktops called "Spaces" in the upcoming release of Leopard. The upcoming iPhone also uses a stylized form of ZUI. Since the iPhone will use Safari and the full web experience, Apple needed a way to pan through the information on a small screen and came up with double tapping to zoom in and out, as well as pushing/panning of the page with swipes of your finger.
Apparently Apple has beat Microsoft efforts in that area to the punch. Look at Microsoft's Deepfish (see Demo) technology. The similarities are obvious, although the navigation of the interface by joystick seems a bit clutsy.

It is always so interesting to me, that all these concepts have been around for a long while. Some companies - usually Apple - just implement them better. Then they are called innovators, but their ideas were never produced out of thin air. It's a process and many smart people have contributed.

Zenzui - Zooming User Experience and Business Model

Zenzui's user interface design has some merits - I would like to check it out. The content seems to be like widgets, rather than a navigational system for real phone applications like email, calendar etc.
I like the tiles' conceptual extention past the boundaries of the users screen resolution. Zooming in and out seems intuitive. Instead of scrolling, which on "non-iPhone" phones is cumbersome to do, they have small interface elements let you navigate in 4 directions or else zoom in or out (essentially opening and closing of widgets/apps/files).



The Zenzui business model aims to satisfy consumers, marketers, advertisers, developers and mobile operators, which means it will be a system filled with ad-crap.

In the video below zenzui sometimes seems latent and slow. The Zenzui interface has the potential to be a direct touch interface (like the iPhone it doesn't really need hardware buttons), but bends to so many different mobile manufacturers that the user experience is largely uncontrollable.

Jeff Han at TED 2006

I've posted on multitouch technology before, but watching Jeff Han from the NYU Research Lab in NY talk about their research just adds another layer. The demo shows him manipulating data and objects with his hand and fingers using a variety of gestures. Rarely have I seen a more convincing way of Human Computer Interaction. It makes complete sense right away and makes you wish time would fly so we could all use machines like that.

Jeff Han: Multi-touch Screens - the steroid version

Started reading John Maeda's "The Laws of Simplicity" today. He keeps a blog lawsofsimplicity.com, that continuously develops the ideas further. A few clicks later I ended up on the Signal-vs-Noise blog and eventually saw this amazing video showing Jeff Han and Phil Davidson demonstrate how multi-touch screens “will change the way we work and play.” Examples include usage in image editing, web browsing, creating 3D animations, etc.

These guys were announcing on their site that they were up to some exciting things. One of my very first posts was about their work. This is the amazing development:



http://link.brightcove.com/services/link/bcpid271543545/bctid422563006

Jeff Han, NYU, Multi-touch Technology

Just must have hid underneath a rock for the last two weeks, if you haven't heard of Apple's new iPhone. What came as a surprise to me was the fact that multi-touch screens have been around for almost 20 years. Sitting there in laboratories...why hasn't anybody picked up on this technology?
Look at this: AWESOME!