That was fast!

Wasn't it not only a few months back that we first heard of multi touch technology, the iPhone and Jeff Han? I recently gave a lecture on touch technology at the University of Salzburg, Austria. I wanted to start my talk by going back to the roots. So I (image-)googled the word "touch" expecting to find images of mothers with babies or perhaps pornography. Instead my search returned pictures of devices equipped with touch technology - over 58,000,000.00 of them!! Has the word "touch" become synonomous with "touch technology"?

I am surprised how fast it trickled into the public conciousness, especially since touch technology only uses a small fraction of our actual sense of touch. The abilities to sense temperature, shape, degrees of softness, texture, pain or the position of your muscles and joints are not playing any part in touch technology so far. A slick surface provides little haptic feedback which for example makes typing difficult.

At this year's Cebit convention, T-Mobile had an multi-touch installation, that was part Minority Report, part Jeff Han Screen and part vertical MS Surface "wall". In this video people seem so bored with the content itself. In fact they are not dealing with the data at all. How long will it take for the novelty of scaling and turning objects to wear off? What kind of interesting public application could this offer?

Tactile Interaction with Mobile Devices

More and more our digital devices and computers are being equipped with sensors. Physical computing is here at last! Apple's labtops have a motion sensor, that picks up fast movement. If a labtop is being dropped, it will shut down the spinning hard drive instantly to prevent damage to it. The upcoming iPhone is going to be equipped with proximity sensors as well as a accelerometer that detect when the device is being rotated from portrait to landscape.

Beyond the commercial applications there is exiting research being done at the university level. Steven Brewster, Faraz Chohan and Lorna Brown from the University of Glasgow introduced and tested tactile vibrational feedback on touchscreen devices. Typically people would like to use their mobile devices to get work done on subways and busses. Entering data is prone to errors because of shaking vehicles and environmental noise. Auditory feedbacks as tested by Brewster improve usability, but add no benefit in noisy conditions. In this system successful and erroneous typing (double taps or slips) each produce a specific feedback that suggests a "smooth" (success) vs. a "rough"(error) sensation. The study showed significant usability improvement. It increased the amount of data entered by the user and more successful error correction.
More information can be found here.

Technologyreview.com reports today (05-16-07) that the "British Telecom tries to wed Nintendo Wii-style technology to a tablet PC". Tilting and rotating the tablet PC would let the user navigate the computer "Etch-A-Scetch" style. While studying for my Masters (1999) Han Gene Paik and I developed an email application prototype that used tilt sensors to navigate in a similar fashion. Our concept departed from the WIMP interface and suggested the on screen environment to be filled with water. Sorting, filtering and selecting the email objects felt like digging for gold nuggets.


"Liquid Mail" by Han Gene Paik and Dirk J. Platzek

Today these accelerometer sensors are a lot more sophisticated and very tiny. They are known as microelectromechanical systems (MEMS) and are used for example for the iPod Sport Kit , a joined effort by Apple and Nike. Accelerometers are placed into special running shoes. A wireless connection transmits the information to the iPod.

Another exciting idea in the area of physical computing was just introduced at the Proceedings of the SIGCHI conference on Human factors in computing systems 2007, in San Jose, California: "Shoogle", - excitatory multimodal interaction on mobile devices has been developed by John Williamson, Rod Murray-Smith and Stephen Hughes. It also uses accelerometers to pick up user gestures in exchange for vibrotactile and audio feedback. The system produces feedback that - for example - feels like balls of different weights bouncing inside a box. From the sound and vibrotactile sensation a user is then able to distinguish length or urgency of a message. It is not necessary for the user to look at the device, which is great in situations where that is not appropriate. The idea takes advantage "of user's familiarity with the dynamics of processes in the physical world to present information in a natural and non-irritating manner". The highly recommended article can be found here.

Apple's Patent for Back Side Multi Touch Interface

Just when I thought the iPhone's multi-touch-the-information was a breakthrough for a consumer product the next exciting interface appears at the horizon. Yesterday macdailynews.com reported that patents have been filed by Apple concerning their 6th G iPod. It looks like you can navigate the front face of the iPod with a touch panel located on the back side. Input and output will be on different sides of the iPod.

UPDATE 05-15-07: This post on Core77.com explains why a touch panel on the back is so smart: This way the user is no longer obscuring the very interface he/she is supposed to interact with...



The greatest innovation is often the timely implementation of a particular technology that often has been around for a while. This only happens after other technical hurdles, such as CPU speed, bandwidth or component size have been solved. The iPhone made me research multi-touch technology and I have written about it in previous posts. During my studies I discovered a touch table interface on youtube.com which utilizes the underside of the table for additional navigation as well.



Matthias Müller Prove recently reminded me of Alan Kay's famous quote: "The computer revolution hasn't happened yet". Physical computing finally makes it to the market place. Only the very best will catch our attention. Apple stock briefly climbed to an all time high yesterday again ....

Neurofeedback Gaming Interfaces

Lately I've noticed a renewed interest in neurofeedback interfaces. Yahoo News reported on "mind reading toys" and mentioned Silicon Valley gaming companies such as NeuroSky Inc and Emotiv Systems Inc.



Electroencephalography, or EEG, is the basis of these technologies. It measures the brain's electrical activity through electrodes placed on the scalp. Mostly these techiques are used in clinical contexts to help patients with migranes or kids who suffer from ADD. Beyond that there have communities obsessed with neurofeedback technologies since the 60s, claiming to achieve optimized mental performance.

Steven Johnson's book Mind Wide Open from 2005 describes his own encounter with neurofeedback devices. He says: "After the initial amazement had worn off while I was playing the space game at Seiden's office, I couldn't help noticing that I couldn't control the ship with nearly as much accuracy as I could have with a joystick or a keyboard. There's a fuzziness to interaction that would be unpleasant were I actually interested in having an efficient conversation with the computer."

and

"The almost unthinkable complexity of the brain's information network is necessarily compressed down to a crude language when a machine listens to the collective rhythms of brain waves through the skull."

NeuroSky is about to unveil a headset with brainwave "dry" sensors that won't require a conducting gel to pick up brain activity.



While looking futuristic and usable I wonder how much brain activity they will be able to pick up, when scientific instruments with a lot more sensors produce "crude" results.

Emotiv Systems Inc. "Project Epoc" headset resembles more the know EEG instruments and interface wirelessly with all gaming consoles.



The company's wiki states: "Emotiv uses proprietary, patented algorithms to formulate detections of activities from the reading of EEG signals. These detections can determine a user’s expressions, feelings and thoughts." Apparently the technology picks ups the player's muscle twitches and thus facial expressions. The player's character would smile in synch with the player or "react" to the player with a range of feelings like surprise or anger. In other words interaction with an artificial intelligence.

Last but not least I found a "brainwave dance performance" on youtube. There is an introduction on how the sensors attached to the dancer work followed by pretty bad dancing:)

The Emotional iPhone Experience

Mac owners always had a more emotional relationship to their machines. That it is a great accomplishment on the part of Apple's designers. I like looking at my MacBook Pro every day! Touching it, using it is a delight. To prove my gushiness isn't rare to find amoung Mac users I'd like you to take a look at this video by Jason Ponton, Editor of The MIT Technology Review. He explains what makes a beautiful machine and his enthusiasm is a great example for the kind of emotional bond I am talking about.



Cell phones are intimate machines. We carry them close to our bodies and use them primarily to communicate, they are connectors to the world around us. I have never built that emotional relationship with any of my cell phones ( I am on my 5th phone now) . The novelty of a new phone wears off after a couple of weeks. It becomes a mere tool I expect to work. Since its features a miserably integrated I don't bother to use them. I am still doing the same few things with my phone I have always done: calling, texting and entering data into the calendar and the address book. My personal feature wish list has not changed for years. I don't need more features just a few done the right way. Nothing but sad compromises so far....

I think the iPhone will create that emotional relationship.

Why?

The integration with my Mac will be perfect. The iPhone will behave like my computer. It will be its ultra portable extension.

It will feel great in my hand, the multi-touch interface will make it feel like a real extension of my body. I will be able to directly touch and manipulate the information!

The overall success of the iPhone may depend on how well it will integrate with the Windows machines. The exclusive provider deal with AT+T is also annoying. No matter how great the iPhone is, I will not change my current phone plan for it.

I guess...

Shiira Web Browser 2.0 Released

I was looking forward to testing the Shiira browser, because the demo video featured these really nice functions:

1. Select and open multiple links in new tabs
2. Tabs Exposé feature.

Use the cursor to highlight any part of the web page, - right click and choose the "Open all Links in new Tabs" command. The drowdown menu tells you how many links you have selected, so you can gage how many tabs will open up at once.



To me that seems more useful then Safari's "Open in Tabs" bookmarks feature. Together with the new Tabs Exposé you get a key feature: quickly scan the open tabs in Exposé mode and find your information faster.



Shiira desperately needs a "close all tabs" (but the first one) command. I tried to open 40 tabs "by mistake". My hard core test could not be corrected. Closing the entire window made the browser crash - repeatedly. It does not appear to be completely stable yet.

Shiira's shelf is well organized. All bookmarks from all browsers on my machine were thoughtfully added. Unlike Firefox you don't have to choose between a bookmark or a history side panel.




download the application here.

Twitter, Dodgeball, Mashups, Web 2.0 and the Ecstacy of Communication

Until I read the post "Map-O-Mania" by Erhard Wimmmer I had no idea what on earth "twitter" meant as far as Web 2.0 goes. I had missed this!



The oxford dictionary defines it as follows:



• verb

1 (of a bird) make a series of light tremulous sounds.

2 talk rapidly in a nervous or trivial way.









Twitter.com concerns itself with one question: What are you doing right now? You can use the phone, IM or the web to submit SMS-length messages that will be broadcasted to your "friends" (all friend receive the twitter SMS). In additional - if I understand correctly - these messages go public on the twitter site. It's like a constant stream of disembodied ramblings that people can chose to contact. What gives?



It gets interesting when Twitter is being used in mashups sites like twittervision.com. The human need to communicate is relentless and we are right smack in the middle of a giant exstacy of communication.



In 2001 Han Gene Paik and I embarked on a project called "Guerillofitti". The concept was different, but the method was the same. This is from our original project proposal:



["...Using cell phones and other handheld devices that are meant to enhance "private" communications, participants instead publicize their opinions (read: twitter) on a censor free message board installed in a "public" space. This conversion of a "private-to-private" communication to a "private-to-public"..."]



We intended to have a projection or video wall installed in public that would display the messages. It seems now that Web 2.0 and our devices have become the public board. We are public with absolutely everything. I can only presume that anonymity and privacy don't concern people. On twittervision.com you even get to see where people are located, since it is mashed with google maps. Other services like Google's Dodgeball also function to locate your friends, - if they happen to be in your vicinity. The purpose is once again social interaction. Personally I find it alarming to be so locatable, but it might be naive to think that I couldn't be found, only because I don't "twitter" or "dodgeball".

ZUIs: Archy, Zenzui, iPhone and Deepfish

When I wrote the post about the Zenzui interface I wasn't aware of the fact that efforts have been made for a long time to develop ZUIs - Zooming User Interfaces. The idea is a desktop that is infinitely expandable in all directions, as well as zoomable to get an overview or to show detail. To my knowledge the Zooming User Interface is one part of "ARCHY", a proposed system for interacting with computers designed by Jef Raskin. He first described these ideas in his book The Humane Interface which was published in 2000.





Apple is making efforts in that direction with the introduction of virtual desktops called "Spaces" in the upcoming release of Leopard. The upcoming iPhone also uses a stylized form of ZUI. Since the iPhone will use Safari and the full web experience, Apple needed a way to pan through the information on a small screen and came up with double tapping to zoom in and out, as well as pushing/panning of the page with swipes of your finger.
Apparently Apple has beat Microsoft efforts in that area to the punch. Look at Microsoft's Deepfish (see Demo) technology. The similarities are obvious, although the navigation of the interface by joystick seems a bit clutsy.

It is always so interesting to me, that all these concepts have been around for a long while. Some companies - usually Apple - just implement them better. Then they are called innovators, but their ideas were never produced out of thin air. It's a process and many smart people have contributed.

Zenzui - Zooming User Experience and Business Model

Zenzui's user interface design has some merits - I would like to check it out. The content seems to be like widgets, rather than a navigational system for real phone applications like email, calendar etc.
I like the tiles' conceptual extention past the boundaries of the users screen resolution. Zooming in and out seems intuitive. Instead of scrolling, which on "non-iPhone" phones is cumbersome to do, they have small interface elements let you navigate in 4 directions or else zoom in or out (essentially opening and closing of widgets/apps/files).



The Zenzui business model aims to satisfy consumers, marketers, advertisers, developers and mobile operators, which means it will be a system filled with ad-crap.

In the video below zenzui sometimes seems latent and slow. The Zenzui interface has the potential to be a direct touch interface (like the iPhone it doesn't really need hardware buttons), but bends to so many different mobile manufacturers that the user experience is largely uncontrollable.

Adobe Apollo Empowers Web Developers

A post on the websector blog convinced me to rethink my remarks from yesterday on Adobe's Apollo technology. Today I realized that iTunes has been doing the same thing for years and I never once thought about the fact that content in iTunes is local as well as web content.

This in-depth article by Adobe's Michael Chambers explains what Apollo is exactly, why Apollo apps are NOT going to replace browser apps, etc.

Adobe Apollo: Why do we need it?

This is a link to a video from DEMO 07, a launch venue for new products, technologies, and companies. Adobe just released the public Alpha of Apollo, which lets developers built desktop applications using web technologies such as Adobe Flash, Adobe Flex, AJAX, (X)HTML or PHP. It shows an example of an eBay Branded Apollo Application.

http://www.adobe.com/devnet/videos/apollo_demo07/index.html

On first impression I find it annoying to mash the web with the desktop. Do I want to interact with ebay outside of the browser paradigm? Toggle to a Apollo-fied version of Ebay when I am already using the browser for something else?

I didn't really see the advantage in this demo and some of its functionalities were lame. For example: Why do I want to save an Ebay watchlist as an Excel speadsheet?

All this smells like a clever marketing ploy: The invasion of eCommerce onto our private desktops!