Much To Do About Microsoft Surface

Microsoft's introduction of surface computing has caused a ruckus on the IxDa, which is the discussion board of the Interaction Design Association. There was a huge backlash on the product and I commented negatively as well. I was disappointed that already at the introduction of the product Microsoft had partnered with chain hotels and casinos to introduce the product to the public in the near future. It just shifted the focus from innovation toward selling the product. Many of the shown applications are lame like showing augmented information about the wine you are drinking right on the surface of the table...

On the other hand I really like the interaction with real objects. Using the table to share data between different devices. There is a lot of promise in that. I wonder if they could have solved the recognition problem with RFID tags instead of cameras. Cameras need distance, which makes the table bulky.

Ideas and innovation happens in synchronicity. All of a sudden a few people have the same ideas around the same time. Much of it has to do with technology moving to the next level opening up the doors for certain new ideas. 
This has happened for multi-touch. The iPhone made it popular and the time for multi touch computing is now. It's everywhere and we are going to see a lot more of it. Exciting times.

This is another example of multi-touch used for a collaboration instrument as shown at the Transmediale 2007 in Berlin



This is one of MS's cheesy ads for Surface:




http://www.microsoft.com/surface/
http://www.microsoft.com/presspass/presskits/surfacecomputing/default.mspx

Quest for the Eternal "Second, Third, Fourth..." Life

Right now my I am hooked on Battlestar Galactica. Last night I watched an episode from the 2nd season entitled "Downloaded". The show produces "Cylon"-androids-want-to-destroy-humankind stories. When a Cylon dies, he/she/its consciousness is being downloaded into a fresh clone-body.



Why do characters like "Data" on Star Trek's Next Generation or "Sharon Valerii" on Battlestar fascinate us so much? Their struggle to be accepted as human asks us to define what makes us human. Data's struggles to teach his inner circuitry to "feel". Does our capacity to feel distinguish us from machines like the Cylons? What if these machines have "feelings" themselves? What if they look exactly like you and I?





"Shanon Valerii" and "Data" - SNG



After all, they are either created deliberately by us (like Data) or the result of machines that have declared their independence from humankind. The examples included storylines from "Terminator", "The Matrix", "2001: A Space Odyssey" and lately "Battlestar Galactica". The more human they appear to be, the more the differences blur and the more we are forced to compare:

We are fragile. Once we are killed or injured there is no clone-body to replace limbs or entire bodies. We are supposed to be unique. It consoles us that there never was and never will be another person like us. At the same time it makes us sad, that along with our death all memories and experiences will vanish as well. What makes our life more important than that of a fruit fly? What is a human life span in comparison to the age of the universe? Being reborn, "downloaded" into a fresh body, having eternal youth ... just visions for coping with our insignificance and imminent death?



Which brings me to the subject of Second Life. People virtualize themselves in droves. Is the creation of virtual representation of yourself a God-like act? Our (many) disembodied selves are free from pain, age and disease. The desire the extend life as we know it is incredible. Clones and Androids and replacement organs grown form stem cells are just a matter of time. In the meantime we have Second Life. Whatever....

Tactile Interaction with Mobile Devices

More and more our digital devices and computers are being equipped with sensors. Physical computing is here at last! Apple's labtops have a motion sensor, that picks up fast movement. If a labtop is being dropped, it will shut down the spinning hard drive instantly to prevent damage to it. The upcoming iPhone is going to be equipped with proximity sensors as well as a accelerometer that detect when the device is being rotated from portrait to landscape.

Beyond the commercial applications there is exiting research being done at the university level. Steven Brewster, Faraz Chohan and Lorna Brown from the University of Glasgow introduced and tested tactile vibrational feedback on touchscreen devices. Typically people would like to use their mobile devices to get work done on subways and busses. Entering data is prone to errors because of shaking vehicles and environmental noise. Auditory feedbacks as tested by Brewster improve usability, but add no benefit in noisy conditions. In this system successful and erroneous typing (double taps or slips) each produce a specific feedback that suggests a "smooth" (success) vs. a "rough"(error) sensation. The study showed significant usability improvement. It increased the amount of data entered by the user and more successful error correction.
More information can be found here.

Technologyreview.com reports today (05-16-07) that the "British Telecom tries to wed Nintendo Wii-style technology to a tablet PC". Tilting and rotating the tablet PC would let the user navigate the computer "Etch-A-Scetch" style. While studying for my Masters (1999) Han Gene Paik and I developed an email application prototype that used tilt sensors to navigate in a similar fashion. Our concept departed from the WIMP interface and suggested the on screen environment to be filled with water. Sorting, filtering and selecting the email objects felt like digging for gold nuggets.


"Liquid Mail" by Han Gene Paik and Dirk J. Platzek

Today these accelerometer sensors are a lot more sophisticated and very tiny. They are known as microelectromechanical systems (MEMS) and are used for example for the iPod Sport Kit , a joined effort by Apple and Nike. Accelerometers are placed into special running shoes. A wireless connection transmits the information to the iPod.

Another exciting idea in the area of physical computing was just introduced at the Proceedings of the SIGCHI conference on Human factors in computing systems 2007, in San Jose, California: "Shoogle", - excitatory multimodal interaction on mobile devices has been developed by John Williamson, Rod Murray-Smith and Stephen Hughes. It also uses accelerometers to pick up user gestures in exchange for vibrotactile and audio feedback. The system produces feedback that - for example - feels like balls of different weights bouncing inside a box. From the sound and vibrotactile sensation a user is then able to distinguish length or urgency of a message. It is not necessary for the user to look at the device, which is great in situations where that is not appropriate. The idea takes advantage "of user's familiarity with the dynamics of processes in the physical world to present information in a natural and non-irritating manner". The highly recommended article can be found here.

Adobe Flash CS3 Unstable Crapware

I need to start a bitch session on Adobe Flash CS3: It is highly unstable. While I am happy to finally run a native application on my MBP, I am pissed that Flash is quitting at least 3 times a day in regular workflow! Has anybody else experienced this?

Apple's Patent for Back Side Multi Touch Interface

Just when I thought the iPhone's multi-touch-the-information was a breakthrough for a consumer product the next exciting interface appears at the horizon. Yesterday macdailynews.com reported that patents have been filed by Apple concerning their 6th G iPod. It looks like you can navigate the front face of the iPod with a touch panel located on the back side. Input and output will be on different sides of the iPod.

UPDATE 05-15-07: This post on Core77.com explains why a touch panel on the back is so smart: This way the user is no longer obscuring the very interface he/she is supposed to interact with...



The greatest innovation is often the timely implementation of a particular technology that often has been around for a while. This only happens after other technical hurdles, such as CPU speed, bandwidth or component size have been solved. The iPhone made me research multi-touch technology and I have written about it in previous posts. During my studies I discovered a touch table interface on youtube.com which utilizes the underside of the table for additional navigation as well.



Matthias Müller Prove recently reminded me of Alan Kay's famous quote: "The computer revolution hasn't happened yet". Physical computing finally makes it to the market place. Only the very best will catch our attention. Apple stock briefly climbed to an all time high yesterday again ....

Neurofeedback Gaming Interfaces

Lately I've noticed a renewed interest in neurofeedback interfaces. Yahoo News reported on "mind reading toys" and mentioned Silicon Valley gaming companies such as NeuroSky Inc and Emotiv Systems Inc.



Electroencephalography, or EEG, is the basis of these technologies. It measures the brain's electrical activity through electrodes placed on the scalp. Mostly these techiques are used in clinical contexts to help patients with migranes or kids who suffer from ADD. Beyond that there have communities obsessed with neurofeedback technologies since the 60s, claiming to achieve optimized mental performance.

Steven Johnson's book Mind Wide Open from 2005 describes his own encounter with neurofeedback devices. He says: "After the initial amazement had worn off while I was playing the space game at Seiden's office, I couldn't help noticing that I couldn't control the ship with nearly as much accuracy as I could have with a joystick or a keyboard. There's a fuzziness to interaction that would be unpleasant were I actually interested in having an efficient conversation with the computer."

and

"The almost unthinkable complexity of the brain's information network is necessarily compressed down to a crude language when a machine listens to the collective rhythms of brain waves through the skull."

NeuroSky is about to unveil a headset with brainwave "dry" sensors that won't require a conducting gel to pick up brain activity.



While looking futuristic and usable I wonder how much brain activity they will be able to pick up, when scientific instruments with a lot more sensors produce "crude" results.

Emotiv Systems Inc. "Project Epoc" headset resembles more the know EEG instruments and interface wirelessly with all gaming consoles.



The company's wiki states: "Emotiv uses proprietary, patented algorithms to formulate detections of activities from the reading of EEG signals. These detections can determine a user’s expressions, feelings and thoughts." Apparently the technology picks ups the player's muscle twitches and thus facial expressions. The player's character would smile in synch with the player or "react" to the player with a range of feelings like surprise or anger. In other words interaction with an artificial intelligence.

Last but not least I found a "brainwave dance performance" on youtube. There is an introduction on how the sensors attached to the dancer work followed by pretty bad dancing:)

The Emotional iPhone Experience

Mac owners always had a more emotional relationship to their machines. That it is a great accomplishment on the part of Apple's designers. I like looking at my MacBook Pro every day! Touching it, using it is a delight. To prove my gushiness isn't rare to find amoung Mac users I'd like you to take a look at this video by Jason Ponton, Editor of The MIT Technology Review. He explains what makes a beautiful machine and his enthusiasm is a great example for the kind of emotional bond I am talking about.



Cell phones are intimate machines. We carry them close to our bodies and use them primarily to communicate, they are connectors to the world around us. I have never built that emotional relationship with any of my cell phones ( I am on my 5th phone now) . The novelty of a new phone wears off after a couple of weeks. It becomes a mere tool I expect to work. Since its features a miserably integrated I don't bother to use them. I am still doing the same few things with my phone I have always done: calling, texting and entering data into the calendar and the address book. My personal feature wish list has not changed for years. I don't need more features just a few done the right way. Nothing but sad compromises so far....

I think the iPhone will create that emotional relationship.

Why?

The integration with my Mac will be perfect. The iPhone will behave like my computer. It will be its ultra portable extension.

It will feel great in my hand, the multi-touch interface will make it feel like a real extension of my body. I will be able to directly touch and manipulate the information!

The overall success of the iPhone may depend on how well it will integrate with the Windows machines. The exclusive provider deal with AT+T is also annoying. No matter how great the iPhone is, I will not change my current phone plan for it.

I guess...