Working with the deaf made me keenly aware of how we can use sound in an effective application interface by designing one where sound was not available. Audible feedback is a powerful tool and often overlooked outside of gaming. We humans are pretty good at localizing and tuning in to a sound source in space-- not as good as in the animal kingdom, but not bad. A study I'd be interested in seeing is one that can determine if an interface can be augmented with localized audio. Imagine a blank screen where a subject is asked to click on the screen where they think a sound is emanating. I'd imagine that with a proper audio setup and balanced hearing that users could do pretty good. I'd then be curious to see how users with unbalanced hearing, or less than optimal hardware setup (speakers placed differently) might affect such a test. The benefit of using audio draw a user's attention to parts of a display is fairly clear. For example, having an IM client moved to the upper left corner of the screen and obscured by another window while doing other work, an audible indicator of an incoming IM could help the user locate the application without resorting to popups or other annoyances. As the user moves the IM window to another part of the screen, the alert sounds it makes are also relocated. This research may have been done as part of the research around telepresence where the voice of speakers in a multiparty video conference session would appear be co-located with the video. However, I have not done a lit-search on that topic.

Depending on the context, it's critical. At CES this year I saw one of the worlds best gamers destroy all challengers. Yes, he was good, but when interviewed he said, flat out, that what made the difference for him was sound. He wore high end headphones and used a high end sound card. The information he received because of that differentiated him from the competition. It gave him a definite edge.

In a gaming or simulation environment, a character's voice or any sound that is relevant to the context ought to come from the side of the board where the character is located or the sound is being generated. One could even consider dividing the board into quadrants and having the upper quadrants have a lower volume of sound to indicate the character or event is further from the user. I would also consider having any alerts or play hints delivered as whispers in one ear or the other, at the user choice. Of course, we have to allow the user to determine if sound if monaural or binaural to accommodate any hearing loss they may have in one or both ears. We also have to provide the same information without sound, for those who either cannot hear or are using the application in an environment where sound is distruptive and headphones are not available.

Error messages could be modal and "attached" to the respective application that is running. So if a process breaks, instead of this big ugly window popping up in front of whatever you are doing and a loud annoying sound made, the application on the taskbar could turn red and perhaps overlay a "warning" icon and at the same time a binaurally directed sound effect could occur to direct the user's attention to the right location. The sound could even binaurally "appear" to be coming from behind the user, so that they know it is a background process that has an error (or even any other status message). Computer gaming interfaces using binaural sounds is obvious and exciting. In a first person shooter, you could be steathily moving through some trees, when, thanks to the binaural sound features of the game, you hear crunching leaves behind you indicating you are being followed. This technology can allow players to directly pinpoint the direction that noises, such as gunfire, are coming from so that they can move towards the action. As it stands, we can only guess from hearing if the sound came from the near/far left or near/far right.

Ok It would be helpfull if you can acces a pop up menu with some combination of keys. The menu can be translucid even invisible and you can navegate it without focussing acording to the sounds of the diferents tools. Sound could work to identify the tools or the sub-menus. So you can still pay atention to work you are doing while you call diferents tools. in 3d virtual worlds you can simulate objects near you. As if you were in a room without lights you know if a wall is at your left or right because it reflect the ambient sounds diferent. Or yoou can asignate a vibration for the diferents tools in the 3d world, and acces the tools just pointing at teh directon of the tool.

a practical example, IMHO: A compiz-fusion user could get great benefit from auditorily knowing which side of the rotational cube is attempting to get his/her attention. While some may scoff at the resource-hogging windows manager, I would argue that an (average) user will start to demand a more dynamic experience from their window manager. Maybe the next version of the paperclip assistant will be walking around this three dimensional space. http://burnurl.com/aMMqt4 Binaural interfacing could greatly improve the assistants ability to communicate with the user in this type of environment.

To add to my comment, one would link use of a devise that immersed someone in a mono or channel inverted environment with before & after scans to understand the implications on how the brain is processing sound. Some interesting compensations may arise when forced to function without normal binaural hearing for a period of time.

Create a headset that takes live real world input and convert it to mono, or swaps the left and right channels. Then one can live in mono, or in a L/R inverted world for an hour, 8 hours, a day, 3 days... however long they care to. Similar things have been done with vision studies. I would guess the brain would compensate for the change relatively quickly, but it may take some time to return back to normal sound recognition after extended immersion in a mono or inversed environment. Such a devise could also add processing to artificially expand stereo separation, modify EQ, reduce or expand dynamic range, or in some other way alter real world input.

Perception is closely tied to cognition. My book has a chapter on getting people to listen in social applications. I have designed audio-based systems that have to coordinate with real-time physical events (mobile or stationary). I know people with diminished hearing in one ear who compensate in complex ways. I've had better results focusing on the user process more than the physics of hearing, treating the latter as a variable that can be controlled to some extent by the user.

Ears are only useful when they are used to listen to the needs of others otherwise it matters not how many ears one has. Read more online at http://www.pypi.info