A real-time audio rendering system is introduced which combines a full room-specific simulation, dynamic crosstalk cancellation, and multitrack binaural synthesis for virtual acoustical imaging. The system is applicable for any room shape (normal, long, flat, coupled), independent of the a priori assumption of a diffuse sound field. This provides the possibility of simulating indoor or outdoor spatially distributed, freely movable sources and a moving listener in virtual environments. In addition to that, near-to-head sources can be simulated by using measured near-field HRTFs. The reproduction component consists of a headphone-free reproduction by dynamic crosstalk cancellation. The focus of the project is mainly on the integration and interaction of all involved subsystems. It is demonstrated that the system is capable of real-time room simulation and reproduction and, thus, can be used as a reliable platform for further research on VR applications.
Technology for 3-D sound reproduction, the so-called “audio front end” or the “acoustic human-machine interface” is an essential component of VR systems, which must be capable of fulfilling high quality standards involving psychoacoustically relevant cues. These cues may differ from one VR application to the next. Some applications require an exact localization.
• Sound in Virtual Reality
Human beings have two, not one, ears at about equal height on both sides of the head. This well-known fact is the basis of many of the outstanding features of human auditory perception. Identifying faint signals in a noisy environment, comprehending a specific talker in a group of people all speaking at the same time, enjoying the "acoustics" of a concert hall, and perceiving "stereo" with our hi-fi system at home, would hardly be possible with only one ear. In their effort to understand and to take advantage of the basic principles of human binaural hearing, engineers have done the groundwork for a new branch of technology - now known as Binaural Technology. Binaural Technology is able to offer a great number of applications capable of having noticeable impact on society. One of these applications is the representation of the auditory sensory domain in so-called Virtual-Reality systems. To this end, physiologically-adequate treatment of the prominent sensory modalities, including the auditory one, is mandatory.
Technically speaking, auditory representation in VR systems is implemented by means of a sound system. However, in contrast to conventional sound systems, the auditory representation is non-stationary and interactive, i.e., among other things, dependent on listeners' actions. This implies, for the auditory representation, that very complex, physiologically-adequate sound signals have to be delivered to the auditory systems of the listeners, namely to their eardrums.
One possible technical way to accomplish this is via transducers positioned at the entrances to the ear canals (headphones). Headphones are fixed to the head and thus move simultaneously with it. Consequently, head and body movements do not modify the coupling between transducers and ear canals (so-called head-related approach to auditory representation) - in contrast to the case where the transducers, e.g. loudspeakers, are positioned away from the head and where the head and body can move in proportion to the sound sources (room-related approach). In any real acoustical situation the transmission paths from the sources to the ear-drums will vary as a result of the listeners' movements in relation to the sound sources- the actual variation being dependent on the directional characteristics of both the sound sources and the external ears (skull, pinna, torso) and on the reflections and reverberation present.
Virtual-reality systems must take account of all these specific variations. Only if this task is performed with sufficient sophistication will the listeners accept their auditory percepts as real - and develop the required sense of presence and immersion.
