Multichannel speech intelligibility and talker recognition using monaural, binaural, and three-dimensional auditory presentation

In a 3D auditory display, sounds are presented over headphones in a way that they seem to originate from virtual sources in a space around the listener. This paper describes a study on the possible merits of such a display for bandlimited speech with respect to intelligibility and talker recognition against a background of competing voices. Different conditions were investigated: speech material (words/sentences), presentation mode (monaural/binaural/3D), number of competing talkers (1-4), and virtual position of the talkers (in 45 degrees-steps around the front horizontal plane). Average results for 12 listeners show an increase of speech intelligibility for 3D presentation for two or more competing talkers compared to conventional binaural presentation. The ability to recognize a talker is slightly better and the time required for recognition is significantly shorter for 3D presentation in the presence of two or three competing talkers. Although absolute localization of a talker is rather poor, spatial separation appears to have a significant effect on communication. For either speech intelligibility, talker recognition, or localization, no difference is found between the use of an individualized 3D auditory display and a general display.     

Combined monaural and binaural masking release

Stimulus conditions were examined where both across-frequency [comodulation masking release (CMR)] and across-ear [binaural masking-level difference (BMLD)] cues were available, as well as conditions where only one of these cue types was available. The main goal of the study was to determine how the two types of cues combine. The effects of comodulation were assessed either by modulating a masking noise and manipulating its bandwidth (experiment 1) or by using two comodulated narrow bands of noise separated in frequency (experiment 2). The masker was always No, and the 500-Hz pure-tone signal was either So or S pi. The effect of the frequency of modulation was examined either by changing the frequency of the modulating stimulus (experiment 1) or by changing the bandwidth of the comodulated narrow-band noise (experiment 2). Four of six subjects showed greater masking release when both BMLD and CMR cues were available than for either type of cue alone, whereas the other two subjects did not show an ability to combine the two cues for additional advantage. For the subjects who were able to combine the two types of cue, the additional advantage was greater for low frequencies of modulation. The results indicate that one component of CMR may be based upon across-frequency envelope comparisons at a stage of processing after binaural analysis.     

Speech perception from monaural and binaural information

Two experiments explored the concept of the binaural spectrogram [Culling and Colburn, J. Acoust. Soc. Am. 107, 517-527 (2000)] and its relationship to monaurally derived information. In each experiment, speech was added to noise at an adverse signal-to-noise ratio in the NoS pi binaural configuration. The resulting monaural and binaural cues were analyzed within an array of spectro-temporal bins and then these cues were resynthesized by modulating the intensity and/or interaural correlation of freshly generated noise. Experiment 1 measured the intelligibility of the resynthesized stimuli and compared them with the original NoSo and NoS pi stimuli at a fixed signal-to-noise ratio. While NoS pi stimuli were approximately equal to 50% intelligible, each cue in isolation produced similar (very low) intelligibility to the NoSo condition. The resynthesized combination produced approximately equal to 25% intelligibility. Modulation of interaural correlation below 1.2 kHz and of amplitude above 1.2 kHz was not as effective as their combination across all frequencies. Experiment 2 measured three-point psychometric functions in which the signal-to-noise ratio of the original NoS pi stimulus was increased in 3-dB steps from the level used in experiment 1. Modulation of interaural correlation alone proved to have a flat psychometric function. The functions for NoS pi and for combined monaural and binaural cues appeared similar in slope, but shifted horizontally. The results indicate that for sentence materials, neither fluctuations in interaural correlation nor in monaural intensity are sufficient to support speech recognition at signal-to-noise ratios where 50% intelligibility is achieved in the NoS pi configuration; listeners appear to synergistically combine monaural and binaural information in this task, to some extent within the same frequency region.     

Integration of monaural and binaural evidence of vowel formants

The intelligibility of speech is sustained at lower signal-to-noise ratios when the speech has a different interaural configuration from the noise. This paper argues that the advantage arises in part because listeners combine evidence of the spectrum of speech in the across-frequency profile of interaural decorrelation with evidence in the across-frequency profile of intensity. To support the argument, three experiments examined the ability of listeners to integrate and segregate evidence of vowel formants in these two profiles. In experiment 1, listeners achieved accurate identification of the members of a small set of vowels whose first formant was defined by a peak in one profile and whose second formant was defined by a peak in the other profile. This result demonstrates that integration is possible. Experiment 2 demonstrated that integration is not mandatory, insofar as listeners could report the identity of a vowel defined entirely in one profile despite the presence of a competing vowel in the other profile. The presence of the competing vowel reduced accuracy of identification, however, showing that segregation was incomplete. Experiment 3 demonstrated that segregation of the binaural vowel, in particular, can be increased by the introduction of an onset asynchrony between the competing vowels. The results of experiments 2 and 3 show that the intrinsic cues for segregation of the profiles are relatively weak. Overall, the results are compatible with the argument that listeners can integrate evidence of spectral peaks from the two profiles.     

Exploring the additivity of binaural and monaural masking release

Experiment 1 examined comodulation masking release (CMR) for a 700-Hz tonal signal under conditions of N(o)S(o) (noise and signal interaurally in phase) and N(o)S(π) (noise in phase, signal out of phase) stimulation. The baseline stimulus for CMR was either a single 24-Hz wide narrowband noise centered on the signal frequency [on-signal band (OSB)] or the OSB plus, a set of flanking noise bands having random envelopes. Masking noise was either gated or continuous. The CMR, defined with respect to either the OSB or the random noise baseline, was smaller for N(o)S(π) than N(o)S(o) stimulation, particularly when the masker was continuous. Experiment 2 examined whether the same pattern of results would be obtained for a 2000-Hz signal frequency; the number of flanking bands was also manipulated (two versus eight). Results again showed smaller CMR for N(o)S(π) than N(o)S(o) stimulation for both continuous and gated masking noise. The CMR was larger with eight than with two flanking bands, and this difference was greater for N(o)S(o) than N(o)S(π). The results of this study are compatible with serial mechanisms of binaural and monaural masking release, but they indicate that the combined masking release (binaural masking-level difference and CMR) falls short of being additive.     

The role of monaural frequency selectivity in binaural analysis

The relation between the monaural critical band and binaural analysis was examined using an NoSm MLD paradigm, in order to resolve ambiguities about the width of the masking spectrum important for binaural detection. A 500-Hz pure-tone signal was presented with a 600-Hz-wide band of masking noise to the signal ear. Bands of noise ranging in width from 25 to 600 Hz, or noise notches (imposed on a 600-Hz-wide band centered on the signal frequency) ranging in width from 0 to 600 Hz were presented to the nonsignal ear. All noise bands and notches were centered on 500 Hz, the frequency of the signal. The effects of varying bandwidth were radically different from those of varying notchwidth: the MLD changed from zero to approximately 8 dB over a bandwidth range of 400 Hz; for notchwidths, however, the MLD changed 8 dB over a range of only 50 Hz. The results support an interpretation that the fine frequency selectivity of monaural analysis is preserved in peripheral binaural interaction, but that a relatively wide frequency range of critical bands is scanned at a later stage of binaural processing. It was suggested that the wide spectral range of binaural analysis may provide a background against which binaural differences due to the signal are detected.     

Influence of monaural spectral cues on binaural localization

Seven subjects located, monaurally and binaurally, narrow bands of noise originating in the horizontal plane. The stimuli were 1.0 kHz wide and centered at 4.0-14.0 kHz in steps of 0.5 kHz. The loudspeakers, 15 deg apart, were arranged in a semicircle (0-270-180 deg, azimuth). In the first part of the experiment all sounds emanated from the loudspeaker at 270 deg, but their apparent locations varied widely as a function of their center frequency. For each subject, the pattern of location judgments under the binaural listening condition corresponded to that recorded for the monaural condition. In the second part of the experiment the loudspeaker from which each of the same narrow bands of noise emanated was varied in irregular order. Again, monaural location judgments were governed by the frequency content of the noise bands. Binaural location judgments were strongly influenced by the sounds' frequency composition when the stimuli originated from 315-225 deg, notwithstanding the presence of interaural differences in time and intensity. For narrow bands of noise emanating off midline, monaural spectral cues significantly override binaural difference cues, and they also determine the resolution of front-back ambiguities.     

The effect of random intensity fluctuation on monaural and binaural detection

Two experiments were performed to determine the effects of random intensity fluctuation on NoSo and NoS pi performance. Noise was used as both signal and masker, and stimuli were bands of noise from either 0-2.0 or 2.0-4.0kHz. Signal and masker were either coherent (from the same source) or noncoherent (from independent sources). In the first experiment, noise fluctuation was achieved by modulating a wide band of noise. In the second experiment, fluctuation was achieved by narrowing the noise bandwidth. Results from both experiments indicated that NoSo performance was adversely affected by fluctuation and by noncoherent relation between signal and masker. NoS pi detection was not adversely affected by fluctuation at low frequency, and was affected less adversely than was NoSo detection at high frequency. This difference between NoSo and NoS pi performance is an important consideration when making inferences about monaural and binaural processing when the stimuli are fluctuating rather than temporally steady.     

Tonal masking and frequency selectivity for the monaural and binaural hearing systems

A series of masking experiments was performed with the aim of comparing frequency selectivity for the monaural and binaural systems. The masking stimulus used in this study combined a sinusoid, which was gated simultaneously with the signal, with a continuous broadband noise. Signal frequency was fixed at 500 Hz. In one condition, the tonal masker and noise were interaurally in phase and the signal was phase reversed. In a second condition, noise, tonal masker, and signal were presented to one ear alone. Signal thresholds were obtained as a function of masker frequency for these two conditions. After making an appropriate selection of noise levels, masking functions for the monaural and binaural system conditions were found to agree closely except for a region about their tips where the binaural condition was more detectable. Two possible interpretations of these results are discussed. Either the monaural and binaural systems contain filters each which have similarly shaped skirts, or the frequency selectivity observed under both diotic and dichotic conditions (for large frequency separations of masker and signal) reflect the operation of a common peripheral filter.     

Late-onset auditory deprivation: effects of monaural versus binaural hearing aids

Performance on tests of pure-tone thresholds, speech-recognition thresholds, and speech-recognition scores for the two ears of each subject were evaluated in two groups of adults with bilateral hearing losses. One group was composed of individuals fitted with binaural hearing aids, and the other group included persons with monaural hearing aids. Performance prior to the use of hearing aids was compared to performance after 4-5 years of hearing aid use in order to determine whether the unaided ear would show effects of auditory deprivation. There were no differences over time for pure-tone thresholds or speech-recognition thresholds for both ears of both groups. Nevertheless, the results revealed that the speech-recognition difference scores of the binaurally fitted subjects remained stable over time whereas they increased for the monaurally fitted subjects. The findings reveal an auditory deprivation effect for the unfitted ears of the subjects with monaural hearing aids.     

Monaural and binaural auditory frequency resolution measured using bandlimited noise and notched-noise masking

Several studies using bandlimited masking noise have indicated that NOSO frequency resolution is better than that for NOS pi. The present study examined NOSO and NOS pi frequency resolution with two different masking methods: bandlimited noise and notched noise. Noise spectrum levels of 10, 30, and 50 dB/Hz were used. Thresholds were determined for a 500-Hz signal, using a three-alternative forced-choice adaptive procedure, as a function of masker bandwidth and notchwidth. For NOSO presentation, 3-dB down points were comparable for the notched-noise and bandlimiting methods. For NOS pi presentation, 3-dB down points were generally greater for the bandlimiting method than the notched noise method. Furthermore, for NOS pi presentation, the 3-dB down estimate increased as noise level increased for the bandlimiting method, but stayed constant for the notched-noise method. It is suggested that the two masking methods measured different aspects of binaural processing.     

Comparing the effects of reverberation and of noise on speech recognition in simulated electric-acoustic listening

Cochlear implant users report difficulty understanding speech in both noisy and reverberant environments. Electric-acoustic stimulation (EAS) is known to improve speech intelligibility in noise. However, little is known about the potential benefits of EAS in reverberation, or about how such benefits relate to those observed in noise. The present study used EAS simulations to examine these questions. Sentences were convolved with impulse responses from a model of a room whose estimated reverberation times were varied from 0 to 1 sec. These reverberated stimuli were then vocoded to simulate electric stimulation, or presented as a combination of vocoder plus low-pass filtered speech to simulate EAS. Monaural sentence recognition scores were measured in two conditions: reverberated speech and speech in a reverberated noise. The long-term spectrum and amplitude modulations of the noise were equated to the reverberant energy, allowing a comparison of the effects of the interferer (speech vs noise). Results indicate that, at least in simulation, (1) EAS provides significant benefit in reverberation; (2) the benefits of EAS in reverberation may be underestimated by those in a comparable noise; and (3) the EAS benefit in reverberation likely arises from partially preserved cues in this background accessible via the low-frequency acoustic component.     

Masked detection and discrimination of tone sequences under conditions of monaural and binaural masking release

Experiment 1 examined detection and discrimination of monaural four-tone sequences composed of 400-, 500-, and 625-Hz sinusoids. In the baseline conditions, the masker was monaural composed of 25-Hz-wide bands of random noise centered on 320, 400, 500, 625, and 781 Hz. In the binaural masking release conditions, the noise was presented diotically. In the monaural masking release conditions, the noise was presented to the same ear as the signal, but it was comodulated. Tones had half-amplitude durations of 30, 60, or 150 ms. There was no delay between successive tones, so the rate of frequency change depended on tone duration. Listeners discriminated between sequences composed of 500-400-625-500 Hz and 500-625-400-500 Hz. Discrimination results were poor for rapid sequences in both monaural and binaural masking release conditions relative to baseline conditions. Results from experiment 2 indicated that poor discrimination for rapid sequences could also occur in the baseline conditions, provided that the frequency separation among tonal components was small. Sluggish processing in the present paradigm was not restricted to conditions relying on binaural cues. It is argued that sluggishness may reflect a long temporal window in monaural and binaural masking release conditions or an interaction between poor cue quality and task difficulty.