Speech perception from monaural and binaural information

Two experiments explored the concept of the binaural spectrogram [Culling and Colburn, J. Acoust. Soc. Am. 107, 517-527 (2000)] and its relationship to monaurally derived information. In each experiment, speech was added to noise at an adverse signal-to-noise ratio in the NoS pi binaural configuration. The resulting monaural and binaural cues were analyzed within an array of spectro-temporal bins and then these cues were resynthesized by modulating the intensity and/or interaural correlation of freshly generated noise. Experiment 1 measured the intelligibility of the resynthesized stimuli and compared them with the original NoSo and NoS pi stimuli at a fixed signal-to-noise ratio. While NoS pi stimuli were approximately equal to 50% intelligible, each cue in isolation produced similar (very low) intelligibility to the NoSo condition. The resynthesized combination produced approximately equal to 25% intelligibility. Modulation of interaural correlation below 1.2 kHz and of amplitude above 1.2 kHz was not as effective as their combination across all frequencies. Experiment 2 measured three-point psychometric functions in which the signal-to-noise ratio of the original NoS pi stimulus was increased in 3-dB steps from the level used in experiment 1. Modulation of interaural correlation alone proved to have a flat psychometric function. The functions for NoS pi and for combined monaural and binaural cues appeared similar in slope, but shifted horizontally. The results indicate that for sentence materials, neither fluctuations in interaural correlation nor in monaural intensity are sufficient to support speech recognition at signal-to-noise ratios where 50% intelligibility is achieved in the NoS pi configuration; listeners appear to synergistically combine monaural and binaural information in this task, to some extent within the same frequency region.     

NoSo and NoS pi detection as a function of masker bandwidth in normal-hearing and cochlear-impaired listeners

NoSo and NoS pi detection thresholds for a 500-Hz pure-tone signal were measured as a function of masking noise bandwidth in normal-hearing and cochlear hearing-impaired subjects. NoSo and NoS pi critical bands were derived from the bandlimited noise functions. A notched noise measure of the monaural critical band was also obtained for each ear. One hypothesis tested was that an asymmetrical monaural critical band would result in a relatively steep improvement of the NoS pi detection threshold as a function of decreasing masker bandwidth and would, therefore, be associated with a wider binaural critical band. This was hypothesized because the outputs of the left and right auditory filters would be more decorrelated the greater the interaural difference in the monaural critical band. However, as the noise bandwidth was narrowed, the decorrelation would lessen, resulting in a relatively steep improvement in NoS pi detection. Results indicated that the masking level difference (MLD) was smaller and that the monaural critical bands were generally wider in cochlear-impaired listeners. NoSo and NoS pi critical bands were somewhat larger in the cochlear hearing-impaired listeners having relatively wide monaural critical bands. There was a significant correlation between monaural critical band asymmetry and the NoS pi critical band; however, this correlation was insignificant when a control was employed for the critical band in the worse ear. Therefore, the present results did not support a strong association between monaural critical band asymmetry and the width of the NoS pi critical band.     

Frequency dependence of binaural performance in listeners with impaired binaural hearing.

Binaural performance was measured as a function of stimulus frequency for four impaired listeners, each with bilaterally symmetric audiograms. The subjects had various degrees and configurations of audiometric losses: two had high-frequency, sensorineural losses; one had a flat sensorineural loss; and one had multiple sclerosis with normal audiometric thresholds. Just noticeable differences (jnd's) in interaural time, interaural intensity, and interaural correlation as well as detection thresholds for NoSo and NoS pi conditions were obtained for narrow-band noise stimuli at octave frequencies from 250-4000 Hz. Performance of the impaired listeners was generally poorer than that of normal-hearing listeners, although it was comparable to normal in a few instances. The patterns of binaural performance showed no apparent relation to the audiometric patterns; even the two subjects with similar degree and configuration of hearing loss have very different binaural performance, both in the level and frequency dependence of their performance. The frequency dependence of performance on individual tests is irregular enough that one cannot confidently interpolate between octaves. In addition, it appears that no subset of the measurements is adequate to characterize the performance in the rest of the measurements with the exception that, within limits, interaural correlation discrimination and NoS pi detection performance are related.     

Masking effects in binaural detection and interaural time discrimination

This study was designed to investigate the effects of masker level and frequency on binaural detection and interaural time discrimination. Detection and interaural time discrimination of a 700-Hz sinusoidal signal were measured as a function of the center frequency and level of a narrow-band masking noise. The masker was a continuous, diotic, 80-Hz-wide noise that varied in center frequency from 250 to 1370 Hz. In the detection experiment, the signal was presented either diotically (NoSo) or interaurally phase reversed (NoS pi). In the interaural time discrimination experiment, the signal level needed to discriminate a 30-microseconds interaural delay was measured. As would be expected, the presence of the masker has a greater effect on NoSo detection than NoS pi detection, and for masker frequencies at or near the signal frequency. In contrast, interaural time discrimination can be improved by the presence of a low-level masker. Also, performance improves more rapidly as the signal/masker frequency separation increases for NoSo detection than for interaural time discrimination and NoS pi detection. For all three tasks, significant upward spread of masking occurs only at the highest masker level; at low masker levels, there is a tendency toward downward spread of masking.     

Interaural correlation sensitivity

Sensitivity to differences in interaural correlation was measured for 1.3-ERB-wide bands of noise using a 2IFC task at six frequencies: 250, 500, 750, 1000, 1250, and 1500 Hz. The sensitivity index, d', was measured for discriminations between a number of fixed pairs of correlation values. Cumulative d' functions were derived for each frequency and condition. The d' for discriminating any two values of correlation may be recovered from the cumulative d' function by the difference between cumulative d's for these values. Two conditions were employed: the noisebands were either presented in isolation (narrow-band condition) or in the context of broad, contiguous flanking bands of correlated noise (fringed condition). The cumulative d' functions showed greater sensitivity to differences in correlation close to 1 than close to 0 at low frequencies, but this difference was less pronounced in the fringed condition. Also, a more linear relationship was observed when cumulative d' was plotted as a function of the equivalent signal-to-noise ratio (SNR) in dB for each correlation value, rather than directly against correlation. The equivalent SNR was the SNR at which the interaural correlation in an NoS(pi) stimulus would equal the interaural correlation of the noise used in the experiment. The maximum cumulative d' declined above 750 Hz. This decline was steeper for the fringed than for the narrow-band condition. For the narrow-band condition, the total cumulative d' was variable across listeners. All cumulative d' functions were closely fitted using a simple two-parameter function. The complete data sets, averaged across listeners, from the fringed and narrow-band conditions were fitted using functions to describe the changes in these parameters over frequency, in order to produce an interpolated family of curves that describe sensitivity at frequencies between those tested. These curves predict the spectra recovered by the binaural system when complex sounds, such as speech, are masked by noise.     

Measurement of the binaural auditory filter using a detection task

The spectral resolution of the binaural system was measured using a tone-detection task in a binaural analog of the notched-noise technique. Three listeners performed 2-interval, 2-alternative, forced choice tasks with a 500-ms out-of-phase signal within 500 ms of broadband masking noise consisting of an "outer" band of either interaurally uncorrelated or anticorrelated noise, and an "inner" band of interaurally correlated noise. Three signal frequencies were tested (250, 500, and 750 Hz), and the asymmetry of the filter was measured by keeping the signal at a constant frequency and moving the correlated noise band relative to the signal. Thresholds were taken for bandwidths of correlated noise ranging from 0 to 400 Hz. The equivalent rectangular bandwidth of the binaural filter was found to increase with signal frequency, and estimates tended to be larger than monaural bandwidths measured for the same listeners using equivalent techniques.     

Binaural masking experiments using noise maskers with frequency-dependent interaural phase differences. II: Influence of frequency and interaural-phase uncertainty

This study investigates whether binaural signal detection is improved by the listener's previous knowledge about the interaural phase relations of masker and test signal. Binaural masked thresholds were measured for a 500-ms dichotic noise masker that had an interaural phase difference of 0 below 500 Hz and of pi above 500 Hz. The thresholds for two difference 20-ms test signals were determined within the same measurement using an interleaved adaptive 3-interval forced-choice (3IFC) procedure. In each 3IFC trial, both signals could occur with equal probability (uncertainty). The two signals differed in frequency and interaural phase in such a way that one signal always had a frequency above the masker edge frequency (500 Hz) and no interaural phase difference (So), whereas the other signal frequency was below 500 Hz and the interaural phase difference was pi (S pi). The frequencies of a signal pair remained fixed during the whole 3IFC track. These two signals thus lead to two different binaural conditions, i.e., NoS pi for the low-frequency signal and N pi So for the high-frequency signal. For comparison, binaural masked thresholds were measured with the same masker for fixed signal frequency and phase. The binaural masking level differences (BMLDs) resulting from the two experimental conditions show no significant difference. This indicates that the binaural system is able to apply different internal transformations or processing strategies simultaneously in different critical bands and even within the same critical band.     

Lateralization of bands of noise and sinusoidally amplitude-modulated tones: effects of spectral locus and bandwidth

Lateralization of narrow bands of noise was investigated while varying interaural temporal disparity (ITD), center frequency, and bandwidth, utilizing an acoustic pointing task. Stimuli were narrow bands of noise centered at octave intervals between 500 Hz and 4 kHz with bandwidths ranging from 50-400 Hz. In a second experiment, lateralization for bands of noise and sinusoidally amplitude-modulated (SAM) tones, whose spectral content was constrained to be no lower than 3.8 kHz, was assessed. Overall, relatively large extents of laterality were obtained from all four listeners for ITDs of low-frequency bands of noise. Increasing the bandwidth of these noises did not yield consistent changes in the extent of laterality across ITDs and listeners. Most targets centered at high frequencies were lateralized near the midline. However, three of the four listeners did exhibit rather large displacements of the intracranial image when the bandwidth of the high-frequency noises was 400 Hz or greater. Interestingly, ITDs within high-frequency SAM tones were relatively ineffective. Thus, it appears that ITDs of relatively wide-band, high-frequency stimuli can mediate rather substantial extents of laterality. However, these effects are highly listener-dependent.     

Spectral interference in a binaural detection task

Several investigations suggest that sensitivity to changes in interaural disparities within select spectral regions may be degraded by the presence of energy at other, even remote, spectral regions. This study assessed whether similar degradations would be observed in an MLD paradigm. Detection thresholds were measured for NoSo and NoS pi. The signal, an 800-Hz tone (100-ms), was presented in continuous, broadband noise. Thresholds were also measured in the presence of a 400-Hz tone (the interferer) presented with an interaural phase disparity of 180 degrees and gated simultaneously with the signal or presented continuously. NoS pi thresholds increased by about 7 dB with the gated interferer at 80 dB SPL. Smaller increases were observed with lower levels of the interferer. Presenting the interferer continuously reduced substantially its effect. NoSo thresholds were affected only slightly by the interferer. Reversing the roles of the signal and interferer (400-Hz signal, 800-Hz interferer) led to smaller, but reliable degradations in performance. Diotic interferers had, in general, smaller effects on performance. The possible relation between the mechanisms that produce interference and those that foster an ability to segregate sources of sound is discussed.     

Lateralization of sinusoidally amplitude-modulated tones: effects of spectral locus and temporal variation

It has long been recognized that listeners are sensitive to interaural temporal disparities (ITDs) of low-frequency (i.e., below 1600 Hz) stimuli. Within the last three decades, it has often been demonstrated that listeners are also sensitive to ITDs within the envelope of high-frequency, complex stimuli. Because these studies, for the most part, employed discrimination tasks, few data exist concerning the extent of laterality produced by ITDs as a function of the spectral locus of the stimulus. To this end, we employed an acoustic "pointing" task in which listeners varied the interaural intensity difference of a 500-Hz narrow-band noise (the pointer) so that it matched the intracranial position of a second, experimenter-controlled stimulus (the target). Targets were sinusoidally amplitude-modulated tones centered on 500 Hz, 1, 2, 3, or 4 kHz and modulated at rates ranging from 50 to 800 Hz. Targets were presented with either the entire waveform delayed or with only the envelope delayed. Our results suggest that: (1) for low-frequency targets, lateralization is influenced by ITDs in the envelope but is dominated by ITDs in the fine structure; (2) for high-frequency targets, envelope-based delays produce displacements of the acoustic images which are affected greatly by the rate of modulation; rather large extents of laterality could be produced with high rates of modulation; these data are consistent with those obtained previously in discrimination experiments; (3) for low rates of modulation (e.g., 100 Hz), delays of the entire waveform (both envelope and fine structure) produce much greater displacements of the acoustic image for low-frequency than for high-frequency targets (where fine-structure-based cues are not utilizable); (4) there appear to be no consistent relations among extent of laterality, rate of modulation, and the frequency of the carrier within and across listeners.     

Detection of tones in reproducible narrow-band noise

Hit and false-alarm rates were measured for detection of a 500-Hz tone target in each of ten reproducible samples of 1/3-oct bandwidth noise centered at 500 Hz for both NoS pi and NoSo conditions. The effects on hit rates of the starting phase of the target relative to individual noise samples were investigated with two target phase angles for three subjects. The major results are: (1) performance varies significantly over masker waveforms; (2) for NoS pi conditions, the effect of target-to-marker phase angle on hit rates is not significant for these narrow-band maskers; (3) for NoSo conditions, the target-to-masker phase angle has a large effect; (4) no significant correlation between NoSo performance and NoS pi performance is seen across masker waveforms. These results are generally consistent wuth previously reported results for wideband maskers [R.H. Gilkey, D.E. Robinson, and T.E. Hanna, "Effects of masker waveform and signal-to-masker phase relation on diotic and dichotic masking by reproducible noise," J. Acoust. Soc. Am. 78, 1207-1219 (1985)] with an important exception. Specifically, in the wideband experiment, significant correlation between NoSo and NoS pi performance across noise samples was found. In addition, in the wideband experiment, a small yet statistically significant effect of target-to-masker phase was observed in the NoS pi condition.     

An analysis of quasi-frequency-modulated noise and random-sideband noise as comparisons for amplitude-modulated noise

Experiments were performed to determine under what conditions quasi-frequency-modulated (QFM) noise and random-sideband noise are suitable comparisons for AM noise in measuring a temporal modulation transfer function (TMTF). Thresholds were measured for discrimination of QFM from random-sideband noise and AM from QFM noise as a function of sideband separation. In the first experiment, the upper spectral edge of the noise stimuli was at 2400 Hz and the bandwidth was 1600 Hz. For sideband separations up to 256 Hz, at threshold sideband levels for discriminating AM from QFM noise, QFM was indiscriminable from random-sideband noise. For the largest sideband separation used (512 Hz), listeners may have used within-stimulus envelope correlation in the QFM noise to discriminate it from the random-sideband noise. Results when stimulus bandwidth was varied suggest that listeners were able to use this cue when the carrier was wider than a critical band, and the sideband separation approached the carrier bandwidth. Within-stimulus envelope correlation was also present in AM noise, and thus QFM noise was a suitable comparison because it made this cue unusable and forced listeners to use across-stimulus envelope differences. When the carrier bandwidth was less than a critical band or was wideband, QFM noise and random-sideband noise were equally suitable comparisons for AM noise. When discrimination thresholds for QFM and random-sideband noise were converted to modulation depth and modulation frequency, they were nearly identical to those for discrimination of AM from QFM noise, suggesting that listeners were using amplitude modulation cues in both cases.     

Frequency resolution for diotic and dichotic listening conditions compared using the bandlimiting measure and a modified bandlimiting measure

Two experiments were performed that examined the relation between frequency selectivity for diotic and dichotic stimuli. Subjects were eight normal-hearing listeners. In each experiment, a 500-Hz pure tone of 400-ms duration was presented in continuous noise. In the diotic listening conditions, a signal and noise were presented binaurally with no interaural differences (So and No, respectively). In the dichotic listening conditions, the signal or noise at one ear was 180 degrees out-of-phase relative to the respective stimulus at the other ear (S pi and N pi, respectively). The first experiment examined frequency selectivity using the bandlimiting measure. Here, signal thresholds were determined as a function of masker bandwidth (50, 100, 250, 500, and 1000 Hz) for SoNo, S pi No, and SoN pi listening conditions. The second experiment used a modified bandlimiting measure. Here, signal thresholds (So and S pi) were determined with a relatively narrow No band of masker energy (50 Hz wide) centered about the signal. Then, a second No narrow-band masker (30 Hz wide) was added at another frequency region, and signal thresholds were reestablished. The results of the two experiments indicated that listeners process a wider band of frequencies when resolving dichotic stimuli than when resolving diotic or monotic stimuli. The results also indicated that the bandlimiting measure may underestimate the spectral band processed upon dichotic stimulation. Results are interpreted in terms of an across-ear and across-frequency processing of waveform amplitude envelope.     

Performance in several binaural-interaction experiments

The relationship between interaural correlation discrimination and binaural detection was investigated using common experimental procedures and common subjects. Psychometric functions were obtained for four normal-hearing subjects at 500 and 4000 Hz using third-octave noise signals for the correlation discrimination experiment, and pure-tone signals and third-octave noise maskers for the detection experiment. Results from these two measurements, which were compared by expressing the signal-to-noise ratio as an equivalent change in interaural correlation, support the idea that interaural correlation discrimination and binaural detection are closely related. Since large intersubject differences in binaural performance were observed in these experiments, interaural-time, interaural-intensity, and monaural-intensity discrimination were measured in a second experiment. The results of the second experiment show large intersubject differences for the interaural tasks, but not for the monaural task.     

Observer weighting strategies in interaural time-difference discrimination and monaural level discrimination for a multi-tone complex

Two experiments measured listeners' abilities to weight information from different components in a complex of 553, 753, and 953 Hz. The goal was to determine whether or not the ability to adjust perceptual weights generalized across tasks. Weights were measured by binary logistic regression between stimulus values that were sampled from Gaussian distributions and listeners' responses. The first task was interaural time discrimination in which listeners judged the laterality of the target component. The second task was monaural level discrimination in which listeners indicated whether the level of the target component decreased or increased across two intervals. For both experiments, each of the three components served as the target. Ten listeners participated in both experiments. The results showed that those individuals who adjusted perceptual weights in the interaural time experiment could also do so in the monaural level discrimination task. The fact that the same individuals appeared to be analytic in both tasks is an indication that the weights measure the ability to attend to a particular region of the spectrum while ignoring other spectral regions.     

Binaural processing of noisy stimuli: internal/external noise ratios for diotic and dichotic stimuli

A set of ten digitized statistically similar Gaussian maskers was used in one-internal tone-in-noise detection experiments under diotic (NoSo) and dichotic (NoS pi) interaural conditions. Stimulus/response matrices were generated for each masker in the presence or absence of a target 500-Hz tone. For both NoSo and NoS pi, nonparametric analyses show that response probabilities and sensitivities vary significantly across noise waveforms, indicating a considerable external noise component in subject response variability. A parametric model is developed that maps individual stimulus waveforms onto a decision axis, facilitating evaluation of internal/external noise variance ratios. For both NoSo and NoS pi, internal and external noise variance are of similar magnitude.     

Lateralization of low-frequency, complex waveforms: the use of envelope-based temporal disparities

Several recent investigations suggest that listeners either cannot or do not use envelope-based interaural temporal disparities (ITDs) to lateralize low-frequency sounds [G.B. Henning, J. Acoust. Soc. Am. 68, 446-453 (1980); G.B. Henning and J. Ashton, Hear. Res. 4, 185-194 (1981); G.B. Henning, Hear. Res. 9, 153-172 (1983)]. We believe listeners in those studies may have been unable to process envelope-based ITDs principally because of the types of stimuli utilized. In this study we employed an acoustic "pointing" task in which listeners varied the interaural intensitive difference of a 500-Hz narrow-band noise (the pointer) so that it matched the intracranial position of a second, experimenter-controlled stimulus (the target). Targets were sinusoidally amplitude-modulated tones centered on 500 Hz or 1 kHz, and modulated at 25, 50, or 100 Hz. Targets were presented with either the entire waveform delayed or with only the envelope delayed. The results suggest that delays of the envelope do affect the lateral position of low-frequency targets. However, the envelope-based cues appear to interact with those provided by the dominant fine structure.     

Estimating the transition bandwidth between two auditory processes: evidence for broadband auditory filters

A spectral discrimination task was used to estimate the frequency range over which information about the temporal envelope is consolidated. The standard consisted of n equal intensity, random phase sinusoids, symmetrically placed around a signal component. The signal was an intensity increment of the central sinusoid, which on average was 1000 Hz. Pitch cues were degraded by randomly selecting the center frequency of the complex and single channel energy cues were degraded with a roving-level procedure. Stimulus bandwidth was controlled by varying the number of tones and the frequency separation between tones. For a fixed frequency separation, thresholds increased as n increased until a certain bandwidth was reached, beyond which thresholds decreased. This discontinuity in threshold functions suggests that different auditory processes predominate at different bandwidths, presumably an envelope analysis at bandwidths less than the breakpoint and across channel level comparisons for wider stimulus bandwidths. Estimates of the "transition bandwidth" for 46 listeners ranged from 100 to 1250 Hz. The results are consistent with a peripheral filtering system having multiple filterbanks.     

Discrimination of depth of sinusoidal amplitude modulation with and without roved carrier levels

Thresholds for the discrimination of the depth of sinusoidal amplitude modulation with a broadband noise carrier were measured for three listeners in a two-alternative, forced-choice task for modulation frequencies of 8, 32, and 128 Hz. Thresholds were measured with the spectrum level of the carrier fixed at 20 dB across all trials and, separately, with the carrier spectrum level roved randomly over a 20-dB range (10-30 dB) in each interval. Mean thresholds were equal or slightly lower (but not significantly so) for the fixed conditions relative to the roved conditions, and the differences between thresholds were too small to be explained by assuming that listeners compared instantaneous intensity at corresponding phases of the modulation cycle (for example, in the troughs). Rather, it appears that listeners discriminated modulation depth by extracting an estimate of the modulation depth within each interval that was independent of the overall level. Consequently, models of envelope extraction must include normalization of the envelope fluctuations to the envelope dc.     

A binaural analog of gap detection.

The temporal resolution of the binaural auditory system was measured using a binaural analog of gap detection. A binaural "gap" was defined as a burst of interaurally uncorrelated noise (Nu) placed between two bursts of interaurally correlated noise (N0). The Nu burst creates a dip in the output of a binaural temporal window integrating interaural correlation, analogous to the dip created by a silent gap in the output of a monaural temporal window integrating intensity. The equivalent rectangular duration (ERD) of the binaural window was used as an index of binaural temporal resolution. In order to derive the ERD, both the shortest-detectable binaural gap and the jnd for a reduction in interaural correlation from unity were measured. In experiment 1, binaural-gap thresholds were measured using narrow-band noise carriers as a function of center frequency from 250 to 2000 Hz (fixed 100-Hz bandwidth) and a function of lower-cutoff frequency from 100 to 400 Hz (fixed 500-Hz upper-cutoff frequency). Binaural-gap thresholds (1) increased significantly with increasing frequency in both tasks, and (2) at frequencies below 500 Hz, were shorter than corresponding silent-gap thresholds measured with the same N0 noises. In experiment 2, interaural-correlation jnd's were measured for the same conditions. The jnd's also increased significantly with increasing frequency. The results were analyzed using a temporal window integrating the output of a computational model of binaural processing. The ERD of the window varied widely across listeners, with a mean value of 140 ms, and did not significantly depend on frequency. This duration is about an order of magnitude longer than the ERD of the monaural temporal window and is, therefore, consistent with "binaural sluggishness."