Monaural and interaural temporal modulation transfer functions measured with 5-kHz carriers

Temporal modulation transfer functions (TMTFs) were measured for detection of monaural sinusoidal amplitude modulation and dynamically varying interaural level differences for a single set of listeners. For the interaural TMTFs, thresholds are the modulation depths at which listeners can just discriminate interaural envelope-phase differences of 0 and 180 degrees. A 5-kHz pure tone and narrowband noises, 30- and 300-Hz wide centered at 5 kHz, were used as carriers. In the interaural conditions, the noise carriers were either diotic or interaurally uncorrelated. The interaural TMTFs with tonal and diotic noise carriers exhibited a low-pass characteristic but the cutoff frequencies changed nonmonotonically with increasing bandwidth. The interaural TMTFs for the tonal carrier began rolling off approximately a half-octave lower than the tonal monaural TMTF (approximately 80 Hz vs approximately 120 Hz). Monaural TMTFs obtained with noise carriers showed effects attributable to masking of the signal modulation by intrinsic fluctuations of the carrier. In the interaural task with dichotic noise carriers, similar masking due to the interaural carrier fluctuations was observed. Although the mechanisms responsible for differences between the monaural and interaural TMTFs are unknown, the lower binaural TMTF cutoff frequency suggests that binaural processing exhibits greater temporal limitation than monaural processing.     

An analysis of quasi-frequency-modulated noise and random-sideband noise as comparisons for amplitude-modulated noise

Experiments were performed to determine under what conditions quasi-frequency-modulated (QFM) noise and random-sideband noise are suitable comparisons for AM noise in measuring a temporal modulation transfer function (TMTF). Thresholds were measured for discrimination of QFM from random-sideband noise and AM from QFM noise as a function of sideband separation. In the first experiment, the upper spectral edge of the noise stimuli was at 2400 Hz and the bandwidth was 1600 Hz. For sideband separations up to 256 Hz, at threshold sideband levels for discriminating AM from QFM noise, QFM was indiscriminable from random-sideband noise. For the largest sideband separation used (512 Hz), listeners may have used within-stimulus envelope correlation in the QFM noise to discriminate it from the random-sideband noise. Results when stimulus bandwidth was varied suggest that listeners were able to use this cue when the carrier was wider than a critical band, and the sideband separation approached the carrier bandwidth. Within-stimulus envelope correlation was also present in AM noise, and thus QFM noise was a suitable comparison because it made this cue unusable and forced listeners to use across-stimulus envelope differences. When the carrier bandwidth was less than a critical band or was wideband, QFM noise and random-sideband noise were equally suitable comparisons for AM noise. When discrimination thresholds for QFM and random-sideband noise were converted to modulation depth and modulation frequency, they were nearly identical to those for discrimination of AM from QFM noise, suggesting that listeners were using amplitude modulation cues in both cases.     

Acoustic temporal modulation detection and speech perception in cochlear implant listeners

The goals of the present study were to measure acoustic temporal modulation transfer functions (TMTFs) in cochlear implant listeners and examine the relationship between modulation detection and speech recognition abilities. The effects of automatic gain control, presentation level and number of channels on modulation detection thresholds (MDTs) were examined using the listeners' clinical sound processor. The general form of the TMTF was low-pass, consistent with previous studies. The operation of automatic gain control had no effect on MDTs when the stimuli were presented at 65 dBA. MDTs were not dependent on the presentation levels (ranging from 50 to 75 dBA) nor on the number of channels. Significant correlations were found between MDTs and speech recognition scores. The rates of decay of the TMTFs were predictive of speech recognition abilities. Spectral-ripple discrimination was evaluated to examine the relationship between temporal and spectral envelope sensitivities. No correlations were found between the two measures, and 56% of the variance in speech recognition was predicted jointly by the two tasks. The present study suggests that temporal modulation detection measured with the sound processor can serve as a useful measure of the ability of clinical sound processing strategies to deliver clinically pertinent temporal information.     

Noise improves modulation detection by cochlear implant listeners at moderate carrier levels

Envelope detection and processing are very important for cochlear implant (CI) listeners, who must rely on obtaining significant amounts of acoustic information from the time-varying envelopes of stimuli. In previous work, Chatterjee and Robert [JARO 2(2), 159-171 (2001)] reported on a stochastic-resonance-type effect in modulation detection by CI listeners: optimum levels of noise in the envelope enhanced modulation detection under certain conditions, particularly when the carrier level was low. The results of that study suggested that a low carrier level was sufficient to evoke the observed stochastic resonance effect, but did not clarify whether a low carrier level was necessary to evoke the effect. Modulation thresholds in CI listeners generally decrease with increasing carrier level. The experiments in this study were designed to investigate whether the observed noise-induced enhancement is related to the low carrier level per se, or to the poor modulation sensitivity that accompanies it. This was done by keeping the carrier amplitude fixed at a moderate level and increasing modulation frequency so that modulation sensitivity could be reduced without lowering carrier level. The results suggest that modulation sensitivity, not carrier level, is the primary factor determining the effect of the noise.     

Lateralization of sinusoidally amplitude-modulated tones: effects of spectral locus and temporal variation

It has long been recognized that listeners are sensitive to interaural temporal disparities (ITDs) of low-frequency (i.e., below 1600 Hz) stimuli. Within the last three decades, it has often been demonstrated that listeners are also sensitive to ITDs within the envelope of high-frequency, complex stimuli. Because these studies, for the most part, employed discrimination tasks, few data exist concerning the extent of laterality produced by ITDs as a function of the spectral locus of the stimulus. To this end, we employed an acoustic "pointing" task in which listeners varied the interaural intensity difference of a 500-Hz narrow-band noise (the pointer) so that it matched the intracranial position of a second, experimenter-controlled stimulus (the target). Targets were sinusoidally amplitude-modulated tones centered on 500 Hz, 1, 2, 3, or 4 kHz and modulated at rates ranging from 50 to 800 Hz. Targets were presented with either the entire waveform delayed or with only the envelope delayed. Our results suggest that: (1) for low-frequency targets, lateralization is influenced by ITDs in the envelope but is dominated by ITDs in the fine structure; (2) for high-frequency targets, envelope-based delays produce displacements of the acoustic images which are affected greatly by the rate of modulation; rather large extents of laterality could be produced with high rates of modulation; these data are consistent with those obtained previously in discrimination experiments; (3) for low rates of modulation (e.g., 100 Hz), delays of the entire waveform (both envelope and fine structure) produce much greater displacements of the acoustic image for low-frequency than for high-frequency targets (where fine-structure-based cues are not utilizable); (4) there appear to be no consistent relations among extent of laterality, rate of modulation, and the frequency of the carrier within and across listeners.     

Discrimination of depth of sinusoidal amplitude modulation with and without roved carrier levels

Thresholds for the discrimination of the depth of sinusoidal amplitude modulation with a broadband noise carrier were measured for three listeners in a two-alternative, forced-choice task for modulation frequencies of 8, 32, and 128 Hz. Thresholds were measured with the spectrum level of the carrier fixed at 20 dB across all trials and, separately, with the carrier spectrum level roved randomly over a 20-dB range (10-30 dB) in each interval. Mean thresholds were equal or slightly lower (but not significantly so) for the fixed conditions relative to the roved conditions, and the differences between thresholds were too small to be explained by assuming that listeners compared instantaneous intensity at corresponding phases of the modulation cycle (for example, in the troughs). Rather, it appears that listeners discriminated modulation depth by extracting an estimate of the modulation depth within each interval that was independent of the overall level. Consequently, models of envelope extraction must include normalization of the envelope fluctuations to the envelope dc.     

Temporal integration in amplitude modulation detection

Thresholds for detecting sinusoidal amplitude modulation (AM) of a wideband noise carrier were measured as a function of the duration of the modulating signal. The carrier was either; (a) gated with a duration that exceeded the duration of modulation by the combined stimulus rise and fall times; (b) presented with a fixed duration that included a 500-ms carrier fringe preceding the onset of modulation; or (c) on continuously. In condition (a), the gated-carrier temporal modulation transfer functions (TMTFs) exhibited a bandpass characteristic. For AM frequencies above the individual subject's TMTF high-pass segment, the mean slope of the integration functions was - 7.46 dB per log unit duration. For the fringe and continuous-carrier conditions [(b) and (c)], the mean slopes of the integration functions were, respectively, - 9.30 and - 9.36 dB per log unit duration. Simulations based on integration of the output of an envelope detector approximate the results from the gated-carrier conditions. The more rapid rates of integration obtained in the fringe and continuous-carrier conditions may be due to "overintegration" where, at brief modulation durations, portions of the unmodulated carrier envelope are included in the integration of modulating signal energy.     

Discrimination of starting phase with sinusoidal envelope modulation

Modulation-filterbank models discard phase information above very low rates of amplitude modulation (AM). The present work evaluated this restriction by measuring thresholds for discriminating the starting phase of sinusoidal modulators of wideband-noise carriers. Results showed a low-pass characteristic with some listeners unable to perform the task once the modulation rate was greater than 12.5 Hz. For others, however, thresholds were obtained with AM rates of up to one to two octaves higher. Intersubject variability may in part relate to the presence of multiple discrimination cues, with only some based on comparison of the ongoing pattern of envelope fluctuation.     

Estimating the transition bandwidth between two auditory processes: evidence for broadband auditory filters

A spectral discrimination task was used to estimate the frequency range over which information about the temporal envelope is consolidated. The standard consisted of n equal intensity, random phase sinusoids, symmetrically placed around a signal component. The signal was an intensity increment of the central sinusoid, which on average was 1000 Hz. Pitch cues were degraded by randomly selecting the center frequency of the complex and single channel energy cues were degraded with a roving-level procedure. Stimulus bandwidth was controlled by varying the number of tones and the frequency separation between tones. For a fixed frequency separation, thresholds increased as n increased until a certain bandwidth was reached, beyond which thresholds decreased. This discontinuity in threshold functions suggests that different auditory processes predominate at different bandwidths, presumably an envelope analysis at bandwidths less than the breakpoint and across channel level comparisons for wider stimulus bandwidths. Estimates of the "transition bandwidth" for 46 listeners ranged from 100 to 1250 Hz. The results are consistent with a peripheral filtering system having multiple filterbanks.     

Factors affecting the use of noise-band vocoders as acoustic models for pitch perception in cochlear implants

Although in a number of experiments noise-band vocoders have been shown to provide acoustic models for speech perception in cochlear implants (CI), the present study assesses in four experiments whether and under what limitations noise-band vocoders can be used as an acoustic model for pitch perception in CI. The first two experiments examine the effect of spectral smearing on simulated electrode discrimination and fundamental frequency (FO) discrimination. The third experiment assesses the effect of spectral mismatch in an FO-discrimination task with two different vocoders. The fourth experiment investigates the effect of amplitude compression on modulation rate discrimination. For each experiment, the results obtained from normal-hearing subjects presented with vocoded stimuli are compared to results obtained directly from CI recipients. The results show that place pitch sensitivity drops with increased spectral smearing and that place pitch cues for multi-channel stimuli can adequately be mimicked when the discriminability of adjacent channels is adjusted by varying the spectral slopes to match that of CI subjects. The results also indicate that temporal pitch sensitivity is limited for noise-band carriers with low center frequencies and that the absence of a compression function in the vocoder might alter the saliency of the temporal pitch cues.     

Temporal processing deficits in the pre-senescent auditory system

This study tested the hypothesis that temporal processing deficits are evident in the pre-senescent (middle-aged) auditory system for listening tasks that involve brief stimuli, across-frequency-channel processing, and/or significant processing loads. A gap duration discrimination (GDD) task was employed that used either fixed-duration gap markers (experiment 1) or random-duration markers (experiment 2). Independent variables included standard gap duration (0, 35, and 250 ms), marker frequency (within- and across-frequency), and task complexity. A total of 18 young and 23 middle-aged listeners with normal hearing participated in the GDD experiments. Middle age was defined operationally as 40-55 years of age. The results indicated that middle-aged listeners performed more poorly than the young listeners in general, and that this deficit was sometimes, but not always, exacerbated by increases in task complexity. A third experiment employed a categorical perception task that measured the gap duration associated with a perceptual boundary. The results from 12 young and 12 middle-aged listeners with normal hearing indicated that the categorical boundary was associated with shorter gaps in the young listeners. The results of these experiments indicate that temporal processing deficits can be observed relatively early in the aging process, and are evident in middle age.     

The ability of cochlear implant users to use temporal envelope cues recovered from speech frequency modulation

Previous studies have demonstrated that normal-hearing listeners can understand speech using the recovered "temporal envelopes," i.e., amplitude modulation (AM) cues from frequency modulation (FM). This study evaluated this mechanism in cochlear implant (CI) users for consonant identification. Stimuli containing only FM cues were created using 1, 2, 4, and 8-band FM-vocoders to determine if consonant identification performance would improve as the recovered AM cues become more available. A consistent improvement was observed as the band number decreased from 8 to 1, supporting the hypothesis that (1) the CI sound processor generates recovered AM cues from broadband FM, and (2) CI users can use the recovered AM cues to recognize speech. The correlation between the intact and the recovered AM components at the output of the sound processor was also generally higher when the band number was low, supporting the consonant identification results. Moreover, CI subjects who were better at using recovered AM cues from broadband FM cues showed better identification performance with intact (unprocessed) speech stimuli. This suggests that speech perception performance variability in CI users may be partly caused by differences in their ability to use AM cues recovered from FM speech cues.     

Effects of envelope shape on interaural envelope delay sensitivity in acoustic and electric hearing

The envelope shape is important for the perception of interaural time difference (ITD) in the envelope as supported by the improved sensitivity for transposed tones compared to sinusoidally amplitude-modulated (SAM) tones. The present study investigated the effects of specific envelope parameters in nine normal-hearing (NH) and seven cochlear-implant (CI) listeners, using high-rate carriers with 27-Hz trapezoidal modulation. In NH listeners, increasing the off time (the silent interval in each modulation cycle) up to 12 ms, increasing the envelope slope from 6 to 8 dB/ms, and increasing the peak level improved ITD sensitivity. The combined effect of the off time and slope accounts for the gain in sensitivity for transposed tones relative to SAM tones. In CI listeners, increasing the off time up to 20 ms improved sensitivity, but increasing the slope showed no systematic effect. A 27-pulses/s electric pulse train, representing a special case of modulation with infinitely steep slopes and maximum possible off time, yielded considerably higher sensitivity compared to the best condition with trapezoidal modulation. Overall, the results of this study indicate that envelope-ITD sensitivity could be improved by using CI processing schemes that simultaneously increase the off time and the peak level of the signal envelope.     

Acoustic cues to tonal contrasts in Mandarin: implications for cochlear implants

The present study systematically manipulated three acoustic cues--fundamental frequency (f0), amplitude envelope, and duration--to investigate their contributions to tonal contrasts in Mandarin. Simplified stimuli with all possible combinations of these three cues were presented for identification to eight normal-hearing listeners, all native speakers of Mandarin from Taiwan. The f0 information was conveyed either by an f0-controlled sawtooth carrier or a modulated noise so as to compare the performance achievable by a clear indication of voice f0 and what is possible with purely temporal coding of f0. Tone recognition performance with explicit f0 was much better than that with any combination of other acoustic cues (consistently greater than 90% correct compared to 33%-65%; chance is 25%). In the absence of explicit f0, the temporal coding of f0 and amplitude envelope both contributed somewhat to tone recognition, while duration had only a marginal effect. Performance based on these secondary cues varied greatly across listeners. These results explain the relatively poor perception of tone in cochlear implant users, given that cochlear implants currently provide only weak cues to f0, so that users must rely upon the purely temporal (and secondary) features for the perception of tone.     

Coding of the fundamental frequency in continuous interleaved sampling processors for cochlear implants

In this study the perception of the fundamental frequency (F0) of periodic stimuli by cochlear implant users is investigated. A widely used speech processor is the Continuous Interleaved Sampling (CIS) processor, for which the fundamental frequency appears as temporal fluctuations in the envelopes at the output. Three experiments with four users of the LAURA (Registered trade mark of Philips Hearing Implants, now Cochlear Technology Centre Europe) cochlear implant were carried out to examine the influence of the modulation depth of these envelope fluctuations on pitch discrimination. In the first experiment, the subjects were asked to discriminate between two SAM (sinusoidally amplitude modulated) pulse trains on a single electrode channel differing in modulation frequency ( deltaf = 20%). As expected, the results showed a decrease in the performance for smaller modulation depths. Optimal performance was reached for modulation depths between 20% and 99%, depending on subject, electrode channel, and modulation frequency. In the second experiment, the smallest noticeable difference in F0 of synthetic vowels was measured for three algorithms that differed in the obtained modulation depth at the output: the default CIS strategy, the CIS strategy in which the F0 fluctuations in the envelope were removed (FLAT CIS), and a third CIS strategy, which was especially designed to control and increase the depth of these fluctuations (F0 CIS). In general, performance was poorest for the FLAT CIS strategy, where changes in F0 are only apparent as changes of the average amplitude in the channel outputs. This emphasizes the importance of temporal coding of F0 in the speech envelope for pitch perception. No significantly better results were obtained for the F0 CIS strategy compared to the default CIS strategy, although the latter results in envelope modulation depths at which sub-optimal scores were obtained in some cases of the first experiment. This indicates that less modulation is needed if all channels are stimulated with synchronous F0 fluctuations. This hypothesis is confirmed in a third experiment where subjects performed significantly better in a pitch discrimination task with SAM pulse trains, if three channels were stimulated concurrently, as opposed to only one.     

The role of the envelope in processing iterated rippled noise.

Iterated rippled noise (IRN) is generated by a cascade of delay and add (the gain after the delay is 1.0) or delay and subtract (the gain is -1.0) operations. The delay and add/subtract operations impart a spectral ripple and a temporal regularity to the noise. The waveform fine structure is different in these two conditions, but the envelope can be extremely similar. Four experiments were used to determine conditions in which the processing of IRN stimuli might be mediated by the waveform fine structure or by the envelope. In experiments 1 and 3 listeners discriminated among three stimuli in a single-interval task: IRN stimuli generated with the delay and add operations (g = 1.0), IRN stimuli generated using the delay and subtract operations (g = -1.0), and a flat-spectrum noise stimulus. In experiment 2 the listeners were presented two IRN stimuli that differed in delay (4 vs 6 ms) and a flat-spectrum noise stimulus that was not an IRN stimulus. In experiments 1 and 2 both the envelope and waveform fine structure contained the spectral ripple and temporal regularity. In experiment 3 only the envelope had this spectral and temporal structure. In all experiments discrimination was determined as a function of high-pass filtering the stimuli, and listeners could discriminate between the two IRN stimuli up to frequency regions as high as 4000-6000 Hz. Listeners could discriminate the IRN stimuli from the flat-spectrum noise stimulus at even higher frequencies (as high as 8000 Hz), but these discriminations did not appear to depend on the pitch of the IRN stimuli. A control experiment (fourth experiment) suggests that IRN discriminations in high-frequency regions are probably not due entirely to low-frequency nonlinear distortion products. The results of the paper imply that pitch processing of IRN stimuli is based on the waveform fine structure.     

Sensitivity of school-aged children to pitch-related cues

Two experiments investigated the ability of 17 school-aged children to process purely temporal and spectro-temporal cues that signal changes in pitch. Percentage correct was measured for the discrimination of sinusoidal amplitude modulation rate (AMR) of broadband noise in experiment 1 and for the discrimination of fundamental frequency (F0) of broadband sine-phase harmonic complexes in experiment 2. The reference AMR was 100 Hz as was the reference F0. A child-friendly interface helped listeners to remain attentive to the task. Data were fitted using a maximum-likelihood technique that extracted threshold, slope, and lapse rate. All thresholds were subsequently standardized to a common d' value equal to 0.77. There were relatively large individual differences across listeners: eight had relatively adult-like thresholds in both tasks and nine had higher thresholds. However, these individual differences did not vary systematically with age, over the span of 6-16 yr. Thresholds were correlated across the two tasks and were about nine times finer for F0 discrimination than for AMR discrimination as has been previously observed in adults.     

Bimodal listeners are not sensitive to interaural time differences in unmodulated low-frequency stimuli (L)

Sensitivity to interaural time differences (ITDs) with unmodulated low-frequency stimuli was assessed in bimodal listeners who had previously shown to be good performers in ITD experiments. Two types of stimuli were used: (1) an acoustic sinusoid combined with an electric transposed signal and (2) an acoustic sinusoid combined with an electric clicktrain. No or very low sensitivity to ITD was found for these stimuli, even though subjects were highly trained on the task and were intensively tested in multiple test sessions. In previous studies with users of a cochlear implant (CI) and a contralateral hearing aid (HA) (bimodal listeners), sensitivity was shown to ITD with modulated stimuli with frequency content between 600 and 3600 Hz. The outcomes of the current study imply that in speech processing design for users of a CI in combination with a HA on the contralateral side, the emphasis should be more on providing salient envelope ITD cues than on preserving fine-timing ITD cues present in acoustic signals.     

On the mechanisms involved in the recovery of envelope information from temporal fine structure

Three experiments were designed to provide psychophysical evidence for the existence of envelope information in the temporal fine structure (TFS) of stimuli that were originally amplitude modulated (AM). The original stimuli typically consisted of the sum of a sinusoidally AM tone and two unmodulated tones so that the envelope and TFS could be determined a priori. Experiment 1 showed that normal-hearing listeners not only perceive AM when presented with the Hilbert fine structure alone but AM detection thresholds are lower than those observed when presenting the original stimuli. Based on our analysis, envelope recovery resulted from the failure of the decomposition process to remove the spectral components related to the original envelope from the TFS and the introduction of spectral components related to the original envelope, suggesting that frequency- to amplitude-modulation conversion is not necessary to recover envelope information from TFS. Experiment 2 suggested that these spectral components interact in such a way that envelope fluctuations are minimized in the broadband TFS. Experiment 3 demonstrated that the modulation depth at the original carrier frequency is only slightly reduced compared to the depth of the original modulator. It also indicated that envelope recovery is not specific to the Hilbert decomposition.