Effects of cochlear implant processing and fundamental frequency on the intelligibility of competing sentences

Speech perception in the presence of another competing voice is one of the most challenging tasks for cochlear implant users. Several studies have shown that (1) the fundamental frequency (F0) is a useful cue for segregating competing speech sounds and (2) the F0 is better represented by the temporal fine structure than by the temporal envelope. However, current cochlear implant speech processing algorithms emphasize temporal envelope information and discard the temporal fine structure. In this study, speech recognition was measured as a function of the F0 separation of the target and competing sentence in normal-hearing and cochlear implant listeners. For the normal-hearing listeners, the combined sentences were processed through either a standard implant simulation or a new algorithm which additionally extracts a slowed-down version of the temporal fine structure (called Frequency-Amplitude-Modulation-Encoding). The results showed no benefit of increasing F0 separation for the cochlear implant or simulation groups. In contrast, the new algorithm resulted in gradual improvements with increasing F0 separation, similar to that found with unprocessed sentences. These results emphasize the importance of temporal fine structure for speech perception and demonstrate a potential remedy for difficulty in the perceptual segregation of competing speech sounds.     

Place-pitch discrimination of single- versus dual-electrode stimuli by cochlear implant users (L)

Simultaneous or near-simultaneous activation of adjacent cochlear implant electrodes can produce pitch percepts intermediate to those produced by each electrode separately, thereby increasing the number of place-pitch steps available to cochlear implant listeners. To estimate how many distinct pitches could be generated with simultaneous dual-electrode stimulation, the present study measured place-pitch discrimination thresholds for single- versus dual-electrode stimuli in users of the Clarion CII device. Discrimination thresholds were expressed as the proportion of current directed to the secondary electrode of the dual-electrode pair. For 16 of 17 electrode pairs tested in six subjects, thresholds ranged from 0.11 to 0.64, suggesting that dual-electrode stimuli can produce 2-9 discriminable pitches between the pitches of single electrodes. Some subjects demonstrated a level effect, with better place-pitch discrimination at higher stimulus levels. Equal loudness was achieved with dual-electrode stimuli at net current levels that were similar to or slightly higher than those for single-electrode stimuli.     

Effect of stimulation rate on phoneme recognition by nucleus-22 cochlear implant listeners

This study investigated the effect of pulsatile stimulation rate on medial vowel and consonant recognition in cochlear implant listeners. Experiment 1 measured phoneme recognition as a function of stimulation rate in six Nucleus-22 cochlear implant listeners using an experimental four-channel continuous interleaved sampler (CIS) speech processing strategy. Results showed that all stimulation rates from 150 to 500 pulses/s/electrode produced equally good performance, while stimulation rates lower than 150 pulses/s/electrode produced significantly poorer performance. Experiment 2 measured phoneme recognition by implant listeners and normal-hearing listeners as a function of the low-pass cutoff frequency for envelope information. Results from both acoustic and electric hearing showed no significant difference in performance for all cutoff frequencies higher than 20 Hz. Both vowel and consonant scores dropped significantly when the cutoff frequency was reduced from 20 Hz to 2 Hz. The results of these two experiments suggest that temporal envelope information can be conveyed by relatively low stimulation rates. The pattern of results for both electrical and acoustic hearing is consistent with a simple model of temporal integration with an equivalent rectangular duration (ERD) of the temporal integrator of about 7 ms.     

Evaluating the function of phonetic perceptual phenomena within speech recognition: an examination of the perception of /d/-/t/ by adult cochlear implant users

This study examined whether cochlear implant users must perceive differences along phonetic continua in the same way as do normal hearing listeners (i.e., sharp identification functions, poor within-category sensitivity, high between-category sensitivity) in order to recognize speech accurately. Adult postlingually deafened cochlear implant users, who were heterogeneous in terms of their implants and processing strategies, were tested on two phonetic perception tasks using a synthetic /da/-/ta/ continuum (phoneme identification and discrimination) and two speech recognition tasks using natural recordings from ten talkers (open-set word recognition and forced-choice /d/-/t/ recognition). Cochlear implant users tended to have identification boundaries and sensitivity peaks at voice onset times (VOT) that were longer than found for normal-hearing individuals. Sensitivity peak locations were significantly correlated with individual differences in cochlear implant performance; individuals who had a /d/-/t/ sensitivity peak near normal-hearing peak locations were most accurate at recognizing natural recordings of words and syllables. However, speech recognition was not strongly related to identification boundary locations or to overall levels of discrimination performance. The results suggest that perceptual sensitivity affects speech recognition accuracy, but that many cochlear implant users are able to accurately recognize speech without having typical normal-hearing patterns of phonetic perception.     

Sensitivity to isolated and concurrent intensity and fundamental frequency increments by cochlear implant users under natural listening conditions

Sensitivity to acoustic cues in cochlear implant (CI) listening under natural conditions is a potentially complex interaction between a number of simultaneous factors, and may be difficult to predict. In the present study, sensitivity was measured under conditions that approximate those of natural listening. Synthesized words having increases in intensity or fundamental frequency (F0) in a middle stressed syllable were presented in soundfield to normal-hearing listeners and to CI listeners using their everyday speech processors and programming. In contrast to the extremely fine sensitivity to electrical current observed when direct stimulation of single electrodes is employed, difference limens (DLs) for intensity were larger for the CI listeners by a factor of 2.4. In accord with previous work, F0 DLs were larger by almost one order of magnitude. In a second experiment, it was found that the presence of concurrent intensity and F0 increments reduced the mean DL to half that of either cue alone for both groups of subjects, indicating that both groups combine concurrent cues with equal success. Although sensitivity to either cue in isolation was not related to word recognition in CI users, the listeners having lower combined-cue thresholds produced better word recognition scores.     

Auditory-visual speech perception in normal-hearing and cochlear-implant listeners

The present study evaluated auditory-visual speech perception in cochlear-implant users as well as normal-hearing and simulated-implant controls to delineate relative contributions of sensory experience and cues. Auditory-only, visual-only, or auditory-visual speech perception was examined in the context of categorical perception, in which an animated face mouthing ba, da, or ga was paired with synthesized phonemes from an 11-token auditory continuum. A three-alternative, forced-choice method was used to yield percent identification scores. Normal-hearing listeners showed sharp phoneme boundaries and strong reliance on the auditory cue, whereas actual and simulated implant listeners showed much weaker categorical perception but stronger dependence on the visual cue. The implant users were able to integrate both congruent and incongruent acoustic and optical cues to derive relatively weak but significant auditory-visual integration. This auditory-visual integration was correlated with the duration of the implant experience but not the duration of deafness. Compared with the actual implant performance, acoustic simulations of the cochlear implant could predict the auditory-only performance but not the auditory-visual integration. These results suggest that both altered sensory experience and improvised acoustic cues contribute to the auditory-visual speech perception in cochlear-implant users.     

Absolute identification of electric pulse rates and electrode positions by cochlear implant patients

In a single interval task, multichannel cochlear implant patients were asked to identify the members of a set of seven electric stimuli differing in electric pulse rate or electrode position. The perceptual sensitivity index (d') between successive stimuli in a stimulus set was calculated from the confusions among the seven stimuli. The results showed that the pulse rate above which the identification task became difficult varied from 200 to 600 pps from patient to patient. For the identification of the positions of seven bipolar electrode pairs, d' measures for stimulus sets differing in spatial separation or spatial extent were compared. Spatial separation is defined as the fixed distance between the two basal (or apical) electrodes of two successive bipolar electrode pairs in a stimulus set, while spatial extent is defined as the fixed distance between the apical and basal electrodes of each bipolar electrode pair in a stimulus set. The results showed that perceptual performance improved in an orderly way with spatial separation, but was not significantly affected by spatial extent.     

Within- and across-channel gap detection in cochlear implant listeners

This study examined within- and across-electrode-channel processing of temporal gaps in successful users of MED-EL COMBI 40+ cochlear implants. The first experiment tested across-ear gap duration discrimination (GDD) in four listeners with bilateral implants. The results demonstrated that across-ear GDD thresholds are elevated relative to monaural, within-electrode-channel thresholds; the size of the threshold shift was approximately the same as for monaural, across-electrode-channel configurations. Experiment 1 also demonstrated a decline in GDD performance for channel-asymmetric markers. The second experiment tested the effect of envelope fluctuation on gap detection (GD) for monaural markers carried on a single electrode channel. Results from five cochlear implant listeners indicated that envelopes associated with 50-Hz wide bands of noise resulted in poorer GD thresholds than envelopes associated with 300-Hz wide bands of noise. In both cases GD thresholds improved when envelope fluctuations were compressed by an exponent of 0.2. The results of both experiments parallel those found for acoustic hearing, therefore suggesting that temporal processing of gaps is largely limited by factors central to the cochlea.     

Characterizing auditory processing and perception in individual listeners with sensorineural hearing loss

This study considered consequences of sensorineural hearing loss in ten listeners. The characterization of individual hearing loss was based on psychoacoustic data addressing audiometric pure-tone sensitivity, cochlear compression, frequency selectivity, temporal resolution, and intensity discrimination. In the experiments it was found that listeners with comparable audiograms can show very different results in the supra-threshold measures. In an attempt to account for the observed individual data, a model of auditory signal processing and perception [Jepsen et al., J. Acoust. Soc. Am. 124, 422-438 (2008)] was used as a framework. The parameters of the cochlear processing stage of the model were adjusted to account for behaviorally estimated individual basilar-membrane input-output functions and the audiogram, from which the amounts of inner hair-cell and outer hair-cell losses were estimated as a function of frequency. All other model parameters were left unchanged. The predictions showed a reasonably good agreement with the measured individual data in the frequency selectivity and forward masking conditions while the variation of intensity discrimination thresholds across listeners was underestimated by the model. The model and the associated parameters for individual hearing-impaired listeners might be useful for investigating effects of individual hearing impairment in more complex conditions, such as speech intelligibility in noise.     

基于全相位滤波器的电子耳蜗汉语音调感知及改进研究

针对电子耳蜗音调信息感知差的问题,在多导电子耳蜗机理模型的基础上,利用零(或全)相位滤波器以及希尔伯特变换,提取并分解16个通道信号的包络和精细结构,用多种方式嵌合成声音,对正常人耳测听合成音的音调信息,以确定信号的包络和精细结构对音调感知的影响。测试结果表明:精细结构相对包络对音调感知起着更重要的作用,且该作用主要表现在低频段(约1.2 kHz以内)通道上。研究发现,在固定通道上,精细结构与通道中心频率和相位信息决定的包络出现时刻对应,其音调感知与低通道电极刺激的脉冲发放时间有关。研究结论:在低频段电极上增加刺激脉冲发放时刻的控制对提高电子耳蜗音调感知是重要的,同时应注意滤波器的相位保真。      

Detection and rate discrimination of amplitude modulation in electrical hearing

Three experiments were designed to examine temporal envelope processing by cochlear implant (CI) listeners. In experiment 1, the hypothesis that listeners' modulation sensitivity would in part determine their ability to discriminate between temporal modulation rates was examined. Temporal modulation transfer functions (TMTFs) obtained in an amplitude modulation detection (AMD) task were compared to threshold functions obtained in an amplitude modulation rate discrimination (AMRD) task. Statistically significant nonlinear correlations were observed between the two measures. In experiment 2, results of loudness-balancing showed small increases in the loudness of modulated over unmodulated stimuli beyond a modulation depth of 16%. Results of experiment 3 indicated small but statistically significant effects of level-roving on the overall gain of the TMTF, but no impact of level-roving on the average shape of the TMTF across subjects. This suggested that level-roving simply increased the task difficulty for most listeners, but did not indicate increased use of intensity cues under more challenging conditions. Data obtained with one subject, however, suggested that the most sensitive listeners may derive some benefit from intensity cues in these tasks. Overall, results indicated that intensity cues did not play an important role in temporal envelope processing by the average CI listener.     

The use of acoustic cues for phonetic identification: effects of spectral degradation and electric hearing

Although some cochlear implant (CI) listeners can show good word recognition accuracy, it is not clear how they perceive and use the various acoustic cues that contribute to phonetic perceptions. In this study, the use of acoustic cues was assessed for normal-hearing (NH) listeners in optimal and spectrally degraded conditions, and also for CI listeners. Two experiments tested the tense/lax vowel contrast (varying in formant structure, vowel-inherent spectral change, and vowel duration) and the word-final fricative voicing contrast (varying in F1 transition, vowel duration, consonant duration, and consonant voicing). Identification results were modeled using mixed-effects logistic regression. These experiments suggested that under spectrally-degraded conditions, NH listeners decrease their use of formant cues and increase their use of durational cues. Compared to NH listeners, CI listeners showed decreased use of spectral cues like formant structure and formant change and consonant voicing, and showed greater use of durational cues (especially for the fricative contrast). The results suggest that although NH and CI listeners may show similar accuracy on basic tests of word, phoneme or feature recognition, they may be using different perceptual strategies in the process.     

Channel weights for speech recognition in cochlear implant users

The purpose of this study was to develop and validate a method of estimating the relative "weight" that a multichannel cochlear implant user places on individual channels, indicating its contribution to overall speech recognition. The correlational method as applied to speech recognition was used both with normal-hearing listeners and with cochlear implant users fitted with six-channel speech processors. Speech was divided into frequency bands corresponding to the bands of the processor and a randomly chosen level of corresponding filtered noise was added to each channel on each trial. Channels in which the signal-to-noise ratio was more highly correlated with performance have higher weights, and conversely, channels in which the correlations were smaller have lower weights. Normal-hearing listeners showed approximately equal weights across frequency bands. In contrast, cochlear implant users showed unequal weighting across bands, and varied from individual to individual with some channels apparently not contributing significantly to speech recognition. To validate these channel weights, individual channels were removed and speech recognition in quiet was tested. A strong correlation was found between the relative weight of the channel removed and the decrease in speech recognition, thus providing support for use of the correlational method for cochlear implant users.     

Pulse-rate discrimination by cochlear-implant and normal-hearing listeners with and without binaural cues

Experiment 1 measured rate discrimination of electric pulse trains by bilateral cochlear implant (CI) users, for standard rates of 100, 200, and 300 pps. In the diotic condition the pulses were presented simultaneously to the two ears. Consistent with previous results with unilateral stimulation, performance deteriorated at higher standard rates. In the signal interval of each trial in the dichotic condition, the standard rate was presented to the left ear and the (higher) signal rate was presented to the right ear; the non-signal intervals were the same as in the diotic condition. Performance in the dichotic condition was better for some listeners than in the diotic condition for standard rates of 100 and 200 pps, but not at 300 pps. It is concluded that the deterioration in rate discrimination observed for CI users at high rates cannot be alleviated by the introduction of a binaural cue, and is unlikely to be limited solely by central pitch processes. Experiment 2 performed an analogous experiment in which 300-pps acoustic pulse trains were bandpass filtered (3900-5400 Hz) and presented in a noise background to normal-hearing listeners. Unlike the results of experiment 1, performance was superior in the dichotic than in the diotic condition.     

Temporal pitch perception at high rates in cochlear implants

A recent study reported that a group of Med-El COMBI 40+CI (cochlear implant) users could, in a forced-choice task, detect changes in the rate of a pulse train for rates higher than the 300 pps "upper limit" commonly reported in the literature [Kong, Y.-Y., et al. (2009). J. Acoust. Soc. Am. 125, 1649-1657]. The present study further investigated the upper limit of temporal pitch in the same group of CI users on three tasks [pitch ranking, rate discrimination, and multidimensional scaling (MDS)]. The patterns of results were consistent across the three tasks and all subjects could follow rate changes above 300 pps. Two subjects showed exceptional ability to follow temporal pitch change up to about 900 pps. Results from the MDS study indicated that, for the two listeners tested, changes in pulse rate over the range of 500-840 pps were perceived along a perceptual dimension that was orthogonal to the place of excitation. Some subjects showed a temporal pitch reversal at rates beyond their upper limit of pitch and some showed a reversal within a small range of rates below the upper limit. These results are discussed in relation to the possible neural bases for temporal pitch processing at high rates.     

Categorization and discrimination of nonspeech sounds: differences between steady-state and rapidly-changing acoustic cues

Different patterns of performance across vowels and consonants in tests of categorization and discrimination indicate that vowels tend to be perceived more continuously, or less categorically, than consonants. The present experiments examined whether analogous differences in perception would arise in nonspeech sounds that share critical transient acoustic cues of consonants and steady-state spectral cues of simplified synthetic vowels. Listeners were trained to categorize novel nonspeech sounds varying along a continuum defined by a steady-state cue, a rapidly-changing cue, or both cues. Listeners' categorization of stimuli varying on the rapidly changing cue showed a sharp category boundary and posttraining discrimination was well predicted from the assumption of categorical perception. Listeners more accurately discriminated but less accurately categorized steady-state nonspeech stimuli. When listeners categorized stimuli defined by both rapidly-changing and steady-state cues, discrimination performance was accurate and the categorization function exhibited a sharp boundary. These data are similar to those found in experiments with dynamic vowels, which are defined by both steady-state and rapidly-changing acoustic cues. A general account for the speech and nonspeech patterns is proposed based on the supposition that the perceptual trace of rapidly-changing sounds decays faster than the trace of steady-state sounds.     

An analysis of quasi-frequency-modulated noise and random-sideband noise as comparisons for amplitude-modulated noise

Experiments were performed to determine under what conditions quasi-frequency-modulated (QFM) noise and random-sideband noise are suitable comparisons for AM noise in measuring a temporal modulation transfer function (TMTF). Thresholds were measured for discrimination of QFM from random-sideband noise and AM from QFM noise as a function of sideband separation. In the first experiment, the upper spectral edge of the noise stimuli was at 2400 Hz and the bandwidth was 1600 Hz. For sideband separations up to 256 Hz, at threshold sideband levels for discriminating AM from QFM noise, QFM was indiscriminable from random-sideband noise. For the largest sideband separation used (512 Hz), listeners may have used within-stimulus envelope correlation in the QFM noise to discriminate it from the random-sideband noise. Results when stimulus bandwidth was varied suggest that listeners were able to use this cue when the carrier was wider than a critical band, and the sideband separation approached the carrier bandwidth. Within-stimulus envelope correlation was also present in AM noise, and thus QFM noise was a suitable comparison because it made this cue unusable and forced listeners to use across-stimulus envelope differences. When the carrier bandwidth was less than a critical band or was wideband, QFM noise and random-sideband noise were equally suitable comparisons for AM noise. When discrimination thresholds for QFM and random-sideband noise were converted to modulation depth and modulation frequency, they were nearly identical to those for discrimination of AM from QFM noise, suggesting that listeners were using amplitude modulation cues in both cases.     

Effects of interaural time differences in fine structure and envelope on lateral discrimination in electric hearing

Bilateral cochlear implant (CI) listeners currently use stimulation strategies which encode interaural time differences (ITD) in the temporal envelope but which do not transmit ITD in the fine structure, due to the constant phase in the electric pulse train. To determine the utility of encoding ITD in the fine structure, ITD-based lateralization was investigated with four CI listeners and four normal hearing (NH) subjects listening to a simulation of electric stimulation. Lateralization discrimination was tested at different pulse rates for various combinations of independently controlled fine structure ITD and envelope ITD. Results for electric hearing show that the fine structure ITD had the strongest impact on lateralization at lower pulse rates, with significant effects for pulse rates up to 800 pulses per second. At higher pulse rates, lateralization discrimination depended solely on the envelope ITD. The data suggest that bilateral CI listeners benefit from transmitting fine structure ITD at lower pulse rates. However, there were strong interindividual differences: the better performing CI listeners performed comparably to the NH listeners.     

Speech processing studies using an acoustic model of a multiple-channel cochlear implant

The speech perception of two multiple-channel cochlear implant patients was compared with that of three normally hearing listeners using an acoustic model of the implant for 22 different speech tests. The tests used included a minimal auditory capabilities battery, both closed-set and open-set word and sentence tests, speech tracking and a 12-consonant confusion study using nonsense syllables. The acoustic model represented electrical current pulses by bursts of noise and the effects of different electrodes were represented by using bandpass filters with different center frequencies. All subjects used a speech processor that coded the fundamental voicing frequency of speech as a pulse rate and the second formant frequency of speech as the electrode position in the cochlea, or the center frequency of the bandpass filter. Very good agreement was found for the two groups of subjects, indicating that the acoustic model is a useful tool for the development and evaluation of alternative cochlear implant speech processing strategies.