Relationship between perception of spectral ripple and speech recognition in cochlear implant and vocoder listeners

Spectral resolution has been reported to be closely related to vowel and consonant recognition in cochlear implant (CI) listeners. One measure of spectral resolution is spectral modulation threshold (SMT), which is defined as the smallest detectable spectral contrast in the spectral ripple stimulus. SMT may be determined by the activation pattern associated with electrical stimulation. In the present study, broad activation patterns were simulated using a multi-band vocoder to determine if similar impairments in speech understanding scores could be produced in normal-hearing listeners. Tokens were first decomposed into 15 logarithmically spaced bands and then re-synthesized by multiplying the envelope of each band by matched filtered noise. Various amounts of current spread were simulated by adjusting the drop-off of the noise spectrum away from the peak (40-5 dBoctave). The average SMT (0.25 and 0.5 cyclesoctave) increased from 6.3 to 22.5 dB, while average vowel identification scores dropped from 86% to 19% and consonant identification scores dropped from 93% to 59%. In each condition, the impairments in speech understanding were generally similar to those found in CI listeners with similar SMTs, suggesting that variability in spread of neural activation largely accounts for the variability in speech perception of CI listeners.     

Noise improves modulation detection by cochlear implant listeners at moderate carrier levels

Envelope detection and processing are very important for cochlear implant (CI) listeners, who must rely on obtaining significant amounts of acoustic information from the time-varying envelopes of stimuli. In previous work, Chatterjee and Robert [JARO 2(2), 159-171 (2001)] reported on a stochastic-resonance-type effect in modulation detection by CI listeners: optimum levels of noise in the envelope enhanced modulation detection under certain conditions, particularly when the carrier level was low. The results of that study suggested that a low carrier level was sufficient to evoke the observed stochastic resonance effect, but did not clarify whether a low carrier level was necessary to evoke the effect. Modulation thresholds in CI listeners generally decrease with increasing carrier level. The experiments in this study were designed to investigate whether the observed noise-induced enhancement is related to the low carrier level per se, or to the poor modulation sensitivity that accompanies it. This was done by keeping the carrier amplitude fixed at a moderate level and increasing modulation frequency so that modulation sensitivity could be reduced without lowering carrier level. The results suggest that modulation sensitivity, not carrier level, is the primary factor determining the effect of the noise.     

Within- and across-channel gap detection in cochlear implant listeners

This study examined within- and across-electrode-channel processing of temporal gaps in successful users of MED-EL COMBI 40+ cochlear implants. The first experiment tested across-ear gap duration discrimination (GDD) in four listeners with bilateral implants. The results demonstrated that across-ear GDD thresholds are elevated relative to monaural, within-electrode-channel thresholds; the size of the threshold shift was approximately the same as for monaural, across-electrode-channel configurations. Experiment 1 also demonstrated a decline in GDD performance for channel-asymmetric markers. The second experiment tested the effect of envelope fluctuation on gap detection (GD) for monaural markers carried on a single electrode channel. Results from five cochlear implant listeners indicated that envelopes associated with 50-Hz wide bands of noise resulted in poorer GD thresholds than envelopes associated with 300-Hz wide bands of noise. In both cases GD thresholds improved when envelope fluctuations were compressed by an exponent of 0.2. The results of both experiments parallel those found for acoustic hearing, therefore suggesting that temporal processing of gaps is largely limited by factors central to the cochlea.     

Understanding speech in modulated interference: cochlear implant users and normal-hearing listeners

Many competing noises in real environments are modulated or fluctuating in level. Listeners with normal hearing are able to take advantage of temporal gaps in fluctuating maskers. Listeners with sensorineural hearing loss show less benefit from modulated maskers. Cochlear implant users may be more adversely affected by modulated maskers because of their limited spectral resolution and by their reliance on envelope-based signal-processing strategies of implant processors. The current study evaluated cochlear implant users' ability to understand sentences in the presence of modulated speech-shaped noise. Normal-hearing listeners served as a comparison group. Listeners repeated IEEE sentences in quiet, steady noise, and modulated noise maskers. Maskers were presented at varying signal-to-noise ratios (SNRs) at six modulation rates varying from 1 to 32 Hz. Results suggested that normal-hearing listeners obtain significant release from masking from modulated maskers, especially at 8-Hz masker modulation frequency. In contrast, cochlear implant users experience very little release from masking from modulated maskers. The data suggest, in fact, that they may show negative effects of modulated maskers at syllabic modulation rates (2-4 Hz). Similar patterns of results were obtained from implant listeners using three different devices with different speech-processor strategies. The lack of release from masking occurs in implant listeners independent of their device characteristics, and may be attributable to the nature of implant processing strategies and/or the lack of spectral detail in processed stimuli.     

Across-site patterns of modulation detection in listeners with cochlear implants

In modern cochlear implants, much of the information required for recognition of important sounds is conveyed by temporal modulation of the charge per phase in interleaved trains of electrical pulses. In this study, modulation detection thresholds (MDTs) were used to assess listeners' abilities to detect sinusoidal modulation of charge per phase at each available stimulation site in their 22-electrode implants. Fourteen subjects were tested. MDTs were found to be highly variable across stimulation sites in most listeners. The across-site patterns of MDTs differed considerably from subject to subject. The subject-specific patterns of across-site variability of MDTs suggest that peripheral site-specific characteristics, such as electrode placement and the number and condition of surviving neurons, play a primary role in determining modulation sensitivity. Across-site patterns of detection thresholds (T levels), maximum comfortable loudness levels (C levels) and dynamic ranges (DRs) were not consistently correlated with across-site patterns of MDTs within subjects, indicating that the mechanisms underlying across-site variation in these measures differed from those underlying across-site variation in MDTs. MDTs sampled from multiple sites in a listener's electrode array might be useful for diagnosing across-subject differences in speech recognition with cochlear implants and for guiding strategies to improve the individual's perception.     

The ability of cochlear implant users to use temporal envelope cues recovered from speech frequency modulation

Previous studies have demonstrated that normal-hearing listeners can understand speech using the recovered "temporal envelopes," i.e., amplitude modulation (AM) cues from frequency modulation (FM). This study evaluated this mechanism in cochlear implant (CI) users for consonant identification. Stimuli containing only FM cues were created using 1, 2, 4, and 8-band FM-vocoders to determine if consonant identification performance would improve as the recovered AM cues become more available. A consistent improvement was observed as the band number decreased from 8 to 1, supporting the hypothesis that (1) the CI sound processor generates recovered AM cues from broadband FM, and (2) CI users can use the recovered AM cues to recognize speech. The correlation between the intact and the recovered AM components at the output of the sound processor was also generally higher when the band number was low, supporting the consonant identification results. Moreover, CI subjects who were better at using recovered AM cues from broadband FM cues showed better identification performance with intact (unprocessed) speech stimuli. This suggests that speech perception performance variability in CI users may be partly caused by differences in their ability to use AM cues recovered from FM speech cues.     

Recognition of spectrally asynchronous speech by normal-hearing listeners and Nucleus-22 cochlear implant users

This experiment examined the effects of spectral resolution and fine spectral structure on recognition of spectrally asynchronous sentences by normal-hearing and cochlear implant listeners. Sentence recognition was measured in six normal-hearing subjects listening to either full-spectrum or noise-band processors and five Nucleus-22 cochlear implant listeners fitted with 4-channel continuous interleaved sampling (CIS) processors. For the full-spectrum processor, the speech signals were divided into either 4 or 16 channels. For the noise-band processor, after band-pass filtering into 4 or 16 channels, the envelope of each channel was extracted and used to modulate noise of the same bandwidth as the analysis band, thus eliminating the fine spectral structure available in the full-spectrum processor. For the 4-channel CIS processor, the amplitude envelopes extracted from four bands were transformed to electric currents by a power function and the resulting electric currents were used to modulate pulse trains delivered to four electrode pairs. For all processors, the output of each channel was time-shifted relative to other channels, varying the channel delay across channels from 0 to 240 ms (in 40-ms steps). Within each delay condition, all channels were desynchronized such that the cross-channel delays between adjacent channels were maximized, thereby avoiding local pockets of channel synchrony. Results show no significant difference between the 4- and 16-channel full-spectrum speech processor for normal-hearing listeners. Recognition scores dropped significantly only when the maximum delay reached 200 ms for the 4-channel processor and 240 ms for the 16-channel processor. When fine spectral structures were removed in the noise-band processor, sentence recognition dropped significantly when the maximum delay was 160 ms for the 16-channel noise-band processor and 40 ms for the 4-channel noise-band processor. There was no significant difference between implant listeners using the 4-channel CIS processor and normal-hearing listeners using the 4-channel noise-band processor. The results imply that when fine spectral structures are not available, as in the implant listener's case, increased spectral resolution is important for overcoming cross-channel asynchrony in speech signals.     

Talker intelligibility differences in cochlear implant listeners

People vary in the intelligibility of their speech. This study investigated whether across-talker intelligibility differences observed in normally-hearing listeners are also found in cochlear implant (CI) users. Speech perception for male, female, and child pairs of talkers differing in intelligibility was assessed with actual and simulated CI processing and in normal hearing. While overall speech recognition was, as expected, poorer for CI users, differences in intelligibility across talkers were consistent across all listener groups. This suggests that the primary determinants of intelligibility differences are preserved in the CI-processed signal, though no single critical acoustic property could be identified.     

Frequency-place compression and expansion in cochlear implant listeners

In multichannel cochlear implants, low frequency information is delivered to apical cochlear locations while high frequency information is delivered to more basal locations, mimicking the normal acoustic tonotopic organization of the auditory nerves. In clinical practice, little attention has been paid to the distribution of acoustic input across the electrodes of an individual patient that might vary in terms of spacing and absolute tonotopic location. In normal-hearing listeners, Ba?kent and Shannon (J. Acoust. Soc. Am. 113, 2003) simulated implant signal processing conditions in which the frequency range assigned to the array was systematically made wider or narrower than the simulated stimulation range in the cochlea, resulting in frequency-place compression or expansion, respectively. In general, the best speech recognition was obtained when the input acoustic information was delivered to the matching tonotopic place in the cochlea with least frequency-place distortion. The present study measured phoneme and sentence recognition scores with similar frequency-place manipulations in six Med-El Combi 40+ implant subjects. Stimulation locations were estimated using the Greenwood mapping function based on the estimated electrode insertion depth. Results from frequency-place compression and expansion with implants were similar to simulation results, especially for postlingually deafened subjects, despite the uncertainty in the actual stimulation sites of the auditory nerves. The present study shows that frequency-place mapping is an important factor in implant performance and an individual implant patient's map could be optimized with functional tests using frequency-place manipulations.     

Pure-tone auditory stream segregation and speech perception in noise in cochlear implant recipients

This study examined the ability of cochlear implant users and normal-hearing subjects to perform auditory stream segregation of pure tones. An adaptive, rhythmic discrimination task was used to assess stream segregation as a function of frequency separation of the tones. The results for normal-hearing subjects were consistent with previously published observations (L.P.A.S van Noorden, Ph.D. dissertation, Eindhoven University of Technology, Eindhoven, The Netherlands 1975), suggesting that auditory stream segregation increases with increasing frequency separation. For cochlear implant users, there appeared to be a range of pure-tone streaming abilities, with some subjects demonstrating streaming comparable to that of normal-hearing individuals, and others possessing much poorer streaming abilities. The variability in pure-tone streaming of cochlear implant users was correlated with speech perception in both steady-state noise and multi-talker babble. Moderate, statistically significant correlations between streaming and both measures of speech perception in noise were observed, with better stream segregation associated with better understanding of speech in noise. These results suggest that auditory stream segregation is a contributing factor in the ability to understand speech in background noise. The inability of some cochlear implant users to perform stream segregation may therefore contribute to their difficulties in noise backgrounds.     

Coding of vowellike signals in cochlear implant listeners

Neural-population interactions resulting from excitation overlap in multi-channel cochlear implants (CI) may cause blurring of the "internal" auditory representation of complex sounds such as vowels. In experiment I, confusion matrices for eight German steady-state vowellike signals were obtained from seven CI listeners. Identification performance ranged between 42% and 74% correct. On the basis of an information transmission analysis across all vowels, pairs of most and least frequently confused vowels were selected for each subject. In experiment II, vowel masking patterns (VMPs) were obtained using the previously selected vowels as maskers. The VMPs were found to resemble the "electrical" vowel spectra to a large extent, indicating a relatively weak effect of neural-population interactions. Correlation between vowel identification data and VMP spectral similarity, measured by means of several spectral distance metrics, showed that the CI listeners identified the vowels based on differences in the between-peak spectral information as well as the location of spectral peaks. The effect of nonlinear amplitude mapping of acoustic into "electrical" vowels, as performed in the implant processors, was evaluated separately and compared to the effect of neural-population interactions. Amplitude mapping was found to cause more blurring than neural-population interactions. Subjects exhibiting strong blurring effects yielded lower overall vowel identification scores.     

Auditory-visual speech perception in normal-hearing and cochlear-implant listeners

The present study evaluated auditory-visual speech perception in cochlear-implant users as well as normal-hearing and simulated-implant controls to delineate relative contributions of sensory experience and cues. Auditory-only, visual-only, or auditory-visual speech perception was examined in the context of categorical perception, in which an animated face mouthing ba, da, or ga was paired with synthesized phonemes from an 11-token auditory continuum. A three-alternative, forced-choice method was used to yield percent identification scores. Normal-hearing listeners showed sharp phoneme boundaries and strong reliance on the auditory cue, whereas actual and simulated implant listeners showed much weaker categorical perception but stronger dependence on the visual cue. The implant users were able to integrate both congruent and incongruent acoustic and optical cues to derive relatively weak but significant auditory-visual integration. This auditory-visual integration was correlated with the duration of the implant experience but not the duration of deafness. Compared with the actual implant performance, acoustic simulations of the cochlear implant could predict the auditory-only performance but not the auditory-visual integration. These results suggest that both altered sensory experience and improvised acoustic cues contribute to the auditory-visual speech perception in cochlear-implant users.     

Frequency modulation detection with simultaneous amplitude modulation by cochlear implant users

To better represent fine structure cues in cochlear implants (CIs), recent research has proposed varying the stimulation rate based on slowly varying frequency modulation (FM) information. The present study investigated the abilities of CI users to detect FM with simultaneous amplitude modulation (AM). FM detection thresholds (FMDTs) for 10-Hz sinusoidal FM and upward frequency sweeps were measured as a function of standard frequency (75-1000 Hz). Three AM conditions were tested, including (1) No AM, (2) 20-Hz Sinusoidal AM (SAM) with modulation depths of 10%, 20%, or 30%, and (3) Noise AM (NAM), in which the amplitude was randomly and uniformly varied over a range of 1, 2, or 3 dB, relative to the reference amplitude. Results showed that FMDTs worsened with increasing standard frequencies, and were lower for sinusoidal FM than for upward frequency sweeps. Simultaneous AM significantly interfered with FM detection; FMDTs were significantly poorer with simultaneous NAM than with SAM. Besides, sinusoidal FMDTs significantly worsened when the starting phase of simultaneous SAM was randomized. These results suggest that FM and AM in CI partly share a common loudness-based coding mechanism and the feasibility of "FM+AM" strategies for CI speech processing may be limited.     

Patterns of phoneme perception errors by listeners with cochlear implants as a function of overall speech perception ability

Many studies have noted great variability in speech perception ability among postlingually deafened adults with cochlear implants. This study examined phoneme misperceptions for 30 cochlear implant listeners using either the Nucleus-22 or Clarion version 1.2 device to examine whether listeners with better overall speech perception differed qualitatively from poorer listeners in their perception of vowel and consonant features. In the first analysis, simple regressions were used to predict the mean percent-correct scores for consonants and vowels for the better group of listeners from those of the poorer group. A strong relationship between the two groups was found for consonant identification, and a weak, nonsignificant relationship was found for vowel identification. In the second analysis, it was found that less information was transmitted for consonant and vowel features to the poorer listeners than to the better listeners; however, the pattern of information transmission was similar across groups. Taken together, results suggest that the performance difference between the two groups is primarily quantitative. The results underscore the importance of examining individuals' perception of individual phoneme features when attempting to relate speech perception to other predictor variables.     

Acoustic parameters measured by a formant-estimating speech processor for a multiple-channel cochlear implant

In order to assess the limitations imposed on a cochlear implant system by a wearable speech processor, the parameters extracted from a set of 11 vowels and 24 consonants were examined. An estimate of the fundamental frequency EF 0 was derived from the zero crossings of the low-pass filtered envelope of the waveform. Estimates of the first and second formant frequencies EF 1 and EF 2 were derived from the zero crossings of the waveform, which was filtered in the ranges 300-1000 and 800-4000 Hz. Estimates of the formant amplitudes EA 1 and EA 2 were derived by peak detectors operating on the outputs of the same filters. For vowels, these parameters corresponded well to the first and second formants and gave sufficient information to identify each vowel. For consonants, the relative levels and onset times of EA 1 and EA 2 and the EF 0 values gave cues to voicing. The variation in time of EA 1, EA 2, EF 1, and EF 2 gave cues to the manner of articulation. Cues to the place of articulation were given by EF 1 and EF 2. When pink noise was added, the parameters were gradually degraded as the signal-to-noise ratio decreased. Consonants were affected more than vowels, and EF 2 was affected more than EF 1. Results for three good patients using a speech processor that coded EF 0 as an electric pulse rate, EF 1 and EF 2 as electrode positions, and EA 1 and EA 2 as electric current levels confirmed that the parameters were useful for recognition of vowels and consonants. Average scores were 76% for recognition of 11 vowels and 71% for 12 consonants in the hearing-alone condition. The error rates were 4% for voicing, 12% for manner, and 25% for place.     

Effects of electrode configuration on cochlear implant modulation detection thresholds

Cochlear implant function, as assessed by psychophysical measures, varies from one stimulation site to another within a patient's cochlea. This suggests that patient performance might be improved by selection of the best-functioning sites for the processor map. In evaluating stimulation sites for such a strategy, electrode configuration is an important variable. Variation across stimulation sites in loudness-related measures (detection thresholds and maximum comfortable loudness levels), is much larger for stimulation with bipolar electrode configurations than with monopolar configurations. The current study found that, in contrast to the loudness-related measures, magnitudes of across-site means and the across-site variances of modulation detection thresholds were not dependent on electrode configuration, suggesting that the mechanisms underlying variation in these various psychophysical measures are not all the same. The data presented here suggest that bipolar and monopolar electrode configurations are equally effective in identifying good and poor stimulation sites for modulation detection but that the across-site patterns of modulation detection thresholds are not the same for the two configurations. Therefore, it is recommended to test all stimulation sites using the patient's clinically assigned electrode configuration when performing psychophysical evaluation of a patient's modulation detection acuity to select sites for the processor map.     

Spectral peak resolution and speech recognition in quiet: normal hearing, hearing impaired, and cochlear implant listeners

Spectral peak resolution was investigated in normal hearing (NH), hearing impaired (HI), and cochlear implant (CI) listeners. The task involved discriminating between two rippled noise stimuli in which the frequency positions of the log-spaced peaks and valleys were interchanged. The ripple spacing was varied adaptively from 0.13 to 11.31 ripples/octave, and the minimum ripple spacing at which a reversal in peak and trough positions could be detected was determined as the spectral peak resolution threshold for each listener. Spectral peak resolution was best, on average, in NH listeners, poorest in CI listeners, and intermediate for HI listeners. There was a significant relationship between spectral peak resolution and both vowel and consonant recognition in quiet across the three listener groups. The results indicate that the degree of spectral peak resolution required for accurate vowel and consonant recognition in quiet backgrounds is around 4 ripples/octave, and that spectral peak resolution poorer than around 1-2 ripples/octave may result in highly degraded speech recognition. These results suggest that efforts to improve spectral peak resolution for HI and CI users may lead to improved speech recognition.     

The impact of reverberant self-masking and overlap-masking effects on speech intelligibility by cochlear implant listeners (L)

The purpose of this study is to determine the relative impact of reverberant self-masking and overlap-masking effects on speech intelligibility by cochlear implant listeners. Sentences were presented in two conditions wherein reverberant consonant segments were replaced with clean consonants, and in another condition wherein reverberant vowel segments were replaced with clean vowels. The underlying assumption is that self-masking effects would dominate in the first condition, whereas overlap-masking effects would dominate in the second condition. Results indicated that the degradation of speech intelligibility in reverberant conditions is caused primarily by self-masking effects that give rise to flattened formant transitions.     

Pitch perception by cochlear implant subjects

Direct electrical stimulation of the auditory nerve can be used to restore some degree of hearing to the profoundly deaf. Percepts due to electrical stimulation have characteristics corresponding approximately to the acoustic percepts of loudness, pitch, and timbre. To encode speech as a pattern of electrical stimulation, it is necessary to determine the effects of the stimulus parameters on these percepts. The effects of the three basic stimulus parameters of level, repetition rate, and stimulation location on subjects' percepts were examined. Pitch difference limens arising from changes in rate of stimulation increase as the stimulating rate increases, up to a saturation point of between 200 and 1000 pulses per second. Changes in pitch due to electrode selection depend upon the subject, but generally agree with a tonotopic organization of the human cochlea. Further, the discriminability of such place-pitch percepts seems to be dependent on the degree of current spread in the cochlea. The effect of stimulus level on perceived pitch is significant but is highly dependent on the individual tested. The results of these experiments are discussed in terms of their impact on speech-processing strategies and their relevance to acoustic pitch perception.     

Comparing spatial tuning curves, spectral ripple resolution, and speech perception in cochlear implant users

Spectral ripple discrimination thresholds were measured in 15 cochlear-implant users with broadband (350-5600 Hz) and octave-band noise stimuli. The results were compared with spatial tuning curve (STC) bandwidths previously obtained from the same subjects. Spatial tuning curve bandwidths did not correlate significantly with broadband spectral ripple discrimination thresholds but did correlate significantly with ripple discrimination thresholds when the rippled noise was confined to an octave-wide passband, centered on the STC's probe electrode frequency allocation. Ripple discrimination thresholds were also measured for octave-band stimuli in four contiguous octaves, with center frequencies from 500 Hz to 4000 Hz. Substantial variations in thresholds with center frequency were found in individuals, but no general trends of increasing or decreasing resolution from apex to base were observed in the pooled data. Neither ripple nor STC measures correlated consistently with speech measures in noise and quiet in the sample of subjects in this study. Overall, the results suggest that spectral ripple discrimination measures provide a reasonable measure of spectral resolution that correlates well with more direct, but more time-consuming, measures of spectral resolution, but that such measures do not always provide a clear and robust predictor of performance in speech perception tasks.