Psychophysically based site selection coupled with dichotic stimulation improves speech recognition in noise with bilateral cochlear implants

The ability to perceive important features of electrical stimulation varies across stimulation sites within a multichannel implant. The aim of this study was to optimize speech processor MAPs for bilateral implant users by identifying and removing sites with poor psychophysical performance. The psychophysical assessment involved amplitude-modulation detection with and without a masker, and a channel interaction measure quantified as the elevation in modulation detection thresholds in the presence of the masker. Three experimental MAPs were created on an individual-subject basis using data from one of the three psychophysical measures. These experimental MAPs improved the mean psychophysical acuity across the electrode array and provided additional advantages such as increasing spatial separations between electrodes and/or preserving frequency resolution. All 8 subjects showed improved speech recognition in noise with one or more experimental MAPs over their everyday-use clinical MAP. For most subjects, phoneme and sentence recognition in noise were significantly improved by a dichotic experimental MAP that provided better mean psychophysical acuity, a balanced distribution of selected stimulation sites, and preserved frequency resolution. The site-selection strategies serve as useful tools for evaluating the importance of psychophysical acuities needed for good speech recognition in implant users.     

Across-site patterns of modulation detection in listeners with cochlear implants

In modern cochlear implants, much of the information required for recognition of important sounds is conveyed by temporal modulation of the charge per phase in interleaved trains of electrical pulses. In this study, modulation detection thresholds (MDTs) were used to assess listeners' abilities to detect sinusoidal modulation of charge per phase at each available stimulation site in their 22-electrode implants. Fourteen subjects were tested. MDTs were found to be highly variable across stimulation sites in most listeners. The across-site patterns of MDTs differed considerably from subject to subject. The subject-specific patterns of across-site variability of MDTs suggest that peripheral site-specific characteristics, such as electrode placement and the number and condition of surviving neurons, play a primary role in determining modulation sensitivity. Across-site patterns of detection thresholds (T levels), maximum comfortable loudness levels (C levels) and dynamic ranges (DRs) were not consistently correlated with across-site patterns of MDTs within subjects, indicating that the mechanisms underlying across-site variation in these measures differed from those underlying across-site variation in MDTs. MDTs sampled from multiple sites in a listener's electrode array might be useful for diagnosing across-subject differences in speech recognition with cochlear implants and for guiding strategies to improve the individual's perception.     

Acoustic temporal modulation detection and speech perception in cochlear implant listeners

The goals of the present study were to measure acoustic temporal modulation transfer functions (TMTFs) in cochlear implant listeners and examine the relationship between modulation detection and speech recognition abilities. The effects of automatic gain control, presentation level and number of channels on modulation detection thresholds (MDTs) were examined using the listeners' clinical sound processor. The general form of the TMTF was low-pass, consistent with previous studies. The operation of automatic gain control had no effect on MDTs when the stimuli were presented at 65 dBA. MDTs were not dependent on the presentation levels (ranging from 50 to 75 dBA) nor on the number of channels. Significant correlations were found between MDTs and speech recognition scores. The rates of decay of the TMTFs were predictive of speech recognition abilities. Spectral-ripple discrimination was evaluated to examine the relationship between temporal and spectral envelope sensitivities. No correlations were found between the two measures, and 56% of the variance in speech recognition was predicted jointly by the two tasks. The present study suggests that temporal modulation detection measured with the sound processor can serve as a useful measure of the ability of clinical sound processing strategies to deliver clinically pertinent temporal information.     

Effects of carrier pulse rate and stimulation site on modulation detection by subjects with cochlear implants

Most modern cochlear-implant speech processors convey speech-envelope information using amplitude-modulated pulse trains. The use of higher-rate carrier pulse trains allows more envelope detail in the signal. However, neural response properties could limit the efficacy of high-rate carriers. This study examined effects of carrier rate and stimulation site, on psychophysical modulation detection thresholds (MDTs). Both of these variables could affect the neural representation of the carrier and thus affect perception of the modulation. Twelve human subjects with cochlear implants were tested. Phase duration of symmetric biphasic pulses was modulated sinusoidally at 40 Hz. MDTs were determined for monopolar stimulation at two carrier rates [250 and 4000 pulses/s (pps)], three stimulation sites (basal, middle, and apical), and five stimulus levels (10%, 30%, 50%, 70%, and 90% of the dynamic range). MDTs were lower for 250 pps carriers than for 4000 pps carriers in 71% of the 180 cases studied. Effects of carrier rate were greatest at the apical stimulation site and effects of stimulation site on MDTs depended on carrier rate. The data suggest a distinct disadvantage to using carrier pulse rates as high as 4000 pps. Stimulation site should be considered in evaluating modulation detection ability.     

Consonant identification under maskers with sinusoidal modulation: masking release or modulation interference?

The present study investigated the effect of envelope modulations in a background masker on consonant recognition by normal hearing listeners. It is well known that listeners understand speech better under a temporally modulated masker than under a steady masker at the same level, due to masking release. The possibility of an opposite phenomenon, modulation interference, whereby speech recognition could be degraded by a modulated masker due to interference with auditory processing of the speech envelope, was hypothesized and tested under various speech and masker conditions. It was of interest whether modulation interference for speech perception, if it were observed, could be predicted by modulation masking, as found in psychoacoustic studies using nonspeech stimuli. Results revealed that masking release measurably occurred under a variety of conditions, especially when the speech signal maintained a high degree of redundancy across several frequency bands. Modulation interference was also clearly observed under several circumstances when the speech signal did not contain a high redundancy. However, the effect of modulation interference did not follow the expected pattern from psychoacoustic modulation masking results. In conclusion, (1) both factors, modulation interference and masking release, should be accounted for whenever a background masker contains temporal fluctuations, and (2) caution needs to be taken when psychoacoustic theory on modulation masking is applied to speech recognition.     

Comodulation masking release in speech identification with real and simulated cochlear-implant hearing

For normal-hearing (NH) listeners, masker energy outside the spectral region of a target signal can improve target detection and identification, a phenomenon referred to as comodulation masking release (CMR). This study examined whether, for cochlear implant (CI) listeners and for NH listeners presented with a "noise vocoded" CI simulation, speech identification in modulated noise is improved by a co-modulated flanking band. In Experiment 1, NH listeners identified noise-vocoded speech in a background of on-target noise with or without a flanking narrow band of noise outside the spectral region of the target. The on-target noise and flanker were either 16-Hz square-wave modulated with the same phase or were unmodulated; the speech was taken from a closed-set corpus. Performance was better in modulated than in unmodulated noise, and this difference was slightly greater when the comodulated flanker was present, consistent with a small CMR of about 1.7 dB for noise-vocoded speech. Experiment 2, which tested CI listeners using the same speech materials, found no advantage for modulated versus unmodulated maskers and no CMR. Thus although NH listeners can benefit from CMR even for speech signals with reduced spectro-temporal detail, no CMR was observed for CI users.     

Relative contribution of off- and on-frequency spectral components of background noise to the masking of unprocessed and vocoded speech

The present study examined the relative influence of the off- and on-frequency spectral components of modulated and unmodulated maskers on consonant recognition. Stimuli were divided into 30 contiguous equivalent rectangular bandwidths. The temporal fine structure (TFS) in each "target" band was either left intact or replaced with tones using vocoder processing. Recognition scores for 10, 15 and 20 target bands randomly located in frequency were obtained in quiet and in the presence of all 30 masker bands, only the off-frequency masker bands, or only the on-frequency masker bands. The amount of masking produced by the on-frequency bands was generally comparable to that produced by the broadband masker. However, the difference between these two conditions was often significant, indicating an influence of the off-frequency masker bands, likely through modulation interference or spectral restoration. Although vocoder processing systematically lead to poorer consonant recognition scores, the deficit observed in noise could often be attributed to that observed in quiet. These data indicate that (i) speech recognition is affected by the off-frequency components of the background and (ii) the nature of the target TFS does not systematically affect speech recognition in noise, especially when energetic masking and/or the number of target bands is limited.     

Sentence recognition in noise promoting or suppressing masking release by normal-hearing and cochlear-implant listeners

Normal-hearing (NH) listeners maintain robust speech understanding in modulated noise by "glimpsing" portions of speech from a partially masked waveform--a phenomenon known as masking release (MR). Cochlear implant (CI) users, however, generally lack such resiliency. In previous studies, temporal masking of speech by noise occurred randomly, obscuring to what degree MR is attributable to the temporal overlap of speech and masker. In the present study, masker conditions were constructed to either promote (+MR) or suppress (-MR) masking release by controlling the degree of temporal overlap. Sentence recognition was measured in 14 CI subjects and 22 young-adult NH subjects. Normal-hearing subjects showed large amounts of masking release in the +MR condition and a marked difference between +MR and -MR conditions. In contrast, CI subjects demonstrated less effect of MR overall, and some displayed modulation interference as reflected by poorer performance in modulated maskers. These results suggest that the poor performance of typical CI users in noise might be accounted for by factors that extend beyond peripheral masking, such as reduced segmental boundaries between syllables or words. Encouragingly, the best CI users tested here could take advantage of masker fluctuations to better segregate the speech from the background.     

Spectral integration in bands of modulated or unmodulated noise

Spectral integration was measured for pure-tone signals masked by unmodulated or modulated noise bands centered at the signal frequencies. The bands were typically 100 Hz wide, and when modulated, they were sinusoidally amplitude modulated at a rate of 8 Hz and a depth of 100%. In experiment 1, thresholds were first measured for each individual pure tone of a triplet in the presence of its respective masker band, and then for those three tones added together at their respective threshold levels, masked by their respective masker bands. Four sets of triplets were used: 250, 1000, 4000 Hz; 354, 1000, 2828 Hz; 500, 1000, 2000 Hz; and 800, 1000, 1200 Hz. When the masker bands were unmodulated, the amount of spectral integration was about 2.4 dB for all triplets, consistent with the integration expected based on the multiband energy detector model. When the bands were modulated, the amount of integration depended upon the spacing between masker bands; for the two widest spacings, the integration was between about 0 and 3 dB, whereas for the two closest spacings, the integration was approximately 5 dB. Experiments 2 and 3 addressed the cause of this greater spectral integration in the presence of the modulated masker bands with closer spacing. The second experiment demonstrated that sensitivity (d') was proportional to signal power regardless of whether the background noise was modulated or not, and thus the greater integration in dB in the presence of the modulated noise bands could not be accounted for by shallower psychometric functions in those conditions. Instead, the third experiment showed that the greater integration was likely due to the fact that the masker bands were comodulated. In other words, it was probably due to cues related to comodulation masking release when all three bands (and signals) were present.     

Effect of amplitude modulation coherence for masked speech signals filtered into narrow bands

Introduction of masker amplitude modulation (AM) can improve signal detection in a number of paradigms. In some cases this advantage depends on the coherence of modulation across a relatively wide frequency range. In the experiments described below, observers were asked to identify masked spondee words produced by a single male talker. The target spondees and masking noise were filtered into nine narrow bands, and the coherence of AM of either the speech signal or noise masker was manipulated. Inherent modulation of the masker bands was manipulated via assignment of real and imaginary values to the associated components of each band in the frequency domain, and AM of speech bands was achieved via multiplication with envelopes extracted from these maskers. Responses were based on two alternatives, four alternatives, or open response sets. The effect of masker AM coherence was highly dependent upon the size of the response set: coherent AM was associated with better thresholds in a two-alternative response set, but poorer thresholds in an open response set. Results with AM speech did not depend critically upon the across-frequency temporal synchrony of AM imposed on the speech material.     

Forward masking in patients with cochlear implants

Forward masking was measured in 12 patients with cochlear implants. The amount of masking (in microamps) decreased linearly as a function of the logarithm of the signal delay from masker offset. Normalized forward-masking recovery functions for cochlear implants were similar to normalized functions of normal-hearing listeners, indicating that the mechanism of psychophysical forward masking is retrocochlear. These results indicate that the logarithm of acoustic amplitude should be mapped to microamps to produce normal forward masking in implanted patients. Despite the fact that the forward-masking recovery functions were similar across all patients, their performance with their speech processor varied widely.     

Modulation detection interference: effects of concurrent and sequential streaming

The presence of amplitude fluctuations in one frequency region can interfere with our ability to detect similar fluctuations in another (remote) frequency region. This effect is known as modulation detection interference (MDI). Gating the interfering and target sounds asynchronously is known to lead to a reduction in MDI, presumably because the two sounds become perceptually segregated. The first experiment examined the relative effects of carrier and modulator gating asynchrony in producing a release from MDI. The target carrier was a 900-ms, 4.3-kHz sinusoid, modulated in amplitude by a 500-ms, 16-Hz sinusoid, with 200-ms unmodulated fringes preceding and following the modulation. The interferer (masker) was a 1-kHz sinusoid, modulated by a narrowband noise with a 16-Hz bandwidth, centered around 16 Hz. Extending the masker carrier for 200 ms before and after the signal carrier reduced MDI, regardless of whether the target and masker modulators were gated synchronously or were gated with onset and offset asynchronies of 200 ms. Similarly, when the carriers were gated synchronously, asynchronous gating of the modulators did not produce a release from MDI. The second experiment measured MDI with a synchronous target and masker and investigated the effect of adding a series of precursor tones, which were designed to promote the forming of a perceptual stream with the masker, thereby leaving the target perceptually isolated. Four modulated or unmodulated precursor tones presented at the masker frequency were sufficient to completely eliminate MDI. The results support the idea that MDI is due to a perceptual grouping of the masker and target, and show that conditions promoting sufficient perceptual segregation of the masker and target can lead to a total elimination of MDI.     

Measurement of auditory temporal processing using modified masking period patterns

A common metric of auditory temporal processing is the difference in the threshold for a pure-tone signal masked by either unmodulated or amplitude-modulated noise. This technique may be viewed as a modification of the masking period pattern technique. Such measurements have been proposed as an efficient means of estimating auditory temporal resolution in a clinical setting, although in many cases threshold differences may reflect additional spectro-temporal processes. The primary purpose of the present experiment was to examine interactions among signal frequency and masker bandwidth and the effects of modulation frequency on modified masking period patterns. The results revealed unmodulated-modulated threshold differences that increased with increasing masker bandwidth and decreased with increasing modulation frequency. There was little effect of signal frequency for narrow-band noise maskers that were equal in absolute bandwidth across frequency. However, unmodulated-modulated threshold differences increased substantially with increasing signal frequency for bandwidths proportional to the signal frequency and for wideband maskers. Although the results are interpreted in terms of a combination of both within-channel and across-channel cues, the specific contributions of these cues in particular conditions are difficult to ascertain. Because modified masking period patterns depend strongly upon a number of specific stimulus parameters, and because it is difficult to determine with any precision the underlying perceptual processes, this technique is not recommended for use as a clinical measure of auditory temporal processing.     

Benefit of modulated maskers for speech recognition by younger and older adults with normal hearing

To assess age-related differences in benefit from masker modulation, younger and older adults with normal hearing but not identical audiograms listened to nonsense syllables in each of two maskers: (1) a steady-state noise shaped to match the long-term spectrum of the speech, and (2) this same noise modulated by a 10-Hz square wave, resulting in an interrupted noise. An additional low-level broadband noise was always present which was shaped to produce equivalent masked thresholds for all subjects. This minimized differences in speech audibility due to differences in quiet thresholds among subjects. An additional goal was to determine if age-related differences in benefit from modulation could be explained by differences in thresholds measured in simultaneous and forward maskers. Accordingly, thresholds for 350-ms pure tones were measured in quiet and in each masker; thresholds for 20-ms signals in forward and simultaneous masking were also measured at selected signal frequencies. To determine if benefit from modulated maskers varied with masker spectrum and to provide a comparison with previous studies, a subgroup of younger subjects also listened in steady-state and interrupted noise that was not spectrally shaped. Articulation index (AI) values were computed and speech-recognition scores were predicted for steady-state and interrupted noise; predicted benefit from modulation was also determined. Masked thresholds of older subjects were slightly higher than those of younger subjects; larger age-related threshold differences were observed for short-duration than for long-duration signals. In steady-state noise, speech recognition for older subjects was poorer than for younger subjects, which was partially attributable to older subjects' slightly higher thresholds in these maskers. In interrupted noise, although predicted benefit was larger for older than younger subjects, scores improved more for younger than for older subjects, particularly at the higher noise level. This may be related to age-related increases in thresholds in steady-state noise and in forward masking, especially at higher frequencies. Benefit of interrupted maskers was larger for unshaped than for speech-shaped noise, consistent with AI predictions.     

The role of visual speech cues in reducing energetic and informational masking

Two experiments compared the effect of supplying visual speech information (e.g., lipreading cues) on the ability to hear one female talker's voice in the presence of steady-state noise or a masking complex consisting of two other female voices. In the first experiment intelligibility of sentences was measured in the presence of the two types of maskers with and without perceived spatial separation of target and masker. The second study tested detection of sentences in the same experimental conditions. Results showed that visual cues provided more benefit for both recognition and detection of speech when the masker consisted of other voices (versus steady-state noise). Moreover, visual cues provided greater benefit when the target speech and masker were spatially coincident versus when they appeared to arise from different spatial locations. The data obtained here are consistent with the hypothesis that lipreading cues help to segregate a target voice from competing voices, in addition to the established benefit of supplementing masked phonetic information.     

Pitch of amplitude-modulated irregular-rate stimuli in acoustic and electric hearing

The pitch of stimuli was studied under conditions where place-of-excitation was held constant, and where pitch was therefore derived from "purely temporal" cues. In experiment 1, the acoustical and electrical pulse trains consisted of pulses whose amplitudes alternated between a high and a low value, and whose interpulse intervals alternated between 4 and 6 ms. The attenuated pulses occurred after the 4-ms intervals in condition A, and after the 6-ms intervals in condition B. For both normal-hearing subjects and cochlear implantees, the period of an isochronous pulse train equal in pitch to this "4-6" stimulus increased from near 6 ms at the smallest modulation depth to nearly 10 ms at the largest depth. Additionally, the modulated pulse trains in condition A were perceived as being lower in pitch than those in condition B. Data are interpreted in terms of increased refractoriness in condition A, where the larger pulses are more closely followed by the smaller ones than in condition B. Consistent with this conclusion, the A-B difference was reduced at longer interpulse intervals. These findings provide a measure of supra-threshold effects of refractoriness on pitch perception, and increase our understanding of coding of temporal information in cochlear implant speech processing schemes.     

Channel weights for speech recognition in cochlear implant users

The purpose of this study was to develop and validate a method of estimating the relative "weight" that a multichannel cochlear implant user places on individual channels, indicating its contribution to overall speech recognition. The correlational method as applied to speech recognition was used both with normal-hearing listeners and with cochlear implant users fitted with six-channel speech processors. Speech was divided into frequency bands corresponding to the bands of the processor and a randomly chosen level of corresponding filtered noise was added to each channel on each trial. Channels in which the signal-to-noise ratio was more highly correlated with performance have higher weights, and conversely, channels in which the correlations were smaller have lower weights. Normal-hearing listeners showed approximately equal weights across frequency bands. In contrast, cochlear implant users showed unequal weighting across bands, and varied from individual to individual with some channels apparently not contributing significantly to speech recognition. To validate these channel weights, individual channels were removed and speech recognition in quiet was tested. A strong correlation was found between the relative weight of the channel removed and the decrease in speech recognition, thus providing support for use of the correlational method for cochlear implant users.     

Recognition of spectrally asynchronous speech by normal-hearing listeners and Nucleus-22 cochlear implant users

This experiment examined the effects of spectral resolution and fine spectral structure on recognition of spectrally asynchronous sentences by normal-hearing and cochlear implant listeners. Sentence recognition was measured in six normal-hearing subjects listening to either full-spectrum or noise-band processors and five Nucleus-22 cochlear implant listeners fitted with 4-channel continuous interleaved sampling (CIS) processors. For the full-spectrum processor, the speech signals were divided into either 4 or 16 channels. For the noise-band processor, after band-pass filtering into 4 or 16 channels, the envelope of each channel was extracted and used to modulate noise of the same bandwidth as the analysis band, thus eliminating the fine spectral structure available in the full-spectrum processor. For the 4-channel CIS processor, the amplitude envelopes extracted from four bands were transformed to electric currents by a power function and the resulting electric currents were used to modulate pulse trains delivered to four electrode pairs. For all processors, the output of each channel was time-shifted relative to other channels, varying the channel delay across channels from 0 to 240 ms (in 40-ms steps). Within each delay condition, all channels were desynchronized such that the cross-channel delays between adjacent channels were maximized, thereby avoiding local pockets of channel synchrony. Results show no significant difference between the 4- and 16-channel full-spectrum speech processor for normal-hearing listeners. Recognition scores dropped significantly only when the maximum delay reached 200 ms for the 4-channel processor and 240 ms for the 16-channel processor. When fine spectral structures were removed in the noise-band processor, sentence recognition dropped significantly when the maximum delay was 160 ms for the 16-channel noise-band processor and 40 ms for the 4-channel noise-band processor. There was no significant difference between implant listeners using the 4-channel CIS processor and normal-hearing listeners using the 4-channel noise-band processor. The results imply that when fine spectral structures are not available, as in the implant listener's case, increased spectral resolution is important for overcoming cross-channel asynchrony in speech signals.     

A masking level difference due to harmonicity

The role of harmonicity in masking was studied by comparing the effect of harmonic and inharmonic maskers on the masked thresholds of noise probes using a three-alternative, forced-choice method. Harmonic maskers were created by selecting sets of partials from a harmonic series with an 88-Hz fundamental and 45 consecutive partials. Inharmonic maskers differed in that the partial frequencies were perturbed to nearby values that were not integer multiples of the fundamental frequency. Average simultaneous-masked thresholds were as much as 10 dB lower with the harmonic masker than with the inharmonic masker, and this difference was unaffected by masker level. It was reduced or eliminated when the harmonic partials were separated by more than 176 Hz, suggesting that the effect is related to the extent to which the harmonics are resolved by auditory filters. The threshold difference was not observed in a forward-masking experiment. Finally, an across-channel mechanism was implicated when the threshold difference was found between a harmonic masker flanked by harmonic bands and a harmonic masker flanked by inharmonic bands. A model developed to explain the observed difference recognizes that an auditory filter output envelope is modulated when the filter passes two or more sinusoids, and that the modulation rate depends on the differences among the input frequencies. For a harmonic masker, the frequency differences of adjacent partials are identical, and all auditory filters have the same dominant modulation rate. For an inharmonic masker, however, the frequency differences are not constant and the envelope modulation rate varies across filters. The model proposes that a lower variability facilitates detection of a probe-induced change in the variability, thus accounting for the masked threshold difference. The model was supported by significantly improved predictions of observed thresholds when the predictor variables included envelope modulation rate variance measured using simulated auditory filters.     

Comparison of level discrimination, increment detection, and comodulation masking release in the audio- and envelope-frequency domains

In general, the temporal structure of stimuli must be considered to account for certain observations made in detection and masking experiments in the audio-frequency domain. Two such phenomena are (1) a heightened sensitivity to amplitude increments with a temporal fringe compared to gated level discrimination performance and (2) lower tone-in-noise detection thresholds using a modulated masker compared to those using an unmodulated masker. In the current study, translations of these two experiments were carried out to test the hypothesis that analogous cues might be used in the envelope-frequency domain. Pure-tone carrier amplitude-modulation (AM) depth-discrimination thresholds were found to be similar using both traditional gated stimuli and using a temporally modulated fringe for a fixed standard depth (ms = 0.25) and a range of AM frequencies (4-64 Hz). In a second experiment, masked sinusoidal AM detection thresholds were compared in conditions with and without slow and regular fluctuations imposed on the instantaneous masker AM depth. Release from masking was obtained only for very slow masker fluctuations (less than 2 Hz). A physiologically motivated model that effectively acts as a first-order envelope change detector accounted for several, but not all, of the key aspects of the data.