Using blind source separation techniques to improve speech recognition in bilateral cochlear implant patients

Bilateral cochlear implants seek to restore the advantages of binaural hearing by improving access to binaural cues. Bilateral implant users are currently fitted with two processors, one in each ear, operating independent of one another. In this work, a different approach to bilateral processing is explored based on blind source separation (BSS) by utilizing two implants driven by a single processor. Sentences corrupted by interfering speech or speech-shaped noise are presented to bilateral cochlear implant users at 0 dB signal-to-noise ratio in order to evaluate the performance of the proposed BSS method. Subjects are tested in both anechoic and reverberant settings, wherein the target and masker signals are spatially separated. Results indicate substantial improvements in performance in both anechoic and reverberant settings over the subjects' daily strategies for both masker conditions and at various locations of the masker. It is speculated that such improvements are due to the fact that the proposed BSS algorithm capitalizes on the variations of interaural level differences and interaural time delays present in the mixtures of the signals received by the two microphones, and exploits that information to spatially separate the target from the masker signals.     

Speech perception,localization, and lateralization with bilateral cochlear implants

Five bilateral cochlear implant users were tested for their localization abilities and speech understanding in noise, for both monaural and binaural listening conditions. They also participated in lateralization tasks to assess the impact of variations in interaural time delays (ITDs) and interaural level differences (ILDs) for electrical pulse trains under direct computer control. The localization task used pink noise bursts presented from an eight-loudspeaker array spanning an arc of approximately 108 degrees in front of the listeners at ear level (0-degree elevation). Subjects showed large benefits from bilateral device use compared to either side alone. Typical root-mean-square (rms) averaged errors across all eight loudspeakers in the array were about 10 degrees for bilateral device use and ranged from 20 degrees to 60 degrees using either ear alone. Speech reception thresholds (SRTs) were measured for sentences presented from directly in front of the listeners (0 degrees) in spectrally matching speech-weighted noise at either 0 degrees, +90 degrees or -90 degrees for four subjects out of five tested who could perform the task. For noise to either side, bilateral device use showed a substantial benefit over unilateral device use when noise was ipsilateral to the unilateral device. This was primarily because of monaural head-shadow effects, which resulted in robust SRT improvements (P<0.001) of about 4 to 5 dB when ipsilateral and contralateral noise positions were compared. The additional benefit of using both ears compared to the shadowed ear (i.e., binaural unmasking) was only 1 or 2 dB and less robust (P = 0.04). Results from the lateralization studies showed consistently good sensitivity to ILDs; better than the smallest level adjustment available in the implants (0.17 dB) for some subjects. Sensitivity to ITDs was moderate on the other hand, typically of the order of 100 micros. ITD sensitivity deteriorated rapidly when stimulation rates for unmodulated pulse-trains increased above a few hundred Hz but at 800 pps showed sensitivity comparable to 50-pps pulse-trains when a 50-Hz modulation was applied. In our opinion, these results clearly demonstrate important benefits are available from bilateral implantation, both for localizing sounds (in quiet) and for listening in noise when signal and noise sources are spatially separated. The data do indicate, however, that effects of interaural timing cues are weaker than those from interaural level cues and according to our psychophysical findings rely on the availability of low-rate information below a few hundred Hz.     

The effects of binaural spectral resolution mismatch on Mandarin speech perception in simulated electric hearing

This study assessed the effects of binaural spectral resolution mismatch on the intelligibility of Mandarin speech in noise using bilateral cochlear implant simulations. Noise-vocoded Mandarin speech, corrupted by speech-shaped noise at 0 and 5?dB signal-to-noise ratios, were presented unilaterally or bilaterally to normal-hearing listeners with mismatched spectral resolution between ears. Significant binaural benefits for Mandarin speech recognition were observed only with matched spectral resolution between ears. In addition, the performance of tone identification was more robust to noise than that of sentence recognition, suggesting factors other than tone identification might account more for the degraded sentence recognition in noise.     

Binaural unmasking with multiple adjacent masking electrodes in bilateral cochlear implant users

Bilateral cochlear implant (BiCI) users gain an advantage in noisy situations from a second implant, but their bilateral performance falls short of normal hearing listeners. Channel interactions due to overlapping electrical fields between electrodes can impair speech perception, but its role in limiting binaural hearing performance has not been well characterized. To address the issue, binaural masking level differences (BMLD) for a 125 Hz tone in narrowband noise were measured using a pair of pitch-matched electrodes while simultaneously presenting the same masking noise to adjacent electrodes, representing a more realistic stimulation condition compared to prior studies that used only a single electrode pair. For five subjects, BMLDs averaged 8.9 ± 1.0 dB (mean ± s.e.) in single electrode pairs but dropped to 2.1 ± 0.4 dB when presenting noise on adjacent masking electrodes, demonstrating a negative impact of the additional maskers. Removing the masking noise from only the pitch-matched electrode pair not only lowered thresholds but also resulted in smaller BMLDs. The degree of channel interaction estimated from auditory nerve evoked potentials in three subjects was significantly and negatively correlated with BMLD. The data suggest that if the amount of channel interactions can be reduced, BiCI users may experience some performance improvements related to binaural hearing.     

Binaural benefit for speech recognition with spectral mismatch across ears in simulated electric hearing

The present study investigated the effects of binaural spectral mismatch on binaural benefits in the context of bilateral cochlear implants using acoustic simulations. Binaural spectral mismatch was systematically manipulated by simulating changes in the relative insertion depths across ears. Sentence recognition, presented unilaterally and bilaterally, were measured in normal-hearing listeners in quiet and noise at +5 dB signal-to-noise ratio. Significant binaural benefits were observed when the interaural difference in insertion depth was 1 mm or less. This result suggests a dependence of the binaural benefit on redundant speech information, rather than on similarity in performance across ears.     

Spatial release from masking in children with normal hearing and with bilateral cochlear implants: effect of interferer asymmetry

Spatial release from masking (SRM) was measured in groups of children with bilateral cochlear implants (BiCIs, average ages 6.0 and 7.9 yr) and with normal hearing (NH, average ages 5.0 and 7.8 yr). Speech reception thresholds (SRTs) were measured for target speech in front (0°), and interferers in front, distributed asymmetrically toward the right (+90°/+90°) or distributed symmetrically toward the right and left (+90°/-90°). In the asymmetrical condition both monaural "better ear" and binaural cues are available. In the symmetrical condition, listeners rely heavily on binaural cues to segregate sources. SRM was computed as the difference between SRTs in the front condition and SRTs in either the asymmetrical or symmetrical conditions. Results showed that asymmetrical SRM was smaller in BiCI users than NH children. Furthermore, NH children showed symmetrical SRM, suggesting they are able to use binaural cues for source segregation, whereas children with BiCIs had minimal or absent symmetrical SRM. These findings suggest that children who receive BiCIs can segregate speech from noise under conditions that maximize monaural better ear cues. Limitations in the CI devices likely play an important role in limiting SRM. Thus, improvement in spatial hearing abilities in children with BiCIs may require binaural processing strategies.     

The benefit of binaural hearing in a cocktail party: effect of location and type of interferer

The "cocktail party problem" was studied using virtual stimuli whose spatial locations were generated using anechoic head-related impulse responses from the AUDIS database [Blauert et al., J. Acoust. Soc. Am. 103, 3082 (1998)]. Speech reception thresholds (SRTs) were measured for Harvard IEEE sentences presented from the front in the presence of one, two, or three interfering sources. Four types of interferer were used: (1) other sentences spoken by the same talker, (2) time-reversed sentences of the same talker, (3) speech-spectrum shaped noise, and (4) speech-spectrum shaped noise, modulated by the temporal envelope of the sentences. Each interferer was matched to the spectrum of the target talker. Interferers were placed in several spatial configurations, either coincident with or separated from the target. Binaural advantage was derived by subtracting SRTs from listening with the "better monaural ear" from those for binaural listening. For a single interferer, there was a binaural advantage of 2-4 dB for all interferer types. For two or three interferers, the advantage was 2-4 dB for noise and speech-modulated noise, and 6-7 dB for speech and time-reversed speech. These data suggest that the benefit of binaural hearing for speech intelligibility is especially pronounced when there are multiple voiced interferers at different locations from the target, regardless of spatial configuration; measurements with fewer or with other types of interferers can underestimate this benefit.     

Binaural versus monaural loudness: supersummation of tone partially masked by noise

A series of three experiments used the method of magnitude estimation to examine binaural summation of the loudness of a 1000-Hz tone heard in the quiet and against various backgrounds of masking noise. In the quiet, binaural loudness as measured in sones, is twice monaural loudness. Two conditions of noise masking acted to increase the ratio of binaural/monaural loudness in sones above 2:1--that is, to produce supersummation. (1) When tone was presented to both ears, but masking noise to just one ear (dichotic stimulation), the loudness of the binaural tone was 30%-35% greater than the sum of the loudness of the monaural components. This increase in summation provides a suprathreshold analog to increases in threshold sensitivity observed with dichotic stimulation (masking-level differences). (2) Supersummation was also evident when tone and noise alike were presented to both ears (diotic stimulation); here, the binaural tone's loudness was 10%-25% greater than the sum of the monaural components. The increase in summation with diotic stimulation may be related to the characteristics of binaural summation of the noise masker itself.     

Pulse-rate discrimination by cochlear-implant and normal-hearing listeners with and without binaural cues

Experiment 1 measured rate discrimination of electric pulse trains by bilateral cochlear implant (CI) users, for standard rates of 100, 200, and 300 pps. In the diotic condition the pulses were presented simultaneously to the two ears. Consistent with previous results with unilateral stimulation, performance deteriorated at higher standard rates. In the signal interval of each trial in the dichotic condition, the standard rate was presented to the left ear and the (higher) signal rate was presented to the right ear; the non-signal intervals were the same as in the diotic condition. Performance in the dichotic condition was better for some listeners than in the diotic condition for standard rates of 100 and 200 pps, but not at 300 pps. It is concluded that the deterioration in rate discrimination observed for CI users at high rates cannot be alleviated by the introduction of a binaural cue, and is unlikely to be limited solely by central pitch processes. Experiment 2 performed an analogous experiment in which 300-pps acoustic pulse trains were bandpass filtered (3900-5400 Hz) and presented in a noise background to normal-hearing listeners. Unlike the results of experiment 1, performance was superior in the dichotic than in the diotic condition.     

The effect of interaural differences in envelope shape on the perceived location of sounds (L)

Users of bilateral cochlear implants and a cochlear implant combined with a contralateral hearing aid are sensitive to interaural time differences (ITDs). The way cochlear implant speech processors work and differences between modalities may result in interaural differences in shape of the temporal envelope presented to the binaural system. The effect of interaural differences in envelope shape on ITD sensitivity was investigated with normal-hearing listeners using a 4?kHz pure tone modulated with a periodic envelope with a trapezoid shape in each cycle. In one ear the onset segment of the trapezoid was transformed by a power function. No effect on the just noticeable difference in ITD was found with an interaural difference in envelope shape, but the ITD for a centered percept was significantly different across envelope shape conditions.     

Multi-microphone adaptive noise reduction strategies for coordinated stimulation in bilateral cochlear implant devices

Bilateral cochlear implant (BI-CI) recipients achieve high word recognition scores in quiet listening conditions. Still, there is a substantial drop in speech recognition performance when there is reverberation and more than one interferers. BI-CI users utilize information from just two directional microphones placed on opposite sides of the head in a so-called independent stimulation mode. To enhance the ability of BI-CI users to communicate in noise, the use of two computationally inexpensive multi-microphone adaptive noise reduction strategies exploiting information simultaneously collected by the microphones associated with two behind-the-ear (BTE) processors (one per ear) is proposed. To this end, as many as four microphones are employed (two omni-directional and two directional) in each of the two BTE processors (one per ear). In the proposed two-microphone binaural strategies, all four microphones (two behind each ear) are being used in a coordinated stimulation mode. The hypothesis is that such strategies combine spatial information from all microphones to form a better representation of the target than that made available with only a single input. Speech intelligibility is assessed in BI-CI listeners using IEEE sentences corrupted by up to three steady speech-shaped noise sources. Results indicate that multi-microphone strategies improve speech understanding in single- and multi-noise source scenarios.     

Speech segregation in rooms: monaural, binaural, and interacting effects of reverberation on target and interferer

Speech reception thresholds were measured in virtual rooms to investigate the influence of reverberation on speech intelligibility for spatially separated targets and interferers. The measurements were realized under headphones, using target sentences and noise or two-voice interferers. The room simulation allowed variation of the absorption coefficient of the room surfaces independently for target and interferer. The direct-to-reverberant ratio and interaural coherence of sources were also varied independently by considering binaural and diotic listening. The main effect of reverberation on the interferer was binaural and mediated by the coherence, in agreement with binaural unmasking theories. It appeared at lower reverberation levels than the effect of reverberation on the target, which was mainly monaural and associated with the direct-to-reverberant ratio, and could be explained by the loss of amplitude modulation in the reverberant speech signals. This effect was slightly smaller when listening binaurally. Reverberation might also be responsible for a disruption of the mechanism by which the auditory system exploits fundamental frequency differences to segregate competing voices, and a disruption of the "listening in the gaps" associated with speech interferers. These disruptions may explain an interaction observed between the effects of reverberation on the targets and two-voice interferers.     

Binaural modulation masking

Modulation thresholds were measured in three subjects for a sinusoidally amplitude-modulated (SAM) wideband noise (the signal) in the presence of a second amplitude-modulated wideband noise (the masker). In monaural conditions (Mm-Sm) masker and signal were presented to only one ear; in binaural conditions (M0-S pi) the masker was presented diotically while the phase of modulation of the SAM noise signal was inverted in one ear relative to the other. In experiment 1 masker modulation frequency (fm) was fixed at 16 Hz, and signal modulation frequency (fs) was varied from 2-512 Hz. For monaural presentation, masking generally decreased as fs diverged from fm, although there was a secondary increase in masking for very low signal modulation frequencies, as reported previously [Bacon and Grantham, J. Acoust. Soc. Am. 85, 2575-2580 (1989)]. The binaural masking patterns did not show this low-frequency upturn: binaural thresholds continued to improve as fs decreased from 16 to 2 Hz. Thus, comparing masked monaural and masked binaural thresholds, there was an average binaural advantage, or masking-level difference (MLD) of 9.4 dB at fs = 2 Hz and 5.3 dB at fs = 4 Hz. In addition, there were positive MLDs for the on-frequency condition (fm = fs = 16 Hz: average MLD = 4.4 dB) and for the highest signal frequency tested (fs = 512 Hz: average MLD = 7.3 dB). In experiment 2 the signal was a SAM noise (fs = 16 Hz), and the masker was a wideband noise, amplitude-modulated by a narrow band of noise centered at fs. There was no effect on monaural or binaural thresholds as masker modulator bandwidth was varied from 4 to 20 Hz (the average MLD remained constant at 8.0 dB), which suggests that the observed "tuning" for modulation may be based on temporal pattern discrimination and not on a critical-band-like filtering mechanism. In a final condition the masker modulator was a 10-Hz-wide band of noise centered at the 64-Hz signal modulation frequency. The average MLD in this case was 7.4 dB. The results are discussed in terms of various binaural capacities that probably play a role in binaural release from modulation masking, including detection of varying interaural intensity differences (IIDs) and discrimination of interaural correlation.     

Speech-reception threshold in noise with one and two hearing aids

The binaural free-field speech-reception threshold (SRT) in 70-dBA noise was measured with conversational sentences for 24 hearing-impaired subjects without hearing aids, with a hearing aid left, right, and left plus right, respectively. The sentences were always presented in front of the listener and the interfering noise, with a spectrum equal to the long-term average spectrum of the sentences, was presented either frontally, from the right, or from the left side. For subjects with only moderate hearing loss, PTA (average air-conduction hearing level at 500, 1000, and 2000 Hz) less than 50 dB, the SRT in 70-dBA noise in both ears is determined by the signal-to-noise ratio even if only one hearing aid is used. For larger hearing losses the SRT appears to be partly determined by the absolute threshold. In conditions with a high noise level relative to the absolute threshold, in which case for both ears the SRT is determined by the signal-to-noise ratio, a second hearing aid, just as a monaural hearing aid, generally does not improve the SRT. However, in the case of a high hearing level, or a low noise level, in which a monaural hearing aid is profitable, the use of two hearing aids is even more profitable. In a separate experiment, acoustic head shadow was measured at the entrance of the ear canal and at the microphone location of a hearing aid. It appeared that, for a lateral noise source and speech frontal, the microphone position of behind-the-ear hearing aids has a negative effect on the signal-to-noise ratio of 2-3 dB.     

Binaural speech unmasking and localization in noise with bilateral cochlear implants using envelope and fine-timing based strategies

Four adult bilateral cochlear implant users, with good open-set sentence recognition, were tested with three different sound coding strategies for binaural speech unmasking and their ability to localize 100 and 500 Hz click trains in noise. Two of the strategies tested were envelope-based strategies that are clinically widely used. The third was a research strategy that additionally preserved fine-timing cues at low frequencies. Speech reception thresholds were determined in diotic noise for diotic and interaurally time-delayed speech using direct audio input to a bilateral research processor. Localization in noise was assessed in the free field. Overall results, for both speech and localization tests, were similar with all three strategies. None provided a binaural speech unmasking advantage due to the application of 700 micros interaural time delay to the speech signal, and localization results showed similar response patterns across strategies that were well accounted for by the use of broadband interaural level cues. The data from both experiments combined indicate that, in contrast to normal hearing, timing cues available from natural head-width delays do not offer binaural advantages with present methods of electrical stimulation, even when fine-timing cues are explicitly coded.     

Studies of interaural attenuation to investigate the validity of a dichotic difference tone response recorded from the inferior colliculus in the chinchilla

In a previous paper (Arnold and Burkard, 1998) a dichotic f2-f1 difference tone (DT) auditory evoked potential from the chinchilla inferior colliculus (IC) was measured while presenting f1 (2000 Hz) to one ear and f2 (2100 Hz) to the other ear. This measurement paradigm could be used as a means to study binaural processing in an unanesthetized animal model. However, it is possible that this response is actually generated peripherally, as a result of acoustic crossover. The purpose of the present set of experiments was to investigate whether the dichotic DT is a true binaural phenomenon. Recordings were made from chronically implanted IC electrodes in unanesthetized, monaural chinchillas (left cochlea destroyed). In experiment 1, interaural attenuation (IA) was measured in two ways. First, IA was measured by comparing IC evoked potential thresholds obtained when stimulating the normal right ear and the dead left ear, using tone bursts (0.5-8 kHz). Mean values of interaural attenuation ranged from 50-65 dB across frequency (55 dB at 2000 Hz). Next, the DT was measured monaurally using f1 = 2000 and f2 = 2100 (L1 = L2). By comparing the mean DT input/output functions for monaural stimulation of the right and left ears, a mean value of IA for the tonal pair was estimated (approximately 69 dB). In experiment 2, the DT was measured with right monaural stimulation, while varying the relative levels of the primaries. A small DT could be seen with primary levels up to 30 dB apart, but not for greater level differences. Differences substantially greater than 30 dB would be expected in the crossover situation based upon IA. In experiment 3, the stimuli were presented dichotically (f1 to right ear, f2 to left ear and vice versa, L1 = L2) to determine whether acoustic crosstalk to the normal right ear would generate a DT. No DT was reliably observed in this condition. Taken together, these results suggest that the dichotic DT is a true binaural phenomenon, and not simply attributable to acoustic crossover.     

Late-onset auditory deprivation: effects of monaural versus binaural hearing aids

Performance on tests of pure-tone thresholds, speech-recognition thresholds, and speech-recognition scores for the two ears of each subject were evaluated in two groups of adults with bilateral hearing losses. One group was composed of individuals fitted with binaural hearing aids, and the other group included persons with monaural hearing aids. Performance prior to the use of hearing aids was compared to performance after 4-5 years of hearing aid use in order to determine whether the unaided ear would show effects of auditory deprivation. There were no differences over time for pure-tone thresholds or speech-recognition thresholds for both ears of both groups. Nevertheless, the results revealed that the speech-recognition difference scores of the binaurally fitted subjects remained stable over time whereas they increased for the monaurally fitted subjects. The findings reveal an auditory deprivation effect for the unfitted ears of the subjects with monaural hearing aids.     

Binaural sensitivity as a function of interaural electrode position with a bilateral cochlear implant user

Experiments were conducted with a single, bilateral cochlear implant user to examine interaural level and time-delay cues that putatively underlie the design and efficacy of bilateral implant systems. The subject's two implants were of different types but custom equipment allowed presentation of controlled bilateral stimuli, particularly those with specified interaural time difference (ITD) and interaural level difference (ILD) cues. A lateralization task was used to measure the effect of these cues on the perceived location of the sensations elicited. For trains of fixed-amplitude, biphasic current pulses at 100 pps, the subject demonstrated sensitivity to an ITD of 300 micros, providing evidence of access to binaural information. The choice of bilateral electrode pair greatly influenced ITD sensitivity, suggesting that electrode pairings are likely to be an important consideration in the effort to provide binaural advantages. The selection of bilateral electrode pairs showing sensitivity to ITD was partially aided by comparisons of the pitch elicited by individual electrodes in each ear (when stimulated alone with fixed-amplitude current pulses at 813 pps): specifically, interaural electrodes with similar pitches were more likely (but not certain) to show ITD sensitivity. Significant changes in lateral position occurred with specific electrode pairs. With five bilateral electrode pairs of 14 tested, ITDs of 300 and 600 micros moved an auditory image significantly from right to left. With these same pairs, ILD changes of approximately 11% of the dynamic range (in microApp) moved an auditory image from the far left to the far right-significantly farther than the nine pairs not showing significant ITD sensitivity. However, even these nine pairs did show response changes as a function of the interaural (or confounding monaural) level cue. Overall, insofar as the access to bilateral cues demonstrated herein generalizes to other subjects, it provides hope that the normal binaural advantages for speech recognition and sound localization can be made available to bilateral implant users.     

Speech intelligibility and localization in a multi-source environment.

Natural environments typically contain sound sources other than the source of interest that may interfere with the ability of listeners to extract information about the primary source. Studies of speech intelligibility and localization by normal-hearing listeners in the presence of competing speech are reported on in this work. One, two or three competing sentences [IEEE Trans. Audio Electroacoust. 17(3), 225-246 (1969)] were presented from various locations in the horizontal plane in several spatial configurations relative to a target sentence. Target and competing sentences were spoken by the same male talker and at the same level. All experiments were conducted both in an actual sound field and in a virtual sound field. In the virtual sound field, both binaural and monaural conditions were tested. In the speech intelligibility experiment, there were significant improvements in performance when the target and competing sentences were spatially separated. Performance was similar in the actual sound-field and virtual sound-field binaural listening conditions for speech intelligibility. Although most of these improvements are evident monaurally when using the better ear, binaural listening was necessary for large improvements in some situations. In the localization experiment, target source identification was measured in a seven-alternative absolute identification paradigm with the same competing sentence configurations as for the speech study. Performance in the localization experiment was significantly better in the actual sound-field than in the virtual sound-field binaural listening conditions. Under binaural conditions, localization performance was very good, even in the presence of three competing sentences. Under monaural conditions, performance was much worse. For the localization experiment, there was no significant effect of the number or configuration of the competing sentences tested. For these experiments, the performance in the speech intelligibility experiment was not limited by localization ability.     

The role of monaural frequency selectivity in binaural analysis

The relation between the monaural critical band and binaural analysis was examined using an NoSm MLD paradigm, in order to resolve ambiguities about the width of the masking spectrum important for binaural detection. A 500-Hz pure-tone signal was presented with a 600-Hz-wide band of masking noise to the signal ear. Bands of noise ranging in width from 25 to 600 Hz, or noise notches (imposed on a 600-Hz-wide band centered on the signal frequency) ranging in width from 0 to 600 Hz were presented to the nonsignal ear. All noise bands and notches were centered on 500 Hz, the frequency of the signal. The effects of varying bandwidth were radically different from those of varying notchwidth: the MLD changed from zero to approximately 8 dB over a bandwidth range of 400 Hz; for notchwidths, however, the MLD changed 8 dB over a range of only 50 Hz. The results support an interpretation that the fine frequency selectivity of monaural analysis is preserved in peripheral binaural interaction, but that a relatively wide frequency range of critical bands is scanned at a later stage of binaural processing. It was suggested that the wide spectral range of binaural analysis may provide a background against which binaural differences due to the signal are detected.