Temporal pitch perception at high rates in cochlear implants

A recent study reported that a group of Med-El COMBI 40+CI (cochlear implant) users could, in a forced-choice task, detect changes in the rate of a pulse train for rates higher than the 300 pps "upper limit" commonly reported in the literature [Kong, Y.-Y., et al. (2009). J. Acoust. Soc. Am. 125, 1649-1657]. The present study further investigated the upper limit of temporal pitch in the same group of CI users on three tasks [pitch ranking, rate discrimination, and multidimensional scaling (MDS)]. The patterns of results were consistent across the three tasks and all subjects could follow rate changes above 300 pps. Two subjects showed exceptional ability to follow temporal pitch change up to about 900 pps. Results from the MDS study indicated that, for the two listeners tested, changes in pulse rate over the range of 500-840 pps were perceived along a perceptual dimension that was orthogonal to the place of excitation. Some subjects showed a temporal pitch reversal at rates beyond their upper limit of pitch and some showed a reversal within a small range of rates below the upper limit. These results are discussed in relation to the possible neural bases for temporal pitch processing at high rates.     

The performance of different synthesis signals in acoustic models of cochlear implants

Synthesis (carrier) signals in acoustic models embody assumptions about perception of auditory electric stimulation. This study compared speech intelligibility of consonants and vowels processed through a set of nine acoustic models that used Spectral Peak (SPEAK) and Advanced Combination Encoder (ACE)-like speech processing, using synthesis signals which were representative of signals used previously in acoustic models as well as two new ones. Performance of the synthesis signals was determined in terms of correspondence with cochlear implant (CI) listener results for 12 attributes of phoneme perception (consonant and vowel recognition; F1, F2, and duration information transmission for vowels; voicing, manner, place of articulation, affrication, burst, nasality, and amplitude envelope information transmission for consonants) using four measures of performance. Modulated synthesis signals produced the best correspondence with CI consonant intelligibility, while sinusoids, narrow noise bands, and varying noise bands produced the best correspondence with CI vowel intelligibility. The signals that performed best overall (in terms of correspondence with both vowel and consonant attributes) were modulated and unmodulated noise bands of varying bandwidth that corresponded to a linearly varying excitation width of 0.4 mm at the apical to 8 mm at the basal channels.     

Enhancement of temporal cues to pitch in cochlear implants: effects on pitch ranking

The abilities to hear changes in pitch for sung vowels and understand speech using an experimental sound coding strategy (eTone) that enhanced coding of temporal fundamental frequency (F0) information were tested in six cochlear implant users, and compared with performance using their clinical (ACE) strategy. In addition, rate- and modulation rate-pitch difference limens (DLs) were measured using synthetic stimuli with F0s below 300 Hz to determine psychophysical abilities of each subject and to provide experience in attending to rate cues for the judgment of pitch. Sung-vowel pitch ranking tests for stimuli separated by three semitones presented across an F0 range of one octave (139-277 Hz) showed a significant benefit for the experimental strategy compared to ACE. Average d-prime (d') values for eTone (d' = 1.05) were approximately three time larger than for ACE (d' = 0.35). Similar scores for both strategies in the speech recognition tests showed that coding of segmental speech information by the experimental strategy was not degraded. Average F0 DLs were consistent with results from previous studies and for all subjects were less than or equal to approximately three semitones for F0s of 125 and 200?Hz.     

Vowel recognition via cochlear implants and noise vocoders: effects of formant movement and duration

Previous work has demonstrated that normal-hearing individuals use fine-grained phonetic variation, such as formant movement and duration, when recognizing English vowels. The present study investigated whether these cues are used by adult postlingually deafened cochlear implant users, and normal-hearing individuals listening to noise-vocoder simulations of cochlear implant processing. In Experiment 1, subjects gave forced-choice identification judgments for recordings of vowels that were signal processed to remove formant movement and/or equate vowel duration. In Experiment 2, a goodness-optimization procedure was used to create perceptual vowel space maps (i.e., best exemplars within a vowel quadrilateral) that included F1, F2, formant movement, and duration. The results demonstrated that both cochlear implant users and normal-hearing individuals use formant movement and duration cues when recognizing English vowels. Moreover, both listener groups used these cues to the same extent, suggesting that postlingually deafened cochlear implant users have category representations for vowels that are similar to those of normal-hearing individuals.     

Effects of stimulation configurations on place pitch discrimination in cochlear implants

The present study aimed to examine the effect of electrode configuration, specifically monopolar (MP) or bipolar (BP) stimulation, on place pitch discrimination in cochlear implants (CIs). Twelve subjects implanted with the Nucleus Freedom device were presented with various pairs of stimulation across the electrode array, with varying degrees of distance between stimulation sites, and asked to judge the higher of the two in pitch. Each pair was presented either in the same mode or in different modes of stimulation for the within-mode or across-mode condition, respectively, at least 20 times. The result of the within-mode condition revealed that subjects, on average, were able to discriminate pitches significantly better in MP than in BP, with the sensitivity index (d') for adjacent channels of 1.2 for MP and 0.8 for BP. The result of the across-mode condition revealed that while individual variability existed, there was a strong tendency for CI subjects to perceive a higher pitch in BP stimulation than in MP for a similar site of stimulation. In other words, an MP channel needed to be shifted in a basal direction by as much as two electrodes on average to elicit a pitch comparable to that of a BP channel.     

Features of stimulation affecting tonal-speech perception: implications for cochlear prostheses

Tone languages differ from English in that the pitch pattern of a single-syllable word conveys lexical meaning. In the present study, dependence of tonal-speech perception on features of the stimulation was examined using an acoustic simulation of a CIS-type speech-processing strategy for cochlear prostheses. Contributions of spectral features of the speech signals were assessed by varying the number of filter bands, while contributions of temporal envelope features were assessed by varying the low-pass cutoff frequency used for extracting the amplitude envelopes. Ten normal-hearing native Mandarin Chinese speakers were tested. When the low-pass cutoff frequency was fixed at 512 Hz, consonant, vowel, and sentence recognition improved as a function of the number of channels and reached plateau at 4 to 6 channels. Subjective judgments of sound quality continued to improve as the number of channels increased to 12, the highest number tested. Tone recognition, i.e., recognition of the four Mandarin tone patterns, depended on both the number of channels and the low-pass cutoff frequency. The trade-off between the temporal and spectral cues for tone recognition indicates that temporal cues can compensate for diminished spectral cues for tone recognition and vice versa. An additional tone recognition experiment using syllables of equal duration showed a marked decrease in performance, indicating that duration cues contribute to tone recognition. A third experiment showed that recognition of processed FM patterns that mimic Mandarin tone patterns was poor when temporal envelope and duration cues were removed.     

Patterns of phoneme perception errors by listeners with cochlear implants as a function of overall speech perception ability

Many studies have noted great variability in speech perception ability among postlingually deafened adults with cochlear implants. This study examined phoneme misperceptions for 30 cochlear implant listeners using either the Nucleus-22 or Clarion version 1.2 device to examine whether listeners with better overall speech perception differed qualitatively from poorer listeners in their perception of vowel and consonant features. In the first analysis, simple regressions were used to predict the mean percent-correct scores for consonants and vowels for the better group of listeners from those of the poorer group. A strong relationship between the two groups was found for consonant identification, and a weak, nonsignificant relationship was found for vowel identification. In the second analysis, it was found that less information was transmitted for consonant and vowel features to the poorer listeners than to the better listeners; however, the pattern of information transmission was similar across groups. Taken together, results suggest that the performance difference between the two groups is primarily quantitative. The results underscore the importance of examining individuals' perception of individual phoneme features when attempting to relate speech perception to other predictor variables.     

Behavioral and physiological correlates of temporal pitch perception in electric and acoustic hearing

In the "4-6" condition of experiment 1, normal-hearing (NH) listeners compared the pitch of a bandpass-filtered pulse train, whose inter-pulse intervals (IPIs) alternated between 4 and 6 ms, to that of isochronous pulse trains. Consistent with previous results obtained at a lower signal level, the pitch of the 4-6 stimulus corresponded to that of an isochronous pulse train having a period of 5.7 ms-longer than the mean IPI of 5 ms. In other conditions the IPI alternated between 3.5-5.5 and 4.5-6.5 ms. Experiment 2 was similar but presented electric pulse trains to one channel of a cochlear implant. In both cases, as overall IPI increased, the pitch of the alternating-interval stimulus approached that of an isochronous train having a period equal to the mean IPI. Experiment 3 measured compound action potentials (CAPs) to alternating-interval stimuli in guinea pigs and in NH listeners. The CAPs to pulses occurring after 4-ms intervals were smaller than responses to pulses occurring after 6-ms intervals, resulting in a modulated pattern that was independent of overall level. The results are compared to the predictions of a simple model incorporating auditory-nerve (AN) refractoriness, and where pitch is estimated from first-order intervals in the AN response.     

Effect of cochlear implants on children's perception and production of speech prosody

Japanese 5- to 13-yr-olds who used cochlear implants (CIs) and a comparison group of normally hearing (NH) Japanese children were tested on their perception and production of speech prosody. For the perception task, they were required to judge whether semantically neutral utterances that were normalized for amplitude were spoken in a happy, sad, or angry manner. The performance of NH children was error-free. By contrast, child CI users performed well below ceiling but above chance levels on happy- and sad-sounding utterances but not on angry-sounding utterances. For the production task, children were required to imitate stereotyped Japanese utterances expressing disappointment and surprise as well as culturally typically representations of crow and cat sounds. NH 5- and 6-year-olds produced significantly poorer imitations than older hearing children, but age was unrelated to the imitation quality of child CI users. Overall, child CI user's imitations were significantly poorer than those of NH children, but they did not differ significantly from the imitations of the youngest NH group. Moreover, there was a robust correlation between the performance of child CI users on the perception and production tasks; this implies that difficulties with prosodic perception underlie their difficulties with prosodic imitation.     

Pitch shifts for complex tones with unresolved harmonics and the implications for models of pitch perception

Complex tone bursts were bandpass filtered, 22nd-30th harmonic, to produce waveforms with five regularly occurring envelope peaks ("pitch pulses") that evoked pitches associated with their repetition period. Two such tone bursts were presented sequentially and separated by an interpulse interval (IPI). When the IPI was varied, the pitch of the whole sequence was shifted by between +2% and -5%. When the IPI was greater than one period, little effect was seen. This is consistent with a pitch mechanism employing a long integration time for continuous stimuli that resets in response to temporal discontinuities of greater than about one period of the waveform. Similar pitch shifts were observed for fundamental frequencies from 100 to 250 Hz. The pitch shifts depended on the IPI duration relative to the period of the complex, not on the absolute IPI duration. The pitch shifts are inconsistent with the autocorrelation model of Meddis and O'Mard [J. Acoust. Soc. Am. 102, 1811-1820 (1997)], although a modified version of the weighted mean-interval model of Carlyon et al. [J. Acoust. Soc. Am. 112, 621-633 (2002)] was successful. The pitch shifts suggest that, when two pulses occur close together, one of the pulses is ignored on a probabilistic basis.     

Speech recognition in noise as a function of the number of spectral channels: comparison of acoustic hearing and cochlear implants

Speech recognition was measured as a function of spectral resolution (number of spectral channels) and speech-to-noise ratio in normal-hearing (NH) and cochlear-implant (CI) listeners. Vowel, consonant, word, and sentence recognition were measured in five normal-hearing listeners, ten listeners with the Nucleus-22 cochlear implant, and nine listeners with the Advanced Bionics Clarion cochlear implant. Recognition was measured as a function of the number of spectral channels (noise bands or electrodes) at signal-to-noise ratios of + 15, + 10, +5, 0 dB, and in quiet. Performance with three different speech processing strategies (SPEAK, CIS, and SAS) was similar across all conditions, and improved as the number of electrodes increased (up to seven or eight) for all conditions. For all noise levels, vowel and consonant recognition with the SPEAK speech processor did not improve with more than seven electrodes, while for normal-hearing listeners, performance continued to increase up to at least 20 channels. Speech recognition on more difficult speech materials (word and sentence recognition) showed a marginally significant increase in Nucleus-22 listeners from seven to ten electrodes. The average implant score on all processing strategies was poorer than scores of NH listeners with similar processing. However, the best CI scores were similar to the normal-hearing scores for that condition (up to seven channels). CI listeners with the highest performance level increased in performance as the number of electrodes increased up to seven, while CI listeners with low levels of speech recognition did not increase in performance as the number of electrodes was increased beyond four. These results quantify the effect of number of spectral channels on speech recognition in noise and demonstrate that most CI subjects are not able to fully utilize the spectral information provided by the number of electrodes used in their implant.     

Effects of the salience of pitch and periodicity information on the intelligibility of four-channel vocoded speech: implications for cochlear implants

Recent simulations of continuous interleaved sampling (CIS) cochlear implant speech processors have used acoustic stimulation that provides only weak cues to pitch, periodicity, and aperiodicity, although these are regarded as important perceptual factors of speech. Four-channel vocoders simulating CIS processors have been constructed, in which the salience of speech-derived periodicity and pitch information was manipulated. The highest salience of pitch and periodicity was provided by an explicit encoding, using a pulse carrier following fundamental frequency for voiced speech, and a noise carrier during voiceless speech. Other processors included noise-excited vocoders with envelope cutoff frequencies of 32 and 400 Hz. The use of a pulse carrier following fundamental frequency gave substantially higher performance in identification of frequency glides than did vocoders using envelope-modulated noise carriers. The perception of consonant voicing information was improved by processors that preserved periodicity, and connected discourse tracking rates were slightly faster with noise carriers modulated by envelopes with a cutoff frequency of 400 Hz compared to 32 Hz. However, consonant and vowel identification, sentence intelligibility, and connected discourse tracking rates were generally similar through all of the processors. For these speech tasks, pitch and periodicity beyond the weak information available from 400 Hz envelope-modulated noise did not contribute substantially to performance.     

Effect of stimulus level and place of stimulation on temporal pitch perception by cochlear implant users

Three experiments studied the effect of pulse rate on temporal pitch perception by cochlear implant users. Experiment 1 measured rate discrimination for pulse trains presented in bipolar mode to either an apical, middle, or basal electrode and for standard rates of 100 and 200 pps. In each block of trials the signals could have a level of -0.35, 0, or +0.35 dB re the standard, and performance for each signal level was recorded separately. Signal level affected performance for just over half of the combinations of subject, electrode, and standard rate studied. Performance was usually, but not always, better at the higher signal level. Experiment 2 showed that, for a given subject and condition, the direction of the effect was similar in monopolar and bipolar mode. Experiment 3 employed a pitch comparison procedure without feedback, and showed that the signal levels in experiment 1 that produced the best performance for a given subject and condition also led to the signal having a higher pitch. It is concluded that small level differences can have a robust and substantial effect on pitch judgments and argue that these effects are not entirely due to response biases or to co-variation of place-of-excitation with level.     

Faciliation of Mandarin tone perception by visual speech in clear and degraded audio: implications for cochlear implants

Cochlear implant (CI) users in tone language environments report great difficulty in perceiving lexical tone. This study investigated the augmentation of simulated cochlear implant audio by visual (facial) speech information for tone. Native speakers of Mandarin and Australian English were asked to discriminate between minimal pairs of Mandarin tones in five conditions: Auditory-Only, Auditory-Visual, CI-simulated Auditory-Only, CI-simulated Auditory-Visual, and Visual-Only (silent video). Discrimination in CI-simulated audio conditions was poor compared with normal audio, and varied according to tone pair, with tone pairs with strong non-F0 cues discriminated the most easily. The availability of visual speech information also improved discrimination in the CI-simulated audio conditions, particularly on tone pairs with strong durational cues. In the silent Visual-Only condition, both Mandarin and Australian English speakers discriminated tones above chance levels. Interestingly, tone-nai?ve listeners outperformed native listeners in the Visual-Only condition, suggesting firstly that visual speech information for tone is available, and may in fact be under-used by normal-hearing tone language perceivers, and secondly that the perception of such information may be language-general, rather than the product of language-specific learning. This may find application in the development of methods to improve tone perception in CI users in tone language environments.     

An analysis of the effects of electrical field interaction with an acoustic model of cochlear implants

Electrical field interaction caused by current spread in a cochlear implant was modeled in an explicit way in an acoustic model (the SPREAD model) presented to six listeners with normal hearing. The typical processing of cochlear implants was modeled more closely than in traditional acoustic models by careful selection of parameters related to current spread or parameters that could amplify the electrical field interactions caused by current spread. These parameters were the insertion depth, electrode spacing, electrical dynamic range, and dynamic range compression function. The hypothesis was that current spread could account for the asymptote in performance in speech intelligibility experiments observed at around seven stimulation channels in a number of cochlear implant studies. Speech intelligibility for sentences, vowels, and consonants at three noise levels (SNR of +15 dB, +10 dB, and +5 dB) was measured as a function of the number of spectral channels (4, 7, and 16). The SPREAD model appears to explain the asymptote in speech intelligibility at seven channels for all noise levels for all speech material used in this study. It is shown that the compressive amplitude mapping used in cochlear implants can have a detrimental effect on the number of effective channels.     

Subspace algorithms for noise reduction in cochlear implants

A single-channel algorithm is proposed for noise reduction in cochlear implants. The proposed algorithm is based on subspace principles and projects the noisy speech vector onto "signal" and "noise" subspaces. An estimate of the clean signal is made by retaining only the components in the signal subspace. The performance of the subspace reduction algorithm is evaluated using 14 subjects wearing the Clarion device. Results indicated that the subspace algorithm produced significant improvements in sentence recognition scores compared to the subjects' daily strategy, at least in stationary noise. Further work is needed to extend the subspace algorithm to nonstationary noise environments.     

Enhancement of temporal periodicity cues in cochlear implants: effects on prosodic perception and vowel identification

Standard continuous interleaved sampling processing, and a modified processing strategy designed to enhance temporal cues to voice pitch, were compared on tests of intonation perception, and vowel perception, both in implant users and in acoustic simulations. In standard processing, 400 Hz low-pass envelopes modulated either pulse trains (implant users) or noise carriers (simulations). In the modified strategy, slow-rate envelope modulations, which convey dynamic spectral variation crucial for speech understanding, were extracted by low-pass filtering (32 Hz). In addition, during voiced speech, higher-rate temporal modulation in each channel was provided by 100% amplitude-modulation by a sawtooth-like wave form whose periodicity followed the fundamental frequency (F0) of the input. Channel levels were determined by the product of the lower- and higher-rate modulation components. Both in acoustic simulations and in implant users, the ability to use intonation information to identify sentences as question or statement was significantly better with modified processing. However, while there was no difference in vowel recognition in the acoustic simulation, implant users performed worse with modified processing both in vowel recognition and in formant frequency discrimination. It appears that, while enhancing pitch perception, modified processing harmed the transmission of spectral information.     

Infant pitch perception: evidence for responding to pitch categories and the missing fundamental

While numerous studies on infant perception have demonstrated the infant's ability to discriminate sounds having different frequencies, little research has evaluated more sophisticated pitch perception abilities such as perceptual constancy and perception of the missing fundamental. In the present study 7-8-month-old infants demonstrated the ability to discriminate harmonic complexes from two pitch categories that differed in pitch by approximately 20% (e.g., 160 vs 200 Hz). Using a visually reinforced conditioned head-turning paradigm, a number of spectrally different tonal complexes that contained varying harmonic components but signaled the same two pitch categories were presented. After learning the basic pitch discrimination, the same infants learned to categorize spectrally different tonal complexes according to the pitches signaled by their fundamental frequencies. That is, the infants showed evidence of perceptual constancy for the pitch of harmonic complexes. Finally, infants heard tonal complexes that signaled the same pitch categories but for which the fundamental frequency was removed. Infants were still able to categorize the harmonic complexes according to their pitch categories. These results suggest that by 7 months of age infants show fairly sophisticated pitch perception abilities similar to those demonstrated by adults.     

Gap detection as a measure of electrode interaction in cochlear implants.

Gap detection thresholds were measured as an indication of the amount of interaction between electrodes in a cochlear implant. The hypothesis in this study was as follows: when the two stimuli that bound the gap stimulate the same electrode, and thus the same neural population, the gap detection threshold will be short. As two stimuli are presented to two electrodes that are more widely separated, the amount of neural overlap of the two stimuli decreases, the stimuli sound more dissimilar, and the gap thresholds increase. Gap detection thresholds can thus be used to infer the amount of overlap in neural populations stimulated by two electrodes. Three users of the Nucleus cochlear implant participated in this study. Gap detection thresholds were measured as a function of the distance between the two electrode pairs and as a function of the spacing between the two electrodes of a bipolar pair (i.e., using different modes of stimulation). The results indicate that measuring gap detection thresholds may provide an estimate of the amount of electrode interaction. Gap detection thresholds were a function of the physical separation of the electrode pairs used for the two stimuli that bound the gap. Lower gap thresholds were observed when the two electrode pairs were closely spaced, and gap thresholds increased as the separation increased, resulting in a "psychophysical tuning curve" as a function of electrode separation. The sharpness of tuning varied across subjects, and for the three subjects in this study, the tuning was generally sharper for the subjects with better speech recognition. The data also indicate that increasing the separation between active and reference electrodes has limited effect on spatial selectivity (or tuning) as measured perceptually.