Speech of cochlear implant patients: a longitudinal study of vowel production.

Acoustic parameters were measured for vowels spoken in /hVd/ context by four postlingually deafened recipients of multichannel (Ineraid) cochlear implants. Three of the subjects became totally deaf in adulthood after varying periods of partial hearing loss; the fourth became totally deaf at age four. The subjects received different degrees of perceptual benefit from the prosthesis. Recordings were made before, and at intervals following speech processor activation. The measured parameters included F1, F2, F0, SPL, duration, and amplitude difference between the first two harmonic peaks in the log magnitude spectrum (H 1-H2). Numerous changes in parameter values were observed from pre- to post-implant, with differences among subjects. Many changes, but not all, were in the direction of normative data, and most changes were consistent with hypotheses about relations among the parameters. Some of the changes tended to enhance phonemic contrasts; others had the opposite effect. For three subjects, H 1-H2 changed in a direction consistent with measurements of their average air flow when reading; that relation was more complex for the fourth subject. The results are interpreted with respect to: characteristics of the individual subjects, including vowel identification scores; mechanical interactions among glottal and supraglottal articulations; and hypotheses about the role of auditory feedback in the control of speech production. Almost all the observed differences could be attributed to changes in the average settings of speaking rate, F0 and SPL, which presumably can be perceived without the need for spectral place information. Some observed F2 realignment may be attributable to the reception of spectral cues.     

Time course of speech changes in response to unanticipated short-term changes in hearing state

The timing of changes in parameters of speech production was investigated in six cochlear implant users by switching their implant microphones off and on a number of times in a single experimental session. The subjects repeated four short, two-word utterances, /dV1n#SV2d/ (S = /s/ or /S/), in quasi-random order. The changes between hearing and nonhearing states were introduced by a voice-activated switch at V1 onset. "Postural" measures were made of vowel sound pressure level (SPL), duration, F0; contrast measures were made of vowel separation (distance between pair members in the formant plane) and sibilant separation (difference in spectral means). Changes in parameter values were averaged over multiple utterances, lined up with respect to the switch. No matter whether prosthetic hearing was blocked or restored, contrast measures for vowels and sibilants did not change systematically. Some changes in duration, SPL and F0 were observed during the vowel within which hearing state was changed, V1, as well as during V2 and subsequent utterance repetitions. Thus, sound segment contrasts appear to be controlled differently from the postural parameters of speaking rate and average SPL and F0. These findings are interpreted in terms of the function of hypothesized feedback and feedforward mechanisms for speech motor control.     

Effects of auditory feedback on fricatives produced by cochlear-implanted adults and children: acoustic and perceptual evidence

Acoustic analyses and perception experiments were conducted to determine the effects of brief deprivation of auditory feedback on fricatives produced by cochlear implant users. The words /si/ and /Si/ were recorded by four children and four adults with their cochlear implant speech processor turned on or off. In the processor-off condition, word durations increased significantly for a majority of talkers. These increases were greater for children compared to adults, suggesting that children may rely on auditory feedback to a greater extent than adults. Significant differences in spectral measures of /S/ were found between processor-on and processor-off conditions for two of the four children and for one of the four adults. These talkers also demonstrated a larger /s/-/S/ contrast in centroid values compared to the other talkers within their respective groups. This finding may indicate that talkers who produce fine spectral distinctions are able to perceive these distinctions through their implants and to use this feedback to fine tune their speech. Two listening experiments provided evidence that some of the acoustic changes were perceptible to normal-hearing listeners. Taken together, these experiments indicate that for certain cochlear-implant users the brief absence of auditory feedback may lead to perceptible modifications in fricative consonants.     

The effect of partially restored hearing on speech production of postlingually deafened adults with multichannel cochlear implants

The effect of auditory feedback on speech production was investigated in five postlingually deafened adults implanted with the 22-channel Nucleus device. Changes in speech production were measured before implant and 1, 6, and 24 months postimplant. Acoustic measurements included: F1 and F2 of vowels in word-in-isolation and word-in-sentence context, voice-onset-time (VOT), spectral range of sibilants, fundamental frequency (F0) of word-in-isolation and word-in-sentence context, and word and sentence duration. Perceptual ratings of speech quality were done by ten listeners. The significant changes after cochlear implantation included: a decrease of F0, word and sentence duration, and F1 values, and an increase of voiced plosives' voicing lead (from positive to negative VOT values) and fricatives' spectral range. Significant changes occurred until 2 years postimplant when most measured values fell within Hebrew norms. Listeners were found to be sensitive to the acoustic changes in the speech from preimplant to 1, 6, and 24 months postimplant. Results suggest that when hearing is restored in postlingually deafened adults, calibration of speech is not immediate and occurs over time depending on the age-at-onset of deafness, years of deafness, and perception skills. The results also concur with hypothesis that the observed changes of some speech parameters are an indirect consequence of intentional changes in other articulatory parameters.     

Effect of different types of auditory stimulation on vowel formant frequencies in multichannel cochlear implant users

Two experiments investigating the effects of auditory stimulation delivered via a Nucleus multichannel cochlear implant upon vowel production in adventitiously deafened adult speakers are reported. The first experiment contrasts vowel formant frequencies produced without auditory stimulation (implant processor OFF) to those produced with auditory stimulation (processor ON). Significant shifts in second formant frequencies were observed for intermediate vowels produced without auditory stimulation; however, no significant shifts were observed for the point vowels. Higher first formant frequencies occurred in five of eight vowels when the processor was turned ON versus OFF. A second experiment contrasted productions of the word "head" produced with a FULL map, OFF condition, and a SINGLE channel condition that restricted the amount of auditory information received by the subjects. This experiment revealed significant shifts in second formant frequencies between FULL map utterances and the other conditions. No significant differences in second formant frequencies were observed between SINGLE channel and OFF conditions. These data suggest auditory feedback information may be used to adjust the articulation of some speech sounds.     

A comparison of three speech coding strategies using an acoustic model of a cochlear implant

Three alternative speech coding strategies suitable for use with cochlear implants were compared in a study of three normally hearing subjects using an acoustic model of a multiple-channel cochlear implant. The first strategy (F2) presented the amplitude envelope of the speech and the second formant frequency. The second strategy (F0 F2) included the voice fundamental frequency, and the third strategy (F0 F1 F2) presented the first formant frequency as well. Discourse level testing with the speech tracking method showed a clear superiority of the F0 F1 F2 strategy when the auditory information was used to supplement lipreading. Tracking rates averaged over three subjects for nine 10-min sessions were 40 wpm for F2, 52 wpm for F0 F2, and 66 wpm for F0 F1 F2. Vowel and consonant confusion studies and a test of prosodic information were carried out with auditory information only. The vowel test showed a significant difference between the strategies, but no differences were found for the other tests. It was concluded that the amplitude and duration cues common to all three strategies accounted for the levels of consonant and prosodic information received by the subjects, while the different tracking rates were a consequence of the better vowel recognition and the more natural quality of the F0 F1 F2 strategy.     

Enhancing Chinese tone recognition by manipulating amplitude envelope: implications for cochlear implants

Tone recognition is important for speech understanding in tonal languages such as Mandarin Chinese. Cochlear implant patients are able to perceive some tonal information by using temporal cues such as periodicity-related amplitude fluctuations and similarities between the fundamental frequency (F0) contour and the amplitude envelope. The present study investigates whether modifying the amplitude envelope to better resemble the F0 contour can further improve tone recognition in multichannel cochlear implants. Chinese tone and vowel recognition were measured for six native Chinese normal-hearing subjects listening to a simulation of a four-channel cochlear implant speech processor with and without amplitude envelope enhancement. Two algorithms were proposed to modify the amplitude envelope to more closely resemble the F0 contour. In the first algorithm, the amplitude envelope as well as the modulation depth of periodicity fluctuations was adjusted for each spectral channel. In the second algorithm, the overall amplitude envelope was adjusted before multichannel speech processing, thus reducing any local distortions to the speech spectral envelope. The results showed that both algorithms significantly improved Chinese tone recognition. By adjusting the overall amplitude envelope to match the F0 contour before multichannel processing, vowel recognition was better preserved and less speech-processing computation was required. The results suggest that modifying the amplitude envelope to more closely resemble the F0 contour may be a useful approach toward improving Chinese-speaking cochlear implant patients' tone recognition.     

Interactions of speaking condition and auditory feedback on vowel production in postlingually deaf adults with cochlear implants

This study investigates the effects of speaking condition and auditory feedback on vowel production by postlingually deafened adults. Thirteen cochlear implant users produced repetitions of nine American English vowels prior to implantation, and at one month and one year after implantation. There were three speaking conditions (clear, normal, and fast), and two feedback conditions after implantation (implant processor turned on and off). Ten normal-hearing controls were also recorded once. Vowel contrasts in the formant space (expressed in mels) were larger in the clear than in the fast condition, both for controls and for implant users at all three time samples. Implant users also produced differences in duration between clear and fast conditions that were in the range of those obtained from the controls. In agreement with prior work, the implant users had contrast values lower than did the controls. The implant users' contrasts were larger with hearing on than off and improved from one month to one year postimplant. Because the controls and implant users responded similarly to a change in speaking condition, it is inferred that auditory feedback, although demonstrably important for maintaining normative values of vowel contrasts, is not needed to maintain the distinctiveness of those contrasts in different speaking conditions.     

Vowel and consonant recognition of cochlear implant patients using formant-estimating speech processors

Vowel and consonant confusion matrices were collected in the hearing alone (H), lipreading alone (L), and hearing plus lipreading (HL) conditions for 28 patients participating in the clinical trial of the multiple-channel cochlear implant. All patients were profound-to-totally deaf and "hearing" refers to the presentation of auditory information via the implant. The average scores were 49% for vowels and 37% for consonants in the H condition and the HL scores were significantly higher than the L scores. Information transmission and multidimensional scaling analyses showed that different speech features were conveyed at different levels in the H and L conditions. In the HL condition, the visual and auditory signals provided independent information sources for each feature. For vowels, the auditory signal was the major source of duration information, while the visual signal was the major source of first and second formant frequency information. The implant provided information about the amplitude envelope of the speech and the estimated frequency of the main spectral peak between 800 and 4000 Hz, which was useful for consonant recognition. A speech processor that coded the estimated frequency and amplitude of an additional peak between 300 and 1000 Hz was shown to increase the vowel and consonant recognition in the H condition by improving the transmission of first formant and voicing information.     

Forward masking in patients with cochlear implants

Forward masking was measured in 12 patients with cochlear implants. The amount of masking (in microamps) decreased linearly as a function of the logarithm of the signal delay from masker offset. Normalized forward-masking recovery functions for cochlear implants were similar to normalized functions of normal-hearing listeners, indicating that the mechanism of psychophysical forward masking is retrocochlear. These results indicate that the logarithm of acoustic amplitude should be mapped to microamps to produce normal forward masking in implanted patients. Despite the fact that the forward-masking recovery functions were similar across all patients, their performance with their speech processor varied widely.     

Young cochlear implant users' response to delayed auditory feedback.

This investigation determined whether the signal provided by the Cochlear Corporation Nucleus cochlear implant can convey enough speech information to induce a response to delayed auditory feedback (DAF), and whether prelingually deafened children who received a cochlear implant relatively late in their speech development are susceptible. Ten children with the Nucleus cochlear implant spoke simple phrases, first without and then with DAF. Three prelingually deafened subjects and the only two postlingually deafened subjects demonstrated longer phrase durations when speaking with DAF than without it. Two of the prelingually deafened subjects who demonstrated a response received their cochlear implants at the age of 5 years.     

Across-site patterns of modulation detection: relation to speech recognition

The aim of this study was to identify across-site patterns of modulation detection thresholds (MDTs) in subjects with cochlear implants and to determine if removal of sites with the poorest MDTs from speech processor programs would result in improved speech recognition. Five hundred millisecond trains of symmetric-biphasic pulses were modulated sinusoidally at 10 Hz and presented at a rate of 900 pps using monopolar stimulation. Subjects were asked to discriminate a modulated pulse train from an unmodulated pulse train for all electrodes in quiet and in the presence of an interleaved unmodulated masker presented on the adjacent site. Across-site patterns of masked MDTs were then used to construct two 10-channel MAPs such that one MAP consisted of sites with the best masked MDTs and the other MAP consisted of sites with the worst masked MDTs. Subjects' speech recognition skills were compared when they used these two different MAPs. Results showed that MDTs were variable across sites and were elevated in the presence of a masker by various amounts across sites. Better speech recognition was observed when the processor MAP consisted of sites with best masked MDTs, suggesting that temporal modulation sensitivity has important contributions to speech recognition with a cochlear implant.     

Effects of Tonsillectomy on Speech Spectrum

Changes in the speech spectrum of vowels and consonants before and after tonsillectomy were investigated to find out the impact of the operation on speech quality. Speech recordings obtained from patients were analyzed using the Kay Elemetrics, Multi-Dimensional Voice Processing (MDVP Advanced) software. Examination of the time-course changes after the operation revealed that certain speech parameters changed. These changes were mainly F3 (formant center frequency) and B3 (formant bandwidth) for the vowel /o/ and a slight decrease in B1 and B2 for the vowel /a/. The noise-to-harmonic ratio (NHR) also decreased slightly, suggesting less nasalized vowels. It was also observed that the fricative, glottal consonant /h/ has been affected. The larger the tonsil had been, the more changes were seen in the speech spectrum. The changes in the speech characteristics (except F3 and B3 for the vowel /o/) tended to recover, suggesting an involvement of auditory feedback and/or replacement of a new soft tissue with the tonsils. Although the changes were minimal and, therefore, have little effect on the extracted acoustic parameters, they cannot be disregarded for those relying on their voice for professional reasons, that is, singers, professional speakers, and so forth.     

个体依恋风格对亲密话语韵律及嗓音特征的影响

个体依恋风格可基于依恋回避、依恋焦虑这两个维度加以定义,并根据其取值高低划分为4种依恋类型.为探究依恋风格对亲密话语语音特征的影响,我们选取12对年轻异性情侣,采用亲密关系体验量表测出各人的依恋回避与焦虑值.通过半开放式的约会剧本,诱导被试产出亲密语气的目标句,再单独朗读这些目标句作为中性话语.基于9个韵律及嗓音参数的...     

Speech processing studies using an acoustic model of a multiple-channel cochlear implant

The speech perception of two multiple-channel cochlear implant patients was compared with that of three normally hearing listeners using an acoustic model of the implant for 22 different speech tests. The tests used included a minimal auditory capabilities battery, both closed-set and open-set word and sentence tests, speech tracking and a 12-consonant confusion study using nonsense syllables. The acoustic model represented electrical current pulses by bursts of noise and the effects of different electrodes were represented by using bandpass filters with different center frequencies. All subjects used a speech processor that coded the fundamental voicing frequency of speech as a pulse rate and the second formant frequency of speech as the electrode position in the cochlea, or the center frequency of the bandpass filter. Very good agreement was found for the two groups of subjects, indicating that the acoustic model is a useful tool for the development and evaluation of alternative cochlear implant speech processing strategies.     

The use of an auditory model in predicting perceptual ratings of breathy voice quality

Despite much research, the relationship between vocal acoustic signals and perceived voice quality is not well understood. The present study used an auditory model proposed by Moore et al¹⁰ to study how changes in the acoustic spectrum may relate to changes in perceptual ratings of breathiness. Perceptual ratings of breathiness were obtained using a multidimensional scaling (MDS) design. The stimulus distances on the dominant MDS dimension were correlated with several commonly used acoustic measures for voice quality. These distances were also compared with measures obtained from the output of the auditory model. Results show that the partial loudness of the harmonic energy obtained with the aspiration noise acting as a masker was the most important predictor of perceptual ratings of breathiness. Results also demonstrate that measures obtained from the auditory spectrum were better predictors of perceptual ratings of breathiness than were commonly used acoustic spectral measures.     

Perceptual sensitivity to first harmonic amplitude in the voice source

Little is known about the perceptual importance of changes in the shape of the source spectrum, although many measures have been proposed and correlations with different vocal qualities (breathiness, roughness, nasality, strain...) have frequently been reported. This study investigated just-noticeable differences in the relative amplitudes of the first two harmonics (H1-H2) for speakers of Mandarin and English. Listeners heard pairs of vowels that differed only in the amplitude of the first harmonic and judged whether or not the voice tokens were identical in voice quality. Across voices and listeners, just-noticeable-differences averaged 3.18 dB. This value is small relative to the range of values across voices, indicating that H1-H2 is a perceptually valid acoustic measure of vocal quality. For both groups of listeners, differences in the amplitude of the first harmonic were easier to detect when the source spectral slope was steeply falling so that F0 dominated the spectrum. Mandarin speakers were significantly more sensitive (by about 1 dB) to differences in first harmonic amplitudes than were English speakers. Two explanations for these results are possible: Mandarin speakers may have learned to hear changes in harmonic amplitudes due to changes in voice quality that are correlated with the tones of Mandarin; or Mandarin speakers' experience with tonal contrasts may increase their sensitivity to small differences in the amplitude of F0 (which is also the first harmonic).     

An analysis of the effects of electrical field interaction with an acoustic model of cochlear implants

Electrical field interaction caused by current spread in a cochlear implant was modeled in an explicit way in an acoustic model (the SPREAD model) presented to six listeners with normal hearing. The typical processing of cochlear implants was modeled more closely than in traditional acoustic models by careful selection of parameters related to current spread or parameters that could amplify the electrical field interactions caused by current spread. These parameters were the insertion depth, electrode spacing, electrical dynamic range, and dynamic range compression function. The hypothesis was that current spread could account for the asymptote in performance in speech intelligibility experiments observed at around seven stimulation channels in a number of cochlear implant studies. Speech intelligibility for sentences, vowels, and consonants at three noise levels (SNR of +15 dB, +10 dB, and +5 dB) was measured as a function of the number of spectral channels (4, 7, and 16). The SPREAD model appears to explain the asymptote in speech intelligibility at seven channels for all noise levels for all speech material used in this study. It is shown that the compressive amplitude mapping used in cochlear implants can have a detrimental effect on the number of effective channels.     

The effects of production and presentation level on the auditory distance perception of speech

Although both perceived vocal effort and intensity are known to influence the perceived distance of speech, little is known about the processes listeners use to integrate these two parameters into a single estimate of talker distance. In this series of experiments, listeners judged the distances of prerecorded speech samples presented over headphones in a large open field. In the first experiment, virtual synthesis techniques were used to simulate speech signals produced by a live talker at distances ranging from 0.25 to 64 m. In the second experiment, listeners judged the apparent distances of speech stimuli produced over a 60-dB range of different vocal effort levels (production levels) and presented over a 34-dB range of different intensities (presentation levels). In the third experiment, the listeners judged the distances of time-reversed speech samples. The results indicate that production level and presentation level influence distance perception differently for each of three distinct categories of speech. When the stimulus was high-level voiced speech (produced above 66 dB SPL 1 m from the talker's mouth), the distance judgments doubled with each 8-dB increase in production level and each 12-dB decrease in presentation level. When the stimulus was low-level voiced speech (produced at or below 66 dB SPL at 1 m), the distance judgments doubled with each 15-dB increase in production level but were relatively insensitive to changes in presentation level at all but the highest intensity levels tested. When the stimulus was whispered speech, the distance judgments were unaffected by changes in production level and only decreased with increasing presentation level when the intensity of the stimulus exceeded 66 dB SPL. The distance judgments obtained in these experiments were consistent across a range of different talkers, listeners, and utterances, suggesting that voice-based distance cueing could provide a robust way to control the apparent distances of speech sounds in virtual audio displays.