Speech coding in the auditory nerve: IV. Sounds with consonant-like dynamic characteristics

Discharge patterns of auditory-nerve fibers in anesthetized cats were obtained for two stimulus levels in response to synthetic stimuli with dynamic characteristics appropriate for selected consonants. A set of stimuli was constructed by preceding a signal that was identified as /da/by another sound that was systematically manipulated so that the entire complex would sound like either /da/, /ada/, /na/, /sa/, /sa/, or others. Discharge rates of auditory-nerve fibers in response to the common /da/-like formant transitions depended on the preceding context. Average discharge rates during these transitions decreased most for fibers whose CFs were in frequency regions where the context had considerable energy. Some effect of the preceding context on fine time patterns of response to the transitions was also found, but the identity of the largest response components (which often corresponded to the formant frequencies) was in general unaffected. Thus the response patterns during the formant transitions contain cues about both the nature of the transitions and the preceding context. A second set of stimuli sounding like /s/ and /c/ was obtained by varying the duration of the rise in amplitude at the onset of a filtered noise burst. At both 45 and 60 dB SPL, there were fibers which showed a more prominent peak in discharge rate at stimulus onset for /c/ than for /s/, but the CF regions that reflected the clearest distinctions depended on stimulus level. The peaks in discharge rate that occur in response to rapid changes in amplitude or spectrum might be used by the central processor as pointers to portions of speech signals that are rich in phonetic information.     

Speech coding in the auditory nerve: V. Vowels in background noise

Responses of auditory-nerve fibers to steady-state, two-formant vowels in low-pass background noise (S/N = 10 dB) were obtained in anesthetized cats. For fibers over a wide range of characteristic frequencies (CFs), the peaks in discharge rate at the onset of the vowel stimuli were nearly eliminated in the presence of noise. In contrast, strong effects of noise on fine time patterns of discharge were limited to CF regions that are far from the formant frequencies. One effect is a reduction in the amplitude of the response component at the fundamental frequency in the high-CF regions and for CFs between F1 and F2 when the formants are widely separated. A reduction in the amplitude of the response components at the formant frequencies, with concomitant increase in components near CF or low-frequency components occurs in CF regions where the signal-to-noise ratio is particularly low. The processing schemes that were effective for estimating the formant frequencies and fundamental frequency of vowels in quiet generally remain adequate in moderate-level background noise. Overall, the discharge patterns contain many cues for distinctions among the vowel stimuli, so that the central processor should be able to identify the different vowels, consistent with psychophysical performance at moderate signal-to-noise ratios.     

Representation of steady-state vowels in the temporal aspects of the discharge patterns of populations of auditory-nerve fibers.

This paper is concerned with the representation of the spectra of synthesized steady-state vowels in the temporal aspects of the discharges of auditory-nerve fibers. The results are based on a study of the responses of large numbers of single auditory-nerve fibers in anesthetized cats. By presenting the same set of stimuli to all the fibers encountered in each cat, we can directly estimate the population response to those stimuli. Period histograms of the responses of each unit to the vowels were constructed. The temporal response of a fiber to each harmonic component of the stimulus is taken to be the amplitude of the corresponding component in the Fourier transform of the unit's period histogram. At low sound levels, the temporal response to each stimulus component is maximal among units with CFs near the frequency of the component (i.e., near its place). Responses to formant components are larger than responses to other stimulus components. As sound level is increased, the responses to the formants, particularly the first formant, increase near their places and spread to adjacent regions, particularly toward higher CFs. Responses to nonformant components, exept for harmonics and intermodulation products of the formants (2F1,2F2,F1 + F2, etc), are suppressed; at the highest sound levels used (approximately 80 dB SPL), temporal responses occur almost exclusively at the first two or three formants and their harmonics and intermodulation products. We describe a simple calculation which combines rate, place, and temporal information to provide a good representation of the vowels' spectra, including a clear indication of at least the first two formant frequencies. This representation is stable with changes in sound level at least up to 80 dB SPL; its stability is in sharp contrast to the behavior of the representation of the vowels' spectra in terms of discharge rate which degenerates at stimulus levels within the conversational range.     

Speech coding in the auditory nerve: II. Processing schemes for vowel-like sounds

Several processing schemes by which phonetically important information for vowels can be extracted from responses of auditory-nerve fibers are analyzed. The schemes are based on power spectra of period histograms obtained in response to a set of nine two-formant, steady-state, vowel-like stimuli presented at 60 and 75 dB SPL. One class of "local filtering" schemes, which was originally proposed by Young and Sachs [J. Acoust. Soc. Am. 66, 1381-1403 (1979)], consists of analyzing response patterns by filters centered at the characteristic frequencies (CF) of the fibers, so that a tonotopically arranged measure of synchronized response can be obtained. Various schemes in this class differ in the characteristics of the filter. For a wide range of filter bandwidths, formant frequencies correspond approximately to the CFs for which the response measure is maximal. If in addition, the bandwidths of the analyzing filters are made compatible with psychophysical measures of frequency selectivity, low-frequency harmonics of the stimulus fundamental are resolved in the output profile, so that fundamental frequency can also be estimated. In a second class of processing schemes, a dominant response component is defined for each fiber from a 1/6 octave spectral representation of the response pattern, and the formant frequencies are estimated from the most frequent values of the dominant component in the ensemble of auditory-nerve fibers. The local filtering schemes and the dominant component schemes can be related to "place" and "periodicity" models of auditory processing, respectively.     

Coding of spectral fine structure in the auditory nerve. I. Fourier analysis of period and interspike interval histograms

The temporal fine structure of discharge patterns of single auditory-nerve fibers in adult cats was analyzed in response to signals consisting of a variable number of equal-intensity, in-phase harmonics of a common low-frequency fundamental. Two analytic methods were employed. The first method considered Fourier spectra of period histograms based on the period of the fundamental, and the second method considered Fourier spectra of interspike interval histograms (ISIH's). Both analyses provide information about fiber tuning properties, but Fourier spectra of ISIH's also allow estimates to be made of the degree of resolution of individual stimulus components. At low intensities (within 20-40 dB of threshold), indices of synchronization to individual components of complex tones were similar to those obtained for pure tones. This was true even when fibers were capable of responding to several signal components simultaneously. Response spectra obtained at low intensities resembled fibers' tuning curves, and fibers with low spontaneous discharge rates tended to provide better resolution of stimulus components than fibers with high spontaneous rates. Strongly nonlinear behavior existed at higher stimulus intensities. In this, information was transmitted about progressively fewer signal components and about frequencies not present in the acoustic stimulus, and the component eliciting the largest response shifted away from the fiber's characteristic frequency and toward the edges of the stimulus spectrum. This high-intensity "edge enhancement" can result from the combined effects of a compressive input-output nonlinearity, suppression, and the fortuitous addition of internally generated combination tones. The data indicate that sufficient information exists for the auditory system to determine the frequencies of narrowly spaced stimulus components from the temporal fine structure of nerve fiber's responses.     

Responses of auditory-nerve fibers to nasal consonant-vowel syllables

Responses of single auditory-nerve fibers in anesthetized cat to spoken nasal consonant-vowel syllables were recorded. Analyses in the form of spectrograms and of three-dimensional spatial-time and spatial-frequency plots were made. Among other features, formant transitions are clearly represented in the fibers' response synchronization properties. During vocalic segments, especially those in /mu/and/ma/, at a stimulus level near 75 dB SPL, a strong dominance in the responses by frequencies near the second formant (F2) is found for most fibers whose characteristic frequencies (CFs) are at or above F2. In contrast, at more moderate levels, the same fibers may show response synchrony to frequencies closer to their own CFs. There are significant differences in the response properties of high and low/medium-spontaneous-rate fibers.     

Coding of spectral fine structure in the auditory nerve. II: Level-dependent nonlinear responses

Phase-locked discharge patterns of single cat auditory-nerve fibers were analyzed in response to complex tones centered at fiber characteristic frequency (CF). Signals were octave-bandwidth harmonic complexes defined by a center frequency F and an intercomponent spacing factor N, such that F/N was the fundamental frequency. Parameters that were manipulated included the phase spectrum, the number of components, and the intensity of the center component. Analyses employed Fourier transforms of period histograms to assess the degree to which responses were synchronized to the frequencies present in the acoustic stimulus. Several nonlinearities were observed in the response as intensity was varied between threshold and 80-90 dB SPL. Response nonlinearities were strong for all signals except those with random phase spectra. The most commonly observed nonlinearity was an emphasis of one or more stimulus components in the response. The degree of nonlinearity usually increased with intensity and signal complexity and decreased with fiber frequency selectivity. Half-wave rectification introduced synchronization to the missing fundamental. The strength of the response at the fundamental was related to stimulus crest factor. Signals with low center frequencies and high crest factors often elicited instantaneous discharge rates at the theoretical maximum of pi CF. This suggests that the probability of spike generation approaches one during high-amplitude waveform segments. Response nonlinearity was interpreted as arising from three sources, namely, cochlear mechanics, compression of instantaneous discharge rate, and saturation of average discharge rate. At near-threshold intensities, fibers with high spontaneous rates exhibited responses that were linear functions of stimulus waveshape, whereas fibers with low spontaneous spike rates produced responses that were best described in terms of an expansive nonlinearity.     

Speech processing in the auditory system. I: The representation of speech sounds in the responses of the auditory nerve

In a previous paper the speech evoked spatio-temporal response patterns recorded in large populations of auditory-nerve fibers in the cat were examined [M.I. Miller and M.B. Sachs, J. Acoust. Soc. Am. 74, 502-517 (1983)]. The distribution of the relative phases of synchronized activity emerges as an important response feature reflecting the stimulus spectral parameters. Specifically, each strong low-order harmonic of the stimulus (less than or equal to 1.5-2 kHz) dominates the synchrony of a relatively broad segment of fibers near its corresponding characteristic frequency (CF) location in a pattern which mirrors the underlying traveling wave component. Each such fiber segment can be roughly subdivided into two regions: (1) a region basal to the point of resonance of the harmonic where the fiber PST histograms accumulate only small delays (or phase shifts) relative to each other reflecting the fast speed of propagation of the traveling wave, and (2) a region at or very near the point of resonance where the responses exhibit drastic relative phase shifts owing to the sudden slow down of the traveling wave and the consequent rapid accumulation of phase shifts. These rapid phase shifts thus manifest themselves as steep and localized spatial discontinuities in an otherwise relatively uniform instantaneous pattern of activity across the fiber array, all occurring at the CF locations corresponding to the low-order harmonics of the stimulus.     

Synchronized discharge rate representation of voice-onset time in the chinchilla auditory nerve

Responses of chinchilla auditory nerve fibers to synthesized stop consonant syllables differing in voice-onset time (VOT) were obtained. The syllables, heard as /ga/-/ka/ or /da/-/ta/, were similar to those previously used by others in psychophysical experiments with human and chinchilla subjects. Synchronized discharge rates of neurons tuned to frequencies near the first formant increased at the onset of voicing for VOTs longer than 20 ms. Stimulus components near the formant or the neuron's characteristic frequency accounted for the increase. In these neurons, synchronized response changes were closely related to the same neuron's average discharge rates [D. G. Sinex and L. P. McDonald, J. Acoust. Soc. Am. 83, 1817-1827 (1988)]. Neurons tuned to frequency regions near the second and third formants usually responded to components near the second formant prior to the onset of voicing. These neurons' synchronized discharges could be captured by the first formant at the onset of voicing or with a latency of 50-60 ms, whichever was later. Since these neurons' average rate responses were unaffected by the onset of voicing, the latency of the synchronized response did provide as much additional neural cue to VOT. Overall, however, discharge synchrony did not provide as much information about VOT as was provided by the best average rate responses. The results are compared to other measurements of the peripheral encoding of speech sounds and to aspects of VOT perception.     

Encoding of steady-state vowels in the auditory nerve: representation in terms of discharge rate

Responses of large populations of auditory-nerve fibers to synthesized steady-state vowels were recorded in anesthetized cats. Driven discharge rate to vowels, normalized by dividing by saturation rate (estimated from the driven rate to CF tones 50 dB above threshold), was plotted versus fiber CF for a number of vowel levels. For the vowels /I/ and /e/, such rate profiles showed a peak in the region of the first formant and another in the region of the second and third formants, for sound levels below about 70 dB SPL. For /a/ at levels below about 40 dB SPL there are peaks in the region of the first and second formants. At higher levels these peaks disappear for all the vowels because of a combination of rate saturation and two-tone suppression. This must be qualified by saying that rate profiles plotted separately for units with spontaneous rates less than one spike per second may retain peaks at higher levels. Rate versus level functions for units with CFs above the first formant can saturate at rates less than the saturation rate to CF to-es or they can be nonmonotonic; these effects are most likely produced by the same mechanism as that involved in two-tone suppression.     

A temporal analysis of auditory-nerve fiber responses to spoken stop consonant-vowel syllables

Auditory-nerve fiber spike trains were recorded in response to spoken English stop consonant-vowel syllables, both voiced (/b,d,g/) and unvoiced (/p,t,k/), in the initial position of syllables with the vowels /i,a,u/. Temporal properties of the neural responses and stimulus spectra are displayed in a spectrographic format. The responses were categorized in terms of the fibers' characteristic frequencies (CF) and spontaneous rates (SR). High-CF, high-SR fibers generally synchronize to formants throughout the syllables. High-CF, low/medium-SR fibers may also synchronize to formants; however, during the voicing, there may be sufficient low-frequency energy present to suppress a fiber's synchronized response to a formant near its CF. Low-CF fibers, from both SR groups, synchronize to energy associated with voicing. Several proposed acoustic correlates to perceptual features of stop consonant-vowel syllables, including the initial spectrum, formant transitions, and voice-onset time, are represented in the temporal properties of auditory-nerve fiber responses. Nonlinear suppression affects the temporal features of the responses, particularly those of low/medium-spontaneous-rate fibers.     

The representation of the spectra and fundamental frequencies of steady-state single- and double-vowel sounds in the temporal discharge patterns of guinea pig cochlear-nerve fibers

Psychophysical results using double vowels imply that subjects are able to use the temporal aspects of neural discharge patterns. To investigate the possible temporal cues available, the responses of fibers in the cochlear nerve of the anesthetized guinea pig to synthetic vowels were recorded at a range of sound levels up to 95 dB SPL. The stimuli were the single vowels /i/ [fundamental frequency (f0) 125 Hz], /a/ (f0, 100 Hz), and /c/ (f0, 100 Hz) and the double vowels were /a(100),i(125)/ and /c(100),i(125)/. Histograms synchronized to the period of the double vowels were constructed, and locking of the discharge to individual harmonics was estimated from them by Fourier transformation. One possible cue for identifying the f0's of the constituents of a double vowel is modulation of the neural discharge with a period of 1/f0. Such modulation was found at frequencies between the formant peaks of the double vowel, with modulation at the periods of 100 and 125 Hz occurring at different places in the fiber array. Generation of a population response based on synchronized responses [average localized synchronized rate (ALSR): see Young and Sachs [J. Acoust. Soc. Am. 66, 1381-1403 (1979)] allowed estimation of the f0's by a variety of methods and subsampling the population response at the harmonics of the f0 of the constituent vowel achieved a good reconstruction of its spectrum. Other analyses using interval histograms and autocorrelation, which overcome some problems associated with the ALSR approach, also allowed f0 identification and vowel segregation. The present study has demonstrated unequivocally that the timing of the impulses in auditory-nerve fibers provides copious possible cues for the identification of the fundamental frequencies and spectra associated with each of the constituents of double vowels.     

Response dynamics of goldfish saccular fibers: effects of stimulus frequency and intensity on fibers with different tuning, sensitivity, and spontaneous activity

The effects of stimulus frequency and intensity on response patterns (PST histograms) to tone burst stimulation were examined in differently tuned saccular fibers of the goldfish. In addition, the sensitivity of these fibers to amplitude-modulated (AM) signals of different carrier frequencies was measured. The response patterns evoked by unmodulated signals were a complex function of tuning, spontaneous activity and sensitivity of the fiber, and the frequency and intensity of the signal. Frequency-dependent response patterns were found in low-frequency fibers with best frequencies (BF) below 200 Hz. Responses in these fibers ranged from tonic to phasic in nonspontaneous fibers and included more complex patterns in spontaneously active fibers, such as suppression of evoked activity below spontaneous levels. Midfrequency fibers (BF = 500-600 Hz) showed responses similar to those in low-frequency fibers, but with less dependence on frequency. In contrast, both high-frequency (BF = 800-1000 Hz) and wideband, untuned fibers showed frequency-invariant patterns of adaptation. High-frequency fibers were equally sensitive to AM signals at all frequencies tested. The sensitivity of low-frequency fibers to AM, however, increased as a function of carrier frequency and corresponded to the degree of adaptation in response to unmodulated tones. In general, the AM sensitivity of a fiber could be predicted more by its pattern of response to unmodulated signals than by its tuning characteristics.     

Extraction and enhancement of spectral structure by the cochlea

The intensity dependence of signal processing in the cat cochlea was studied in responses of single auditory-nerve fibers for harmonic complexes having various amplitude and phase spectra. Analyses were based on information present in temporal discharge cadences, and they consisted of assessing Fourier spectra of period histograms synchronized to the period of the waveform fundamental. At low intensities, response spectra resembled filtered versions of the stimulus spectrum, with the amounts of filtering being determined by fibers' tuning curves. At high intensities, response spectra exhibited nonlinear behavior and could differ dramatically from spectra obtained at low intensities. The high-intensity response typically emphasized one or more aspects of the stimulus spectrum. When the stimulus possessed equal component amplitudes and phases, the features that were emphasized at high intensities were the high- and low-frequency edges of the spectrum, and when the component at fiber CF was changed in phase or amplitude relative to the others, fibers primarily signaled the presence of the phase- or amplitude-shifted component. Many of the intensity-dependent changes in response spectra are accounted for by considering the effects of the compressive input-output nonlinearity operating at or peripheral to the hair-cell level on the temporal waveform.     

Spectral properties of fricative consonants in children

Twenty-one preadolescent children were asked to produce stimulus words with the fricatives /f/, /v/, /s/, /z/, /s/, and /3/ in various phonetic environments. Results indicated that the major resonances were higher than values reported for both male and female adult speakers. Also significant differences existed among the major resonances of the consonants studied.     

Stimulus factors influencing the identification of voiced stop consonants by normal-hearing and hearing-impaired adults

The effects of mild-to-moderate hearing impairment on the perceptual importance of three acoustic correlates of stop consonant place of articulation were examined. Normal-hearing and hearing-impaired adults identified a stimulus set comprising all possible combinations of the levels of three factors: formant transition type (three levels), spectral tilt type (three levels), and abruptness of frequency change (two levels). The levels of these factors correspond to those appropriate for /b/, /d/, and /g/ in the /ae/ environment. Normal-hearing subjects responded primarily in accord with the place of articulation specified by the formant transitions. Hearing-impaired subjects showed less-than-normal reliance on formant transitions and greater-than-normal reliance on spectral tilt and abruptness of frequency change. These results suggest that hearing impairment affects the perceptual importance of cues to stop consonant identity, increasing the importance of information provided by both temporal characteristics and gross spectral shape and decreasing the importance of information provided by the formant transitions.     

Neuronal responses to amplitude-modulated and pure-tone stimuli in the guinea pig inferior colliculus, and their modification by broadband noise

Neuronal responses were recorded to pure and to sinusoidally amplitude-modulated (AM) tones at the characteristic frequency (CF) in the central nucleus of the inferior colliculus of anesthetized guinea pigs. Temporal (synchronized) and mean-rate measures were derived from period histograms locked to the stimulus modulation waveform to characterize the modulation response. For stimuli presented in quiet, the modulation gain at low frequencies of modulation (approx less than 50 Hz) was inversely proportional to the neuron's mean firing rate in response to both the modulated stimulus and to a pure tone at an equivalent level. In 43% of units the mean discharge rates in response to the AM stimuli were greatest for those modulation frequencies that generated the largest temporal responses. These discharge-rate maxima occurred at signal intensities corresponding to the steeply sloping part of the neuron's pure-tone rate-intensity function (RIF). The change in mean-rate response to modulated stimuli, as a function of intensity, was qualitatively similar to the pure-tone RIF. Adding broadband noise to the modulated stimulus increased the neuron's temporal response to low modulation frequencies. This increase in modulation gain was correlated with mean firing rate in response to the modulation but did not bear a simple relationship to the noise-induced shift in the RIF measured for a pure tone.     

Masking of short stimuli by noises with spiked spectra: II. Time summation and frequency selectivity of hearing in a narrow frequency range

The thresholds of masking of short high-frequency pulses with either different durations (1.25–25 ms) and similar central frequency or different central frequencies (3.6–4.4 kHz) but similar durations were measured to reveal manifestations of the properties of peripheral encoding in auditory perception. Noises with a spiked amplitude spectrum structure were used as maskers. The central frequency and the frequency band of a masker were 4 and 1 kHz, respectively. The central frequencies of a stimulus and a masker being equal, the noise the central frequency of which coincided with the frequency corresponding to a dip of an indented spectrum was called an off_(rip)-frequency masker. Owing to the off_(rip)-masker, stimuli-induced masking thresholds were formed taking into account excitation in a narrow region of a basila membrane and auditory nerve fibers with characteristic frequencies from a narrow range. High-frequency pulses with an envelope in the form of the Gaussian function and sinusoidal filling were used as stimuli. At masker levels of 30 dB above the auditory threshold, frequencies of off_(rip)-masker spectra spikes of 500–2000 Hz, and a central stimulus frequency of 4 kHz, the thresholds of tonal stimuli (25 ms in duration) masking in two out of three probationers were higher than the thresholds of masking of compact stimuli (1.25 ms in duration). In the third probationer, on the contrary, the thresholds of tonal stimuli masking were lower than the thresholds of compact stimuli masking. At masker levels of 50 dB, individual threshold differences disappeared. The obtained results were interpreted in the context of implementation of different methods of auditory encoding of the intensity. The methods were based on either the average frequency of auditory nerve pulsations or the number of fibers participating in the response. The interpretation was also carried out in the context of revealing manifestations of nonlinear properties of basila membrane displacements in auditory thresholds. The fact that the dependence of detection thresholds of compact stimuli on their central frequency in one of the two probationers did not reveal the minimum in case of coincidence of off_(rip)-masker and stimulus frequencies pointed to the presence of an auditory “problem zone” that was likely to be localized at the periphery of the auditory system.