Dead regions and pitch perception

The perception of pitch for pure tones with frequencies falling inside low- or high-frequency dead regions (DRs) was examined. Subjects adjusted a variable-frequency tone to match the pitch of a fixed tone. Matches within one ear were often erratic for tones falling in a DR, indicating unclear pitch percepts. Matches across ears of subjects with asymmetric hearing loss, and octave matches within ears, indicated that tones falling within a DR were perceived with an unclear pitch and/or a pitch different from "normal" whenever the tones fell more than 0.5 octave within a low- or high-frequency DR. One unilaterally impaired subject, with only a small surviving region between 3 and 4 kHz, matched a fixed 0.5-kHz tone in his impaired ear with, on average, a 3.75-kHz tone in his better ear. When asked to match the 0.5-kHz tone with an amplitude-modulated tone, he adjusted the carrier and modulation frequencies to about 3.8 and 0.5 kHz, respectively, suggesting that some temporal information was still available. Overall, the results indicate that the pitch of low-frequency tones is not conveyed solely by a temporal code. Possibly, there needs to be a correspondence between place and temporal information for a normal pitch to be perceived.     

Octave illusion elicited by overlapping narrowband noises

The octave or Deutsch illusion occurs when two tones, separated by about one octave, are presented simultaneously but alternating between ears, such that when the low tone is presented to the left ear the high tone is presented to the right ear and vice versa. Most subjects hear a single tone that alternates both between ears and in pitch; i.e., they hear a low pitched tone in one ear alternating with a high pitched tone in the other ear. The present study examined whether the illusion can be elicited by aperiodic signals consisting of low-frequency band-pass filtered noises with overlapping spectra. The amount of spectral overlap was held constant, but the high- and low-frequency content of the signals was systematically varied. The majority of subjects perceived an auditory illusion in terms of a dominant ear for pitch and lateralization by frequency, as proposed by Deutsch [(1975a) Sci. Am. 233, 92-104]. Furthermore, the salience of the illusion increased as the high frequency of the content in the signal increased. Since no harmonics were present in the stimuli, it is highly unlikely that this illusion is perceived on the basis of binaural diplacusis or harmonic binaural fusion.     

Learning to perceive pitch differences

This paper reports two experiments concerning the stimulus specificity of pitch discrimination learning. In experiment 1, listeners were initially trained, during ten sessions (about 11,000 trials), to discriminate a monaural pure tone of 3000 Hz from ipsilateral pure tones with slightly different frequencies. The resulting perceptual learning (improvement in discrimination thresholds) appeared to be frequency-specific since, in subsequent sessions, new learning was observed when the 3000-Hz standard tone was replaced by a standard tone of 1200 Hz, or 6500 Hz. By contrast, a subsequent presentation of the initial tones to the contralateral ear showed that the initial learning was not, or was only weakly, ear-specific. In experiment 2, training in pitch discrimination was initially provided using complex tones that consisted of harmonics 3-7 of a missing fundamental (near 100 Hz for some listeners, 500 Hz for others). Subsequently, the standard complex was replaced by a standard pure tone with a frequency which could be either equal to the standard complex's missing fundamental or remote from it. In the former case, the two standard stimuli were matched in pitch. However, this perceptual relationship did not appear to favor the transfer of learning. Therefore, the results indicated that pitch discrimination learning is, at least to some extent, timbre-specific, and cannot be viewed as a reduction of an internal noise which would affect directly the output of a neural device extracting pitch from both pure tones and complex tones including low-rank harmonics.     

The "overshoot" effect and sensory hearing impairment

The threshold for the detection of a brief tone masked by a longer-duration noise burst is higher when the tone is presented shortly after the onset of the noise than at longer delay times. This finding has been termed the "overshoot" effect [E. Zwicker, J. Acoust. Soc. Am. 37, 653-663 (1965)]. The present letter compared the size of the effect in the better and more impaired ear of six subjects with high-frequency unilateral or asymmetric hearing losses of sensory origin. Thresholds were measured for 5-ms 4-kHz tones presented 10, 200, and 390 ms after the onset of a 400-ms, 2- to 8-kHz noise burst. The better ear of each subject was tested using two noise levels, one equal in sound-pressure level and one equal in sensation level to that used for the impaired ear. Thresholds for all subjects and all ears decreased monotonically with increasing delay time, with the size of the effect typically 5 dB. Thus a small overshoot effect was observed regardless of hearing impairment.     

The relation between the human frequency-following response and the low pitch of complex tones

The relation between the auditory brain stem potential called the frequency-following response (FFR) and the low pitch of complex tones was investigated. Eleven complex stimuli were synthesized such that frequency content varied but waveform envelope periodicity was constant. This was accomplished by repeatedly shifting the components of a harmonic complex tone upward in frequency by delta f of 20 Hz, producing a series of six-component inharmonic complex tones with constant intercomponent spacing of 200 Hz. Pitch-shift functions were derived from pitch matches for these stimuli to a comparison pure tone for each of four normal hearing adults with extensive musical training. The FFRs were recorded for the complex stimuli that were judged most divergent in pitch by each subject and for pure-tone signals that were judged equal in pitch to these complex stimuli. Spectral analyses suggested that the spectral content of the FFRs elicited by the complex stimuli did not vary consistently with component frequency or the first effect of pitch shift. Furthermore, complex and pure-tone signals judged equal in pitch did not elicit FFRs of similar spectral content.     

Pitch circularity from tones comprising full harmonic series

This paper describes an algorithm for producing pitch circularity using tones that each comprise a full harmonic series, and reports an experiment that demonstrates such circularity. Banks of 12 tones (i.e., scales) were created, with F0 varying in semitone steps. For each scale, as F0 descended, the amplitudes of the odd-numbered harmonics were reduced relative to the even-numbered ones by 3.5 dB for each semitone step. In consequence, the tone with the lowest F0 was heard as though displaced up an octave. In an experiment employing two such scales, all possible ordered tone pairs from each scale were presented, making 132 ordered tone pairs for each scale. Sixteen subjects judged for each tone pair whether the second tone was higher or lower than the first. The data derived from these pairwise comparisons were subjected to Kruskal's nonmetric multidimensional scaling, and excellent circularities were obtained. Individual differences in the subjects' judgments were also explored. The findings support the argument that musical pitch should be characterized as varying along two dimensions: the monotonic dimension of pitch height and the circular dimension of pitch class.     

Pitch discrimination and phase sensitivity in young and elderly subjects and its relationship to frequency selectivity.

Frequency difference limens for pure tones (DLFs) and for complex tones (DLCs) were measured for four groups of subjects: young normal hearing, young hearing impaired, elderly with near-normal hearing, and elderly hearing impaired. The auditory filters of the subjects had been measured in earlier experiments using the notched-noise method, for center frequencies (fc) of 100, 200, 400, and 800 Hz. The DLFs for both impaired groups were higher than for the young normal group at all fc's (50-4000 Hz). The DLFs at a given fc were generally only weakly correlated with the sharpness of the auditory filter at that fc, and some subjects with broad filters had near-normal DLFs at low frequencies. Some subjects in the elderly normal group had very large DLFs at low frequencies in spite of near-normal auditory filters. These results suggest a partial dissociation of frequency selectivity and frequency discrimination of pure tones. The DLCs for the two impaired groups were higher than those for the young normal group at all fundamental frequencies (fo) tested (50, 100, 200, and 400 Hz); the DLCs for the elderly normal group were intermediate. At fo = 50 Hz, DLCs for a complex tone containing only low harmonics (1-5) were markedly higher than for complex tones containing higher harmonics, for all subject groups, suggesting that pitch was conveyed largely by the higher, unresolved harmonics. For the elderly impaired group, and some subjects in the elderly normal group, DLCs were larger for a complex tone with lower harmonics (1-12) than for tones without lower harmonics (4-12 and 6-12) for fo's up to 200 Hz. Some elderly normal subjects had markedly larger-than-normal DLCs in spite of near-normal auditory filters. The DLCs tended to be larger for complexes with components added in alternating sine/cosine phase than for complexes with components added in cosine phase. Phase effects were significant for all groups, but were small for the young normal group. The results are not consistent with place-based models of the pitch perception of complex tones; rather, they suggest that pitch is at least partly determined by temporal mechanisms.     

Effects of signal envelope on the pitch of short complex tones

Envelope-induced pitch shifts were measured for exponentially decaying complex tones consisting of two sinusoidal components with frequencies f1 = nf0 + 50 Hz and f2 = (n + 1) f0 + 50 Hz, where n equals 3, 4, or 5 and exponential decay rates were 0, 0.5, 1, and 2 dB/ms. Four subjects adjusted a sinusoidal comparison tone to match the virtual pitch of the (missing) fundamental and the pitches of the lower and upper partials f1 and f2. Pitch shifts for f1 are generally less, and pitch shifts for f2 always greater, than envelope-induced shifts observed in isolated sinusoidal tones of comparable frequency and envelope decay rate. Pitch-shift functions for virtual pitch are similar in magnitude and shape to average pitch-shift functions of the partials, which supports the idea that virtual pitch depends on spectral pitch.     

How we hear what is not there: a neural mechanism for the missing fundamental illusion

How the brain estimates the pitch of a complex sound remains unsolved. Complex sounds are composed of more than one tone. When two tones occur together, a third lower pitched tone is often heard. This is referred to as the "missing fundamental illusion" because the perceived pitch is a frequency (fundamental) for which there is no actual source vibration. This phenomenon exemplifies a larger variety of problems related to how pitch is extracted from complex tones, music and speech, and thus has been extensively used to test theories of pitch perception. A noisy nonlinear process is presented here as a candidate neural mechanism to explain the majority of reported phenomenology and provide specific quantitative predictions. The two basic premises of this model are as follows: (I) The individual tones composing the complex tones add linearly producing peaks of constructive interference whose amplitude is always insufficient to fire the neuron (II): The spike threshold is reached only with noise, which naturally selects the maximum constructive interferences. The spacing of these maxima, and consequently the spikes, occurs at a rate identical to the perceived pitch for the complex tone. Comparison with psychophysical and physiological data reveals a remarkable quantitative agreement not dependent on adjustable parameters. In addition, results from numerical simulations across different models are consistent, suggesting relevance to other sensory modalities.     

Pitch shift of pure and complex tones induced by masking noise

Psychoacoustic experiments were performed to measure the pitch-shift effects of pure and complex tones resulting from the addition of a masking noise to the tonal stimuli. Harmonic residue tones with either two or three harmonics and a fundamental frequency of 200 Hz were chosen as test tones. The pitch shifts of virtual and spectral pitches of the residue tones were measured as a function of the intensity of a low-pass noise with 600-Hz cutoff frequency. The SPL of this noise varied between 30 and 70 dB. In another experiment, the pitch shifts of single pure tones corresponding to the frequencies and SPLs of the harmonics of the residue tones were measured using the same masking noise. The results from five subjects for the harmonic residue tones show only a weak dependence of pitch shift on masking noise intensity. This dependence exists for both spectral and virtual pitches. In the case of single pure tones, pitch shift depends more distinctly on noise intensity. Pitch shifts of up to 5% were found in the range of noise intensity investigated. The magnitude of pitch shift shows pronounced interindividual differences, but the direction of the shift effect is always the same. In all cases pitch increases with higher masking noise levels.     

Dissociation of pitch from timbre in auditory short-term memory

In three experiments, untrained listeners made same/different judgments on pairs of pure or complex tones with periods that eventually differed by +/- 4%. On each trial, the two test tones were separated by 4.3 s, during which other tones (I) were heard but had to be ignored. The period (p) of the first test tone was randomly selected between 1/600 and 1/300 s. The period of each I tone was randomly selected among four possible values, close to p (+/- 3% or 6% apart) in some conditions, and remote from p in other conditions. In addition, from condition to condition, the spectral content of the I tones was varied independently of their periods: The I tones could have the same harmonic content as the test tones, or a very different harmonic content. Subjects' performances were much better when the periods of the I tones were remote from p than when they were close to p, as expected from previous findings by D. Deutsch [e.g., Science 175, 1020-1022 (1972)]. But, more importantly, the relation between the spectral contents of the I tones and the test tones had, by itself, practically no effect on performance. Thus performance was affected by the pitches of the I tones, but not by their timbres. These results suggest that pitch is processed independently of timbre in auditory short-term memory.     

Ipsilateral masking between acoustic and electric stimulations

Residual acoustic hearing can be preserved in the same ear following cochlear implantation with minimally traumatic surgical techniques and short-electrode arrays. The combined electric-acoustic stimulation significantly improves cochlear implant performance, particularly speech recognition in noise. The present study measures simultaneous masking by electric pulses on acoustic pure tones, or vice versa, to investigate electric-acoustic interactions and their underlying psychophysical mechanisms. Six subjects, with acoustic hearing preserved at low frequencies in their implanted ear, participated in the study. One subject had a fully inserted 24 mm Nucleus Freedom array and five subjects had Iowa/Nucleus hybrid implants that were only 10 mm in length. Electric masking data of the long-electrode subject showed that stimulation from the most apical electrodes produced threshold elevations over 10 dB for 500, 625, and 750 Hz probe tones, but no elevation for 125 and 250 Hz tones. On the contrary, electric stimulation did not produce any electric masking in the short-electrode subjects. In the acoustic masking experiment, 125-750 Hz pure tones were used to acoustically mask electric stimulation. The acoustic masking results showed that, independent of pure tone frequency, both long- and short-electrode subjects showed threshold elevations at apical and basal electrodes. The present results can be interpreted in terms of underlying physiological mechanisms related to either place-dependent peripheral masking or place-independent central masking.     

Accuracy of pitch matching for pure tones and for complex tones with overlapping or nonoverlapping harmonics.

The discrimination of the fundamental frequency (fo) of pairs of complex tones with no common harmonics is worse than the discrimination of fo for tones with all harmonics in common. These experiments were conducted to assess whether this effect is a result of pitch shifts between pairs of tones without common harmonics or whether it reflects influences of spectral differences (timbre) on the accuracy of pitch perception. In experiment 1, pitch matches were obtained between sounds drawn from the following types: (1) pure tones (P) with frequencies 100, 200, or 400 Hz; (2) a multiple-component complex tone, designated A, with harmonics 3, 4, 8, 9, 10, 14, 15, and fo = 100, 200, or 400 Hz; (3) A multiple-component complex tone, designated B, with harmonics 5, 6, 7, 11, 12, 13, 16, and with fo = 100, 200 or 400 Hz. The following matches were made; A vs A, B vs B, A vs P, B vs P and P vs P. Pitch shifts were found between the pure tones and the complex tones (A vs P and B vs P), but not between the A and B tones (A vs B). However, the variability of the A vs B matches was significantly greater than that of the A vs A or B vs B matches. Also, the variability of the A vs P and B vs P matches was greater than that for the A vs B matches. In a second experiment, frequency difference limens (DLCs) were measured for the A vs A, B vs B, and A vs B pairs of sounds. The DLCs were larger for the A vs B pair than for A vs A or B vs B. The results suggest that the poor frequency discrimination of tones with no common harmonics does not result from pitch shifts between the tones. Rather, it seems that spectral differences between tones interfere with judgements of their relative pitch.     

Frequency discrimination of complex tones with overlapping and non-overlapping harmonics

These experiments address the following issues. (1) When two complex tones contain different harmonics, do the differences in timbre between them impair the ability to discriminate the pitches of the tones? (2) When two complex tones have only a single component in common, and that component is the most discriminable component in each tone, is the frequency discrimination of the component affected by differences in residue pitch between the two tones? (3) How good is the pitch discrimination of complex tones with no common components when each tone contains multiple harmonics, so as to avoid ambiguity of pitch? (4) Is the pitch discrimination of complex tones with common harmonics impaired by shifting the component frequencies to nonharmonic values? In all experiments, frequency difference limens (DLCs) were measured for multiple-component complex tones, using an adaptive two-interval, two-alternative, forced-choice task. Three highly trained subjects were used. The main conclusions are as follows. (1) When two tones have the first six harmonics in common, DLCs are larger when the upper harmonics are different than when the upper harmonics are in common or are absent. It appears that differences in timbre impair DLCs. (2) Discrimination of the frequency of a single common partial in two complex tones is worse when the two tones have different residue pitches than when they have the same residue pitch. (3) DLCs for complex tones with no common harmonics are generally larger than those for complex tones with common harmonics. For the former, large individual differences occur, probably because subjects are affected differently by differences in timbre. (4) DLCs for harmonic complex tones are smaller than DLCs for complex tones in which the components are mistuned from harmonic values. This can probably be attributed to the less distinct residue pitch of the inharmonic complexes, rather than to reduced discriminability of partials. Overall, the results support the idea that DLCs for complex tones with common harmonics depend on residue pitch comparisons, rather than on comparisons of the pitches of partials.     

Influence of spectral locus and F0 changes on the pitch and timbre of complex tones.

Harmonic complex tones comprising components in different spectral regions may differ considerably in timbre. While the pitch of "residue" tones of this type has been studied extensively, their timbral properties have received little attention. Discrimination of F0 for such tones is typically poorer than for complex tones with "corresponding" harmonics [A. Faulkner, J. Acoust. Soc. Am. 78, 1993-2004 (1985)]. The F0 DLs may be higher because timbre differences impair pitch discrimination. The present experiment explores effects of changes in spectral locus and F0 of harmonic complex tones on both pitch and timbre. Six normally hearing listeners indicated if the second tone of a two-tone sequence was: (1) same, (2) higher in pitch, (3) lower in pitch, (4) same in pitch but different in "something else," (5) higher in pitch and different in "something else," or (6) lower in pitch and different in "something else" than the first. ("Something else" is assumed to represent timbre.) The tones varied in spectral loci of four equal-amplitude harmonics m, m + 1, m + 2, and m + 3 (m = 1,2,3,4,5,6) and ranged in F0 from 200 to 200 +/- 2n Hz (n = 0,1,2,4,8,16,32). Results show that changes in F0 primarily affect pitch, and changes in spectral locus primarily affect timbre. However, a change in spectral locus can also influence pitch. The direction of locus change was reported as the direction of pitch change, despite no change in F0 or changes in F0 in the opposite direction for delta F0 < or = 0-2%. This implies that listeners may be attending to the "spectral pitch" of components, or to changes in a timbral attribute like "sharpness," which are construed as changes in overall pitch in the absence of strong F0 cues. For delta F0 > or = 2%, the direction of reported pitch change accord with the direction of F0 change, but the locus change continued to be reported as a timbre change. Rather than spectral-pitch matching of corresponding components, a context-dependent spectral evaluation process is thus implied in discernment of changes in pitch and timbre. Relative magnitudes of change in derived features of the spectrum such as harmonic number and F0, and absolute features such as spectral frequencies are compared. What is called "spectral pitch," contributes to the overall pitch, but also appears to be an important dimension of the multidimensional percept, timbre.     

Thresholds for hearing mistuned partials as separate tones in harmonic complexes

When a low harmonic in a harmonic complex tone is mistuned from its harmonic value by a sufficient amount it is heard as a separate tone, standing out from the complex as a whole. This experiment estimated the degree of mistuning required for this phenomenon to occur, for complex tones with 10 or 12 equal-amplitude components (60 dB SPL per component). On each trial the subject was presented with a complex tone which either had all its partials at harmonic frequencies or had one partial mistuned from its harmonic frequency. The subject had to indicate whether he heard a single complex tone with one pitch or a complex tone plus a pure tone which did not "belong" to the complex. An adaptive procedure was used to track the degree of mistuning required to achieve a d' value of 1. Threshold was determined for each ot the first six harmonics of each complex tone. In one set of conditions stimulus duration was held constant at 410 ms, and the fundamental frequency was either 100, 200, or 400 Hz. For most conditions the thresholds fell between 1% and 3% of the harmonic frequency, depending on the subject. However, thresholds tended to be greater for the first two harmonics of the 100-Hz fundamental and, for some subjects, thresholds increased for the fifth and sixth harmonics. In a second set of conditions fundamental frequency was held constant at 200 Hz, and the duration was either 50, 110, 410, or 1610 ms. Thresholds increased by a factor of 3-5 as duration was decreased from 1610 ms to 50 ms. The results are discussed in terms of a hypothetical harmonic sieve and mechanisms for the formation of perceptual streams.     

Testing the concept of softness imperception: loudness near threshold for hearing-impaired ears

Buus and Florentine [J. Assoc. Res. Otolaryngol. 3, 120-139 (2002)] have proposed that loudness recruitment in cases of cochlear hearing loss is caused partly by an abnormally large loudness at absolute threshold. This has been called "softness imperception." To evaluate this idea, loudness-matching functions were obtained using tones at very low sensation levels. For subjects with asymmetrical hearing loss, matches were obtained for a single frequency across ears. For subjects with sloping hearing loss, matches were obtained between tones at two frequencies, one where the absolute threshold was nearly normal and one where there was a moderate hearing loss. Loudness matching was possible for sensation levels (SLs) as low as 2 dB. When the fixed tone was presented at a very low SL in an ear (or at a frequency) where there was hearing impairment, it was matched by a tone with approximately the same SL in an ear (or at a frequency) where hearing was normal (e.g., 2 dB SL matched 2 dB SL). This relationship held for SLs up to 4-10 dB, depending on the subject. These results are not consistent with the concept of softness imperception.     

Perceptual components of pitch: spatial representation using a multidimensional scaling technique

Paired comparison experiments were carried out on the pitch of 18 computer-generated complex tones synthesized by slight modification of the frequency structure of Shepard's endless scale sounds. The results were analyzed by a multidimensional scaling technique, and simple helixes were obtained in which two components of pitch (tone height and tone chroma) were represented. Some individual differences in perception of pitch were observed according to the weight that the subject gives to each component.     

On the binding of successive sounds: perceiving shifts in nonperceived pitches

It is difficult to hear out individually the components of a "chord" of equal-amplitude pure tones with synchronous onsets and offsets. In the present study, this was confirmed using 300-ms random (inharmonic) chords with components at least 1/2 octave apart. Following each chord, after a variable silent delay, listeners were presented with a single pure tone which was either identical to one component of the chord or halfway in frequency between two components. These two types of sequence could not be reliably discriminated from each other. However, it was also found that if the single tone following the chord was instead slightly (e.g., 1/12 octave) lower or higher in frequency than one of its components, the same listeners were sensitive to this relation. They could perceive a pitch shift in the corresponding direction. Thus, it is possible to perceive a shift in a nonperceived frequency/pitch. This paradoxical phenomenon provides psychophysical evidence for the existence of automatic "frequency-shift detectors" in the human auditory system. The data reported here suggest that such detectors operate at an early stage of auditory scene analysis but can be activated by a pair of sounds separated by a few seconds.     

Beneficial acoustic speech cues for cochlear implant users with residual acoustic hearing

This study investigated which acoustic cues within the speech signal are responsible for bimodal speech perception benefit. Seven cochlear implant (CI) users with usable residual hearing at low frequencies in the non-implanted ear participated. Sentence tests were performed in near-quiet (some noise on the CI side to reduce scores from ceiling) and in a modulated noise background, with the implant alone and with the addition, in the hearing ear, of one of four types of acoustic signals derived from the same sentences: (1) a complex tone modulated by the fundamental frequency (F0) and amplitude envelope contours; (2) a pure tone modulated by the F0 and amplitude contours; (3) a noise-vocoded signal; (4) unprocessed speech. The modulated tones provided F0 information without spectral shape information, whilst the vocoded signal presented spectral shape information without F0 information. For the group as a whole, only the unprocessed speech condition provided significant benefit over implant-alone scores, in both near-quiet and noise. This suggests that, on average, F0 or spectral cues in isolation provided limited benefit for these subjects in the tested listening conditions, and that the significant benefit observed in the full-signal condition was derived from implantees' use of a combination of these cues.