Reduced contribution of a nonsimultaneous mistuned harmonic to residue pitch

Ciocca and Darwin [V. Ciocca and C. J. Darwin, J. Acoust. Soc. Am. 105, 2421-2430 (1999)] reported that the shift in residue pitch caused by mistuning a single harmonic (the fourth out of the first 12) was the same when the mistuned harmonic was presented after the remainder of the complex as when it was simultaneous, even though subjects were asked to ignore the pure-tone percept. The present study tried to replicate this result, and investigated the role of the presence of the nominally mistuned harmonic in the matching sound. Subjects adjusted a "matching" sound so that its pitch equaled that of a subsequent 90-ms complex tone (12 harmonics of a 155-Hz F0), whose mistuned (+/-3%) third harmonic was presented either simultaneously with or after the remaining harmonics. In experiment 1, the matching sound was a harmonic complex whose third harmonic was either present or absent. In experiments 2A and 2B, the target and matching sound had nonoverlapping spectra. Pitch shifts were reduced both when the mistuned component was nonsimultaneous, and when the third harmonic was absent in the matching sound. The results indicate a shorter than originally estimated time window for obligatory integration of nonsimultaneous components into a virtual pitch.     

Grouping in pitch perception: effects of onset asynchrony and ear of presentation of a mistuned component.

Three experiments investigated how the onset asynchrony and ear of presentation of a single mistuned frequency component influence its contribution to the pitch of an otherwise harmonic complex tone. Subjects matched the pitch of the target complex by adjusting the pitch of a second similar but strictly periodic complex tone. When the mistuned component (the 4th harmonic of a 155 Hz fundamental) started 160 ms or more before the remaining harmonics but stopped simultaneously with them, it made a reduced contribution to the pitch of the complex. It made no contribution if it started more than 300 ms before. Pitch shifts and their reduction with onset time were larger for short (90 ms) sounds than for long (410 ms). Pitch shifts were slightly larger when the mistuned component was presented to the same ear as the remaining 11 in-tune harmonics than to the opposite ear. Adding a "captor" complex tone with a fundamental of 200 Hz and a missing 3rd harmonic to the contralateral ear did not augment the effect of onset time, even though the captor was synchronous with the mistuned harmonic, the mistuned component was equal in frequency to the missing 3rd harmonic of the captor complex tone and it was played to the same ear as the captor. The results show that a difference in onset time can prevent a resolved frequency component from contributing to the pitch of a complex tone even though it is present throughout that complex tone.     

Effects of asynchrony and ear of presentation on the pitch of mistuned partials in harmonic and frequency-shifted complex tones

When a partial of a periodic complex is mistuned, its change in pitch is greater than expected. Two experiments examined whether these partial-pitch shifts are related to the computation of global pitch. In experiment 1, stimuli were either harmonic or frequency-shifted (25% of F0) complexes. One partial was mistuned by +/- 4% and played with leading and lagging portions of 500 ms each, relative to the other components (1 s), in both monaural and dichotic contexts. Subjects indicated whether the mistuned partial was higher or lower in pitch when concurrent with the other components. Responses were positively correlated with the direction of mistuning in all conditions. In experiment 2, stimuli from each condition were compared with synchronous equivalents. Subjects matched a pure tone to the pitch of the mistuned partial (component 4). The results showed that partial-pitch shifts are not reduced in size by asynchrony. Similar asynchronies are known to produce a near-exclusion of a mistuned partial from the global-pitch computation. This mismatch indicates that global and partial pitch are derived from different processes. The similarity of the partial-pitch shifts observed for harmonic and frequency-shifted stimuli suggests that they arise from a grouping mechanism that is sensitive to spectral regularity.     

Evidence against an effect of grouping by spectral regularity on the perception of virtual pitch

Two experiments investigated the role of the regularity of the frequency spacing of harmonics, as a separate factor from harmonicity, on the perception of the virtual pitch of a harmonic series. The first experiment compared the shifts produced by mistuning the 3rd, 4th, and 5th harmonics in the pitch of two harmonic series: the odd-H and the all-H tones. The odd-H tone contained odd harmonics 1 to 11, plus the 4th harmonic; the all-H tone contained harmonics 1 to 12. Both tones had a fundamental frequency of 155 Hz. Pitch shifts produced by mistuning the 3rd harmonic, but not the 4th and 5th harmonics, were found to be significantly larger for the odd-H tone than for the all-H tone. This finding was consistent with the idea that grouping by spectral regularity affects pitch perception since an odd harmonic made a larger contribution than an adjacent even harmonic to the pitch of the odd-H tone. However, an alternative explanation was that the 3rd mistuned harmonic produced larger pitch shifts within the odd-H tone than the 4th mistuned harmonic because of differences in the partial masking of these harmonics by adjacent harmonics. The second experiment tested these explanations by measuring pitch shifts for a modified all-H tone in which each mistuned odd harmonic was tested in the presence of the 4th harmonic, but in the absence of its other even-numbered neighbor. The results showed that, for all mistuned harmonics, pitch shifts for the modified all-H tone were not significantly different from those for the odd-H tone. These findings suggest that the harmonic relations among frequency components, rather than the regularity of their frequency spacing, is the primary factor for the perception of the virtual pitch of complex sounds.     

Perceptual segregation and pitch shifts of mistuned components in harmonic complexes and in regular inharmonic complexes.

It is unclear whether the perceptual segregation of a mistuned harmonic from a periodic complex tone depends specifically on harmonic relations between the other components. A procedure used previously for harmonic complexes [W. M. Hartmann et al., J. Acoust. Soc. Am. 88, 1712-1724 (1990)] was adapted and extended to regular inharmonic complexes. On each trial, subjects heard a 12-component complex followed by a pure tone in a continuous loop. In experiment 1, a mistuning of +/- 4% was applied to one of the components 2-11. The complex was either harmonic, frequency shifted, or spectrally stretched. Subjects adjusted the pure tone to match the pitch of the mistuned component. Near matches were taken to indicate segregation, and were almost as frequent in the inharmonic conditions as in the harmonic case. Also, small but consistent mismatches, pitch shifts, were found in all conditions. These were similar in direction and size to earlier findings for harmonic complexes. Using a range of mistunings, experiment 2 showed that the segregation of components from regular inharmonic complexes could be sensitive to mistunings of 1.5% or less. These findings are consistent with the proposal that aspects of spectral regularity other than harmonic relations can also influence auditory grouping.     

Thresholds for hearing mistuned partials as separate tones in harmonic complexes

When a low harmonic in a harmonic complex tone is mistuned from its harmonic value by a sufficient amount it is heard as a separate tone, standing out from the complex as a whole. This experiment estimated the degree of mistuning required for this phenomenon to occur, for complex tones with 10 or 12 equal-amplitude components (60 dB SPL per component). On each trial the subject was presented with a complex tone which either had all its partials at harmonic frequencies or had one partial mistuned from its harmonic frequency. The subject had to indicate whether he heard a single complex tone with one pitch or a complex tone plus a pure tone which did not "belong" to the complex. An adaptive procedure was used to track the degree of mistuning required to achieve a d' value of 1. Threshold was determined for each ot the first six harmonics of each complex tone. In one set of conditions stimulus duration was held constant at 410 ms, and the fundamental frequency was either 100, 200, or 400 Hz. For most conditions the thresholds fell between 1% and 3% of the harmonic frequency, depending on the subject. However, thresholds tended to be greater for the first two harmonics of the 100-Hz fundamental and, for some subjects, thresholds increased for the fifth and sixth harmonics. In a second set of conditions fundamental frequency was held constant at 200 Hz, and the duration was either 50, 110, 410, or 1610 ms. Thresholds increased by a factor of 3-5 as duration was decreased from 1610 ms to 50 ms. The results are discussed in terms of a hypothetical harmonic sieve and mechanisms for the formation of perceptual streams.     

Hearing a mistuned harmonic in an otherwise periodic complex tone

The ability of a listener to detect a mistuned harmonic in an otherwise periodic tone is representative of the capacity to segregate auditory entities on the basis of steady-state signal cues. By use of a task in which listeners matched the pitch of a mistuned harmonic, this ability has been studied, in order to find dependences on mistuned harmonic number, fundamental frequency, signal level, and signal duration. The results considerably augment the data previously obtained from discrimination experiments and from experiments in which listeners counted apparent sources. Although previous work has emphasized the role of spectral resolution in the segregation process, the present work suggests that neural synchrony is an important consideration; our data show that listeners lose the ability to segregate mistuned harmonics at high frequencies where synchronous neural firing vanishes. The functional form of this loss is insensitive to the spacing of the harmonics. The matching experiment also permits the measurement of the pitches of mistuned harmonics. The data exhibit shifts of a form that argues against models of pitch shifts that are based entirely upon partial masking.     

Virtual pitch integration for asynchronous harmonics

This experiment examined the generation of virtual pitch for harmonically related tones that do not overlap in time. The interval between successive tones was systematically varied in order to gauge the integration period for virtual pitch. A pitch discrimination task was employed, and both harmonic and nonharmonic tone series were tested. The results confirmed that a virtual pitch can be generated by a series of brief, harmonically related tones that are separated in time. Robust virtual pitch information can be derived for intervals between successive 40-ms tones of up to about 45 ms, consistent with a minimum estimate of integration period of about 210 ms. Beyond intertone intervals of 45 ms, performance becomes more variable and approaches an upper limit where discrimination of tone sequences can be undertaken on the basis of the individual frequency components. The individual differences observed in this experiment suggest that the ability to derive a salient virtual pitch varies across listeners.     

Processing simultaneous auditory objects: infants' ability to detect mistuning in harmonic complexes

The ability to separate simultaneous auditory objects is crucial to infant auditory development. Music in particular relies on the ability to separate musical notes, chords, and melodic lines. Little research addresses how infants process simultaneous sounds. The present study used a conditioned head-turn procedure to examine whether 6-month-old infants are able to discriminate a complex tone (240 Hz, 500 ms, six harmonics in random phase with a 6 dB roll-off per octave) from a version with the third harmonic mistuned. Adults perceive such stimuli as containing two auditory objects, one with the pitch of the mistuned harmonic and the other with pitch corresponding to the fundamental of the complex tone. Adult thresholds were between 1% and 2% mistuning. Infants performed above chance levels for 8%, 6%, and 4% mistunings, with no significant difference between conditions. However, performance was not significantly different from chance for 2% mistuning and significantly worse for 2% compared to all larger mistunings. These results indicate that 6-month-old infants are sensitive to violations of harmonic structure and suggest that they are able to separate two simultaneously sounding objects.     

Neural activity associated with distinguishing concurrent auditory objects

The neural processes underlying concurrent sound segregation were examined by using event-related brain potentials. Participants were presented with complex sounds comprised of multiple harmonics, one of which could be mistuned so that it was no longer an integer multiple of the fundamental. In separate blocks of trials, short-, middle-, and long-duration sounds were presented and participants indicated whether they heard one sound (i.e., buzz) or two sounds (i.e., buzz plus another sound with a pure-tone quality). The auditory stimuli were also presented while participants watched a silent movie in order to evaluate the extent to which the mistuned harmonic could be automatically detected. The perception of the mistuned harmonic as a separate sound was associated with a biphasic negative-positive potential that peaked at about 150 and 350 ms after sound onset, respectively. Long duration sounds also elicited a sustained potential that was greater in amplitude when the mistuned harmonic was perceptually segregated from the complex sound. The early negative wave, referred to as the object-related negativity (ORN), was present during both active and passive listening, whereas the positive wave and the mistuning-related changes in sustained potentials were present only when participants attended to the stimuli. These results are consistent with a two-stage model of auditory scene analysis in which the acoustic wave is automatically decomposed into perceptual groups that can be identified by higher executive functions. The ORN and the positive waves were little affected by sound duration, indicating that concurrent sound segregation depends on transient neural responses elicited by the discrepancy between the mistuned harmonic and the harmonic frequency expected based on the fundamental frequency of the incoming stimulus.     

Pitch shifts of mistuned partials: a time-domain model.

Mistuning one partial of a complex harmonic tone makes that partial easier to hear as a tone separate from the complex. At the same time, two pitch shifts may be observed. First, the low pitch of the complex is shifted in the direction of the mistuning, as if it were "pulled" by the partial. Second, the mistuning of the partial is perceptually exaggerated, as if the pitch of the partial were "pushed" away from the harmonic series defined by the complex. This paper shows how the latter effect can emerge within a hypothetical neural circuit. The circuit involves a gating neuron fed by three pathways, one direct and excitatory and the other two delayed and inhibitory. The neuron responds to any excitatory input spike unless it is accompanied by an inhibitory input spike on either delayed input, thus acting as a kind of "anticoincidence counter." The first delay is fixed and tuned to the period of the background harmonic complex. Its purpose is to weaken correlates of in-tune components and allow the mistuned partial to stand out. The second delay is variable and used to estimate the period of the mistuned partial, by searching for a minimum output as a function of delay. With an appropriate choice of parameters, the estimate is subject to shifts that are of the same sign as the mistuning and that peak at about 4% mistuning and decrease beyond, as observed experimentally.     

Effects of signal envelope on the pitch of short complex tones

Envelope-induced pitch shifts were measured for exponentially decaying complex tones consisting of two sinusoidal components with frequencies f1 = nf0 + 50 Hz and f2 = (n + 1) f0 + 50 Hz, where n equals 3, 4, or 5 and exponential decay rates were 0, 0.5, 1, and 2 dB/ms. Four subjects adjusted a sinusoidal comparison tone to match the virtual pitch of the (missing) fundamental and the pitches of the lower and upper partials f1 and f2. Pitch shifts for f1 are generally less, and pitch shifts for f2 always greater, than envelope-induced shifts observed in isolated sinusoidal tones of comparable frequency and envelope decay rate. Pitch-shift functions for virtual pitch are similar in magnitude and shape to average pitch-shift functions of the partials, which supports the idea that virtual pitch depends on spectral pitch.     

Thresholds for the detection of inharmonicity in complex tones

Thresholds were measured for the detection of inharmonicity in complex tones. Subjects were required to distinguish a complex tone whose partials were all at exact harmonic frequencies from a similar complex tone with one of the partials slightly mistuned. The mistuning which allowed 71% correct identification in a two-alternative forced-choice task was estimated for each partial in turn. In experiment I the fundamental frequency was either 100, 200, or 400 Hz, and the complex tones contained the first 12 harmonics at equal levels of 60 dB SPL per component. The stimulus duration was 410 ms. For each fundamental the thresholds were roughly constant when expressed in Hz, having a mean value of about 4 Hz (range 2.4-7.3 Hz). In experiment II the fundamental frequency was fixed at 200 Hz, and thresholds for inharmonicity were measured for stimulus durations of 50, 110, 410, and 1610 ms. For harmonics above the fifth the thresholds increased from less than 1 Hz to about 40 Hz as duration was decreased from 1610-50 ms. For the lower harmonics (up to the fourth) threshold changed much less with duration, and for the three shorter durations thresholds for each duration were roughly a constant proportion of the harmonic frequency. The results suggest that inharmonicity is detected in different ways for high and low harmonics. For low harmonics the inharmonic partial appears to "stand out" from the complex tone as a whole. For high harmonics the mistuning is detected as a kind of "beat" or "roughness," presumably reflecting a sensitivity to the changing relative phase of the mistuned harmonic relative to the other harmonics.(ABSTRACT TRUNCATED AT 250 WORDS)     

Object-related brain potentials associated with the perceptual segregation of a dichotically embedded pitch

The cortical mechanisms of perceptual segregation of concurrent sound sources were examined, based on binaural detection of interaural timing differences. Auditory event-related potentials were measured from 11 healthy subjects. Binaural stimuli were created by introducing a dichotic delay of 500-ms duration to a narrow frequency region within a broadband noise, and resulted in a perception of a centrally located noise and a right-lateralized pitch (dichotic pitch). In separate listening conditions, subjects actively discriminated and responded to randomly interleaved binaural and control stimuli, or ignored random stimuli while watching silent cartoons. In a third listening condition subjects ignored stimuli presented in homogenous blocks. For all listening conditions, the dichotic pitch stimulus elicited an object-related negativity (ORN) at a latency of about 150-250 ms after stimulus onset. When subjects were required to actively respond to stimuli, the ORN was followed by a P400 wave with a latency of about 320-420 ms. These results support and extend a two-stage model of auditory scene analysis in which acoustic streams are automatically parsed into component sound sources based on source-relevant cues, followed by a controlled process involving identification and generation of a behavioral response.     

Pitch matches between unresolved complex tones differing by a single interpulse interval

The experiment compared the pitches of complex tones consisting of unresolved harmonics. The fundamental frequency (F0) of the tones was 250 Hz and the harmonics were bandpass filtered between 5500 and 7500 Hz. Two 20-ms complex-tone bursts were presented, separated by a brief gap. The gap was an integer number of periods of the waveform: 0, 4, or 8 ms. The envelope phase of the second tone burst was shifted, such that the interpulse interval (IPI) across the gap was reduced or increased by 0.25 or 0.75 periods (1 or 3 ms). A "no shift" control was also included, where the IPI was held at an integer number of periods. Pitch matches were obtained by varying the F0 of a comparison tone with the same temporal parameters as the standard but without the shift. Relative to the no-shift control, the variations in IPI produced substantial pitch shifts when there was no gap between the bursts, but little effect was seen for gaps of 4 or 8 ms. However, for some conditions with the same IPI in the shifted interval, an increase in the IPI of the comparison interval from 4 to 8 ms (gap increased from 0 to 4 ms) changed the pitch match. The presence of a pitch shift suggests that the pitch mechanism is integrating information across the two tone bursts. It is argued that the results are consistent with a pitch mechanism employing a long integration time for continuous stimuli that is reset in response to temporal discontinuities. For a 250-Hz F0, an 8-ms IPI may be sufficient for resetting. Pitch models based on a spectral analysis of the simulated neural spike train, on an autocorrelation of the spike train, and on the mean rate of pitch pulses, all failed to account for the observed pitch matches.     

Frequency discrimination of complex tones with overlapping and non-overlapping harmonics

These experiments address the following issues. (1) When two complex tones contain different harmonics, do the differences in timbre between them impair the ability to discriminate the pitches of the tones? (2) When two complex tones have only a single component in common, and that component is the most discriminable component in each tone, is the frequency discrimination of the component affected by differences in residue pitch between the two tones? (3) How good is the pitch discrimination of complex tones with no common components when each tone contains multiple harmonics, so as to avoid ambiguity of pitch? (4) Is the pitch discrimination of complex tones with common harmonics impaired by shifting the component frequencies to nonharmonic values? In all experiments, frequency difference limens (DLCs) were measured for multiple-component complex tones, using an adaptive two-interval, two-alternative, forced-choice task. Three highly trained subjects were used. The main conclusions are as follows. (1) When two tones have the first six harmonics in common, DLCs are larger when the upper harmonics are different than when the upper harmonics are in common or are absent. It appears that differences in timbre impair DLCs. (2) Discrimination of the frequency of a single common partial in two complex tones is worse when the two tones have different residue pitches than when they have the same residue pitch. (3) DLCs for complex tones with no common harmonics are generally larger than those for complex tones with common harmonics. For the former, large individual differences occur, probably because subjects are affected differently by differences in timbre. (4) DLCs for harmonic complex tones are smaller than DLCs for complex tones in which the components are mistuned from harmonic values. This can probably be attributed to the less distinct residue pitch of the inharmonic complexes, rather than to reduced discriminability of partials. Overall, the results support the idea that DLCs for complex tones with common harmonics depend on residue pitch comparisons, rather than on comparisons of the pitches of partials.     

Test of a model of auditory object formation using intensity and interaural time difference discrimination.

A model of auditory object formation and an experimental evaluation of the model are described. Specifically, predictions for intensity discrimination and interaural time difference discrimination for the central component of a three-component harmonic complex are evaluated empirically. The onset time of the central, target component is varied relative to the onset times of the remaining, interferer components in order to vary the degree of fusion (versus perceptual segregation) of the target and the interferers. The model, which is based on the idea of attenuation of the components in the nonattended auditory image (in the case of segregated images), predicts lower sensitivity to information at the target component for the fused versus segregated target, and equal sensitivities for completely segregated targets and targets presented in isolation. Results are presented for four subjects with component frequencies of 400, 600, and 800 Hz and with onset time differences of 0 or 250 ms. The target duration was always 100 ms and offset times were the same for all components. The subjective results were as expected, with synchronous onsets yielding one sound object, and asynchrony of the central component yielding two sound objects. Also, the empirical results on interference in the synchronous case were in qualitative agreement with the above predictions. However, significantly more interference was found than was predicted for both synchronous and asynchronous conditions. In fact, the amount of interference found contradicts the simple attenuation model of object formation.     

Speech masking. I. Simultaneous and nonsimultaneous masking within stop /d/ and flap closures

This article reports on investigations of the relative roles of simultaneous and nonsimultaneous masking on detection thresholds using natural speech utterances. Thresholds were obtained for 15-ms probe tones placed in the stop or flap closures of /ada/ and /idi/. Threshold elevations due to simultaneous and nonsimultaneous masking could be explained by the dynamics of neighboring speech spectra. Nonsimultaneous effects were related to spectra at least 30 ms around the probe tone. Although simultaneous masking is usually stronger than nonsimultaneous masking, the relative amplitude of adjacent speech segments in natural speech is sufficiently high near formant regions to cause noticeable effects of nonsimultaneous masking.     

The influence of duration on the perception of pitch in single and simultaneous complex tones

The influence of duration on the virtual pitch of complex tones was measured using an absolute identification paradigm. If performance with two-tone complexes is expressed in terms of a single central frequency-coding noise function, this function is found to depend on duration in about the same way as the pure-tone difference limen function. The function is further found to be a reasonably good predictor of pitch identification performance with multitone complexes. Another experimental finding was that subjects tend to switch to the analytic mode of pitch perception when complex tones are shortened (i.e., they tend to hear the spectral pitches instead of the virtual ones). A third finding was that with simultaneous complex tones the degradation of each pitch percept depends not only on duration and harmonic order of the tone but also on the harmonic order of the other tone.     

Difference limens for phase in normal and hearing-impaired subjects

These experiments measure the ability to detect a change in the relative phase of a single component in a harmonic complex tone. Complex tones containing the first 20 harmonics of 50, 100, or 200 Hz, all at equal amplitude, were used. All of the harmonics except one started in cosine phase. The remaining harmonic started in cosine phase, but was shifted in phase half-way through either the first or the second of the two stimuli comprising a trial. The subject had to identify the stimulus containing the phase-shifted component. For normally hearing subjects tested at a level of 70 dB SPL per component, thresholds for detecting the phase shift [i.e., phase difference limens (DLs)] were smallest (2 degrees-4 degrees) for harmonics above the eighth and for the lowest fundamental frequency (F0). Changes in phase were not detectable for harmonic numbers below three or four at the lowest F0 and below 5-13 at the highest F0. The DLs increased slightly for the highest harmonics in the complexes. The DLs increased markedly with decreasing level, except for the highest harmonic, where only a small effect of level was found. Subjects reported that the phase-shifted harmonic appeared to "pop out" and was heard with a pure-tone quality. A pitch-matching experiment demonstrated that the pitch of this tone corresponded to the frequency of the phase-shifted component. For the highest harmonic, the phase shift was associated with a downward shift of the edge pitch heard in the reference (all cosine phase) stimulus. When the phases of the components in the reference stimulus were randomized, phase DLs were much higher (and often impossible to measure), the pop-out phenomenon was not observed, and no edge pitch was heard. Subjects with unilateral cochlear hearing impairment generally showed poorer phase sensitivity in their impaired than in their normal ears, when the two ears were compared at equal sound-pressure levels. However, at comparable sensation levels, the impaired ears sometimes showed lower phase DLs. The results are explained by considering the waveforms that would occur at the outputs of the auditory filters in response to these stimuli.