Localization of sound in rooms. IV: The Franssen effect

The Franssen effect is an illusion that causes human listeners to make large errors in localizing a sound source. This paper describes steps taken to convert the illusion into an experiment in order to study the localization precedence effect as it operates in rooms. The results of the experiment suggest that there are two components to the illusion: The first is the inability of listeners to localize a sine tone in a room in the absence of an onset; the second is the obscuring of modulation cues by the irregular transient response of a room. Experiments show that the Franssen effect fails completely in an anechoic environment, as expected if the effect depends upon the implausibility of steady-state cues in a room. The Franssen effect also fails when the spectrum of the sound is dense.     

Localization of sound in rooms

This paper is concerned with the localization of sources of sounds by human listeners in rooms. It presents the results of source-identification experiments designed to determine whether the ability to localize sound in a room depends upon the room acoustics, and how it depends upon the nature of the source signal. The experiments indicate that the localization of impulsive sounds, with strong attack transients, is independent of the room reverberation time, though it may depend upon the room geometry. For sounds without attack transients, localization improves monotonically with the spectral density of the source. Localization of continuous broadband noise does depend upon room reverberation time, and we propose the concept of direct signal to reverberant noise ratio to study that effect. Source identification experiments reveal certain localization biases, invisible to minimum-audible-angle experiments, and of uncertain origin. Appendices to this paper develop the statistics of the source-identification paradigm and show how they relate to the minimum audible angle.     

Auditory constraints on the perception of voice-onset time: the influence of lower tone frequency on judgments of tone-onset simultaneity

The experiments reported employed nonspeech analogs of speech stimuli to examine the perceptual interaction between first-formant onset frequency and voice-onset time, acoustic cues to the voicing distinction in English initial stop consonants. The nonspeech stimuli comprised two pure tones varying in relative onset time, and listeners were asked to judge the simultaneity of tone onsets. These judgments were affected by the frequency of the lower tone in a manner that parallels the influence of first-formant onset frequency on voicing judgments. This effect was shown to occur regardless of prior learning and to be systematic over a wide range of lower tone frequencies including frequencies beyond the range of possible first-formant frequencies of speech, suggesting that the effect in speech is not attributable to (tacit) knowledge of production constraints, as some current theories suggest.     

Localization dominance in the median-sagittal plane: effect of stimulus duration

Localization dominance is an aspect of the precedence effect (PE) in which the leading source dominates the perceived location of a simulated echo (lagging source). It is known to be robust in the horizontal/azimuthal dimension, where binaural cues dominate localization. However, little is known about localization dominance in conditions that minimize binaural cues, and most models of precedence treat the phenomena as "belonging" to the binaural system. Here, localization dominance in the median-sagittal plane was studied where binaural cues are greatly reduced, and monaural spectral/level cues are thought to be the primary cues used for localization. Lead-lag pairs of noise bursts were presented from locations spaced in 15 degrees increments in the frontal, median-sagittal plane, with a 2-ms delay in their onsets, for source durations of 1, 10, 25, and 50-ms. Intermixed with these trials were single-speaker trials, in which lead and lag were summed and presented from one speaker. Listeners identified the speaker that was nearest to the perceived source location. With single-speaker stimuli, localization improves as signal duration is increased. Furthermore, evidence of elevation compression was found with a dependence on duration. With lead-lag pairs, localization dominance occurs in the median plane, and becomes more robust with increased signal duration. These results suggest that accurate localization of a co-located lead-lag pair is necessary for localization dominance to occur when the lag is spatially separated from the lead.     

Investigation of the relationship among three common measures of precedence: fusion, localization dominance, and discrimination suppression

Listeners have a remarkable ability to localize and identify sound sources in reverberant environments. The term "precedence effect" (PE; also known as the "Haas effect," "law of the first wavefront," and "echo suppression") refers to a group of auditory phenomena that is thought to be related to this ability. Traditionally, three measures have been used to quantify the PE: (1) Fusion: at short delays (1-5 ms for clicks) the lead and lag perceptually fuse into one auditory event; (2) Localization dominance: the perceived location of the leading source dominates that of the lagging source; and (3) Discrimination suppression: at short delays, changes in the location or interaural parameters of the lag are difficult to discriminate compared with changes in characteristics of the lead. Little is known about the relation among these aspects of the PE, since they are rarely studied in the same listeners. In the present study, extensive measurements of these phenomena were made for six normal-hearing listeners using 1-ms noise bursts. The results suggest that, for clicks, fusion lasts 1-5 ms; by 5 ms most listeners hear two sounds on a majority of trials. However, localization dominance and discrimination suppression remain potent for delays of 10 ms or longer. Results are consistent with a simple model in which information from the lead and lag interacts perceptually and in which the strength of this interaction decreases with spatiotemporal separation of the lead and lag. At short delays, lead and lag both contribute to spatial perception, but the lead dominates (to the extent that only one position is ever heard). At the longest delays tested, two distinct sounds are perceived (as measured in a fusion task), but they are not always heard at independent spatial locations (as measured in a localization dominance task). These results suggest that directional cues from the lag are not necessarily salient for all conditions in which the lag is subjectively heard as a separate event.     

On the influence of interaural differences on temporal perception of noise bursts of different durations

The perception of a composite sound's temporal cues, like synchronous onsets, is considered essential to correct perceptual grouping of its constituent components. The processing of a single sound's spatial cues, already present at its onset, may interact with temporal perception of the onset. The current study investigated the influence of interaural differences on temporal perception of a sound's onset. As a measure of temporal perception, the ability to position the onset of a temporally displaced target sound to the regular meter of diotic reference marker sound onsets was measured for various target sound lateralizations, sensation levels, and target and marker sound durations. For target sounds presented in quiet, no influence of interaural differences on temporal positioning of the onset was found. However, increasing a sound's duration systematically shifted the perceived onset position into its "interior." For target sounds presented at low sensation levels in a noise masker, the precision of temporally positioning the onset generally degraded, though faster for dichotic conditions and for longer durations. The level below which temporal perception precision starts to degrade was found to depend on signal-to-noise ratio rather than on sensation level or duration, and is influenced by the presence of interaural differences.     

Manipulations of listeners' echo perception are reflected in event-related potentials

To gain information from complex auditory scenes, it is necessary to determine which of the many loudness, pitch, and timbre changes originate from a single source. Grouping sound into sources based on spatial information is complicated by reverberant energy bouncing off multiple surfaces and reaching the ears from directions other than the source's location. The ability to localize sounds despite these echoes has been explored with the precedence effect: Identical sounds presented from two locations with a short stimulus onset asynchrony (e.g., 1-5 ms) are perceived as a single source with a location dominated by the lead sound. Importantly, echo thresholds, the shortest onset asynchrony at which a listener reports hearing the lag sound as a separate source about half of the time, can be manipulated by presenting sound pairs in contexts. Event-related brain potentials elicited by physically identical sounds in contexts that resulted in listeners reporting either one or two sources were compared. Sound pairs perceived as two sources elicited a larger anterior negativity 100-250 ms after onset, previously termed the object-related negativity, and a larger posterior positivity 250-500 ms. These results indicate that the models of room acoustics listeners form based on recent experience with the spatiotemporal properties of sound modulate perceptual as well as later higher-level processing.     

The effect of hearing impairment on localization dominance for single-word stimuli

Localization dominance (one of the phenomena of the "precedence effect") was measured in a large number of normal-hearing and hearing-impaired individuals and related to self-reported difficulties in everyday listening. The stimuli (single words) were made-up of a "lead" followed 4 ms later by a equal-level "lag" from a different direction. The stimuli were presented from a circular ring of loudspeakers, either in quiet or in a background of spatially diffuse babble. Listeners were required to identify the loudspeaker from which they heard the sound. Localization dominance was quantified by the weighting factor c [B.G. Shinn-Cunningham et al., J. Acoust. Soc. Am. 93, 2923-2932 (1993)]. The results demonstrated large individual differences: Some listeners showed near-perfect localization dominance (c near 1) but many showed a much reduced effect. Two-thirds (64/93) of the listeners gave a value of c of at least 0.75. There was a significant correlation with hearing loss, such that better hearing listeners showed better localization dominance. One of the items of the self-report questionnaire ("Do you have the impression of sounds being exactly where you would expect them to be?") showed a significant correlation with the experimental results. This suggests that reductions in localization dominance may affect everyday auditory perception.     

Spatial cues alone produce inaccurate sound segregation: the effect of interaural time differences

To clarify the role of spatial cues in sound segregation, this study explored whether interaural time differences (ITDs) are sufficient to allow listeners to identify a novel sound source from a mixture of sources. Listeners heard mixtures of two synthetic sounds, a target and distractor, each of which possessed naturalistic spectrotemporal correlations but otherwise lacked strong grouping cues, and which contained either the same or different ITDs. When the task was to judge whether a probe sound matched a source in the preceding mixture, performance improved greatly when the same target was presented repeatedly across distinct distractors, consistent with previous results. In contrast, performance improved only slightly with ITD separation of target and distractor, even when spectrotemporal overlap between target and distractor was reduced. However, when subjects localized, rather than identified, the sources in the mixture, sources with different ITDs were reported as two sources at distinct and accurately identified locations. ITDs alone thus enable listeners to perceptually segregate mixtures of sources, but the perceived content of these sources is inaccurate when other segregation cues, such as harmonicity and common onsets and offsets, do not also promote proper source separation.     

Influence of the precedence effect on word identification by normally hearing and hearing-impaired subjects

The influence of the precedence effect on word identification was investigated binaurally and monaurally with normally hearing and hearing-impaired subjects. The Modified Rhyme Test was processed through a PDP-12 computer to produce delay times of 0, 5, 10, 20, 40, 80, or 160 ms. The sounds were reproduced in a room by two loudspeakers positioned at +/-30 degrees azimuths in front of a subject at 50 dB SPL for normals and at the most comfortable level for impaireds. A babble of eight voices was added to reduce scores about 15% from the best values measured in quiet. Binaural and monaural word identification remained constant over a range of delays from 0 to 20 ms and declined for longer delays for both groups of subjects. The shapes of the word-identification curves were explained by self-masking (an overlap of consonants with their own repetitions) and masking (an overlap of consonants with preceding vowels or preceding and following words in sentence). Binaural responses for ten selected initial and final consonants showed various patterns of perception with delay. Some hearing impaireds showed more deterioration in word identification than others which might indicate that they experience more perceptual difficulties than normal listeners in places with reverberation or sound amplification.     

Hearing a mistuned harmonic in an otherwise periodic complex tone

The ability of a listener to detect a mistuned harmonic in an otherwise periodic tone is representative of the capacity to segregate auditory entities on the basis of steady-state signal cues. By use of a task in which listeners matched the pitch of a mistuned harmonic, this ability has been studied, in order to find dependences on mistuned harmonic number, fundamental frequency, signal level, and signal duration. The results considerably augment the data previously obtained from discrimination experiments and from experiments in which listeners counted apparent sources. Although previous work has emphasized the role of spectral resolution in the segregation process, the present work suggests that neural synchrony is an important consideration; our data show that listeners lose the ability to segregate mistuned harmonics at high frequencies where synchronous neural firing vanishes. The functional form of this loss is insensitive to the spacing of the harmonics. The matching experiment also permits the measurement of the pitches of mistuned harmonics. The data exhibit shifts of a form that argues against models of pitch shifts that are based entirely upon partial masking.     

Acoustic cues to tonal contrasts in Mandarin: implications for cochlear implants

The present study systematically manipulated three acoustic cues--fundamental frequency (f0), amplitude envelope, and duration--to investigate their contributions to tonal contrasts in Mandarin. Simplified stimuli with all possible combinations of these three cues were presented for identification to eight normal-hearing listeners, all native speakers of Mandarin from Taiwan. The f0 information was conveyed either by an f0-controlled sawtooth carrier or a modulated noise so as to compare the performance achievable by a clear indication of voice f0 and what is possible with purely temporal coding of f0. Tone recognition performance with explicit f0 was much better than that with any combination of other acoustic cues (consistently greater than 90% correct compared to 33%-65%; chance is 25%). In the absence of explicit f0, the temporal coding of f0 and amplitude envelope both contributed somewhat to tone recognition, while duration had only a marginal effect. Performance based on these secondary cues varied greatly across listeners. These results explain the relatively poor perception of tone in cochlear implant users, given that cochlear implants currently provide only weak cues to f0, so that users must rely upon the purely temporal (and secondary) features for the perception of tone.     

The influence of early reflections on the identification and lateralization of vowels

Sound coming directly from a source is often accompanied by reflections arriving from different directions. However, the "precedence effect" occurs when listeners judge such a source's direction: information in the direct, first-arriving sound tends to govern the direction heard for the overall sound. This paper asks whether the spectral envelope of the direct sound has a similar, dominant influence on the spectral envelope perceived for the whole sound. A continuum between two vowels was produced and then a "two-part" filter distorted each step. The beginning of this filter's unit-sample response simulated a direct sound with no distortion of the spectral envelope. The second part simulated a reflection pattern that distorted the spectral envelope. The reflections' frequency response was designed to give the spectral envelope of one of the continuum's end-points to the other end-point. Listeners' identifications showed that the reflections in two-part filters had a substantial influence because sounds tended to be identified as the positive vowel of the reflection pattern. This effect was not reduced when the interaural delays of the reflections and the direct sound were substantially different. Also, when the reflections were caused to precede the direct sound, the effects were much the same. By contrast, in measurements of lateralization the precedence effect was obtained. Here, the lateral position of the whole sound was largely governed by the interaural delay of the direct sound, and was hardly affected by the interaural delay of the reflections.     

Lateralization of dichotic noise bursts: effect of onset and offset disparity

Sound image position associated with the interaural onset or offset disparity of a signal was quantified by a scaling procedure in three experiments. Lateralization cues derived from the steady-state portion of the broadband noise signal that would support a specific image position were minimized by the use of independent noise sources for each ear. Onset disparities produced lateralization toward the ear at which the sound was presented first, while offset disparity produced lateralization toward the ear at which the sound remained on longer. Disparity was systematically varied between 0 and 10 ms and for a given disparity, a greater shift in the sound image position was obtained when the disparity was at the onset rather than the offset. The duration of the shorter signal ranged from 2.5-100 ms and for either onset or offset disparity, the image of stimuli of long duration tended to remain near the center of the head, while those of shorter duration could be moved to more extreme positions. In an attempt to rule out dichotic loudness cues as a basis for the lateralization associated with offset disparity, stimuli were presented with equal energy at each ear. Image position for equal energy was virtually identical to that for equal sound pressure, suggesting that loudness differences are not mediating lateralization associated with offset disparity.     

Equal autophonic level curves under different room acoustics conditions

The indirect auditory feedback from one's own voice arises from sound reflections at the room boundaries or from sound reinforcement systems. The relative variations of indirect auditory feedback are quantified through room acoustic parameters such as the room gain and the voice support, rather than the reverberation time. Fourteen subjects matched the loudness level of their own voice (the autophonic level) to that of a constant and external reference sound, under different synthesized room acoustics conditions. The matching voice levels are used to build a set of equal autophonic level curves. These curves give an indication of the amount of variation in voice level induced by the acoustic environment as a consequence of the sidetone compensation or Lombard effect. In the range of typical rooms for speech, the variations in overall voice level that result in a constant autophonic level are on the order of 2 dB, and more than 3 dB in the 4 kHz octave band. By comparison of these curves with previous studies, it is shown that talkers use acoustic cues other than loudness to adjust their voices when speaking in different rooms.     

Temporal order discrimination of tonal sequences by younger and older adults: the role of duration and rate

The effect of tone duration and presentation rate on the discrimination of the temporal order of the middle two tones of a four-tone sequence was investigated in young normal-hearing (YNH) and older hearing-impaired (OHI) listeners. The frequencies and presentation level of the tone sequences were selected to minimize the effect of hearing loss on the performance of the OHI listeners. Tone durations varied from 20 to 400 ms and presentation rates from 2.5 to 25 toness. Two experiments were conducted with anisochronous (nonuniform duration and rate across entire sequence) and isochronous (uniform rate and duration) sequences, respectively. For the YNH listeners, performance for both isochronous and anisochronous sequences was determined primarily by presentation rate such that performance decreased at rates faster than 5 toness. For anisochronous tone sequences alone, the effects of rate were more pronounced at short tone durations. For the OHI listeners, both presentation rate and tone duration had an impact on performance for both isochronous and anisochronous sequences such that performance decreased as rate increased above 5 toness or duration decreased below 40 ms. Temporal masking was offered as an explanation for the interaction of short durations and fast rates on temporal order discrimination for the anisochronous sequences.     

Wavelet packet based analysis of sound fields in rooms using coincident microphone arrays

This paper presents a passive analysis method for determining the spatio-temporal characteristics of sound fields in small rooms. The analysis finds an approximate directional reflectogram (ADR) which reveals the approximate arrival directions, time delays and amplitudes of the direct sound and early reflections without using a special or known sound source. A coincident microphone array is used to obtain directional recordings. The recordings are analysed by wavelet packet decomposition to determine the direction of the sound source and select wavelet packet coefficients to reconstruct the estimate of the direct sound. ADR is then computed via deconvolution using this estimate. Experiments have been carried out using synthesized recordings that were obtained from actual room impulse responses measured in two rooms for various source locations. The method estimates the source direction with a mean absolute error of about 7°. Calculated ADRs provide a good estimate of the time delays and arrival directions of acoustical reflections, whereas the amplitudes differ slightly.     

On the importance of early reflections for speech in rooms

This paper presents the results of new studies based on speech intelligibility tests in simulated sound fields and analyses of impulse response measurements in rooms used for speech communication. The speech intelligibility test results confirm the importance of early reflections for achieving good conditions for speech in rooms. The addition of early reflections increased the effective signal-to-noise ratio and related speech intelligibility scores for both impaired and nonimpaired listeners. The new results also show that for common conditions where the direct sound is reduced, it is only possible to understand speech because of the presence of early reflections. Analyses of measured impulse responses in rooms intended for speech show that early reflections can increase the effective signal-to-noise ratio by up to 9 dB. A room acoustics computer model is used to demonstrate that the relative importance of early reflections can be influenced by the room acoustics design.     

The effect of construction material, contents and room geometry on the sound field in dwellings at low frequencies

An experimentally validated finite element method is used to model the sound level in rooms at low frequencies. It is demonstrated that the dimensions of rectangular rooms strongly influence the sound pressure level difference. Additional factors were investigated which are not normally considered in the frequency range where diffuse sound field conditions can be assumed. Three effects were investigated: room damping due to wall vibrations, furniture, the effect of small deviations from simple rectangular shapes. It is confirmed by field measurements that the vibrations of masonry walls and floors introduce less damping than surfaces of lightweight construction. Assigning to the FE model a damping equivalent to a surface absorption of 0.02 reproduces the effect of walls of heavyweight construction. Damping equivalent to a surface absorption of 0.15 reproduces the effects of plastered timber-frame walls, floors and ceilings. The work was briefly extended to a room pair built with heavyweight and lightweight material of construction. The modification of the shape of the room frequency response highlights well the effect of material of construction. In-situ and laboratory measurements show that furniture has little effect on steady-state room response below 100 Hz. Modelling a wall recess smaller than 0.5 m improved the agreement between prediction and measurements but the assumption of a simple rectangular room remains appropriate.     

Discrimination of rising and falling simulated single-formant frequency transitions: practice and transition duration effects

Two experiments evaluated discrimination of simulated single-format frequency transitions. In the first experiment, listeners received practice with trial-by-trial feedback in discriminating either rising or falling frequency transitions of three different durations (30, 60, and 120 ms). Transitions either occurred in isolation or were followed by a steady-state sound matched in frequency to the transition end point. Some improvement in discrimination over practice runs occurred for the shortest transitions. Whether performance was evaluated at the beginning or end of practice, there were no differences attributable to transition direction or to whether transitions were followed by steady-state sound. Discrimination, however, was significantly better for the longest transitions. Just noticeable differences (jnd's) for the longest transitions, measured in Hz at transition onsets, were of approximately the same magnitude as jnd's for steady-state sounds that were equal in frequency to the midpoints of the transitions. Subjects of the second experiment discriminated the longer rising and falling transitions, but did not receive extensive practice. Results of experiment 2 replicated results of experiment 1 in showing similar jnd's. Experiment 2 also showed no differences attributable to transition direction or to the presence of the steady-state sound following transitions.