Perceptual interactions in the loudness of combined auditory and vibrotactile stimuli

The loudness of auditory (A), tactile (T), and auditory-tactile (A+T) stimuli was measured at supra-threshold levels. Auditory stimuli were pure tones presented binaurally through headphones; tactile stimuli were sinusoids delivered through a single-channel vibrator to the left middle fingertip. All stimuli were presented together with a broadband auditory noise. The A and T stimuli were presented at levels that were matched in loudness to that of the 200-Hz auditory tone at 25 dB sensation level. The 200-Hz auditory tone was then matched in loudness to various combinations of auditory and tactile stimuli (A+T), and purely auditory stimuli (A+A). The results indicate that the matched intensity of the 200-Hz auditory tone is less when the A+T and A+A stimuli are close together in frequency than when they are separated by an octave or more. This suggests that A+T integration may operate in a manner similar to that found in auditory critical band studies, further supporting a strong frequency relationship between the auditory and somatosensory systems.     

The role of frequency selectivity in measures of auditory and vibrotactile temporal resolution.

The purpose of this study was to compare the role of frequency selectivity in measures of auditory and vibrotactile temporal resolution. In the first experiment, temporal modulation transfer functions for a sinusoidally amplitude modulated (SAM) 250-Hz carrier revealed auditory modulation thresholds significantly lower than corresponding vibrotactile modulation thresholds at SAM frequencies greater than or equal to 100 Hz. In the second experiment, auditory and vibrotactile gap detection thresholds were measured by presenting silent gaps bounded by markers of the same or different frequency. The marker frequency F1 = 250 Hz preceded the silent gap and marker frequencies after the silent gap included F2 = 250, 255, 263, 310, and 325 Hz. Auditory gap detection thresholds were lower than corresponding vibrotactile thresholds for F2 markers less than or equal to 263 Hz, but were greater than the corresponding vibrotactile gap detection thresholds for F2 markers greater than or equal to 310 Hz. When the auditory gap detection thresholds were transformed into filter attenuation values, the results were modeled well by a constant-percentage (10%) bandwidth filter centered on F1. The vibrotactile gap detection thresholds, however, were independent of marker frequency separation. In a third experiment, auditory and vibrotactile rate difference limens (RDLs) were measured for a 250-Hz carrier at SAM rates less than or equal to 100 Hz. Auditory RDLs were lower than corresponding vibrotactile RDLs for standard rates greater than 10 Hz. Combination tones may have confounded auditory performance for standard rates of 80 and 100 Hz. The results from these experiments revealed that frequency selectivity influences auditory measures of temporal resolution, but there was no evidence of frequency selectivity affecting vibrotactile temporal resolution.     

The localization of low- and high-frequency vibrotactile stimuli

Four experiments were conducted to determine whether spatial localization on the skin varied in acuity as a function of frequency of vibratory stimulation. The glabrous skin of the palm over the hypothenar eminence was selected as the site for stimulation by two frequencies, one at 25 Hz to stimulate non-Pacinian receptors, and one at 250 Hz to excite Pacinian receptors. Because the Pacinian receptors have larger receptive fields than the non-Pacinians, it was thought that the subjects' ability to localize would be poorer when the Pacinians were the class of receptor stimulated. In addition to frequency of vibration, the presence of a surround, the site of stimulation, the separation of the stimulator pair used in the 2AFC method, and the use of an impulse stimulus were all conditions varied to determine whether a simple direct correlation exists between receptor category and spatial acuity for vibratory stimuli. A significant difference in acuity was found as a function of vibration frequency at a proximal locus on the palm, but this vanished at a distal locus. The results have been interpreted to suggest that receptor density and its gradient across the skin areas involved may be as important as receptor type in the determination of spatial acuity.     

Gap detection and the auditory filter: phase effects using sinusoidal stimuli

Psychometric functions were determined for the detection of temporal gaps in sinusoidal signals at center frequencies between 0.2 and 2.0 kHz. A continuous notched-noise masker was used to restrict listening to the signal frequency region. The gap always started when the signal was at a positive-going zero crossing. There were three different conditions for the starting phase of the signal at the termination of the gap. In the standard-phase condition the signal restarted at a positive-going zero crossing, in the reversed-phase condition at a negative-going zero crossing, and in the preserved-phase condition at the phase the signal would have had if the gap had not been present. In the standard-phase and reversed-phase conditions the psychometric functions were nonmonotonic, showing oscillations with a period equal to that of the signal; maxima in the functions for the standard-phase condition coincided with minima in the functions for the reversed-phase condition, and vice versa. In the preserved-phase condition the psychometric functions were monotonic and the 75% points were roughly independent of center frequency, having a value of about 5 ms. The general form of the results can be modeled by a filter bank followed by a square-law device and a temporal integrator, but good agreement between the data and the model could not be attained across the whole range of gap durations. The deviations between data and model suggest that subjects are sensitive to the brief transitions in phase (or, equivalently, in frequency) in some conditions.     

Encoding voice fundamental frequency into vibrotactile frequency.

Measured in this study was the ability of eight hearing and five deaf subjects to identify the stress pattern in a short sentence from the variation in voice fundamental frequency (F0), when presented aurally (for hearing subjects) and when transformed into vibrotactile pulse frequency. Various transformations from F0 to pulse frequency were tested in an attempt to determine an optimum transformation, the amount of F0 information that could be transmitted, and what the limitations in the tactile channel might be. The results indicated that a one- or two-octave reduction of F0 vibrotactile frequency (transmitting every second or third glottal pulse) might result in a significant ability to discriminate the intonation patterns associated with moderate-to-strong patterns of sentence stress in English. However, accurate reception of the details of the intonation pattern may require a slower than normal pronounciation because of an apparent temporal indeterminacy of about 200 ms in the perception of variations in vibrotactile frequency. A performance deficit noted for the two prelingually, profoundly deaf subjects with marginally discriminable encodings offers some support for our previous hypothesis that there is a natural association between auditory pitch and perceived vibrotactile frequency.     

Child and adult vibrotactile thresholds for sinusoidal and pulsatile stimuli

Three experiments were performed to obtain vibrotactile sensitivity thresholds from hearing children and adults, and from deaf children. An adaptive two-interval forced-choice procedure was used to obtain estimates of the 70.7% point on the psychometric sensitivity curve. When hearing children of 5-6 and 9-10 years of age and adults were tested with sinusoids and haversine pulse stimuli, at 10, 100, 160, and 250 Hz or pps, respectively, only the 10-Hz stimulus resulted in an age effect. For this stimulus, young children were significantly less sensitive than adults. When sinusoids were again tested at 20, 40, 80, and 160 Hz, a small overall effect of age was observed with a significant effect only at 20 Hz. Two prelingually profoundly deaf children were tested with haversine pulse trains at 10, 50, 100, 160, and 250 pps. Both children were at least as sensitive to the tactile stimulation as were the hearing children and adults. Pulsatile stimulation, compared to sinusoidal stimulation, exhibited relatively flat threshold versus frequency functions. The present results, demonstrating no age effect for pulsatile stimulation and similar performance for deaf and hearing children, suggest that pulsatile stimulation would be appropriate in vibrotactile speech communication aids for the deaf.     

Coding of vibrotactile stimulus frequency by Pacinian corpuscle afferents

Psychophysical and electrophysiological techniques were used to study the encoding and processing of information about the frequency content of vibrational stimuli applied to glabrous skin in humans and cats. Trained human subjects were asked to discriminate changes in stimulus frequency and harmonic content for pairs of mono- and diharmonic sinusoidal vibrations applied to the fingertips. These psychophysical tests supplied data on what information is available to the central nervous system about the frequency components of vibratory stimuli. Electrophysiological recordings from nerves innervating the glabrous skin of the paw in cats during presentation of the same stimuli used in the psychophysical study provided data on how the peripheral nervous system encodes information about the physical parameters of cutaneous vibratory stimuli. The two sets of data indicated that the subjects derived information about the frequency of vibrotactile stimuli from the mean interval between action potentials in afferent nerve fibers activated by the stimulus.     

Speechreading sentences with single-channel vibrotactile presentation of voice fundamental frequency

The main goal of this study was to investigate the efficacy of four vibrotactile speechreading supplements. Three supplements provided single-channel encodings of fundamental frequency (F0). Two encodings involved scaling and shifting glottal pulses to pulse rate ranges suited to tactual sensing capabilities; the third transformed F0 to differential amplitude of two fixed-frequency sinewaves. The fourth supplement added to one of the F0 encodings a second vibrator indicating high-frequency speech energy. A second goal was to develop improved methods for experimental control. Therefore, a sentence corpus was recorded on videodisc using two talkers whose speech was captured by video, microphone, and electroglottograph. Other experimental control issues included use of visual-alone control subjects, a multiple-baseline, single-subject design replicated for each of 15 normal-hearing subjects, sentence and syllable pre- and post-tests balanced for difficulty, and a speechreading screening test for subject selection. Across 17 h of treatment and 5 h of visual-alone baseline testing, each subject performed open-set sentence identification. Covariance analyses showed that the single-channel supplements provided a small but significant benefit, whereas the two-channel supplement was not effective. All subjects improved in visual-alone speechreading and maintained individual differences across the experiment. Vibrotactile benefit did not depend on speechreading ability.     

Neuromagnetic steady-state responses to auditory stimuli

Steady-state magnetic responses to clicks presented at rates between 10 and 70 Hz have been recorded in healthy humans. The responses were highest in amplitude around 40 Hz. This amplitude enhancement is satisfactorily explained by summation of responses evoked by single clicks. The field maps suggest activation of the auditory cortex at all stimulus frequencies. Similar responses were obtained with gated noise bursts and by pauses in a series of clicks. The mean "apparent latency," determined from the phase lag at rates 30-70 Hz, was 54 ms. The physiological relevance of this quantity is shown to be questionable.     

Temporal and spatio-temporal vibrotactile displays for voice fundamental frequency: an initial evaluation of a new vibrotactile speech perception aid with normal-hearing and hearing-impaired individuals.

Four experiments were performed to evaluate a new wearable vibrotactile speech perception aid that extracts fundamental frequency (F0) and displays the extracted F0 as a single-channel temporal or an eight-channel spatio-temporal stimulus. Specifically, we investigated the perception of intonation (i.e., question versus statement) and emphatic stress (i.e., stress on the first, second, or third word) under Visual-Alone (VA), Visual-Tactile (VT), and Tactile-Alone (TA) conditions and compared performance using the temporal and spatio-temporal vibrotactile display. Subjects were adults with normal hearing in experiments I-III and adults with severe to profound hearing impairments in experiment IV. Both versions of the vibrotactile speech perception aid successfully conveyed intonation. Vibrotactile stress information was successfully conveyed, but vibrotactile stress information did not enhance performance in VT conditions beyond performance in VA conditions. In experiment III, which involved only intonation identification, a reliable advantage for the spatio-temporal display was obtained. Differences between subject groups were obtained for intonation identification, with more accurate VT performance by those with normal hearing. Possible effects of long-term hearing status are discussed.     

Functional MRI of working memory and selective attention in vibrotactile frequency discrimination

Background  

Focal lesions of the frontal, parietal and temporal lobe may interfere with tactile working memory and attention. To characterise the neural correlates of intact vibrotactile working memory and attention, functional MRI was conducted in 12 healthy young adults. Participants performed a forced-choice vibrotactile frequency discrimination task, comparing a cue stimulus of fixed frequency to their right thumb with a probe stimulus of identical or higher frequency. To investigate working memory, the time interval between the 2 stimuli was pseudo-randomized (either 2 or 8 s). To investigate selective attention, a distractor stimulus was occasionally presented contralaterally, simultaneous to the probe.     

Modeling auditory evoked brainstem responses to transient stimuli

A quantitative model is presented that describes the formation of auditory brainstem responses (ABRs) to tone pulses, clicks, and rising chirps as a function of stimulation level. The model computes the convolution of the instantaneous discharge rates using the "humanized" nonlinear auditory-nerve model of Zilany and Bruce [J. Acoust. Soc. Am. 122, 402-417 (2007)] and an empirically derived unitary response function which is assumed to reflect contributions from different cell populations within the auditory brainstem, recorded at a given pair of electrodes on the scalp. It is shown that the model accounts for the decrease of tone-pulse evoked wave-V latency with frequency but underestimates the level dependency of the tone-pulse as well as click-evoked latency values. Furthermore, the model correctly predicts the nonlinear wave-V amplitude behavior in response to the chirp stimulation both as a function of chirp sweeping rate and level. Overall, the results support the hypothesis that the pattern of ABR generation is strongly affected by the nonlinear and dispersive processes in the cochlea.     

Auditory stream segregation in monkey auditory cortex: effects of frequency separation, presentation rate, and tone duration

Auditory stream segregation refers to the organization of sequential sounds into "perceptual streams" reflecting individual environmental sound sources. In the present study, sequences of alternating high and low tones, "...ABAB...," similar to those used in psychoacoustic experiments on stream segregation, were presented to awake monkeys while neural activity was recorded in primary auditory cortex (A1). Tone frequency separation (AF), tone presentation rate (PR), and tone duration (TD) were systematically varied to examine whether neural responses correlate with effects of these variables on perceptual stream segregation. "A" tones were fixed at the best frequency of the recording site, while "B" tones were displaced in frequency from "A" tones by an amount = delta F. As PR increased, "B" tone responses decreased in amplitude to a greater extent than "A" tone responses, yielding neural response patterns dominated by "A" tone responses occurring at half the alternation rate. Increasing TD facilitated the differential attenuation of "B" tone responses. These findings parallel psychoacoustic data and suggest a physiological model of stream segregation whereby increasing delta F, PR, or TD enhances spatial differentiation of "A" tone and "B" tone responses along the tonotopic map in A1.     

Latency of auditory brain-stem responses and otoacoustic emissions using tone-burst stimuli

A comparison of the latency of auditory brain-stem responses (ABR) and evoked otoacoustic emissions (EOAE) has led to an interpretation for the travel of transients in the peripheral auditory system that is consistent with both sets of data. The "cochlear echo" theory for the origin of the EOAE indicates that the latency of a particular frequency component back to the ear canal should be twice the forward latency of its characteristic place in the cochlea. The latency of wave V of the ABR to tone-burst stimuli can be described as the sum of two components: (1) a component that varies with intensity and frequency in an orderly and predictable manner and (2) a component that is independent of both intensity and frequency. Because the EOAE data can be predicted by taking twice the value of component (1) of the ABR latency, this component is interpreted to be due to mechanical travel through the cochlea. A consequence of this interpretation is that the remaining neural component of the ABR latency must be relatively independent of frequency and intensity.     

Decoupling of auditory pitch and stimulus frequency: the Shepard demonstration revisited

With ten successive octave tones, Shepard demonstrated that upward shifts in frequency might be heard as downward shifts in pitch. This decoupling of auditory pitch and frequency has important consequences for any representational theory of pitch. Experiments are reported that attempt to identify the crucial features of Shepard's demonstration.     

Temporal resolution and temporal masking properties of transient stimuli: data and an auditory model.

Temporal resolution is often measured using the detection of temporal gaps or signals in temporal gaps embedded in long-duration stimuli. In this study, psychoacoustical paradigms are developed for measuring the temporal encoding of transient stimuli. The stimuli consisted of very short pips which, in two experiments, contained a steady state portion. The carrier was high-pass filtered, dynamically compressed noise, refreshed for every stimulus presentation. The first experiment shows that, with these very short stimuli, gap detection thresholds are about the same as obtained in previous investigations. Experiments II and III show that, using the same stimuli, temporal-separation thresholds and duration-discrimination thresholds are better than gap-detection thresholds. Experiment IV investigates the significance of residual spectral cues for the listeners' performance. In experiment V, temporal separation thresholds were measured as a function of the signal-pip sensation level (SL) in both forward- and backward-masking conditions. The separation thresholds show a strong temporal asymmetry with good separation thresholds independent of signal-pip SL in backward-masking conditions and increasing separation thresholds with decreasing signal-pip SL in forward-masking conditions. A model of the auditory periphery is used to stimulate the gap-detection and temporal-separation thresholds quantitatively. By varying parameters like auditory-filter width and transduction time constants, the model provides some insight into how the peripheral auditory system may cope with temporal processing tasks and thus represents a more physiology-related complement to current models of temporal processing.