Integration of auditory and vibrotactile stimuli: effects of frequency

Perceptual integration of vibrotactile and auditory sinusoidal tone pulses was studied in detection experiments as a function of stimulation frequency. Vibrotactile stimuli were delivered through a single channel vibrator to the left middle fingertip. Auditory stimuli were presented diotically through headphones in a background of 50 dB sound pressure level broadband noise. Detection performance for combined auditory-tactile presentations was measured using stimulus levels that yielded 63% to 77% correct unimodal performance. In Experiment 1, the vibrotactile stimulus was 250 Hz and the auditory stimulus varied between 125 and 2000 Hz. In Experiment 2, the auditory stimulus was 250 Hz and the tactile stimulus varied between 50 and 400 Hz. In Experiment 3, the auditory and tactile stimuli were always equal in frequency and ranged from 50 to 400 Hz. The highest rates of detection for the combined-modality stimulus were obtained when stimulating frequencies in the two modalities were equal or closely spaced (and within the Pacinian range). Combined-modality detection for closely spaced frequencies was generally consistent with an algebraic sum model of perceptual integration; wider-frequency spacings were generally better fit by a Pythagorean sum model. Thus, perceptual integration of auditory and tactile stimuli at near-threshold levels appears to depend both on absolute frequency and relative frequency of stimulation within each modality.     

Psychophysical studies relevant to the design of a digital electrotactile speech processor

Psychophysical tests were carried out to investigate the perception of electrocutaneous stimuli delivered to the digital nerve bundles. The tests provided data for defining the operating range of a tactile aid for patients with profound-to-total hearing loss, as well as the individual differences between subjects and the information that could be transmitted. Monopolar biphasic constant current pulses with variable pulse widths were used. Threshold pulse widths varied widely between subjects and between fingers for the same subject. Thresholds were reasonably stable, but maximum comfortable levels increased with time. Perceived intensity was weakly dependent on pulse rate. Absolute identification of stimuli differing in pulse width gave information transmissions from 1.3-2.1 bits, limited by the dynamic ranges of the stimuli (3-17 dB). Stimuli from electrodes placed on either side of each finger were identified easily by all subjects. Absolute identification of stimuli differing in pulse rate gave information transmissions from 0.5-2.0 bits. Difference limens for pulse rate varied between subjects and were generally poor above 100 pps. On the basis of the results, an electrotactile speech processor is proposed, which codes the speech amplitude as pulse width, the fundamental frequency as pulse rate, and the second formant frequency as electrode position. Variable performances on tasks relying on amplitude and fundamental frequency cues are expected to arise from the intersubject differences in dynamic range and pulse rate discrimination. The psychophysical results for electrotactile stimulation are compared with previously published results for electroauditory stimulation with a multiple-channel cochlear implant.     

Perceptual interactions in the loudness of combined auditory and vibrotactile stimuli

The loudness of auditory (A), tactile (T), and auditory-tactile (A+T) stimuli was measured at supra-threshold levels. Auditory stimuli were pure tones presented binaurally through headphones; tactile stimuli were sinusoids delivered through a single-channel vibrator to the left middle fingertip. All stimuli were presented together with a broadband auditory noise. The A and T stimuli were presented at levels that were matched in loudness to that of the 200-Hz auditory tone at 25 dB sensation level. The 200-Hz auditory tone was then matched in loudness to various combinations of auditory and tactile stimuli (A+T), and purely auditory stimuli (A+A). The results indicate that the matched intensity of the 200-Hz auditory tone is less when the A+T and A+A stimuli are close together in frequency than when they are separated by an octave or more. This suggests that A+T integration may operate in a manner similar to that found in auditory critical band studies, further supporting a strong frequency relationship between the auditory and somatosensory systems.     

Categorical perception--phenomenon or epiphenomenon: evidence from experiments in the perception of melodic musical intervals

Categorical perception was investigated in a series of experiments on the perception of melodic musical intervals (sequential frequency ratios). When procedures equivalent to those typically used in speech-perception experiments were employed, i.e., determination of identification and discrimination functions for stimuli separated by equal physical increments), musical intervals were perceived categorically by trained musicians. When a variable-step-size (adaptive) discrimination procedure was used, evidence of categorical perception (in the form of smaller interval-width DL's for ratios at identification category boundaries than for ratios within categories), although present initially, largely disappeared after subjects had reached asymptotic performance. However, equal-step-size discrimination functions obtained after observers had reached asymptotic performance in the adaptive paradigm were not substantially different from those initially obtained. The results of other experiments imply that this dependence of categorical perception on procedure may be related to differences in stimulus uncertainty between the procedures. An experiment on the perception of melodic intervals by musically untrained observers showed no evidence for the existence of "natural" categories for musical intervals.     

Bottlenose dolphin (Tursiops truncatus) steady-state evoked responses to multiple simultaneous sinusoidal amplitude modulated tones

Auditory steady-state evoked potentials were measured in a bottlenose dolphin (Tursiops truncatus) in response to single and multiple sinusoidal amplitude modulated (SAM) tones. Tests were conducted in air using a "jawphone" sound projector. Evoked potentials were recorded noninvasively using surface electrodes embedded in suction cups. Sound stimuli consisted of SAM tones with 1, 2, 3, or 4 carrier frequencies (10, 20, 30, 40 kHz), each with a unique modulation frequency. Stimulus sound pressure levels were varied in 5-dB steps from approximately 120 to 60-75 dB re 1 microPa, depending on frequency. Evoked potentials followed the temporal envelope of each stimulus, resulting in spectral components at each unique modulation frequency. Spectral analysis was used to evaluate the response amplitude for each carrier as a function of stimulus level. There were no significant differences between thresholds obtained with single and multiple stimuli at 10, 30, and 40 kHz. At 20 kHz, thresholds obtained with three components were higher than those obtained with four components, possibly revealing interactions between stimuli with less than one octave frequency separation. The use of multiple SAM stimuli may offer substantial advantages for studies of marine mammal hearing, where testing time and access to subjects are typically limited.     

Auditory brainstem response recovery in the dolphin as revealed by double sound pulses of different frequencies.

Recovery of auditory brainstem responses (ABR) in a bottlenose dolphin was studied in conditions of double-pip stimulation when two stimuli in a pair differed in frequency and intensity. When the conditioning and test stimuli were of equal frequencies, the test response was markedly suppressed at short interstimulus intervals; complete recovery appeared at intervals from about 2 ms (when two stimuli were of equal intensity) to 10-20 ms (when the conditioning stimulus exceeded the test by up to 40 dB). When the two stimuli were of different frequencies, the suppression diminished and was almost absent at a half-octave difference even if the conditioning stimulus exceeded the test one by 40 dB. Frequency-dependence curves (ABR amplitude dependence on frequency difference between the two stimuli) had equivalent rectangular bandwidth from +/-0.2 oct at test stimuli of 20 dB above threshold to +/-0.5 oct at test stimuli of 50 dB above threshold.     

Masking of short stimuli by noises with spiked spectra: II. Time summation and frequency selectivity of hearing in a narrow frequency range

The thresholds of masking of short high-frequency pulses with either different durations (1.25–25 ms) and similar central frequency or different central frequencies (3.6–4.4 kHz) but similar durations were measured to reveal manifestations of the properties of peripheral encoding in auditory perception. Noises with a spiked amplitude spectrum structure were used as maskers. The central frequency and the frequency band of a masker were 4 and 1 kHz, respectively. The central frequencies of a stimulus and a masker being equal, the noise the central frequency of which coincided with the frequency corresponding to a dip of an indented spectrum was called an off_(rip)-frequency masker. Owing to the off_(rip)-masker, stimuli-induced masking thresholds were formed taking into account excitation in a narrow region of a basila membrane and auditory nerve fibers with characteristic frequencies from a narrow range. High-frequency pulses with an envelope in the form of the Gaussian function and sinusoidal filling were used as stimuli. At masker levels of 30 dB above the auditory threshold, frequencies of off_(rip)-masker spectra spikes of 500–2000 Hz, and a central stimulus frequency of 4 kHz, the thresholds of tonal stimuli (25 ms in duration) masking in two out of three probationers were higher than the thresholds of masking of compact stimuli (1.25 ms in duration). In the third probationer, on the contrary, the thresholds of tonal stimuli masking were lower than the thresholds of compact stimuli masking. At masker levels of 50 dB, individual threshold differences disappeared. The obtained results were interpreted in the context of implementation of different methods of auditory encoding of the intensity. The methods were based on either the average frequency of auditory nerve pulsations or the number of fibers participating in the response. The interpretation was also carried out in the context of revealing manifestations of nonlinear properties of basila membrane displacements in auditory thresholds. The fact that the dependence of detection thresholds of compact stimuli on their central frequency in one of the two probationers did not reveal the minimum in case of coincidence of off_(rip)-masker and stimulus frequencies pointed to the presence of an auditory “problem zone” that was likely to be localized at the periphery of the auditory system.     

Vibrotactile temporal gap detection as a function of age

The ability of subjects to detect temporal gaps between bursts of sinusoids or bursts of bandlimited noise was measured to evaluate the phenomenon of tactile "sensory persistence" in older persons. Vibratory stimuli were delivered to the right thenar eminence of 27 subjects ranging in age from 8-75 years. The subjects' task was to detect the presence of a silent interval or "gap" between flanking 350-ms vibrotactile stimuli. The gap-detection threshold, expressed as the amplitude of vibration relative to the absolute detection threshold, decreased as the gap duration increased and was higher for gaps in noise than for gaps in sinusoids. The threshold for detecting short gaps increased with age for noise stimuli, but not for sinusoidal stimuli. Furthermore, the gap-detection threshold recovered more rapidly in older subjects for noise stimuli, but less rapidly in older subjects for sinusoidal stimuli. Because of these differences, it appears that the effects of age on gap detection cannot be due to a simple increase in sensory persistence, but may be due to multiple processes.     

A model of spatiotemporal tactile sensitivity linking psychophysics to tissue mechanics

Sensitivities were measured for tangible spatiotemporal sinusoids applied to the index fingertip. The sinusoids had temporal frequencies of 8 and 128 Hz, in order to selectively activate the non-Pacinian I (NP I) and Pacinian (P) cutaneous mechanoreceptor systems, respectively, and had spatial frequencies from 0.00-1.03 cycles/mm. The sensitivity of the NP I system increased as the spatial frequency increased, whereas the sensitivity of the P system generally decreased as the spatial frequency increased. A mechanical model of the fingertip was used to calculate the normal and shear strains in the tissue, and a psychophysical linking hypothesis was introduced to predict tactile sensitivities based on the calculated strains. Specifically, the fingertip was modeled as a slab of a linear, isotropic, homogeneous, viscoelastic material. The boundary conditions were imposed by the spatiotemporal sinusoid at the top of the slab and the rigidly attached bone at the bottom of the slab. It was then assumed that the detection threshold was equal to the stimulus amplitude, which produced a constant, criterion strain at the location of the receptor. For both the P and NP I responses, the agreement between the predicted and measured sensitivities was best for calculations based on the normal strain, and for spatial frequencies below 0.5 cycles/mm. At higher spatial frequencies, the measured sensitivities were higher than predicted. The model also predicted the location of the P and NP I receptors in the tissue, the thickness of the tissue, and the value of the threshold strain for both receptor types. The predicted values agreed reasonably well with independent anatomical and physiological measurements.     

Functional MRI of working memory and selective attention in vibrotactile frequency discrimination

Background  

Focal lesions of the frontal, parietal and temporal lobe may interfere with tactile working memory and attention. To characterise the neural correlates of intact vibrotactile working memory and attention, functional MRI was conducted in 12 healthy young adults. Participants performed a forced-choice vibrotactile frequency discrimination task, comparing a cue stimulus of fixed frequency to their right thumb with a probe stimulus of identical or higher frequency. To investigate working memory, the time interval between the 2 stimuli was pseudo-randomized (either 2 or 8 s). To investigate selective attention, a distractor stimulus was occasionally presented contralaterally, simultaneous to the probe.     

Evaluation of two multichannel tactile aids for the hearing impaired

Two multichannel tactile devices for the hearing impaired were compared in speech perception tasks of varying levels of complexity. Both devices implemented the "vocoder" principle in their stimulus processing: One device had a 16-element linear vibratory array worn on the forearm and displayed activity in 16 overlapping frequency channels; the other device delivered tactile stimulation to a linear array of 16 electrodes worn on the abdomen. Subjects were tested in several phoneme discrimination tasks, ranging from discrimination of pairs of words differing in only one phoneme under tactile aid alone conditions to identification of stimuli in a larger set under tactile aid alone, lipreading alone, and lipreading plus tactile aid conditions. Results showed both devices to be better transmitters of manner and voicing features of articulation than of place features, when tested in single-item tasks. No systematic differences in performance with the two devices were observed. However, in a connected discourse tracking task, the vibrotactile vocoder in conjunction with lipreading yielded much greater improvements over lipreading alone than did the electrotactile vocoder. One possible explanation for this difference in performance, the inclusion of a noise suppression circuit in the electrotactile aid, was evaluated, but did not appear to account for the differences observed. Results are discussed in terms of additional differences between the two devices that may influence performance.     

Frequency resolution for diotic and dichotic listening conditions compared using the bandlimiting measure and a modified bandlimiting measure

Two experiments were performed that examined the relation between frequency selectivity for diotic and dichotic stimuli. Subjects were eight normal-hearing listeners. In each experiment, a 500-Hz pure tone of 400-ms duration was presented in continuous noise. In the diotic listening conditions, a signal and noise were presented binaurally with no interaural differences (So and No, respectively). In the dichotic listening conditions, the signal or noise at one ear was 180 degrees out-of-phase relative to the respective stimulus at the other ear (S pi and N pi, respectively). The first experiment examined frequency selectivity using the bandlimiting measure. Here, signal thresholds were determined as a function of masker bandwidth (50, 100, 250, 500, and 1000 Hz) for SoNo, S pi No, and SoN pi listening conditions. The second experiment used a modified bandlimiting measure. Here, signal thresholds (So and S pi) were determined with a relatively narrow No band of masker energy (50 Hz wide) centered about the signal. Then, a second No narrow-band masker (30 Hz wide) was added at another frequency region, and signal thresholds were reestablished. The results of the two experiments indicated that listeners process a wider band of frequencies when resolving dichotic stimuli than when resolving diotic or monotic stimuli. The results also indicated that the bandlimiting measure may underestimate the spectral band processed upon dichotic stimulation. Results are interpreted in terms of an across-ear and across-frequency processing of waveform amplitude envelope.     

A critical experimental study of the classical tactile threshold theory

Background  

The tactile sense is being used in a variety of applications involving tactile human-machine interfaces. In a significant number of publications the classical threshold concept plays a central role in modelling and explaining psychophysical experimental results such as in stochastic resonance (SR) phenomena. In SR, noise enhances detection of sub-threshold stimuli and the phenomenon is explained stating that the required amplitude to exceed the sensory threshold barrier can be reached by adding noise to a sub-threshold stimulus. We designed an experiment to test the validity of the classical vibrotactile threshold. Using a second choice experiment, we show that individuals can order sensorial events below the level known as the classical threshold. If the observer's sensorial system is not activated by stimuli below the threshold, then a second choice could not be above the chance level. Nevertheless, our experimental results are above that chance level contradicting the definition of the classical tactile threshold.     

Frequency specificity of chirp-evoked auditory brainstem responses

This study examines the usefulness of the upward chirp stimulus developed by Dau et al. [J. Acoust. Soc. Am. 107, 1530-1540 (2000)] for retrieving frequency-specific information. The chirp was designed to produce simultaneous displacement maxima along the cochlear partition by compensating for frequency-dependent traveling-time differences. In the first experiment, auditory brainstem responses (ABR) elicited by the click and the broadband chirp were obtained in the presence of high-pass masking noise, with cutoff frequencies of 0.5, 1, 2, 4, and 8 kHz. Results revealed a larger wave-V amplitude for chirp than for click stimulation in all masking conditions. Wave-V amplitude for the chirp increased continuously with increasing high-pass cutoff frequency while it remains nearly constant for the click for cutoff frequencies greater than 1 kHz. The same two stimuli were tested in the presence of a notched-noise masker with one-octave wide spectral notches corresponding to the cutoff frequencies used in the first experiment. The recordings were compared with derived responses, calculated offline, from the high-pass masking conditions. No significant difference in response amplitude between click and chirp stimulation was found for the notched-noise responses as well as for the derived responses. In the second experiment, responses were obtained using narrow-band stimuli. A low-frequency chirp and a 250-Hz tone pulse with comparable duration and magnitude spectrum were used as stimuli. The narrow-band chirp elicited a larger response amplitude than the tone pulse at low and medium stimulation levels. Overall, the results of the present study further demonstrate the importance of considering peripheral processing for the formation of ABR. The chirp might be of particular interest for assessing low-frequency information.     

Using electrical noise to enhance the ability of humans to detect subthreshold mechanical cutaneous stimuli

Stochastic resonance (SR) is a phenomenon wherein the response of a nonlinear system to a weak input signal is optimized by the presence of a particular, nonzero level of noise. Our objective was to demonstrate cross-modality SR in human sensory perception. Specifically, we were interested in testing the hypothesis that the ability of an individual to detect a subthreshold mechanical cutaneous stimulus can be significantly enhanced by introducing a particular level of electrical noise. Psychophysical experiments were performed on 11 healthy subjects. The protocol consisted of the presentation of: (a) a subthreshold mechanical stimulus plus electrical noise, or (b) no mechanical stimulus plus electrical noise. The intensity of the electrical noise was varied between trials. Each subject's ability to identify correctly the presence of the mechanical stimulus was determined as a function of the noise intensity. In 9 of the 11 subjects, the introduction of a particular level of electrical noise significantly enhanced the subject's ability to detect the subthreshold mechanical cutaneous stimulus. In 2 of the 11 subjects, the introduction of electrical noise did not significantly change the subject's ability to detect the mechanical stimulus. These findings indicate that input electrical noise can serve as a negative masker for subthreshold mechanical tactile stimuli, i.e., electrical noise can increase the detectability of weak mechanical signals. Thus, for SR-type effects to be observed in human sensory perception, the noise and stimulus need not be of the same modality. From a bioengineering and clinical standpoint, this work suggests that an electrical noise-based technique could be used to improve tactile sensation in humans when the mechanical stimulus is around or below threshold. (c) 1998 American Institute of Physics.     

Asymmetrical hippocampal connectivity in mesial temporal lobe epilepsy: evidence from resting state fMRI

Background

Recent studies have shown that the human right-hemispheric auditory cortex is particularly sensitive to reduction in sound quality, with an increase in distortion resulting in an amplification of the auditory N1m response measured in the magnetoencephalography (MEG). Here, we examined whether this sensitivity is specific to the processing of acoustic properties of speech or whether it can be observed also in the processing of sounds with a simple spectral structure. We degraded speech stimuli (vowel /a/), complex non-speech stimuli (a composite of five sinusoidals), and sinusoidal tones by decreasing the amplitude resolution of the signal waveform. The amplitude resolution was impoverished by reducing the number of bits to represent the signal samples. Auditory evoked magnetic fields (AEFs) were measured in the left and right hemisphere of sixteen healthy subjects.

Results

We found that the AEF amplitudes increased significantly with stimulus distortion for all stimulus types, which indicates that the right-hemispheric N1m sensitivity is not related exclusively to degradation of acoustic properties of speech. In addition, the P1m and P2m responses were amplified with increasing distortion similarly in both hemispheres. The AEF latencies were not systematically affected by the distortion.

Conclusions

We propose that the increased activity of AEFs reflects cortical processing of acoustic properties common to both speech and non-speech stimuli. More specifically, the enhancement is most likely caused by spectral changes brought about by the decrease of amplitude resolution, in particular the introduction of periodic, signal-dependent distortion to the original sound. Converging evidence suggests that the observed AEF amplification could reflect cortical sensitivity to periodic sounds.     

Temporal onset-order discrimination through the tactual sense

Tactual temporal-onset order thresholds were measured for two sinusoidal vibrations of different frequencies delivered to two separate locations (thumb and index finger) of a multi-finger tactual stimulating device. The frequency delivered to the thumb was fixed at 50 Hz and that to the index finger at 250 Hz. The amplitude and duration of each of the two sinusoidal vibrations were roved independently from trial to trial in a 1-interval, 2AFC procedure. Performance, measured as a function of stimulus-onset asynchrony (SOA), indicated that the temporal-onset-order threshold averaged 34 ms across four subjects. The data were further classified into subsets according to both the amplitude and duration of the two stimuli in each trial of the roving-discrimination paradigm. The results indicated that the amplitude differences of the two stimuli in each trial had a substantial effect on onset-order discrimination, while duration differences generally had little effect. The effects of amplitude differences are explained qualitatively in terms of amplitude latency relationships and stimulus interactions such as temporal masking. Overall, the results not only contribute to an enhanced understanding of the temporal sensitivity of the tactual system but also provide guidelines for the design of tactual aids for hearing-impaired persons.     

The effects of neural synchronization and peripheral compression on the acoustic-reflex threshold

This study investigates the acoustic reflex threshold (ART) dependency on stimulus phase utilizing low-level reflex audiometry [Neumann et al., Audiol. Neuro-Otol. 1, 359-369 (1996)]. The goal is to obtain optimal broadband stimuli for elicitation of the acoustic reflex and to obtain objective determinations of cochlear hearing loss. Three types of tone complexes with different phase characteristics were investigated: A stimulus that compensates for basilar-membrane dispersion, thus causing a large overall neural synchrony (basilar-membrane tone complex-BMTC), the temporally inversed stimulus (iBMTC), and random-phase tone complexes (rTC). The ARTs were measured in eight normal-hearing and six hearing-impaired subjects. Five different conditions of peak amplitude and stimulus repetition rate were used for each stimulus type. The results of the present study suggest that the ART is influenced by at least two different factors: (a) the degree of synchrony of neural activity across frequency, and (b) the fast-acting compression mechanism in the cochlea that is reduced in the case of a sensorineural hearing loss. The results allow a clear distinction of the two subjects groups based on the different ART for the utilized types and conditions of the stimuli. These differences might be useful for objective recruitment detection in clinical diagnostics.     

Vibrotactile identification of vowels

The ability of subjects to identify vowels in vibrotactile transformations of consonant-vowel syllables was measured for two types of displays: a spectral display (frequency by intensity), and a vocal tract area function display (vocal tract location by cross-sectional area). Both displays were presented to the fingertip via the tactile display of the Optacon transducer. In the first experiments the spectral display was effective for identifying vowels in /b/V/ context when as many as 24 or as few as eight spectral channels were presented to the skin. However, performance fell when the 12- and 8-channel displays were reduced in size to occupy 1/2 or 1/3 of the 24-row tactile matrix. The effect of reducing the size of the display was greater when the spectrum was represented as a solid histogram ("filled" patterns) than when it was represented as a simple spectral contour ("unfilled" patterns). Spatial masking within the filled pattern was postulated as the cause for this decline in performance. Another experiment measured the utility of the spectral display when the syllables were produced by multiple speakers. The resulting increase in response confusions was primarily attributable to variations in the tactile patterns caused by differences in vocal tract resonances among the speakers. The final experiment found an area function display to be inferior to the spectral display for identification of vowels. The results demonstrate that a two-dimensional spectral display is worthy of further development as a basic vibrotactile display for speech.