Results of a pilot study on the involvement of bilateral inferior frontal gyri in emotional prosody perception: an rTMS study

Background  

The right hemisphere may play an important role in paralinguistic features such as the emotional melody in speech. The extent of this involvement however is unclear. Imaging studies have shown involvement of both left and right inferior frontal gyri in emotional prosody perception. The present pilot study examined whether these brain areas are critically involved in the processing of emotional prosody and of semantics in 9 healthy subjects. Repetitive transcranial magnetic stimulation was used with a coil centred over left and right inferior frontal gyri, as localized by neuronavigation based on the subject's MRI. A sham condition was included. An online-TMS approach was applied; an emotional language task was completed during stimulation. This computerized task consisted of sentences pronounced by actors. In the semantics condition an emotion (fear, anger or neutral) was expressed in the content pronounced with a neutral intonation. In the prosody condition the emotion was expressed in the intonation, while the content was neutral.     

Effect of cochlear implants on children's perception and production of speech prosody

Japanese 5- to 13-yr-olds who used cochlear implants (CIs) and a comparison group of normally hearing (NH) Japanese children were tested on their perception and production of speech prosody. For the perception task, they were required to judge whether semantically neutral utterances that were normalized for amplitude were spoken in a happy, sad, or angry manner. The performance of NH children was error-free. By contrast, child CI users performed well below ceiling but above chance levels on happy- and sad-sounding utterances but not on angry-sounding utterances. For the production task, children were required to imitate stereotyped Japanese utterances expressing disappointment and surprise as well as culturally typically representations of crow and cat sounds. NH 5- and 6-year-olds produced significantly poorer imitations than older hearing children, but age was unrelated to the imitation quality of child CI users. Overall, child CI user's imitations were significantly poorer than those of NH children, but they did not differ significantly from the imitations of the youngest NH group. Moreover, there was a robust correlation between the performance of child CI users on the perception and production tasks; this implies that difficulties with prosodic perception underlie their difficulties with prosodic imitation.     

Presynaptic action of neurotensin on dopamine release through inhibition of D2 receptor function

Background

Statistical learning is a candidate for one of the basic prerequisites underlying the expeditious acquisition of spoken language. Infants from 8 months of age exhibit this form of learning to segment fluent speech into distinct words. To test the statistical learning skills at birth, we recorded event-related brain responses of sleeping neonates while they were listening to a stream of syllables containing statistical cues to word boundaries.

Results

We found evidence that sleeping neonates are able to automatically extract statistical properties of the speech input and thus detect the word boundaries in a continuous stream of syllables containing no morphological cues. Syllable-specific event-related brain responses found in two separate studies demonstrated that the neonatal brain treated the syllables differently according to their position within pseudowords.

Conclusion

These results demonstrate that neonates can efficiently learn transitional probabilities or frequencies of co-occurrence between different syllables, enabling them to detect word boundaries and in this way isolate single words out of fluent natural speech. The ability to adopt statistical structures from speech may play a fundamental role as one of the earliest prerequisites of language acquisition.     

The performance of different synthesis signals in acoustic models of cochlear implants

Synthesis (carrier) signals in acoustic models embody assumptions about perception of auditory electric stimulation. This study compared speech intelligibility of consonants and vowels processed through a set of nine acoustic models that used Spectral Peak (SPEAK) and Advanced Combination Encoder (ACE)-like speech processing, using synthesis signals which were representative of signals used previously in acoustic models as well as two new ones. Performance of the synthesis signals was determined in terms of correspondence with cochlear implant (CI) listener results for 12 attributes of phoneme perception (consonant and vowel recognition; F1, F2, and duration information transmission for vowels; voicing, manner, place of articulation, affrication, burst, nasality, and amplitude envelope information transmission for consonants) using four measures of performance. Modulated synthesis signals produced the best correspondence with CI consonant intelligibility, while sinusoids, narrow noise bands, and varying noise bands produced the best correspondence with CI vowel intelligibility. The signals that performed best overall (in terms of correspondence with both vowel and consonant attributes) were modulated and unmodulated noise bands of varying bandwidth that corresponded to a linearly varying excitation width of 0.4 mm at the apical to 8 mm at the basal channels.     

On the robustness of overall F0-only modifications to the perception of emotions in speech

Emotional information in speech is commonly described in terms of prosody features such as F0, duration, and energy. In this paper, the focus is on how F0 characteristics can be used to effectively parametrize emotional quality in speech signals. Using an analysis-by-synthesis approach, F0 mean, range, and shape properties of emotional utterances are systematically modified. The results show the aspects of the F0 parameter that can be modified without causing any significant changes in the perception of emotions. To model this behavior the concept of emotional regions is introduced. Emotional regions represent the variability present in the emotional speech and provide a new procedure for studying speech cues for judgments of emotion. The method is applied to F0 but can be also used on other aspects of prosody such as duration or loudness. Statistical analysis of the factors affecting the emotional regions, and discussion of the effects of F0 modifications on the emotion and speech quality perception are also presented. The results show that F0 range is more important than F0 mean for emotion expression.     

Explicit attention interferes with selective emotion processing in human extrastriate cortex

Background  

Brain imaging and event-related potential studies provide strong evidence that emotional stimuli guide selective attention in visual processing. A reflection of the emotional attention capture is the increased Early Posterior Negativity (EPN) for pleasant and unpleasant compared to neutral images (~150–300 ms poststimulus). The present study explored whether this early emotion discrimination reflects an automatic phenomenon or is subject to interference by competing processing demands. Thus, emotional processing was assessed while participants performed a concurrent feature-based attention task varying in processing demands.     

面向情感语音转换的韵律转换方法

面向情感语音转换,该文提出了一种韵律转换方法。该方法包含基频转换和时长转换两个部分,前者选择离散余弦变换(DCT)参数化基频,根据基频的层次结构特点,将基频分解为短语层和音节层两个层次,使用基于混合高斯模型(GMM)的转换方法对两个层次分别进行转换;后者使用基于分类回归树(CART)的方法以声韵母为基本单位对时长进行转换。一个包含三种基本情感的语料库用作训练和测试,客观评测以及主观评测实验结果显示该方法可有效进行情感韵律转换,其中悲伤情感在主观实验中达到了接近100%的正确率。      

Age-related changes in talker recognition with reduced spectral cues

Temporal information provided by cochlear implants enables successful speech perception in quiet, but limited spectral information precludes comparable success in voice perception. Talker identification and speech decoding by young hearing children (5-7 yr), older hearing children (10-12 yr), and hearing adults were examined by means of vocoder simulations of cochlear implant processing. In Experiment 1, listeners heard vocoder simulations of sentences from a man, woman, and girl and were required to identify the talker from a closed set. Younger children identified talkers more poorly than older listeners, but all age groups showed similar benefit from increased spectral information. In Experiment 2, children and adults provided verbatim repetition of vocoded sentences from the same talkers. The youngest children had more difficulty than older listeners, but all age groups showed comparable benefit from increasing spectral resolution. At comparable levels of spectral degradation, performance on the open-set task of speech decoding was considerably more accurate than on the closed-set task of talker identification. Hearing children's ability to identify talkers and decode speech from spectrally degraded material sheds light on the difficulty of these domains for child implant users.     

Processing of emotional words measured simultaneously with steady-state visually evoked potentials and near-infrared diffusing-wave spectroscopy

Background  

Emotional stimuli are preferentially processed compared to neutral ones. Measuring the magnetic resonance blood-oxygen level dependent (BOLD) response or EEG event-related potentials, this has also been demonstrated for emotional versus neutral words. However, it is currently unclear whether emotion effects in word processing can also be detected with other measures such as EEG steady-state visual evoked potentials (SSVEPs) or optical brain imaging techniques. In the present study, we simultaneously performed SSVEP measurements and near-infrared diffusing-wave spectroscopy (DWS), a new optical technique for the non-invasive measurement of brain function, to measure brain responses to neutral, pleasant, and unpleasant nouns flickering at a frequency of 7.5 Hz.     

Pitch ranking ability of cochlear implant recipients: a comparison of sound-processing strategies

Pitch ranking of sung vowel stimuli, separated in fundamental frequency (F0) by half an octave, was measured with a group of eleven Nucleus 24 cochlear implant recipients using different sound coding strategies. In three consecutive studies, either two or three different sound coding strategies were compared to the Advanced Combinational Encoder (ACE) strategy. These strategies included Continuous Interleaved Sampling (CIS), Peak Derived Timing (PDT), Modulation Depth Enhancement (MDE), F0 Synchronized ACE (FOSync), and Multi-channel Envelope Modulation (MEM), the last four being experimental strategies. While pitch ranking results on average were poor compared to those expected for most normal hearing listeners, significantly higher scores were obtained using the MEM, MDE, and FOSync strategies compared to ACE. These strategies enhanced coding of temporal F0 cues by providing deeper modulation cues to F0 coincidentally in time across all activated electrodes. In the final study, speech recognition tests were also conducted using ACE, CIS, MDE, and MEM. Similar results among all strategies were obtained for word tests in quiet and between ACE and MEM for sentence tests in noise. These findings demonstrate that strategies such as MEM may aid perception of pitch and still adequately code segmental speech features as per existing coding strategies.     

When decisions of others matter to me: an electrophysiological analysis

Background

Actions of others may have immediate consequences for oneself. We probed the neural responses associated with the observation of another person's action using event-related potentials in a modified gambling task. In this task a "performer" bet either a higher or lower number and could win or lose this amount. Three different groups of "observers" were also studied. The first (neutral) group simply observed the performer's action, which had no consequences for the observers. In the second (parallel) group, wins/losses of the performer were paralleled by similar wins and losses by the observer. In the third (reverse) group, wins of the performer led to a loss of the observer and vice versa.

Results

ERPs of the performers showed a mediofrontal feedback related negativity (FRN) to losses. The neutral and parallel observer groups did similarly show an FRN response to the performer's losses with a topography indistinguishable from that seen in the performers. In the reverse group, however, the FRN occurred for wins of the performer which translated to losses for the observer.

Conclusions

Taking into account previous experiments, we suggest that the FRN response in observers is driven by two evaluative processes (a) related to the benefit/loss for oneself and (b) related to the benefit/loss of another person.     

A glimpsing account for the benefit of simulated combined acoustic and electric hearing

The benefits of combined electric and acoustic stimulation (EAS) in terms of speech recognition in noise are well established; however the underlying factors responsible for this benefit are not clear. The present study tests the hypothesis that having access to acoustic information in the low frequencies makes it easier for listeners to glimpse the target. Normal-hearing listeners were presented with vocoded speech alone (V), low-pass (LP) filtered speech alone, combined vocoded and LP speech (LP+V) and with vocoded stimuli constructed so that the low-frequency envelopes were easier to glimpse. Target speech was mixed with two types of maskers (steady-state noise and competing talker) at -5 to 5 dB signal-to-noise ratios. Results indicated no advantage of LP+V in steady noise, but a significant advantage over V in the competing talker background, an outcome consistent with the notion that it is easier for listeners to glimpse the target in fluctuating maskers. A significant improvement in performance was noted with the modified glimpsed stimuli over the original vocoded stimuli. These findings taken together suggest that a significant factor contributing to the EAS advantage is the enhanced ability to glimpse the target.     

Asymmetrical hippocampal connectivity in mesial temporal lobe epilepsy: evidence from resting state fMRI

Background

Recent studies have shown that the human right-hemispheric auditory cortex is particularly sensitive to reduction in sound quality, with an increase in distortion resulting in an amplification of the auditory N1m response measured in the magnetoencephalography (MEG). Here, we examined whether this sensitivity is specific to the processing of acoustic properties of speech or whether it can be observed also in the processing of sounds with a simple spectral structure. We degraded speech stimuli (vowel /a/), complex non-speech stimuli (a composite of five sinusoidals), and sinusoidal tones by decreasing the amplitude resolution of the signal waveform. The amplitude resolution was impoverished by reducing the number of bits to represent the signal samples. Auditory evoked magnetic fields (AEFs) were measured in the left and right hemisphere of sixteen healthy subjects.

Results

We found that the AEF amplitudes increased significantly with stimulus distortion for all stimulus types, which indicates that the right-hemispheric N1m sensitivity is not related exclusively to degradation of acoustic properties of speech. In addition, the P1m and P2m responses were amplified with increasing distortion similarly in both hemispheres. The AEF latencies were not systematically affected by the distortion.

Conclusions

We propose that the increased activity of AEFs reflects cortical processing of acoustic properties common to both speech and non-speech stimuli. More specifically, the enhancement is most likely caused by spectral changes brought about by the decrease of amplitude resolution, in particular the introduction of periodic, signal-dependent distortion to the original sound. Converging evidence suggests that the observed AEF amplification could reflect cortical sensitivity to periodic sounds.     

Auditory temporal processing in healthy aging: a magnetoencephalographic study

Background  

Impaired speech perception is one of the major sequelae of aging. In addition to peripheral hearing loss, central deficits of auditory processing are supposed to contribute to the deterioration of speech perception in older individuals. To test the hypothesis that auditory temporal processing is compromised in aging, auditory evoked magnetic fields were recorded during stimulation with sequences of 4 rapidly recurring speech sounds in 28 healthy individuals aged 20 – 78 years.     

Comparing models of the combined-stimulation advantage for speech recognition

The "combined-stimulation advantage" refers to an improvement in speech recognition when cochlear-implant or vocoded stimulation is supplemented by low-frequency acoustic information. Previous studies have been interpreted as evidence for "super-additive" or "synergistic" effects in the combination of low-frequency and electric or vocoded speech information by human listeners. However, this conclusion was based on predictions of performance obtained using a suboptimal high-threshold model of information combination. The present study shows that a different model, based on Gaussian signal detection theory, can predict surprisingly large combined-stimulation advantages, even when performance with either information source alone is close to chance, without involving any synergistic interaction. A reanalysis of published data using this model reveals that previous results, which have been interpreted as evidence for super-additive effects in perception of combined speech stimuli, are actually consistent with a more parsimonious explanation, according to which the combined-stimulation advantage reflects an optimal combination of two independent sources of information. The present results do not rule out the possible existence of synergistic effects in combined stimulation; however, they emphasize the possibility that the combined-stimulation advantages observed in some studies can be explained simply by non-interactive combination of two information sources.     

Comparing the effects of reverberation and of noise on speech recognition in simulated electric-acoustic listening

Cochlear implant users report difficulty understanding speech in both noisy and reverberant environments. Electric-acoustic stimulation (EAS) is known to improve speech intelligibility in noise. However, little is known about the potential benefits of EAS in reverberation, or about how such benefits relate to those observed in noise. The present study used EAS simulations to examine these questions. Sentences were convolved with impulse responses from a model of a room whose estimated reverberation times were varied from 0 to 1 sec. These reverberated stimuli were then vocoded to simulate electric stimulation, or presented as a combination of vocoder plus low-pass filtered speech to simulate EAS. Monaural sentence recognition scores were measured in two conditions: reverberated speech and speech in a reverberated noise. The long-term spectrum and amplitude modulations of the noise were equated to the reverberant energy, allowing a comparison of the effects of the interferer (speech vs noise). Results indicate that, at least in simulation, (1) EAS provides significant benefit in reverberation; (2) the benefits of EAS in reverberation may be underestimated by those in a comparable noise; and (3) the EAS benefit in reverberation likely arises from partially preserved cues in this background accessible via the low-frequency acoustic component.     

Beneficial acoustic speech cues for cochlear implant users with residual acoustic hearing

This study investigated which acoustic cues within the speech signal are responsible for bimodal speech perception benefit. Seven cochlear implant (CI) users with usable residual hearing at low frequencies in the non-implanted ear participated. Sentence tests were performed in near-quiet (some noise on the CI side to reduce scores from ceiling) and in a modulated noise background, with the implant alone and with the addition, in the hearing ear, of one of four types of acoustic signals derived from the same sentences: (1) a complex tone modulated by the fundamental frequency (F0) and amplitude envelope contours; (2) a pure tone modulated by the F0 and amplitude contours; (3) a noise-vocoded signal; (4) unprocessed speech. The modulated tones provided F0 information without spectral shape information, whilst the vocoded signal presented spectral shape information without F0 information. For the group as a whole, only the unprocessed speech condition provided significant benefit over implant-alone scores, in both near-quiet and noise. This suggests that, on average, F0 or spectral cues in isolation provided limited benefit for these subjects in the tested listening conditions, and that the significant benefit observed in the full-signal condition was derived from implantees' use of a combination of these cues.     

Maturation of auditory temporal integration and inhibition assessed with event-related potentials (ERPs)

Background

We examined development of auditory temporal integration and inhibition by assessing electrophysiological responses to tone pairs separated by interstimulus intervals (ISIs) of 25, 50, 100, 200, 400, and 800 ms in 28 children aged 7 to 9 years, and 15 adults.

Results

In adults a distinct neural response was elicited to tones presented at ISIs of 25 ms or longer, whereas in children this was only seen in response to tones presented at ISIs above 100 ms. In adults, late N1 amplitude was larger for the second tone of the tone pair when separated by ISIs as short as 100 ms, consistent with the perceptual integration of successive stimuli within the temporal window of integration. In contrast, children showed enhanced negativity only when tone pairs were separated by ISIs of 200 ms. In children, the amplitude of the P1 component was attenuated at ISIs below 200 ms, consistent with a refractory process.

Conclusions

These results indicate that adults integrate sequential auditory information into smaller temporal segments than children. These results suggest that there are marked maturational changes from childhood to adulthood in the perceptual processes underpinning the grouping of incoming auditory sensory information, and that electrophysiological measures provide a sensitive, non-invasive method allowing further examination of these changes.     

Arithmetic mismatch negativity and numerical magnitude processing in number matching

Background

After a prolonged exposure to a paired presentation of different types of signals (e.g., color and motion), one of the signals (color) becomes a driver for the other signal (motion). This phenomenon, which is known as contingent motion aftereffect, indicates that the brain can establish new neural representations even in the adult's brain. However, contingent motion aftereffect has been reported only in visual or auditory domain. Here, we demonstrate that a visual motion aftereffect can be contingent on a specific sound.

Results

Dynamic random dots moving in an alternating right or left direction were presented to the participants. Each direction of motion was accompanied by an auditory tone of a unique and specific frequency. After a 3-minutes exposure, the tones began to exert marked influence on the visual motion perception, and the percentage of dots required to trigger motion perception systematically changed depending on the tones. Furthermore, this effect lasted for at least 2 days.

Conclusions

These results indicate that a new neural representation can be rapidly established between auditory and visual modalities.