Transitional zones of meaning and semantics in music and language: Reply to comments on “Towards a neural basis of processing musical semantics”

Processing of meaning is critical for language perception, and therefore the majority of research on meaning processing has focused on the semantic, lexical, conceptual, and propositional processing of language. However, music is another a means of communication, and meaning also emerges from the interpretation of musical information. This article provides a framework for the investigation of the processing of musical meaning, and reviews neuroscience studies investigating this issue. These studies reveal two neural correlates of meaning processing, the N400 and the N5 (which are both components of the event-related electric brain potential). Here I argue that the N400 can be elicited by musical stimuli due to the processing of extra-musical meaning, whereas the N5 can be elicited due to the processing of intra-musical meaning. Notably, whereas the N400 can be elicited by both linguistic and musical stimuli, the N5 has so far only been observed for the processing of meaning in music. Thus, knowledge about both the N400 and the N5 can advance our understanding of how the human brain processes meaning information.     

Computational models of music perception and cognition I: The perceptual and cognitive processing chain

We present a review on perception and cognition models designed for or applicable to music. An emphasis is put on computational implementations. We include findings from different disciplines: neuroscience, psychology, cognitive science, artificial intelligence, and musicology. The article summarizes the methodology that these disciplines use to approach the phenomena of music understanding, the localization of musical processes in the brain, and the flow of cognitive operations involved in turning physical signals into musical symbols, going from the transducers to the memory systems of the brain. We discuss formal models developed to emulate, explain and predict phenomena involved in early auditory processing, pitch processing, grouping, source separation, and music structure computation. We cover generic computational architectures of attention, memory, and expectation that can be instantiated and tuned to deal with specific musical phenomena. Criteria for the evaluation of such models are presented and discussed. Thereby, we lay out the general framework that provides the basis for the discussion of domain-specific music models in Part II.     

Effects of musical training and absolute pitch ability on event-related activity in response to sine tones.

The neural correlates of music perception have received relatively little scientific attention. The neural activity of listeners without musical training (N = 11), highly trained musicians (N = 14), and musicians possessing "absolute pitch" (AP) ability (N = 10) have been measured. Major differences were observed in the P3, an endogenous event-related potential (ERP), which is thought to be a neurophysiological manifestation of working memory processing. The P3 was elicited using the classical "oddball" paradigm with a sine-tone series. Subjects' musical backgrounds were evaluated with a survey questionnaire. AP ability was verified with an objective pitch identification test. The P3 amplitude, latency and wave shape were evaluated along with each subjects' performance score and musical background. The AP subjects showed a significantly smaller P3 amplitude than either the musicians or nonmusicians, which were nearly identical. The P3 latency was shortest for the AP subjects, and was longer for the nonmusicians. Performance scores were uniformly high in all three groups. It is concluded that AP subjects do indeed exhibit P3 ERPs, albeit with smaller amplitudes and shorter latencies. The differences in neural activity between the musicians and AP subjects were not due to musical training, as the AP subjects had similar musical backgrounds to the musician group. It is also concluded that persons with the AP ability may have superior auditory sensitivity at cortical levels and/or use unique neuropsychological strategies when processing tones.     

Mapping perception to action in piano practice: a longitudinal DC-EEG study

Background  

Performing music requires fast auditory and motor processing. Regarding professional musicians, recent brain imaging studies have demonstrated that auditory stimulation produces a co-activation of motor areas, whereas silent tapping of musical phrases evokes a co-activation in auditory regions. Whether this is obtained via a specific cerebral relay station is unclear. Furthermore, the time course of plasticity has not yet been addressed.     

Perception of acoustic scale and size in musical instrument sounds

There is size information in natural sounds. For example, as humans grow in height, their vocal tracts increase in length, producing a predictable decrease in the formant frequencies of speech sounds. Recent studies have shown that listeners can make fine discriminations about which of two speakers has the longer vocal tract, supporting the view that the auditory system discriminates changes on the acoustic-scale dimension. Listeners can also recognize vowels scaled well beyond the range of vocal tracts normally experienced, indicating that perception is robust to changes in acoustic scale. This paper reports two perceptual experiments designed to extend research on acoustic scale and size perception to the domain of musical sounds: The first study shows that listeners can discriminate the scale of musical instrument sounds reliably, although not quite as well as for voices. The second experiment shows that listeners can recognize the family of an instrument sound which has been modified in pitch and scale beyond the range of normal experience. We conclude that processing of acoustic scale in music perception is very similar to processing of acoustic scale in speech perception.     

Musical emotions: Functions,origins, evolution

Theories of music origins and the role of musical emotions in the mind are reviewed. Most existing theories contradict each other, and cannot explain mechanisms or roles of musical emotions in workings of the mind, nor evolutionary reasons for music origins. Music seems to be an enigma. Nevertheless, a synthesis of cognitive science and mathematical models of the mind has been proposed describing a fundamental role of music in the functioning and evolution of the mind, consciousness, and cultures. The review considers ancient theories of music as well as contemporary theories advanced by leading authors in this field. It addresses one hypothesis that promises to unify the field and proposes a theory of musical origin based on a fundamental role of music in cognition and evolution of consciousness and culture. We consider a split in the vocalizations of proto-humans into two types: one less emotional and more concretely-semantic, evolving into language, and the other preserving emotional connections along with semantic ambiguity, evolving into music. The proposed hypothesis departs from other theories in considering specific mechanisms of the mind–brain, which required the evolution of music parallel with the evolution of cultures and languages. Arguments are reviewed that the evolution of language toward becoming the semantically powerful tool of today required emancipation from emotional encumbrances. The opposite, no less powerful mechanisms required a compensatory evolution of music toward more differentiated and refined emotionality. The need for refined music in the process of cultural evolution is grounded in fundamental mechanisms of the mind. This is why today's human mind and cultures cannot exist without today's music. The reviewed hypothesis gives a basis for future analysis of why different evolutionary paths of languages were paralleled by different evolutionary paths of music. Approaches toward experimental verification of this hypothesis in psychological and neuroimaging research are reviewed.     

Pitch-interval discrimination and musical expertise: Is the semitone a perceptual boundary?

The ability to discriminate pitch changes (or intervals) is foundational for speech and music. In an auditory psychophysical experiment, musicians and non-musicians were tested with fixed- and roving-pitch discrimination tasks to investigate the effects of musical expertise on interval discrimination. The tasks were administered parametrically to assess performance across varying pitch distances between intervals. Both groups showed improvements in fixed-pitch interval discrimination as a function of increasing interval difference. Only musicians showed better roving-pitch interval discrimination as interval differences increased, suggesting that this task was too demanding for non-musicians. Musicians had better interval discrimination than non-musicians across most interval differences in both tasks. Interestingly, musicians exhibited improved interval discrimination starting at interval differences of 100 cents (a semitone in Western music), whereas non-musicians showed enhanced discrimination at interval differences exceeding 125 cents. Although exposure to Western music and speech may help establish a basic interval-discrimination threshold between 100 and 200 cents (intervals that occur often in Western languages and music), musical training presumably enhances auditory processing and reduces this threshold to a semitone. As musical expertise does not decrease this threshold beyond 100 cents, the semitone may represent a musical training-induced intervallic limit to acoustic processing.     

An investigation into the linear and nonlinear correlation of two music walk sequences

In order to investigate the features of a multiple-part musical score which enhance its appeal to the listener’s ear, this study performs a robust analysis of the correlation between two musical sequences. In the proposed approach, a series of notes are extracted from seven well-known classical pieces of music and are converted into one-variable “music walks”. The linear correlation between pairs of music walks is assessed using the conventional linear correlation coefficient, while the nonlinear correlation is examined using the mutual information concept. The results show that even though two musical time walks may exhibit virtually no linear correlation, they invariably have a certain degree of nonlinear correlation. In other words, to truly understand the correlation between two musical sequences, it is necessary to consider not only the linear correlation between them, but also the nonlinear correlation. In addition, it is shown that the normalized mutual information coefficient between musical sequences has a relatively low value and varies significantly over the course of the musical score. Thus, it can be inferred that the appeal of a musical score stems at least in part from significant variations in both the melody and the rhythm of the constituent parts such that the overall score has a rich and unpredictable property.     

The origins of language and the evolution of music: A comparative perspective

According to Darwin [Darwin, CR. The descent of man, and selection in relation to sex. London: John Murray; 1871], the human musical faculty 'must be ranked amongst the most mysterious with which he is endowed'. Music is a human cultural universal that serves no obvious adaptive purpose, making its evolution a puzzle for evolutionary biologists. This review examines Darwin's hypothesis of similarities between language and music indicating a shared evolutionary history. In particular, the fact that both are human universals, have phrase structure, and entail learning and cultural transmission, suggests that any theory of the evolution of language will have implications for the evolution of music, and vice versa. The argument starts by describing variable predispositional musical capabilities and the ontogeny of prosodic communication in human infants and young children, presenting comparative data regarding communication systems commonly present in living nonhuman primate species. Like language, the human music faculty is based on a suite of abilities, some of which are shared with other primates and some of which appear to be uniquely human. Each of these subcomponents may have a different evolutionary history, and should be discussed separately. After briefly considering possible functions of human music for language acquisition, the review ends by discussing the phylogenetic history of music. It concludes that many strands of evidence support Darwin's hypothesis of an intermediate stage of human evolutionary history, characterized by a communication system that resembled music more closely than language, but was identical to neither. This pre-linguistic system, which could probably referred to as "prosodic protolanguage", provided a precursor for both modern language and music.     

Fine-grained pitch processing of music and speech in congenital amusia

Congenital amusia is a lifelong disorder of music processing that has been ascribed to impaired pitch perception and memory. The present study tested a large group of amusics (n=17) and provided evidence that their pitch deficit affects pitch processing in speech to a lesser extent: Fine-grained pitch discrimination was better in spoken syllables than in acoustically matched tones. Unlike amusics, control participants performed fine-grained pitch discrimination better for musical material than for verbal material. These findings suggest that pitch extraction can be influenced by the nature of the material (music vs speech), and that amusics' pitch deficit is not restricted to musical material, but extends to segmented speech events.     

Enhancing dominant modes in nonstationary time series by means of the symbolic resonance analysis

We present the symbolic resonance analysis (SRA) as a viable method for addressing the problem of enhancing a weakly dominant mode in a mixture of impulse responses obtained from a nonlinear dynamical system. We demonstrate this using results from a numerical simulation with Duffing oscillators in different domains of their parameter space, and by analyzing event-related brain potentials (ERPs) from a language processing experiment in German as a representative application. In this paradigm, the averaged ERPs exhibit an N400 followed by a sentence final negativity. Contemporary sentence processing models predict a late positivity (P600) as well. We show that the SRA is able to unveil the P600 evoked by the critical stimuli as a weakly dominant mode from the covering sentence final negativity.     

语后聋人工耳蜗使用者电刺激听觉部位音调感知研究

系统地研究了人工耳蜗植入者的电刺激听觉部位音调感知,全面地探讨了部位音调感知与人工耳蜗植入者言语识别和音乐感知的关系。4位成人语后聋人工耳蜗植入者参与了该研究。通过电极音调排序测试度量植入者的部位音调感知能力。言语能力测试和音乐音高分辨测试分别用米考察植入者的言语识别和音乐感知能力。结果显示,随着电极刺激部位从蜗尖移向蜗底,所有受试者均可获得从"低"到"高"的音调感知变化,但个体差异较大。受试者的言语识别结果与其电刺激听觉部位音调感知能力相关,但受到天花板效应影响,对应关系并不明显。受试者的音乐音高分辨成绩与其电刺激听觉部位音调感知能力呈较好的对应关系。结果表明,当前人工耳蜗声音编码策略所传递的声信号特征已可使植入者获得良好的言语识别效果;且安静环境下言语识别对植入者的部位音调感知能力要求不高。但当前的声音编码策略并未能有效对音乐信号进行编码;植入者在理解音乐这类复杂声信号时,其电刺激听觉部位音调感知能力一定程度决定了其听音效果。      

Place-pitch perception in electrical hearing with post-lingually deafened cochlear implant users

The main goal of this study was to systematically investigate place-pitch perception in electrical hearing and the relative relationship between place-pitch perception ability,speech understanding and musical pitch discrimination by cochlear implant(CI) users.Electrode pitch ranking test was carried out to evaluate the place-pitch perception ability of CI users. Four post-lingually deafened CI users were recruited.They also participated in the speech recognition test and musical pitch discrimination test.Results showed that place pitch were generally ordered from apical to basal electrodes.The apical electrodes were judged lower in pitch than basal electrodes.Large individual difference was found.Comparing pitch and speech performance,the speech recognition result was related to the place-pitch perception ability of CI users,but this relationship was limited by the ceiling effects.However,a correlative relationship was found between musical pitch discrimination result and place-pitch ability of CI users.It indicated that the current signal processing of CI system can provide sufficient information for speech understanding but not for music perception of CI users.To a certain extent,music perception of CI users was determined by their place-pitch abilities.     

Composing Music Inspired by Sculpture: A Cross-Domain Mapping and Genetic Algorithm Approach

In this article, a system that takes a 3D model of a sculpture as starting point to compose music is presented. We raised the hypothesis that cross-domain mapping can be an approach to model inspiration. The semantic meaning of the sculpture is not used directly but rather a more abstract approach was used. A Genetic Algorithm was used to obtain results with more musical interest. The results were promising: the majority of the participants gave a classification of 4 out of 5 to the preferred interpretations of the compositions and related them to the respective sculpture. This is a step toward a possible model for inspiration.     

Pitch discrimination abilities in classical Arab-music listeners

The current experiment examined pitch discrimination thresholds in listeners of classical Arab music, and listeners of Western popular music. Classical Arab music is characterized by modes (“Maqamat”, plural of “Maqam” in Arabic language) of which the smallest interval is a quarter tone. In contrast, the smallest interval in Western music is a semitone. We hypothesized that daily exposure to a musical style involving minuscule pitch differences may have a positive effect on pitch discrimination abilities. Results demonstrate superior pitch discrimination abilities in the classical Arab music listeners. These results indicate that musical cultures may differ in their influence on perceptual abilities, depending on their basic acoustic characteristics.     

基于音色单元分布的音乐结构分析

音乐的结构是音乐作品表达作者思想的一种重要形式,也是听众理解音乐作品内涵的有效途径。本文研究了基于音乐特征的音色单元建模方法,研究了在Fisher准则下,根据局部范围音色单元的分布,采用非监督聚类方法分析音乐的结构。实验结果证明了基于离散余弦变换的音色特征,用音色单元分布聚类算法进行音乐结构分析的有效性。      

The emergence of semantic categorization in early visual processing: ERP indices of animal vs. artifact recognition

Background

Neuroimaging and neuropsychological literature show functional dissociations in brain activity during processing of stimuli belonging to different semantic categories (e.g., animals, tools, faces, places), but little information is available about the time course of object perceptual categorization. The aim of the study was to provide information about the timing of processing stimuli from different semantic domains, without using verbal or naming paradigms, in order to observe the emergence of non-linguistic conceptual knowledge in the ventral stream visual pathway. Event related potentials (ERPs) were recorded in 18 healthy right-handed individuals as they performed a perceptual categorization task on 672 pairs of images of animals and man-made objects (i.e., artifacts).

Results

Behavioral responses to animal stimuli were ~50 ms faster and more accurate than those to artifacts. At early processing stages (120–180 ms) the right occipital-temporal cortex was more activated in response to animals than to artifacts as indexed by posterior N1 response, while frontal/central N1 (130–160) showed the opposite pattern. In the next processing stage (200–260) the response was stronger to artifacts and usable items at anterior temporal sites. The P300 component was smaller, and the central/parietal N400 component was larger to artifacts than to animals.

Conclusion

The effect of animal and artifact categorization emerged at ~150 ms over the right occipital-temporal area as a stronger response of the ventral stream to animate, homomorphic, entities with faces and legs. The larger frontal/central N1 and the subsequent temporal activation for inanimate objects might reflect the prevalence of a functional rather than perceptual representation of manipulable tools compared to animals. Late ERP effects might reflect semantic integration and cognitive updating processes. Overall, the data are compatible with a modality-specific semantic memory account, in which sensory and action-related semantic features are represented in modality-specific brain areas.     

Unforgettable film music: The role of emotion in episodic long-term memory for music

Background  

Specific pieces of music can elicit strong emotions in listeners and, possibly in connection with these emotions, can be remembered even years later. However, episodic memory for emotional music compared with less emotional music has not yet been examined. We investigated whether emotional music is remembered better than less emotional music. Also, we examined the influence of musical structure on memory performance.     

基于高夹角分辨率扩散磁振造影神经径路追踪的人脑语言机率路径图谱

大脑皮质内部的联系神经束,对于大脑皮质之间的信息传递担任非常重要的角色.传统的语言模型理论提出人类的2个主要语言中枢分别位于大脑皮质的左侧额下回的布罗卡区域(Broca’s area,BA44andBA45)以及颞上回处的维尼基区域(Wernicke’s area,BA22),而联系这2个区域的纤维束,也就是弓状束(arcuate fasciculus).另外,近期研究也发现下顶叶(inferior parietal cortex,BA39and BA40)在语音处理历程具重要性.扩散磁振造影(Diffu-sionMRI)可以提供大脑白质精细的组织结构,配合神经径路追踪(tractography)便能撷取出复杂的神经纤维连结路径.该研究利用扩散磁振影像中的高夹角分辨率扩散磁振造影(high angular resolution diffusion imaging)与神经径路追踪技术,呈现与语言相关的大脑机率神经连结路径(probabilistic language pathway).     

From everyday emotions to aesthetic emotions: Towards a unified theory of musical emotions

The sound of music may arouse profound emotions in listeners. But such experiences seem to involve a ‘paradox’, namely that music – an abstract form of art, which appears removed from our concerns in everyday life – can arouse emotions – biologically evolved reactions related to human survival. How are these (seemingly) non-commensurable phenomena linked together? Key is to understand the processes through which sounds are imbued with meaning. It can be argued that the survival of our ancient ancestors depended on their ability to detect patterns in sounds, derive meaning from them, and adjust their behavior accordingly. Such an ecological perspective on sound and emotion forms the basis of a recent multi-level framework that aims to explain emotional responses to music in terms of a large set of psychological mechanisms. The goal of this review is to offer an updated and expanded version of the framework that can explain both ‘everyday emotions’ and ‘aesthetic emotions’. The revised framework – referred to as BRECVEMA – includes eight mechanisms: Brain Stem Reflex, Rhythmic Entrainment, Evaluative Conditioning, Contagion, Visual Imagery, Episodic Memory, Musical Expectancy, and Aesthetic Judgment. In this review, it is argued that all of the above mechanisms may be directed at information that occurs in a ‘musical event’ (i.e., a specific constellation of music, listener, and context). Of particular significance is the addition of a mechanism corresponding to aesthetic judgments of the music, to better account for typical ‘appreciation emotions’ such as admiration and awe. Relationships between aesthetic judgments and other mechanisms are reviewed based on the revised framework. It is suggested that the framework may contribute to a long-needed reconciliation between previous approaches that have conceptualized music listeners? responses in terms of either ‘everyday emotions’ or ‘aesthetic emotions’.