Economy of effort in different speaking conditions. I. A preliminary study of intersubject differences and modeling issues

This study explores the hypothesis that clear speech is produced with greater "articulatory effort" than normal speech. Kinematic and acoustic data were gathered from seven subjects as they pronounced multiple repetitions of utterances in different speaking conditions, including normal, fast, clear, and slow. Data were analyzed within a framework based on a dynamical model of single-axis frictionless movements, in which peak movement speed is used as a relative measure of articulatory effort (Nelson, 1983). There were differences in peak movement speed, distance and duration among the conditions and among the speakers. Three speakers produced the "clear" condition utterances with movements that had larger distances and durations than those for "normal" utterances. Analyses of the data within a peak speed, distance, duration "performance space" indicated increased effort (reflected in greater peak speed) in the clear condition for the three speakers, in support of the hypothesis. The remaining four speakers used other combinations of parameters to produce the clear condition. The validity of the simple dynamical model for analyzing these complex movements was considered by examining several additional parameters. Some movement characteristics differed from those required for the model-based analysis, presumably because the articulators are complicated structurally and interact with one another mechanically. More refined tests of control strategies for different speaking styles will depend on future analyses of more complicated movements with more realistic models.     

Control of oral closure in lingual stop consonant production

Previous work has shown that the lips are moving at a high velocity when the oral closure occurs for bilabial stop consonants, resulting in tissue compression and mechanical interactions between the lips. The present experiment recorded tongue movements in four subjects during the production of velar and alveolar stop consonants to examine kinematic events before, during, and after the stop closure. The results show that, similar to the lips, the tongue is often moving at a high velocity at the onset of closure. The tongue movements were more complex, with both horizontal and vertical components. Movement velocity at closure and release were influenced by both the preceding and the following vowel. During the period of oral closure, the tongue moved through a trajectory of usually less than 1 cm; again, the magnitude of the movement was context dependent. Overall, the tongue moved in forward-backward curved paths. The results are compatible with the idea that the tongue is free to move during the closure as long as an airtight seal is maintained. A new interpretation of the curved movement paths of the tongue in speech is also proposed. This interpretation is based on the principle of cost minimization that has been successfully applied in the study of hand movements in reaching.     

Control of phonemic length contrast and speech rate in vocalic and consonantal syllable nuclei

This paper investigates the mechanisms controlling the phonemic quantity contrast and speech rate in nonsense p(1)Np(2)a words read by five Slovak speakers in normal and fast speech rate. N represents a syllable nucleus, which in Slovak corresponds to long and short vowels and liquid consonants. The movements of the lips and the tongue were recorded with an electromagnetometry system. Together with the acoustic durations of p(1), N, and p(2), gestural characteristics of three core movements were extracted: p(1) lip opening, tongue movement for (N)ucleus, and p(2) lip closing. The results show that, although consonantal and vocalic nuclei are predictably different on many kinematic measures, their common phonological behavior as syllabic nuclei may be linked to a stable temporal coordination of the consonantal gestures flanking the nucleus. The functional contrast between phonemic duration and speech rate was reflected in the bias in the control mechanisms they employed: the strategies robustly used for signaling phonemic duration, such as the degree of coproduction of the two lip movements, showed a minimal effect of speech rate, while measures greatly affected by speech rate, such as p(2) acoustic duration, or the degree of p(1)-N gestural coproduction, tended to be minimally influenced by phonemic quantity.     

Articulatory-acoustic kinematics: the production of American English /s/

Due to its aerodynamic, articulatory, and acoustic complexities, the fricative /s/ is known to require high precision in its control, and to be highly resistant to coarticulation. This study documents in detail how jaw, tongue front, tongue back, lips, and the first spectral moment covary during the production of /s/, to establish how coarticulation affects this segment. Data were obtained from 24 speakers in the Wisconsin x-ray microbeam database producing /s/ in prevocalic and pre-obstruent sequences. Analysis of the data showed that certain aspects of jaw and tongue motion had specific kinematic trajectories, regardless of context, and the first spectral moment trajectory corresponded to these in some aspects. In particular contexts, variability due to jaw motion is compensated for by tongue-tip motion and bracing against the palate, to maintain an invariant articulatory-aerodynamic goal, constriction degree. The change in the first spectral moment, which rises to a peak at the midpoint of the fricative, primarily reflects the motion of the jaw. Implications of the results for theories of speech motor control and acoustic-articulatory relations are discussed.     

An EMA study of VCV coarticulatory direction

This study addresses three issues that are relevant to coarticulation theory in speech production: whether the degree of articulatory constraint model (DAC model) accounts for patterns of the directionality of tongue dorsum coarticulatory influences; the extent to which those patterns in tongue dorsum coarticulatory direction are similar to those for the tongue tip; and whether speech motor control and phonemic planning use a fixed or a context-dependent temporal window. Tongue dorsum and tongue tip movement data on vowel-to-vowel coarticulation are reported for Catalan VCV sequences with vowels /i/, /a/, and /u/, and consonants /p/, /n/, dark /l/, /s/, /S/, alveolopalatal /n/ and /k/. Electromidsagittal articulometry recordings were carried out for three speakers using the Carstens articulograph. Trajectory data are presented for the vertical dimension for the tongue dorsum, and for the horizontal dimension for tongue dorsum and tip. In agreement with predictions of the DAC model, results show that directionality patterns of tongue dorsum coarticulation can be accounted for to a large extent based on the articulatory requirements on consonantal production. While dorsals exhibit analogous trends in coarticulatory direction for all articulators and articulatory dimensions, this is mostly so for the tongue dorsum and tip along the horizontal dimension in the case of lingual fricatives and apicolaminal consonants. This finding results from different articulatory strategies: while dorsal consonants are implemented through homogeneous tongue body activation, the tongue tip and tongue dorsum act more independently for more anterior consonantal productions. Discontinuous coarticulatory effects reported in the present investigation suggest that phonemic planning is adaptative rather than context independent.     

Lip-jaw and tongue-jaw coordination during rate-controlled syllable repetitions

The present study investigated the relationship between functionally relevant compound gestures and single-articulator component movements of the jaw and the constrictors lower lip and tongue tip during rate-controlled syllable repetitions. In nine healthy speakers, the effects of speaking rate (3 vs 5 Hz), place of articulation, and vowel type during stop consonant-vowel repetitions (/pa/, /pi/, /ta/, /ti/) on the amplitude and peak velocity of differential jaw and constrictor opening-closing movements were measured by means of electromagnetic articulography. Rather than homogeneously scaled compound gestures, the results suggest distinct control mechanisms for the jaw and the constrictors. In particular, jaw amplitude was closely linked to vowel height during bilabial articulation, whereas the lower lip component amplitude turned out to be predominantly rate sensitive. However, the observed variability across subjects and conditions does not support the assumption that single-articulator gestures directly correspond to basic phonological units. The nonhomogeneous effects of speech rate on articulatory subsystem parameters indicate that single structures are differentially rate sensitive. On average, an increase in speech rate resulted in a more or less proportional increase of the steepness of peak velocity/amplitude scaling for jaw movements, whereas the constrictors were less rate sensitive in this respect. Negative covariation across repetitions between jaw and constrictor amplitudes has been considered an indicator of motor equivalence. Although significant in some cases, such a relationship was not consistently observed across subjects. Considering systematic sources of variability such as vowel height, speech rate, and subjects, jaw-constrictor amplitude correlations showed a nonhomogeneous pattern strongly depending on place of articulation.     

Control of rate and duration of speech movements

A computerized pulsed-ultrasound system was used to monitor tongue dorsum movements during the production of consonant-vowel sequences in which speech rate, vowel, and consonant were varied. The kinematics of tongue movement were analyzed by measuring the lowering gesture of the tongue to give estimates of movement amplitude, duration, and maximum velocity. All three subjects in the study showed reliable correlations between the amplitude of the tongue dorsum movement and its maximum velocity. Further, the ratio of the maximum velocity to the extent of the gesture, a kinematic indicator of articulator stiffness, was found to vary inversely with the duration of the movement. This relationship held both within individual conditions and across all conditions in the study such that a single function was able to accommodate a large proportion of the variance due to changes in movement duration. As similar findings have been obtained both for abduction and adduction gestures of the vocal folds and for rapid voluntary limb movements, the data suggest that a wide range of changes in the duration of individual movements might all have a similar origin. The control of movement rate and duration through the specification of biomechanical characteristics of speech articulators is discussed.     

Lip kinematics in long and short stop and fricative consonants

This paper examines lip and jaw kinematics in the production of labial stop and fricative consonants where the duration of the oral closure/constriction is varied for linguistic purposes. The subjects were speakers of Japanese and Swedish, two languages that have a contrast between short and long consonants. Lip and jaw movements were recorded using a magnetometer system. Based on earlier work showing that the lips are moving at a high velocity at the oral closure, it was hypothesized that speakers could control closure/constriction duration by varying the position of a virtual target for the lips. According to this hypothesis, the peak vertical position of the lower lip during the oral closure/constriction should be higher for the long than for the short consonants. This would result in the lips staying in contact for a longer period. The results show that this is the case for the Japanese subjects and one Swedish subject who produced non-overlapping distributions of closure/ constriction duration for the two categories. However, the peak velocity of the lower lip raising movement did not differ between the two categories. Thus if the lip movements in speech are controlled by specifying a virtual target, that control must involve variations in both the position and the timing of the target.     

Managing the distinctiveness of phonemic nasal vowels: articulatory evidence from Hindi

There is increasing evidence that fine articulatory adjustments are made by speakers to reinforce and sometimes counteract the acoustic consequences of nasality. However, it is difficult to attribute the acoustic changes in nasal vowel spectra to either oral cavity configuration or to velopharyngeal opening (VPO). This paper takes the position that it is possible to disambiguate the effects of VPO and oropharyngeal configuration on the acoustic output of the vocal tract by studying the position and movement of the tongue and lips during the production of oral and nasal vowels. This paper uses simultaneously collected articulatory, acoustic, and nasal airflow data during the production of all oral and phonemically nasal vowels in Hindi (four speakers) to understand the consequences of the movements of oral articulators on the spectra of nasal vowels. For Hindi nasal vowels, the tongue body is generally lowered for back vowels, fronted for low vowels, and raised for front vowels (with respect to their oral congeners). These movements are generally supported by accompanying changes in the vowel spectra. In Hindi, the lowering of back nasal vowels may have originally served to enhance the acoustic salience of nasality, but has since engendered a nasal vowel chain shift.     

Temporal measures of anticipatory labial coarticulation for the vowel/u/: within- and cross-subject variability.

The timing of upper lip protrusion movements and accompanying acoustic events was examined for multiple repetitions of word pairs such as "lee coot" and "leaked coot" by four speakers of American English. The duration of the intervocalic consonant string was manipulated by using various combinations of /s/, /t/, /k/, /h/, and /#/. Pairwise comparisons were made of consonant string duration (acoustic /i/ offset to acoustic /u/ onset) with durations of: protrusion movement beginning to acoustic /u/ onset, maximum acceleration of the movement to acoustic /u/ onset, and acoustic /u/ onset to movement end. There were some consonant-specific protrusion effects, primarily on the movement beginning event for /s/. Inferences from measures of the maximum acceleration and movement end events for the non-/s/ subset suggested the simultaneous and variable expression of three competing constraints: (1) end the protrusion movement during the voiced part of the /u/; (2) use a preferred movement duration; and (3) begin the /u/-related protrusion movement when permitted by relaxation of the perceptually motivated constraint that the preceding /i/ be unrounded. The subjects differed in the degree of expression of each constraint, but the results generally indicate that anticipatory coarticulation of lip protrusion is influenced both by acoustic-phonetic context dependencies and dynamical properties of movements. Because of the extensive variation in the data and the small number of subjects, these ideas are tentative; additional work is needed to explore them further.     

Tongue movement kinematics in long and short Japanese consonants

This paper examines tongue movements in stop and fricative consonants where the duration of the oral closure/constriction for the consonant is varied for linguistic purposes. Native speakers of Japanese served as subjects. The linguistic material consisted of Japanese word pairs that only differed in the duration of the lingual consonant, which was either long or short. Recordings were made of tongue movements using a magnetometer system. Results show a robust difference in closure duration between the long and short consonants. Overall, the path of the tongue movement during the consonant was longer for the long than for the short consonant. All speakers decreased the speed of the tongue movement during the long consonant. These adjustments in tongue movements were most likely made to maintain the contact between the tongue and the palate for the closure and constriction.     

Mandible movements during increasingly rapid articulations of single syllables: preliminary observations

A previous study of mandible movements in normal speech [W.L. Nelson, J. Acoust. Soc. Am. Suppl. 1 68, S32 (1980)] suggested that the speech motor control process resulted in a relationship between movement time, distance, and peak velocity which implied (1) some adaptation to the physical effort required for the movement, and (2) that the force limit effective during speech was considerably below that which the mandibular muscles are capable of producing. In the present study, mandible movements were measured during maximally rapid opening and closing tasks, and during increasingly rapid repetitions of a spoken syllable and a nonspeech "syllable." The results indicate that (1) peak acceleration levels for the repeated mandible movements were less than half of those for the maximally rapid single mandible movements, and (2) a rather different mode of control is used for the repeated speech movements as compared to the nonspeech movements.     

Effect of speaking rate on vowel formant movements

The purpose of this experiment was to study the effects of changes in speaking rate on both the attainment of acoustic vowel targets and the relative time and speed of movements toward these presumed targets. Four speakers produced a number of different CVC and CVCVC utterances at slow and fast speaking rates. Spectrographic measurements showed that the midpoint format frequencies of the different vowels did not vary as a function of rate. However, for fast speech the onset frequencies of second formant transitions were closer to their target frequencies while CV transition rates remained essentially unchanged, indicating that movement toward the vowel simply began earlier for fast speech. Changes in both speaking rate and lexical stress had different effects. For stressed vowels, an increase in speaking rate was accompanied primarily by a decrease in duration. However, destressed vowels, even if they were of the same duration as quickly produced stressed vowels, were reduced in overall amplitude, fundamental frequency, and to some extent, vowel color. These results suggest that speaking rate and lexical stress are controlled by two different mechanisms.     

Kinematic and electromyographic responses to perturbation of the jaw

The task-dependent organization of sensorimotor mechanisms during the production of speech was investigated using a perturbation paradigm. Six subjects received unanticipated jaw perturbations before and during tongue elevation for [aedae], in which the lips do not participate, and bilabial closure for [aebae], in which the tongue does not participate. A strain gauge system was used to monitor inferior-superior displacements of the upper lip, lower lip, and jaw, while hooked-wire electrodes monitored muscle activity in various muscles of the lips, jaw, and tongue. Results indicated significant compensatory kinematic adjustments to jaw perturbations in the lips and/or jaw during [aebae], but no labial compensations during [aedae] (with the exception of one subject). EMG responses were inconsistent and not necessarily indicative of the kinematic findings. Individual subjects responded to perturbations reliably but differently, using different combinations of involved articulators to achieve bilabial closure and lingua-alveolar contact. The current study supports earlier research which suggests that the components of the motor system are flexibly assembled, based on the requirements of the specific task. That is, compensatory responses to sensory information occur only when such responses are functionally necessary.     

Lingual kinematics and coarticulation for alveolopalatal and velar consonants in Catalan

Vertical lingual movement data for the alveolopalatal consonants /?/ and /?/ and for the dorsovelar consonant /k/ in Catalan /aCa/ sequences produced by three speakers reveal that the tongue body travels a smaller distance at a slower speed and in a longer time during the lowering period extending from the consonant into the following vowel (CV) than during the rising period extending from the preceding vowel into the consonant (VC). For two speakers, two-phase trajectories characterized by two successive velocity peaks occur more frequently during the former period than during the latter, whether associated with tongue blade and dorsum (for alveolopalatals) or with the tongue dorsum articulator alone (for velars). Greater tongue dorsum involvement for /?/ and /k/ than for /?/ accounts for a different kinematic relationship between the four articulatory phases. The lingual gesture for alveolopalatals and, less so, that for velars may exert more prominent spatial and temporal effects on V2 than on V1 which is in agreement with the salience of the C-to-V carryover component associated with these consonants according to previous coarticulation studies. These kinematic and coarticulation data may be attributed to tongue dorsum biomechanics to a large extent.     

Physiological Studies of Retrusive Movements of the Human Tongue

SUMMARY: This study identified that physiologically the superior pharyngeal constrictor muscle at the level of the base of the tongue contributes to retrusive movement of the tongue with constriction of the mid-pharyngeal cavity and possesses unique properties in terms of motor speech control along with the genioglossus muscle. From a kinematic study involving trans-nasal fiberscopy and lateral X-ray fluorography, retrusive movement of the tongue was highly correlated with constrictive movement of the mid-pharyngeal cavity. An electromyographic study revealed that the superior pharyngeal constrictor muscle at the level of the base of the tongue contributes to retrusive movement of the tongue and that the genioglossus muscle contributes to protrusive movement. We also noted that this relationship between the activities of these two muscles were in response to postural changes during vowel productions without changes in the acoustic features. These findings suggest that these two muscles act not only antagonistically to produce retrusive and protrusive movement of the tongue, but also they complement each other to conserve the shape of the vocal tract for speech production. The functional relationship between these two muscles could contribute the consecutive movement of human speech production under various conditions and might be useful when applying rehabilitation approaches for the patients with neurological speech and swallowing disorders.     

Influences of tongue biomechanics on speech movements during the production of velar stop consonants: a modeling study

This study explores the following hypothesis: forward looping movements of the tongue that are observed in VCV sequences are due partly to the anatomical arrangement of the tongue muscles, how they are used to produce a velar closure, and how the tongue interacts with the palate during consonantal closure. The study uses an anatomically based two-dimensional biomechanical tongue model. Tissue elastic properties are accounted for in finite-element modeling, and movement is controlled by constant-rate control parameter shifts. Tongue raising and lowering movements are produced by the model mainly with the combined actions of the genioglossus, styloglossus, and hyoglossus. Simulations of V1CV2 movements were made, where C is a velar consonant and V is [a], [i], or [u]. Both vowels and consonants are specified in terms of targets, but for the consonant the target is virtual, and cannot be reached because it is beyond the surface of the palate. If V1 is the vowel [a] or [u], the resulting trajectory describes a movement that begins to loop forward before consonant closure and continues to slide along the palate during the closure. This pattern is very stable when moderate changes are made to the specification of the target consonant location and agrees with data published in the literature. If V1 is the vowel [i], looping patterns are also observed, but their orientation was quite sensitive to small changes in the location of the consonant target. These findings also agree with patterns of variability observed in measurements from human speakers, but they contradict data published by Houde [Ph.D. dissertation (1967)]. These observations support the idea that the biomechanical properties of the tongue could be the main factor responsible for the forward loops when V1 is a back vowel, regardless of whether V2 is a back vowel or a front vowel. In the [i] context it seems that additional factors have to be taken into consideration in order to explain the observations made on some speakers.     

Speaker Recognition Using Constrained Convolutional Neural Networks in Emotional Speech

Speaker recognition is an important classification task, which can be solved using several approaches. Although building a speaker recognition model on a closed set of speakers under neutral speaking conditions is a well-researched task and there are solutions that provide excellent performance, the classification accuracy of developed models significantly decreases when applying them to emotional speech or in the presence of interference. Furthermore, deep models may require a large number of parameters, so constrained solutions are desirable in order to implement them on edge devices in the Internet of Things systems for real-time detection. The aim of this paper is to propose a simple and constrained convolutional neural network for speaker recognition tasks and to examine its robustness for recognition in emotional speech conditions. We examine three quantization methods for developing a constrained network: floating-point eight format, ternary scalar quantization, and binary scalar quantization. The results are demonstrated on the recently recorded SEAC dataset.     

A three-dimensional model of tongue movement based on ultrasound and x-ray microbeam data

Point-tracking techniques provide timing information about structural movements of the tongue. Imaging techniques provide information about cross-sectional and pharyngeal tongue shape and movement. This study joined these techniques in a single subject. Five pellets on the tongue surface were tracked using x-ray microbeam, and the midsagittal and coronal planes of the tongue were imaged using real-time ultrasound. The speech materials were the consonants [s] and [l] and the vowels [i], [a], and [o] combined in VCVCe utterances. Analyses concentrated on the difference in tongue movements related to the two consonants. A model of tongue movement was developed, in which critical features of consonant shape and position dominated the tongue opening movement. In this model, the tongue is divided into subdivisions termed "functional segments" in both the sagittal and coronal planes. Movements of the functional segments created observable opening movement patterns.     

An approach to real-time magnetic resonance imaging for speech production

Magnetic resonance imaging (MRI) has served as a valuable tool for studying static postures in speech production. Now, recent improvements in temporal resolution are making it possible to examine the dynamics of vocal-tract shaping during fluent speech using MRI. The present study uses spiral k-space acquisitions with a low flip-angle gradient echo pulse sequence on a conventional GE Signa 1.5-T CV/i scanner. This strategy allows for acquisition rates of 8-9 images per second and reconstruction rates of 20-24 images per second, making veridical movies of speech production now possible. Segmental durations, positions, and interarticulator timing can all be quantitatively evaluated. Data show clear real-time movements of the lips, tongue, and velum. Sample movies and data analysis strategies are presented.