Visual Speech Recognition with Lightweight Psychologically Motivated Gabor Features

Extraction of relevant lip features is of continuing interest in the visual speech domain. Using end-to-end feature extraction can produce good results, but at the cost of the results being difficult for humans to comprehend and relate to. We present a new, lightweight feature extraction approach, motivated by human-centric glimpse-based psychological research into facial barcodes, and demonstrate that these simple, easy to extract 3D geometric features (produced using Gabor-based image patches), can successfully be used for speech recognition with LSTM-based machine learning. This approach can successfully extract low dimensionality lip parameters with a minimum of processing. One key difference between using these Gabor-based features and using other features such as traditional DCT, or the current fashion for CNN features is that these are human-centric features that can be visualised and analysed by humans. This means that it is easier to explain and visualise the results. They can also be used for reliable speech recognition, as demonstrated using the Grid corpus. Results for overlapping speakers using our lightweight system gave a recognition rate of over 82%, which compares well to less explainable features in the literature.     

Reliability of Speaking and Maximum Voice Range Measures in Screening for Dysphonia

Speech range profile (SRP) is a graphical display of frequency-intensity occurring interactions during functional speech activity. Few studies have suggested the potential clinical applications of SRP. However, these studies are limited to qualitative case comparisons and vocally healthy participants. The present study aimed to examine the effects of voice disorders on speaking and maximum voice ranges in a group of vocally untrained women. It also aimed to examine whether voice limit measures derived from SRP were as sensitive as those derived from voice range profile (VRP) in distinguishing dysphonic from healthy voices. Ninety dysphonic women with laryngeal pathologies and 35 women with normal voices, who served as controls, participated in this study. Each subject recorded a VRP for her physiological vocal limits. In addition, each subject read aloud the "North Wind and the Sun" passage to record SRP. All the recordings were captured and analyzed by Soundswell's computerized real-time phonetogram Phog 1.0 (Hitech Development AB, T?by, Sweden). The SRPs and the VRPs were compared between the two groups of subjects. Univariate analysis results demonstrated that individual SRP measures were less sensitive than the corresponding VRP measures in discriminating dysphonic from normal voices. However, stepwise logistic regression analyses revealed that the combination of only two SRP measures was almost as effective as a combination of three VRP measures in predicting the presence of dysphonia (overall prediction accuracy: 93.6% for SRP vs 96.0% for VRP). These results suggest that in a busy clinic where quick voice screening results are desirable, SRP can be an acceptable alternate procedure to VRP.     

重庆方言语音识别系统的设计与实现

语音识别赋予了计算机能够识别出语音内容的功能,是人机交互技术领域的重要研究内容。随着计算机技术的发展,语音识别已经得到了成熟的发展。但是关于方言的语音识别还有很大的发展空间。中国是一个幅员辽阔、人口众多的国家,因此方言种类繁多,其中有3000多万人交流使用的重庆方言就是其中之一。采集了重庆方言的部分词语的文本文件和对应的语音文件建立语料库,根据重庆方言的发音特点,选取重庆方言的声韵母作为声学建模基元,选取隐马尔可夫模型(Hidden Markov Model, HMM)为声学模型设计了一个基于HMM的重庆方言语音识别系统。在训练过程利用语料库中训练集语料对声学模型进行训练,形成HMM模型库;在识别过程利用语料库中的测试集语料进行识别测试。实验结果表明,该系统能够实现重庆方言的语音识别,并且识别的正确率为100%。     

The speaker''s formant in male voices

Spectral analysis of vowels during connected speech can be performed using the spectral intensity distribution within critical bands corresponding to a natural scale on the basilar membrane. Normalization of the spectra provides the opportunity to make objective comparisons independent from the recording level. An increasing envelope peak between 3,150 and 3,700 Hz has been confirmed statistically for a combination of seven vowels in three groups of male speakers with hoarse, normal, and professional voices. Each vowel is also analyzed individually. The local energy maximum is called “the speaker's formant” and can be found in the region of the fourth formant. The steepness of the spectral slope (i.e. the rate of decline) becomes less pronounced when the sonority or the intensity of the voice increases. The speaker's formant is connected with the sonorous quality of the voice. It increases gradually and is approximately 10 dB higher in professional male voices than in normal male voices at neutral loudness (60 dB at 0.3 min). The peak intensity becomes stronger (30 dB above normal voices) when the overall speaking loudness is increased to 80 dB. Shouting increases the spectral energy of the adjacent critical bands but not the speaker's formant itself.     

第四讲语音信号处理的现状和展望

文章简要介绍了“语音信号处理”这一分支学科形成和发展的历史过程．指出了它在现代信息科学技术中的地位和作用．介绍了语音信号处理在应用领域的一些重要课题，如语音的低速率编码，语音的规则合成和文一语转换系统，语音识别和人一机语音对话等，这些仍然是当前研究的热点．文章最后展望了语音信号处理的发展前景，指出在这个领域还有很多难题等待人们去研究探索．     

Study of the formant and duration in Chinese whispered vowel speech

Study on the acoustical characteristic is important to speech and speaker recognition in Chinese whispered speech. In this paper, the characteristics of whispered speech are introduced and the acoustical characteristics in Chinese whispered speech are discussed. There is no fundamental frequency in the whispered speech, so other characteristics such as the duration and frequency of formant are extracted and analyzed. From experiments with six simple Chinese whispered vowels, it is proved that the duration and the frequency of formant can be used as the main acoustical characteristics in the Chinese whispered recognition.     

Att-U-Net：融合注意力机制的U-Net骨导语声增强*

近年来大量全卷积网络、U-Net等编解码网络结构应用于语音增强,它们具有计算复杂度低、模型参数少等优势。然而,与长短时记忆模型等方法相比,这些编解码结构仍存在不能充分利用先后时间之间和高低频率之间的关联信息等缺点,尤其对于长序列数据的输入,编解码结构存在信息丢失的问题。为保持计算效率的同时考虑更充分的时频关联信息建模,本文提出一种融合注意力机制的U-Net网络的骨导语音增强方法（Att-U-Net）,通过在跳跃连接中引入注意力机制,生成一个权重矩阵,将编码层中的全局信息根据权重融入对应的解码层中,使网络在编解码过程中能够关注输入数据中与增强目标相关程度高的重要信息,同时抑制不相关的信息。在骨导语音数据集上的实验表明,融合注意力机制的U-Net网络能在保持模型轻量化的同时有效提升骨导语音的增强效果,增强后的语音在各项客观评价指标上均优于基线模型。通过对编解码网络中间层的可视化分析发现,在解码过程中注意力机制有效地保留了有声段的信息,滤除了骨导语音由于骨导传声特性带来的中频共振,从而使得增强后的骨导语音具有较好的听觉效果。     

汉语儿童情感语声合成

情感语声合成技术对于人机交互具有重要的意义。面对儿童情感语声合成所需汉语语声数据资源缺乏以及模型训练时长较长等问题,该文提出利用迁移学习实现汉语儿童情感语声合成的方法。首先基于汉语语声数据库训练深度学习模型实现中文语声端到端合成模型,再使用高质量大样本的中文情感语料库完成情感语声合成模型,最后利用自行采样的小样本汉语儿童情感语料对模型进行迁移学习实现低资源的语声合成。客观实验结果中梅尔倒谱失真指标为4.91,主观听辨实验指标分别为3.61和4.17。通过实验对比表明,该文的方法在情感语声合成技术的应用上具有良好的性能表现,并且优于现有先进的低资源情感语声合成方法。     

一种适用于双微阵列的语音增强算法研究

考虑到传统单通道语音增强算法对噪声抑制的局限性,本文采用由两个微型麦克风阵列组成的双微阵列,利用该阵列空间结构的时空域特性对含噪语音进行处理,提出了一种适用于双微阵列的语音增强算法。该增强算法是将各通道采集到的带噪语音信号先使用对数最小均方误差(Logarithmic Minimunm Mean Square Error,LogMMSE)提升其信噪比,然后利用频域宽带最小方差无畸变响应(MVDR)通过对目标声源信号的获取,保留目标声源方向的信号并抑制其他方向的信号干扰,最后通过一个改进可懂度结合改进最小控制递归平均(Improved Minimum Controlled Recursive Average Algorithm,IMCRA)噪声估计的维纳滤波器来去除噪声残留提升语音质量。仿真实验结果表明,相比传统的单通道语音增强算法,该算法具有良好的噪声抑制性能。