自动发音错误检测中基于F1值最大化的声学模型训练方法

摘要为了提高计算机辅助语言学习中自动发音错误检测系统的性能,提出一种声学模型的区分性训练方法。该方法将经过正确度标注的非母语语音数据库上的发音错误检测的F₁值的最大化作为模型参数的训练准则。采用Sigmoid 函数对F₁值函数进行平滑构造目标函数,并利用构造弱意义辅助函数的方法以及扩展Baum-Welch 形式的参数更新公式进行优化。提出在模型参数更新与音素门限同时优化的策略保证目标函数增长的单调性。发音错误检测实验表明该方法能够有效地增大训练和测试数据检错的F₁值。同时训练数据和测试数据上的精确度、召回率以及检测正确度都有明显改进。      

一种基于音素模型感知度的发音质量评价方法

为了提高发音质量判别精度,提出了一种基于音素模型感知度的发音质量评价方法。它采用不同语音样本集合下样本声学特征的对数后验概率期望差作为音素模型对变异发音的感知度,并以此为基础,生成各音素对应的识别模型候选集。实验表明,所提出的方法使语音识别网络候选音素模型集合尺寸减少约95%;在非母语语音数据库上,该方法评分与人工专家打分相关性为0.828,基于该方法得到的声韵母错误检出率为70.8%,声调错误检出率为42.5%,均优于其它方法。      

汉语发音质量评估的实验研究

研究了发音评估系统中通用的置信度测度——后验概率算法,针对它存在的不足,提出了两种改进方案。首先,为了降低计算复杂度,传统算法采用了求最大值算法代替求和算法,在被测发音偏离目标音素集的情况下,这会严重降低后验概率的计算精度,本文提出基于扩展的音素混淆网络的后验概率算法。其次,为使置信度能评估不同语音段长的发音质量优劣,传统算法采用了后验概率的段长规整策略,研究分析发现声学似然值与时间的关系更为紧密,所以本文提出了基于声学似然值的时间规整方案。试验结果表明:与传统算法相比,采用改进的置信度算法能使平均打分错误率相对降低35%左右,有效地改善了计算机辅助语言学习系统的性能。      

用于语音识别置信度的发音特征各维度分析和子集优化

提出了基于发音特征单个维度的置信度算法,并基于此对发音特征的各个维度展开分析.分析不仅验证了融合的必要性,同时也展示了发音特征各维度之间以及和隐马尔可夫模型之间的大量冗余.为了去除冗余,提出了用子集选择的方法进行优化.对比所有都用的情况,基于发音特征紧凑子集的语音识别置信度估计,在等错率上取得了12.7%的相对下降.把经过优化后的基于发音特征的语音识别置信度估计和基于隐马尔可夫模型的语音识别置信度进行融合,在保持集内识别率不损失的前提下,显著提高了语法外输入测试的拒识性能:在相同参数下,在开发集和测试集上分别取得了34%和35.3%的显著改善.     

基于发音特征的汉语普通话语音声学建模

将表征汉语普通话语音特点的发音特征引入汉语普通话语音识别的声学建模中,根据普通话发音特点,确定了用于区别普通话元音、辅音以及声调信息的9种发音特征,并以此为目标值训练神经网络得到语音信号属于各类发音特征的后验概率,将此概率作为语音识别的输入特征建立声学模型。在汉语普通话非特定人大词表自然口语对话识别系统中进行了实验验证,并与基于频谱特征的声学模型进行了比较,在相同解码速度下,由此方法建立的声学模型汉字错误率相对下降6.8%;将发音特征和频谱特征进行了融合实验,融合以后的识别系统相对基于频谱特征系统的汉字错误率相对下降10.1%。上述结果表明,基于发音特征的声学模型更加有效的实现了对语音特性的表征,通过利用发音特征和频谱特征的互补性,能够进一步实现对语音识别性能的提高。      

汉语普通话水平测试中儿化音的自动检测与评价

提出一种汉语普通话水平测试中儿化音的自动检测与评价方法。在现有计算机辅助发音评测系统的框架下,深入分析儿化音的发音规律和声学特性,将儿化音的检测与评价转化成典型的分类问题进行处理。经过挑选多个有代表性的声学特征,并尝试多种不同的分类算法,结果表明,集成分类回归树(Boosting CART)强化分类模型,能充分利用儿化音的各种声学特征,分类正确率达到92.41%。通过对声学特征组的进一步分析,发现共振峰、发音置信度、时长是表达儿化音的最重要线索,利用这些线索能有效地实现对儿化音的自动检测与评价。      

基于随机轨迹模型的汉语连续语音识别方法研究

本文在指出隐马尔可夫模型（HMM）不合理假设的基础上,介绍了随机轨迹模型（STM）的理论机制及优越性。随机轨迹模型将语音基元的声学观察表示为参数空间中轨迹的聚类,并将轨迹建模为状态随机序列概率密度函数的混合,该模型可以克服HMM的不合理假设,在理论上更合理。根据STM的特点及汉语语音特色,本文对汉语连续语音识别基元的选取进行了讨论,提出了音素类单元作为识别系统的识别基元。基于STM的汉语连续语音识别的实验结果证明了STM的有效性和音素类单元的一致性。     

连接时序分类准则声学建模方法优化

对基于连接时序分类准则(connectionist temporal classification,CTC)的端到端声学建模方法进行研究和优化。研究分析了不同声学特征、建模单元以及神经网络结构对CTC声学模型性能的影响,针对CTC模型中blank符号共享导致的建模缺陷提出了建模单元相关的非共享blank方法进行改进,并引入融合建模单元关联信息的模型初始化方法进一步提高CTC模型的性能。在300小时标准英文数据集Switchboard的实验结果显示,结合非共享blank、时延神经网络以及融合建模单元关联信息的初始化方法,CTC声学模型相对于基线系统在词错误率上取得绝对1.1%的下降,同时在训练速度上取得3.3倍的提高,实验结果证明本文针对端到端声学建模提出的优化方法是有效的。      

潜变量机器学习方法在咖啡NIR定量分析中的应用

采用近红外（NIR）光谱快检技术实现对咖啡蛋白质的定量检测,研究支持向量机（SVM）和极限学习机（ELM）等机器学习方法在建模分析中的实用性。结合潜变量分析技术,建立潜变量SVM（LV-SVM）模型和潜变量ELM（LV-ELM）模型,通过调试潜变量个数和机器学习关键参数的联合优选,实现数据降维和机器学习关键参数的同过程优化。运用定标-验证-测试机制,利用定标集样本建立咖啡蛋白质的NIR分析模型,随参数变动形成三维随动优选结构的建模预测结果,结合验证集样本对模型进行联合优选,然后将优化模型应用于测试集样本进行模型评价。LV-SVM建模优选的验证集预测均方根误差为6.797,对应的测试集预测均方根误差为8.384。LV-ELM建模优选的验证集预测均方根误差为6.118,对应的测试集预测均方根误差为7.837。与常规偏最小二乘（PLS）方法相比较,LV-SVM和LV-ELM方法均取得更好的预测结果,验证了潜变量机器学习方法在近红外定量分析中的应用优势,该方法有望应用于不同类型的咖啡各成分含量检测。     

近红外光谱法分析油页岩含油率中异常样品识别和剔除方法的研究

研究了漫反射近红外(NIR)光谱法分析油页岩含油率过程中异常样品的识别和剔除方法。在近红外光谱定量分析中,环境变化和操作失误等都会产生异常样品,异常样品的存在会导致模型的预测能力下降,因此异常样品的剔除是建模过程中的关键步骤。分别采用主成分分析—马氏距离(PCA-MD)法和半数重采样(RHM)法识别油页岩光谱数据中的异常样品,通过剔除异常样品后所建的偏最小二乘(PLS)分析模型的性能来评价PCA-MD与RHM方法对异常样品的识别能力。实验中考察了不同MD阈值和RHM置信度对异常样品剔除结果的影响,比较了单独和同时应用PCA-MD及RHM法识别并剔除异常样品后所得PLS模型的预测能力。结果表明：与所有样品参与建模时预测偏差均方根(RMSEP)相比,采用PCA-MD法时阈值取平均值与标准偏差之和时RMSEP降低了48.3%;采用RHM法时置信度取85%时RMSEP降低了27.5%;同时采用PCA-MD法和RHM法时RMSEP降低了44.8%,研究内容有效地提高了分析模型的预测能力。     

基于鼻韵尾分离的汉语声韵母识别模型

研究汉语自然口语识别中的建模单元选择问题。在HMM三状态模型中,声韵母单元与音素单元作为两种最流行的建模单元各有优劣。一方面从自然口语音变严重的问题出发,倾向采用粗粒度的声韵母单元以概括各种音变;另一方面从三状态结构可能无法有效描述复杂单元的问题出发,又倾向采用细粒度的音素单元。本文在实验语音学理论研究成果与声韵母时长分析实验结果的基础上,主张对扩展声韵母单元进行有选择的拆分,提出了基于鼻韵尾分离的声韵母拆分方法。实验结果表明本文的方法与扩展声韵母单元、音素单元相比,识别性能有了明显改善,其字错误率分别降低2.23%和9.45%。      

Influence of the method of acoustic feature representation on the phoneme recognition efficiency with perceptron

We study different methods of acoustic feature representation for the phoneme recognition problem using an artificial neural network. Feature representation methods are compared using the results of phoneme recognition and clustering of the parameters retrieved from speech signals. The best results of phoneme recognition are obtained by using a filter bank for acoustic feature representation. __________ Translated from Izvestiya Vysshikh Uchebnykh Zavedenii, Radiofizika, Vol. 50, No. 4, pp. 350–356, April 2007.     

多任务最小二乘支持向量回归机及其在近红外光谱分析技术中的应用研究

在近红外光谱定量分析中,许多模型分开考虑各种样品成分含量,失去了样品成分间潜在的联系.针对该问题,文章将建模分析每种样品成分含量的问题看作一个任务,将同时建模分析所有样品成分含量的问题转换为多任务学习问题.在LS-SVR的基础上,提出了多任务LS-SVR(MTLS-SVR),并给出一种有效的大规模问题求解算法.最后,以高粱样品数据集为实验材料,建立了三种样品成分(蛋白质,赖氨酸及淀粉)的同时定量分析模型.三种样品成分的预测值与实际值的平均相对误差分别为1.52%,3.04%和1.01%,相关系数分别为0.993 1,0.894 0和0.940 6,经分析比较,发现MTLS-SVR模型优于PLS,LS-SVR以及多因变量LS-SVR(MLS-SVR),从而验证了MTLS-SVR模型的可行性和有效性.     

重庆方言语音识别系统的设计与实现

语音识别赋予了计算机能够识别出语音内容的功能,是人机交互技术领域的重要研究内容。随着计算机技术的发展,语音识别已经得到了成熟的发展。但是关于方言的语音识别还有很大的发展空间。中国是一个幅员辽阔、人口众多的国家,因此方言种类繁多,其中有3000多万人交流使用的重庆方言就是其中之一。采集了重庆方言的部分词语的文本文件和对应的语音文件建立语料库,根据重庆方言的发音特点,选取重庆方言的声韵母作为声学建模基元,选取隐马尔可夫模型(Hidden Markov Model, HMM)为声学模型设计了一个基于HMM的重庆方言语音识别系统。在训练过程利用语料库中训练集语料对声学模型进行训练,形成HMM模型库;在识别过程利用语料库中的测试集语料进行识别测试。实验结果表明,该系统能够实现重庆方言的语音识别,并且识别的正确率为100%。     

Effects and modeling of phonetic and acoustic confusions in accented speech

Accented speech recognition is more challenging than standard speech recognition due to the effects of phonetic and acoustic confusions. Phonetic confusion in accented speech occurs when an expected phone is pronounced as a different one, which leads to erroneous recognition. Acoustic confusion occurs when the pronounced phone is found to lie acoustically between two baseform models and can be equally recognized as either one. We propose that it is necessary to analyze and model these confusions separately in order to improve accented speech recognition without degrading standard speech recognition. Since low phonetic confusion units in accented speech do not give rise to automatic speech recognition errors, we focus on analyzing and reducing phonetic and acoustic confusability under high phonetic confusion conditions. We propose using likelihood ratio test to measure phonetic confusion, and asymmetric acoustic distance to measure acoustic confusion. Only accent-specific phonetic units with low acoustic confusion are used in an augmented pronunciation dictionary, while phonetic units with high acoustic confusion are reconstructed using decision tree merging. Experimental results show that our approach is effective and superior to methods modeling phonetic confusion or acoustic confusion alone in accented speech, with a significant 5.7% absolute WER reduction, without degrading standard speech recognition.     

Phoneme discrimination using connectionist networks

The application of connectionist networks to speech recognition is assessed using a set of eight representative phonetic discrimination problems chose with respect to a theory of phonetics. A connectionist network model called the temporal flow model (TFM) is defined which represents temporal relationships using delay links and permits general patterns of connectivity. It is argued that the model has properties appropriate for time varying signals such as speech. Networks are trained using gradient descent methods of iterative nonlinear optimization to reduce the mean-squared error between the actual and the desired response of the output units. Separate network solutions are demonstrated for all eight phonetic discrimination problems for one male speaker. The network solutions are analyzed carefully and are shown in every case to make use of known acoustic phonetic cues. The network solutions vary in the degree to which they make use of context-dependent cues to achieve phoneme recognition. The network solutions were tested on data not used for training and achieved an average accuracy of 99.5%. It is concluded that acoustic phonetic speech recognition can be accomplished using connectionist networks.     

Multi-task learning approach for modulation and wireless signal classification for 5G and beyond: Edge deployment via model compression

Future communication networks must address the scarce spectrum to accommodate extensive growth of heterogeneous wireless devices. Efforts are underway to address spectrum coexistence, enhance spectrum awareness, and bolster authentication schemes. Wireless signal recognition is becoming increasingly more significant for spectrum monitoring, spectrum management, secure communications, among others. Consequently, comprehensive spectrum awareness on the edge has the potential to serve as a key enabler for the emerging beyond 5G (fifth generation) networks. State-of-the-art studies in this domain have (i) only focused on a single task – modulation or signal (protocol) classification – which in many cases is insufficient information for a system to act on, (ii) consider either radar or communication waveforms (homogeneous waveform category), and (iii) does not address edge deployment during neural network design phase. In this work, for the first time in the wireless communication domain, we exploit the potential of deep neural networks based multi-task learning (MTL) framework to simultaneously learn modulation and signal classification tasks while considering heterogeneous wireless signals such as radar and communication waveforms in the electromagnetic spectrum. The proposed MTL architecture benefits from the mutual relation between the two tasks in improving the classification accuracy as well as the learning efficiency with a lightweight neural network model. We additionally include experimental evaluations of the model with over-the-air collected samples and demonstrate first-hand insight on model compression along with deep learning pipeline for deployment on resource-constrained edge devices. We demonstrate significant computational, memory, and accuracy improvement of the proposed model over two reference architectures. In addition to modeling a lightweight MTL model suitable for resource-constrained embedded radio platforms, we provide a comprehensive heterogeneous wireless signals dataset for public use.