基于大数据分布式技术架构在上网行为分析中的解决方案

基于大数据技术上的上网行为分析是运营商经营分析发展的必然趋势。构建大数据的高效率平台,建立满足大数据全生命周期的数据存储与信息管理基础架构,是大数据对企业信息技术部门最重要的挑战。就综合运营商的上网日志、信令等对大数据分布式技术架构在运营商上网行为分析中的必要性和特点进行了描述,最后给出了基于大数据分布式技术架构在上网行为分析中的解决方案以及相应的核心关键技术。     

面向中文文本分类的字符级对抗样本生成方法

对抗样本生成是一种通过添加较小扰动信息,使得神经网络产生误判的技术,可用于检测文本分类模型的鲁棒性。目前,中文领域对抗样本生成方法主要有繁体字和同音字替换等,这些方法都存在对抗样本扰动幅度大,生成对抗样本质量不高的问题。针对这些问题,该文提出一种字符级对抗样本生成方法(PGAS),通过对多音字进行替换可以在较小扰动下生成高质量的对抗样本。首先,构建多音字字典,对多音字进行标注;然后对输入文本进行多音字替换;最后在黑盒模式下进行对抗样本攻击实验。实验在多种情感分类数据集上,针对多种最新的分类模型验证了该方法的有效性。     

基于一种混合语言模型的自动文本分类技术研究

随着Internet以及Intranet中大量可利用信息的爆炸式增长,文本分类成为处理和组织大量文档数据的关键技术之一。该文提出一种本体论和统计方法相结合的混合语言模型,用以解决自动文本分类问题。首先,通过学习不同类别的训练语料,分别获得各自类别的语言本体知识库,构造成为不同类别的分类器。对于实际文档,将基于不同类别的语言本体知识库分别获得对文档的评价值,并以所获得的最高评价值决定该文档的类别归属。与Bayes,k-nearest neighbor,support vector machine等3种典型的文本分类器进行了比较。实验结果表明,该文方法的分类性能均胜于其上述3种方法。     

基于类别分布差异和VPRS特征选择的文本分类方法

权值计算和特征降维是影响文本分类的精度和效率的两个重要步骤。该文首先根据特征词的类别分布差异进行特征过滤;然后,分析传统的权值公式TF-IDF的缺点,采用改进的权值计算公式简记为TF-CDF,依据TF-CDF公式计算每个特征词的权值,生成文档集的向量空间模型VSM;接着,提出了一种基于可变精度粗糙理论(VPRS)的特征选择进一步选择对分类贡献度大的特征,并用SQL实现。最后利用支持向量机LibSVM分类器进行实验,实验结果表明特征过滤和选择方法及TF-CDF权值公式有助于提高分类精度和分类效率。     

Indexing text with approximate q-grams

We present a new index for approximate string matching. The index collects text q-samples, that is, disjoint text substrings of length q, at fixed intervals and stores their positions. At search time, part of the text is filtered out by noticing that any occurrence of the pattern must be reflected in the presence of some text q-samples that match approximately inside the pattern. Hence the index points out the text areas that could contain occurrences and must be verified. The index parameters permit load balancing between filtering and verification work, and provide a compromise between the space requirement of the index and the error level for which the filtration is still efficient. We show experimentally that the index is competitive against others that take more space, being in fact the fastest choice for intermediate error levels, an area where no current index is useful.     

Fast,Flexible Text Search Using Genomic Short‐Read Mapping Model

The searching of an extensive document database for documents that are locally similar to a given query document, and the subsequent detection of similar regions between such documents, is considered as an essential task in the fields of information retrieval and data management. In this paper, we present a framework for such a task. The proposed framework employs the method of short‐read mapping, which is used in bioinformatics to reveal similarities between genomic sequences. In this paper, documents are considered biological objects; consequently, edit operations between locally similar documents are viewed as an evolutionary process. Accordingly, we are able to apply the method of evolution tracing in the detection of similar regions between documents. In addition, we propose heuristic methods to address issues associated with the different stages of the proposed framework, for example, a frequency‐based fragment ordering method and a locality‐aware interval aggregation method. Extensive experiments covering various scenarios related to the search of an extensive document database for documents that are locally similar to a given query document are considered, and the results indicate that the proposed framework outperforms existing methods.     

Landscape of ROD9 Island: Functional annotations and biological network of hypothetical proteins in Salmonella enterica

Salmonella, an Enterobacteria is a therapeutically important pathogen for the host. The advancement of genome sequencing of S. enterica serovar Enteritidis have identified a distinct ROD9 pathogenic island, imparting virulence. The occurrence of 17 ROD9 hypothetical proteins, necessitates subsequent bioinformatics approach for structural and functional aspects of protein-protein relations or networks in different pathogenic phenotypes express. A collective analysis using predictive bioinformatics tools that includes NCBI-BLASTp and BLAST2GO annotated the motif patterns and functional significance. The VFDB identified 10 virulence proteins at both genomic and metagenomic level. Phylogenetic analysis revealed a divergent and convergent relationship between 17 ROD9 and 41 SP-1 proteins. Here, combining a comprehensive approach from sequence based, motif recognitions, domain identification, virulence ability to structural modelling provides a precise function to ROD9 proteins biological network, for which no experimental information is available.     

A new universality class in corpus of texts; A statistical physics study

Text can be regarded as a complex system. There are some methods in statistical physics which can be used to study this system. In this work, by means of statistical physics methods, we reveal new universal behaviors of texts associating with the fractality values of words in a text. The fractality measure indicates the importance of words in a text by considering distribution pattern of words throughout the text. We observed a power law relation between fractality of text and vocabulary size for texts and corpora. We also observed this behavior in studying biological data.