A statistical approach to crosslingual natural language tasks

The existence of huge volumes of documents written in multiple languages on Internet leads to investigate novel algorithmic approaches to deal with information of this kind. However, most crosslingual natural language processing (NLP) tasks consider a decoupled approach in which monolingual NLP techniques are applied along with an independent translation process. This two-step approach is too sensitive to translation errors, and in general to the accumulative effect of errors. To solve this problem, we propose to use a direct probabilistic crosslingual NLP system which integrates both steps, translation and the specific NLP task, into a single one. In order to perform this integrated approach to crosslingual tasks, we propose to use the statistical IBM 1 word alignment model (M1). The M1 model may show a non-monotonic behaviour when aligning words from a sentence in a source language to words from another sentence in a different, target language. This is the case of languages with different word order. In English, for instance, adjectives appear before nouns, whereas in Spanish it is exactly the opposite. The successful experimental results reported in three different tasks - text classification, information retrieval and plagiarism analysis - highlight the benefits of the statistical integrated approach proposed in this work.     

Link test--A statistical method for finding prostate cancer biomarkers

We present a new method, link-test, to select prostate cancer biomarkers from SELDI mass spectrometry and microarray data sets. Biomarkers selected by link-test are supported by data sets from both mRNA and protein levels, and therefore results in improved robustness. Link-test determines the level of significance of the association between a microarray marker and a specific mass spectrum marker by constructing background mass spectra distributions estimated by all human protein sequences in the SWISS-PROT database. The data set consist of both microarray and mass spectrometry data from prostate cancer patients and healthy controls. A list of statistically justified prostate cancer biomarkers is reported by link-test. Cross-validation results show high prediction accuracy using the identified biomarker panel. We also employ a text-mining approach with OMIM database to validate the cancer biomarkers. The study with link-test represents one of the first cross-platform studies of cancer biomarkers.     

Iterative Denoising

One problem in many fields is knowledge discovery in heterogeneous, high-dimensional data. As an example, in text mining an analyst often wishes to identify meaningful, implicit, and previously unknown information in an unstructured corpus. Lack of metadata and the complexities of document space make this task difficult. We describe Iterative Denoising, a methodology for knowledge discovery in large heterogeneous datasets that allows a user to visualize and to discover potentially meaningful relationships and structures. In addition, we demonstrate the features of this methodology in the analysis of a heterogeneous Science News corpus.     

不确定性推理在文本分类上的应用研究

在文本分类中k-NN分类方法简洁而有效,但在多类分类问题中,由于类的重叠和属性的不足导致训练文本和类边界出现不确定性,而传统k-NN分类方法无法处理这种不确定性.该文借助于几种经典的不确定性推理方法:DS证据理论、模糊集理论、模糊-粗糙集理论,来改进传统k-NN文本分类方法,实验表明基于不确定性推理的方法能够提高文本分类的精度和召回率.     

Two-phase biomedical named entity recognition using CRFs

As a fundamental step of biomedical text mining, Biomedical Named Entity Recognition (Bio-NER) remains a challenging task. This paper explores a so-called two-phase approach to identify biomedical entities, in which the recognition task is divided into two subtasks: Named Entity Detection (NED) and Named Entity Classification (NEC). And the two subtasks are finished in two phases. At the first phase, we try to identify each named entity with a Conditional Random Fields (CRFs) model without identifying its type; at the second phase, another CRFs model is used to determine the correct entity type for each identified entity. This treatment can reduce the training time significantly and furthermore, more relevant features can be selected for each subtask. In order to achieve a better performance, post-processing algorithms are employed before NEC subtask. Experiments conducted on JNLPBA2004 datasets show that our two-phase approach can achieve an F-score of 74.31%, which outperforms most of the state-of-the-art systems.     

The Mathematics of Internet Search Engines

This article presents a survey of techniques for ranking results in search engines, with emphasis on link-based ranking methods and the PageRank algorithm. The problem of selecting, in relation to a user search query, the most relevant documents from an unstructured source such as the WWW is discussed in detail. The need for extending classical information retrieval techniques such as boolean searching and vector space models with link-based ranking methods is demonstrated. The PageRank algorithm is introduced, and its numerical and spectral properties are discussed. The article concludes with an alternative means of computing PageRank, along with some example applications of this new method.     

基于文本分类的火控系统故障诊断研究

以某型装备火控系统为例,将文本分类技术同基于支持向量机的故障诊断方法结合,通过建立故障特征词库、采用布尔模型形成故障向量库,运用SVM算法对该装备火控系统的故障进行了训练评估,并获得了较理想的试验结果,最大识别率达到了70%。通过这种方法进行装备故障诊断,对于装备维修特别是战场抢修有极其重要的意义,使维修人员从繁琐的仪器检查中解脱出来,通过已有的故障库快捷简便地确定故障检测点,实现装备的快速抢修,为抢夺战场主动权创造有利条件。     

基于论域划分的无监督文本特征选择方法

由于缺乏类信息,使得无监督文本特征选择问题一直未较好地加以解决。为此,对该问题进行了研究并提出了一个基于论域划分的无监督文本特征选择。该方法主要是把论域划分的思想引入到无监督文本特征选择之中,其首先使用一种新型无监督文档进行文本特征初选以过滤低频的噪声词,然后再使用所给的基于论域划分的属性约简进行文本特征优选。实验结果表明这个方法能够克服文本聚类时缺乏类的先验知识的不足,可以较好地解决无监督文本特征选择问题。     

Fast and flexible packed string matching

Searching for all occurrences of a pattern in a text is a fundamental problem in computer science with applications in many other fields, like natural language processing, information retrieval and computational biology. In the last two decades a general trend has appeared trying to exploit the power of the word RAM model to speed-up the performances of classical string matching algorithms. In this model an algorithm operates on words of length w, grouping blocks of characters, and arithmetic and logic operations on the words take one unit of time.In this paper we use specialized word-size packed string matching instructions, based on the Intel streaming SIMD extensions (SSE) technology, to design a very fast string matching algorithm. We evaluate our solution in terms of efficiency, stability and flexibility, where we propose to use the deviation in running time of an algorithm on distinct equal length patterns as a measure of stability.From our experimental results it turns out that, despite their quadratic worst case time complexity, the new presented algorithm becomes the clear winner on the average in many cases, when compared against the most recent and effective algorithms known in literature.     

文本语法与历史遗产类旅游文本的英译

本文侧重于人文景观中的历史遗产类旅游文本的英译。本文尝试对从Werlich文本语法入手,探讨如何通过有效的翻译提高历史遗产类旅游文本的效果。在功能翻译原则指引下,对文本的外部制约和内部制约进行分析对比。从而总结出目前历史遗产类旅游文本存在的不足。