期刊论文

中文

计算机应用研究

1001-3695

2016

33

2

375-377,383

10.3969/j.issn.1001-3695.2016.02.012

本校为第一机构

为了降低中文文本相似度计算方法的时间消耗、提高文本聚类的准确率,提出了一种PST_LDA(词性标注潜在狄利克雷模型)中文文本相似度计算方法。首先,对文本中的名词、动词和其他词进行词性标注;然后,分别对名词、动词和其他词建立相应的LDA主题模型;最后,按照一定的权重比例综合这三个主题模型,计算文本之间的相似度。由于考虑了不同词性的词集对文本相似度计算的贡献差异,利用文本的语义信息提高了文本聚类准确率。将分离后的三个词集的LDA建模过程并行化,减少建模的时间消耗,提高文本聚类速度。在TanCorp-12数据集分别用LDA和PST_LDA方法进行中文文本相似度计算模拟实验。实验结果显示,PST_LDA方...

This paper introduced a new text similarity algorithm, which was based on the PST_LDA （part-of-speech tagging LDA） , to reduce the time complexity of Chinese text similarity calculation and improve the accuracy of text clustering. The al- gorithm had three procedures. Firstly, it divided the words into noun set, verb set and last words set according to the part of speech. Secondly, applied a LDAimodel to each set. Finally, it combined three models according to the certain proportion and computed the distance of two texts by JS similarity dist...