期刊论文

Research and application of a multidimensional association rules mining algorithm based on Hadoop

中文

计算机工程与科学

1007-130X

2019

41

12

2127-2133

10.3969/j.issn.1007-130X.2019.12.005

国家自然科学基金(61532008) 国家重点研发计划(2017YFC0909502).

本校为第一机构

传统的Apriori算法要多次扫描数据集,随着数据量的快速增长,传统的Apriori算法已经不能很好地适用于大数据分析,针对该情况设计了IPApriori算法。首先通过剪枝策略设计了一种适用于多维数据的IApriori算法,再将IApriori算法与Hadoop分布式框架相结合,实现了多维关联规则挖掘算法的并行化。将IPApriori算法运用到手机用户行为预测关联分析中,分析影响手机用户行为的一些主要因素,挖掘出手机用户行为与年龄维度、性别维度、时间维度、地点维度和手机品牌维度属性之间可能存在的某种关联。最后通过实验证明,算法的并行化和建立结构的方法可以降低系统的I/O负荷,提高算法的执行效率。

The traditional Apriori algorithm has to scan the data set multiple times.With the rapid growth of data volume,it cannot be applied to big data analysis.For this problem,an improved parallel Apriori algorithm is designed.Firstly,an IApriori algorithm for multidimensional data is designed by pruning strategy.Secondly,the IApriori algorithm is combined with the Hadoop distributed framework to realize the parallelization of multidimensional association rules mining algorithm.This paper applies the IPApriori algorithm to the correlation analysis of mobile phone user behavior prediction,analyzes so...