基于Hadoop的质检大数据分析的研究

随着社会生产的发展,政府监督和企业委托的产品质量检验产生了规模非常巨大的质检数据.这些数据持续快速增长,数据类型多样化,针对对海量的质检数据进行高效率的访问,并进行有效的分析利用,为政府和公众提供质量监管、预测和预警的帮助.现提出了一种基于Hadoop分布式计算技术进行存储和分析质检大数据的方法,设计了质检大数据分析的架构和主要部分的实现代码.该设计具有良好的高扩展性和高可用性,能满足质检大数据的存储和挖掘等要求.

Research on quality inspect big data analysis based on Hadoop

With the development of the social production, large-scale data of quality inspection are produced by the government supervision and enterprise commission. These data have been continued rapid growth, data types are various. How to efficiently access and use massive data of quality inspection for helping government and public to enhance quality supervision, quality prediction, quality warning. The paper presents a method of storing and analyzing quality inspection data based on Hadoop, designs the architecture and realizes the major code. The design has good scalability and high availability, meets the requirements of storage and mining.