异构调度环境下ETL工作流监控系统的设计与实现*

本文提出一种分布式ETL在异构调度环境下的工作流统一监控方法。通过对异构调度系统元数据的提取和抽象，采用增量抽取和跟踪监控对象的方式，通过网络消息传送将监控对象加载到监控数据中心并统一管理，实现了分布式ETL工作流在异构调度环境下的的全局监控，提高了数据恢复能力和仓库的数据质量。

Design and implementation of monitoring system for the ETL workflows in heterogeneous scheduling environment

Abstract: During the Data Warehouse building stage, the process of the data Extraction, Transformation and Loading (ETL) affect the quality of the Data Warehouse directly, and eventually affect the accuracy and the effectiveness of data mining and analyzing results which based on the warehouse. In order to manage the data processing, master the data flowing direction and improve the data quality effectively, this paper provided a unified monitoring method of the Distributed ETL workflows in heterogeneous environment. By extracting and abstracting the metadata of each workflow and tracing the monitored objects incrementally, all the ETL jobs could be monitored and managed by the monitor data center, meanwhile improving the quality of the Data Warehouse.