Uniform Representation Model for Metadata of Data Warehouse

A uniform metadata representation is introduced for heterogeneous databases, multi media information and other information sources. Some features about metadata are analyzed. The limitation of existing metadata model is compared with the new one. The metadata model is described in XML which is fit for metadata denotation and exchange. The well structured data, semi structured data and those exterior file data without structure are described in the metadata model. The model provides feasibility and extensibility for constructing uniform metadata model of data warehouse.

ROLE OF META-MODEL IN ENGINEERING DATA WAREHOUSE

Engineering data are separately organized and their schemas are increasingly complex and variable. Engineering data management systems are needed to be able to manage the unified data and to be both customizable and extensible. The design of the systems is heavily dependent on the flexibility and self-description of the data model. The characteristics of engineering data and their management facts are analyzed. Then engineering data warehouse (EDW) architecture and multi-layer metamodels are presented. Also an approach to manage anduse engineering data by a meta object is proposed. Finally, an application flight test EDW system (FTEDWS) is described and meta-objects to manage engineering data in the data warehouse are used. It shows that adopting a meta-modeling approach provides a support for interchangeability and a sufficiently flexible environment in which the system evolution and the reusability can be handled.

ETL Maturity Model for Data Warehouse Systems:A CMMI Compliant Framework

The effectiveness of the Business Intelligence(BI)system mainly depends on the quality of knowledge it produces.The decision-making process is hindered,and the user’s trust is lost,if the knowledge offered is undesired or of poor quality.A Data Warehouse(DW)is a huge collection of data gathered from many sources and an important part of any BI solution to assist management in making better decisions.The Extract,Transform,and Load(ETL)process is the backbone of a DW system,and it is responsible for moving data from source systems into the DW system.The more mature the ETL process the more reliable the DW system.In this paper,we propose the ETL Maturity Model(EMM)that assists organizations in achieving a high-quality ETL system and thereby enhancing the quality of knowledge produced.The EMM is made up of five levels of maturity i.e.,Chaotic,Acceptable,Stable,Efficient and Reliable.Each level of maturity contains Key Process Areas(KPAs)that have been endorsed by industry experts and include all critical features of a good ETL system.Quality Objectives(QOs)are defined procedures that,when implemented,resulted in a high-quality ETL process.Each KPA has its own set of QOs,the execution of which meets the requirements of that KPA.Multiple brainstorming sessions with relevant industry experts helped to enhance the model.EMMwas deployed in two key projects utilizing multiple case studies to supplement the validation process and support our claim.This model can assist organizations in improving their current ETL process and transforming it into a more mature ETL system.This model can also provide high-quality information to assist users inmaking better decisions and gaining their trust.

Refreshing File Aggregate of Distributed Data Warehouse in Sets of Electric Apparatus

Integrating heterogeneous data sources is a precondition to share data for enterprises. Highly-efficient data updating can both save system expenses, and offer real-time data. It is one of the hot issues to modify data rapidly in the pre-processing area of the data warehouse. An extract transform loading design is proposed based on a new data algorithm called Diff-Match,which is developed by utilizing mode matching and data-filtering technology. It can accelerate data renewal, filter the heterogeneous data, and seek out different sets of data. Its efficiency has been proved by its successful application in an enterprise of electric apparatus groups.

The Proposal of Data Warehouse Validation

The analysis of relevant standards and guidelines proved the lack of information on actions and activities concerning data warehouse testing. The absence of the complex data warehouse testing methodology seems to be crucial particularly in the phase of the data warehouse implementation. The aim of this article is to suggest basic data warehouse testing activities as a final part of data warehouse testing methodology. The testing activities that must be implemented in the process of the data warehouse testing can be split into four logical units regarding the multidimensional database testing, data pump testing, metadata and OLAP （Online Analytical Processing） testing. Between main testing activities can be included： revision of the multidimensional database scheme, optimizing of fact tables number, problem of data explosion, testing for correctness of aggregation and summation of data etc.

A trajectory data warehouse solution for workforce management decision-making

In modern workforce management,the demand for new ways to maximize worker satisfaction,productivity,and security levels is endless.Workforce movement data such as those source data from an access control system can support this ongoing process with subsequent analysis.In this study,a solution to attaining this goal is proposed,based on the design and implementation of a data mart as part of a dimensional trajectory data warehouse(TDW)that acts as a repository for the management of movement data.A novel methodological approach is proposed for modeling multiple spatial and temporal dimensions in a logical model.The case study presented in this paper for modeling and analyzing workforce movement data is to support human resource management decision-making and the following discussion provides a representative example of the contribution of a TDW in the process of information management and decision support systems.The entire process of exporting,cleaning,consolidating,and transforming data is implemented to achieve an appropriate format for final import.Structured query language(SQL)queries demonstrate the convenience of dimensional design for data analysis,and valuable information can be extracted from the movements of employees on company premises to manage the workforce efficiently and effectively.Visual analytics through data visualization support the analysis and facilitate decisionmaking and business intelligence.

Research on the Construction of a Data Warehouse Model for College Student Performance

Students’grades not only serve as an effective indicator of their learning achievements but also to some extent reflect the completion of teaching tasks by the instructors.Currently,many universities across the country have collected and recorded various information about students and teachers in the school’s information management system,but it is only a simple storage record and has not effectively excavated hidden information,and data have not been fully utilized.Student performance information,enrolment information,course information,teaching plans,and teacher-related information are currently stored in separate databases,which are independent of each other,making it difficult to perform effective data analysis.Data warehousing technology can integrate various information and use data analysis software to excavate more high-value information,which is convenient for teaching evaluation and optimizing teaching strategies.Based on data warehousing technology,the article uses the hierarchical concept of data warehousing to construct the ODS layer,DWD layer,DWS layer and ETL layer.Facing the data warehousing topic,the article designs the data warehousing conceptual model,logical model,and physical model based on student performance,providing a model basis for later data mining.

Modelling and implementing big data warehouses for decision support

In the era of Big Data,many NoSQL databases emerged for the storage and later processing of vast volumes of data,using data structures that can follow columnar,key-value,document or graph formats.For analytical contexts,requiring a Big Data Warehouse,Hive is used as the driving force,allowing the analysis of vast amounts of data.Data models in Hive are usually defined taking into consideration the queries that need to be answered.In this work,a set of rules is presented for the transformation of multidimensional data models into Hive tables,making available data at different levels of detail.These several levels are suited for answering different queries,depending on the analytical needs.After the identification of the Hive tables,this paper summarizes a demonstration case in which the implementation of a specific Big Data architecture shows how the evolution from a traditional Data Warehouse to a Big Data Warehouse is possible.

Research on Multidimensional Data Model in Data Warehouse

Data structure and semantics of the traditional data model cannot effectively represent the data warehouse, it is difficult to effectively support online analytical processing （referred to as OLAP）. This paper is propose a new multidimensional data model based on the partial ordering and mapping. The data model can fully express the complex data structure and semantics of data warehouse, and provide an OLAP operation as the core of the operation of algebra, support structure in levels of complex aggregation operation sequence, which can effectively support the application of OLAE The data model supports the concept of aggregation function constraint, and provides constraint mechanism of the hierarchy aggregation function.

Performance optimization of grid aggregation in spatial data warehouses

The problem of storage and querying of large volumes of spatial grids is an issue to solve.In this paper,we propose a method to optimize queries to aggregate raster grids stored in databases.In our approach,we propose to estimate the exact result rather than calculate the exact result.This approach reduces query execution time.One advantage of our method is that it does not require implementing or modifying functionalities of database management systems.Our approach is based on a new data structure and a specific model of SQL queries.Our work is applied here to relational data warehouses.

Constructing a raster-based spatio-temporal hierarchical data model for marine fisheries application

Marine information has been increasing quickly. The traditional database technologies have disadvantages in manipulating large amounts of marine information which relates to the position in 3-D with the time. Recently, greater emphasis has been placed on GIS （geographical information system）to deal with the marine information. The GIS has shown great success for terrestrial applications in the last decades, but its use in marine fields has been far more restricted. One of the main reasons is that most of the GIS systems or their data models are designed for land applications. They cannot do well with the nature of the marine environment and for the marine information. And this becomes a fundamental challenge to the traditional GIS and its data structure. This work designed a data model, the raster-based spatio-temporal hierarchical data model （RSHDM）, for the marine information system, or for the knowledge discovery fi＇om spatio-temporal data, which bases itself on the nature of the marine data and overcomes the shortages of the current spatio-temporal models when they are used in the field. As an experiment, the marine fishery data warehouse （FDW） for marine fishery management was set up, which was based on the RSHDM. The experiment proved that the RSHDM can do well with the data and can extract easily the aggregations that the management needs at different levels.

抢先赢得商机的Data Mining──基于数据仓库的数据挖掘技术

首先介绍了数据仓库以及在此技术上产生的数据挖掘技术,其次阐述了实现数据挖掘应用的几种工具以及选用工具时应遵循的原则,最后说明了数据挖掘技术现存的问题及他现在重要的商业地位。

Organization and storage model of marine information and its application in the "China Digital Ocean"

Based on the experience and achievement of the＂China Digital Ocean＂, the classification plan for Marine data elements is made, which can be classified into five, including marine point elements, marine line elements, marine polygon elements, marine grid elements and marine dynamic elements. In this paper, the technology of features and object-oriented method, a spatial-temporal data model is proposed, which can be applied in the large information system engineering like the ＂Digital Ocean＂, and this paper discusses the application of spatial data model, marine three-dimensional raster data model and relation data model in the building of Data Warehouse in ＂China Digital Ocean＂, and concludes the merits of these models.

Applying XML for designing and interchanging information for multidimensional model

In order to exchange and share information among the conceptual models of data warehouse, and to build a solid base for the integration and share of metadata, a new multidimensional concept model is presented based on XML and its DTD is defined, which can perfectly describe various semantic characteristics of multidimensional conceptual model. According to the multidimensional conceptual modeling technique which is based on UML, the mapping algorithm between the multidimensional conceptual model is described based on XML and UML class diagram, and an application base for the wide use of this technique is given.

如何用Warehouse Studio创建和管理数据仓库

介绍了数据仓库的体系结构和WarehouseStudio,然后结合实例详细叙述了利用WarehouseStudio创建、管理和维护数据仓库的过程。

Data Warehouse for Quality Management Systems

The objectives of quality management systems which are based on data warehouses are to acquire, store, and process quality control data within an enterprise, and to facilitate analysis, control and decision making based on this data. This paper discusses the DB/ODS/DW (traditional database/operational data store/data warehouse) architecture, data granularity and data partition in the data warehouse, describes the data model, and presents the client/server platform model.

Model-Apriori模型在医疗决策支持系统中的应用

本文主要讨论了Model-Apriori模型在医疗决策支持系统中的应用。模型Model-Apriori是单维关联规则挖掘模型。它主要用于完成从医院现有数据库中挖掘知识,以便使医院的管理者通过挖掘到的知识来调整对医院的管理。本模型主要应用关联规则中的Apriori算法。

基于大数据的高校教务域数仓应用研究

针对当前业界大数据方法论的实践对高校场景化业务支撑有限、大数据视角的教育数据价值未能得到有效挖掘利用的问题,探索高校数据仓库建设方法路径。以构建可持续迭代的智慧校园数字基础设施为目标,以大数据维度建模方法论为基础,结合高校数据自身的特点,以教务域数仓的建设为例,通过深入了解源业务系统、丰富数仓建模知识,总结出了一套对高校来说切实可行的建模思路和方法论,其创新性主要体现在数据整合、标准化数据管理、建模方法和建设方式等方面。该方法为发掘数据价值、促进校级能力整合、推动高校实现数字化转型提供了基本保障。

基于Hadoop技术的加速器大数据安全存储与高效分析系统设计

为了解决当前加速器控制系统在数据管理方面所面临的海量数据安全存储和高效分析处理的问题,在现有的基础上引入了Hadoop大数据框架,并结合大数据其他相关组件,构建一个分布式数据仓库系统。文章详细阐述了数据仓库的搭建过程,包括软硬件架构以及将数据从现有数据库抽取、转换和加载到数据仓库的方案。特别是针对系统归档数据的存储和分析需求,根据实际应用场景设计一个基于HBase的存储解决方案。在系统部署完成后,进行了吞吐量测试,并与当前使用的传统数据库进行性能对比。测试结果显示,基于Hadoop的数据仓库系统在海量数据存储、高性能查询以及数据分析处理方面都表现出明显的优势。这一改进为加速器控制系统提供了更强大的数据管理和处理能力,有望为加速器在未来的发展提供更多可能性。

基于AdaBoost模型和SVM模型的铁水温度预测

以某炼铁厂3号高炉实际生产数据为基础,针对原始数据存在的重复值、缺失值和异常值等问题进行数据处理,选用AdaBoost模型和SVM模型对铁水温度进行预测。结果表明,AdaBoost模型相较于SVM模型取得更好的预测效果,R达到0.878,±5℃预测准确率为85.21%,可满足高炉实际生产需要。基于高炉炼铁数据仓库系统,建立FineBI前端工具与铁水温度预测应用的数据连接,构建由特征参数、相关性分析、预测结果、模型评估和预测曲线等模块组成的前端界面,实现了高炉铁水温度预测应用的可视化展示。