The rapidly increasing scale of data warehouses is challenging today＇s data analytical technologies. A con- ventional data analytical platform processes data warehouse queries using a star schema -- it normalizes the...The rapidly increasing scale of data warehouses is challenging today＇s data analytical technologies. A con- ventional data analytical platform processes data warehouse queries using a star schema -- it normalizes the data into a fact table and a number of dimension tables, and during query processing it selectively joins the tables according to users＇ demands. This model is space economical. However, it faces two problems when applied to big data. First, join is an expensive operation, which prohibits a parallel database or a MapReduce-based system from achieving efficiency and scalability simultaneously. Second, join operations have to be executed repeatedly, while numerous join results can actually be reused by different queries. In this paper, we propose a new query processing frame- work for data warehouses. It pushes the join operations par- tially to the pre-processing phase and partially to the post- processing phase, so that data warehouse queries can be transformed into massive parallelized filter-aggregation oper- ations on the fact table. In contrast to the conventional query processing models, our approach is efficient, scalable and sta- ble despite of the large number of tables involved in the join. It is especially suitable for a large-scale parallel data ware- house. Our empirical evaluation on Hadoop shows that our framework exhibits linear scalability and outperforms some existing approaches by an order of magnitude.展开更多
In this paper, the authors present the design and implementation of an Interoperable Object Platform for Multi-Databases (IOPMD). The aim of the system is to provide a uniform object view and a set of tools for object...In this paper, the authors present the design and implementation of an Interoperable Object Platform for Multi-Databases (IOPMD). The aim of the system is to provide a uniform object view and a set of tools for object manipu lation and query based on heterogeneous multiple data sources under client/server environment. The common object model is compatible with ODMG2.0 and OMG'sCORBA, which provides main OO features such as OID, attribute, method, inheri tance, reference, etc. Three types of interfaCes, namely Vface, IOQL and C++ API, are given to provide the database programmer with tools and functionalities for application development. Nested transactions and compensating technology are adopted in transaction manager. In discussing some key 'implementation techlliques, translation and mapping approaches from various schemata to a common object schemaare proposed. Buffer management provides the data caching policy and consistency maintenance of cached data. Version managemellt presellts some operations based on the definitions in semantic version model, and introduces the implemelltation of the semantic version graph.展开更多
文摘The rapidly increasing scale of data warehouses is challenging today＇s data analytical technologies. A con- ventional data analytical platform processes data warehouse queries using a star schema -- it normalizes the data into a fact table and a number of dimension tables, and during query processing it selectively joins the tables according to users＇ demands. This model is space economical. However, it faces two problems when applied to big data. First, join is an expensive operation, which prohibits a parallel database or a MapReduce-based system from achieving efficiency and scalability simultaneously. Second, join operations have to be executed repeatedly, while numerous join results can actually be reused by different queries. In this paper, we propose a new query processing frame- work for data warehouses. It pushes the join operations par- tially to the pre-processing phase and partially to the post- processing phase, so that data warehouse queries can be transformed into massive parallelized filter-aggregation oper- ations on the fact table. In contrast to the conventional query processing models, our approach is efficient, scalable and sta- ble despite of the large number of tables involved in the join. It is especially suitable for a large-scale parallel data ware- house. Our empirical evaluation on Hadoop shows that our framework exhibits linear scalability and outperforms some existing approaches by an order of magnitude.
文摘In this paper, the authors present the design and implementation of an Interoperable Object Platform for Multi-Databases (IOPMD). The aim of the system is to provide a uniform object view and a set of tools for object manipu lation and query based on heterogeneous multiple data sources under client/server environment. The common object model is compatible with ODMG2.0 and OMG'sCORBA, which provides main OO features such as OID, attribute, method, inheri tance, reference, etc. Three types of interfaCes, namely Vface, IOQL and C++ API, are given to provide the database programmer with tools and functionalities for application development. Nested transactions and compensating technology are adopted in transaction manager. In discussing some key 'implementation techlliques, translation and mapping approaches from various schemata to a common object schemaare proposed. Buffer management provides the data caching policy and consistency maintenance of cached data. Version managemellt presellts some operations based on the definitions in semantic version model, and introduces the implemelltation of the semantic version graph.
