Multi-source heterogeneous data access management framework and key technologies for electric power Internet of Things

The power Internet of Things(IoT)is a significant trend in technology and a requirement for national strategic development.With the deepening digital transformation of the power grid,China’s power system has initially built a power IoT architecture comprising a perception,network,and platform application layer.However,owing to the structural complexity of the power system,the construction of the power IoT continues to face problems such as complex access management of massive heterogeneous equipment,diverse IoT protocol access methods,high concurrency of network communications,and weak data security protection.To address these issues,this study optimizes the existing architecture of the power IoT and designs an integrated management framework for the access of multi-source heterogeneous data in the power IoT,comprising cloud,pipe,edge,and terminal parts.It further reviews and analyzes the key technologies involved in the power IoT,such as the unified management of the physical model,high concurrent access,multi-protocol access,multi-source heterogeneous data storage management,and data security control,to provide a more flexible,efficient,secure,and easy-to-use solution for multi-source heterogeneous data access in the power IoT.

Design and Implementation of a Heterogeneous Database System over the Intranet

Aim To develop a heterogeneous database united system(HDBUS)that combines the local database of Oracle, Sybase and SQL server distributed on different server into a global database,and supports the global transaction management and parallel query over the Intranet Methods In the designing and implementation of HDBUS two important concepts heterogeneous tables join. Results and Conclu- tion The first concept can be used to process the parallel query of multiple database server, the second one is the key technology of heterogeneous is the key technology of heterogeneous distribute database.

Accessing Web Heterogeneous Databases Based on CORBA

The future usage of heterogeneous databases will consist of the WWW and CORBA environments. The integration of the WWW databases and CORBA standards are discussed. These two techniques need to merge together to make distributed usage of heterogeneous databases user friendly. In an environment integrating WWW databases and CORBA technologies, CORBA can be used to access heterogeneous data sources in the internet. This kind of applications can achieve distributed transactions to assure data consistency and integrity. The application of this technology is with a good prospect.

Sharing and Implementation of Heterogeneous Database for Education Resource Based on XML

The problem of sharing heterogeneous database for accessing different educational resources has to be considered. The study is carried out to realize the heterogeneous database sharing for educational resources using multi-media educa-tional resources as the researching object. XML is applied as middleware for the practical requirements of education. The study has important practical significance for the intellectualization of educational and teaching resource platform.

A Web-Based Approach for the Efficient Management of Massive Multi-source 3D Models

Effectively managing extensive,multi-source,and multi-level real-scene 3D models for responsive retrieval scheduling and rapid visualization in the Web environment is a significant challenge in the current development of real-scene 3D applications in China.In this paper,we address this challenge by reorganizing spatial and temporal information into a 3D geospatial grid.It introduces the Global 3D Geocoding System(G_(3)DGS),leveraging neighborhood similarity and uniqueness for efficient storage,retrieval,updating,and scheduling of these models.A combination of G_(3)DGS and non-relational databases is implemented,enhancing data storage scalability and flexibility.Additionally,a model detail management scheduling strategy(TLOD)based on G_(3)DGS and an importance factor T is designed.Compared with mainstream commercial and open-source platforms,this method significantly enhances the loadable capacity of massive multi-source real-scene 3D models in the Web environment by 33%,improves browsing efficiency by 48%,and accelerates invocation speed by 40%.

多数据库系统集成平台CMDatabase体系结构

在比较传统的两种实现多数据库系统的体系结构后，针对大规模的数据集成提出 了一种新的体系结构，该体系结构方便了用户的操作，提高了运行效率。

Western North Pacific Tropical Cyclone Database Created by the China Meteorological Administration

This paper describes the access to, and the content, characteristics, and potential applications of the tropical cyclone(TC) database that is maintained and actively developed by the China Meteorological Administration, with the aim of facilitating its use in scientific research and operational services. This database records data relating to all TCs that have passed through the western North Pacific(WNP) and South China Sea(SCS) since 1949. TC data collection has expanded over recent decades via continuous TC monitoring using remote sensing and specialized field detection techniques,allowing collation of a multi-source TC database for the WNP and SCS that covers a long period, with wide coverage and many observational elements. This database now comprises a wide variety of information related to TCs, such as historical or real-time locations(i.e., best track and landfall), intensity, dynamic and thermal structures, wind strengths, precipitation amounts, and frequency. This database will support ongoing research into the processes and patterns associated with TC climatic activity and TC forecasting.

Research on Heterogeneous Data Sharing in Early Warning System for Grain Security

The data nodes with heterogeneous database in early warning system for grain security seriously hampered the effective data collection in this system. In this article,the existing middleware technologies was analyzed,the problem-solution approach of heterogeneous data sharing was discussed through middleware technologies. Based on this method,and according to the characteristics of early warning system for grain security,the technology of data sharing in this system were researched and explored to solve the issues of collection of heterogeneous data sharing.

Hetero-DB： Next Generation High-Performance Database Systems by Best Utilizing Heterogeneous Computing and Storage Resources

With recent advancement on hardware technologies, new general-purpose high-performance devices have been widely adopted, such as the graphics processing unit （GPU） and solid state drive （SSD）. GPU may offer an order of higher throughput for applications with massive data parallelism, compared with the multicore CPU. Moreover, new storage device SSD is also capable of offering a much higher I/O throughput and lower latency than a traditional hard disk device （HDD）. These new hardware devices can significantly boost the performance of many applications;thus the database community has been actively engaging in adopting them into database systems. However, the performance benefit cannot be easily reaped if the new hardwares are improperly used. In this paper, we propose Hetero-DB, a high-performance database system by exploiting both the characteristics of the database system and the special properties of the new hardware devices in system’s design and implementation. Hetero-DB develops a GPU-aware query execution engine with GPU device memory management and query scheduling mechanism to support concurrent query execution. Furthermore, with the SSD-HDD hybrid storage system, we redesign the storage engine by organizing HDD and SSD into a two-level caching hierarchy in Hetero-DB. To best utilize the hybrid hardware devices, the semantic information that is critical for storage I/O is identified and passed to the storage manager, which has a great potential to improve the e？ciency and performance. Hetero-DB aims to maximize the performance benefits of GPU and SSD, and demonstrates the effectiveness for designing next generation database systems.

智能云平台异构数据库协同检索算法研究

智能云平台是一种集成各种资源和功能的高效计算平台,可以为用户提供灵活的数据存储和高效的数据检索服务.随着信息技术飞速发展,异构数据库中数据呈爆炸式增长.为了提升网络异构数据库检索效果,提出智能云平台异构数据库协同检索算法.构建智能云平台异构数据库,均衡异构数据库中的节点能耗;排序云平台多源异构数据,预处理异构数据;建立以索引库为核心的检索服务引擎,利用神经网络提取多源异构数据特征,实现异构数据库的匹配检索.测试结果表明,所提算法查准率为96%,查全率为94%,数据丢失量仅为1.由此证明,所提方法有效提高了网络异构数据库检索效果.

BioMGE——一个用于采集和分析生物医用材料和多组学数据的数据库

随着机器学习技术的发展,生物医用材料的科学研究也逐渐走向数据驱动,利用生物测序技术来测试生物医用材料的生物功能,需要对生物医用材料进行进一步优化.因此,一个开放、共享的基础设施来存储来自不同研究领域的异构科学数据是多学科交叉联合分析的基石.本文介绍了BioMGE,一个基于灵活的自定义平台NMDMS(国家材料数据管理与服务平台)实现的数据库建设案例,用于收集生物医用材料和多组学测序数据.NMDMS的动态容器框架允许用户定义个性化的数据提交模式,存储来自生物医用材料和多组学研究领域的数据.自2019年以来,BioMGE已收集了1547100个生物医用材料和多组学数据集.BioMGE提供了数据导出接口,方便用户直接导出数据以进行数据分析.以组学数据可视化为例,提供BioMGE-viewer模块以实现对生物染色质结构数据从一维到三维的数据可视化.该数据库可为其他跨领域研究的数据共享提供了新的思路和平台.

跨院区分布式医疗数据采集融合平台设计与实现

目的 解决区域性数据中心或多院区医疗机构进行跨院区数据采集、融合所面临的突出困难。方法综合运用全增量无缝衔接、数据变更捕获、一致性比对、容错机制及带宽优化等关键技术,设计了数据汇聚部署、同步组织、管理控制等平台功能模块。结果 构建了分布式数据采集融合平台,解决多源异构数据归一性、完整性和保鲜性问题,实现跨院区、跨系统、跨领域的医疗数据实时汇聚融合。结论 分布式数据采集融合平台支持区域性数据中心或多院区医疗机构对医疗、管理、科研等方面数据价值进行挖掘,对满足全量实时贴源数据的迫切需求进行有效的技术性探索。

多灾害多源异构数据库构建与管理系统实现

近年来全球极端天气频发,诱发了大量灾害,如洪水、泥石流、崩塌等。构建多灾害多源异构数据库与数据管理系统,对于防灾减灾以及灾后应急救援、灾后重建等均具有重要意义。首先,介绍多灾害多源异构数据库与管理系统的构架;其次,详细介绍了基于MySQL关系型数据库的管理系统设计,并应用Python语言开发数据采集程序,实现对地震、海啸、洪水、波浪、火灾五类极端灾害数据信息的高效采集和标准化;再次,应用序列化与反序列化、事务处理两项关键技术实现了数据库的高效存储与检索;最后,进行数据库管理系统功能测试。结果表明,所建数据库及管理系统运行良好。该数据库的构建与管理系统的实现可为减小极端灾害风险以及增加各地抗灾韧性提供重要技术支持。

基于模糊C均值聚类的异构数据信息检索仿真

不同数据库系统可能使用不同查询语义和查询表达方式,为了获取高质量的异构数据库信息检索结果,提出一种混合云环境下异构数据库信息检索算法。优先建立局部本体,在局部本体的基础上形成信息领域本体,抽取局部信息。在信息领域本体的指导下对局部信息展开集成,获取全局信息,完成混合云环境下异构数据库信息集成。通过对全部信息展开检索特征匹配,组建异构数据库空间规划模型,采用模糊C均值聚类算法对信息聚类,最终实现混合云环境下异构数据库信息检索。实验结果表明,采用所提算法展开异构数据库信息检索的精度更高,且相对检索增益较高,说明其检索效果较好。

基于异构数据库的物联网系统鉴权模块设计

随着信息通信技术的发展,物联网的应用规模和多样性呈现出爆炸式增长,使用物联网设备的人员也越来越多,如何对不同的人员给设备设置相对应的权限也越来越复杂,过去简单的将每个人员的权限存在Mysql里这种方式的效率已经不适用于越来越庞大的物联网系统。文章提出了一种基于异构数据库的权限验证方法,该方法使用Redis数据库用于权限数据的缓存,使用Mysql数据库存储和管理各类人员的权限数据以及各类数据的属性数据。此方法可以提升不同人员在操作物联网设备时对权限验证的效率。

我国较大及以上生产安全事故特征及严重程度分析

为实现我国较大及以上生产安全事故(MWSA)数据的有效管理,减少事故发生及降低事故严重程度,基于2010—2022年国务院安委会挂牌督办的278起MWSA,建立通用的MWSA数据库,从事故发生时间、空间、行业、类型维度探究事故特征分布,并依据事故严重程度的综合量化指标,构建分位数回归模型全面识别事故严重程度的影响因素。结果表明:在时空分布上,6—9月的事故起数和死亡人数处于高峰,日照充足的白天(7:00—18:00)事故起数和死亡人数占比多达2/3,经济发达省/直辖市(京津江浙沪闽)事故较少;爆炸和车辆伤害为主要事故类型。在0.05的显著性水平下,事故类型、发生日期、季节、日照条件、天气、平均温度、企业员工规模、企业成立时长等8个因素分别在不同分位点处与事故严重程度显著相关。其中,天气、平均温度是低严重度事故中影响事故后果的重要因素,100人以下的企业员工规模因素增大了极端严重事故发生的可能性。

实用化数据同步技术在新型配电系统规划多人协同机制中的应用

随着能源互联网时代的到来,将各能源部门内部或部门之间使用的不同数据库整合成统一的数据平台,既适应能源互联网发展趋势,又能为新型配电系统网格化规划多人协同机制的实现奠定基础。异构数据库的同步,关键在于源端数据变更的捕获、目的端数据操作的辨识以及目的端数据的同步。在源端变更数据捕获方面,通过创建临时表获取源端的变更数据,大大减少了传输数据量。在数据操作辨识方面,采用源端利用组合编码统一标识更新数据、目的端自行解析的方式,避免了解读的混乱。为省去大量同类语句的解析时间,在目的端事先制定好各类数据操作的存储过程,以实现同步数据的便捷写入。将该实用化数据同步技术应用在株洲市新型配电系统能源互联网规划多人协同机制中,成功提高了效率,证明了方法的有效性和可行性。实用化数据同步技术既能将源端的变更数据以系统开销最小的方式准确抓取出来,又能使目的端的数据写入更加快捷。

云上异构数据库管理技术(DMS)研究

为了解决企业中因国产数据库及国外数据库等异构数据库增多而带来的管理难度增加、运维成本高涨等问题,文中基于云服务的异构数据库管理(DMS),采用了Web模式。其提供了异构数据库对象研究、异构数据库SQL脚本兼容、异构数据库SQL执行分析、异构数据库SQL结果展示及审计等能力,解决了当前数据库管理工具良莠不齐,存在费用高、安装困难、部署耗时、难上手的问题,致力于帮助企业高效、安全的挖掘数据价值,助力企业数字化转型。

多源异构数据融合与高性能图数据库查询引擎设计

文章针对多源异构数据整合及高性能图数据库查询引擎设计问题,提出创新技术解决方案。通过深度融合实现数据整合与高效存储,并引入先进图数据库技术,以快速响应复杂查询请求。实验证明,该系统在多源异构数据融合和图数据库查询方面具有明显的性能优势,为实际应用提供了可行解决方案。

多源异构数据融合与高性能图数据库查询引擎设计

随着信息时代的快速发展,多源异构数据的整合和利用变得日益重要。文中着眼于解决多源异构数据融合与高性能图数据库查询引擎设计的问题,提出了一种创新的方法。首先,探讨了数据融合的重要性以及在整合过程中面临的挑战,特别是在异构数据类型、格式和来源多样的情况下。接着,介绍了高性能图数据库的基本原理,并根据数据融合的需求设计了一种高效的查询引擎。文中设计了一个多层次的数据融合模型,并提出了相应的查询优化算法,以应对数据融合过程的复杂性和多样性。