Spectral clustering eigenvector selection of hyperspectral image based on the coincidence degree of data distribution

Spectral clustering is a well-regarded subspace clustering algorithm that exhibits outstanding performance in hyperspectral image classification through eigenvalue decomposition of the Laplacian matrix.However,its classification accuracy is severely limited by the selected eigenvectors,and the commonly used eigenvectors not only fail to guarantee the inclusion of detailed discriminative information,but also have high computational complexity.To address these challenges,we proposed an intuitive eigenvector selection method based on the coincidence degree of data distribution(CDES).First,the clustering result of improved k-means,which can well reflect the spatial distribution of various types was used as the reference map.Then,the adjusted Rand index and adjusted mutual information were calculated to assess the data distribution consistency between each eigenvector and the reference map.Finally,the eigenvectors with high coincidence degrees were selected for clustering.A case study on hyperspectral mineral mapping demonstrated that the mapping accuracies of CDES are approximately 56.3%,15.5%,and 10.5%higher than those of the commonly used top,high entropy,and high relevance eigenvectors,and CDES can save more than 99%of the eigenvector selection time.Especially,due to the unsupervised nature of k-means,CDES provides a novel solution for autonomous feature selection of hyperspectral images.

A Robust Framework for Multimodal Sentiment Analysis with Noisy Labels Generated from Distributed Data Annotation

Multimodal sentiment analysis utilizes multimodal data such as text,facial expressions and voice to detect people’s attitudes.With the advent of distributed data collection and annotation,we can easily obtain and share such multimodal data.However,due to professional discrepancies among annotators and lax quality control,noisy labels might be introduced.Recent research suggests that deep neural networks(DNNs)will overfit noisy labels,leading to the poor performance of the DNNs.To address this challenging problem,we present a Multimodal Robust Meta Learning framework(MRML)for multimodal sentiment analysis to resist noisy labels and correlate distinct modalities simultaneously.Specifically,we propose a two-layer fusion net to deeply fuse different modalities and improve the quality of the multimodal data features for label correction and network training.Besides,a multiple meta-learner(label corrector)strategy is proposed to enhance the label correction approach and prevent models from overfitting to noisy labels.We conducted experiments on three popular multimodal datasets to verify the superiority of ourmethod by comparing it with four baselines.

A parallel matching algorithm based on order relation for HLA data distribution management

In distribution simulation based on High-level architecture(HLA),data distribution management(DDM)is one of HLA services for the purpose of filtering the unnecessary data transferring over the network.DDM admits the sending federates and the receiving federates to express their interest using update regions and subscription regions in a multidimensional routing space.There are several matching algorithms to obtain overlap information between the update regions and subscription regions.When the number of regions increase sharply,the matching process is time consuming.However,the existing algorithms is hard to be parallelized to take advantage of the computing capabilities of multi-core processors.To reduce the computational overhead of region matching,we propose a parallel algorithm based on order relation to accelerate the matching process.The new matching algorithm adopts divide-and-conquer approach to divide the regions into multiple region bound sublists,each of which comprises parts of region bounds.To calculate the intersection inside and amongst the region bound sublists,two matching rules are presented.This approach has good performance since it performs region matching on the sublists parallel and does not require unnecessary comparisons within regions in different sublists.Theoretical analysis has been carried out for the proposed algorithm and experimental result shows that the proposed algorithm has better performance than major existing DDM matching algorithms.

Supporting Flexible Data Distributionin Software DSMs

Page-based software DSM systems suffer from false sharing caused by the large sharing granularity, and only support one-dimension Block or Cyclicblock data distribution schemes. Thus applications running on them will suffer from poor data locality and will be able to exploit parallelism only when using a large number of processors. In this paper, a way towards supporting flexible data distribution (FDD) on software DSM system is presented. Small granularity-tunable blocks, the size of which can be set by compiler or programmer, are used to overlap the working data sets distributed among processors. The FDD was implemented on a software DSM system called JIAJIA. Compared with Block/Cyclic-block distribution schemes used by most DSM systems now, experiments show that the proposed way of flexible data distribution is more effective. The performance of the applications used in the experiments is significantly improved.

Co-seismic fault geometry and slip distribution of the 26 December 2004, giant Sumatra–Andaman earthquake constrained by GPS, coral reef, and remote sensing data

We analyze co-seismic displacement field of the 26 December 2004, giant Sumatra–Andaman earthquake derived from Global Position System observations,geological vertical measurement of coral head, and pivot line observed through remote sensing. Using the co-seismic displacement field and AK135 spherical layered Earth model, we invert co-seismic slip distribution along the seismic fault. We also search the best fault geometry model to fit the observed data. Assuming that the dip angle linearly increases in downward direction, the postfit residual variation of the inversed geometry model with dip angles linearly changing along fault strike are plotted. The geometry model with local minimum misfits is the one with dip angle linearly increasing along strike from 4.3oin top southernmost patch to 4.5oin top northernmost path and dip angle linearly increased. By using the fault shape and geodetic co-seismic data, we estimate the slip distribution on the curved fault. Our result shows that the earthquake ruptured ＊200-km width down to a depth of about 60 km.0.5–12.5 m of thrust slip is resolved with the largest slip centered around the central section of the rupture zone78N–108N in latitude. The estimated seismic moment is8.2 9 1022 N m, which is larger than estimation from the centroid moment magnitude（4.0 9 1022 N m）, and smaller than estimation from normal-mode oscillation data modeling（1.0 9 1023 N m）.

Comparative Study and Spatial-Temporal Distribution of Regolith and Rock Geochemical Data from Xingmeng-North China

1 Introduction Geochemical mapping at national and continental scales continues to present challenges worldwide due to variations in geologic and geotectonic units.Use of the proper sampling media can provide rich information on

An Adaptive Privacy Preserving Framework for Distributed Association Rule Mining in Healthcare Databases

It is crucial,while using healthcare data,to assess the advantages of data privacy against the possible drawbacks.Data from several sources must be combined for use in many data mining applications.The medical practitioner may use the results of association rule mining performed on this aggregated data to better personalize patient care and implement preventive measures.Historically,numerous heuristics(e.g.,greedy search)and metaheuristics-based techniques(e.g.,evolutionary algorithm)have been created for the positive association rule in privacy preserving data mining(PPDM).When it comes to connecting seemingly unrelated diseases and drugs,negative association rules may be more informative than their positive counterparts.It is well-known that during negative association rules mining,a large number of uninteresting rules are formed,making this a difficult problem to tackle.In this research,we offer an adaptive method for negative association rule mining in vertically partitioned healthcare datasets that respects users’privacy.The applied approach dynamically determines the transactions to be interrupted for information hiding,as opposed to predefining them.This study introduces a novel method for addressing the problem of negative association rules in healthcare data mining,one that is based on the Tabu-genetic optimization paradigm.Tabu search is advantageous since it removes a huge number of unnecessary rules and item sets.Experiments using benchmark healthcare datasets prove that the discussed scheme outperforms state-of-the-art solutions in terms of decreasing side effects and data distortions,as measured by the indicator of hiding failure.

考虑特性分类批处理负荷可调节能力的数据中心微网灵活性设备分布鲁棒容量配置方法

微网灵活性设备主要用于平抑源荷两侧的波动,其容量配置方法应考虑源荷不确定性的影响,而含数据中心微网的灵活性设备容量配置方法还应进一步考虑数据中心负荷的可调节特性。考虑数据中心批处理负荷的可调节能力和源荷不确定性因素,提出一种灵活性设备容量配置方法。根据负荷特性的不同,将批处理负荷划分为2类以更加准确地量化其可调节能力,一类为带宽时序可变限时可平移负荷,另一类为带宽时序不变可中断平移负荷,对这2类批处理负荷进行详细分析并给出一般性的建模方法;构建数据驱动下的min-max-min两阶段分布鲁棒优化容量配置模型,利用1-范数和∞-范数约束场景概率分布模糊集,采用列和约束生成算法对该模型进行化简求解。对某省数据中心微网进行算例分析,验证了所提方法的有效性。

计及最恶劣场景概率和供需灵活性的综合能源系统分布鲁棒低碳优化调度

随着可再生能源渗透率的提升,其不确定性给综合能源系统(integrated energy system,IES)的经济性和鲁棒性带来了极大挑战。为了促进可再生能源消纳以及降低碳排放量,提出了一种基于数据驱动的分布鲁棒优化(distributionally robust optimization,DRO)调度策略。首先,构建了由有机朗肯循环(organic Rankine cycle,ORC)、氢燃料电池和电动汽车等构成的供需灵活响应模型,并引入阶梯碳交易机制来约束系统碳排放量。其次,为了能获取最恶劣情况下的场景概率分布,采用综合范数对风电输出场景的概率分布置信集合进行约束。然后,以在最恶劣场景概率分布下综合能源系统运行总成本最低为目标建立两阶段鲁棒优化模型,并通过列和约束生成(column and constraint generation,CCG)算法对模型进行迭代求解。最后,算例仿真结果表明了所提模型和求解方法的有效性,并分析了阶梯碳交易机制和供需灵活响应模型对提高系统灵活性和低碳经济性的影响。

A Survey and Experimental Review on Data Distribution Strategies for Parallel Spatial Clustering Algorithms

The advent of Big Data has led to the rapid growth in the usage of parallel clustering algorithms that work over distributed computing frameworks such as MPI,MapReduce,and Spark.An important step for any parallel clustering algorithm is the distribution of data amongst the cluster nodes.This step governs the methodology and performance of the entire algorithm.Researchers typically use random,or a spatial/geometric distribution strategy like kd-tree based partitioning and grid-based partitioning,as per the requirements of the algorithm.However,these strategies are generic and are not tailor-made for any specific parallel clustering algorithm.In this paper,we give a very comprehensive literature survey of MPI-based parallel clustering algorithms with special reference to the specific data distribution strategies they employ.We also propose three new data distribution strategies namely Parameterized Dimensional Split for parallel density-based clustering algorithms like DBSCAN and OPTICS,Cell-Based Dimensional Split for dGridSLINK,which is a grid-based hierarchical clustering algorithm that exhibits efficiency for disjoint spatial distribution,and Projection-Based Split,which is a generic distribution strategy.All of these preserve spatial locality,achieve disjoint partitioning,and ensure good data load balancing.The experimental analysis shows the benefits of using the proposed data distribution strategies for algorithms they are designed for,based on which we give appropriate recommendations for their usage.

基于分布鲁棒模型预测控制的微电网多时间尺度优化调度

源荷多元不确定性给“源荷储”一体化微电网优化调度带来了诸多挑战,但传统方案存在优化模型过于片面极端和时间尺度单一导致调度不合理的问题,无法兼顾可靠性和经济性,并且难以协调不确定性分析方法与不同时间尺度之间的配合关系。文中基于数据驱动的多离散场景分布鲁棒方法,提出一种微电网两阶段分布鲁棒日前优化调度模型,使用列和约束生成算法进行求解。结合改进分布鲁棒优化的不确定方法、多时间尺度调度策略和模型预测控制理论,通过逐级细化调度时间尺度和减小预测周期的长度来提高精度,以最小化发电成本以及调节成本等为目标建立日前-日内多时间尺度滚动优化调度模型,较大程度上抵抗系统不确定性因素的影响。结合算例仿真分析,进一步说明了所提模型能够有效消纳新能源、降低运行成本同时兼顾安全性和经济性。

Remote Control for the HT-7 Distributed Data Acquisition System

HT-7 is the first superconducting tokamak device for fusion research in China. Many experiments have been done in the machine since 1994, and lots of satisfactory results have been achieved in the fusion research field on HT-7 tokamak [1]. With the development of fusion research, remote control of experiment becomes more and more important to improve experimental efficiency and expand research results. This paper will describe a RCS (Remote Control System), the combined model of Browser/Server and Client/Server, based on Internet of HT-7 distributed data acquisition system (HT7DAS). By means of RCS, authorized users all over the world can control and configure HT7DAS remotely. The RCS is designed to improve the flexibility, opening, reliability and efficiency of HT7DAS. In the paper, the whole process of design along with implementation of the system and some key items are discussed in detail. The System has been successfully operated during HT-7 experiment in 2002 campaign period.

Statecharts for Distributed Product Data Management System Modelling

Product data management (PDM) has been accepted as an important tool for the manufacturing industries. In recent years, more and mor e researches have been conducted in the development of PDM. Their research area s include system design, integration of object-oriented technology, data distri bution, collaborative and distributed manufacturing working environment, secur ity, and web-based integration. However, there are limitations on their rese arches. In particular, they cannot cater for PDM in distributed manufacturing e nvironment. This is especially true in South China, where many Hong Kong (HK) ma nufacturers have moved their production plants to different locations in Pearl R iver Delta for cost reduction. However, they retain their main offices in HK. Development of PDM system is inherently complex. Product related data cover prod uct name, product part number (product identification), drawings, material speci fications, dimension requirement, quality specification, test result, log size, production schedules, product data version and date of release, special tooling (e.g. jig and fixture), mould design, project engineering in charge, cost spread sheets, while process data includes engineering release, engineering change info rmation management, and other workflow related to the process information. Accor ding to Cornelissen et al., the contemporary PDM system should contains manageme nt functions in structure, retrieval, release, change, and workflow. In system design, development and implementation, a formal specification is nece ssary. However, there is no formal representation model for PDM system. Theref ore a graphical representation model is constructed to express the various scena rios of interactions between users and the PDM system. Statechart is then used to model the operations of PDM system, Fig.1. Statechart model bridges the curr ent gap between requirements, scenarios, and the initial design specifications o f PDM system. After properly analyzing the PDM system, a new distributed PDM (DPDM) system is proposed. Both graphical representation and statechart models are constructed f or the new DPDM system, Fig.2. New product data of DPDM and new system function s are then investigated to support product information flow in the new distribut ed environment. It is found that statecharts allow formal representations to capture the informa tion and control flows of both PDM and DPDM. In particular, statechart offers a dditional expressive power, when compared to conventional state transition diagr am, in terms of hierarchy, concurrency, history, and timing for DPDM behavioral modeling.

A Data Mining Algorithm Based on Distributed Decision-Tree in Grid Computing Environments

Recently, researches on distributed data mining by making use of grid are in trend. This paper introduces a data mining algorithm by means of distributed decision-tree,which has taken the advantage of conveniences and services supplied by the computing platform-grid,and can perform a data mining of distributed classification on grid.

A Two-Phase Paradigm for Joint Entity-Relation Extraction

An exhaustive study has been conducted to investigate span-based models for the joint entity and relation extraction task.However,these models sample a large number of negative entities and negative relations during the model training,which are essential but result in grossly imbalanced data distributions and in turn cause suboptimal model performance.In order to address the above issues,we propose a two-phase paradigm for the span-based joint entity and relation extraction,which involves classifying the entities and relations in the first phase,and predicting the types of these entities and relations in the second phase.The two-phase paradigm enables our model to significantly reduce the data distribution gap,including the gap between negative entities and other entities,aswell as the gap between negative relations and other relations.In addition,we make the first attempt at combining entity type and entity distance as global features,which has proven effective,especially for the relation extraction.Experimental results on several datasets demonstrate that the span-based joint extraction model augmented with the two-phase paradigm and the global features consistently outperforms previous state-ofthe-art span-based models for the joint extraction task,establishing a new standard benchmark.Qualitative and quantitative analyses further validate the effectiveness the proposed paradigm and the global features.

新能源电力系统灵活性供需量化及分布鲁棒优化调度

电网中新能源渗透率的提升导致电力系统在局部时段灵活性严重不足。针对现有处理电力系统灵活性和供需不确定性过于保守或冒险的问题,提出一种数据驱动的分布鲁棒优化调度模型。首先,考虑风光出力的时空相关性,基于Copula理论构建出力集合。结合场景法与区间法对电力系统灵活性需求进行量化,引入灵活调节因子表征各类资源参与灵活性调节的能力,建立灵活性供需平衡约束。其次,考虑电动汽车等需求侧资源的灵活性供给能力,以灵活性资源运行成本与电网灵活性缺额惩罚成本最优作为目标函数,建立数据驱动的两阶段分布鲁棒模型。为降低保守性,采用综合范数对其概率分布进行约束,减小了灵活性需求出现极端情况的概率。针对两阶段鲁棒模型求解问题,利用零和博弈思想将模型解耦为主问题和子问题,采用列与约束生成算法进行迭代求解。最后,仿真算例表明,所提模型相比于传统不确定性模型对提高电力系统灵活性裕度与经济性具有积极作用。

基于数据驱动分布鲁棒优化的梯级水光蓄联合优化调度

多种可再生能源互补联合发电技术因其独特优越性正在成为“双碳”背景下电力系统优选供电方案之一,而其不确定性复杂耦合特性下的互补联合调度问题越来越受到人们关注。针对不确定性优化调度问题,本文引入能较好平衡不确定性及鲁棒性的数据驱动分布鲁棒优化理论(data-driven DRO),提出了一种新的基于数据驱动DRO梯级水光蓄联合发电系统协同优化调度方法。首先,考虑系统互补经济调度成本建立两阶段调度模型,制定各电站日前出力调度计划;然后,引入综合范数约束限定概率置信区间,并考虑最恶劣分布下的实时运行调整成本,获取日前调度计划的最优调整方案,日调度计划和调度调整方案形成最优调度计划;最后,本方法采用MP-SP框架,引入CCG算法展开两阶段协同求解。为验证所提方法的性能,引入四川示范区实际运行数据,开展了有效性验证、性能对比分析、计算效率仿真验证等。结果表明:本调度方法的有效性在数据规模、置信度水平两个维度得到了验证;对于SO、RO及本方法鲁棒性及经济性等性能指标的对比,本方法可获得高于SO的鲁棒性及高于RO的经济性;将本调度方法与概率性时序生产模拟方法的计算耗时进行对比,该方法实现了相同计算精度的较高计算效率。基于两阶段调度模型及循环迭代求解的DRO梯级水光蓄联合优化调度方法实现了协同调度结果经济性与保守性的均衡,其高效性能得到验证,为多种可再生能源互补协同调度提供了新思路。

基于联邦学习的网络化ICU呼吸机和镇静剂管理方法

医疗物联网设备的激增和丰富的医疗数据为智慧医疗提供了新的可能。重症监护室(ICU)的病人依靠众多医疗边缘设备来持续监测管理患者的健康状况。在ICU常见的治疗干预措施中,有创机械通气和镇静剂的注射多用于维持患者的呼吸功能,提高治疗质量,而现有的治疗干预措施很大程度上依赖于医生的判断。文中提出了一种基于联邦学习的临床辅助决策方法——MFed,可以基于网络化ICU分布式协作学习最佳干预政策。该方法应用基于差分隐私的联邦学习方法,打破了医疗数据隐私方面的限制以及医疗数据孤岛的窘境;用分布鲁棒优化确保最坏情况下的性能并结合伪孪生网络实现自适应地滤除噪声数据。最后,在现实ICU数据集上的实验表明,与其他最先进的基线相比,所提方法的准确率提高了36.75%。

基于最优决策树的多能系统快速鲁棒优化调度

新能源渗透率的持续增长造成了多能系统快速协调调度的巨大挑战,包括调度结果过于保守以及日内调度低效等问题。为此,提出了一种基于最优决策树分布式鲁棒优化的多能系统协调快速调度方法,所构建模型考虑电网日内经济调度,引入基于范数约束的概率分布置信集精准描述新能源的不确定性,防止调度结果过于保守。同时,根据新能源日内运行数据,分别通过可解释的最优分类树和最优回归树算法,优化日内机组启停状态和出力水平的初始决策量,解决日内鲁棒调度的低效问题。在四川某地区电网的验证结果表明,该模型可在兼顾调度成本和鲁棒性的同时,实现水风光多能系统的日内快速协调调度。

全球气象资料客观分析系统的分布式并行化

It is difficult to parallelize a subsistent sequential algorithm. Through analyzing the sequential algorithm of a Global Atmospheric Data Objective Analysis System, this article puts forward a distributed parallel algorithm that statically distributes data on a massively parallel processing (MPP) computer. The algorithm realizes distributed parailelization by extracting the analysis boxes and model grid point Iatitude rows with leaped steps, and by distributing the data to different processors. The parallel algorithm achieves good load balancing, high parallel efficiency, and low parallel cost. Performance experiments on a MPP computer arc also presented.