Multi-UAV cooperative maneuver decision-making for pursuitevasion using improved MADRL

Aiming at the problem of multi-UAV pursuit-evasion confrontation, a UAV cooperative maneuver method based on an improved multi-agent deep reinforcement learning(MADRL) is proposed. In this method, an improved Comm Net network based on a communication mechanism is introduced into a deep reinforcement learning algorithm to solve the multi-agent problem. A layer of gated recurrent unit(GRU) is added to the actor-network structure to remember historical environmental states. Subsequently,another GRU is designed as a communication channel in the Comm Net core network layer to refine communication information between UAVs. Finally, the simulation results of the algorithm in two sets of scenarios are given, and the results show that the method has good effectiveness and applicability.

Optimal decision threshold for soft decision cooperative spectrum sensing

In order to achieve higher spectrum efficiency in cognitive radio （CR） systems, a closed-form expression of the optimal decision threshold for soft decision cooperative spectrum sensing based on the minimum total error probability criterion is derived. With the analytical expression of the optimal decision threshold, the impact of different sensing parameters on the threshold value is studied. Theoretical analyses show that the optimal threshold achieves an efficient trade-off between the missed detection probability and the false alarm probability. Simulation results illustrate that the average signal-to-noise ratio （SNR） and the soft combination schemes have a great influence on the optimal threshold value, whereas the number of samples has a weak impact on the optimal threshold value. Furthermore, for the maximal ratio combing （MRC） and the modified deflection coefficient （MDC） schemes, the optimal decision threshold value increases and approaches a corresponding individual limit value while the number of CR users increases. But the number of CR users has a weak influence on the optimal decision threshold for the equal gain combining （EGC） scheme.

Cooperative maneuver decision making for multi-UAV air combat based on incomplete information dynamic game

Cooperative autonomous air combat of multiple unmanned aerial vehicles(UAVs)is one of the main combat modes in future air warfare,which becomes even more complicated with highly changeable situation and uncertain information of the opponents.As such,this paper presents a cooperative decision-making method based on incomplete information dynamic game to generate maneuver strategies for multiple UAVs in air combat.Firstly,a cooperative situation assessment model is presented to measure the overall combat situation.Secondly,an incomplete information dynamic game model is proposed to model the dynamic process of air combat,and a dynamic Bayesian network is designed to infer the tactical intention of the opponent.Then a reinforcement learning framework based on multiagent deep deterministic policy gradient is established to obtain the perfect Bayes-Nash equilibrium solution of the air combat game model.Finally,a series of simulations are conducted to verify the effectiveness of the proposed method,and the simulation results show effective synergies and cooperative tactics.

Cooperative Spectrum Sensing for Cognitive Radio-Wireless Sensors Network Based on OR Rule Decision to Enhance Energy Consumption in Greenhouses

Wireless sensors networks (WSNs) combined with cognitive radio have developed and solved the limited space of the frequency spectrum. In this paper, we propose different types of spectrums sensing and their own decisions depend on the probabilities that applied into fusion center, and how these probabilities’ techniques help to enhance the energy consumption of WSNs. In the same way, the importance of designing balanced distribution between the wireless sensors networks and their own sinks. This research also provides an overview of security issues in CR-WSN, especially in Spectrum Sensing Data Falsification (SSDF) attacks that enforces harmful effects on spectrum sensing and spectrum sharing. We adopt OR rule as four types of CRSN sensing protocolin greenhouses application by using Matlab and Netsim simulators. Our results show that the designing balanced wireless sensors and their sinks in greenhouses are very significant to decrease the energy, which is due to the traffic congestion in the sink range area. Furthermore, by applying OR rule has enhanced the energy consumption, and improved the sensors network lifetime compared to cognitive radio network.

A Cooperation-planning Model Based on Bilevel Programming Decision

This paper is based on a resource constrained active network project;the constraint of the local resource and the time constraint of the cooperation resource are considered simultaneously.And the respective benefit of the manager and cooperation partners is also considered simultaneously.And a cooperation planning model based on bilevel multi-objective programming is de- signed,according to the due time and total cost.And an extended CNP based on the permitted range for resource and time requests is presented.A larger task set in scheduling cycle is on the permitting for the request of cooperation resource and time while the task manager itself may be permitted biding for tasks.As a result,the optimization space for the cooperation planning is enlarged.So not every bidding task is successfully bid by invitee,and the task manager itself takes on some bidding tasks.Finally,the genetic algorithm is given and the validity and feasibility of the model is proved by a case.

Farmers using insurance and cooperatives to manage agricultural risks: A case study of the swine industry in China

Purchasing agricultural insurance and joining agricultural cooperatives are two prevalent instruments used by farmers in China for dealing with agricultural risks. Data from 443 swine farmers in Jiangsu and Henan provinces of China were collected. Factors affecting the farmers’ decision to purchase agricultural insurance and join agricultural cooperatives were assessed. The possibility of simultaneous use of both instruments and the potential correlation between these two decisions was considered as well. Results showed that the farmers’ decision to use agricultural insurance and cooperatives was positively correlated, indicating that farmers who purchased agricultural insurance which mainly used to mitigate production risks were more likely to join agricultural cooperatives which more used to share market risks, and vice versa. Farmers’ knowledge of swine insurance and trust in the local government positively impacted the purchase of agricultural insurance, while education, years involved in swine production and scale of swine production positively impacted farmers joining agricultural cooperatives.

On intelligent Cooperative System

The paper presents our research efforts motivated by the apparent need to combine conventional,preexisting computing functions with novel knowledge--based functions. This has been likened to what occurred in the evolution of primates, where the 'new brain' (the cortex) was added to, layered upon, and given control over the 'old brain' common to the less complex animals.

Markov decision evolutionary game theoretic learning for cooperative sensing of unmanned aerial vehicles

As one of the major contributions of biology to competitive decision making, evolutionary game theory provides a useful tool for studying the evolution of cooperation. To achieve the optimal solution for unmanned aerial vehicles （UAVs） that are car- rying out a sensing task, this paper presents a Markov decision evolutionary game （MDEG） based learning algorithm. Each in- dividual in the algorithm follows a Markov decision strategy to maximize its payoff against the well known Tit-for-Tat strate- gy. Simulation results demonstrate that the MDEG theory based approach effectively improves the collective payoff of the roam. The proposed algorithm can not only obtain the best action sequence but also a sub-optimal Markov policy that is inde- pendent of the game duration. Furthermore, the paper also studies the emergence of cooperation in the evolution of self-regarded UAVs. The results show that it is the adaptive ability of the MDEG based approach as well as the perfect balance between revenge and forgiveness of the Tit-for-Tat strategy that the emergence of cooperation should be attributed to.

HEURISTIC QUANTUM GENETIC ALGORITHM FOR AIR COMBAT DECISION MAKING ON COOPERATIVE MULTIPLE TARGET ATTACK

In order to achieve the optimal attack outcome in the air combat under the beyond visual range(BVR)condition,the decision-making(DM)problem which is to set a proper assignment for the friendly fighters on the hostile fighters is the most crucial task for cooperative multiple target attack(CMTA).In this paper,a heuristic quantum genetic algorithm(HQGA)is proposed to solve the DM problem.The originality of our work can be supported in the following aspects:(1)the HQGA assigns all hostile fighters to every missile rather than fighters so that the HQGA can encode chromosomes with quantum bits(Q-bits);(2)the relative successful sequence probability(RSSP)is defined,based on which the priority attack vector is constructed;(3)the HQGA can heuristically modify quantum chromosomes according to modification technique proposed in this paper;(4)last but not the least,in some special conditions,the HQGA gets rid of the constraint described by other algorithms that to obtain a better result.In the end of this paper,two examples are illustrated to show that the HQGA has its own advantage over other algorithms when dealing with the DM problem in the context of CMTA.

考虑居民用户可调节潜力的负荷聚合商日前投标决策优化模型研究

负荷聚合商能够充分挖掘居民用户资源参与需求响应,深化居民用电可调节潜力量化分析对负荷精准调控与负荷聚合商模式智能推广具有重要作用。首先,基于居民用电负荷调控差异性,考虑居民典型用电电器综合满意度计量,构建居民柔性负荷可调节潜力分析模型;然后,考虑负荷聚合商参与市场交易的运营模式,构建基于非合作博弈理论的日前投标决策优化模型;最后,通过算例仿真验证模型的有效性及适用性。结果表明该模型能够兼顾调控潜力,有效支撑负荷聚合商最优投标决策,为负荷聚合商推广下统筹资源优化调控提供借鉴与参考。

供应受限下考虑采购环节的订货-价格竞合决策模型

在上游供应受限的背景下,为保障供应链的有效运转,在品牌商采用“控制”采购模式的代工供应链中,引入一个存在产能限制及规模经济效应的供应商,构建制造商-品牌商的订货-价格决策模型(策略1)。以经典竞合模型为基础,建立制造商开通直销渠道的博弈模型(策略2)。通过对博弈模型进行分析,得到了供应结构中的均衡结果。结合解析和数值分析,说明了制造商接受极限高价采购(零部件)和搭便车采购两种情境下,竞合博弈的效果以及制造商的策略偏好。与以往模型不同,由于考虑了制造商和品牌商的零部件采购环节,从而形成了环形供应结构,并发现了以渠道交换订单的效应。研究结论可以为供应受限条件下制造商的转型升级提供建议。

世界卫生组织数字健康治理决策证据研究

目的对世界卫生组织发布的数字健康文件开展科学计量分析,为提升我国数字健康治理能力和治理水平,扩大我国在全球健康决策方面的影响力提供参考。方法识别世界卫生组织发布的数字健康文件引文类型,并开展科学计量,包括统计发文机构、国别、年份并分析引文主题等。结果世界卫生组织在数字健康领域发布多个战略和指导性文件,参考文献为电子文献、期刊文献和少量专著,电子文献以国际机构发布的各类文件为主。期刊论文来自多个医学相关高水平期刊,文献具有较强时效性,研究主题包括随机对照试验、队列研究、卫生教育、行为干预、吸烟、基础设施建设等。美国、英国和瑞士机构的研究成果获得最多引用。结论应合理采纳不同类型机构研究成果,科学优化我国数字健康在重点疾病领域的规划措施,积极参与全球数字健康治理,鼓励我国有能力的社会组织、高校等机构世界卫生组织和其他国际组织研究项目并扩大成果影响力,加强国际合作。

自我-群体决策对象和分配方式影响合作公平评价的ERP研究

采用事件相关技术,探讨在合作任务中,决策对象和分配方式如何影响个体公平评价的时间进程。使用2(决策对象:自我、群体)×2(分配方式:按贡献分配、平均分配)被试内设计。结果发现自我决策比为群体决策诱发更大的P2和更小的FRN;自我决策时,按贡献分配诱发的P3显著小于平均分配诱发的P3;为群体决策时,两种方式诱发的P3无显著差异。结果表明决策对象在早期阶段被优先加工与考虑,而晚期阶段的公平评价受到决策对象和分配方式的联合调节。

复杂产品企业主导产学研深度融合策略博弈

基于创新行为决策学说,提出研发服务协议发挥效力时的温和型融合策略,基于“奖罚兼顾”的协同管理理论,对产学研道德风险存在下的4种深度融合创新均衡进行多维度估测研判。温和解或聚点行为对深度融合系统获益及公正性都极为不利,相比负向的三方非合作模式及保障举措,若支付可以被无任何附加条件地流转,则跨机构深度融合创新系统的总收益最优。从议价行为设计、平台化运作方面提出了复杂产品系统研制企业主导下三方深度融合创新系统强化策略。

基于合作博弈的网联车辆合流序列优化策略

为提升道路合流区交通通行效率,提出了一种基于合作博弈的网联车辆合流序列优化决策算法。首先,构建包含油耗、舒适度的多优化目标联合成本函数,通过分析多人合作博弈问题推导出合流区全局最小成本策略;然后,将多车合作博弈分解为多个两车合作博弈,预测不同策略的最小合流成本,利用成本矩阵法求得两车博弈的帕累托有效解,确定最优合流序列;最后,在三车道合并至一车道合流场景下,MATLAB仿真结果表明,上述优化合流序列能有效降低油耗,缩短出行时间,提高乘客的舒适性。

合作社介入对农产品供应链合作的影响研究

针对农产品供应链的农产品采购问题,本文基于“公司+合作社+农民”模式构建了“讨价还价”模型,通过设计和对比统一采购合同和差异化采购合同两种合同方式,重点分析了合作社介入模式及相关因素对农产品供应链的影响。研究发现:(1)合作社对农民种植信息(包括农产品类别和农民种植能力等信息)的整合对农产品供应链中农产品的采购方式具有重要影响,即整合信息决定了合作社提供的信息类别,进而促使公司和合作社达成不同的采购合同;(2)社会型企业只关注整体农民的福利,而合作社负责协调整体福利在农民间的分配;(3)为了满足消费者个性化需求,应由合作社根据农民的差异规划和组织不同农产品的种植,进而增加各方的效用。最后,研究结论为“公司+合作社+农民”模式的进一步发展提供理论支持和管理启示。

基于深度确定性策略梯度与注意力Critic的多智能体协同清障算法

动态障碍物一直是阻碍智能体自主导航发展的关键因素,而躲避障碍物和清理障碍物是两种解决动态障碍物问题的有效方法。近年来,多智能体躲避动态障碍物(避障)问题受到了广大学者的关注,优秀的多智能体避障算法纷纷涌现。然而,多智能体清理动态障碍物(清障)问题却无人问津,相对应的多智能体清障算法更是屈指可数。为解决多智能体清障问题,文中提出了一种基于深度确定性策略梯度与注意力Critic的多智能体协同清障算法(Multi-Agent Cooperative Algorithm for Obstacle Clearance Based on Deep Deterministic Policy Gradient and Attention Critic, MACOC)。首先,创建了首个多智能体协同清障的环境模型,定义了多智能体及动态障碍物的运动学模型,并根据智能体和动态障碍物数量的不同,构建了4种仿真实验环境;其次,将多智能体协同清障过程定义为马尔可夫决策过程(Markov Decision Process, MDP),构建了多智能体t的状态空间、动作空间和奖励函数;最后,提出一种基于深度确定性策略梯度与注意力Critic的多智能体协同清障算法,并在多智能体协同清障仿真环境中与经典的多智能体强化学习算法进行对比。实验证明,相比对比算法,所提出的MACOC算法清障的成功率更高、速度更快,对复杂环境的适应性更好。

基于数字孪生和强化学习的低空智联网协同认知干扰

针对低空智联网协同认知干扰决策过程中,多架电子干扰无人机对抗多部多功能雷达的干扰资源分配问题,提出了一种基于数字孪生和深度强化学习的认知干扰决策方法。首先,将协同电子干扰问题建模为马尔可夫决策问题,建立认知干扰决策系统模型,综合考虑干扰对象、干扰功率和干扰样式选择约束,构建智能体动作空间、状态空间和奖励函数。其次,在近端策略优化(Proximal policy optimization,PPO)深度强化学习算法的基础上,提出了自适应学习率近端策略优化(Adaptive learning rate proximal policy optimization,APPO)算法。同时,为了以高保真的方式提高深度强化学习算法的训练速度,提出了一种基于数字孪生的协同电子干扰决策模型训练方法。仿真结果表明,与已有的深度强化学习算法相比,APPO算法干扰效能提升30%以上,所提训练方法能够提高50%以上的模型训练速度。

基于深度确定性梯度学习的集群多目标分配方法

针对多弹协同作战进行目标分配时,存在敌方平台和反舰导弹数量不确定性和类型多样化,导致目标分配算法难以建模的问题,为提升高动态协同攻击条件下的攻击效能,建立动态战场环境模型和多目标分配的单回合马尔可夫决策模型,提出一种改进深度确定性策略梯度的分配算法.通过与模拟器的交互自动求解最佳分配策略,利用mask方法对动作空间进行掩码操作,实现算法对平台数量和类型的适应能力.实验结果表明,在各种不同舰船的防御配置和红蓝双方数量配置下,算法求解得到的攻击策略相对于随机策略的性能提升约为87.5%,模型推理时间约为0.04ms.研究结果将加速基于深度确定性梯度学习的方法在高动态环境下智能决策中的应用,对集群自主决策方法的研究具有推动作用.

人力资本、激励与有限合伙制

本研究基于私募基金中GP和LP的博弈视角,在合作博弈框架下,通过决策权归属对公司制和有限合伙制进行界定,引入GP能力参数,探讨了有限合伙制的优势和适用性问题,以及人力资本对企业组织形式和企业价值的影响机制.研究发现由于决策权由LP转移到GP,有限合伙制能够充分发挥人力资本作用,实现更高的人力资本激励和企业价值,但是决策权的转移会丧失LP在收益分配中的优势.只有当GP能力水平达到一定程度,使得有限合伙制下联盟总体价值的上升足以弥补收益占比下降的负面作用时,有限合伙制下GP和LP收益均会上升,有限合伙制占优于公司制,实现帕累托改进,从而解释了私募基金等人力资本密集型行业采用有限合伙制的原因.为更好地发挥私募基金支持实体经济作用,培养专业领域的高水平人才是私募基金健康可持续发展的关键,有助于形成人力资本提升和产业转型升级良性循环,提高人力资本在收益分配中的相对优势.