MADDPG-D2: An Intelligent Dynamic Task Allocation Algorithm Based on Multi-Agent Architecture Driven by Prior Knowledge

Aiming at the problems of low solution accuracy and high decision pressure when facing large-scale dynamic task allocation(DTA)and high-dimensional decision space with single agent,this paper combines the deep reinforce-ment learning(DRL)theory and an improved Multi-Agent Deep Deterministic Policy Gradient(MADDPG-D2)algorithm with a dual experience replay pool and a dual noise based on multi-agent architecture is proposed to improve the efficiency of DTA.The algorithm is based on the traditional Multi-Agent Deep Deterministic Policy Gradient(MADDPG)algorithm,and considers the introduction of a double noise mechanism to increase the action exploration space in the early stage of the algorithm,and the introduction of a double experience pool to improve the data utilization rate;at the same time,in order to accelerate the training speed and efficiency of the agents,and to solve the cold-start problem of the training,the a priori knowledge technology is applied to the training of the algorithm.Finally,the MADDPG-D2 algorithm is compared and analyzed based on the digital battlefield of ground and air confrontation.The experimental results show that the agents trained by the MADDPG-D2 algorithm have higher win rates and average rewards,can utilize the resources more reasonably,and better solve the problem of the traditional single agent algorithms facing the difficulty of solving the problem in the high-dimensional decision space.The MADDPG-D2 algorithm based on multi-agent architecture proposed in this paper has certain superiority and rationality in DTA.

An Intelligent Algorithm for Solving Weapon-Target Assignment Problem:DDPG-DNPE Algorithm

Aiming at the problems of traditional dynamic weapon-target assignment algorithms in command decisionmaking,such as large computational amount,slow solution speed,and low calculation accuracy,combined with deep reinforcement learning theory,an improved Deep Deterministic Policy Gradient algorithm with dual noise and prioritized experience replay is proposed,which uses a double noise mechanism to expand the search range of the action,and introduces a priority experience playback mechanism to effectively achieve data utilization.Finally,the algorithm is simulated and validated on the ground-to-air countermeasures digital battlefield.The results of the experiment show that,under the framework of the deep neural network for intelligent weapon-target assignment proposed in this paper,compared to the traditional RELU algorithm,the agent trained with reinforcement learning algorithms,such asDeepDeterministic Policy Gradient algorithm,Asynchronous Advantage Actor-Critic algorithm,Deep Q Network algorithm performs better.It shows that the use of deep reinforcement learning algorithms to solve the weapon-target assignment problem in the field of air defense operations is scientific.In contrast to other reinforcement learning algorithms,the agent trained by the improved Deep Deterministic Policy Gradient algorithm has a higher win rate and reward in confrontation,and the use of weapon resources is more efficient.It shows that the model and algorithm have certain superiority and rationality.The results of this paper provide new ideas for solving the problemof weapon-target assignment in air defense combat command decisions.

Based on network pharmacology to explore the mechanism of hepatotoxicity of Fructus Meliae Toosendan

Objective:To explore the potential mechanism of hepatotoxicity induced by Fructus Meliae Toosendan(FMT)through network pharmacology.Methods:The active components and targets of FMT were identified and screened by Traditional Chinese Medicine Systems Pharmacology Database and Analysis Platform Database,PubChem Database and Swiss Target Prediction database,etc.Genecards,pharmGKB,and OMIM databases were used to collect relevant targets of hepatotoxicity,and intersect them with the targets of active ingredients to obtain the potential targets of hepatotoxicity caused by FMT.A compound-target network was constructed with Cytoscape 3.8.0 software.The String 11.0 database was used to construct the protein-protein interaction(PPI)network of the targets and to screen out the core targets.In addition,Gene Ontology(GO)terms and Kyoto Encyclopedia of Genes and Genomes(KEGG)pathway enrichment analyses were conducted by R software,and then the pathways directly related to hepatotoxicity were integrated.Results:In this study,9 active ingredients of FMT and 265 targets were obtained.There are 533 hepatotoxicity-related targets,and 76 potential targets for hepatotoxicity caused by FMT,among which quercetin,melianone,and nimbolin A are the key active components for hepatotoxicity caused by FMT,and MYC,STAT3,JUN,and RELA were the core target proteins of FMT’s hepatotoxicity.There were 2353 GO entries(P<0.05),including 2181 Biological Process(BP),41 Cellular Component(CC)and 131 Molecular Function(MF).KEGG enrichment analysis revealed 165 pathways(P<0.05),of which Th17 cell differentiation,HIF-1 signaling pathway,PI3K-Akt signaling pathway were strongly correlated with the hepatotoxicity of FMT.Conclusion:Through network pharmacology,it was found that many potential components in azadirachia chinaberry may be involved in the regulation of apoptosis,excessive inflammatory response and mitochondrial dynamics through multi-target and multi-pathway,resulting in the generation of hepatotoxicity.

Mechanism of Wumei Pill in the Treatment of Non-Erosive reflux disease from the Perspective of Network Pharmacology and Molecular docking

Objective:Based on network pharmacology and molecular docking to explore the mechanism of Wumei Pill in the treatment of non-erosive reflux disease(NERD).Method:We collected the active ingredients and targets of Wumei Pill by Traditional Chinese Medicine Systems Pharmacology Database and Analysis Platform(TCMSP),and collected NERD related targets through Genecards,PharmGKB,Drugbank,DisGeNET,OMIM,CTD and TTD databases.Intersection targets of Wumei Pill targets and NERD related targets were the potential targets of Wumei Pill in the treatment of NERD.We imported the intersection targets into the STRING database to obtain the PPI network,and obtained the hub targets.The network diagram of"Drugs-Potential active ingredients-Potential targets"was constructed by Cytoscape 3.7.2 software.We used R software to perform Gene Ontology function enrichment analysis(GO)and Kyoto Encyclopedia of Genes and Genomes pathway enrichment analysis(KEGG)on hub targets,and then performed molecular docking verification.Results:There were 129 active ingredients and 213 drug targets of Wumei Pill of which 114 were the intersection targets.1587 GO enrichment items were identified(P<0.05),including 1,491 biological processes,11 cell components,and 85 molecular functions.143 KEGG pathways(P<0.05),mainly related to Kaposi sarcoma-associated herpesvirus infection,IL-17 signaling pathway,the TNF signaling pathway,MAPK signaling pathway.Results of molecular docking showed that the potential active ingredients in Wumei Pill had relatively stable binding activity to the key targets.Conclusion:Wumei pill for the treatment of non-erosive reflux disease are main active ingredients quercetin,kaempferol,beta sitosterol,Isocorypalmine,Stigmasterol,rutaecarpine,etc,the main targets is JUN,TP53,AKT1,may inhibit excessive inflammation,antioxidant therapy effect into full play.This provided a certain theoretical basis for clinical application.

Based on network pharmacology to explore the mechanism of nephrotoxicity of Xanthii Fructus

Objective:To explore the potential mechanism of nephrotoxicity induced by Xanthii Fructus(XF)through network pharmacology.Methods:The active ingredients and targets of XF were screened through Traditional Chinese Medicine Systems Pharmacology Database,PubChem Database and Swiss Target Prediction Database,and the related pathogenic genes of nephrotoxicity were downloaded from Genecards,pharmGKB and OMIM databases.The potential target is the intersection of XF and the pathogenic gene of nephrotoxicity.Cytoscape software was used to construct a network of“XF-Potential active ingredient-Target-Nephrotoxicity network”.Protein protein interaction(PPI)network was constructed by String database,and the data was analyzed and sorted out to obtain the key active components and core target proteins of XF.GO functional enrichment analysis and KEGG pathway enrichment analysis of key target proteins were performed by R software.Results:11 active components,167 targets,6839 nephrotoxicity related pathogenic genes and 139 intersection target genes were screened from XF.After constructing the network of“XF-Potential active ingredient-target-nephrotoxicity network”and PPI network,the key active ingredients of XF were identified as Moupinamide,Beta-sitosterol and Stigmasterol,and the core target proteins were CASP3,HSP90AA1 and TP53.There were 2072 GO entries(P<0.05),including 1876 Biological Process(BP),64 Cellular Component(CC)and 132 Molecular Function(MF).KEGG enrichment analysis revealed 138 pathways(P<0.05),mainly involving p53 signaling pathway,IL-17 signaling pathway,PI3K-Akt signaling pathway were strongly correlated with the nephrotoxicity of XF.Conclusion:The core target protein of XF HSP90AA1,CASP3 and TP53 participate in p53 signaling Pathway,PI3K-Akt signaling Pathway,IL-17 signaling Pathway and other pathways to regulate apoptosis and induce inflammatory responses,resulting in kidney damage.