Multi-Agent Deep Reinforcement Learning for Cross-Layer Scheduling in Mobile Ad-Hoc Networks

Due to the fading characteristics of wireless channels and the burstiness of data traffic,how to deal with congestion in Ad-hoc networks with effective algorithms is still open and challenging.In this paper,we focus on enabling congestion control to minimize network transmission delays through flexible power control.To effectively solve the congestion problem,we propose a distributed cross-layer scheduling algorithm,which is empowered by graph-based multi-agent deep reinforcement learning.The transmit power is adaptively adjusted in real-time by our algorithm based only on local information(i.e.,channel state information and queue length)and local communication(i.e.,information exchanged with neighbors).Moreover,the training complexity of the algorithm is low due to the regional cooperation based on the graph attention network.In the evaluation,we show that our algorithm can reduce the transmission delay of data flow under severe signal interference and drastically changing channel states,and demonstrate the adaptability and stability in different topologies.The method is general and can be extended to various types of topologies.

Incorporation of Perception-based Information in Robot Learning Using Fuzzy Reinforcement Learning Agents

Robot learning in unstructured environments has been proved to be an extremely challenging problem, mainly because of many uncertainties always present in the real world. Human beings, on the other hand, seem to cope very well with uncertain and unpredictable environments, often relying on perception-based information. Furthermore, humans beings can also utilize perceptions to guide their learning on those parts of the perception-action space that are actually relevant to the task. Therefore, we conduct a research aimed at improving robot learning through the incorporation of both perception-based and measurement-based information. For this reason, a fuzzy reinforcement learning (FRL) agent is proposed in this paper. Based on a neural-fuzzy architecture, different kinds of information can be incorporated into the FRL agent to initialise its action network, critic network and evaluation feedback module so as to accelerate its learning. By making use of the global optimisation capability of GAs (genetic algorithms), a GA-based FRL (GAFRL) agent is presented to solve the local minima problem in traditional actor-critic reinforcement learning. On the other hand, with the prediction capability of the critic network, GAs can perform a more effective global search. Different GAFRL agents are constructed and verified by using the simulation model of a physical biped robot. The simulation analysis shows that the biped learning rate for dynamic balance can be improved by incorporating perception-based information on biped balancing and walking evaluation. The biped robot can find its application in ocean exploration, detection or sea rescue activity, as well as military maritime activity.

Removal of Dye Using Lignin-Based Biochar/Poly(ester amide urethane)Nanocomposites from Contaminated Wastewater

The pursuit of incorporating eco-friendly reinforcing agents in polymer composites has accentuated the exploration of various natural biomass-derived materials.The burgeoning environmental crisis spurred by the discharge of synthetic dyes into wastewater has catalyzed the search for effective and sustainable treatment technologies.Among the various sorbent materials explored,biochar,being renewable,has gained prominence due to its excellent adsorption properties and environmental sustainability.It has also emerged as a focal point for its potential to replace other conventional reinforcing agents,viz.,fumed silica,aluminum oxide,treated clays,etc.This study introduces a novel class of polymer nanocomposites comprising of lignin-based biochar particles and poly(ester amide urethane)matrix via a feasible method.The structural evaluation of these nanocomposites was accomplished using Fourier-transform infrared spectroscopy,X-ray photoelectron spectroscopy,and powder X-ray diffraction.The polymer nanocomposites exhibited superior mechanical properties with an increment in tensile strength factor by 45%in comparison to its pristine matrix,along with an excellent toughness value of 90.22 MJm^(−3)at a low loading amount of only 1 wt%.The composites showed excellent improvement in thermal properties with a sharp rise in the glass transition temperature(Tg)value from−28.15℃to 84℃,while also championing sustainability through inherent biodegradability attributes.Beyond their structural prowess,these polymer nanocomposites demonstrated excellent potential as adsorbents,displaying efficient removal of malachite green and tartrazine dyes from aqueous systems with a removal efficiency of 87.25%and 73.98%,respectively.The kinetics study revealed the pseudo second order model to be the precision tool to assess the dye removal study.Complementing this,the Langmuir adsorption isotherm provided a framework to assess the sorption features of the polymer nanocomposites.Overall,these renewable biochar integrated polymer matrices boast remarkable recovery capabilities up to seven cycles of usage with an excellent dye recovery percentage of 95.21%for the last cycle,thereby defining sustainability as well as economic feasibility.

Investigation of nano-talc as a filling material and a reinforcing agent in high density polyethylene (HDPE)

An experiment of producing high density polyethylene (HDPE) nano-composite filled with 4wt.% talc was presented. Acting as filler and a reinforcing agent in the HDPE, talc powder, sized at around 5 μm, was surface-treated with aluminum diethylene glycol dinitrate coupling agent before adding to the HDPE. Analyses of the reinforced HDPE nano-composite show significant improvement in its mechanical properties including, tensile strength (>26 MPa), break elongation (<1.1%), flexural strength (>22 MPa), and friction coefficients<0.11. The results demonstrate that, after surface-treated, talc can be used as a promising filling material and a reinforcing agent in making HDPE nano-composite.

Synergistic action on hypnosia: Yinao capsules with pentobarbital sodium of threshold and sub-threshold dosages

BACKGROUND： Sedative and hypnotic drugs could cure insomnia in a dependent manner, and traditional Chinese medicine has some superiority in treating insomnia. OBJECTIVE： To observe the synergistic action of yinao capsules with pentobarbital sodium in threshold and sub-threshold dosages in hypnosia and sedation. DESIGN： A completely randomized grouping design and control experiment. SETTING： Pharmacological laboratory, College of Pharmacy, Henan College of Traditional Chinese Medicine. MATERIALS： Totally 200 grade II Kunming mice of 18 - 21 g, either male or female, were used. Yinao capsules, main ingredients of which were turtleback glue, thinleaf milkwort root, Chinese magnoliavine fruit, mythic fungus, tangshen, ginseng and grassleaf sweetflag rhizome, were offered by Chinese-American Huayi Pharmacy, Co.,Ltd. （ratified number： 040901）; Kangnaoshuai capsules, main ingredients of which were prepared rehmannia root, tuber fleeceflower root, ginseng, membranous milkvetch root, thinleaf milkwort root, Fushen, grassleaf sweetflag rhizome, spine date seed, lecithin, barbary wolfberry fruit, pueraria root, vitamin E, etc., were produced by Shijiazhuang Siyao, Co.,Ltd. （ratified number： 040964）; Pentobarbital sodium was produced by China Medicine （Group） Shanghai Chemical Reagent, Co,.Ltd. （Ratified number： 030816）. ZZ-6 mice spontaneous activity apparatus was produced by Chengdu Taimeng Science and Technology, Co.,Ltd. METHODS： The experiment was carried out in the Animal Experimental Center, Henan College of Traditional Chinese Medicine from October to December in 2005. （1） Influence of Yinao capsules on the spontaneous activity of mice： Fifty mice were randomly divided into five groups with 10 mice in each group： Mice in the large, middle and small dosages of Yinao capsules groups were intragastrically infused with Yinao capsules suspension （1.36, 0.68, 0.34 g/kg）; Those in the Kangnaoshuai capsules group were infused with Kangnaoshuai suspension （1.12 g/kg）; Those in the control group were given physiologic saline of the same volume. The mice were administrated once a day for 7 days continuously, and they were placed into the mice spontaneous activity apparatus after 60 minutes from the last administration, the times of spontaneous activities and the times of arising within 10 minutes were recorded after adaptation for 5 minutes. （2） Synergistic action in hypnosia by Finao capsules with pentobarbital sodium of threshold dosage： Seventy mice were randomly divided into 5 groups as above-mentioned with 14 mice in each group, and they were treated the same as above. They were intraperitoneally injected with 50 mg/kg pentobarbital sodium after 60 minutes from the last administration, then conditions of falling asleep were observed. The disappearance of righting reflex was taken as the index of falling asleep, the latency of falling asleep was the duration from intraperitoneal injection of pentobarbital sodium to fall asleep, the sleeping time was from falling asleep to the disappearance of righting reflex. （3） Synergistic action in hypnosia by Yinao capsules with pentobarbital sodium of sub-threshold dosage： Eighty mice were randomly divided into 5 groups as above-mentioned with 16 mice in each group, and they were treated the same as above. They were intraperitoneally injected with 27 mg/kg pentobarbital sodium after 60 minutes from the last administration. Mice whose righting reflex disappeared for at least 1 minute within 30 minutes were taken as falling asleep, the number of sleeping animals in each group was recorded, and the rate of falling asleep was calculated. MAIN OUTCOME MEASURES： Synergistic action of Yinao capsules with pentobarbital sodium of threshold and sub-threshold dosages in hypnosia and sedation. RESULTS： All the 200 mice were involved in the analysis of results. （1） Influence of Yinao capsules on the spontaneous activity of mice： The times of spontaneous activities within 10 minutes in the large and middle dosage of Yinao capsules groups and Kangnaoshuai capsules group [（138.0±37.0）, （156.8±28.3）, （133.3±46.1） times] were obviously fewer than those in the control group [（204.3±61.3） times, P 〈 0.05- 0.01]. The arising times within 10 minutes in the middle and small dosages of Yinao capsules groups and Kangnaoshuai capsules group [（30.7 ± 18.3）, （26.5± 11.2）, （24.2±11.6） times] were obviously fewer than those in the control [（71.7±38.6） times, P 〈 0.01 ]. （2） Synergistic action in hypnosia by Yinao capsules with pentobarbital sodium of threshold dosage： The sleeping latencies in the large and middle dosages of Yinao capsules groups and Kangnaoshuai capsules group [（4.49±1.84）, （4.83±1.72）, （3.85± 1.94） minutes] were obviously shorter than that in the control group [（6.73 ±2.75） minutes, P 〈 0.05 - 0.01 ]. The sleeping time in the large, middle and small dosages of Yinao capsules groups and Kangnaoshuai capsules group [（89.0± 38.42）, （67.21 ±24.07）, （66.28±18.94）, （84.36±29.81） minutes] were obviously longer than that in the control group [（45.78±20.78） minutes, P 〈 0.05- 0.01]. （3） Synergistic action in hypnosia by Yinao capsules with pentobarbital sodium of sub-threshold dosage： The rates of falling asleep in the large and middle dosages of Yinao capsules groups were higher than that in the control group [56% （9/1 6）, 38% （6/1 6）, 6% （6/16）, P 〈 0.01, 0.05], whereas Kangnaoshuai capsules and small dosage of Yinao capsules had no synergistic action with pentobarbital sodium of sub-threshold dosage （P 〉 0.05）. CONCLUSION： Yinao capsules have synergistic action with pentobarbital sodium in hypnosia. Yinao capsules possess obvious sedative and hypnotic effects in a dosage-dependent manner.

Study and application on fire extinguishing and preventing with light-paste material for coalmine

For preventing and extinguishing the spontaneous combustion of coal seam, based on the importance for sealing air leakage channels in roadway, developed a material of light-paste for coalmine, and research its ingredient, proportion and performance. The result indicates that the component ratio of the material is which PB exciting agent is 20%, complex reinforcing agent is 2%, complex expansion agent is 1%, and A-material is 77%. The performances of compressive strength, swelling ratio and expansion time are optimum. The light-paste material can effectively seal air leakage channels for preventing and extinguishing the spontaneous combustion of coal seam in coalmine.

Research on active defense decision-making method for cloud boundary networks based on reinforcement learning of intelligent agent

The cloud boundary network environment is characterized by a passive defense strategy,discrete defense actions,and delayed defense feedback in the face of network attacks,ignoring the influence of the external environment on defense decisions,thus resulting in poor defense effectiveness.Therefore,this paper proposes a cloud boundary network active defense model and decision method based on the reinforcement learning of intelligent agent,designs the network structure of the intelligent agent attack and defense game,and depicts the attack and defense game process of cloud boundary network;constructs the observation space and action space of reinforcement learning of intelligent agent in the non-complete information environment,and portrays the interaction process between intelligent agent and environment;establishes the reward mechanism based on the attack and defense gain,and encourage intelligent agents to learn more effective defense strategies.the designed active defense decision intelligent agent based on deep reinforcement learning can solve the problems of border dynamics,interaction lag,and control dispersion in the defense decision process of cloud boundary networks,and improve the autonomy and continuity of defense decisions.

Inhibitory potential of three Yin-tonification herbal formulas on the activities of human major cytochrome P450 and UDP-glucuronosyltransferases isozymes in vitro

OBJECTIVE:To investigate the influence of Yin-tonification herbal formulas Jaeumganghwa-tang(Ziyin Jianghuo Tang,JEGHT),Ssanghwa-tang(Shuanghe Tang,SHT) and Yukmijihwang-tang(Liuwei Di huang Tang,YMJHT) on the activities of human major cytochrome P450(CYP450 s) and UDP-glucuronosyltransferases isozymes(UGTs) in vitro.METHODS:The activities of CYP450 s(CYP1 A2,CYP3 A4,CYP2 B6,CYP2 C9,CYP2 C19,CYP2 D6 and CYP2 E1) and UGTs(UGT1 A1,UGT1 A4 and UGT2 B7)were assessed using in vitro fluorescence-and luminescence-based enzyme assays,respectively.The effects of herbal formulas on the activities of CYP450 s and UGTs were presented as IC50 values.RESULTS:JEGHT showed the potent inhibition of the CYP2 D6 activity,with weak inhibition on the activities of CYP1 A2,CYP2 B6,CYP2 C9,CYP2 C19,CYP2 E1,CYP3 A4,UGT1 A1,UGT1 A4 and UGT2 B7.SHT inhibited the activities of CYP1 A2 and CYP2 E1,whereas the negligible inhibition of the activities of CYP2 B6,CYP2 C9,CYP2 C19,CYP2 D6,CYP3 A4,UGT1 A1,UGT1 A4 and UGT2 B7 through SHT was observed.YMJHT inhibited CYP2 E1 activity,with a negligible inhibition on the activities of CYP1 A2,CYP2 B6,CYP2 C9,CYP2 C19,CYP2 D6,CYP3 A4,UGT1 A1,UGT1 A4 and UGT2 B7.CONCLUSION:These findings provide information about the potential interactions between three Yin-tonification herbal formulas and conventional drugs.