Knowledge transfer in multi-agent reinforcement learning with incremental number of agents

In this paper, the reinforcement learning method for cooperative multi-agent systems(MAS) with incremental number of agents is studied. The existing multi-agent reinforcement learning approaches deal with the MAS with a specific number of agents, and can learn well-performed policies. However, if there is an increasing number of agents, the previously learned in may not perform well in the current scenario. The new agents need to learn from scratch to find optimal policies with others,which may slow down the learning speed of the whole team. To solve that problem, in this paper, we propose a new algorithm to take full advantage of the historical knowledge which was learned before, and transfer it from the previous agents to the new agents. Since the previous agents have been trained well in the source environment, they are treated as teacher agents in the target environment. Correspondingly, the new agents are called student agents. To enable the student agents to learn from the teacher agents, we first modify the input nodes of the networks for teacher agents to adapt to the current environment. Then, the teacher agents take the observations of the student agents as input, and output the advised actions and values as supervising information. Finally, the student agents combine the reward from the environment and the supervising information from the teacher agents, and learn the optimal policies with modified loss functions. By taking full advantage of the knowledge of teacher agents, the search space for the student agents will be reduced significantly, which can accelerate the learning speed of the holistic system. The proposed algorithm is verified in some multi-agent simulation environments, and its efficiency has been demonstrated by the experiment results.

STUDIES ON TBP-NO_x——A NEW OXIDIZING AGENT PREPARED FROM NITROGEN OXIDES

Nitrogen oxides(NO_2 and NO)are absorbed by tributyl phosphorate(TBP)to fom a new complex mixture of TBP-NO_x. which is used as a selective oddizing agent to oxidize benzylalcohols to corresponding sldehydes or ketones In high yield. In the reaction process, nitrogen oxides are llberated mildly and mainly reduced to nitrogen, while tributyl phosphorate is recovered end recycled.

Covalent Attachment of DNA to Glass Supports Using A New Silane Coupling Agent and Chemiluminescent Detection

A new kind of silane coupling agent, N- (β-aminoethyl ) - γ-aminopropyl triet hoxysilane, was used for DNA direct attachment on the surfaces of glass supports, then the immobilized DNA was hybridized with horseradish peroxidase (HRP)-labeled probe, and detected by using enhanced chemiluminescent method. In comparison with γ-aminopropyl triethoxysilane, the detection limits (S/N) of DNA were 10 pg and 75 pg respectively. Several experimental conditions of DNA attachING to glass supports were investigated, and the system of hybridization of nucleic acid on the surfaces of glass supports was developed.

A NEW BIFUNCTIONAL CHELATING AGENT α,ε-N,N'-BIS(L-CYSTEINYL)-L-LYSINE FOR RADIOLABELLING OF MONOCLONAL ANTIBODIES WITH TECHNETIUM-99M

α,ε-N,N'-bis(L-cysteinyl)-L-lysine was synthesized and char- acterized for the first time.It was then employed as a bifunctional chelating agent to chelate technetium-99m and subsequently conjugated to fragment F(ab')_2 of anti-gastric tumor monoclonal antibody 3G9.The radiolabelled antibody was satisfactorily stable and immunoreactive.

THE SYNTHESIS OF ZSM-35 WITH A NEW KIND OF TEMPLATING AGENT

The present paper covers the synthesis of ZSM-35 molecular sieve with 1,6-hexanediamine as templating agent,the influence of the synthesis conditions, and a discussion concerning the crystal structures of the products.

Application of ultrasound contrast in identification and diagnosis of ocular spaceoccupying lesions

AIM: To analyze the application significance of ultrasound contrast agent in identification and diagnosis of ocular spaceoccupying lesions, and mainly analyze its advantages and problems. METHODS: Thirty-two representative literatures about the application of ultrasound contrast agent in diagnosis of spaceoccupying lesions at home and abroad were collected after focused on sorting the literature reporting the application of ultrasound contrast diagnostic technology in the diagnosis and identification of ocular spaceoccupying lesions in recent years. Its advantages and problems were retrospectively analyzed, and reasonable assessment on Existing problems was made and possible solutions to the problems were proposed. RESULTS: As a new imaging diagnostic technique, the contrast-enhanced ultrasound, which can enhance the display of tumor microcirculation vessels and improve the tumor's ultrasound diagnostic capability, was analyzed. Through sorting and comprehensively analyzing the collected literatures, the positive rate of ocular spaceoccupying lesion diagnosis could be significantly improved with ultrasound contrast technology. Thus, the vascular perfusion in normal tissues and lesions was reflected objectively. According to the lesion's perfusion characteristics of the contrast agent plus with the performance features of two-dimensional ultrasound, the ocular spaceoccupying lesions can be accurately diagnosed, and this could provide clinicians with reliable research basis in this field. CONCLUSION: Ultrasound contrast examination is a new testing method, and ultrasound contrast agent can significantly enhance the ultrasonic detection signal, clearly show the blood perfusion in vessels and tissues, increase the image contrast resolution, and improve the lesion's detection capability in the microcirculation perfusion level, especially its important value in the diagnosis of ocular tumor.

Study on Plugging Technology in Oil Test

In view of the shortcomings of poor temperature resistance, poor pumpability and poor pressure-bearing capacity of commonly used gel plugging materials, polyacrylamide (HPAM) and N,N-methylene bisacrylamide (BIS) were selected for crosslinking and compounding to make a new type of gel plugging material with high temperature and pressure resistance. The compressive strength and yield stress were measured by inverted observation method to evaluate the gel strength. The anti-temperature, anti-pressure, anti-dilution and gel-breaking properties of the gel were evaluated. Finally, 71 type high temperature and high pressure water loss instrument and high temperature and high pressure filter with slit plate were used to evaluate the plugging capacity of gel plugging agent. The experimental results show that the new gel plugging system between 100°C - 120°C, gelation time can be controlled at about 5 h;it has strong temperature resistance, compression resistance, dilution resistance and gel breaking performance. In the face of permeability and fracture leakage simulation experiments, when the ambient temperature below 120°C, pressure within 5 MPa, the filter loss of gel plugging agent is far less than the market two conventional plugging agent, has excellent plugging performance.

Resolution of racemic R.S-α-arylethylamine by a new resolving agent (R)thiazolidine-2-thione-4-carboxylic acid

Racemic R.S-α-arylethylamine was resolved by R (-) thiazolidine-2-thione-4-carboxylic acid, a new resolving agent abbreviated as [R (-) TTCA], by which R (-) TTCA.S(-) arylethylamine salts2a-2e, [α] D 20 =-47.24° — 64.40° and optically active R(+)-a-arylethylamines3a – 3e, 74. 54%-94. 45% e, e., were obtained. Optically active S (-) -α-arylethylamines4a-4e, 72.84%-90.36% e.e., were obtained by the decomposition of2a-2e in basic solutions. The influence of substitutive group of the benzene ring on the basicity of the amino group was studied by semiempirical PM3 method. The structures of the R (-) TTCA.S (-) -α-phenylethylamine salt (2a(R-S) configuration) and R (-) TTCA-R(+)-a-phenylethylamine salt (2a(R-R) configuration) have been established by means of X-ray diffraction. They crystallize in a monoclinic system. Space group isP21. The cell constants of2a(R-S) configuration were obtained as follows: α = 1.387 8(2), b = 0.664 05(101,c = 1.580 O(2) nm; β = 90.844(10)° Z = 4; those obtained for2a(R-R) configuration were α = 1.080 6(2),b = 0.584 80(12),c = 1.2188(2) nm, β= 110.38(3)dg, V = 0.7220nm3,Z = 2. There are intermolecular hydrogen bonds in the crystals of the two kinds of configurations of the amine salt. The hydrogen bond number in the unit cell of R (-) TTCA.S (-)-α-phenylethylamine salt is twice as much as that of R (-) TTCA.R(+)-a-phenylethylamine salt.