Network Defense Decision-Making Based on Deep Reinforcement Learning and Dynamic Game Theory

Existing researches on cyber attackdefense analysis have typically adopted stochastic game theory to model the problem for solutions,but the assumption of complete rationality is used in modeling,ignoring the information opacity in practical attack and defense scenarios,and the model and method lack accuracy.To such problem,we investigate network defense policy methods under finite rationality constraints and propose network defense policy selection algorithm based on deep reinforcement learning.Based on graph theoretical methods,we transform the decision-making problem into a path optimization problem,and use a compression method based on service node to map the network state.On this basis,we improve the A3C algorithm and design the DefenseA3C defense policy selection algorithm with online learning capability.The experimental results show that the model and method proposed in this paper can stably converge to a better network state after training,which is faster and more stable than the original A3C algorithm.Compared with the existing typical approaches,Defense-A3C is verified its advancement.

Integrated Heat and Power Dispatch Model for Wind-CHP System with Solid Heat Storage Device Based on Robust Stochastic Theory

This paper built a combined heat and power（CHP） dispatch model for wind-CHP system with solid heat storage device（SHS） aiming at minimizing system coal consumption, and set system demand-supply balance and units＇ operation conditions as the operation constraints. Furthermore, robust stochastic optimization theory was used to describe wind power output uncertainty. The simulation result showed that SHS increased CHP peak-valley shifting capability and reduced abandoned wind rate from 12% to 6%, and reduced 5% coal consumption, compared with the original system operation by flexible charging electric power and heating. The payback period of employing SHS in wind-CHP system is far shorter than SHS expected service life.

A Stochastic Optimal Control Theory to Model Spontaneous Breathing

Respiratory variables, including tidal volume and respiratory rate, display significant variability. The probability density function (PDF) of respiratory variables has been shown to contain clinical information and can predict the risk for exacerbation in asthma. However, it is uncertain why this PDF plays a major role in predicting the dynamic conditions of the respiratory system. This paper introduces a stochastic optimal control model for noisy spontaneous breathing, and obtains a Shrödinger’s wave equation as the motion equation that can produce a PDF as a solution. Based on the lobules-bronchial tree model of the lung system, the tidal volume variable was expressed by a polar coordinate, by use of which the Shrödinger’s wave equation of inter-breath intervals (IBIs) was obtained. Through the wave equation of IBIs, the respiratory rhythm generator was characterized by the potential function including the PDF and the parameter concerning the topographical distribution of regional pulmonary ventilations. The stochastic model in this study was assumed to have a common variance parameter in the state variables, which would originate from the variability in metabolic energy at the cell level. As a conclusion, the PDF of IBIs would become a marker of neuroplasticity in the respiratory rhythm generator through Shr?dinger’s wave equation for IBIs.

Research on problem about technology insurance pricing based on backward stochastic differential equation theory

The development of Backward Stochastic Differential Equation Theory is just a thing happened in the past years. Although its development and application is far behind Forward Stochastic Differential Equation, its wide application prospect on financial mathematics gets more and more attention. The meaning of Backward Stochastic Differential Equation is that change a already-known final （usually uncertain） goal into a present certain answer to make a present resolution. But Insurance Pricing happens to know the final result, it＇ s certain that the result is uncertain, that is to say, to get out of danger or not. And then make present insurance price according to the future uncertain result. The Insurance Pricing just follows the meaning of Backward Stochastic Differential Equation. Insurance Pricing itself is also a research field sprang up in past scores of years, because insurance pricing is the indisputable core of insurance work, and gets quite a few researchers＇ attention. This article adopts backward stochastic differential equation theory and do research on problem about technology insurance pricing.

Prediction of subsurface settlement induced by shield tunnelling in sandy cobble stratum

This study focuses on the analytical prediction of subsurface settlement induced by shield tunnelling in sandy cobble stratum considering the volumetric deformation modes of the soil above the tunnel crown.A series of numerical analyses is performed to examine the effects of cover depth ratio(C/D),tunnel volume loss rate(h t)and volumetric block proportion(VBP)on the characteristics of subsurface settle-ment trough and soil volume loss.Considering the ground loss variation with depth,three modes are deduced from the volumetric deformation responses of the soil above the tunnel crown.Then,analytical solutions to predict subsurface settlement for each mode are presented using stochastic medium theory.The influences of C/D,h t and VBP on the key parameters(i.e.B and N)in the analytical expressions are discussed to determine the fitting formulae of B and N.Finally,the proposed analytical solutions are validated by the comparisons with the results of model test and numerical simulation.Results show that the fitting formulae provide a convenient and reliable way to evaluate the key parameters.Besides,the analytical solutions are reasonable and available in predicting the subsurface settlement induced by shield tunnelling in sandy cobble stratum.

Extracting hidden weak sinusoidal signal with short duration from noisy data: Analytical theory and computational realization

Signal detection is both a fundamental topic of data science and a great challenge for practical engineering. One of the canonical tasks widely investigated is detecting a sinusoidal signal of known frequency ω with time duration T：I（t）=Acos ω t＋Γ（t）, embedded within a stationary noisy data. The most direct, and also believed to be the most efficient, method is to compute the Fourier spectral power at ω：B=｜2/T∫0^T I（t）e^iωtdt｜. Whether one can out-perform the linear Fourier approach by any other nonlinear processing has attracted great interests but so far without a consensus. Neither a rigorous analytic theory has been offered. We revisit the problem of weak signal, strong noise, and finite data length T=O（1）, and propose a signal detection method based on resonant filtering. While we show that the linear approach of resonant filters yield a same signal detection efficiency in the limit of T→∞, for finite time length T=O（1）, our method can improve the signal detection due to the highly nonlinear interactions between various characteristics of a resonant filter in finite time with respect to transient evolution. At the optimal match between the input I（t）, the control parameters, and the initial preparation of the filter state, its performance exceeds the above threshold B considerably. Our results are based on a rigorous analysis of Gaussian processes and the conclusions are supported by numerical computations.

Predictions of ground surface settlement for shield tunnels in sandy cobble stratum based on stochastic medium theory and empirical formulas

This paper focuses on the prediction of ground surface settlement induced by shield tunnelling in sandy cobble stratum.Based on the stochastic medium theory,an analytical solution to predict the surface settlement is developed considering the difference between soil and tunnel volume loss.Then,the effects of tunnel geometries,influence angle and volume loss on the characteristics of surface settlement are discussed.Through back analysis,a total of 103 groups of field monitoring data of surface settlement induced by shield tunnelling in sandy cobble stratum are examined to investigate the statistical characteristics of the maximum settlement,settlement trough width and volume loss.An empirical prediction is presented based on the results of back analysis.Finally,the analytical solution and empirical expression are validated by the comparisons with the results of model tests and field monitoring.Results show that the soil at ground surface has an overall dilative response for most of the shield tunnelling in sandy cobble stratum.In addition,the developed analytical solution is applicable and reasonable for surface settlement prediction.Meanwhile,the proposed empirical formula also shows good per-formance in some cases,providing an approach or a reference for engineering designers to preliminarily evaluate the surface settlement.

Trajectory and Correction Maneuver During the Transfer from Earth to Halo Orbit

This article addresses the design of the trajectory transferring from Earth to Halo orbit, and proposes a timing closed-loop strategy of correction maneuver during the transfer in the frame of circular restricted three body problem （CR3BP）. The relation between the Floquet multipliers and the magnitudes of Halo orbit is established, so that the suitable magnitude for the aerospace mission is chosen in terms of the stability of Halo orbit. The stable manifold is investigated from the Poincar6 mapping defined which is different from the previous researches, and six types of single-impulse transfer trajectories are attained from the geometry of the invariant manifolds. Based on one of the trajectories of indirect transfer which are ignored in the most of literatures, the stochastic control theory for imperfect information of the discrete linear stochastic system is applied to design the trajectory correction maneuver. The statistical dispersion analysis is performed by Monte-Carlo simulation,

Unintentional forking analysis in wireless blockchain networks

The forking problem plays a key role in the security issue,which is a major concern in the blockchain system.Although many works studied the attack strategy,consensus mechanism,privacy-protecting and security performance analysis,most of them only address the intentional forking caused by a malicious attacker.In fact,without any attacker,unintentional forking still remains due to transmission delay and failure,especially in wireless network scenarios.To this end,this paper investigates the reason for generating unintentional forking and derives the forking probability expression in Wireless Blockchain Networks(WBN).Furthermore,in order to illustrate the unintentional forking on the blockchain system,the performances in terms of resource utilization rate,block generation time,and Transaction Per Second(TPS)are investigated.The numerical results show that the target difficulty of hash algorithm in generating a new block,the delay time of broadcasting,the network scale,and the transmission failure probability would affect the unintentional forking probability significantly,which can provide a reliable basis for avoiding forking to save resource consumption and improving system performance.

Rotational Brownian Motion on Sphere Surface and Rotational Relaxation

The spatial components of the autocorrelation function of noninteracting dipoles are analytically obtained in terms of rotational Brownian motion on the surface of a unit sphere using multi-level jumping formalism based on Debye＇s rotational relaxation model, and the rotational relaxation functions are evaluated.

Mechanism of grout bulb expansion and its effect on ground uplifting

In order to study the uplifting effect of compensation grouting on ground surface and the upper structures,the ground heave induced by stratum expansion was considered as a stochastic process and the stochastic medium theory was applied to determine the heave and deformation of ground surface under uniform and non-uniform expansion models of spherical grout bulb.The corresponding calculating formulas and simplified methods were derived based on the hypotheses of radial expansion.Then,a numerical model,in which radial velocity was imposed on the outer nodes of grout bulb to simulate the expansion process reaching a required volume strain,was established simultaneously.This new method avoids repeated trial calculation needed in the traditional method which applies a "fictitious" expanding pressure in the grouting elements.The results show that the numerical solutions have good consistency with the theoretical ones.Meanwhile,though the heave resulting from non-uniform expansion is larger than that from uniform expansion for shallow grouting,both of them tend to be convergent with the increasing of grouting depth.

Prediction of ground surface displacement caused by grouting

Ground surface displacement caused by grouting was calculated with stochastic medium theory. Ground surface displacement was assumed to be caused by the cavity expansion of grouting, slurry seepage, and slurry contraction. A prediction method of ground surface displacement was developed. The reliability of the presented method was validated through a comparison between theoretical results and results from engineering practice. Results show that the present method is effective. The effect of parameters on uplift displacement was illustrated under different grouting conditions. Through analysis, it can be known that the ground surface uplift is mainly caused by osmosis of slurry and the primary influence angle of stratum β determines the influence range of surface uplift. Besides, the results show that ground surface uplift displacement decreases notably with increasing depth of the grouting cavity but it increases with increasing diffusion radius of grout and increasing grouting pressure.

Synergy and Redundancy in a Signaling Cascade with Different Feedback Mechanisms

Feedback plays an important role in various biological signal transmission systems. In this paper, a signaling cascade system(including three layers: input(S), intermediate(V), output(X) components) is employed to study the fluctuations and net synergy in information transmission, in which the V component is regulated by itself or the X component, and each feedback on V is either positive or negative. The Fano factor, the net synergy, and the signalto-noise ratio(SNR) of signaling cascade with the four possible feedback types are theoretically derived by using linear noise approximation of the master equation, and the ability of information transmission through the signaling cascade is characterized by using the partial information decomposition of information theory. It is found that the signaling cascade exhibits different responses to the four feedback mechanisms, which depend on the relationships between degradation rates of components. Our results not only clarify the dependence of the Fano factor, net synergy, and SNR on the feedback regulations with the varying of degradation rates of components, but also imply that living cells could utilize different feedback mechanisms to adapt to the external fluctuating environments.