A Heterogeneous Information Fusion Deep Reinforcement Learning for Intelligent Frequency Selection of HF Communication

The high-frequency(HF) communication is one of essential communication methods for military and emergency application. However, the selection of communication frequency channel is always a difficult problem as the crowded spectrum, the time-varying channels, and the malicious intelligent jamming. The existing frequency hopping, automatic link establishment and some new anti-jamming technologies can not completely solve the above problems. In this article, we adopt deep reinforcement learning to solve this intractable challenge. First, the combination of the spectrum state and the channel gain state is defined as the complex environmental state, and the Markov characteristic of defined state is analyzed and proved. Then, considering that the spectrum state and channel gain state are heterogeneous information, a new deep Q network(DQN) framework is designed, which contains multiple sub-networks to process different kinds of information. Finally, aiming to improve the learning speed and efficiency, the optimization targets of corresponding sub-networks are reasonably designed, and a heterogeneous information fusion deep reinforcement learning(HIF-DRL) algorithm is designed for the specific frequency selection. Simulation results show that the proposed algorithm performs well in channel prediction, jamming avoidance and frequency channel selection.

Accurate time synchronization of power reference station based on BD3 system

A Beidou 3(BD3)system-based power reference station can provide high-precision time synchronization for power distribution systems by sending synchronization data packets to devices in a multi-hop routing fashion.However,optimizing route selection to reduce both time synchronization error and delay is a challenging problem.In this paper,we establish a software-defined network-enabled power reference station time synchronization framework based on BD3.Then,we formulate the joint problem to minimize cumulative synchronization error and delay through multi-hop route selection optimization.A back propagation(BP)neural network-improved intelligent time synchronization route selection algorithm named BP-RS is proposed to learn the optimal route selection,which uses a BP neural network to dynamically adjust the exploration factor to achieve rapid convergence.Simulation results show the superior performance of BP-RS in synchronization delay,synchronization error,and adaptability with changing routing topologies.