Reinforcement Learning Based Dynamic Spectrum Access in Cognitive Internet of Vehicles

Cognitive Internet of Vehicles(CIoV)can improve spectrum utilization by accessing the spectrum licensed to primary user(PU)under the premise of not disturbing the PU’s transmissions.However,the traditional static spectrum access makes the CIoV unable to adapt to the various spectrum environments.In this paper,a reinforcement learning based dynamic spectrum access scheme is proposed to improve the transmission performance of the CIoV in the licensed spectrum,and avoid causing harmful interference to the PU.The frame structure of the CIoV is separated into sensing period and access period,whereby the CIoV can optimize the transmission parameters in the access period according to the spectrum decisions in the sensing period.Considering both detection probability and false alarm probability,a Q-learning based spectrum access algorithm is proposed for the CIoV to intelligently select the optimal channel,bandwidth and transmit power under the dynamic spectrum states and various spectrum sensing performance.The simulations have shown that compared with the traditional non-learning spectrum access algorithm,the proposed Q-learning algorithm can effectively improve the spectral efficiency and throughput of the CIoV as well as decrease the interference power to the PU.

A DYNAMIC SPECTRUM ACCESS NETWORK BASED ON COGNITIVE RADIO

Dynamic spectrum access technologies based on Cognitive Radio(CR) is under intensive research carried out by the wireless communication society and is expected to solve the problem of spectrum scarcity.However,most enabling technologies related to dynamic spectrum access are con-sidered individually.In this paper,we consider these key technologies jointly and introduce a new implementation scheme for a Dynamic Spectrum Access Network Based on Cognitive Radio(DSAN-BCR).We start with a flexible hardware platform for DSAN-BCR,as well as a flexible protocol structure that dominates the operation of DSAN-BCR.We then focus on the state of the art of key technologies such as spectrum sensing,spectrum resources management,dynamic spectrum access,and routing that are below the network layer in DSAN-BCR,as well as the development of technologies related to higher layers.Last but not the least,we analyze the challenges confronted by these men-tioned technologies in DSAN-BCR,and give the perspectives on the future development of these technologies.The DSAN-BCR introduced is expected to provide a system level guidance to alleviate the problem of spectrum scarcity.

Orthogonal Codes-Based Dynamic Spectrum Access in Cognitive Radio Networks

Dynamic spectrum access(DSA) in cognitive radio(CR) networks became a challenging research area recently. In CR technology, the DSA between primary users(PUs) and secondary users(SUs) simultaneously can be achieved without degrading the performance of the PUs by SUs interference. This can be achieved by donating incentive power to the PUs in order to compensate the interference caused by the SUs. Consequently, PUs allow SUs to share the spectrum. In this paper, orthogonal codes-based dynamic spectrum access(OC-DSA) technique is proposed. OC-DSA technique employs orthogonality between PUs and SUs transmitted data symbols in addition to the incentive power donation. Compared to other techniques, the proposed technique uses a simple encoder at the SU network for the same PU network infrastructure. By applying orthogonal codes, the interference caused by SUs is canceled and hence the donated power to incentivize the PUs is reduced. Also, the SU packet rate is increased significantly. The simulation results show that the proposed technique provides effective improvements over other existing techniques in the signal strength and the bit error rate performance of both the PU network and the SU network at the receiver side. Moreover, the proposed technique requires less donated power to incentivize the PU and has higher packet rate.

QoS Provisioning Energy Saving Dynamic Access Policy for Overlay Cognitive Radio Networks with Hidden Markov Channels

Dynamic spectrum access policy is crucial in improving the performance of over- lay cognitive radio networks. Most of the previ- ous works on spectrum sensing and dynamic spe- ctrum access consider the sensing effective- ness and spectrum utilization as the design cri- teria, while ignoring the energy related issues and QoS constraints. In this article, we propose a QoS provisioning energy saving dynamic acc- ess policy using stochastic control theory con- sidering the time-varying characteristics of wir- eless channels because of fading and mobility. The proposed scheme determines the sensing action and selects the optimal spectrum using the corresponding power setting in each decis- ion epoch according to the channel state with the objective being to minimise both the flame error rate and energy consumption. We use the Hidden Markov Model （HMM） to model a wir- eless channel, since the channel state is not dir- ectly observable at the receiver, but is instead embedded in the received signal. The proced- ure of dynamic spectrum access is formulated as a Markov decision process which can be sol- ved using linear programming and the primal- dual index heuristic algorithm, and the obta- ined policy has an index-ability property that can be easily implemented in real systems. Sim- ulation results are presented to show the per- formance improvement caused by the propo- sed approach.

MULTI-CHANNEL TRANSMISSION STRATEGY FOR DYNAMIC SPECTRUM ACCESS

Cognitive radio is a promising technology that deals with the scarcity of radio spectrum. In this paper, we propose a new multi-channel transmission strategy for dynamic spectrum access in cognitive radio network. Starting with the model of spectrum activities, we present the multi-channel transmission strategy which has primary user's protection mechanism to improve the spectrum efficiency and study its performance under perfect and imperfect sensing. Numerical example results reveal that the performance of the proposed strategy has the superiority with respect to capacity performance of cognitive users and the protection of primary users.

Dynamic Spectrum Access Protocol for the Digital Dividend

It is not more and more, easy to satisfy the important and growing spectrum demands in the context of the static conventional policy spectrum allocation. Therefore, to find a suitable solution to this problem, we are to days observing the apparition of flexible dynamic spectrum allocation methods. These methods that ought to improve more significantly the spectrum use have gained much interest. In fact, the digital dividend due to the change-over from the analog television to the digital terrestrial television must be efficiently used. So the Dynamic Spectrum Access (DSA) can potentially play a key role in shaping the future digital dividend use. In the DSA, two kinds of users or networks coexist on different channels. The first one, known as the primary user, accesses to a channel with high priority;and the second one, known as secondary user has a low priority. This paper presents a dynamic spectrum access protocol based on an auction framework. Our protocol is an interesting tool that allows the networks to bid and obtain on the available spectrum, the rights to be primary and secondary users according their valuations and traffic needs. Based on certain offers, our protocol selects primary and secondary users for each idle channel in order to realize the maximum economic for the regulator or social benefits. We deal with the case in which the offers of the networks are independent one another even if they will share the same channels. We design an algorithm in accordance with our dynamic spectrum access protocol. The algorithm is used here to find an optimal solution to the access allocation problem, specifically to digital dividend. Finally, the results in the numeric section, regarding the three suggested scenarios, show that the proposed dynamic spectrum access protocol is viable. The algorithm is able to eliminate all non-compliant bidders for the available spectrum sharing. We notice that the revenue or social benefits of the regulator is maximized when we have on each channel, one primary user and the maximum number of secondary users.

Space-Time Correlation Based Fast Regional Spectrum Sensing in Cognitive Radio

In this paper,a space-time correlation based fast regional spectrum sensing(RSS)scheme is proposed to reduce the time and energy consumption of traditional spatial spectrum sensing. The target region is divided into small meshes,and all meshes are clustered into highly related groups using the spatial correlation among them. In each group,some representative meshes are selected as detecting meshes(DMs)using a multi-center mesh(MCM)clustering algorithm,while other meshes(EMs)are estimated according to their correlations with DMs and the Markov modeled dependence on history by MAP principle. Thus,detecting fewer meshes saves the sensing consumption. Since two independent estimation processes may provide contradictory results,minimum entropy principle is adopted to merge the results. Tested with data acquired by radio environment mapping measurement conducted in the downtown Beijing,our scheme is capable to reduce the consumption of traditional sensing method with acceptable sensing performance.

The investigation of DARC etch back in DRAM capacitor oxide mask opening

Opening the silicon oxide mask of a capacitor in dynamic random access memory is a critical process on a capacitive coupled plasma(CCP)etch tool.Three steps,dielectric anti-reflective coating(DARC)etch back,silicon oxide etch and strip,are contained.To acquire good performance,such as low leakage current and high capacitance,for further fabricating capacitors,we should firstly optimize DARC etch back.We developed some experiments,focusing on etch time and chemistry,to evalu-ate the profile of a silicon oxide mask,DARC remain and critical dimension.The result shows that etch back time should be con-trolled in the range from 50 to 60 s,based on the current equipment and condition.It will make B/T ratio higher than 70%mean-while resolve the DARC remain issue.We also found that CH_(2)F_(2) flow should be~15 sccm to avoid reversed CD trend and keep in-line CD.

Opportunistic Usage of Maritime VHF Band—Deployment Challenges for a New Regulatory Framework

Similarly to other domains, maritime community requests for broadband services have been significantly increasing. Worldwide navigation footprint and the lack of practical alternatives to Satellite Communications (SATCOM) empower VHF band as the natural choice to support most of those demands. Nevertheless, the major challenge for an implementation of maritime broadband VHF services is unquestionably the spectrum availability and management. Eventually, the solution must include spectrum sharing, using a Cognitive Radio (CR) based approach, but unfortunately current regulatory framework and spectrum management regime are not appropriate for such concepts and emerging technologies. To overcome such constraints, it is necessary to address a whole field of regulatory and standardization issues in order to prepare an evolution towards a more flexible and dynamic approach to spectrum management and a transition that would ensure incumbents live operations and legacy systems. The required paradigm change encompasses a new policy definition, an enforcement mechanism implementation and a comprehensive transition plan. The presented analysis pretends to address the regulatory feasibility of a framework change, discusses its evolving process and points some challenges related with practical aspects associated to Quality of Service (QoS) enforcement metrics definition, centering the arguments in maritime VHF band.

Cooperative spectrum sharing of multiple primary users and multiple secondary users

This paper proposes a multiple-input multiple-output （MIMO） based cooperative dynamic spectrum access （DSA） framework that enables multiple primary users （PUs） and multiple secondary users （SUs） to cooperate in spectrum sharing. By exploiting MIMO in cooperative DSA, SUs can relay the primary traffic and send their own data at the same time, which greatly improves the performance of both PUs and SUs when compared to the non- MIMO time-division spectrum sharing schemes. Especially, we focus on the relay selection optimization problem among multiple PUs and multiple SUs. The network-wide cooperation and competition are formulated as a bargaining game, and an algorithm is developed to derive the optimal PU-SU relay assignment and resource allocation. Evaluation results show that both primary and secondary users achieve significant utility gains with the proposed framework, which gives all of them incentive for cooperation.

An Intelligent Fuzzy Based Energy Detection Approach for Cooperative Spectrum Sensing

Cognitive radio systems are helpful to access the unused spectrum using the popular technique, referred to as spectrum sensing. Spectrum sensing involves the detection of primary user (PU) signal using dynamic spectrum access. Cooperative spectrum sensing takes advantage of the spatial diversity in multiple cognitive radio user networks to improve the sensing accuracy. Though the cooperative spectrum sensing schemes significantly improve the sensing accuracy, it requires the noise variance and channel state information which may lead to transmission overhead. To overcome the drawbacks in conventional cooperative spectrum sensing, this paper proposes a fuzzy system based cooperative spectrum sensing. Selection combining (SC) and maximum ratio combining (MRC) are used at fuzzy based fusion center to obtain the value of the sensing energy. These energy values are utilized in finding the presence of PU, results in improved sensing accuracy. In addition, an intelligent fuzzy fusion algorithm determines the PU presence without the channel state information based on multiple threshold values. Simulation results show that the proposed scheme outperforms the existing schemes in terms of sensing accuracy.

Self-Coexistence Among Cognitive Radio Networks Using Risk-Motivated Channel Selection Based Deference Structure

Among cognitive radio networks there is a persistent trend of competition to acquire under-utilized and idle channels for data transmission. The competition for spectrum resources often results in the misuse of the spectrum resources as networks experience contention in attempt to access unoccupied spectrum bands. The competitive scenario causes cognitive radio networks to incur a huge amount of loss, which constitutes a major problem of self-coexistence among networks. As a way to minimize these losses we present a self-coexistence mechanism that allows cognitive radio networks to coexist with each other by implementing a risk-motivated channel selection based on deference structure. Cognitive radio networks form deference structure community to have more efficient access to a channel of interest and can defer transmission to one another on that channel, thereby minimizing the chances of conflicts. As part of the decision making process to become a member of a deference structure community, cognitive radio networks rely on a risk-motivated channel selection scheme to evaluate the tentative deference structure channel. We provide numerical and simulation results that demonstrates the benefits of the proposed self-coexistence mechanism and show how it helps networks to coordinate their spectrum activities, minimize contention experienced and improve their utility. We also emphasize on the importance of the deference structure community size with regards to the average performance of member networks.

A Complex Systems Science Perspective on Wireless Networks

The paper targets a future world where all wireless networks are self-organising entities and in which the predominant mode of spectrum access is dynamic. The paper explores whether the behaviour of a collection of autonomous self-organising wireless systems can be treated as a complex system and whether complex systems science can shed light on the design and deployment of these networks. The authors focus on networks that self-organise from a frequency perspective to understand the behaviour of a collection of wireless self-organising nodes. Each autonomous network is modelled as a cell in a lattice and follows a simple set of self-organisation rules. Two scenarios are considered, one in which each cell is based on cellular automata and which provides an abstracted view of interference and a second in which each cell uses a self-organising technique which more accurately accounts for interference. The authors use excess entropy to measure complexity and in combination with entropy gain an understanding of the structure emerging in the lattice for the self-organising networks. The authors show that the self-organising systems presented here do exhibit complex behaviour. Finally,the authors look at the robustness of these complex systems and show that they are robust against changes in the environment.

RBC:A Memory Architecture for Improved Performance and Energy Efficiency

DRAM-based memory suffers from increasing row buffer conflicts,which causes significant performance degradation and power consumption.As memory capacity increases,the overheads of the row buffer conflict are increasingly worse as increasing bitline length,which results in high row activation and precharge latencies.In this work,we propose a practical approach called Row Buffer Cache(RBC)to mitigate row buffer conflict overheads efficiently.At the core of our proposed RBC architecture,the rows with good spatial locality are cached and protected,which are exempted from being interrupted by the accesses for rows with poor locality.Such an RBC architecture significantly reduces the overheads of performance and energy caused by row activation and precharge,and thus improves overall system performance and energy efficiency.We evaluate RBC architecture using SPEC CPU2006 on a DDR4 memory compared to a commodity baseline memory system.Results show that RBC improves the overall performance by up to 2:24(16:1%on average)and reduces the memory energy by up to 68:2%(23:6%on average)for single-core simulations.For multi-core simulations,RBC increases the overall performance by up to1:55(17%on average)and reduces memory energy consumption by up to 35:4%(21:3%on average).

Guardband Analysis for Distributed OFDMA with User Heterogeneity

This paper presents an in-depth analysis of the interference strength and required guardband width between coexistent users for distributed orthogonal frequency division multiple access （OFDMA）. In dynamic spectrum access networks, the cross-band interference between spectrally adjacent users is considered harmful with frequency guardbands inserted between spectrum blocks to eliminate the interference. However, the strength of the cross-band interference depends heavily on the user heterogeneity in different OFDM configurations. The cross-band interference due to the three user heterogeneity artifacts of power heterogeneity, sampling rate heterogeneity, and symbol length heterogeneity is investigated to determine the required guardband width. Analytical and simulation results show that the greater user heterogeneity requires larger guardbands with the sampling rate heterogeneity having the greatest effect. These results can be used to assist the design of spectrum allocation strategies.

Spectrum Occupancy Prediction for Realistic Traffic Scenarios: Time Series versus Learning-Based Models

Spectrum occupancy information is neces-sary in a cognitive radio network(CRN)as it helps in modeling and predicting the spectrum availability for efficient dynamic spectrum access(DSA).However,in a CRN,it is difficult to ascertain a priori the pattern of the spectrum usage of the primary user due to its stochastic behavior.In this context,the spectrum occupancy predic-tion proves to be very useful in enhancing the quality of experience of the secondary user.This paper investigates the practical prowess of various time-series modeling approaches and the machine learning(ML)techniques for predicting spectrum occupancy,based on a spectrum measurement campaign conducted in Jaipur,Rajasthan,India.Moreover,the comparison analysis conducted between the above two approaches highlights the trade-off in terms of the respective performance depending upon the nature of the spectrum occupancy data.Nevertheless,prediction through ML-based recurrent neural network proves to perform reasonably well,thereby providing an accurate future spectrum occupancy information for DSA.

Intelligent cognitive spectrum collaboration:Convergence of spectrum sensing,spectrum access,and coding technology

For a future scenario where everything is connected,cognitive technology can be used for spectrum sensing and access,and emerging coding technologies can be used to address the erasure of packets caused by dynamic spectrum access and realize cognitive spectrum collaboration among users in mass connection scenarios.Machine learning technologies are being increasingly used in the implementation of smart networks.In this paper,after an overview of several key technologies in the cognitive spectrum collaboration,a joint optimization algorithm of dynamic spectrum access and coding is proposed and implemented using reinforcement learning,and the effectiveness of the algorithm is verified by simulations,thus providing a feasible research direction for the realization of cognitive spectrum collaboration.