Towards a multi-agent reinforcement learning approach for joint sensing and sharing in cognitive radio networks

The adoption of the Fifth Generation(5G)and beyond 5G networks is driving the demand for learning approaches that enable users to co-exist harmoniously in a multi-user distributed environment.Although resource-constrained,the Cognitive Radio(CR)has been identified as a key enabler of distributed 5G and beyond networks due to its cognitive abilities and ability to access idle spectrum opportunistically.Reinforcement learning is well suited to meet the demand for learning in 5G and beyond 5G networks because it does not require the learning agent to have prior information about the environment in which it operates.Intuitively,CRs should be enabled to implement reinforcement learning to efficiently gain opportunistic access to spectrum and co-exist with each other.However,the application of reinforcement learning is straightforward in a single-agent environment and complex and resource intensive in a multi-agent and multi-objective learning environment.In this paper,(1)we present a brief history and overview of reinforcement learning and its limitations;(2)we provide a review of recent multi-agent learning methods proposed and multi-agent learning algorithms applied in Cognitive Radio(CR)networks;and(3)we further present a novel framework for multi-CR reinforcement learning and conclude with a synopsis of future research directions and recommendations.

Estimation of Postmortem Interval Using the Radiological Techniques,Computed Tomography:A Pilot Study

Estimation of postmortem interval(PMI)has been an important and difficult subject in the forensic study.It is a primary task of forensic work,and it can help guide the work in field investigation.With the development of computed tomography(CT)technology,CT imaging techniques are now being more frequently applied to the field of forensic medicine.This study used CT imaging techniques to observe area changes in different tissues and organs of rabbits after death and the changing pattern of the average CT values in the organs.The study analyzed the relationship between the CT values of different organs and PMI with the imaging software Max Viewer and obtained multiparameter nonlinear regression equation of the different organs,and the study provided an objective and accurate method and reference information for the estimation of PMI in the forensic medicine.In forensic science,PMI refers to the time interval between the discovery or inspection of corpse and the time of death.CT,magnetic resonance imaging,and other imaging techniques have become important means of clinical examinations over the years.Although some scholars in our country have used modem radiological techniques in various fields of forensic science,such as estimation of injury time,personal identification of bodies,analysis of the cause of death,determination of the causes of injury,and identification of the foreign substances of bodies,there are only a few studies on the estimation of time of death.We detected the process of subtle changes in adult rabbits after death,the shape and size of tissues and organs,and the relationship between adjacent organs in three-dimensional space in an efiort to develop new method for the estimation of PMI.The bodies of the dead rabbits were stored at 20℃ room temperature,sealed condition,and prevented exposure to flesh flies.The dead rabbits were randomly divided into comparison group and experimental group.The whole-body CT scans were performed on the experimental group of rabbits at different PMIs.NeuViz dual-slice spiral CT scanner(made by Neusoft Medical in China,2 mm×10 mm high-speed rare earth ceramic detector)is a 360°scan that could obtain two images,capable of providing a wide range of high-speed continuous spiral scan.Max Viewer(Version:1.0.0131,Neusoft,Shenyang,China)is a CT image viewing software developed by Neusoft Medical.The software can be used to view and process images in various common methods and to measure a number of parameters,such as length,area,angle,and CT values.Statistical analysis was performed using SPSS1 Statistics 19.0(SPSS,Inc.,Chicago,IL,USA).AP<0.05 was considered statistically significant.We obtained the binomial regression equation of the CT values and the related coefficient(R^(2)).In the future,we suggests that comprehensive analyses of various indicators of different organs could establish a diversified pattern to remedy the deficiencies and make the study of PMI estimation more scientific and enhance the operability.

THE EMERGING CHALLENGE:THE VALIDITY OF COMPUTATIONAL ORGANIZATION MODEL AND METHODOLOGY

The validity of the computational organization model is a necessary condition and also poses a bottleneck in computational organization theory development.Established on the frontline of computational organization theory,this paper provides an overall discussion on the difficulties in validation and validation methodology of the computational organization model.First,different from natural engineering system model,and also exceeding the traditional empirical method,the computational organization model is faced with various subjective and objective difficulties during the validation progress;second,in developing the situational validation methodology based on relationships among problem field,modeling purpose and referents,it is important for the computational organization model that the validation is conducted according to certain degree;third,the establishment of verification,validation and accreditation approach(VV&A),which is different from the natural engineering system,is an irresistible trend for future development of the computational organization model.Model validation should be focused on concept validation,operation validation,and data validation,as well as the principle of iterative validation approach and such validation should be conducted throughout the modeling process;finally,the validation of computational organization model should be the process to enhance people’s confidence in the model.From the perspective of review,if the model is able to pass through all the validation tests,it is helpful for a better understanding of the model’s ability,limitation and applicability.In this case,research from interdisciplinary experts is required urgently.