Autonomous Vehicle Platoons In Urban Road Networks:A Joint Distributed Reinforcement Learning and Model Predictive Control Approach

In this paper, platoons of autonomous vehicles operating in urban road networks are considered. From a methodological point of view, the problem of interest consists of formally characterizing vehicle state trajectory tubes by means of routing decisions complying with traffic congestion criteria. To this end, a novel distributed control architecture is conceived by taking advantage of two methodologies: deep reinforcement learning and model predictive control. On one hand, the routing decisions are obtained by using a distributed reinforcement learning algorithm that exploits available traffic data at each road junction. On the other hand, a bank of model predictive controllers is in charge of computing the more adequate control action for each involved vehicle. Such tasks are here combined into a single framework:the deep reinforcement learning output(action) is translated into a set-point to be tracked by the model predictive controller;conversely, the current vehicle position, resulting from the application of the control move, is exploited by the deep reinforcement learning unit for improving its reliability. The main novelty of the proposed solution lies in its hybrid nature: on one hand it fully exploits deep reinforcement learning capabilities for decisionmaking purposes;on the other hand, time-varying hard constraints are always satisfied during the dynamical platoon evolution imposed by the computed routing decisions. To efficiently evaluate the performance of the proposed control architecture, a co-design procedure, involving the SUMO and MATLAB platforms, is implemented so that complex operating environments can be used, and the information coming from road maps(links,junctions, obstacles, semaphores, etc.) and vehicle state trajectories can be shared and exchanged. Finally by considering as operating scenario a real entire city block and a platoon of eleven vehicles described by double-integrator models, several simulations have been performed with the aim to put in light the main f eatures of the proposed approach. Moreover, it is important to underline that in different operating scenarios the proposed reinforcement learning scheme is capable of significantly reducing traffic congestion phenomena when compared with well-reputed competitors.

Towards Collaborative Robotics in Top View Surveillance:A Framework for Multiple Object Tracking by Detection Using Deep Learning

Collaborative Robotics is one of the high-interest research topics in the area of academia and industry.It has been progressively utilized in numerous applications,particularly in intelligent surveillance systems.It allows the deployment of smart cameras or optical sensors with computer vision techniques,which may serve in several object detection and tracking tasks.These tasks have been considered challenging and high-level perceptual problems,frequently dominated by relative information about the environment,where main concerns such as occlusion,illumination,background,object deformation,and object class variations are commonplace.In order to show the importance of top view surveillance,a collaborative robotics framework has been presented.It can assist in the detection and tracking of multiple objects in top view surveillance.The framework consists of a smart robotic camera embedded with the visual processing unit.The existing pre-trained deep learning models named SSD and YOLO has been adopted for object detection and localization.The detection models are further combined with different tracking algorithms,including GOTURN,MEDIANFLOW,TLD,KCF,MIL,and BOOSTING.These algorithms,along with detection models,help to track and predict the trajectories of detected objects.The pre-trained models are employed;therefore,the generalization performance is also investigated through testing the models on various sequences of top view data set.The detection models achieved maximum True Detection Rate 93%to 90%with a maximum 0.6%False Detection Rate.The tracking results of different algorithms are nearly identical,with tracking accuracy ranging from 90%to 94%.Furthermore,a discussion has been carried out on output results along with future guidelines.

ResIoT:An IoT Social Framework Resilient to Malicious Activities

The purpose of the next internet of things(Io T)is that of making available myriad of services to people by high sensing intelligent devices capable of reasoning and real time acting.The convergence of Io T and multi-agent systems(MAS)provides the opportunity to benefit from the social attitude of agents in order to perform machine-to-machine(M2 M)cooperation among smart entities.However,the selection of reliable partners for cooperation represents a hard task in a mobile and federated context,especially because the trustworthiness of devices is largely unreferenced.The issues discussed above can be synthesized by recalling the well known concept of social resilience in Io T systems,i.e.,the capability of an Io T network to resist to possible attacks by malicious agent that potentially could infect large areas of the network,spamming unreliable information and/or assuming unfair behaviors.In this sense,social resilience is devoted to face malicious activities of software agents in their social interactions,and do not deal with the correct working of the sensors and other information devices.In this setting,the use of a reputation model can be a practicable and effective solution to form local communities of agents on the basis of their social capabilities.In this paper,we propose a framework for agents operating in an Io T environment,called Res Io T,where the formation of communities for collaborative purposes is performed on the basis of agent reputation.In order to validate our approach,we performed an experimental campaign by means of a simulated framework,which allowed us to verify that,by our approach,devices have not any economic convenience to performs misleading behaviors.Moreover,further experimental results have shown that our approach is able to detect the nature of the active agents in the systems(i.e.,honest and malicious),with an accuracy of not less than 11%compared to the best competitor tested and highlighting a high resilience with respect to some malicious activities.

Guest Editorial for Special Issue on Blockchain for Internet-of-Things and Cyber-Physical Systems

Cyber-physical systems(CPS)are increasingly commonplace,with applications in energy,health,transportation,and many other sectors.One of the major requirements in CPS is that the interaction between cyber-world and man-made physical world(exchanging and sharing of data and information with other physical objects and systems)must be safe,especially in bi-directional communications.In particular,there is a need to suitably address security and/or privacy concerns in this human-in-the-loop CPS ecosystem.However,existing centralized architecture models in CPS,and also the more general IoT systems,have a number of associated limitations,in terms of single point of failure,data privacy,security,robustness,etc.Such limitations reinforce the importance of designing reliable,secure and privacy-preserving distributed solutions and other novel approaches,such as those based on blockchain technology due to its features(e.g.,decentralization,transparency and immutability of data).This is the focus of this special issue.

Resilient Control in Large-Scale Networked Cyber-Physical Systems: Guest Editorial

RECENT advances in sensing,communication and computing have open the door to the deployment of largescale networks of sensors and actuators that allow fine-grain monitoring and control of a multitude of physical processes and infrastructures.The appellation used by field experts for these paradigms is Cyber-Physical Systems(CPS)because the dynamics among computers,networking media/resources and physical systems interact in a way that multi-disciplinary technologies(embedded systems,computers,communications and controls)are required to accomplish prescribed missions.Moreover,they are expected to play a significant role in the design and development of future engineering applications such as smart grids,transportation systems,nuclear plants and smart factories.

Energy Theft Detection in Smart Grids:Taxonomy,Comparative Analysis,Challenges,and Future Research Directions

Electricity theft is one of the major issues in developing countries which is affecting their economy badly.Especially with the introduction of emerging technologies,this issue became more complicated.Though many new energy theft detection(ETD)techniques have been proposed by utilising different data mining(DM)techniques,state&network(S&N)based techniques,and game theory(GT)techniques.Here,a detailed survey is presented where many state-of-the-art ETD techniques are studied and analysed for their strengths and limitations.Three levels of taxonomy are presented to classify state-of-the-art ETD techniques.Different types and ways of energy theft and their consequences are studied and summarised and different parameters to benchmark the performance of proposed techniques are extracted from literature.The challenges of different ETD techniques and their mitigation are suggested for future work.It is observed that the literature on ETD lacks knowledge management techniques that can be more effective,not only for ETD but also for theft tracking.This can help in the prevention of energy theft,in the future,as well as for ETD.

Evaluating Group Formation in Virtual Communities

In this paper, we are interested in answering the following research question: "Is it possible to form effective groups in virtual communities by exploiting trust information without significant overhead, similarly to real user communities?"In order to answer this question, instead of adopting the largely used approach of exploiting the opinions provided by all the users of the community(called global reputation), we propose to use a particular form of reputation, called local reputation. We also propose an algorithm for group formation able to implement the proposed procedure to form effective groups in virtual communities. Another interesting question is how to measure the effectiveness of groups in virtual communities. To this aim we introduce the index in a measure of the effectiveness of the group formation. We tested our algorithm by realizing some experimental trials on real data from the real world EPINIONS and CIAO communities, showing the significant advantages of our procedure w.r.t. another prominent approach based on traditional global reputation.