A Learning Automata Based Area Coverage Algorithm for Wireless Sensor Networks

One way to reduce energy consumption in wireless sensor networks is to reduce the number of active nodes in the network. When sensors are redundantly deployed, a subset of sensors should be selected to actively monitor the field （referred to as a ＂cover＂）, whereas the rest of the sensors should be put to sleep to conserve their batteries. In this paper, a learning automata based algorithm for energy-efficient monitoring in wireless sensor networks （EEMLA） is proposed. Each node in EEMLA algorithm is equipped with a learning automaton which decides for the node to be active or not at any time during the operation of the network. Using feedback received from neighboring nodes, each node gradually learns its proper state during the operation of the network. Experimental results have shown that the proposed monitoring algorithm in comparison to other existing methods such as Tian and LUC can better prolong the network lifetime.

A Stable and Energy-Efficient Routing Algorithm Based on Learning Automata Theory for MANET

The mobile Ad Hoc network(MANET)is a self-organizing and self-configuring wireless network,consisting of a set of mobile nodes.The design of efficient routing protocols for MANET has always been an active area of research.In existing routing algorithms,however,the current work does not scale well enough to ensure route stability when the mobility and distribution of nodes vary with time.In addition,each node in MANET has only limited initial energy,so energy conservation and balance must be taken into account.An efficient routing algorithm should not only be stable but also energy saving and balanced,within the dynamic network environment.To address the above problems,we propose a stable and energy-efficient routing algorithm,based on learning automata(LA)theory for MANET.First,we construct a new node stability measurement model and define an effective energy ratio function.On that basis,we give the node a weighted value,which is used as the iteration parameter for LA.Next,we construct an LA theory-based feedback mechanism for the MANET environment to optimize the selection of available routes and to prove the convergence of our algorithm.The experiments show that our proposed LA-based routing algorithm for MANET achieved the best performance in route survival time,energy consumption,energy balance,and acceptable per-formance in end-to-end delay and packet delivery ratio.

A Reinforcement-learning-based Bidding Strategy for Power Suppliers with Limited Information

The power market is a typical imperfectly competitive market where power suppliers gain higher profits through strategic bidding behaviors.Most existing studies assume that a power supplier is accessible to the sufficient market information to derive an optimal bidding strategy.However,this assumption may not be true in reality,particularly when a power market is newly launched.To help power suppliers bid with the limited information,a modified continuous action reinforcement learning automata algorithm is proposed.This algorithm introduces the discretization and Dyna structure into continuous action reinforcement learning automata algorithm for easy implementation in a repeated game.Simulation results verify the effectiveness of the proposed learning algorithm.

Model learning:a survey of foundations,tools and applications

Software systems are present all around us and playing their vital roles in our daily life.The correct functioning of these systems is of prime concern.In addition to classical testing techniques,formal techniques like model checking are used to reinforce the quality and reliability of software systems.However,obtaining of behavior model,which is essential for model-based techniques,of unknown software systems is a challenging task.To mitigate this problem,an emerging black-box analysis technique,called Model Learning,can be applied.It complements existing model-based testing and verification approaches by providing behavior models of blackbox systems fully automatically.This paper surveys the model learning technique,which recently has attracted much attention from researchers,especially from the domains of testing and verification.First,we review the background and foundations of model learning,which form the basis of subsequent sections.Second,we present some well-known model learning tools and provide their merits and shortcomings in the form of a comparison table.Third,we describe the successful applications of model learning in multidisciplinary fields,current challenges along with possible future works,and concluding remarks.

Cognitive Power Management in Wireless Sensor Networks

Dynamic power management （DPM） in wireless sensor nodes is a well-known technique for reducing idle energy consumption. DPM controls a node＇s operating mode by dynamically toggling the on/off status of its units based on predictions of event occurrences. However, since each mode change induces some overhead in its own right, guaranteeing DPM＇s eificiency is no mean feat in environments exhibiting non-determinism and uncertainty with unknown statistics. Our solution suite in this paper, collectively referred to as cognitive power management （CPM）, is a principled attempt toward enabling DPM in statistically unknown settings and gives two different analytical guarantees. Our first design is based on learning automata and guarantees better-than-pure-chance DPM in the face of non-stationary event processes. Our second solution caters tor an even more general setting in which event occurrences may take on an adversarial character. In this case, we formulate the interaction of an individual mote with its environment in terms of a repeated zero-sum game in which the node relies on a no-external-regret procedure to learn its mini-max strategies in an online fashion. We conduct numerical experiments to measure the performance of our schemes in terms of network lifetime and event loss percentage.