Delay-Optimal Random Access in Large-Scale Energy Harvesting IoT Networks Based on Mean Field Game

With energy harvesting capability, the Internet of things(IoT) devices transmit data depending on their available energy, which leads to a more complicated coupling and brings new technical challenges to delay optimization. In this paper,we study the delay-optimal random access(RA) in large-scale energy harvesting IoT networks. We model a two-dimensional Markov decision process(MDP)to address the coupling between the data and energy queues, and adopt the mean field game(MFG) theory to reveal the coupling among the devices by utilizing the large-scale property. Specifically, to obtain the optimal access strategy for each device, we derive the Hamilton-Jacobi-Bellman(HJB) equation which requires the statistical information of other devices.Moreover, to model the evolution of the states distribution in the system, we derive the Fokker-PlanckKolmogorov(FPK) equation based on the access strategy of devices. By solving the two coupled equations,we obtain the delay-optimal random access solution in an iterative manner with Lax-Friedrichs method. Finally, the simulation results show that the proposed scheme achieves significant performance gain compared with the conventional schemes.

A Fractional Micro-Macro Model for Crowds of Pedestrians Based on Fractional Mean Field Games

Modeling a crowd of pedestrians has been considered in this paper from different aspects. Based on fractional microscopic model that may be much more close to reality, a fractional macroscopic model has been proposed using conservation law of mass. Then in order to characterize the competitive and cooperative interactions among pedestrians, fractional mean field games are utilized in the modeling problem when the number of pedestrians goes to infinity and fractional dynamic model composed of fractional backward and fractional forward equations are constructed in macro scale. Fractional micromacro model for crowds of pedestrians are obtained in the end.Simulation results are also included to illustrate the proposed fractional microscopic model and fractional macroscopic model,respectively.

Data Security Defense and Algorithm for Edge Computing Based on Mean Field Game

With the development of the Internet of Things,the edge devices are increasing.Cyber security issues in edge computing have also emerged and caused great concern.We propose a defense strategy based on Mean field game to solve the security issues of edge user data during edge computing.Firstly,an individual cost function is formulated to build an edge user data security defense model.Secondly,we research the𝜀𝜀-Nash equilibrium of the individual cost function with finite players and prove the existence of the optimal defense strategy.Finally,by analyzing the stability of edge user data loss,it proves that the proposed defense strategy is effective.

A Trading Execution Model Based on Mean Field Games and Optimal Control

We present a trading execution model that describes the behaviour of a big trader and of a multitude of retail traders operating on the shares of a risky asset. The retail traders are modeled as a population of “conservative” investors that: 1) behave in a similar way, 2) try to avoid abrupt changes in their trading strategies, 3) want to limit the risk due to the fact of having open positions on the asset shares, 4) in the long run want to have a given position on the asset shares. The big trader wants to maximize the revenue resulting from the action of buying or selling a (large) block of asset shares in a given time interval. The behaviour of the retail traders and of the big trader is modeled using respectively a mean field game model and an optimal control problem. These models are coupled by the asset share price dynamic equation. The trading execution strategy adopted by the retail traders is obtained solving the mean field game model. This strategy is used to formulate the optimal control problem that determines the behaviour of the big trader. The previous mathematical models are solved using the dynamic programming principle. In some special cases explicit solutions of the previous models are found. An extensive numerical study of the trading execution model proposed is presented. The interested reader is referred to the website: http://www.econ.univpm.it/recchioni/finance/w19 to find material including animations, an interactive application and an app that helps the understanding of the paper. A general reference to the work of the authors and of their coauthors in mathematical finance is the website:?http://www.econ.univpm.it/recchioni/finance.

Mean field game of optimal relative investment with jump risk

In this paper,we study the n-player game and the mean field game under the constant relative risk aversion relative performance on terminal wealth,in which the interaction occurs by peer competition.In the model with n agents,the price dynamics of underlying risky assets depend on a common noise and contagious jump risk modeled by a multi-dimensional nonlinear Hawkes process.With a continuum of agents,we formulate the mean field game problem and characterize a deterministic mean field equilibrium in an analytical form under some conditions,allowing us to investigate some impacts of model parameters in the limiting model and discuss some financial implications.Moreover,based on the mean field equilibrium,we construct an approximate Nash equilibrium for the n-player game when n is sufficiently large.The explicit order of the approximation error is also derived.

Linear Quadratic Mean Field Games:Decentralized O(1/N)-Nash Equilibria

This paper studies an asymptotic solvability problem for linear quadratic(LQ)mean field games with controlled diffusions and indefinite weights for the state and control in the costs.The authors employ a rescaling approach to derive a low dimensional Riccati ordinary differential equation(ODE)system,which characterizes a necessary and sufficient condition for asymptotic solvability.The rescaling technique is further used for performance estimates,establishing an O(1/N)-Nash equilibrium for the obtained decentralized strategies.

Mean Field Games with Common Noises and Conditional Distribution Dependent FBSDEs

In this paper,the authors consider the mean field game with a common noise and allow the state coefficients to vary with the conditional distribution in a nonlinear way.They assume that the cost function satisfies a convexity and a weak monotonicity property.They use the sufficient Pontryagin principle for optimality to transform the mean field control problem into existence and uniqueness of solution of conditional distribution dependent forward-backward stochastic differential equation(FBSDE for short).They prove the existence and uniqueness of solution of the conditional distribution dependent FBSDE when the dependence of the state on the conditional distribution is sufficiently small,or when the convexity parameter of the running cost on the control is sufficiently large.Two different methods are developed.The first method is based on a continuation of the coefficients,which is developed for FBSDE by[Hu,Y.and Peng,S.,Solution of forward-backward stochastic differential equations,Probab.Theory Rel.,103(2),1995,273–283].They apply the method to conditional distribution dependent FBSDE.The second method is to show the existence result on a small time interval by Banach fixed point theorem and then extend the local solution to the whole time interval.

Mean field games of controls:Propagation of monotonicities

The theory of Mean Field Game of Controls considers a class of mean field games where the interaction is through the joint distribution of the state and control.It is well known that,for standard mean field games,certain monotonicity conditions are crucial to guarantee the uniqueness of mean field equilibria and then the global wellposedness for master equations.In the literature the monotonicity condition could be the Lasry–Lions monotonicity,the displacement monotonicity,or the anti-monotonicity conditions.In this paper,we investigate these three types of monotonicity conditions for Mean Field Games of Controls and show their propagation along the solutions to the master equations with common noises.In particular,we extend the displacement monotonicity to semi-monotonicity,whose propagation result is new even for standard mean field games.This is the first step towards the global wellposedness theory for master equations of Mean Field Games of Controls.

基于平均场博弈的钢铁生产企业电-碳市场非合作博弈均衡分析

随着电力市场改革和全国碳市场建设工作的推进,负荷侧的高耗能、高排放企业需参与市场化运行。数量众多的高耗能、高排放企业在电–碳市场中的策略性行为将对自身运行、电价、碳排放权价格产生重要影响。该文以钢铁生产企业为例,分析其对电–碳市场的影响。首先,针对钢铁生产企业的生产工艺流程,构建其碳排放以及电能消费模型。其次,基于电力价格以及碳排放权市场交易模型,构建钢铁生产企业参与电–碳市场的运行交易决策模型,以及市场博弈均衡模型。此外,针对策略性主体数目众多时均衡点求解困难的实际问题,提出了基于平均场博弈理论的均衡点求解方法,并证明所采用平均场博弈方法均衡点的存在性、唯一性以及与原博弈问题纳什均衡点的等价性。采用Mann型迭代格式,设计均衡点求解的加速算法。最后,通过算例仿真证明了所提模型的有效性以及算法的计算效率优势。

基于平均场博弈的电动汽车计算卸载及能源管理联合策略

在能源互联网中,电动汽车(EV)不仅是一种能够参与能源需求响应的分布式移动能源节点,也是一种能够执行计算任务并通过无线传输将部分任务卸载给移动边缘计算(MEC)中心的计算节点。由于计算耗能不可忽略,文中从优化成本的角度考虑具有计算需求及能源需求的大规模EV的计算卸载及能源管理问题。考虑到EV的传输速率和充放电功率等策略受其他EV策略的影响,将EV的决策过程建模为平均场博弈(MFG)模型。首先,定义并计算MEC计算价格、传输干扰和瞬时电价等平均场项;然后,通过求解HJB-FPK方程组得到基于MFG的最优计算卸载及能源管理策略。算例分析结果表明了所提策略不仅具有最优的经济效益,而且在维护电网稳定运行方面具有明显的优越性。

最优投资与风险控制策略的多人非零和博弈及平均场博弈

金融市场中存在大量的机构投资者,机构投资者追求高回报高财富的特性导致市场竞争日益激烈,竞争的市场环境使得机构投资者不仅追求自身财富的最大化,还关注与竞争对手之间的财富差距。本文研究多个机构投资者策略互动下的投资与风险控制问题。假设每个投资者均可以将财富投资于金融市场中以实现财富增值,同时通过购买保险等方式将面临的风险部分转移给其他金融机构。使用投资者自身财富与市场平均财富之差描述的相对业绩刻画市场竞争,投资者的目标是最大化终端时刻相对绩效的期望效用,在非零和博弈框架下构建了多人投资与风险控制博弈模型,以CARA效用函数为例,运用随机微分博弈理论和平均场博弈理论求出Nash均衡状态下的最优投资与风险控制策略,并进行参数的敏感性分析。研究发现:竞争将导致风险投资攀升,风险控制减弱,从而导致金融市场的系统性风险增加;机构投资者自身及竞争对手的风险偏好和市场竞争程度均会影响均衡投资与风险控制策略;盈余波动影响风险控制策略发生同向改变,但这种影响在波动轻微时较为明显,当波动超过一定程度时,波动对风险控制策略影响甚微。研究为机构投资者的投资与风险控制策略选择提供了有益指导。

群视角下的多智能体强化学习方法综述

多智能体系统是分布式人工智能领域的前沿研究概念,传统的多智能体强化学习方法主要聚焦群体行为涌现、多智能体合作与协调、智能体间交流与通信、对手建模与预测等主题,但依然面临环境部分可观、对手策略非平稳、决策空间维度高、信用分配难理解等难题,如何设计满足智能体数量规模比较大、适应多类不同应用场景的多智能体强化学习方法是该领域的前沿课题。首先简述了多智能体强化学习的相关研究进展;其次着重从规模可扩展与种群自适应两个视角对多种类、多范式的多智能体学习方法进行了综合概述归纳,系统梳理了集合置换不变性、注意力机制、图与网络理论、平均场理论共四大类规模可扩展学习方法,迁移学习、课程学习、元学习、元博弈共四大类种群自适应强化学习方法,给出典型应用场景;最后从基准平台开发、双层优化架构、对抗策略学习、人机协同价值对齐和自适应博弈决策环共5个方面进行了前沿研究方向展望,该研究可为多模态环境下多智能强化学习的相关前沿重点问题研究提供参考。

Control and Nash Games with Mean Field Effect

Mean field theory has raised a lot of interest in the recent years (see in particular the results of Lasry-Lions in 2006 and 2007,of Gueant-Lasry-Lions in 2011,of HuangCaines-Malham in 2007 and many others).There are a lot of applications.In general,the applications concern approximating an infinite number of players with common behavior by a representative agent.This agent has to solve a control problem perturbed by a field equation,representing in some way the behavior of the average infinite number of agents.This approach does not lead easily to the problems of Nash equilibrium for a finite number of players,perturbed by field equations,unless one considers averaging within different groups,which has not been done in the literature,and seems quite challenging.In this paper,the authors approach similar problems with a different motivation which makes sense for control and also for differential games.Thus the systems of nonlinear partial differential equations with mean field terms,which have not been addressed in the literature so far,are considered here.

条件期望在微分博弈中的应用

以条件期望为工具,研究了一类在部分信息下线性二次非零和平均场微分博弈问题.通过取关于部分信息的条件数学期望,给出了纳什均衡点的表达式.利用条件数学期望的定义和性质,证明了该表达式的合理性.

基于平均场均衡的Ad hoc网络路由协议

为了简化使用完美马尔科夫均衡方法可能引起的复杂计算过程,本文依据博弈论方法,提出一种平均场均衡的无线自组织网络路由协议(mean field equilibrium AODV,MFEA)。该方法要求每个节点利用所有其他节点的信息来分析自己的最优策略,而不需要知道每一个局中人的信息,并且在足够大的局中人数目情况下性能更加近似马尔科夫均衡。仿真实验显示:提出的MFEA路由协议在包投递率、时延和归一化开销方面均优于AODV(Ad hoc on-demand distance vector routing)协议,在节点密集的无线自组织网络中仍可获得比较好效果。

一个虚假而有意义的问题——对“中国哲学学科合法性问题”的解读

中国哲学学科的合法性问题是近年来哲学界讨论的热点。本文认为 ,当人们以西方哲学作为参照系 ,从哲学应该具有什么样的内容的角度来探讨中国哲学学科的合法性时 ,这个问题乃是一个假问题 ,因为在不同的文明中 ,都存在着一个宗教、艺术和科学所无法取代的领域 ,只要这个领域存在 ,研究这个领域的学科也就获得了自己的合法性。至于人们用什么样的名称去指谓它 ,乃是一个与合法性无涉的问题。当人们不是从中国哲学应有的内容 ,而是从其作为一门学科所应有的形式出发去讨论合法性问题时 ,这一问题才真正地获得自己的意义。

一种密集无人机网络下行发射功率控制算法研究

为解决密集无人机网络的下行功率控制问题,降低无人机的相互干扰、提高系统能量效率,提出了一种密集无人机网络下行发射功率控制算法。首先,将无人机网络的功率控制问题建模为高维系统状态下的平均场博弈论模型,降低无人机间的相互干扰;其次,将平均场博弈论模型转化为马尔可夫决策过程,以得到无人机密集部署下的均衡解;另外,提出了一种基于深度强化学习的平均场博弈论算法,通过使用深度神经网络获取系统的最优功率控制策略,从而最大化系统能量效率。最后,通过仿真分析将所提方法与其他3种算法进行对比分析。实验结果表明,所提方法能够使无人机与环境进行有效交互,有效降低无人机的相互干扰,增强系统网络通信性能;同时,与其他3种方法相比,所提方法的收敛速度更快、能量效率更高,具有良好的收敛性与可靠性。

噪声可退化且依赖于状态和分布的平均场博弈

文章考虑状态方程关于状态和控制仿射,效用关于状态和控制凸的平均场博弈,允许状态方程的扩散项可退化且依赖状态和分布.由于允许漂移项和扩散项关于分布可以线性增长,因此可以包含线性二次平均场博弈,且允许状态的期望以线性形式出现在状态方程中.作者证明了对应的McKean-Vlasov型正倒向微分方程解的存在性,并获得了对应的解耦函数的正则性.最后作者证明了用平均场博弈的解和解耦函数可以以N-1/d+4的速度逼近多人博弈的Nash均衡.

基于平均场博弈的新冠肺炎最优防控力度切换策略

本文以新冠肺炎(COVID-19)在美国各州感染人数的公开数据为例,提出新冠肺炎大流行期的多地区随机动态传染模型.为解决何时"开放"或"限制"经济社会活动的问题,本文建立基于平均场交互的期望效用最大化的多地区最优防控切换Nash均衡策略.随后,考虑地区个数为无穷时对应的代表性地区感染人数模型和最优防控切换问题,证明其为有限地区情形Nash均衡的极限.最后,通过比较及分析不同状态下的最优切换边界,我们对何时及如何调整防控力度给出了具体的建议.

部分信息下带跳线性二次平均场类型的二人零和微分对策问题

讨论在部分信息下带跳线性二次平均场类型的二人零和微分对策问题,其中状态方程是由布朗运动和泊松随机鞅测度共同驱动,且包含仿射项的平均场类型的随机微分方程.通过二人零和微分对策中两个决策者的相互作用,引入两个Riccati方程,再利用经典的变分技术和配方法,建立开环鞍点的状态反馈表示和最优的对策值函数,最后通过讨论该问题的一个特例,得到其相应最优控制的反馈表示.