This paper studies the multi-agent differential game based problem and its application to cooperative synchronization control. A systematized formulation and analysis method for the multi-agent differential game is proposed and a data-driven methodology based on the reinforcement learning (RL) technique is given. First, it is pointed out that typical distributed controllers may not necessarily lead to global Nash equilibrium of the differential game in general cases because of the coupling of networked interactions. Second, to this end, an alternative local Nash solution is derived by defining the best response concept, while the problem is decomposed into local differential games. An off-policy RL algorithm using neighboring interactive data is constructed to update the controller without requiring a system model, while the stability and robustness properties are proved. Third, to further tackle the dilemma, another differential game configuration is investigated based on modified coupling index functions. The distributed solution can achieve global Nash equilibrium in contrast to the previous case while guaranteeing the stability. An equivalent parallel RL method is constructed corresponding to this Nash solution. Finally, the effectiveness of the learning process and the stability of synchronization control are illustrated in simulation results.

Yu SHI;Yongzhao HUA;Jianglong YU;Xiwang DONG;Zhang REN

School of Automation Science and Electrical Engineering,Beihang University,Beijing 100191,China;Institute of Artificial Intelligence,Beihang University,Beijing 100191,China

信息与电子工程前沿（英文）

[1]Yu SHI;Yongzhao HUA;Jianglong YU;Xiwang DONG;Zhang REN-.Multi-agent differential game based cooperative synchronization control using a data-driven method)[J].信息与电子工程前沿（英文）,2022(07):1043-1056

systematized

Multi,agent,differential,cooperative,synchronization,using,data,driven,This,paper,studies,multi,problem,its,application,formulation,analysis,proposed,methodology,reinforcement,learning,RL,technique,given,First,pointed,that,typical,distributed,controllers,may,necessarily,lead,global,Nash,equilibrium,general,cases,because,coupling,networked,interactions,Second,this,end,alternative,local,solution,derived,by,defining,best,response,concept,while,decomposed,into,games,An,off,policy,algorithm,neighboring,interactive,constructed,update,without,requiring,model,stability,robustness,properties,are,proved,Third,further,tackle,dilemma,another,configuration,investigated,modified,functions,can,achieve,contrast,previous,guaranteeing,equivalent,parallel,corresponding,Finally,effectiveness,process,illustrated,simulation,results

0.540062