With the deep integration of advanced information technologies,such as artificial intelligence and traditional energy technologies,smart energy systems have been proposed as a method to provide the best solution for the coordination,balance,and control of the entire energy system.As a new way of energy balance and interaction in the user side energy market,a peer-to-peer(P2P)electricity transaction can effectively promote energy sharing within the user group and improve the economic benefits of users participating in the energy market.Reinforcement learning(RL)is an artificial intelligence method in which agents continuously acquire relevant experience and knowledge during the interaction with the environment,automatically update their decision-making behavior,and achieve maximum return.It is a suitable approach for P2P transaction decision analysis of small-scale users in the context of smart energy.First,this paper establishes a P2P transaction model that includes a participant model,equipment model and price model.Secondly,the transaction problem is equivalent to a Markov decision process(MDP)and each learning element model is established.Then,the MDP problem is solved and analyzed using the SARSA RL algorithm with average discrete processing.Finally,a case study of a community with multiple users is conducted to verify the effectiveness,economy,and security of the RL method in solving energy storage action selection and transaction decision problems of energy storage users.

Dan Wang;Bo Liu;Hongjie Jia;Ziyang Zhang;Jingcheng Chen;Deyu Huang

Key Laboratory of Smart Grid of Ministry of Education,Tianjin University,Tianjin 300072,China;Key Laboratory of Smart Energy&Information Technology of Tianjin Municipality,Tianjin 30072,China;State Grid Jiangsu Power Co.Ltd.,Zhenjiang Power Supply Company,Zhenjiang 212000,Jiangsu Province,China;State Grid Tianjin Electric Power Company,Hebei District,Tianjin 300010,China

中国电机工程学会电力与能源系统学报（英文版）

[1]Dan Wang;Bo Liu;Hongjie Jia;Ziyang Zhang;Jingcheng Chen;Deyu Huang-.Peer-to-peer Electricity Transaction Decisions of the User-side Smart Energy System Based on the SARSA Reinforcement Learning)[J].中国电机工程学会电力与能源系统学报（英文版）,2022(03):826-837

Peer,peer,Electricity,Transaction,Decisions,User,side,Smart,Energy,System,Based,SARSA,Reinforcement,Learning,With,deep,integration,advanced,information,technologies,such,artificial,intelligence,traditional,energy,smart,systems,have,been,proposed,method,provide,best,solution,coordination,balance,control,entire,new,way,interaction,market,P2P,electricity,transaction,can,effectively,promote,sharing,within,group,improve,economic,benefits,users,participating,learning,RL,which,agents,continuously,acquire,relevant,experience,knowledge,during,environment,automatically,update,their,decision,making,behavior,achieve,maximum,return,It,suitable,approach,analysis,small,scale,context,First,this,paper,establishes,model,that,includes,participant,equipment,price,Secondly,equivalent,Markov,MDP,each,element,established,Then,solved,analyzed,using,algorithm,average,discrete,processing,Finally,case,study,community,multiple,conducted,verify,effectiveness,economy,security,solving,storage,selection,problems

0.591157