This paper studies price-based residential demand response management (PB-RDRM) in smart grids, in which non-dispatchable and dispatchable loads (including general loads and plug-in electric vehicles (PEVs)) are both involved. The PB-RDRM is composed of a bi-level optimization problem, in which the upper-level dynamic retail pricing problem aims to maximize the profit of a utility company (UC) by selecting optimal retail prices (RPs), while the lower-level demand response (DR) problem expects to minimize the comprehensive cost of loads by coordinating their energy consumption behavior. The challenges here are mainly two-fold: 1) the uncertainty of energy consumption and RPs; 2) the flexible PEVs' temporally coupled constraints, which make it impossible to directly develop a model-based optimization algorithm to solve the PB-RDRM. To address these challenges, we first model the dynamic retail pricing problem as a Markovian decision process (MDP), and then employ a model-free reinforcement learning (RL) algorithm to learn the optimal dynamic RPs of UC according to the loads' responses. Our proposed RL-based DR algorithm is benchmarked against two model-based optimization approaches (i.e., distributed dual decomposition-based (DDB) method and distributed primal-dual interior (PDI)-based method), whichrequire exact load and electricity price models. The comparison results show that, compared with the benchmark solutions, our proposed algorithm can not only adaptively decide the RPs through on-line learning processes, but also achieve larger social welfare within an unknown electricity market environment.

Yanni Wan

Department of Automation,University of Science and Technology of China,Hefei 230027,China;Institute of Artificial Intelligence,Hefei Comprehensive National Science Center,Hefei 230088,China;School of Engineering,RMIT University,VIC 3000,Australia;State Key Laboratory of Synthetical Automation for Process Industries,Northeastern University,Shenyang 110819,China;Department of Automation,State Key Laboratory of Fire Science,Institute of Advanced Technology,University of Science and Technology of China,Hefei 230027,China;Key Laboratory of Technology in Geo-Spatial Information Processing and Application Systems,Chinese Academy of Sciences,Beijing 100190,China

自动化学报（英文版）

[1]Yanni Wan-.Price-Based Residential Demand Response Management in Smart Grids: A Reinforcement Learning-Based Approach)[J].自动化学报（英文版）,2022(01):123-134

RDRM,dispatchable,RPs,whichrequire

Price,Based,Residential,Demand,Response,Management,Smart,Grids,Reinforcement,Learning,Approach,This,paper,studies,residential,demand,management,PB,smart,grids,loads,including,general,plug,vehicles,PEVs,both,involved,composed,bi,level,optimization,problem,upper,dynamic,retail,pricing,aims,maximize,profit,utility,company,UC,by,selecting,optimal,prices,while,lower,expects,minimize,comprehensive,cost,coordinating,their,energy,consumption,behavior,challenges,here,mainly,two,fold,uncertainty,flexible,temporally,coupled,constraints,make,impossible,directly,develop,algorithm,solve,To,address,these,first,Markovian,decision,MDP,then,employ,free,reinforcement,learning,RL,according,responses,Our,proposed,benchmarked,against,approaches,distributed,dual,decomposition,DDB,method,primal,interior,PDI,exact,electricity,models,comparison,results,show,that,compared,solutions,our,can,not,only,adaptively,decide,through,line,processes,also,achieve,larger,social,welfare,within,unknown,market,environment

0.57233