Optimization of Aggregated Energy Resources using Sequential Decision Making

Due to the limited non-renewable sources and focus on clean energy in the contemporary world, most governments are focusing more on renewable energy sources for electrical power. However, the unpredictable variations in the renewable energy source itself can pose serious threats to the grid's operation unless managed. A plausible solution is to manage a combination of resources that can complement each other's generation to meet the total demand reliably. In this paper, we consider a utility with solar energy as a renewable resource, battery for storage, and interaction with the electrical grid. The objective is to develop a decision-making approach to help the utilities better manage their resources to decide when and how to dispatch battery, solar power, and how much electricity to buy/sell from the grid. Here, the management problem has been formulated as Markov Decision Process. We have implemented and compared the performance of a conventional approach of dynamic programming and a Q-learning approach to solve the optimization problem. The advantages and disadvantages of these approaches are also highlighted in this paper.