In a real electricity market, complete information of rivals’ behavior is not available to market participants. Therefore, they make their bidding strategies based on the historical information of the market clearing price. In this paper, a new market simulator is introduced for a joint energy and spinning reserve market, in which market participants’ learning process is modeled using Q-learning algorithm. The main feature of this simulator is simulating a real market, in which market participants make decisions based on incomplete information of the market. Using the proposed simulator, the clearing price for each submarket is computed considering the participants’ behavior, under different load levels and/or contingency conditions. The results show that Q-learning approach can modify the agent’s strategy under different market situations.

Keywords