Abstract—In this paper a partially model-free reinforcement learning (RL) algorithm based on experience replay is developed for finding online the Nash equilibrium solution of the multi-player nonzero-sum (NZS) differential games. In order to avoid the performance degradation or even system instability, the amplitude limitation on the control inputs is considered in the design procedure. The proposed algorithm is implemented on actor-critic structure for every player in the game, where both actor and critic networks are tuned at the same time. The game players learn online the solution of the constrained coupled Hamilton-Jacobi (HJ) equations, without using any knowledge on the internal system dynamics. The idea of experience replay is used to relax the requirement for checking the restrictive persistence of excitation (PE) condition which is difficult to verify or implement online. The closed-loop stability is analyzed and the convergence to the Nash equilibrium of the game is shown. A simulation study example is provided showing the effectiveness of the proposed approach.

Keywords