Title : ( Undiscounted reinforcement learning for infinite-time optimal output tracking and disturbance rejection of discrete-time LTI systems with unknown dynamics )
Authors: Ali Amirparast , Seyed Kamal Hosseini Sani ,Access to full-text not allowed by authors
Abstract
This paper proposes a novel control structure to solve the infinite-time linear quadratic tracking (LQT) problem. The major challenge in the LQT problem is the boundedness issue of the cost function in an infinite time framework. In many studies, a discount factor is utilised to overcome the challenge. However, it can affect the stability of the closed-loop system and the steady-state error. This paper proposes an optimal control structure that guarantees zero steady-state error with bounded cost function without utilising the discount factor. The optimal gains of the proposed control structure are computed via model-based and model-free reinforcement learning (RL) algorithms. As a novelty in model-based RL algorithms, a model predictive RL algorithm is proposed to reduce the number of iterations in the learning phase. A model-free reinforcement learning algorithm is utilised to obtain optimal control for tracking the reference online and without any knowledge of system dynamics. Finally, the simulation results verify the advantages of the proposed optimal control structure.
Keywords
, Linear quadratic, tracking optimal, control reinforcement learning, policy iteration@article{paperid:1094722,
author = {Amirparast, Ali and Hosseini Sani, Seyed Kamal},
title = {Undiscounted reinforcement learning for infinite-time optimal output tracking and disturbance rejection of discrete-time LTI systems with unknown dynamics},
journal = {International Journal of Systems Science},
year = {2023},
volume = {54},
number = {10},
month = {July},
issn = {0020-7721},
pages = {2175--2195},
numpages = {20},
keywords = {Linear quadratic- tracking optimal- control reinforcement learning-policy iteration},
}
%0 Journal Article
%T Undiscounted reinforcement learning for infinite-time optimal output tracking and disturbance rejection of discrete-time LTI systems with unknown dynamics
%A Amirparast, Ali
%A Hosseini Sani, Seyed Kamal
%J International Journal of Systems Science
%@ 0020-7721
%D 2023