Automatica, ( ISI ), Volume (50), Year (2014-9) , Pages (1167-1175)

Title : ( Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics )

Authors: Bahareh Kiumarsi Khomartash , Frank L. Lewis , Hamidreza Modares , Ali Karimpour , Mohammad Bagher Naghibi Sistani ,

Access to full-text not allowed by authors

Citation: BibTeX | EndNote

Abstract

In this paper, a novel approach based on the Q-learning algorithm is proposed to solve the infinite-horizon linear quadratic tracker (LQT) for unknown discrete-time systems in a causal manner. It is assumed that the reference trajectory is generated by a linear command generator system. An augmented system composed of the original system and the command generator is constructed and it is shown that the value function for the LQT is quadratic in terms of the state of the augmented system. Using the quadratic structure of the value function, a Bellman equation and an augmented algebraic Riccati equation (ARE) for solving the LQT are derived. In contrast to the standard solution of the LQT, which requires the solution of an ARE and a noncausal difference equation simultaneously, in the proposed method the optimal control input is obtained by only solving an augmented ARE. A Q-learning algorithm is developed to solve online the augmented ARE without any knowledge about the system dynamics or the command generator. Convergence to the optimal solution is shown. A simulation example is used to verify the effectiveness of the proposed control scheme.

Keywords

, Linear quadratic tracker, Reinforcement learning, Policy iteration, Algebraic Riccati equation
برای دانلود از شناسه و رمز عبور پرتال پویا استفاده کنید.

@article{paperid:1049612,
author = {Kiumarsi Khomartash, Bahareh and Frank L. Lewis and Modares, Hamidreza and Karimpour, Ali and Naghibi Sistani, Mohammad Bagher},
title = {Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics},
journal = {Automatica},
year = {2014},
volume = {50},
month = {September},
issn = {0005-1098},
pages = {1167--1175},
numpages = {8},
keywords = {Linear quadratic tracker; Reinforcement learning; Policy iteration; Algebraic Riccati equation},
}

[Download]

%0 Journal Article
%T Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics
%A Kiumarsi Khomartash, Bahareh
%A Frank L. Lewis
%A Modares, Hamidreza
%A Karimpour, Ali
%A Naghibi Sistani, Mohammad Bagher
%J Automatica
%@ 0005-1098
%D 2014

[Download]