AIAA Guidance, Navigation, and Control (GNC) Conference , 2013-08-19

Title : ( Adaptive Optimal Control of Partially-unknown Constrained-input Systems using Policy Iteration with Experience Replay )

Authors: Hamidreza Modares , Mohammad Bagher Naghibi Sistani , Frank L. Lewis , Girish Chowdhary , Tansel Yucelen ,

Access to full-text not allowed by authors

Citation: BibTeX | EndNote

Abstract

This paper develops an online learning algorithm to find optimal control solutions for partially-unknown continuous-time systems subject to input constraints. The input constraints are encoded into the optimal control problem through a nonquadratic performance functional. An online policy iteration algorithm that uses integral reinforcement knowledge is developed to learn the solution to the optimal control problem online without knowing the full dynamics model. The policy iteration algorithm is implemented on an actor-critic structure, where two neural network approximators are tuned online and simultaneously to generate the optimal control law. A novel technique based on experience replay is introduced to retain past data in updating the neural network weights. This uses the recorded data concurrently with current data for adaptation of the critic neural network weights. Concurrent learning provides an easy-to-check real-time condition for persistence of excitation that is sufficient to guarantee convergence to a near optimal control law. Stability of the proposed feedback control law is shown and its performance is evaluated through simulations.

Keywords

, optimal control, reinforcement learning, policy iteration, partially-unknown continuous-time systems
برای دانلود از شناسه و رمز عبور پرتال پویا استفاده کنید.

@inproceedings{paperid:1040141,
author = {Modares, Hamidreza and Naghibi Sistani, Mohammad Bagher and Frank L. Lewis and Girish Chowdhary and Tansel Yucelen},
title = {Adaptive Optimal Control of Partially-unknown Constrained-input Systems using Policy Iteration with Experience Replay},
booktitle = {AIAA Guidance, Navigation, and Control (GNC) Conference},
year = {2013},
location = {Boston, Massachusetts, USA},
keywords = {optimal control; reinforcement learning; policy iteration; partially-unknown continuous-time systems},
}

[Download]

%0 Conference Proceedings
%T Adaptive Optimal Control of Partially-unknown Constrained-input Systems using Policy Iteration with Experience Replay
%A Modares, Hamidreza
%A Naghibi Sistani, Mohammad Bagher
%A Frank L. Lewis
%A Girish Chowdhary
%A Tansel Yucelen
%J AIAA Guidance, Navigation, and Control (GNC) Conference
%D 2013

[Download]