Markov Decision Process (MDP) has enormous applications in science, engineering, economics and management. Most of decision processes have Markov property and can be modeled as MDP. Reinforcement Learning (RL) is an approach to deal with Markov Decision Processes. RL methods are based on Dynamic Programming (DP) algorithms, such as Policy Evaluation, Policy Iteration and Value Iteration. In this paper, policy evaluation algorithm is represented in the form of a discrete-time dynamical system, namely a Discrete-Time Control system. Hence, using Discrete- Time Control methods, behavior of agent and properties of various policies, can be analyzed. Two grid-world problems are solved and analyzed using this approach. Therefore general case of gridworld problems is addressed, and some important results are obtained for this type of problems, For example, equivalent dynamical system of an optimal policy for a grid-world problem, is always a dead-beat system in the framework of Discrete-Time Control systems.

کلمات کلیدی