Adaptive PID Controller based on Reinforcement Learning for Wind Turbine Control

Adaptive PID Controller based on Reinforcement Learning for Wind Turbine Control

Abstract

A self tuning PID control strategy using reinforcement learning is proposed in this paper to deal with the control of wind energy conversion systems (WECS). Actor-Critic learning is used to tune PID parameters in an adaptive way by taking advantage of he model-free and on-line learning properties of reinforcement learning effectively. In order to reduce the demand of storage space and to improve the learning efficiency, a single RBF neural network is used to approximate the policy function of Actor and the value function of Critic simultaneously. The inputs of RBF network are the system error, as well as the first and the second-order differences of error. The Actor can realize the mapping from the system state to PID parameters, while the Critic evaluates the outputs of the Actor and produces TD error. Based on TD error performance index and gradient descent method, the updating rules of RBF kernel function and network weights were given. Simulation results show that the proposed controller is efficient for WECS and it is perfectly adaptable and strongly robust, which is better than that of a conventional PID controller.

INTRODUCTION

As a result of increasing environmental concerns, the impact of conventional electricity generation on the environment is being minimized and efforts are made to generate electricity from renewable sources. The main advantages of electricity generation from renewable sources are the absence of harmful emissions and the infinite availability of the prime mover that is converted into electricity. One way of generating electricity from renewable sources is to use wind turbines that convert the energy contained in flowing air into electricity. Various electromechanical schemes for generating electricity from the wind have been suggested, but the main drawback is that the resulting system is highly nonlinear, and thus a nonlinear control strategy is required to place the system in its optimal generation point.

Different intelligent approaches have successfully been applied to identify and nonlinearly control the WECS and other plants. For instance, Kanellos and Hatziargyriou [1], Yong-tong and Cheng-zhi [2] and Zhao-da et al [3] have suggested neural networks as powerful building blocks for nonlinear control strategies. The most famous topologies for this purpose are multilayer perceptron (MLP) and radial basis function (RBF) networks [4]. Mayosky and Cancelo [5] proposed a neural-network-based structure for Wind turbine control that consists of two combined control actions, a supervisory control and an RBF etwork-based adaptive controller. Sedighizadeh et al [6,7,8] suggested an adaptive controller using neural network frame Morlet wavelets together with an adaptive PI controller using RASP1 wavenets for Wind turbine control. In this paper, the reinforcement learning is used to design of controller. This learning method unlike supervised learning of neural network adopts a ‘trial and error’ mechanism existing in human and animal learning. This method emphasizes that an agent can learn to obtain a goal from interactions with the environment. At first, a reinforcement learning agent exploits the environment actively and then evaluates the exploitation results, based on which controller is modified. It can realize unsupervised on-line learning without a system model [9-10]. Actor-Critic learning proposed by Barto et al is one of the most important reinforcement learning methods, which provides a working method of finding the optimal action and the expected value simultaneously [11]. Actor-Critic learning is widely used in artificial intelligence, robot planning and control, optimization and scheduling fields. Based on this analysis, in this paper a new adaptive PID controller based on reinforcement learning for WECS control is proposed. PID parameters are tuned on-line and adaptively by using the Actor-Critic learning method, which can solve the deficiency of realizing effective control for complex and time-varying systems by conventional PID controllers. The next section presents details of the wind energy
conversion system in this simulation. Section III describes the adaptive network algorithmic implementation. Then, the section IV introduces controller design steps. After that, the section V presents the simulation results and finally, the section VI explains conclusion.

Download full seminar papers At
http://www.enjineer.com/forum