E. de Weerdt, Q.P. Chu, and J.A. Mulder (The Netherlands)
Reconfigurable control; Reinforcement Learning; Neu ral networks; Newton-Gauss; Parameter identifiability; Recency-effect
The field of reconfigurable control has become more and more active in the last few years. Many control strategies for constructing an autonomous adapting control system have been developed in the past. However, many of those strategies require a Failure Detection and Isolation (FDI) system such that the control laws can be adapted properly. The major drawback of an FDI system is that the designer needs to foresee all possible failure scenarios, something virtually impossible for even simple systems. In this paper, a reconfigurable control system based on Reinforcement Learning (RL) is proposed. Reinforcement Learning does not require a FDI system, which makes it ideal for reconfig urable control and for controlling unknown plants. Neural networks are used to solve the ’curse of dimensionality’ inherent to a RL controller. A novel method of batch up dating in combination with a new training algorithm based on the Newton-Gauss method are applied to solve two ma jor problems of neural networks, i.e. the ’recency’-effect and the identifiability problem. Closely related to the iden tifiability problem is the optimization of the neural network structure. To guarantee the optimal amount of network pa rameters an optimization algorithm is proposed. The tech niques are implemented on a simple system identification and a pole balancing task. Experiments show that the use of ‘anti-recency’ points circumvent the recency-effect in case of system identification and speed up stabilization for the RL controller.
Important Links:
Go Back
