Jiahang Liu, Xin Xu, Zhenhua Huang, and Chuanqiang Lian
Reinforcement learning, multiple kernel learning, approximate dynamic programming, model-free, nonlinear systems
Reinforcement learning (RL) has become an important research topic to solve learning control problems of nonlinear dynamic systems. In RL, feature representation is a critical factor for improving the performance of online or offline learning controllers. Although multi-kernel learning has been studied in supervised learning problems, there is little work on multi-kernel-based feature representation in RL algorithms. In this paper, a model-free multi-kernel learning control (MMLC) approach is proposed for a class of nonlinear discrete-time systems. MMLC has advantages over other single- kernel-based RL algorithms in that the parameters in the kernel functions can be learned adaptively. Furthermore, MMLC uses a model-free actor–critic learning structure, where the critic is designed to approximate the derivatives of value functions. Different from the popularly studied dual heuristical programming algorithm, the proposed MMLC approach can learn the dynamics of the nonlinear system in a data-driven way. To evaluate the performance of MMLC, single-link and double-link inverted pendulums are employed as two benchmarks. The effectiveness of the MMLC algorithm has been demonstrated in simulation. It is shown that MMLC can achieve better performance than previous kernel-based dual heuristic programming with partial model information.
Important Links:
Go Back
