Abstract: A distributed kernel-based reinforcement learning method is proposed to optimize the multi-robot formation control.Firstly,the basic formation control is realized based on a distributed leader-follower strategy by adding a virtualleader -robot.Secondly,a kernel-based reinforcement learning method,which combines the least squares policy iteration with the least squares policy evaluation,is proposed.The kernel-based least squares policy iteration method is used to obtain an initial formation optimal control policy offline,and then the kernel-based least squares policy evaluation method is used to optimize the control policy online.Finally,the experimental results for formation control show that the proposed method can optimize the control policy adaptively and improve the multi-robot formation control performance.