An End-to-End Mapless Navigation Method Based on TS-TD3 in Dynamic Environment

Aiming at the problems of map-based mobile robot navigation framework deployed in dynamic complex environment, a mapless navigation method based on TS-TD3(time series twin delayed deep deterministic policy gradient) is proposed. Firstly, navigation tasks in dynamic scenarios(usually with partially observable environment) are defined as partially observable Markov decision process(POMDP). Secondly, the historical information processed by the long short-term memory components is introduced as the input of the model. The historical information baseline is introduced into the deterministic policy gradient of the actor network to process the state information hidden in the environmental observation set.The critic criteria concerned with the temporal correlation of navigation actions is introduced into the critic network. Thirdly, the expert experience network is used to guide the output of the actor network in the early stage of training to standardize the navigation actions. Finally, the deep reinforcement learning(DRL) based end-to-end model of the actor-critic framework is established, and the actions are controlled directly according to the sensor perception. Compared with the mainstream DRL methods, the motion trajectory obtained by the proposed method is natural, stable and continuous in the simulation experiment, the intersection of multiple dynamic obstacles can be dealed with, and the overall navigation performance is optimal.In the test in real dynamic environment, the model is directly deployed in an unknown environment without adjustment, and the navigation effect and generalization of the model are verified.