Abstract: A method of constraint and optimization control on the similar stepping-upstairs process for humanoid robots is proposed to solve the problem that the fixed tracks captured from the human locomotion can't be used in the changeable target environment in the similar motion planning of humanoid robots. Firstly, the similar stepping-upstairs processes are divided into the sub phases and key poses, and the corresponding kinematics constraints are set as well. Secondly, the dynamics constraints are also set for the locomotion stability, walking direction and landing-buffer process. Thirdly, the HRPSO (particle swarm optimization with hierarchical reinforcement learning) algorithm is proposed to optimize the tracks of stepping upstairs for humanoid robots. The experiment proves the validity of the proposed method.