HUANG Zhong, REN Fuji, HU Min, LIU Juan. A Real-time Expression Mimicking Method for Humanoid Robot Based on Dual LSTM Fusion[J]. ROBOT, 2019, 41(2): 137-146. DOI: 10.13973/j.cnki.robot.180145
Citation: HUANG Zhong, REN Fuji, HU Min, LIU Juan. A Real-time Expression Mimicking Method for Humanoid Robot Based on Dual LSTM Fusion[J]. ROBOT, 2019, 41(2): 137-146. DOI: 10.13973/j.cnki.robot.180145

A Real-time Expression Mimicking Method for Humanoid Robot Based on Dual LSTM Fusion

  • To improve space-time similarity and motion smoothness in robot expression imitation, a real-time expression mimicking method for humanoid robot based on dual LSTM (long short-term memory) fusion is proposed by combining with the sequence to sequence deep learning model. In offline mechanical modeling phase, an inverse mechanical model is constructed firstly to fulfill the inverse mapping from facial feature sequence to motor control sequence, and a motion tendency model is further presented to wrap the smoothness of continuous motor motion. Secondly, a weighted objective function is addressed to implement the fusion of the above two models as well as the parameter optimization. In online expression transfer phase, the facial feature sequence of performer is regarded as the input of the fusion model, and the frame-to-frame expression mimicking of robot is achieved by means of the end-to-end translation from the performer facial feature sequence to robot control sequence under the optimal parameters. The experimental results indicate that the control deviations of the fusion model is lower than 8%, meanwhile, the space-time similarity and motion smoothness in expression mimicking is greater than 85%. Compared with related methods, the proposed method has a significant improvement in the control deviation, space-time similarity and motion smoothness.
  • loading

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return