Sequence-enhancement-based Human Detection and Posture Recognition of Mobile Robots in Low Illumination Scenes

Low illumination or even complete darkness is a very serious problem for lots of application scenes, such as disaster relief and underground space development, which brings a challenge to target search and recognition of robot. So a method for human detection and posture recognition in low illumination scenes using the image sequences collected by an infrared depth camera is proposed. Firstly, the AlphaPose algorithm based on YOLO v4 to detect human bounding boxes and key points is used. Then, a method to recover the missed human bounding box based on feature matching is proposed to reduce the missing detection rate. Meanwhile, the D-S (Dempster-Shafer) evidence theory is used to fuse the detection results of human bounding boxes and key points, in order to reduce the detection error rate. Finally, a sequence-based hierarchical recognition method to classify the human postures is designed, which extracts the torso features of human body and uses the sequential torso features in multiple frames to recognize the human posture accurately. Experimental results demonstrate that the proposed method can achieve good performance of human detection and posture recognition in low illumination scenes, and the accuracy of posture recognition can reach 95.36%.