Abstract The vision based hand posture recognition, which can benefit natural human robot interaction, is an important part of human robot interaction. As an important channel of human robot interaction, hand posture recognition and other channel (for example, audio recognition) can constitute multi modal human robot interaction system. The main challenges in vision based hand posture recognition are the large variety of existing postures, 2D appearance depending on the camera viewpoint, degrees of freedom and so on. There are some key technologies in the field of hand posture recognition, for example, hand detection/segmentation, hand posture recognition. This paper presents the review of achievements and progress about some key technologies of hand posture recognition and multi modal human robot interaction according to the recently published literatures, and points out the critical problems in the research in detail and the key challenging problems and some future research directions.