Abstract: A method of dynamic hand gesture recognition based on SURF(speeded up robust feature) tracking is proposed. The main characteristic is that the hand trajectory is described only by tracking the dominant movement directions of matched SURF points in adjacent frames with no need of the previous detection and segmentation of the hand region.The dynamic hand gesture is then modeled by a series of trajectory direction data streams after time warping.Accordingly,the data stream clustering method based on correlation analysis is developed to recognize a dynamic hand gesture and to speed up calculation. The proposed algorithm is tested on 26 alphabetic hand gestures and yields a satisfactory recognition rate,which is 87.1% on the training set and 84.6%on the testing set.Its successful application to the motion control of our self-developed robot Hunter also establishes the effectiveness of the approach.