Abstract: Existing methods based on appearance models cannot track targets correctly when illumination varies or occlusion occurs. To solve the problem, considering SIFT (scale-invariant feature transform) feature invariabilities for illumination, scale and affine, a new method is proposed in which target model is constructed by SIFT feature and particle filter is used to track object. In tracking process, the target model is updated automatically according to the matching result between target model and candidate targets in time window. As a result, the target model can adapt well to appearance variation. Simulation results show that the proposed method is robust in various scenes.