Abstract: In order to satisfy the requirements of UAV's aerial safety monitoring and surveillance of sensitive areas, a robust vision system for the rotor UAV is designed and implemented, which includes visual airborne subsystem, ground station subsystem and wireless communication subsystem. Complete sky-ground and human-computer interaction loops are constructed. Based on the developed UAV vision platform, a real-time target tracking algorithm under the mean shift tracking framework is developed. The joint color-texture histogram is used to represent the target robustly. With the help of moment information, the scale and the orientation of the tracked target is estimated adaptively during the tracking process. A model updating scheme for the target and the background is introduced to reduce the interferences from background and the locating biases. Numerical simulation and real flight tracking experiments demonstrate that the overall visual tracking system is effective and has superior performance against several state-of-the-art algorithms.