Abstract: In order to improve target detection speed and accuracy, simplify particle filter based target tracking algorithm for histogram calculation, and improve the speed of detection and tracking algorithm in the DSP (digital signal processor)- based active vision systems, a DSP-based active vision system is proposed. By improved the EMCV (Embedded Computer Vision Library) and heuristic search methods, the system implements the AdaBoost detection algorithm on the DSP. And in that system, the color histogram and edge orientation histogram in particle filter are calculated by using the incremental histogram calculation algorithm, the histogram is integrated into the observation model, and the target tracking algorithm is optimized on the DSP. The experiment proves the rapidity and the robustness of the proposed algorithm in the active vision system.