Abstract: The existing visual obstacle localization algorithms can't locate the obstacles moving out of sight, and there are noises in localization. For these problems, an obstacle memory localization and denoising algorithm based on binocular ORB-SLAM (simultaneous localization and mapping system based on the oriented BRIEF feature) is proposed. Based on the detection results of the obstacle, the algorithm projects the obstacle points to the ground grid through the SLAM (simultaneous localization and mapping) map frame-by-frame firstly, calculates the number of standard obstacle projection points in the grid, and removes the localization noises by the Otsu method. So, the accurate obstacle memory position is obtained finally. The experimental results show that the obstacle can still be localized after moving out of sight, the denoising success rate of a single obstacle is 95.3%, and the average processing speed reaches 8 key frames per second. It is proved that the algorithm can realize the obstacle memory localization, and has good denoising performance and real-time performance.