Abstract: Based on the concept of semantic context of frame images, an adaptive online terrain inference strategy (TIBSC, terrain inference based on semantic) is proposed, which incorporates spatiotemporal information of different scenes. Firstly, feature vectors and terrain categories of close-field-of-view pixels of different scene images are extracted to construct the terrain sample candidate database. Secondly, samples with most similar semantic context to that of the distant-field-of-view region of current scene are selected from the terrain sample candidate database to further construct the terrain discriminant database of the current scene. Finally, based on the discriminant database of the current scene and the Bayesian rule, terrain categories of distant-field-of-view pixels of the current scene are inferred. Results of the simulation experiments based on database show that the optimal sample selection based on semantic distance criterion and the online sample expansion play the dominant role among all the factors influencing the inference accuracy of TIBSC. And the results also indicate that, TIBSC model outperforms other existing methods in the term of inference accuracy.