Abstract: Semantic segmentation of images is combined with simultaneous localization and mapping (SLAM) to create three-dimensional semantic map. Through ORB-SLAM, the input image sequences are screened to obtain key frames. Then, an improved semantic segmentation method based on DeepLab algorithm is proposed. The up-sampling convolutional network is added behind the last layer of original convolution network to improve the coarse sampling caused by bilinear interpolation. The depth of the key frame is used as gating signals to control the choice of different convolution operations, as a result, the small details are preserved for remote objects and larger receptive fields are preserved for near objects simultaneously. The segmented image is aligned with the depth map. Then, three-dimensional dense semantic map of the scene is formed by using the spatial correspondence between adjacent key frames. Experimental results show that the proposed algorithm, for indoor and outdoor scenes, can implement accurate semantic segmentation and create satisfactory semantic map by reverse projection in the three-dimensional space. Compared with existing methods based on DeepLab and deconvolution algorithms, the proposed algorithm can obtain better semantic map.