Abstract: Traditional trajectory segmentation approaches for surgical robot are time consuming, low-accuracy and prone to over-segmentation. For those problems, a multi-modality surgical trajectory segmentation approach is proposed based on DCED-Net (densely-concatenated convolutional encoder-decoder network) feature extraction network. DCED-Net adopts an unsupervised approach and a densely-concatenated structure, and the time consuming manual annotation is not required. Therefore, the image information can be transferred more effectively between convolutional layers, and the quality of extracted features is improved. The kinematic data and video data obtained after feature extraction are input into a transition state clustering (TSC) model to get pre-segmentation results. To further improve the segmentation accuracy, a post-merger processing algorithm based on the similarity between trajectory segments is proposed. By measuring four similarity indicators between trajectory segments, including principal component analysis, mutual information, data center distance, and dynamic time warping, the segments with high similarity are iteratively merged to reduce the impact of over-segmentation. A lot of experiments on the public data set JIGSAWS show that the proposed approach can increase the segmentation accuracy by up to 48.4% and accelerate the segmentation speed by more than 6 times, compared with the classical trajectory segmentation and clustering methods.