Abstract: A road segmentation model for fusing image and point cloud based on fusion via hierarchical CRF (conditional random field) is proposed, to combine the multi-scale features of image and the spatial features of point cloud in road detection, so that the interferences of shadow and light in road scene can be removed by the detection algorithm efficiently. Firstly, the Meanshift algorithm is used to generate multi-scale super-pixel segmentation, and a multi-scale hierarchical CRF based on image is created. The point cloud data of Lidar is projected to the image plane, multi-scale hierarchical CRF based on point cloud is created. And a multi-scale fusion model is created by establishing connections between the pixel layer and the point cloud layer of CRF. Then, for every pixel layer and every point cloud layer in the multi-scale fusion model, the image features and the point cloud features of the corresponding scale are extracted. A gradient boosting tree classifier is trained for every layer based on the extracted features, and the data cost of the corresponding layer is obtained by the classifier of every layer. Finally, the α-expansion algorithm is adopted to optimize the fusion model jointly. The experimental results on KITTI Road dataset show that the proposed method obtains a good performance for road detection.