A Multimodal Object Detection Method for Robots in Underground Environments

A lightweight dual-modal object detection method is proposed to address the dual challenges of low-light interference and limited computational resources faced by robotic perception systems in complex underground environments. By constructing a dual-branch network architecture that fuses LiDAR point clouds with RGB images, multi-scale feature fusion is achieved at shallow, intermediate, and deep levels. In the proposed method, the StarFusion model, featuring element-wise multiplication to enhance cross-modal feature interaction, is introduced, and depthwise separable convolutions and channel compression strategies are adopted, collectively reducing the model parameters to 2.3 million. To overcome the bottleneck of algorithm validation, a low-light multimodal dataset is constructed, containing 4 categories of typical underground targets, with image brightness (25±8.3) and sharpness (18.6±6.9) that are significantly lower than those of conventional datasets. Experimental results demonstrate that the method achieves an mAP50 (mean average precision with intersection-over-union = 0.5) of 86.1% on the custom dataset, representing a 2.6% improvement over the baseline YOLOv8 model, while achieving an inference speed of 20 frames per second. Practical deployment on an exploration robot equipped with the Jetson Orin NX platform verifies that the dual-modal complementary mechanism effectively overcomes the perception limitations of singlesensor systems in low-light conditions, providing a reliable real-time environmental perception solution for autonomous underground operations.