WANG Deming, YAN Yi, ZHOU Guangliang, LI Yongqi, LIU Chengju, LIN Limin, CHEN Qijun. 3D Vision-Based Picking System with Instance Segmentation Network and Iterative Optimization Method[J]. ROBOT, 2019, 41(5): 637-648. DOI: 10.13973/j.cnki.robot.180806
Citation: WANG Deming, YAN Yi, ZHOU Guangliang, LI Yongqi, LIU Chengju, LIN Limin, CHEN Qijun. 3D Vision-Based Picking System with Instance Segmentation Network and Iterative Optimization Method[J]. ROBOT, 2019, 41(5): 637-648. DOI: 10.13973/j.cnki.robot.180806

3D Vision-Based Picking System with Instance Segmentation Network and Iterative Optimization Method

  • A workpiece recognition and picking system based on instance segmentation network and iterative optimization method is proposed for object detection and pose estimation of scattered and stacked texture-less industrial objects. This system consists of three modules, including image acquisition module, target detection module and pose estimation module. In image acquisition module, a dual RGB-D (RGB-depth) camera structure is designed to get higher quality depth data by merging three depth images. The target detection module modifies the instance segmentation network Mask R-CNN (region-based convolutional neural network). The modified network takes RGB images and HHA (horizontal disparity, height above ground, angle with gravity) features containing three-dimensional information as input, and adds STN (spatial transformer network) modules inside to improve the segmentation performance of texture-less objects. Then the module can combine point cloud information to obtain the target point cloud. On this basis, the improved 4PCS (4-points congruent set) algorithm and ICP (iterative closest point) algorithm are used in pose estimation module to match the segmented point cloud with the target model and fine-tune the pose, and thus the final result of pose estimation is obtained. The robots accomplish picking action according to the estimated pose. The experiment results on our workpiece dataset and the actual picking system indicate that the proposed method can achieve fast target recognition and pose estimation for scattered and stacked objects with different shapes and less textures. Its performance can meet the requirements of practical applications with 1 mm position error and 1° angle error.
  • loading

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return