王德明, 颜熠, 周光亮, 李勇奇, 刘成菊, 林立民, 陈启军. 基于实例分割网络与迭代优化方法的3D视觉分拣系统[J]. 机器人, 2019, 41(5): 637-648. DOI: 10.13973/j.cnki.robot.180806
引用本文: 王德明, 颜熠, 周光亮, 李勇奇, 刘成菊, 林立民, 陈启军. 基于实例分割网络与迭代优化方法的3D视觉分拣系统[J]. 机器人, 2019, 41(5): 637-648. DOI: 10.13973/j.cnki.robot.180806
WANG Deming, YAN Yi, ZHOU Guangliang, LI Yongqi, LIU Chengju, LIN Limin, CHEN Qijun. 3D Vision-Based Picking System with Instance Segmentation Network and Iterative Optimization Method[J]. ROBOT, 2019, 41(5): 637-648. DOI: 10.13973/j.cnki.robot.180806
Citation: WANG Deming, YAN Yi, ZHOU Guangliang, LI Yongqi, LIU Chengju, LIN Limin, CHEN Qijun. 3D Vision-Based Picking System with Instance Segmentation Network and Iterative Optimization Method[J]. ROBOT, 2019, 41(5): 637-648. DOI: 10.13973/j.cnki.robot.180806

基于实例分割网络与迭代优化方法的3D视觉分拣系统

3D Vision-Based Picking System with Instance Segmentation Network and Iterative Optimization Method

  • 摘要: 针对工业上常见的弱纹理、散乱堆叠的物体的检测和位姿估计问题,提出了一种基于实例分割网络与迭代优化方法的工件识别抓取系统.该系统包括图像获取、目标检测和位姿估计3个模块.图像获取模块中,设计了一种对偶RGB-D相机结构,通过融合3张深度图像来获得更高质量的深度数据;目标检测模块对实例分割网络Mask R-CNN(region-based convolutional neural network)进行了改进,同时以彩色图像和包含3维信息的HHA(horizontal disparity,height above ground,angle with gravity)特征作为输入,并在其内部增加了STN(空间变换网络)模块,提升对弱纹理物体的分割性能,结合点云信息分割目标点云;在目标检测模块的基础上,位姿估计模块利用改进的4PCS(4-points congruent set)算法和ICP(迭代最近点)算法将分割出的点云和目标模型的点云进行匹配和位姿精修,得到最终位姿估计的结果,机器人根据此结果完成抓取动作.在自采工件数据集上和实际搭建的分拣系统上进行实验,结果表明,该抓取系统能够对不同形状、弱纹理、散乱堆叠的物体实现快速的目标识别和位姿估计,位置误差可达1 mm,角度误差可达1°,其性能可满足实际应用的要求.

     

    Abstract: A workpiece recognition and picking system based on instance segmentation network and iterative optimization method is proposed for object detection and pose estimation of scattered and stacked texture-less industrial objects. This system consists of three modules, including image acquisition module, target detection module and pose estimation module. In image acquisition module, a dual RGB-D (RGB-depth) camera structure is designed to get higher quality depth data by merging three depth images. The target detection module modifies the instance segmentation network Mask R-CNN (region-based convolutional neural network). The modified network takes RGB images and HHA (horizontal disparity, height above ground, angle with gravity) features containing three-dimensional information as input, and adds STN (spatial transformer network) modules inside to improve the segmentation performance of texture-less objects. Then the module can combine point cloud information to obtain the target point cloud. On this basis, the improved 4PCS (4-points congruent set) algorithm and ICP (iterative closest point) algorithm are used in pose estimation module to match the segmented point cloud with the target model and fine-tune the pose, and thus the final result of pose estimation is obtained. The robots accomplish picking action according to the estimated pose. The experiment results on our workpiece dataset and the actual picking system indicate that the proposed method can achieve fast target recognition and pose estimation for scattered and stacked objects with different shapes and less textures. Its performance can meet the requirements of practical applications with 1 mm position error and 1° angle error.

     

/

返回文章
返回