A Visualization System for AFM-Based Nano-Manipulation on Soft Samples
WANG Chao1,2, WU Yinan1,2, FANG Yongchun1,2, FAN Zhi1,2, LIU Cunhuan1,2
1. Institute of Robotics and Automatic Information System, College of Artificial Intelligence, Nankai University, Tianjin 300350, China; 2. Tianjin Key Laboratory of Intelligent Robotics, Tianjin 300350, China
王超, 武毅男, 方勇纯, 樊志, 刘存桓. 面向柔软样品的AFM纳米操作可视化系统[J]. 机器人, 2021, 43(3): 331-339.DOI: 10.13973/j.cnki.robot.200205.
WANG Chao, WU Yinan, FANG Yongchun, FAN Zhi, LIU Cunhuan. A Visualization System for AFM-Based Nano-Manipulation on Soft Samples. ROBOT, 2021, 43(3): 331-339. DOI: 10.13973/j.cnki.robot.200205.
摘要在对细胞、生物大分子等柔软样品进行纳米操作时，原子力显微镜（atomic force microscope，AFM）面临缺乏实时视觉反馈的问题．为此，搭建了一套面向柔软样品的AFM纳米操作可视化系统．具体而言，首先建立了AFM形貌图像坐标系到虚拟场景坐标系之间的映射关系，从而得到虚拟场景中样品的顶点信息，进而通过3维图形引擎渲染样品的虚拟形貌．在此基础上，提出了一种基于接触力学理论的样品形变估计和仿真方法，对探针按压导致的样品形变进行了虚拟视觉反馈，从而使得刻画的虚拟形貌能够和样品的真实形貌保持一致，并准确地还原按压过程中样品表面的形貌变化．仿真和实验结果表明，所设计的纳米操作可视化系统能够在虚拟场景中实时呈现AFM纳米操作过程．
Abstract：When nano-manipulation is performed on soft samples, such as cells and biological macromolecules, an atomic force microscope (AFM) faces the drawback of lacking of real-time visual feedback. To address this issue, a visualization system for AFM-based nano-manipulation on soft samples is constructed. Specifically, the mapping relationship between the AFM topography image coordinate system and the virtual scene coordinate system is established firstly to obtain the vertex information of the sample in the virtual scene, so as to construct the virtual topography via the render of a 3-dimensional graphics engine. On this basis, a sample deformation estimation and simulation algorithm is developed based on the theory of contact mechanics, and virtual visual feedback is provided for the sample deformation caused by the probe pressing, so that the depicted virtual topography can be kept consistent with the real topography of the sample, and the topography changes on the sample surface during the pressing process can be accurately restored. The simulation and experimental results demonstrate that the designed nano-manipulation visual system can present the AFM-based nano-manipulation process in real time in the virtual scene.
[1] Binnig G, Quate C F, Gerber C. Atomic force microscope[M]//Perspectives in Condensed Matter Physics, Vol.6. Berlin, Germany: Springer, 1986: 55-58. [2] Krieg M, Flaschner G, Alsteens D, et al. Atomic force microscopy-based mechanobiology[J]. Nature Reviews: Physics, 2019, 1: 41-57. [3] Li M, Xi N, Wang Y C, et al. Advances in atomic force microscopy for single-cell analysis[J]. Nano Research, 2019, 12: 703-718. [4] Moreno-Moreno M, Ares P, Moreno C, et al. AFM manipulation of gold nanowires to build electrical circuits[J]. Nano Letters, 2019, 19(8): 5459-5468. [5] 李龙海，刘连庆，董再励. 面向可控自组装的DNA纳米管可编程AFM操作[J].微纳电子技术， 2015， 52(4)： 240-245. Li L H, Liu L Q, Dong Z L. Programmable AFM manipulation for DNA nanotubes with the controlled self-assembly[J]. Micronanoelectronic Technology, 2015, 52(4): 240-245. [6] Helle S C J, Feng Q, Aebersold M J, et al. Mechanical force induces mitochondrial fission[J]. eLife, 2017. DOI: 10. 7554/eLife.30292.001. [7] Guillaume-Gentil O, Grindberg R V, Kooger R, et al. Tunable single-cell extraction for molecular analyses[J]. Cell, 2016, 166(2): 506-516. [8] Xie H, Rgnier S. High-efficiency automated nanomanipulation with parallel imaging/manipulation force microscopy[J]. IEEE Transactions on Nanotechnology, 2012, 11(1): 21-33. [9] Shen Y J, Nakajima M, Zhang Z H, et al. Dynamic force characterization microscopy based on integrated nanorobotic AFM and SEM system for detachment process study[J]. IEEE/ASME Transactions on Mechatronics, 2015, 20(6): 3009-3017. [10] Shi Q, Yang Z, Guo Y N, et al. A vision-based automated manipulation system for the pick-up of carbon nanotubes[J]. IEEE/ASME Transactions on Mechatronics, 2017, 22(2): 845- 854. [11] Li G Y, Xi N, Chen H P, et al. “Videolized” atomic force microscopy for interactive nanomanipulation and nanoassembly[J]. IEEE Transactions on Nanotechnology, 2005, 4(5): 605- 615. [12] Korayem M H, Esmaeilzadehha S. Virtual reality interface for nano-manipulation based on enhanced images[J]. The International Journal of Advanced Manufacturing Technology, 2012, 63: 1153-1166. [13] Korayem M H, Hoshiar A K, Ghofrani M. Comprehensive modelling and simulation of cylindrical nanoparticles manipulation by using a virtual reality environment[J]. Journal of Molecular Graphics and Modelling, 2017, 75: 266-276. [14] Onal C D, Sitti M. Teleoperated 3-D force feedback from the nanoscale with an atomic force microscope[J]. IEEE Transactions on Nanotechnology, 2010, 9(1): 46-54. [15] Li G Y, Xi N, Yu M M, et al. Development of augmented reality system for AFM-based nanomanipulation[J]. IEEE/ASME Transactions on Mechatronics, 2004, 9(2): 358-365. [16] Vogl W, Ma B K L, Sitti M. Augmented reality user interface for an atomic force microscope-based nanorobotic system[J]. IEEE Transactions on Nanotechnology, 2006, 5(4): 397-406. [17] Wu Y N, Fang Y C, Wang C, et al. An optimized scanning based AFM fast imaging method[J]. IEEE/ASME Transactions on Mechatronics, 2020, 25(2): 535-546. [18] Sunday D. Fast polygon area and Newell normal computation[J]. Journal of Graphics Tools, 2002, 7(2): 9-13. [19] Sneddon I N. The relation between load and penetration in the axisymmetric Boussinesq problem for a punch of arbitrary profile[J]. International Journal of Engineering Science, 1965, 3(1): 47-57.
