Abstract: This paper presents a direct image adaptive control scheme for visual tracking of moving target. It is different from feature based visual servo. The manipulating process is as follows: first of all, move the arm to the desired manipulating position and orientation with respect to the object of interest then sample the image of the object by eye-in-hand camera. In the manipulating process, the scheme, which compares the real time sampled-image with the desired one, forms the image error. Then the control action composed of image error feedback and adaptive motion compensation-steers the gripper to track the moving target in order that the image error is eliminated. If the manipulating-process does not satisfy very well the initial error condition, the control law can be used for to track the moving target with accurate and robust properties.