In order to let the mechanical arm to accurately capture targets on the assembly line,the target accurately track become one of the key requirements. This paper presents a real-time tracking the target and direction positioning method. By this method mechanical arm can more effectively, accurately and quickly response to grab target.The RGB image is processed to get the accurate contour and selecting appropriate goals,meanwhile least squares ellipse fitting is used to determine the target bearing.Then extract the target center coordinates, using Mean-shift algorithm for real-time tracking. In the trace, only analyze the selected target ignoring others can effectively improve the tracking speed of the program. The method can better satisfy the requirement of real-time tracking and robustness,also can make control system of mechanical arm to feedback in timely, and effectively improve the efficiency of mechanical arm to grab.