Robot vision-based control

Robot Vision-based control | Mechatronicsly

Approaches to robot vision-based control

There are several distinct approaches to vision-based control. These vary based primarily on system configuration and how image data is used.

Where to place the camera

Perhaps the first decision to be made when constructing a vision-based control system is where to place the camera. There are essentially two options: the camera can be mounted in a fixed location in the workspace, or it can be attached to the manipulator. These are often referred to as fixed camera vs.
eye-in-hand configurations, respectively.

With a fixed camera configuration, the camera is positioned so that it can observe the manipulator and any objects to be manipulated. There are several advantages to this approach. Since the camera position is fixed, the field of view does not change as the manipulator moves. The geometric relationship between the camera and the workspace is fixed, and can be calibrated off line.

A disadvantage to this approach is that as the manipulator moves through the workspace, it can occlude the camera’s field of view. This can be particularly important for tasks that require high precision. For example, if an insertion task is to be performed, it may be difficult to find a position from which the camera can view the entire insertion task without occlusion from the end effector.

With an eye-in-hand system, the camera is often attached to the manipulator above the wrist, i.e., the motion of the wrist does not affect the camera motion. In this way, the camera can observe the motion of the end effector at a fixed resolution and without occlusion as the manipulator moves through the workspace. One difficulty that confronts the eye-in-hand configuration is that the geometric relationship between the camera and the workspace changesas the manipulator moves.

The field of view can change drastically for even small motion of the manipulator, particularly if the link to which the camera is attached experiences a change in orientation. For example, a camera attached
to link three of an elbow manipulator (such as the one shown in Figure 3.1) will experience a significant change in field of view when joint 3 moves. For either configuration, motion of the manipulator will produce changes in the images obtained by the camera.

Using the image data

There are two basic ways to approach the problem of vision-based control, and these are distinguished by the way in the data provided by the vision system is used. These two approaches can also be combined in various ways to yield what are known as partitioned control schemes.

The first approach to vision-based control is known as position-based visual servo control. With this approach, the vision data is used to build a partial 3D representation of the world. For example, if the task is to grasp an object, the perspective projection equations can be solved to determine the 3D coordinates of the grasp points relative to the camera coordinate frame.

If these 3D coordinates can be obtained in real time, then they can be provided as set points to the robot controller, and many control techniques can be used. The main difficulties with position-based methods are related to the difficulty of building the 3D representation in real time.

In particular, these methods tend to not be robust with respect to errors in calibration. Furthermore, with position-based methods, there is no direct control over the image itself. Therefore, a common problem with position-based methods is that camera motion can cause the object of interest to leave the camera field of view.

A second method known as image-based visual servo control directly uses the image data to control the robot motion. An error funtion is defined in terms of quantities that can be directly measured in an image (e.g., image coordinates of points, the orientation of lines in an image), and a control law is constructed that maps this error directly to robot motion.

To date, the most common approach has been to use easily detected points on an object as feature points. The error function is then the vector difference between the desired and measured locations of these points in the image. Typically, relatively simple control laws are used to map the image error to robot motion.

Finally, these two approaches can be combined by using position-based methods to control certain degrees of freedom of the robot motion and using image-based methods to control the remaining degrees of freedom. Such methods essentially partition the set of degrees of freedom into disjoint sets, and are
thus known as partitioned methods.

A very good engineering reference is: A Vision-Based Formation Control Framework