 |
Description  |
|
|
BACKGROUND OF THE INVENTION
1. Field of the Invention
This invention relates to an apparatus for recognizing and approaching a
target in a space which is not directly visible, whereby a rotating camera
on the end of a boom of an industrial robot observes the environment of
the boom and sends images to a monitor at a remote control station.
2. Description of the Prior Art
Industrial robots are manually remote-controlled movement devices for
performing manipulation tasks. They are frequently used if the
manipulation tasks must be performed in areas which are not visible or
inaccessible to human beings, and may include, for example, the inside of
the pressure vessel of a radioactive nuclear reactor. In connection with
the present invention, the term "industrial robot" is also used to mean an
occasionally manually-controlled industrial robot (VDI Guideline 2860,
Sheet 1, 1982 Draft Edition, Paragraph 5.1.1).
It is known that industrial robots can be controlled by means of images
from cameras mounted on boom arms ("Industrial Robots", Pergamon Press,
Oxford, UK, Page 90, corresponding Japanese Patent No. 52.91265). However,
it appears that the type of camera movement and the method of controlling
the axes of the industrial robot was not disclosed therein.
The publication "Sensor Review", January 1983, page 23-26, also describes
an anthropomorphic robot with a camera located on the tool head parallel
to a gripper. However, no images are taught to be transmitted for a manual
remote control since the disclosure is, instead, directed to parallax-free
pattern recognition.
Great Britain Patent No. 2 087 107 describes an industrial robot of a
cylindrical type, which has a camera on the tool head. Using the
"teach-in" method by the industrial robot, a three-dimensional weld seam
can be produced, read in, and inspected by the camera. It does not appear
that a direct approach to a three-dimensional target is an object of the
patent.
U.S. Pat. No. 4,258,425 describes a cartesian-type robot, which can be
remotely controlled manually, on rails or in the playback mode. Although
the coordinates are displayed on a screen, there does not appear to be any
camera guidance or point-target control.
On a remote-controlled, off-road manipulator vehicle ("Kerntechnik", 1975,
Volume 12, Page 527-532), there are mono and stereo television cameras for
optical control and inspection on the vehicle in a pivoting and tilting
head. The camera position is not optimized for a precise, targeted
guidance of the tool head or gripper, since the manipulator arm or the
track might conceal the working positions of the gripper. It also fails to
include a combined multi-axis control of the gripper.
OBJECT OF THE INVENTION
It is therefore an object of the invention to provide a process and
apparatus for the recognition and linear approach to a target point by
means of a camera in any space not directly visible within the maximum
range of an industrial robot.
It is also an object, regardless of the type of robot in question, to
manually or automatically approach a target with the tool.
It is a further object to perform the control operations in the same
logical direction, forward, backward, up, down, left or right, as the
optical axis of the camera during the approach to the target.
SUMMARY OF THE INVENTION
These and other objects of the invention are provided in a preferred
embodiment thereof including an apparatus for recognizing and approaching
a target in a space with an industrial robot. The industrial robot is of a
type which has a camera on an end of a boom thereof. The camera is used to
observe an environment of the space including the target and to send
images thereof to a monitor at a remote control station. The preferred
apparatus includes the industrial robot having a plurality of axes having
degrees of freedom. The camera has a lens with an optical axis. The camera
and the optical axis thereof can be selectively rotated by 360 degrees
around a first axis of the plurality of axes and selectively pivoted by
180 degrees around a second axis of the plurality of axes which is
perpendicular to the first axis. The selective rotating and the selective
pivoting is used to move the camera and the optical axis thereof until the
target can be seen in the monitor and located at a target position
therein. The camera has a camera position relative to the optical axis
thereof and the optical axis has a target direction when the target is at
the target position in the monitor. The industrial robot can be moved
through the degrees of freedom for each of the axes of the plurality of
axes and includes the selective rotating and the selective pivoting. A
control system is used for controlling various movements of the industrial
robot. The control system includes sensing the camera position and the
target direction to the optical axis. The control system maintains the
camera position and the target direction during controlled movement of the
end of the boom along the target direction to the target. The controlled
movement by the control system includes moving a first group of the axes
of the plurality of axes while the axes of a second group of the plurality
of axes are limited to synchronous movement in direct response to the
controlled movement of the axis of the first group.
The apparatus according to the invention can be used to directly address
and approach any three-dimensional target and to follow the control
process on a monitor. The visual approach to a three-dimensional target by
means of an industrial robot has heretofore required an extraordinary
degree of concentration on the part of the operator since, with a
conventional camera system, there is no three-dimensional orientation.
This is particularly true for industrial robots. With industrial robots,
targets in a closed space must be approached by simultaneously moving the
robot system in three or more axes.
Such devices include industrial robots of the following types:
Type A Cartesian type with three translation axes,
Type B Anthropomorphic robot with exclusively rotational degrees of
freedom,
Type C Combination of a cartesian and a cylindrical robot, e.g. the one
described in German Laid Open Patent Appln. No. 31 42 500 and
corresponding U.S. Pat. No. 4,666,050,
Type D Combination of an anthropomorphic and a cylindrical robot, e.g. the
one described in German Laid Open Patent Appln. No. 35 06 146 and
corresponding European Patent No. 0 192 174 B1.
For all these types of robots, remotely-controlled operations are possible
on any three-dimensional target. Such operations include linear motion
toward any three-dimensional target. However, some of the industrial
robots must be controlled with more than three shafts, which could very
easily overtax the capabilities of an operator.
This is particularly true if some of the shafts to be controlled are
operated in synchronization with the other controlled shafts.
"Synchronization" is used to mean that either the shafts of the other
degrees of freedom, which are not controlled, remain stationary or are
moved in the same direction or in the opposite direction at the same
angular velocity.
With certain restrictions, the process of the present invention is also
suitable for polar or cylindrical types of robots, as long as the
three-dimensional target is not concealed by the robot body. Accordingly,
an assumption of direct visibility is a natural pre-requisite.
Tests have shown that a camera need only be rotated or pivoted in two axes
to completely scan a space until the three-dimensional target is
recognized. A target is "recognized" when it is in the center of the
monitor, so that it can then be approached in a straight line as a
function of the direction of the optical camera axis.
Since the zero positions of all the robot system axes are known, the
direction and also the motion can be precisely determined and defined by
angular and distance measurements. As a result, the corresponding
coordinates can be stored in real time in a computer so that, on the one
hand, the target can be calculated from this starting position and, on the
other hand, can be executed automatically if necessary for repetitions of
the movement sequence.
If a known or unknown object blocks the path in the direction of the
target, the operator can also manually control the industrial robot to
bypass the obstacle. For this reason, the current coordinates and
directions of movement of the tool head, on which the camera or a tool is
located, are displayed on the monitor.
According to the invention, even with repeated approaches to a
three-dimensional target, as might occur with different tools, the
operator has the ability to monitor the path of the tool. In such cases, a
camera film recorded on the occasion of the first approach may be used.
To allow the operator to concentrate completely on the work with the tool,
the invention makes it possible to store the first path or approach to the
three-dimensional target in the computer and to automatically configure
the next approach to the three-dimensional target with a tool so that the
operator need only execute the starting operation.
A constant orientation for the operator in all three dimensional axes is
particularly important for operations in a space which is not visible.
Such an orientation is even more important when sudden failures of the
industrial robot make manual intervention necessary or when unexpected
obstacles appear in the path during the approach to the three-dimensional
target and must be bypassed with the tool head. For this purpose,
according to the invention, each drive of the movable robot shafts is
coupled with a manual operation circuit, so that control movements of the
operator can be executed in a logically correct sequence, which may be
both simultaneous and parallel with the movements executed by the
equipment.
When working on a three-dimensional target, it is sometimes useful to
monitor the operations on the three-dimensional target with a camera from
another location, which does not lie in the same axis as the principal
axis of the tool head. With a camera in the same axis as that of the tool
head, the spatial separation between the tool tip and the
three-dimensional target cannot be recognized with any certainty. For this
purpose, such a camera would also be controlled from the remote-control
station by means of monitors.
In this regard, it is also possible to have a pivoting or rotating camera,
or one which can execute a translation movement, on the tool head, in
order for the camera to be outside the tool axis. Again, such a camera
should be controlled, according to the invention, from the monitor and
where any parallax errors which occur between the optical axis and the
principal tool axis can be corrected by the computer This is possible,
according to the invention, because all of the movements of the tool are
stored and the direction of the three-dimensional target is known.
This parallax error correction is not necessary if the camera, according to
the invention, can be controlled separately on the tool head, so that, by
rotating or pivoting the camera, the optical axis of the camera and the
axis of the tool intersect at the three-dimensional target.
Such a correction can refer either to the optical axis or to the principal
axis of the tool as the reference line. For example, for a gripper tool,
either the optical axis of the camera can be corrected or the position of
the gripper can be changed, so that the object to be viewed can be
directly and properly approached or, after the approach to the target, the
gripper can be correctly oriented in relation to the object.
Simple error corrections in the approach are possible, according to the
invention, if the automatic control can be overridden by a manual
operation or if, for the manual control of the industrial robot, the
automatic control can be temporarily deactivated.
For several reasons, the process can be advantageously simplified, so that
a collision of the tool or the camera or of the tool head, in general,
with another component can be avoided. When approaching the
three-dimensional target, the automatic approach can be deactivated at a
defined distance from the object. Such a process could be used for the
prevention of a collision with another robot component or another body in
the working space. In fact, it is also possible to use the same method,
for example, to maintain a pre-determined hole depth for a drilling tool.
The advantage of the process sequence according to the invention can also
be defined as follows:
On the one hand, because the operator receives optical information as if he
himself were at the tip of the tool head and, on the other hand, since all
the movements of the tool head he orders are executed in the logically
correct sequence which is in the same direction as his operating
movements, he can orient himself at any time in the space in which the
industrial robot is working.
The simplest case of a targeted search according to the invention and the
approach to a target in a rectangular space by means of a cartesian
industrial robot (Type A) is shown in Table I. Six axes are defined, the
three spatial axes X, Y and Z, plus, as the fourth axis, the full circle
axis for the camera (tool head) in the XY-plane and as the fifth axis, the
pivot axis for the camera (tool head) in the XZ-plane. For the target
search in the above-mentioned space, the camera need only be moved around
the fourth and fifth axes, until the optical axis of the camera has
acquired the target. To be able to approach the target, depending on the
position of the three-dimensional target, movement must be executed in
from one to three axes, i.e. the X and/or Y and/or Z axis of the robot.
This must be performed without the camera losing the target. The sixth
axis, which is for the tool itself, is not used.
TABLE I
______________________________________
Robot Axes
Function of axes with robot type:
Axis A B C D
______________________________________
1 Translation
Foot rotation
Turn ring Rotation
x-axis of bottom
flat joint
2 Translation
Pivot Translation of
Rotation
y-axis upper arm horizontal sleds
of bottom
diagonal joint
3 Translation
Pivot Translation of
Translation
z-axis lower arm vertical sleds
axis
4 Rotate tool
Rotate tool
Rotate Rotation
head head mast of top
diagonal joint
5 Pivot tool
Pivot tool Rotate tool
Rotation
head head head of top
flat joint
6 Rotate tool
Rotate tool
Pivot tool
Pivot tool
head head
7 -- -- Rotate tool
Rotate tool
______________________________________
On the other hand, the control task is significantly more complicated for a
Type C industrial robot. Accordingly, the primary explanation of the
invention will be directed to a Type C industrial robot similar to that
disclosed in U.S. Pat. No. 4,666,050. This patent, as well as the other
patents, patent applications, papers, articles, and publications listed
and discussed hereinabove are incorporated herein by reference as if the
entire contents thereof were fully set forth herein.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 includes a simplified, sectional side view of a Type C robot
including various features of the invention.
FIGS. 2-4 include schematic top views of the robot of FIG. 1 including the
relative positions of the various components and the process sequence
according to the invention for approaching a target at three different
points in time.
FIG. 5 includes a simplified view of an alternative embodiment including
various features of the invention.
FIG. 6 includes a schematic block diagram of the preferred apparatus
including various features of the invention.
DESCRIPTION OF THE PREFERRED EMBODIMENT
As in FIG. 1, a Type C industrial robot can be used for dismantling a
nuclear reactor vessel in a nuclear power plant. The industrial robot is
operated from a remote control station (which is not shown). Additionally,
for purposes of illustration, the drives and the power transmission
elements are not shown.
A slewing ring 1 with a diameter of 3460 mm is mounted so that it can
rotate by 360 degrees in the horizontal plane by means of bearings 2 in
the frame 3. The preferred ring 1 can be driven by toothed gears 4. On a
widened circular section of the ring 1 there are guide rails 5, which
extend like the chord of a circle and are better seen in FIGS. 2, 3 and 4.
A horizontally movable sled 6 is mounted on the guide rails 5 and may be
moved to the left or the right of the central location shown in FIG. 1.
The horizontally movable sled 6 includes a friction bearing 7 which
supports a vertical post 15. The post 15 can be rotated by 360 degrees
around its vertical axis and may be as long as 15000 mm. Fastened to the
post 15 is a vertical guide 13, in which a vertical support 16 slides up
and down.
A rotation mechanism 17 is located on an arm which extends from the
vertical support 16 and is provided for rotation of the tool head 14
around the vertical axis at the end of the extended arm. The extended arm
in FIG. 1 extends outwardly toward the viewer to separate the tool head 14
from the vertical support 16 by a fixed distance (a distance of about M-Z
as seen in FIGS. 2 through 4). A pivoting mechanism 8 can execute a
tipping movement of the tool head 14 relative to the vertical axis at the
end of the extended arm.
On the tip of the tool head 14, in a tool mounting, there is a camera 20
for the transmission of images to a monitor 42 at a remote control station
44, as seen in FIG. 6. This camera 20 can be replaced by a tool, such as a
drill.
FIGS. 2-4 are schematic top views showing the relative positions of the
various components of the robot of FIG. 1 to demonstrate the process
sequence at three different times during the approach to the target, as it
is qualitatively illustrated in Table II and quantitatively illustrated in
Table III. In each of the Tables II and III, the particular movement at
the axes 1 through 7 is identical to that described in Table I for the
C-type robot. The seventh axis (tool rotation) always remains in the idle
position, since while locating the target and during the approach to the
target, only the camera 20 (FIG. 1) is used. An X means that the
particular axis is being controlled, and a--means that the particular axis
is not being controlled. If an axis is being operated in synchronization
with a controlled axis, a corresponding comment will be included to
indicate such movement of that particular axis.
TABLE II
______________________________________
Target Search Target
Axis Movements Approach Approach
Target Axis Z 1-Z 2 Z 2-Z 3
______________________________________
1 -- -- X
2 -- X --
3 -- X X
4 -- X X
5 X synchronous with
synchronous with
Axis 4 Axis 1;4
6 X -- --
7 -- -- --
______________________________________
TABLE III
______________________________________
Axis Positions
Axis FIG. 2/Z 1 FIG. 3/Z 2 FIG. 4/Z 3
______________________________________
1 0 degrees 0 degrees
133 degrees
2 0 mm 825 mm 825 mm
3 -2000 mm -5826 mm -9000 mm
4 178.5 degrees 90 degrees
123 degrees
5 36.5 degrees 125 degrees
201 degrees
6 49.7 degrees 49.7 degrees
49.7 degrees
7 Idle Position at 0
______________________________________
From a starting point Z 1 (FIG. 2), a three-dimensional target is sought at
the target point Z 3 in order for it to eventually be approached. For the
target search phase (Table II), only the fifth axis for rotating the tool
head by a maximum 360 degrees and the sixth axis for pivoting the tool
head by a maximum 180 degrees need be moved, until the target point Z 3 is
acquired by the camera 20 (FIG. 1) from the starting point Z 1. At the
starting point Z 1, the arm extending from the vertical support 16 to the
tool head 14 extends to the side of the guide rail 5 by the rotation of
the vertical post 15 in that direction. Because the arm has a length,
generally M-Z, the starting point Z 1 is generally separated by that
length or distance from the axis of the post 15.
On the way from Z 1 to Z 2, which represents any randomly-selected
intermediate point on the line between Z 1 and Z 3, as shown in Table II,
only the second, third and fourth axes of the industrial robot need be
controlled. The fifth axis, which according to the present invention turns
in the opposite direction to the fourth axis at the same angular velocity
as the fourth axis, is controlled by the computer of a control system 48
as seen in FIG. 6 and discussed in detail hereinbelow. The optical axis of
the camera 20 remains rigidly fixed on the target point Z 3 throughout the
approach. During the further approach to the target from Z 2 to Z 3, at Z
2 the second axis remains stationary, and the first axis is moved together
with the third and fourth axis. The fifth axis, since it requires the
linear target approach, is controlled in synchronization by the control
system 48, sometimes with the first axis and sometimes with the fourth
axis.
FIG. 2 shows the instantaneous starting position Z 1 of the industrial
robot. In all the figures, the representation of the vertical movement
(third axis) has been omitted. In the zero position of the industrial
robot, a camera 20 (FIG. 1), such as a video camera, is mounted on the
tool head by means of an automatic tool change system (not shown), and
moved toward the starting position Z 1. The ring 1 is in the zero position
(Table III), which is designated R 1.
The line of movement 5' (corresponding to the path of the sled 6) of the
post center point 15' runs parallel to the guide 5. The translation zero
position M 1 of the post center point 15' is at the end of the guide rails
5. Initially, the post 15 is rotated by 178.5 degrees to its rotational
zero position to locate the end of the tool head 14 at the zero position
21. The distance M 1/Z 1, generally M-Z, which represents the length of
the arm from the vertical support 16 to the tool head 14, is constant.
The initial vertical position (third axis) is 2000 mm below the level of
the ring 1.
The translation axis movements are controlled by the control system 48
(FIG. 6) through a position sensor and the rotational axis movements are
controlled by the control system 48 through an angular position sensor.
Each of the sensors are respectively included in a sensor system 50 (FIG.
6). From the starting point Z 1, the target point Z 3 is determined by
rotating and pivoting the tool head 14. In the preferred example, the
target is then viewed at a rotational angle of 36.5 degrees (fifth axis)
and a pivot angle of 49.7 degrees (sixth axis). This direction of the
optical camera axis is maintained during the approach to the target, so
that, in other words, during the following movements it is operated in
synchronization with the other axes.
On the way to the three-dimensional target (FIG. 3), the camera 20 reaches
the intermediate point Z 2 by a simultaneous reverse rotation of the post
15 to 90 degrees, movement of the vertical support 16 to -5826 mm and a
horizontal movement of the sled 6 from M 1 through the turning point M U
to the position M 2 which is 825 mm from the zero position at M 1.
From the position of FIG. 3, the second axis (sled 6) is deactivated, and
the ring 1 is rotated out of the rest position R 1 until the ring position
R 3 at 133 degrees (FIG. 4) from the zero position is reached at the
target point Z 3.
Simultaneously, the vertical support 16 is lowered to -9000 mm and the post
15 is rotated continuously forward to the position 123 degrees.
The rotation mechanism 17 of the tool head 14 (FIG. 1), which is first
synchronized with the fourth axis, and later also with the first axis, is
thereby rotated from Z 1 at 36.5 degrees through Z 2 at 125 degrees to Z 3
at 201 degrees, while the pivoting mechanism 8 remains in a position of
49.7 degrees.
FIG. 5 shows an embodiment of an apparatus according to the invention for
an industrial robot in which the pivoting movement of the tool head, as
described above, is replaced. Such an adaptation can also be used to
replace or supplement the rotational movement of the tool head. The
apparatus can be used in particular if, in addition to the tool, for
example, a drilling machine 12, a viewing apparatus 21 (such as a camera)
is used to monitor the work at a target point Z 4.
A pin 22 of a tool head 14 (such as that of FIG. 1) has a mounting 18 at
right angles, to which an arc-shaped guide 19 for the viewing apparatus 21
is fastened.
Holes can be drilled at the target Z 4 by means of the drilling machine 12
which is fastened to the pin 22 and includes a vertically-movable drive
shaft 11 and drill head 10 to hold a drill 9.
The distance between the drill 9 and the target point Z 4 can be controlled
by means of a measurement apparatus (not shown), which simultaneously
functions as a collision warning device during the approach to the target
Z 4.
For the three-dimensional search, the required rotational movement of the
viewing apparatus 21 can be performed by rotation of the pin 22 with a
fixed mounting 18 or by rotation of the mounting 18 with a fixed pin 22.
The pivoting movement is executed by the viewing apparatus 21 along a
curved guide 19 powered by a drive mechanism (not shown). The curved guide
19 is designed so that the optical axis of the viewing apparatus 21 always
intersects the axis of the drill 9 at its working point which, in this
case, is the target point Z 4. The orientation of the viewing apparatus 21
takes place as described above with reference to FIG. 2, but the pivoting
movement is not executed by the tool head 14 but along the curved guide
19.
If necessary, a remotely-controlled rotation of the mounting 18 can point
the viewing apparatus 21 at the target point Z 4 from the opposite side of
the drill 9. The advantage of this arrangement is that the target search,
the optical monitoring, the approach to the target and the work at the
target point Z 4 can all be observed with the same viewing apparatus 21.
As seen in FIG. 6, the preferred invention includes an apparatus for
recognizing and approaching a target in a space with a boom of an
industrial robot 40 of the type discussed hereinabove which has a camera
20 on the end of the boom. The camera is used to observe the environment
of the space including the target and to send images thereof to a monitor
42 at a remote control station 44. The industrial robot 40 has a plurality
of axes having degrees of freedom as discussed hereinabove. The camera may
be positioned on the tool head, such as tool head 14, at the end of the
boom of the industrial robot 40. The camera 20 includes a lens having an
optical axis. There is included through the control panel 46 of the remote
control station 44 means for selectively rotating the camera through a
control system 48. The camera 20 and the optical axis thereof can be
rotated 360 degrees around a first axis of the plurality of axes. There is
also included on the control panel 46 means for selectively pivoting the
camera 20 and the optical axis thereof through the control system 48
through 180 degrees about a second axis of the plurality of axes which is
perpendicular to the first axis.
Accordingly, the operator at the remote control station 44 is capable of
using the control panel 46 for selectively rotating and selectively
pivoting the camera 20 and the optical axis thereof until the target can
be seen in the monitor 42 and is specifically located at a target position
therein. In other words, the operator at the remote control station 44 can
adjust the rotation and pivot of the camera 20 from the control panel 46
while viewing the monitor 42 until a view of the target is properly
positioned within the monitor 42 for further operation of the robot 40.
After the camera is properly positioned, a camera position is sensed by the
sensor system 50 as it relates to the optical axis of the camera 20.
Additionally, the optical axis includes a target direction which is sensed
by the sensor system 50 to determine the position of the camera 20 and the
optical axis thereof when the target is properly positioned within the
monitor 42. Both the camera position and the target direction of the
optical axis are transmitted to the control system 48 and eventually to
the memory system 52 for future reference. Additionally, such information
as the coordinates and/or direction of the target direction and,
eventually, the direction of the tool head of the robot 40 will be
included from the control system 48 for display on a display panel 54 of
the remote control station 44.
After proper positioning of the camera 20 and the target direction of the
optical axis are received in the control system 48, the operator may
activate the control system 48 at the control panel 46 for automatic
movement of the various axes of the industrial robot 40 through the
degrees of freedom thereof in order to move the camera 20 directly toward
the target. Specifically, the control system 48 includes means for moving
the robot through the degrees of freedom for each of the axes of the
plurality of axes including the means for selectively rotating and the
means for selectively pivoting the camera 20. The control means 48
receives information from the sensor system 50 for continuous monitoring
of the actual location of the various axes in order to determine the
location of the various axes of the robot 40 as specific controls are
given to the motors thereof to produce the desired movement of the camera
20 toward the target. The control system 48 maintains the camera position
and the target direction of the optical axis during control movement of
the various axes in order to be able to cause the tool head of the robot
40 to move along the target direction to the target. The control system 48
causes controlled movement to be produced by the means for moving a first
group of the plurality of axes while each axis of a second group of the
plurality of axes is limited to synchronous movement in direct response to
the controlled movement of the axes of the first group. More specifically,
the control system 48 includes an axes selection system 56 including means
for selectively including each axis in the first group and for selectively
excluding each axis from the first group to limit the first group of axes
to include a maximum of three of the axes of the plurality of axes. The
control system uses the information regarding the location of the camera
and the target direction to the target in order to select with the axes
selection system 56 which axes are to be positively controlled for
movement toward the target and includes programmed information to ensure
that other axes which are directly affected by movement of the controlled
axes will automatically include synchronous movement in response thereto
in order to properly move the tool head and/or the camera 20 attached
thereto toward the target.
With the system as basically described, the operator is capable of finding
the location of | | |
