 |
Description  |
|
|
BACKGROUND OF THE INVENTION
The present invention relates to a remote manipulation method using a master manipulator and a slave manipulator and a system using said method.
In the remote manipulation, a human operator operates a master manipulator or a handling lever (to be collectively referred to as a master herebelow) and then a slave manipulator (to be referred to as a slave herebelow) is caused to operate
according to the operation of the human operator, thereby achieving an objective job. According to the prior art technology, in many remote manipulation systems, the attitude of the slave follows the attitude of the master so as to transfer the reaction
force on the slave to the human operator. Generally, operations are performed by also using a TV camera and a TV display to monitor the slave attitude and the state of work being performed. As described in the "Proceedings of '85 International
Conference on Advanced Robotics", pp. 329-336 and the "Proceedings of the 1984 National Topical Meeting on Robotics and Remote Handling in Hostile Environments", pp. 367-374, there exist some systems in which with an input of a command by a human
operator, a computer imposes a restriction on a portion of the motion of the slave (for example, to retain the tip of the hand of the slave to be horizontal), thereby aiding a portion of the manual operation.
The conventional technology is attended with difficulties and troublesome operations because the human operator operates the master to indirectly move the slave located at a remote position for performing work. Furthermore, depending on the
skill required (degree of dexterity), the efficiency of work (for example, a period of time required for performing a piece of work and the quality of the result of work) is greatly varied.
SUMMARY OF THE INVENTION
It is therefore an object of the invention to provide a remote manipulation method and a system using the same wherein even when human operators having different levels of skill operate the system, the required work can be effectively performed
according to the levels of skill of the respective human operators.
According to an aspect of the present invention, an operation procedure matching the level of skill of a human operator is automatically determined and operations which form part of the work are achieved partly through operations of a computer
and partly as a result of manual operations of the human operator.
According to another object of the present invention, in working involving remote manipulation using a master and a slave, an operation procedure plan, including combinations of manual and automatic operations, is generated by use of data
indicating the level of skill of the operator and a goal for the work to be performed, the automatic operations are automatically performed by a motion management of the computer according to the operation procedure plan, and the manual operations are
manually accomplished by the human operator.
According to still another aspect of the present invention, there are provided planning means for generating a procedure mixing manual operations and automatic operations based on skill data indicating a level of skill of a human operator and a
goal of the work to be performed, slave managing means for aiding manual operations and performing automatic operations in the automatic operation state, and interface means for effecting management to guide the human operations in the manual operation
state for a smooth execution of the work, wherein a plan of operation is first generated and then the operation procedures are sequentially accomplished.
The planning means generates operation procedures by mixing automatic operations and manual operations depending on data indicating the skill of the operator and the goal work supplied thereto. This makes it possible to generate operation
procedures for the operators having different levels of skill, thereby developing a high work efficiency. The slave managing means executes automatic operations and aids the operation in the manual operation state. This removes the unnecessary jobs of
the operator to proceed the work. The interface means requests the operator's intervention during the manual operation mode and supplies the human operator with feedback information such as a sense of force, which enables the human operator to
understand the contents of the operation to be performed and to smoothly accomplish the work.
The initial point of the present invention is the application of the artificial intelligence to the remote manipulation and a division of a part of the work to be controlled by a computer. However, a decision to divide portions of work and
operations among the computer and the human operator is a difficult job. As a unique problem, when the remote manipulation is achieved by the human operator, there arises a difficulty in performing an indirect operation of a remote slave while watching
the operation on a TV monitor or the like. To perform such a difficult work, the human operator is required to obtain a sense of directions and particular behaviors of the mechanical system and the control system. In the actual manipulation work, the
skill greatly varies between the beginner and the experienced operator, namely, it has been found that some operations executed by the experienced operator cannot be achieved by the beginner and that the operation speed and correctness of the same
operation also considerably varies between the beginner and the experienced operator. This includes, for example, an operation to grasp a fragile object with an appropriate force, fitting work, a positioning operation, the correctness to align axes, and
the time and smoothness in an operation to move the gripper of the slave to a desired, position. Even the experienced operator becomes fatigued in 2 to 3 hours and the level of skill of the operation is at that point greatly decreased.
On the other hand, for the operation conducted by the computer, for example, the positioning of the gripper with a coarse precision can be achieved at a very high speed; however, a long period of time is required to move the gripper while
detouring an obstacle and to achieve the positioning with a high precision. Moreover, there exist some operations which can be performed only by the human operator.
The inventors of the present invention have consequently recognized that operations involving remote manipulation may be divided into operations to be accomplished by the human operator, operations to be performed by the computer, and operations
to be achieved by both the computer and the human operator depending on the level of skill of the human operator. Based on this recognition, according to the present invention, a plan of operation comprising computer operations, human operations, and
cooperative operations of the computer and the human operator is generated by mixing or combining the objects of work and the skill of the human operator, whereby the portion to be executed by the computer is achieved in the automatic operation state;
moreover, the portion of the human operations is preferably performed depending on a fundamental principle including an operation request for the human operator. This leads to the most smooth and high-speed remote manipulation.
BRIEF DESCRIPTION
OF THE DRAWINGS
FIGS. 1-2 block diagrams illustrating the overall configuration of an embodiment of the present invention.
FIG. 3 is a schematic explanatory diagram illustrating in detail the 6th data storage section (status data) of FIG. 2.
FIG. 4 is an explanatory diagram showing in detail a portion of FIG. 3.
FIG. 5 is a perspective view of an example for explaining the present invention.
FIGS. 6-7 are explanatory diagrams illustrating in detail the 2nd data storage section (work data) of FIG. 2.
FIGS. 8A-8G are schematic diagrams illustrating in detail the 5th data storage section (operation procedure plan) of FIG. 2.
FIG. 9 is a flow chart for explaining the general operation of the embodiment shown in FIGS. 1 and 2.
FIG. 10 is a block diagram showing the planning processor means.
FIGS. 11-12 are schematic diagrams illustrating the block diagram and an operation example of the interface processor means.
FIGS. 13-17 are flowcharts of processing in the interface processor means.
FIGS. 18-19 are flowcharts of the slave managing processor means.
FIGS. 20, 21A, and 21B are schematic diagrams showing an operation example of the slave managing processor means.
FIG. 22 is a block diagram illustrating another embodiment of the present invention having means for evaluating the skill of the human operator.
FIG. 23 is a block diagram illustrating the operator's skill evaluating means of FIG. 22.
FIGS. 24-26 are explanatory diagrams for explaining parameters.
FIG. 27 is a flowchart for explaining the fundamental principle of the operator's skill evaluating means.
FIG. 28 is an explanatory diagram of parameters.
FIG. 29 is a block diagram illustrating another embodiment of the present invention.
DESCRIPTION OF THE PREFERRED EMBODIMENTS
An embodiment of the present invention will be described with reference to FIG. 1. This embodiment includes an interface processor 103 connected to a master 101 and a TV monitor 102, a planning processor 104, a slave managing processor 105, a
slave controlling processor 106, slave 107 to be controlled by the slave controlling processor 106, a sensing device 108 for sensing the state of the slave 107, a memory means 109 in which information concerning read and write operations are effected by
the elements 103, 104, 105, 106, and 108, and a first data storage section 111 disposed in the memory 109 for storing data indicating the operation skill of the human operator, namely, the human operator's skill data.
FIG. 2 is a schematic block diagram for explaining the embodiment of FIG. 1 further in detail. In this diagram, the contents of the memory 109 include, in addition to the first data storage section 111, a 2nd data storage section 201 storing
work data, a 3rd data storage section 202 storing environment data, a 4th data storage section 203 storing system data, a 5th data storage section 204 storing an operation procedure plan, a 6th data storage section 205 storing system status data, a 7th
data storage section 206 storing control input data, and an 8th data storage section 207 storing sensed information data. The blocks of FIG. 2 will be described in the following paragraphs.
<6th data storage section 205>
As shown in FIG. 3, the 6th data storage section 205 contains an operator number 3a of an operator currently being executed, a flag 3b indicating the end of execution, an operator number 3c of an operator to be next executed, and a description 3d
of the system status. Since the items 3a, 3b, and 3c will be described in detail later in conjunction with the 5th data storage section, the description of system status 3d will be given here. As shown in FIG. 4, the content of the description of
system status 3d includes a state of system 4a and a state of the object undergoing work 4b of the master and the slave.
The state of system 4a contains information indicating a joint angle of the master and slave, a position of the gripper, a direction of the gripper, and an information item indicating whether or not the gripper is grasping an object.
The state of the object undergoing work 4b contains a data item 4b1 indicating a mutual relation of the work object and a data item 4b2 indicating a position and an attitude.
Let us assume an example of FIG. 5 for a concrete description of each information. In this example, a valve is disassembled through a manipulation operation, namely, a valve body 54 is taken out from a location inside a flange 52 and a valve
casing 53 fixed to each other with a bolt 51. There may be used, for example, 18 bolts; however, only a single bolt is assumed to be used for simplification of description. Referring again to FIG. 4, the description will be given. The information of
mutual relation 4b1 describes the state that the flange 52 and the valve casing 53 are fixed to each other, that the valve body 54 is located in the valve casing 53, and that the bolt 51 fixes the flange 52 to the valve casing 53. The information of the
position/attitude 4b2 contains position vectors of x.sub.i, x.sub.j, and x.sub.R of each part in the coordinate system of the space and the attitude angle vector .theta..sub.i, .theta..sub.j, and .theta..sub.k.
<2nd data storage section 201>
The contents of the 2nd data storage section 201 include a goal for the work 6a, a definition of an operator 6b, and data concerning shape and dimensions of an object undergoing work 6c as shown in FIG. 6. The goal for work 6a defines a target
state that the valve body is located on a work stand and that the position and the attitude are represented by x.sub.r and .theta..sub.r, respectively. Moreover, data associated with the shape of the flange and the bolt is contained as data concerning
the work object.
The definition of operator 6b will be described with reference to FIG. 7. An operator is means for changing a state. In order to establish the state of the goal for work (for example, as indicated by 6a of FIG. 6) from the state of object
undergoing work (for example, as indicated by 4b of FIG. 4), operations are required to be sequentially achieved by use of appropriate means. The means (operators) are classified into several levels for reasons to be described later. In the example of
FIG. 7, a definition is made for a task-level operator, "demounting of flange". Since the description of the level will be given later, the preconditions 7a and the change of state 7b will be here described.
The content of the preconditions 7a indicates conditions under which the pertinent operator is available. In the example of this diagram, the preconditions 7a represents that the flange is not fixed and is located on the valve body. The
contents of the change of state 7b are described in two parts, namely, a deletion item and an addition item. The deletion item field indicates items to be deleted from the description of the original state after the operator is effected. In the example
of FIG. 7, the item indicating that the flange is located on the valve body and the item related to the position and the attitude of the flange. The addition item field contains items to be added. In the example of the diagram, an item indicating that
the flange is located on a floor and an item concerning the new position and attitude of the flange. The cost field 7c contains a value of a cost (to be described later) or a function to be evaluated when this operator is executed.
<3rd data storage section 202>
The 3rd data storage section 202 stores data related to positions, dimensions, and the shape of devices other than the work object and the slave. The data is used to prevent, for example, a collision between the slave and other devices when the
slave is caused to operate.
<1st data storage section 111>
The 1st data storage section 111 stores data indicating points or a level for an item representing the skill of the human operator so as to indicate the degree of the skill. The skill items include various data as follows.
(1) Gripper moving time to trajectory accuracy
(2) Gripper positioning time to positioning accuracy
(3) Gripper shaft aligning time to aligning accuracy
(4) Reaction force sensitivity
<4th data storage section 203>
The 4th data storage section 203 stores the following data.
(1) Data of dimensions and shape of slave 107
(2) Performance and function of automatic operations by the computer
<Sensing means 108 and 8th data storage section 207>
The sensing means 108 includes a TV camera or the like for displaying a scene on the TV monitor 102 used for the automatic operations of an encoder, a potentiometer, a tacho-generator, a force sensor, and a computer disposed for the control of
the slave 107 and supplies data to the 8th data storage section 207 storing sensed information data.
The 8th data storage section 208 stores the following information.
(1) Data resulting from various sensing operations
(2) Positions and performances of various sensors, position and characteristics of the TV camera, the camera number of the camera currently displaying a scene on the monitor 102, etc.
<Slave controlling processor means 106 and 7th data storage section 206>
The slave controlling processor means 106 controls the position, the velocity, and the force on each joint of the slave 107. The types and target values for the control are contained in the 7th data storage section 206.
<Planning processor means 104 and 5th data storage section 204>
Since a concrete embodiment of the planning processor means 104 will be described later, an outline thereof will be now described. The planning processor means 104 generates a plan of operation procedures by judging the sequence of operators
(FIG. 7) necessary to establish the state of the goal for work (for example, 6a of FIG. 6) from the state of the system (for example, 4b of FIG. 4) and stores the results in the 5th data storage section 204. To indicate that the plan of operation
procedures has been contained in the 5th data storage section 204, the first number of a plan is stored in the field of the number of the operator to be executed next (3c of FIG. 3) of the 6th data storage section 205. An example is shown in FIG. 8A in
which the 5th data storage section 204 stores plans configured in five levels. The plan of each level includes operators defined as shown in FIG. 7. Examples thereof will be illustrated in FIGS. 8A-8G. First, in FIGS. 8A-8B, the plan of the highest
level (purpose level plan 8a) comprises only an operator 8f, which represents a concept comprehensively including the overall work. This operator is instantiated with a task level plan 8b (FIG. 8), namely, the procedure is formed as "demounting of bolt"
8g .fwdarw. "demounting of flange" 8h .fwdarw. "demounting of valve body" 8i. In the plan, the operator number field indicates the sequence to execute the operator, and the number in the upper level pointer field indicates the number of the operator
which exists in the plan next higher in the level as compared with this plan and from which this item has been created. Namely, in the example of the diagram, three operators of the task level plan 8b of FIG. 8 are generated from the operator No. 1 of
the purpose level plan field 8a of FIG. 8B. As the level lowers, the operator becomes more concrete, and at the lowest level, namely, the operator of the trajectory level 8e, the data becomes to be executed by the human operator or the computer. FIGS.
8D-8E are schematic diagrams showing examples of the procedure levels 8c and 8d, respectively. "Demounting of bolt" 8g at the task level 8b in FIG. 8C is divided into "loosen bolt" 9a and "remove bolt" 9b at the sequence level 8c in FIG. 8D. "Loosen
bolt" 9a is a task to be conducted by use of a special tool. The lower levels of the "remove bolt" 9b will be described. As shown in FIG. 8E, at the motion level 8d, there are provided concrete items, namely, "approach bolt" 9c, "grip bolt" 9d, "rewind
bolt" 9e, and "move x.sub.d " 9f. These items respectively mean "move the gripper to a position to grasp the bolt", "grip the bolt", "rewind the bolt", and "move the gripper to the position x.sub.d ". Of these operations, the items 9c, 9d, and 9e are
instantiated as shown in the example of FIG. 8F. This diagram illustrates the lowest level, namely, the trajectory level 8e. As shown in this diagram, "approach bolt" 9c is made to be concrete as "move x.sub.1 : cond.sub.1 ", "move x.sub.2 : cond.sub.2
", "move x.sub.3, cond.sub.3 ", and "op-move x.sub.4, cond.sub.4 ". This means that in order to move the gripper to the position x.sub.4 to grip the head of the bolt, the gripper is moved from x.sub.1 to x.sub.3 via x.sub.2 in the automatic mode and is
further moved from x.sub.3 to x.sub.4 in the manual mode (op move). As described above, the operator associated with the manual operation is, for example, marked with "op-" in its name so as to be discriminated from the operators to be executed in the
automatic mode. The symbol such as "cond.sub.1 " added to an operator indicates an attendant condition to be effected when the operator is executed. The attendant conditions are stored in a separate location, for example, as shown in 10g of FIG. 8G.
Field 10g describes the attitude of the gripper, a restriction concerning the position, namely, the manual operation is to be guided by the computer, and other operations. The attendant conditions further include various conditions such as compliance
conditions and sensing means to be used. "Op-move" 10d is followed by "close-hand" 10e which is a more specific form of "grip bolt" 9d. The attendant condition, cond.sub.5 of this operator includes the descriptions associated with a force to grip the
bolt and the like. The next operator "rotate-hand" 10f means that the bolt is rotated ten turns and then is moved. The attendant conditions are described such that as the bolt is made to be looser, the position of the hand is gradually moved according
to the compliance control and the attitude of the gripper is kept fixed.
<Slave managing processor means 105>
The operators stored in the 5th data storage section 204 are sequentially executed according to the "number of operator to be executed next" 3c (FIG. 3) contained in the 6th data storage section 205. The operators to be executed are at the
trajectory level 8e. In these operations, the slave managing processor means 15 executes the operations depending on the contents of the operators and the attendant conditions. For example, when "move x.sub.1 : cond.sub.1 " 10a is executed, the means
105 controls the slave 107 via the 7th data storage section 206 so that the gripper reaches the position x.sub.1 while satisfying the condition indicated by cond.sub.1. Furthermore, the processor means 105 judges the end of execution by checking the
content of the 8th data storage section 207 and thereby updates the respective items 3a-3d (FIG. 3) of the 6th data storage section 205.
<Interface processor means 103>
If the next operator to be executed is a manual operation, namely, if "op-" is added to the operator, the interface processor means 103 guides the operation of the human operator 110 through a display and a voice output on the monitor 102 and a
force generated at the master 101. The interface processor means 103 reads the force applied to the master 101 from the human operator and the change in the force, writes appropriate control input data in the 7th data storage section 206, and reads the
sensed information from the 8th data storage section 207 at the same time to generate a reaction force and the like in the master, thereby supplying the feedback information such as the sense of force to the human operator. Moreover, the interface
processor means 103 updates the respective items 3a-3d (FIG. 3) of the 6th data storage section 205.
Even for an operator related to the manual operation, the slave managing processor means 105 may function in some cases. This applies to a case where the manual operation is aided, for example, the attendant conditions of the operator impose
restrictions on the attitude of the gripper. In such a case, the interface processor means 103 writes data in a control input data buffer 208 of the 1st data storage section 111. This data is read by the slave managing processor means 105, which then
writes in the 7th data storage section 206 the control input to which the restrictive condition is added. In this case, the update of the 6th data storage section 205 is accomplished only by the interface processor means 103.
<General flow>
A description will be given of an outline of the operation procedures of the manipulation system including the block described above (FIG. 2). Assume that the work data excepting the goal for work, the environment data, the system data, and the
human operator's skill data are beforehand stored in the 2nd, 3rd, 4th, and 1st data storage sections 201, 202, 203, and 111, respectively. In the flowchart FIG. 9, the initial values are set to the goal for work 6a (FIG. 6) and the 6th data storage
section 205 (lla). If "No. 1 of purpose level" is beforehand stored as the initial value in the field of number of operator to be executed next 3c (FIG. 3) in the 6th data storage section 205, since this entry is not an operator at the trajectory level,
the system does not execute the automatic operation or the manual operation. If an operator at a level other than the projectory level is stored in the field 3c, the planning processor means 104 is caused to operate, which then further instantiates the
plan at the purpose level to generate a plan at the projectory level, stores the generated plan in the 5th data storage section 204, and thereafter sets the number of the trajectory-level operator to be first executed to the field of number of operator
to be executed next 3c of the 6th data storage section 205 (llb). When the data at the trajectory level is stored in the field 3c, the interface processor means 103 or the slave managing processor means 105 is caused to function, namely, an operator is
executed (11c), the 6th data storage section 205 is updated, a check is effected to determine whether or not an unexecuted operator exists (11d), and the execution is finished if there does not exist such an operator (11e). The unexecuted operator
includes not only a projectory-level operator not executed but also such operators at a level higher than the trajectory level and not instantiated to the trajectory level. The means 103 or 105 then selects a number for the next execution from the
unexecuted operators and sets the number to the field of number of operator to be executed next 3c, thereby executing the next operator. In this case, if the number written in the field 3c is associated with an operator having a level higher than the
trajectory level, the planning processor means 104 initiates the operator, while the processor means 103 or 105 does nothing until the initiation (planning) is completed (11f).
<Detailed description of planning processor means 104>
Since the function of the planning processor means 104 has already been outlined, the method for realizing the planning processor means 104 will be here described. As already described, when an operator number of an operator having a level
higher than the trajectory level is written in the field of operator to be executed next 3, the planning processor means 104 is initiated to instantiate the operator. As a consequence, the planning processor means 104 is structured as shown in FIG. 10
to instantiate operators at the various levels. The processor means 104 further includes planners 104a-104e for the respective levels. Each planner instantiates an operator at a next higher level to obtain a plan of operators at its own level.
FIGS. 8A-8G shows examples in which operators at a level cause to generate more operators as the operators are instantiated to the lower level. Although the operators can be instantiated at a time to the lowest level, namely, the trajectory
level, there exists another method in which in the course of the operator instantiation, only the first operator at each level is instantiated. In this case, the other operators are temporarily stored in the fields of the respective levels 8a-84 (FIG.
8) and are later selected as "the number of operator to be executed next" for the instantiation.
The planner at each level fundamentally performs a similar operation. The generation of a plan is a process for determining a sequence of operators to be executed to establish a state of goal from the initial state. Ordinarily, there exist
various ways of transition from the initial state to the goal state, namely, the transition is achieved through various intermediate states. The "state" means the description of the system and objects in the form of the example shown in FIG. 4.
However, the state here is created hypothetically, and thus not identical to the one in FIG. 4. If a plurality of operators (for which the state A satisfies the precondition field 7a (FIG. 7) can be executed on the state A, the state A is changeable to
be another state B, C, or D. Moreover, these states B, C, and D can also be changed to be other states. This makes it possible draw a transition graph in which each state is represented as node. In the graph, there exist two or more paths to reach a
goal state, a cost is added to each node, to select a path leading to the goal state at the minimum cost. Such a selection can be implemented by use of a general method called a graph search method. When the path is determined, a list of operators to
be executed and a list of states (nodes) changing in sequence are obtained and are then respectively stored in the plan fields of the respective levels 8a-- 8a of the 5th data storage section 204 and the node sections 12a-12e. When instantiating an
operator .alpha. stored in the 5th data storage section 204, a node P and a node Q immediately before and after the execution of the operator .alpha. are supplied to a planner at a next lower level, which in turn generates a plan by using the node P as
the initial state and the node Q as the goal state in the similar fashion. The instantiation of an operator is sequentially achieved as described above.
One of the characteristics of the present invention is a method for evaluating the cost of an operator. For example, assume that when an operator .beta. is effected on a node i, the node i is changed to be a node j. In this case, the cost of
the node j has a value reflecting the cost of the node i and the cost of the operator .alpha.. As a consequence, the plan selected as a result of the graph search is a list of operators each with a relatively low cost. Namely, the plan to be generated
greatly varies depending on the method for evaluating the cost of each operator. In this embodiment, the cost field 7c (FIG. 7) contains a cost evaluation function in many cases. The cost evaluation function is used to effect a cost evaluation of an
operation at the pertinent time and position by use of the data concerning shape and dimensions of an object undergoing work 6c stored in the 2nd data storage section 201, the environment data stored in the 3rd data storage section 202, the system data
stored in the 4th data storage section 203, the system status data stored in the 6th data storage section 205, and the human operator's skill data stored in the 2nd data storage section 111. In particular, when evaluating the cost of an operator for the
manual operation, namely, an operator having "op-" added to the title thereof, the cost is determined with reference to the human operator's skill data in any case. Consequently, a manual-operation operator which is evaluated for a low cost in a plan
when the skill of the human operator is high is changed to be an operator for the automatic operation if the skill of the human operator is low.
Next, a description will be given of the attendant condition 10g (FIG. 8) of an operator at the trajectory level. Some attendant conditions 10g are beforehand specified as definitions for the associated operators and some attendant conditions
10g are determined to minimize the cost of an operator when evaluating the cost thereof. Particularly, in the cost evaluation for an operator to be effected in the manual operation, the attendant conditions greatly influence the cost depending on the
skill of the human operator. For example, since the operation to align axes is difficult for the beginner or unexperienced human operator, if an attendant condition is added to generate an appropriate compliance from the visual sensor information, the
cost of the operator can be considerably reduced.
<Details of interface processor means 103>
The interface processor means 103 will be described in detail with reference to FIG. 11 and subsequent diagrams. FIG. 11 shows the configuration of the interface processor means 103. Each component thereof will be here described. An interface
control section 1301 includes a memory, a processor, or the like therein and controls the entire interface processor section 103. A bus 1310 is used to effect a data exchange between the interface control section 1301 and the other components of the
interface processor section 103 and the memory means 109. A master control section 1303 controls the master 101 by use of the control data stored in a master control input section 1302. The master control input section 1302 contains the control data to
be periodically updated by the interface control section 1301. Based on the content of the operator currently being executed supplied from the interface control section 1301 and the position data of the gripper, an operation command display generator
1304 displays on the TV monitor 102 the contents of work (for example, data 1402-1404 of FIG. 12) and the goal position (for example, the item 1405 of FIG. 12) of the gripper of the slave. In a display mixer section 1305, the CRT display data output
from the operation command display generator 1304 to the TV monitor 102 is superimposed onto the TV monitor screen information (the images of the object and the gripper of the slave monitored by the TV camera) stored in the 8th data storage section 207
of the memory means 109 (FIG. 2).
FIG. 12 shows an example of a screen image. The following information items are displayed on the CRT display screen 1401 of the TV monitor 102.
(1) Actual images of the object undergoing work and the gripper of the slave
(2) Contents of current task items
In this diagram, "demount of bolt", "removal of bolt", and "approach bolt" are displayed in the task, sequence, and motion level operation indicators 1402, 1403, and 1404, respectively.
(3) Graphic display of goal position of the gripper
The goal position of the trajectory-level operator currently in motion is graphically displayed with broken lines i | | |
