WikiPatents - Community Patent Review
Create Free Account  |  License or Sell Your Patent  |  WikiPatents Marketplace  |  WikiPatents Blog
Username:  Password:  
    
Advanced Search
Apparatus and method for remote control using a visual information stream    
United States Patent5652849   
Link to this pagehttp://www.wikipatents.com/5652849.html
Inventor(s)Conway; Lynn A. (Jackson, MI), Cohen; Charles J. (Philadelphia, PA)
AbstractSystems and methods for controlling devices remotely by embedding command information within video signals. The command information includes gestures or icons which are imbedded into the video signal and enable the user to interpret the command information by viewing the icon or gesture on a monitor or display device. By viewing the monitor or display device, the user can determine what command information is conveyed within the video signal. Further, imbedding the visually interpretable icon or gesture within the video signal facilitates transmitting the video signal having the command information imbedded therein using analog to digital conversions and reconversions and transmission through any of a various number of switching networks. The remote devices may by any of a number of electronically controlled electrical or mechanical devices. A device controller at the remote end receives, the decoded command information and generates the appropriate commands to operate the controlled device.
   














 Title Information Submit all comments and votes
 
Patent Text Patent PDF Print Page Summary File History
Plain text PDF images Print Summary File History
Drawing from US Patent 5652849
Apparatus and method for remote control using a visual information stream - US Patent 5652849 Drawing
Apparatus and method for remote control using a visual information stream
Inventor     Conway; Lynn A. (Jackson, MI) , Cohen; Charles J. (Philadelphia, PA)
Owner/Assignee     Regents of the University of Michigan (Ann Arbor, MI)
Patent assignment
All assignments
Publication Date     July 29, 1997
Application Number     08/404,886
PAIR File History     Application Data   Transaction History
Image File Wrapper   Patent Term   Fees
Litigation
Filing Date     March 16, 1995
US Classification     715/719 348/115 348/E5.043 348/E7.085 701/25 715/835 715/863
Int'l Classification    
Examiner     Bayerl; Raymond J.
Assistant Examiner     Katbab; A.
Attorney/Law Firm     Harness, Dickey & Pierce, P.L.C.
Address
Parent Case    
Priority Data    
USPTO Field of Search     395/155 395/156 395/157 395/158 395/159 395/160 395/161 395/327 348/115 348/114 348/113 364/424.029
Patent Tags     remote control visual information stream
   
Enter a comma (,) or semicolon (;) between multiple tag words/phrases.
Describe this patent:
 Amusing   
 Clever   
 Complex   
 Efficient   
 Historic   
 Important   
 Innovative   
 Interesting   
 Practical   
 Simple   
[no votes]
Patent WIKI

Share information and news about this patent, including information and news about the technology, inventors, company, ligation and licensing.

 References Submit all comments and votes
 
*references marked with an asterisk below are user-added references
 U.S. References
 
Add a new US reference:  
ReferenceRelevancyCommentsReferenceRelevancyComments
5331413
Diner

Jul,1994

[0 after 0 votes]
5236199
Thompson, Jr.

Aug,1993

[0 after 0 votes]
5182641
Diner et al.

Jan,1993

[0 after 0 votes]
5155683
Rahim

Oct,1992

[0 after 0 votes]
5109228
Banaszak

Apr,1992

[0 after 0 votes]
5079634
Hosono

Jan,1992

[0 after 0 votes]
4996592
Yoshida

Feb,1991

[0 after 0 votes]
4887223
Christian

Dec,1989

[0 after 0 votes]
4855822
Narendra et al.

Aug,1989

[0 after 0 votes]
4636137
Lemelson

Jan,1987

[0 after 0 votes]
3691295
Fisk

Sep,1972

[0 after 0 votes]
3542015
Campbell

Nov,1970

[0 after 0 votes]
2978538
Breese

Apr,1961

[0 after 0 votes]
 Foreign References
 Other References
 Market Review Submit all comments and votes
   
Market Size
Estimate the gross annual revenues of the relevant market sector:
> $10B
$5B - $10B
$2B - $5B
$500M - $2B
$100M - $500M
$10M - $100M
$1M - $10M
$500K - $1M
$100K - $500K
< $100K
[No votes]
$0
 
$0   $2.5B   $5B   $7.5B   $10B
Market Share
Estimate the percentage of the relevant market sector this invention will capture:
75% - 100%
50% - 74.99%
25% - 49.99%
10 - 24.99%
5 - 9.99%
2 - 4.99%
1 - 1.99%
< 1%
[No votes]
0.0%
 
0%   25%   50%   75%   100%
Reasonable Royalty
What percentage of gross sales should the inventor or assignee be paid?
75% - 100%
50% - 74.99%
25% - 49.99%
10 - 24.99%
5 - 9.99%
2 - 4.99%
1 - 1.99%
< 1%
[No votes]
0.0%
 
0%   25%   50%   75%   100%
Public's "Guesstimation" of Royalty Value
Market SizeN/A[No votes]
xMarket ShareN/A[No votes]
xReasonable RoyaltyN/A[No votes]

N/A

License Availablity
If you are NOT the owner or assignee, answer here:
Yes, license is available for purchase

No, license is not currently available



[No votes]
License Availablity
If you ARE the owner or assignee, answer here:
Yes, license is available for purchase

No, license is not currently available



[No votes]
Competitive Advantage
Does this invention have a significant competitive advantage over similar technologies?
Yes

No



[No votes]
Most helpful competitive advantage comment
[No comments]

Commercial Alternatives
Are there viable commercial alternatives for this invention?
Yes

No



[No votes]
Most helpful commercial alternative comment
[No comments]

 Technical Review Submit all comments and votes
 Claims Submit all comments and votes
 


We claim:

1. An apparatus for enabling an operator to receive an incoming video signal and to imbed command information in an outgoing video signal in order to control a device, comprising:

means for generating the command information in the form of a visually iconic gesture overlaid onto the incoming video signal to define the outgoing video signal, the visually iconic gesture being interpretable by the operator and conveying the command information in accordance with its location within the outgoing video signal;

processing means for receiving the outgoing video signal having the visually iconic gesture and decoding the iconic gesture into the command information conveyed by the iconic gesture, the processing means outputting the command information; and

device control means for receiving the command information output by the processing means and transforming the command information into control commands for operating the device in accordance with the command information conveyed in the visually iconic gesture.

2. The apparatus as defined in claim 1 further comprising:

a local site at which is located the operator and the generating means; and

a remote site at which is located the processing means and the device control means.

3. The apparatus as defined in claim 2 further comprising a communications interface for establishing a communications link between the local site and the remote site, the communications link enabling transmission of the incoming video signal from the remote site to the local site and enabling transmission of the overlaid video signal from the local site to the remote site.

4. The apparatus as defined in claim 3 further comprising a monitor means interconnected to the means for generating command information, the monitor means displaying the received video signal and the visually iconic gesture overlaid thereon in order to enable an operator to view the iconic gesture and its relative position within the received video signal.

5. The apparatus as defined in claim 4 wherein the means for generating command information further comprises:

means for positioning the iconic gesture, the iconic gesture being overlaid into the receiving video signal and viewable by the operator on the monitoring means; and

means for selecting at least one of a plurality of candidate iconic gestures, the selected at least one iconic gesture being positionable in accordance with operation of the positioning means.

6. The apparatus as defined in claim 1 wherein the means for generating command information further comprises:

means for positioning the iconic gesture, the iconic gesture being overlaid into the receiving video signal and viewable by the operator on the monitoring means; and

means for selecting at least one of a plurality of candidate iconic gestures, the selected at least one iconic gesture being positionable in accordance with operation of the positioning means.

7. The apparatus as defined in claim 6 wherein the means for generating the iconic gesture further comprises a video overlay mixer for overlaying onto the incoming video signal the command information in the form of the iconic gesture and the position within the incoming video signal, the means for generating the iconic gesture outputting an overlaid video signal to a monitor.

8. The apparatus as defined in claim 1 wherein the means for generating the iconic gesture further comprises a video overlay mixer for overlaying onto the incoming video signal the command information in the form of the iconic gesture and the position within the incoming video signal, the means for generating the iconic gesture outputting an overlaid video signal to a monitor and to the processing means.

9. The apparatus as defined in claim 1 wherein the processing means for decoding the outgoing video signal further comprises:

means for extracting from the overlaid video signal information including a command icon and a position of the command icon; and

means for interpreting the iconic information extracted from the overlaid video signal.

10. The apparatus as defined in claim 9 wherein said means for extracting further comprises:

memory for storing a digitized video signal;

pattern memory for storing in digitized form at least a portion of a candidate icon recognizable by the extracting means; and

icon recognizer means for comparing the at least a portion of the candidate icon stored in pattern memory with the digitized video signal stored in memory, the icon recognizer outputting a signal in accordance with a recognized icon.

11. The apparatus as defined in claim 9 wherein the incoming signal is an analog signal and the apparatus further comprises a digitizing means for digitizing the analog incoming video signal to yield a digital representation of the incoming video signal.

12. The apparatus as defined in claim 9 wherein the incoming video signal is a digital signal and the apparatus further includes signal processing means to condition the incoming video signal for storage in memory.

13. The apparatus as defined in claim 1 wherein the device to be controlled comprises:

a controllable camera base arranged to enable motion in at least one degree of freedom, said camera base comprising:

a camera mounted to said camera base;

a first means for enabling the camera base to operate in the at least one degree of freedom; and

communication means for communicating with the device control means, the communication means sending and receiving control signals to operate the camera base in the at least one degree of freedom.

14. The apparatus as defined in claim 13 wherein the enabling means comprises a motor.

15. The apparatus as defined in claim 13 wherein the means for generating command information is operable to insert visual information defining a centering icon, and the control means is operable to position the controllable base to center the camera view area about a point selected in accordance with the centering icon.

16. The apparatus as defined in claim 13 wherein the means for generating command information is operable to insert a visual iconic gesture defining a pan/tilt icon, and the control means is operable to position the controllable base in a direction and a rate in accordance with the direction and magnitude, respectively, of a displacement of the pan/tilt icon from a center of the camera view area.

17. The apparatus as defined in claim 13 wherein the means for generating command information is operable to insert a visual iconic gesture information defining a move icon, and the control means is operable to position the controllable base in accordance with the position of a first and second move gesture where the first move icon defines a starting point for the move and a second move gesture defines a final point for the move where camera base is displaced such that the final point of the view area is displaced to the initial point of the view area.

18. The apparatus as defined in claim 1 wherein the controlled device is a robot arm which receives control signals from the device control means and is manipulable in accordance with the control signal.

19. The apparatus as defined in claim 1 wherein the controlled device is an electronically controlled device which receives control signals from the device control means and is operable in accordance with the control signal.

20. A remote control video apparatus in which an operator at a local site which receives an incoming video signal from a remote site can generate commands to operate a device located at the remote cite, comprising:

a visual signaller for receiving the incoming video signal from the remote site, the visual signaller operable to overlay visual command information onto the incoming video signal to define an outgoing video signal, the visual signaler being operable to output the outgoing video signal for transmission to the remote site, the visual command information being visually interpretable by a user at the local or remote site;

a visual interpreter for decoding the overlaid video signal in order to determine the visual command information overlaid onto the incoming video signal and for outputting the decoded visual command information; and

a device controller for receiving visual command information decoded by the visual interpreter, the device controller generating a command for operating the device in accordance with the visual command information overlaid onto the overlaid video signal.

21. The apparatus as defined in claim 20 wherein the apparatus further comprises a monitor for displaying the overlaid video signal output by the visual signaller.

22. The apparatus as defined in claim 21 wherein the visual signaller further comprises:

a graphics locator means for manipulating a pointer overlaid onto the video signal to enable the operator to view the position of the visual command information; and

an icon selector to enable an operator to select at least one of a plurality of icons corresponding to at least one of a plurality of candidate commands to operate the device at the remote site.

23. The apparatus as defined in claim 20 wherein the visual signaller further comprises:

a graphics locator means for manipulating a pointer overlaid onto the video signal to enable the operator to view the position of the visual command information; and

an icon selector to enable an operator to select at least one of a plurality of icons corresponding to at least one of a plurality of candidate commands to operate the device at the remote site.

24. The apparatus as defined in claim 20 wherein the device to be operated at the remote cite is a controllable camera base for supporting a camera, where the controllable base is controllable for positioning the camera.

25. The apparatus as defined in claim 24 wherein the controllable base is operable to pan in a horizontal direction and tilt in a vertical direction.

26. A visual controller for receiving a video signal and generating visually interpretable control commands related to the received video signal and inserted into an outgoing video signal for controlling a remote device, comprising:

a selector for inserting into the outgoing video signal at least one visually interpretable gesture, the gesture being decodable into a partial control command for controlling the remote device; and

a gesture positioner for positioning the at least one selected gesture onto the outgoing video signal where the position of the at least one selected gesture supplements the control command in accordance with the position of the at least one selected gesture.

27. The visual control device defined in claim 26 further comprising a display means for displaying the incoming video signal and the at least one selected gesture overlaid onto the incoming video signal, wherein an operator may view on the display means the command information to be inserted into the outgoing video signal.

28. The visual control device as defined in claim 26 wherein the outgoing signal includes the incoming video signal having overlaid thereon the gesture information inserted by the visual control device.

29. The visual control device as defined in claim 26 further comprising a video overlay mixer for receiving the incoming video signal and a signal representing the at least one selected gesture positioned by the gesture positioner, and for providing the outgoing video signal including the incoming signal having inserted thereon the at least one selected gesture.

30. The visual signaller as defined in claim 26 wherein the gesture positioner is one of a joy stick, a graphics locator pad, and a track ball.

31. A visual interpreter for detecting the presence of at least one gesture in an incoming video signal and generating device control commands in accordance with the control commands in the incoming video signal, comprising:

means for extracting the at least one gesture from the incoming video signal, the gesture being visually interpretable when displayed on a display means; and

means for interpreting the extracted gesture and providing device control commands to a device controller, the device controller operating the controlled device in accordance with the device control commands.

32. The visual interpreter as defined in claim 31 wherein the incoming video signal is an analog signal, and the means for extracting further comprises a digitizer for creating a digitized representation of the analog signal and memory for storing the digitized representation of the analog signal.

33. The visual interpreter as defined in claim 31 wherein the incoming video signal is a digital signal, and the means for extracting further comprises a memory for storing the digital incoming video signal.

34. The visual interpreter as defined in claim 33 further comprising signal processing means for conditioning the incoming video signal.

35. The visual interpreter as defined in claim 31 further comprising a memory for storing the incoming video signal.

36. The visual interpreter as defined in claim 31 further comprising a gesture pattern memory having stored therein at least one of a plurality of candidate gestures.

37. The visual interpreter as defined in claim 31 wherein:

the incoming video signal is an analog signal, and the means for extracting further comprises a digitizer for creating a digitized representation of the analog signal and memory for storing the digitized representation of the analog signal; and

a gesture pattern memory having stored therein at least one of a plurality of candidate gestures, where the means for extracting the at least one gesture compares at least a portion of the digitized representation of the analog signal with at least one of the plurality of candidate gestures.

38. The apparatus as defined in claim 37 further wherein:

the incoming video signal is a digital signal, and the means for extracting further comprises a memory for storing the digital incoming video signal; and

a gesture pattern memory having stored therein at least one of a plurality of candidate gestures recognized by the means for extracting, where the means for extracting the at least one gesture compares at least a portion of the digitized representation of the analog signal with at least one of the plurality of candidate gestures.

39. An apparatus for enabling an operator to remotely control a device, comprising:

a received video signal input to the apparatus, the video signal having an object located therein;

means for generating command information in the form of a visually iconic gesture overlaid onto the received video signal to define an output video signal, where the visually iconic gesture has a meaning which varies in accordance with the position of the visually iconic gesture relative to the object, and where the visually iconic gesture is interpretable by the operator;

processing means for receiving outgoing video signal and decoding the outgoing video signal into the command information conveyed by the visually iconic gesture and the position of the visually iconic gesture relative to the object in the incoming video signal, the processing means outputting decoded command information to control the object on which the visually iconic gesture is located; and

device control means for receiving the decoded command information output by the processing means and transforming the command information into control commands for operating the object in accordance with the command information conveyed in the visually iconic gesture.

40. The apparatus as defined in claim 39 further comprising:

an imaging device to provide the received video signal, the imaging device scanning and capturing a portion of a predetermined area defined in accordance with a specific orientation of the imaging device;

memory having stored therein candidate objects and a corresponding location of each candidate object, the object being determinable in accordance with its predetermined location;

mapping means for correlating the orientation of the imaging device and the objects stored in memory in accordance with the location of the object and the orientation of the imaging means.

41. A method for controlling a remotely located device, comprising the steps of:

generating command information in the form of a visually iconic gesture overlaid onto an outgoing video signal, the visually iconic gesture being interpretable by the operator and conveying command information in accordance with its location within the outgoing video signal;

processing the outgoing video signal having the visually iconic gesture to decode the iconic gesture into the command information conveyed by the iconic gesture, the processing means outputting the command information; and

transforming the command information into control commands for operating the device in accordance with the command information conveyed in the visually iconic gesture.
 Description Submit all comments and votes
 


This invention relates generally to video telecommunications to control remotely located devices. More particularly, this invention relates to inclusion of iconic information within the video signal received from the remote site, where the video signal is returned to the remote site and the iconic information is decoded into control information to control the remotely located device.

BACKGROUND OF THE INVENTION

The availability of relatively inexpensive transmission of video signals has increased dramatically over the last few years. Specifically, many phone and cable television companies have been allowed by the Federal Communications Commission (FCC) to install bundled telephone and cable television systems using the most recent technologies, typically fiber optic cables. With the proliferation of such systems and the bundling of telephone and video communications, it is readily apparent that in the near future traditional audio telephone communications will most likely migrate towards audio/video communications systems whereby an operator at one site receives audio and video signals which enables the operator to both see and hear what occurs at a remote site, and vise versa.

Initial systems will include merely fixed cameras and video monitors. A method and apparatus for controlling pointers in video images at remote sites from local sites is disclosed in U.S. patent application Ser. No. 07/989,126, filed Dec. 11, 1992, entitled System and Method for Teleinteraction, now U.S. Pat. No. 6,444,476, assigned to the Assignee of the present invention and herein incorporated by reference. Other improved systems will most likely include the capability of tracking the operator around the room and enable the operator more freedom while conducting the televideo conference. In yet another configuration, however, one can readily envision where an operator at one site may desire to control the camera at a remote site without requiring the operator at the remote site to reposition the camera. Such systems lend themselves easily to, for example, home security systems where users could initiate a telephone conference to their own home and select and pan through rooms where the cameras are located in order to check the condition of those particular rooms. For example, a user may want to initiate a telephone conference with their own home on an Integrated Services Digital Networks (ISDN) phone-based video link when no one is home. The user could then pan and tilt the remote video camera to view the room and insure its condition.

Presently, some security camera control systems do enable control of cameras remote to an operator's location, and the operator can manipulate controls such as a mouse or a joy stick to cause the camera to pan and tilt in the specified direction. These systems, however, use a separate, digital communication link to send encoded camera control signals to the remote camera, separate from the video signals. A disadvantage of such systems is that the separate communication link limits control of the camera to a specific configuration of the camera and the operator control into a specific, unique system of hardware and low level communications protocols.

In other video communications systems, the idea of imbedding control information on a separate, digital communication link may be overcome in video communications systems by encoding control signals into the actual video stream. These systems typically utilize the currently unused scan lines in the video frames and provide control information which is used by decoding devices at receiving site to add additional information to the picture. For example, in closed caption systems, the unused video lines are used to encode information which is decoded at the receiving site. A decoding device decodes and inserts the decoded information onto the screen so that subtitles appear which correspond to the words being spoken by the characters in the picture. Such systems have proved useful to the hearing impaired and are relatively effective for systems where the receiving unit receives the video signal directly from the source. Examples of such systems include a local television station broadcasting a signal through an antenna where the signal is received by the antenna of a television set or a cable television system receiving and signals with an antenna located at the broadcasting source. That is, the communication link is a direct link with minimal compression or transfer through switching networks.

However, other audio/video communications arenas in which this invention is most useful involve substantial compression, transmission through digital switching networks, and decompression. Such signal decompression, conditioning, switching, and the like typically degrade the information encoded into the unused scan lines of a closed caption video signal so that the signal cannot reasonably be coded into the information to be placed on the screen as subtitles. Furthermore, such communication may occur across several different video format standards (such as NTSC, PAL, SECAM, and HDTV). The conversion of the video signal from one to another of these standards can potentially also modify or remove any unused scan line codes. Thus, it is desirable to provide a more robust approach to embedding control information into transmitted digital signals where the control information experiencing degradation through multiple compression, standards conversions, and/or switching networks remains useable at the remote site to effect the desired control.

Further, current systems which offer any form of embedded control information in audio or video signals also suffer from the requirement that expensive equipment be located at both the sites generating the control information and sites receiving the control information. Such configurations typically are turnkey systems where specific control information generating devices at the local site must match exactly the devices which decode the control information at the remote site. Thus, the signalling and decoding devices tend to be function specific rather than enabling universal control of a number of devices. Ideally, but hereto not yet achieved, the site generating the control information includes a multipurpose device which desirably controls a number of varied remote sites and remote devices. In this manner, a user need not purchase a specific signalling device and place a specifically designed decoding and control device for use at the remote site. Thus, it is further desirable to provide a relatively inexpensive, universal signalling device which is operable to control a number of remote devices and to provide a system having remote device controllers which can interpret a variety of incoming control information from a variety of control information generating sources.

Thus, it is an object of this invention to provide a communication link between remote and local sites which merely requires a two-way video link, which would normally be present rather than two independent, communication links, thereby limiting hardware requirements to relatively inexpensive hardware.

It is a further object of the invention to intercept a visual information stream from the remote site, add iconic visual information to the information stream using a visual signaller that encodes the desired control actions, and send the augmented video signal back to the original site.

It is a further object of the invention that an image processor decodes the visual control information using a visual interpreter at the remote end to determine the required control actions for a remote device by means of remote device control instructions communicated in iconic visual form.

It is a further object of this invention to provide visually iconic information in a format related to the desired control responses to facilitate use and to make use of the command generator to make device control more intuitive to the operator.

It is a further object of this invention to provide iconic information which is visually readable by an operator at a local site and visually readable by an operator at a remote site so that the visually iconic information when read by a human defines to the human the desired control response.

It is a further object of the invention to provide at a local site an apparatus for generating control information to control a device at a remote site where the apparatus generates information in iconic format visually decipherable by a human being and the iconic information describes general and specific operational goals for the remote device.

It is a further object of the invention that the apparatus for generating control information generates iconic information which is decipherable by a plurality of remote device controllers which are operable to control specific devices, and iconic information generated by the apparatus for generating control information indicates operational goals for the controlled device where the operational goal information depends on the particular device being controlled.

SUMMARY OF THE INVENTION

A method and apparatus for enabling an operator to receive an incoming video signal and to imbed command information in an outgoing video signal in order to remotely control a device. The apparatus includes a device for generating command information in the form of a visually iconic gesture. The visually iconic gesture is interpretable by the operator and conveys command information in accordance with its appearance and location within the outgoing video signal. A processor receives the outgoing video signal having the visually iconic gesture and decodes the iconic gesture into the command information conveyed by the iconic gesture. The processor then outputs the command information to a device controller. The device control receives the command information output by the processor and translates the command information into control commands for operating the device in accordance with the command information conveyed in the visually iconic gesture.

Further objects, features and advantages of the invention will become apparent from a consideration of the following description and the appended claims when taken in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of the visual, remote control system disclosed according to the principles of the present invention;

FIG. 2 is an expanded block diagram of the visual signaller depicted in FIG. 1 for generating visual, iconic control information;

FIG. 3 is an expanded block diagram of the visual interpreter depicted in FIG. 1 for extracting and identifying the iconic, visual information from the visual stream to control the remotely located device;

FIG. 4 is an expanded block diagram of the icon extractor shown in FIG. 3 for extracting iconic information from the video signal overlaid by the visual signaller;

FIGS. 5a, 5b and 5c depict pattern masks which represent portions of recognizable icons which are used to identify the icons which could be overlaid by the visual signaller;

FIG. 6 is a flow diagram for comparing portions of the digitized image to the pattern masks stored in pattern mask memory;

FIG. 7 is a flow diagram of the convolution performed by the icon extractor in order to determine correlations between the digitized image and the pattern mask stored in pattern recognition memory;

FIG. 8 is a state diagram for defining the states of the icon recognizer processor;

FIG. 9 is an expanded block diagram of the icon sequence interpreter depicted in FIG. 3, including a state machine, timers, and memory;

FIG. 10 is a state table defining the states of the state machine FIG. 9 for executing a centering command;

FIG. 11 is a transition table corresponding to the state table of FIG. 10 and defining the state of the state machine of FIG. 9 for executing a centering command;

FIG. 12a and 12b are examples of the change in the view area upon executing a centering command;

FIG. 13 is a state diagram defining the states of the state machine of FIG. 9 for executing a pan/tilt command;

FIG. 14 is an example of the view area of the monitor displayed when executing a pan/tilt command;

FIG. 15 is a state diagram defining the states of the state machine of FIG. 9 for executing a move command;

FIGS. 16a and 16b are examples of the view area of the monitor displayed when executing a move command;

FIG. 17 is a side view of the controllable camera base for holding the camera and shows a detailed view of the drive arrangement for the pan motor;

FIG. 18 is a front view of the controllable camera base for holding the camera and shows a detailed view of the drive arrangement for the tilt motor;

FIG. 19 is an expanded block diagram for the device controller of FIG. 1 for transforming visual command information to motor control information to manipulate the camera base;

FIG. 20 depicts a zoom command for effecting zoom-in and zoom-out control of the camera in addition to manipulating the camera base;

FIG. 21 depicts a mobile camera base which translates across the surface on which the base rests and which has an integral visual interpreter and device controller;

FIG. 22 depicts a robot arm and a camera which are both controlled by an integral visual interpreter and device controller; and

FIG. 23 depicts a remote site with a single robot arm and multiple cameras, each camera being associated with a particular local user to enable two local users to view a remote site individually and effect control of the robot arm at the remote site;

FIG. 24 depicts a remote camera site having an object in the view field of the camera which may be controlled in accordance with control commands from the visual signaller;

FIG. 25 depicts a viewing area of the television monitor located at the local site for controlling the device found at the remote site of FIG. 24;

FIG. 26 depicts the configuration of FIG. 1 having a re-registration module for comparing the signal sent to the local site from the remote site from the remote site with the return signal received from the local site;

FIG. 27 depicts a block diagram of the re-registration module depicted in FIG. 26; and

FIG. 28 depicts a block diagram with the re-registration module depicted in FIG. 25 including the additional feature of having CAD models with which to compare the signals sent and received by the remote site for identifying objects of known shapes at predetermined locations within the CAD models.

DETAILED DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts a block diagram of the remote control system 10. The remote control system 10 includes a local site 12 and a remote site 14. By convention, local site 12 refers to the controlling site or the site where an operator resides and directs control of a device at the remote site 14. Similarly, remote site 14 typically refers to the site where the device to be controlled is located and where an operator may, but need not be stationed. A two-way video communications link 18a and 18b interconnects local site 12 and remote site 14. Video communication links or lines 18a and 18b combine to form a bi-directional, video communication line for transmitting video signals in analog or digital format from remote site 14 to local site 12, depicted as video link 18a, and for transmitting a video signal in analog or digital format from local site 12 to remote site 14, depicted on video link 18b. It should be noted that while shown separately, video links or lines 18a and 18b represent a typical, bi-directional audio/video communication line, as are well known in the art. Of course, it would be understood by one skilled in the art that video communications lines 18a and 18b are separate lines bundled to effect bi-directional communication as shown in FIG. 1 and may optionally provide audio and visual communications, depending upon the particular application.

Because communications between local site 12 and remote site 14 may not normally occur over direct, hard-wired lines, but may occur over switching networks which may be multiplexed, the video signals transmitted between local site 12 and remote site 14 are shown as conditioned for improved for more accurate or efficiency transmission then decoded. That is, in order to transmit a video signal from remote site 14 to local site 12 and from local site 12 to remote site 14, the video signals may be converted from analog to digital format (or may already be in digital format) compressed using data compression algorithms well known in the art in order to decrease the bandwidth requirement of the channel on which the video signals are transferred. For example, the video signals may be transmitted over a digital switching network in order to allow for compression and more efficient data transfer using compression algorithms. Communications interface transmitters 22a and 22b at local site 12 and remote site 14, respectively provide the digitization and data compression for transmission to the remote site 14 and local site 12. The video signals received by local site 12 and remote site 14, respectively, are then decompressed and optionally reconverted to analog format. The digitization and compression described herein may typically be useful for implementation on a T1 or ISDN telephone teleconference link, or other digitized, encoded, compressed (then decompressed at the receiving end) communication links. Such digitization and compression is also useful for internet communications using CUSeeMe formats or slow frame video.

Referring to local site 12, communications adapter 20a receives and decodes the incoming video signal and outputs a decoded video signal which is input to visual signaller 16. Visual signaller 16 comprises a graphics locator pad 26a (or joy stick or track ball controller) and a plurality of icon selectors 26b. Output from the visual signaller 16 is input into television monitor 28 on output line 30a and via output video line 30b. Visual signaller 16 outputs a video signal similar to the incoming video signal with the addition of one or a number of graphics pointers or icons overlaid onto the video signal. The operator uses the graphics locator pad 26a to position a pointer overlaid on the incoming video signal 24, the result of which is displayed on monitor 28. Visual signaller 16 also includes one or a plurality of icon selectors 26b. Icon selectors 26b enable the operator to select a desired function by pressing a selected one or more of the icons in the icon selector 26b associated with that particular function. To be described further herein, icon selector 26b may modify the icon or pointer overlaid onto the incoming video signal and appearing on the video monitor 28. Visual signaller 16 also outputs a signal, in analog or digital format, depending upon the implementation, on output line 30b which is input to communications interface transmitter 22a. The video signal output by visual signaller 16 on output line 30b may optionally be a video overlaid signal comprising an icon overlaid by visual signaller 16 onto the incoming video signal received from the remote site 14. Alternatively, the video signal output on line 30b may consist only of the icon selected by icon generator 26b as positioned by the graphics locater pad 26a, but with a background of a predetermined solid color rather than a background of the video signal received from remote site 14.

Referring to remote site 14, camera 32 captures a video image which it then outputs on line 34 to communications interface transmitter 22b for optional compression, digitization, or the like (depending upon the application) and transmission to local site 12. Camera 32 may be any of a number of video cameras known in the art may output a signal in either analog or digital format, depending upon the particular application. In will be understood by one skilled in the art that the video signals may be either analog or digital signals, depending upon the particular implementation