User interface control for communication system

What is claimed:

1. A system comprising:

first and second stations at respective first and second locations, the first station comprising a first display and a first user input device, the second station comprising a second display and a second user input device;

first and second communication devices, the first communication device being at the first location with the first station and the second communication device being at the second location with the second station;

switching means for performing a switching operation in response to a switch control signal; the switching operation affecting a connection between the first and second communication devices; and

control means connected for receiving signals from the first and second user input devices requesting switches in the connection; the control means further being connected for providing the switch control signal to the switching means in response to a respective one of the signals from the first and second user input devices; the control means comprising a connection data structure containing connection data indicating the connection; the control means updating the connection data structure after providing the switch control signal; the control means further being connected for providing update data so that the first and second displays each present a respective representation of the connection between the first and second communication devices that is consistent with the updated connection data structure.

2. The system of claim 1 in which the control means comprises a data network for providing the switch control signal to the switching means, the switching means comprising:

a switch for making and breaking a set of connections including the connection between the first and second communication devices; and

a switching server for receiving the switch control signal from the data network and for providing commands to the switch based on the switch control signal.

3. The system of claim 1 in which the first and second communication devices are positioned in first and second groups, respectively; the first station including the first group and the second station including the second group.

4. The system of claim 3 in which the first communication device comprises a camera and the second communication device comprises a monitor, the switching means comprising a video switch for switching the connection between the first and second communication devices so that the monitor displays an image which is based on a video signal from the camera.

5. The system of claim 1 in which the control means provides the up date data after the switching means switches the connection so that the respective representations presented by the first and second displays each represent the switched connection.

6. The system of claim 1 in which the control means is further for modifying the switch control signal that is provided based on the respective one of the signals from the first and second user input devices, the control means modifying the switch control signal in response to a modification request signal from the first user input device requesting a modification.

7. The system of claim 6 in which the control means further comprises a signal data structure containing data for generating the switch control signal based on the respective one of the signals from the first and second user input devices, the signal data structure being accessed in response to the modification request signal.

8. The system of claim 1 in which the control means further operates, upon receiving a respective one of said switch request signals from each of the first and second user input devices, to provide a corresponding switch control signal based on the respective switch request signal from the first user input device before providing a corresponding switch control signal based on the respective switch request signals from the second user input device.

9. The system of claim 1, further comprising a third station, the third station comprising a third display; the control means further being connected for providing the update data so that the third display presents a representation of the connection that is consistent with the updated connection data structure.

10. A system comprising:

first and second communication devices;

switching means connected for performing a respective switching operation in response to a switch control signal; the respective switching operation switching a connection between the first and second communication devices;

a user interface that includes a first display and a first user input device; and

control means connected for receiving a signal from the first user input device requesting a switch in the connection; the control means further being connected for providing the switch control signal to the switching means in response to the signal from the first user input device; the control means further being connected for providing update data to the user interface after providing the switch control signal, the update data indicating the connection after the respective switching operation;

the user interface, in response to the update data, providing a representation of the connection indicated by the update data on the first display, the user interface providing the representation by providing first and second device display features and a connection display feature on the first display, the first and second device display features representing respectively the first and second communication devices, each of the first and second device display features having a respective display location, the respective display location of each of the first and second device display features being different from the respective display location of the other, the connection display feature representing the connection between the first and second communication devices, the connection display feature extending between the respective display locations of the first and second device display features.

11. The system of claim 10 in which each of said first and second device display features is a display object, said connection display feature being a link between display objects.

12. The system of claim 10 in which the signal from the first user input device requesting a switch is a sequence of switch request signals, the switch control signal being one of a plurality of switch control signals provided in response to the signal from the first user input device, the switch control signals and the switch request signals each having a respective semantics, the respective semantics of the switch request signals being different than the respective semantics of the switch control signals, the control means operating to generate the switch control signals equivalent to the switch request signals.

13. The system of claim 10 in which the control means comprises a data network for providing the switch control signal to the switching means, the switching means comprising:

a switch for making and breaking a set of connections including the connection between the first and second communication devices; and

a switching server for receiving the switch control signal from the data network and for providing commands to the switch based on the switch control signal.

14. The system of claim 10 in which the first and second communication devices are positioned in first and second groups, respectively; the first and second groups being apart from each other; the user interface being positioned with the first group.

15. The system of claim 14 in which the first communication device comprises a camera and the second communication device comprises a monitor, the switching means comprising a video switch for switching the connection between the first and second communication devices so that the monitor displays an image that is based on a video signal from the camera.

16. A system comprising:

a plurality of stations, each station comprising a respective display, a respective user input device, and a respective communication device; each station being at a respective location;

switching means for performing a switching operation in response to a switch control signal; the switching operation affecting a connection between first and second ones of the respective communication devices; and

control means connected for receiving signals from the respective user input devices requesting switches in the connection; the control means further being connected for providing the switch control signal to the switching means in response to one of the signals from the respective user input devices; the control means comprising a connection data structure containing connection data indicating the connection between the first and second respective communication devices; the control means updating the connection data structure after providing the switch control signal; the control means further being connected for providing update data so that the respective displays each present a respective representation of the connection between the first and second respective communication devices that is consistent with the updated connection data structure.

17. A system comprising:

first and second stations, the first station comprising a user interface that includes a first display and a first user input device, the second station comprising a second display and a second user input device;

first and second communication devices; the first and second communication devices being positioned in first and second groups, respectively; the first station including the first group and the second station including the second group;

switching means for performing a switching operation in response to a switch control signal; the switching operation affecting a connection between the first and second communication devices; and

control means connected for receiving signals from the first and second user input devices requesting switches in the connection; the control means further being connected for providing the switch control signal to the switching means in response to a respective one of the signals from the first and second user input devices; the control means comprising a connection data structure containing connection data indicating the connection; the control means updating the connection data structure after providing the switch control signal; the control means further being connected for providing update data so that the first and second displays each present a respective representation of the connection that is consistent with the updated connection data structure;

the user interface being responsive to the update data provided by the control means for providing a respective representation of each of said first and second groups on the first display and for providing a representation of the connection between the first and second communication devices on the first display as a connection between the respective representations of the first and second groups.

18. The system of claim 17 in which the user interface is further for including in said respective representation of the first group a respective visual cue identifying said first group.

Description Submit all comments and votes

BACKGROUND OF THE INVENTION

The present invention relates to techniques by which a user is able to control a video and audio communication system. More specifically, the invention relates to techniques that provide each user with a display enabling that user to control a switch that makes and breaks connections between communication devices.

Pressman et al., U.S. Pat. No. 4,645,872, describe a videophone network system which includes a control console at each of a plurality of stations. Each station also includes video and audio inputs and outputs, with central switching devices connecting the inputs and outputs. As shown in FIG. 3, each control console includes pushbuttons for selecting functions and stations. Two LEDs assocaited with each station's pushbutton indicate the status of that station. The LEDs indicate whether the station is unconnected, connected or busy, for example. A user can use the pushbuttons to control the switching devices, and the LEDs provide helpful information to the user.

Boerger et al., U.S. Pat. No. 4,650,929, describe a similar videoconferencing system. FIG. 12 shows a controller unit at a participant location, including a display. The participants receive checkback information on the status of the controls and other components of the network through this display, which may include signal lamps, alphanumeric display and so forth.

Fields, U.S. Pat. No. 4,400,724, describes a teleconference system for multiple stations in which one object is to provide a virtual conference space or a naturalistic rendition of a physical space to simulate a face-to-face conference. In other words, cameras and displays are positioned so that the conferees have the same relative locations at each station even though they are at separate locations. Each conferee has a control box with control switches governing interconnections between stations through relays at a central control room.

Face et al., U.S. Pat. No. 3,668,307, describe a two-way closed circuit television system in which a terminal has a control unit for communication with the control center and thence to other terminals under control of the control center, so that the control center delegates part of its supervisory authority to a selected terminal for a period of time. Poirier et al., U.S. Pat. No. 4,054,908, describe a videotelephone conference system in which switching is controlled by signals based on the loudness of speech signal for a microphone or cumulatively for a room. Schober, U.S. Pat. No. 4,264,928, similarly describes a conference video system which uses microphones coupled through circuit elements to a servomotor which positions a mirror to focus on a speaker and aims the camera field of vision toward active audio.

D. C. Swinehart, L. C. Stewart and S. M. Ornstein, "Adding Voice to an Office Computer Network", Proceedings of IEEE GlobeCom '83 Conference, Nov. 1983, reprinted as Xerox PARC CSL-83-8, describe a telephone system in which workstations connected by an Ethernet network manage voice switching. A user may use a workstation to provide commands to a telephone control server which in turn controls switching between standardized telephone peripherals connected to the network.

It would be advantageous to have improved techniques for user control of a video and audio system.

SUMMARY OF THE INVENTION

The present invention provides techniques which enable a user to control the switching of a communication network based on information about connections between stations. Furthermore, the invention provides techniques for controlling the presentation of the connection information to the user, permitting a wide variety of user interfaces for controlling such a network.

One aspect of the invention is based on the recognition that a user of a conventional communication network lacks adequate information about the existing connections. This aspect is based on the further recognition that this problem can be solved with a system which includes a display, such as a cathode ray tube (CRT), and a user input device, such as a keyboard and mouse. A control means provides data so that the display presents a representation of the connections. The control means also provides a sequence of switch control signals based on a sequence of signals from the user input device requesting a switching operation, and a switching means performs switching in response to these switch control signals. As a result, a user has adequate information to request switching of connections. When the switching means performs switching in response to the switch control signals, the displayed representation can be updated to represent the switched connections.

Another aspect of the invention is based on the recognition that information about connections between a number of devices is typically difficult for a human to comprehend. This aspect is based on the discovery that a human can readily comprehend connection information if the devices being connected are represented by display features at separate locations on the display and the connections between them are represented by display features extending between the device representations. If the devices are grouped at a number of stations, for example, each station could be represented by a corresponding display feature, each station's display feature being at a location on the display different from the locations of the display features of other stations. If devices at two stations are connected, the connection could be represented by a line or other link between those stations' display features. As a result of viewing such a representation of the connections, the user has a sense that the stations are positioned at relative locations within a single space.

A closely related aspect is based on the recognition that the development and improvement of techniques for graphically representing connections is an iterative process, and further that no single technique is likely to provide a representation which is suitable for every use. This aspect of the invention solves these problems by enabling the user to modify the sequence of switch control signals provided in response to a switch request signal sequence. Each switch control signal is one of a set of standard commands to the switching means which can be generated from a wide variety of user input signals, and the user can modify the manner in which the control means provides these standard commands based on the user input signals. Also, the invention can be implemented on a system that permits the user to modify the manner in which the devices and connections between devices are represented on the display. As a result, a system according to the invention can be used to modify and develop an unlimited variety of user interface techniques for communication networks.

Another closely related aspect is based on the recognition that a user typically thinks of achieving a desired state of connections in terms that are different from switching operations. Typically, there is not a one-to-one relationship between the user input signals and the appropriate switch control signals to perform the requested switch operation. The user is likely to describe a desired state in abstract terms unrelated to switching, such as by naming a group of people in various locations who should be able to see and hear each other and who sould be protected from interruptions from elsewhere by an appropriate level of privacy. The switch control signals, on the other hand, must define switching operations that cause those changes in the state of the connections that are necessary to provide a desired result. A user is typically occupied with matters other than switching, and would be distracted by translating a desire for a new state of connections into switch control signals. A system that requires the user to do this translation is therefore inefficient. This problem can be solved by a system that receives a sequence of signals in terms that are convenient to the user and that translates those signals into an equivalent switch control signal sequence, freeing the user from thinking about the details of switching. A signal data structure is provided that, for each user input signal requesting a switching operation, provides data to generate the correct sequence of switch control signals. Furthermore, the contents of the signal data structure can be changed to modify the switch control signal sequence provided based on a given sequence of signals from a user.

The representations of connections presented to all users should be consistent at any given time. This problem can similarly be solved by a data structure, in this case a connection data structure containing data indicating the connections. In addition, the switching means can include one or more switching servers on the data network with the workstations, each switching server controlling at least one switch; the connection data structure can be updated based on data from the switching servers. The control means in turn provides data so that all the displayed representations are consistent with the connection data structure.

In a network with several stations, two users could simultaneously request inconsistent switching operations. This problem can be solved with control means that operate to serialize user input signals. If the control means is implemented in a centralized manner, it can serialize user input signals based on the sequence in which it receives them. But if the control means is distributed, it can serialize the user input signals through a shared file that has a lock that can only be held by one workstation at a time. As a result, signals from the users are handled serially, in the sequence in which the lock is held.

The invention thus provides communication system control through a user interface which displays the state of connections of the system, facilitating switching in accordance with the user's desires. More than one user may control the system, with a single data structure containing data indicating the state of switch connections. The handling of switch requests avoids interference between users. A user can also modify the manner in which switch control signals are provided based on user input signals.

These and other objects, features and advantages of the invention will be more fully understood from the following description, together with the attached claims and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram showing general functional components of a communication system according to the invention.

FIG. 2 is a flow chart showing general steps in the operation of the system of FIG. 1 according to the invention.

FIG. 3 is a block diagram showing in more detail components of a communication system according to the invention.

FIG. 4A is a schematic view of a representation of connections between communication devices appropriate to the system of FIG. 3.

FIG. 4B is a schematic view of another representation of connections between communication devices appropriate to the system of FIG. 3.

FIG. 5 is a flow chart showing operations of the control means of FIG. 3.

FIG. 6 is a schematic representation of an entry in the signal data structure of FIG. 3.

FIG. 7 is a schematic representation of an entry in the connection data structure of FIG. 3.

FIG. 8 is a flow chart showing operations of the switching server of FIG. 3.

FIG. 9 is a schematic block diagram of a multi-switch, multi-site communication system according to the invention.

FIG. 10 is a schematic block diagram illustrating components of the control means, switching means and user interface in the system of FIG. 9.

DETAILED DESCRIPTION

A. General Description

FIGS. 1, 2, 3, 4A and 4B show general features of the invention, together with some more detailed features of an implementation of the invention. FIG. 1 shows the broad functional components of a communication system 10 according to the invention. FIG. 2 shows a general sequence of steps followed by the control means of system 10. FIG. 3 shows in more detail the components of one implementation of communication system 10. FIGS. 4A and 4B show examples of representations of a state of connection of system 10 appropriate for the implementation of FIG. 3.

Communication system 10 in FIG. 1 includes control means 12 and switching means 13. Signal paths link control means 12 to display 14a and to user input device 14b, through which a user can provide input data to system 10. Signal paths also link switching means 13 and a number of communication devices, such as pick up device 15a for picking up a signal and rendering device 15b for rendering a signal into perceptible form. As discussed below, pick up device 15a could be a microphone or camera, while rendering device 15b could be a speaker or display monitor. Signal paths could link switching means 13 to other types of communication devices, including recording and playback devices and other signal processing devices. Switching means 13 performs its switching function by selectively providing signal path connections between communication devices in response to switch control signals from control means 12. Switching means 13 can be any combination of hardware and software components capable of responding to switch control signals to obtain the appropriate resulting connections.

Control means 12 performs a control function which gives the user control of the connections provided by switching means 13. FIG. 2 shows the way it does so in the form of a general sequence of steps. In box 16, control means 12 receives a signal from user input device 14b. If the test in box 17 determines that those signals request a change in the connections between communication devices, control means 12 provides appropriate switch control signals to switching means 13 based on the user input signals, in box 18a. Control means 12 also provides update in box 18b so that display 14a presents a current representation of the state of connections between communication devices. As a result, the user can control the system more effectively based on information about the state of connections. If control means 12 provides the update data after switching means 13 changes connections in response to the switch control signals, the representation of the state of connections is more likely to be accurate at all times, resulting in better control. Finally, if the test in box 17 determines that the user input signal is not a signal requesting a switch operation, control means 12 handles the user input signal by performing the appropriate non-switch operation.

FIG. 3 shows a multi-station implementation of communication system 10 with a number of additional details. As shown, system 10 includes control means 20 and switching means 30, corresponding respectively to control means 12 and switching means 13 in FIG. 1. A user exchanges signals with control means 20 through user interface 40, and with switching means 30 through communication devices 50. User interface 40 and devices 50 are located together at station 60, designated "Station A", and, as shown, system 10 may include other stations 62, 64, and so forth, respectively designated "Station B", "Station C", etc. Each station may include at least one user interface and a number of communication devices. Optionally, some stations may have only a user interface with no devices and other stations may have only communication devices with no user interface.

Control means 20 performs its control function generally in the way set forth in FIG. 2, through the operations of controller 22. Controller 22 exchanges signals with each user interface in system 10, including user interface 40 at station 60. In the illustrated implementation, control means 20 also includes connection data structure 24, containing data indicating the state of connection of system 10, and signal data structure 26, containing data indicating the appropriate response to each signal from one of the user interfaces. If a user input signal is received requesting a switching operation or requesting another operation requiring a response from switching means 30, controller 22 accesses signal data structure 26 to obtain a sequence of switch control signals that it then sends to switching means 30. In the illustrated implementation, switching server 32 receives the switch control signals and responds appropriately. If a switch control signal requests a switching operation, switching server 32 causes switch 34 to modify the existing connections accordingly. Switching server 32 also provides acknowledge and status signals to controller 22 in response to appropriate switch control signals. Controller 22 updates connection data structure 24 based on status signals from switching server 32, and provides update data to the user interfaces so that the representations they present to users are kept current.

User interface 40 includes display 42, illustratively a CRT, and a user input device, illustratively keyboard 44 with mouse 46. User interface 40 may thus be the user interface of a conventional workstation. Communication devices 50 include camera 52 for picking up a visual image and providing a video signal, microphone 54 for picking up sounds and providing an audio signal, and monitor 56 for rendering both video and audio signals into perceptible sounds and images. These devices need not have a direct connection to user interface 40, but are positioned so that a user can provide video and audio signals through camera 52 and microphone 54 while viewing monitor 56 and while facing user interface 40.

During operation of system 10, switching means 30 performs its switching function by selectively providing connections between the communication devices of system 10 in response to switch control signals from control means 20. Control means 20 provides data to display 42 so that it presents a representation 70 of these connections to the user. The data includes updates so that representation 70 reflects the current state of connections in system 10, in accordance with the data in connection data structure 24. As a result, the user can see the current state of connections.

FIG. 4A shows representation 70, a text-based representation of connections. Each of the stations in FIG. 3 is represented in FIG. 4A by the corresponding character, and each connection between the communication devices of two stations is represented by a line of text in window 72. Each line of text includes a source character representing the station whose pick up devices are connected, a destination character representing the station whose rendering devices are connected, and an arrow from the source character to the destination character representing the connection itself. Window 74 is a prompter window within which the user can request a new connection by indicating a source and a destination. Cursor 76 in window 74 is the current input point. Rather than a single window, two prompter windows could be provided, one for the source character and one for the destination character. In either case, the user can use keyboard 44 and mouse 46 to request a desired connection between two stations.

FIG. 4B shows representation 80, a graphical representation of connections. Each of the stations in FIG. 3 is represented in FIG. 4B by a corresponding device display object, each device display object being at a separate location from the others. Thus each device display object can represent all the devices at a single location, which reduces the number of display objects and is intuitively logical, since there is ordinarily no need to show connections among pick up and rendering devices at the same location. Each device display object could instead represent a single device or an arbitrary group of devices. Rather than showing each group of devices as an object, a display feature could be provided to show each station as a room or other region within a group of similar regions, with the device display objects of a given station being shown together in the respective region.

Each device display object 82, 84 and 86 includes a visual cue identifying the station it represents, specifically a shape identifying the user at the station it represents. Display objects 82, 84 and 86 correspond respectively to stations A, B and C in FIG. 3. Any other appropriate visual cues could be used, including an image of a face for each station, a name, a character, a number, or any other identifier of each station.

In addition to device display objects, display 80 includes connecting links 92 and 94, each extending between two of the device display objects and each indicating the direction of connection, such as an arrow as shown in FIG. 4B. Connecting link 92 extends between objects 72 and 74, and its arrow indicates that at least one pick up device at Station A is connected to at least one rendering device at Station B, so that Station A can be viewed at Station B. Similarly, connecting link 94 extends between objects 74 and 76, so that Station B can be viewed at Station C. The lack of other connecting links indicates that, for example, Station C cannot be seen or heard at the other stations, while a user at Station A cannot see or hear any of the other stations.

A user viewing representation 80 is also able to use keyboard 44 and mouse 46 to indicate a change in the connections. For example, a user at Station A could request a connection between a pick up device at Station C and a rendering device at Station A, permitting the user to see Station C. This request could be made, for example, by appropriate clicks of buttons on mouse 46 to select objects 86 and 82 and by depressing a key on keyboard 44 to indicate the desired operation of establishing a connection. Or this request could be made by a mouse button click to obtain a pop-up menu on which the operation could then be selected, after which the user would indicate with further cl