 |
Description  |
|
|
BACKGROUND OF THE INVENTION
1. Field of the Invention
This invention relates to multimedia multiparty communications systems and,
more particularly, to methods for interfacing with application programs
and for managing resources used for multimedia, multiparty communications.
The invention facilitates the management of resources for multimedia,
multiparty communications by providing a common framework for application
developers to use to indicate when resources are required for multimedia
multiparty communications and when resources are no longer required for
multimedia multiparty communications. The invention further facilitates
the management of other aspects of communications such as negotiation.
2. Description of the Related Art
The Windows Telephony Application Programming Interface (WTAPI) provided a
framework for programmers to develop user applications that use the
telephone. WTAPI permitted programmers to combine the graphical user
interface (GUI) capabilities of Microsoft Windows with telephony
applications. WTAPI also permitted integrated messaging for user
applications, which permits users to use their computers for electronic
mail, voice mail, and to receive incoming facsimiles.
Although WTAPI allowed programmers to produce certain types of telephony
applications, these applications could not take advantage of the many
other developments in the telecommunications industry.
For example, the telecommunications industry has been developing new
telecommunications networks called broadband networks, such as B-ISDN,
which can simultaneously transmit sound, video, and data. Broadband
networks may include new protocols for multimedia communications and
additional functions based on new network resources such as multimedia
hardware for mixing, combining, and transcoding sound, video, and data
from different sources.
Broadband networks will permit multimedia, multiparty calls between users
with heterogeneous desktop computers or other terminal equipment. User
applications cannot handle multimedia, multiparty calls using WTAPI. To do
so requires a framework, including sophisticated methods for managing
multimedia multiparty calls. Such a framework must permit several user
applications to combine many calls using multimedia hardware, regardless
of whether the hardware is in the broadband network or in the terminal
equipment (e.g. desktop computer). The framework must also permit several
independently-developed applications to share control of different aspects
of the same multimedia multiparty call.
WTAPI cannot support this framework because WTAPI does not logically
distinguish between different objects, such as multiple parties,
communications services, or communications links, that make up multimedia
multiparty calls. WTAPI users can only manage a call as a monolithic
object.
In addition, WTAPI requires application developers to write separate
program code to handle multiparty hardware which mixes, combines, and
transcodes sound, video, and data located in the broadband network and
locally at a desktop computer or local area network.
SUMMARY OF THE INVENTION
Accordingly, the present invention is directed to methods for interfacing
with application programs and for managing resources used for multimedia
multiparty communications that obviate one or more of the problems due to
limitations and disadvantages of the related art.
Features and advantages of the invention will be set forth in the
description which follows, and in part will be apparent from the
description, or may be learned by practice of the invention. The
objectives and other advantages of the invention will be realized and
attained by the method and apparatus particularly pointed out in the
written description and claims thereof as well as the appended drawings.
To achieve the objects of this invention and attain its advantages, broadly
speaking, this invention includes three components. The first component
sends and receives multiparty, multimedia calls. The second component
stores building blocks representing the multiparty, multimedia calls. The
building blocks include objects, which correspond to different aspects of
multiparty, multimedia calls, and relationships between the objects. The
third component monitors the objects and their relationships, and notifies
when changes occur in the objects or their relationships.
It is to be understood that both the foregoing general description and the
following detailed description are exemplary and explanatory and are
intended to provide further explanation of the invention as claimed.
BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings which are incorporated in and which constitute
part of this specification, illustrate a presently preferred
implementation of the invention and, together with the description, serve
to explain the principles of the invention.
In the drawings:
FIG. 1 is a block diagram of a system including the preferred
implementation;
FIG. 2 is a block diagram of multimedia hardware that may be included in
the personal computer and/or telecommunications network of FIG. 1;
FIG. 3 is an illustration of the object-oriented call model used by the
preferred implementation to represent different aspects of calls;
FIG. 4 illustrates a system table used by the preferred implementation to
represent the object-oriented call model illustrated in FIG. 3 in an
active database;
FIG. 5 illustrates a transaction table used by the preferred implementation
to manage transactions in the active database;
FIGS. 6-8 illustrate the concept of how the preferred implementation
updates the system and transaction tables of the active database. In
particular:
FIG. 6 illustrates a state of the active database before an update;
FIG. 7 illustrates a state of the active database after an update;
FIG. 8 illustrates both the states included in FIGS. 6 and 7 and the
information in the active database used for the update;
FIG. 9 is a flow diagram of the general operation of the preferred
implementation to create, modify, or delete a call;
FIG. 10 is a flow diagram of the general operation of the preferred
implementation to receive a call;
FIGS. 11-15 are examples of states of the active database, including a
system table and a transaction table, which are used to explain the
process of creating a call, as shown generally in FIG. 9;
FIG. 16 is a flow diagram of steps performed by an Infrastructure API
Support Component to get the state of an active database;
FIG. 17 is a flow diagram of steps performed by the Infrastructure API
Support Component to update an active database;
FIG. 18 is a flow diagram of steps performed by the Infrastructure API
Support Component to set triggers in an active database;
FIG. 19 is a flow diagram of steps performed by the Infrastructure API
Support Component to remove triggers from an active database;
FIG. 20 is a flow diagram of steps performed by the Infrastructure API
Support Component to deliver triggers fired by an active database;
FIG. 21 is a flow diagram of steps performed by a User API Support
Component to send messages to the Infrastructure API Support Component to
change the transaction state of an object in an active database;
FIG. 22 is a flow diagram of steps performed by the User API Support
Component to send messages to the Infrastructure API Support Component to
change the transaction state of an object in an active database;
FIG. 23 is a flow diagram of steps performed by the User API Support
Component to send messages to the Infrastructure API Support Component to
update an active database;
FIG. 24 is a flow diagram of steps performed by a Resource Manager
Component to send messages to the Infrastructure API Support Component to
set triggers in the active database;
FIG. 25 is a flow diagram of steps performed by the Resource Manager
Component to send messages to the Infrastructure API Support Component to
update the resource information in the active database;
FIG. 26 is a flow diagram of steps performed by the Resource Manager
Component to send messages to the Infrastructure API Support Component to
update the resource information in the active database;
FIG. 27 is a flow diagram of steps performed by the Transaction Manager
Component to send messages to the Infrastructure API Support Component to
set triggers in the active database;
FIG. 28 is a flow diagram of steps performed by the Transaction Manager
Component to send signalling messages to the telecommunications network;
FIG. 29 is a flow diagram of the steps performed by the Transaction Manager
Component to send signalling messages to the telecommunications network;
FIG. 30 is a flow diagram of steps performed by the Transaction Manager
Component when receiving an "INFORM" signalling message from the
telecommunications network;
FIG. 31 is a flow diagram of steps performed by the Transaction Manager
Component when receiving signalling messages, other than an "INFORM"
signalling message, from the telecommunications network; and
FIG. 32 is a flow diagram of the steps performed by the Transaction Manager
Component when receiving signalling messages, other than the "INFORM"
signalling message, from the telecommunications network.
DETAILED DESCRIPTION OF THE PREFERRED IMPLEMENTATION
Reference will now be made in detail to the preferred implementation of the
present invention as illustrated in the accompanying drawings. Whereever
possible, the same reference numbers will be used throughout the drawings
and the following description to refer to the same or like parts.
The present invention provides the capability for applications to send and
receive network calls and to store the building blocks of network calls,
including multiple objects and relationships between the objects. It also
provides monitoring of the stored objects and relationships, which
includes signalling applications of changes in the stored objects and
relationships. By using multiple objects to represent network calls,
multiparty multimedia network calls can be easily initiated, modified, or
released.
Applications send and receive network calls by specifying the calls in
terms of objects and their relationships. The details of the
communications network operations necessary to initiate, modify, or
release the network calls are left to the present invention.
The present invention provides the capability to interface with
applications and to manage resources used, for example, in multimedia
multiparty communications. The Application Programming Interface ("API")
allows independently developed applications at one platform to cooperate
in controlling a session state, which includes the state of local
resources (i.e., resources available to the platform) and
telecommunication network resources.
A. The Major Components of System
FIG. 1 illustrates a system 10 in which the present invention may be
implemented. In FIG. 1, system 10 contains a LAN (local area network) 100
and a personal computer 200. LAN 100 is connected by a telecommunications
network 175 to personal computer 200. LAN 100 and personal computer 200
can be either heterogenous or homogenous. For example, LAN 100 may be a
UNIX-based network and the personal computer 200 may use the MS-DOS
operating system. Alternatively, both the LAN 100 and personal computer
200 may use the MS-DOS operating system.
Telecommunications network 175 may include multimedia hardware, as
explained below. LAN 100 may communicate with personal computer 200 via a
protocol specified by telecommunications network 175. This protocol can
create, modify, and delete multimedia multiparty calls.
LAN 100 includes a LAN interconnect 125 connecting LAN applications 105 and
a personal computer 130. LAN interconnect 125 connects the applications,
workstations, personal computers, including personal computer 130, in LAN
100.
LAN applications 105 include three components of the preferred
implementation: Transaction Manager Component 110, Resource Manager
Component 112, and Infrastructure API Support Component 115. LAN
applications 105 also include an active database 120. LAN applications 105
may also include other applications that may be stored, run or displayed
on any personal computer in LAN 100.
Personal computer 130 includes a CPU 135 and a memory 155. CPU 135 includes
a user application 140 together with a User API Support Component 145. CPU
135 also includes operating system 150.
LAN applications 105 are illustrated separate from the personal computer
130 in the LAN 100 to show that Transaction Manager Component 110,
Resource Manager Component 112, Infrastructure API Support Component 115,
and active database 120 need not be physically located in the same
personal computer as User API Support Component 145. In fact, Transaction
Manager Component 110, Resource Manager Component 112, Infrastructure API
Support Component 115, and active database 120 may be available to all
personal computers connected in LAN 100, including personal computer 130.
Also, although, user application 140 and User API Support Component 145
are shown as part of personal computer 130, they could be among the LAN
applications 105 and available to all personal computers in the LAN 100.
The present invention may be used by personal computers in LAN 100 to
communicate with each other, or by a single personal computer in LAN 100,
like personal computer 130, to communicate with another personal computer
outside the LAN 100, like computer 200. In the latter case, personal
computer 130 in the LAN 100 would communicate with outside personal
computer 200 via the telecommunications network 175. Alternatively,
personal computer 130 in LAN 100 communicates directly with personal
computer 200 via a conventional data connection instead of intermediate
telecommunications network 175. This, however, would require certain
modifications local to Transaction Manager Components 110 and 210. The
details of these modifications are not important to an understanding of
the present invention, however, so they will not be discussed.
Personal computer 200 includes a CPU 235 and a memory 255. CPU 235 includes
user application (A) 260, user application (B) 270, Transaction Manager
Component 210, Infrastructure API Support Component 215, active database
220, and Resource Manager Component 222. User applications (A) 260 and (B)
270 each have a corresponding User API Support Components 265 and 275,
respectively, which are also included in CPU 235. The CPU 235 also
includes operating system 250.
User applications 140, 260, and 270 may be off-the-shelf programs modified
to work with the present invention, or specially-written programs to take
advantage of the services provided by the present invention. For purposes
of this description, user applications either invoke operations to be
performed in accordance with this invention, or respond to invocations by
components of the preferred implementation which may originate from other
user applications. In the preferred implementation, the examples of user
applications 140, 260 and 270 include applications capable of performing
functions associated with multimedia multiparty communications.
Associated with the user applications 140, 260, and 270 are User API
Support Components 145, 265, and 275, respectively. This association is
illustrated by the dotted lines. The dotted lines also show that the User
API Support Components 145, 265, and 275 are part of the preferred
implementation and separate from the user applications 140, 260, 270,
respectively. In one example, the User API Support Components 145, 265,
and 275 may be program code in a library to be compiled with user
applications 140, 260, and 270, respectively.
As illustrated in FIG. 1, only the Infrastructure API Support Components
115 and 215 can be connected directly to the active databases 120 and 220,
respectively. This connection depends on the interface required to access
the active databases 120 and 220. For example, the connection between
components 115 and 120 may differ from the connection between components
215 and 220 because the active database 120 is one type of relational
database and 220 is another type.
The interfaces are the same between the User API Support Components 145,
265, and 275 and the Infrastructure API Support Components 115 and 215,
respectively; between the Resource Manager Components 112 and 222 and the
Infrastructure API Support Component 115 and 215, respectively; and
between the Transaction Manager Components 110 and 210 and the
Infrastructure API Support Component 115 and 215, respectively. They are
dictated by the protocol of the Infrastructure API Support Components 115
and 215.
In the preferred implementation, components 110, 112, 115, 210, 215, and
222 represent software processes which communicate with one another
through standard interprocess communication mechanisms. Further, in the
preferred implementation the user applications 140, 260, and 270 are also
processes and the components 145, 265, and 275, respectively, are parts of
those processes. While the Resource Manager Components 112 and 222, and
the user applications 140, 260, and 270 (with each corresponding support
component 145, 265, and 275, respectively) can be disconnected from the
system, Infrastructure API Support Components 115 and 215 will preferably
always be present in the system 10. Additionally, new applications can be
added to the system 10 at any time.
Although this description discusses Transaction Manager Components 110 and
210 and Resource Manager Components 112 and 222, there may also be other
"managers" communicating with the Infrastructure API Support Components
115 and 215. Managers are applications that oversee aspects of the active
databases 120 and 220 and, in response to changes in active databases 120
and 220, further change the active databases 120 and 220 to assist user
applications, for example, user applications 140, 260, and 270.
For example, a directory service manager may be connected to the
Infrastructure API Support Component 115. When a user application 140
indicates that it wants to place a call to another party with the name of
that party, without the telephone number, the directory service manager
causes a change to the active database 120 to include the telephone number
of the party to be called. This example shows how directory service
manager must include functions correlate party names and their telephone
numbers.
Operating systems 150 and 250 are standard operating systems which are tied
to the corresponding CPUs 135 and 235. For example, operating system 150
and operating system 250 might both be MS-DOS. Though it is not
illustrated in FIG. 1, a LAN operating system, such as WindowsNT.TM.,
could also be included among the LAN applications 105.
Memories 155 and 255 may be any type of storage device (e.g., magnetic
storage or optical storage) and serve several functions. One function is,
of course, to provide general storage for the associated personal computer
130 and 200. Another function is to store user applications 140, 260, and
270, components 110, 115, 120, 220, 210, 215, 265, and 275 and operating
systems 150 and 250. In addition, memory 155 may contain information for
the LAN, including applications and data that may be used by other
personal computers, connected in LAN 100. For example, since personal
computer 130 is part of the LAN 100, memory 155 may store one or more LAN
Applications.
As mentioned above, telecommunications network 175 includes certain
multimedia communications hardware to provide functions for multimedia,
multiparty communications. Such functions may include mixing, combining,
and transcoding sound, video, and data from different sources.
FIG. 2 is a block diagram of multimedia communications hardware 280 the
preferred implementation uses for multimedia, multiparty communications.
Hardware 280 includes a mixer 282 and a transcoder 284. Mixer 282 combines
multiple input signals into a single output signal and transcoder 284
performs format conversion, such as from digital video to NTSC analog
video.
In one embodiment, telecommunications network 175 includes hardware 280,
which may then be considered among the "network resources." Alternative
configurations may also be possible. For example, personal computer 200
may include hardware 280. In this case, Resource Manager Component 222
would manage operations using hardware 280. Resource Manager Component 222
in this example may also manage other local resources in the personal
computer 200, such as a camera or a speaker.
B. Database Architecture
Active databases 120 and 220 may be relational databases or some other type
of databases, e.g., hierarchical databases. The functions of the active
databases 120 and 220 may also be performed by a custom built means for
storing and manipulating data as well as responding to manipulation of the
data.
Active database 120 for the LAN 100 is among the LAN applications 105.
Active database 120 is characterized as being similar to other types of
LAN applications 105 because it includes not only data and relationships
between the data, but also "trigger" information. Trigger information is
defined as conditions or rules that signal events to be initiated when the
data and/or relationships in the database change.
For example, the active database 120 may include a trigger that initiates a
process to signal user application (A) 260 whenever data is added to the
active database 120. Another trigger might initiate a process to signal
user application (B) 270 under more specific conditions, i.e., whenever a
specific change is made to the active database 220.
Similarly, the active database 220 for the personal computer 200 is
illustrated in the CPU 235. This is because, while the data in the active
database 220 is resident in the memory 255, the database is an "active"
database.
The preferred implementation does not limit the number or types of triggers
that active databases 120 and 220 may include. All user applications 140,
260, and 270, which may be of any type (including video conferencing
applications, picture-in-picture applications, voice mail applications,
etc.), may "register" triggers (via the Infrastructure API Support
Components 115 and 215 and User API Support Components 145, 265, and 275,
respectively) with the active databases 120 and 220, respectively.
Additionally, components of the preferred implementation, such as
Transaction Manager Components 110 and 210 and Resource Manager Components
112 and 222, may also register triggers. Registering triggers involves
adding new triggers to the active databases 120 and 220.
For the most part active databases 120 and 220 represent both a
"telecommunications network state" and a "local resource state." The
telecommunications network state includes objects in the active database
120 or 220 that correspond to the different aspects of a call viewed by
the telecommunications network. In contrast, the local resource state
includes objects in the active database 120 or 220 that correspond to
local resources. Throughout this description references to the "state of
the active database" refer to both the telecommunications network state
and the local resource state.
Active databases 120 and 220 include "objects." Each object has an
identifier for identification and one or more attributes to represent some
aspect of the object. For example, a "Party" object might include two
attributes, the first identifying the name of the party and the second
specifying the party's telephone number.
One skilled in the art will recognize that any type of representation for
these objects may also be used. For example, the objects may be
represented in a hierarchical database using another format. Further,
while the present implementation uses certain object attributes, other
attributes may also be added by, for example, user applications.
Among the types of objects included in the active databases 120 and 220 are
"call objects" and "transaction objects." FIG. 3 illustrates examples of
most of the call objects and their relationships. The call objects fall
into eight types: Party Set (not shown), Party 445, Party Service Link
450, Abstract Service 455, Network Mapping Element 460, Access Channel
465, Local Mapping Element 470, and User Device 475.
Call objects represent aspects of "calls" in the active databases 120 and
220. Calls are communications between users with homogeneous or
heterogeneous computers, or between other terminal equipment, for example,
LAN 100 and personal computer 200. Calls may include multiple users and
multimedia.
FIG. 3 also illustrates two categories of call objects: network call
objects and local call objects. Network call objects represent aspects of
calls related to a telecommunications network such as network 175 in FIG.
1, and local call objects represent aspects of calls related to hardware,
including personal computer, LAN, telephone, etc., connected to the
telecommunications network 175.
Network call objects include five types of objects: Party 445, Party
Service Link 450, Abstract Service 455, Network Mapping Element 460, and
Access Channel 465. These types of objects can be defined as follows:
______________________________________
Party 445: identifies an individual or
entity in a call;
Party-Service defines the link between a
Link 450: Party 445 and an Abstract
Service 455 being used in a
call;
Abstract Service 455:
defines the medium of
communication used in a call;
Network Mapping defines the link between an
Element 460: Abstract Service 455 and an
Access Channel 465; and
Access Channel 465:
defines a telecommunications
network endpoint that
describes encoding as well as
the physical and logical
channels for a call.
______________________________________
Active databases 120 and 220 also store one type of network call object not
illustrated in FIG. 3, the Party Set object, which identifies a group of
Party 445 objects in a call. These types of network call objects are
described in greater detail in Steven Minzer, "A Signaling Protocol for
Complex Multimedia Services," IEEE J. Select. Areas Comm., vol. 9, pp.
1383-1394, December 1991, which is incorporated herein by reference.
Because this article uses different terminology for network call objects,
the following translation relates the two terminologies:
______________________________________
Party Set .fwdarw.
Call
Party .fwdarw.
Remote Vertex
Party Service Link
.fwdarw.
Connection Edge
Abstract Service .fwdarw.
T-connection
Network Mapping Element
.fwdarw.
Mapping Element
Access Channel .fwdarw.
Access Edge
______________________________________
Local call objects include Local Mapping Element objects 470 and User
Device objects 475. Local Mapping Element objects 470 represent links
between Access Channel 465 objects and User Device 475 objects. User
Device objects 475 define terminal equipment, such as cameras, monitors,
microphones, or speakers, used in a call.
FIG. 3 shows a possible relationship between objects. A Party object 447,
with the object identifier "anne," is related to a Party Service Link
object 452, which in turn is related to the Abstract Service object 457.
The object identifier for the Abstract Service Link object 457 is
"video.sub.-- 1." Similarly, the Abstract Service object 457 is related to
a Network Mapping Element object 461, which in turn is related to an
Access Channel object 467. The object identifier for the Access Channel
object 467 is "ntsc.sub.-- on.sub.-- atm." The Access Channel object 467
is related to a Local Mapping Element 471, which in turn is related to a
User Device object 477. The object identifier for the User Device Object
477 is "camera.sub.-- 1."
Additionally, the Party Service Link objects 450, Network Mapping Element
objects 460, and Local Mapping Element objects 470 also have object
identifiers. For example, the object identifier for the Party Service Link
object 452 is "high.sub.-- quality.sub.-- video"; the object identifier
for the Network Mapping Element object 461 is "up.sub.-- video.sub.--
net.sub.-- 1"; and the object identifier for the Local Mapping Element
object 471 is "up.sub.-- video.sub.-- loc."
The remaining objects illustrated in FIG. 3, Party 445, Party Service Link
450, Abstract Service 455, Network Mapping Element 460, Access Channel
465, Local Mapping Element 470, and User Device 475 also have object
identifiers, as shown in FIG. 3. Object 449 has an identifier "bob,"
object 458 has an identifier "audio.sub.-- 1," object 459 has an
identifier "audio.sub.-- 2," object 469 has an identifier "7khz.sub.--
audio," object 479 has an identifier "monitor.sub.-- 1," and object 481
has an identifier "speaker.sub.-- 1." Further, the remaining Party Service
Link objects 453 and 454, Network Mapping Element objects 462, 463, and
464, and Local Mapping Element objects 471, 472 and 473 also have object
identifiers. These object identifiers are as follows:
______________________________________
Party Service Link object 453
.fwdarw.
"high.sub.- quality.sub.- audio"
Party Service Link object 454
.fwdarw.
"low.sub.- quality.sub.- audio"
Network Mapping Element object 462
.fwdarw.
"down.sub.- video.sub.- net.sub.- 1"
Network Mapping Element object 463
.fwdarw.
"down.sub.- audio.sub.- net.sub.- 1"
Network Mapping Element object 464
.fwdarw.
"down.sub.- audio.sub.- net.sub.- 1"
Local Mapping Element object 471
.fwdarw.
"up.sub.- video.sub.- loc"
Local Mapping Element object 472
.fwdarw.
"down.sub.- video.sub.- loc"
Local Mapping Element object 473
.fwdarw.
"down.sub.- audio.sub.- loc"
______________________________________
The preferred implementation uses transaction objects to represent
transactions on call objects. These transactions differ from the database
transactions, which those skilled in the art will recognize as involving
updates on a database. Transactions on call objects comprise a set of
operations to be performed as a unit altogether or not at all. These
operations are called "atomic actions." For example, the transactions of
call objects include sets of operations for setting up, connecting,
disconnecting, creating, deleting, modifying, etc. calls.
As illustrated in FIG. 4, relationships between objects in the active
databases 120 and 2 | | |
