 |
Claims  |
|
|
I claim:
1. In a computer network having a plurality of computer systems coupled by
a communication medium, one of said plurality of computer systems being a
server system, a different one of said plurality of computer systems being
a client system, a method for synchronizing a data stream transferred from
said server system to said client system, said method comprising the steps
of:
capturing information into a data packet, said capturing step being
performed by said server system;
storing time information into said data packet, said time information
indicating when said capturing step is complete;
transferring said data packet to said client system across said
communication medium;
initializing a virtual server system clock if said data packet is a first
data packet in said data stream, said initializing step including the step
of computing a difference based on time information stamped on said first
data packet and time information on a server system clock;
retrieving said time information from said data packet;
comparing said time information with said virtual server system clock; and
processing said information in said data packet if said time information
indicates a time equal to a time indicated by said virtual server system
clock.
2. The method as claimed in claim 1 further including the step of:
discarding said data packet if said time information indicates a time
earlier than a time indicated by said virtual server system clock.
3. The method as claimed in claim 1 further including the steps of:
calculating an index into said information in said data packet
corresponding to a value in said virtual server system clock;
discarding a first portion of said data packet from a beginning position in
said data packet to a position in said data packet indicated by said
index; and
processing a second portion of said data packet from a position in said
data packet indicated by said index to an ending position of said data
packet.
4. The method as claimed in claim 1 wherein said information captured in
said data packet comprises either video or audio information.
5. The method of claim 1, wherein said step of computing of a difference
based on a first data packet and time information on a server clock
further comprises the steps of:
computing a value by subtracting the capture time period of said first data
packet from the time information stamped on said first data packet; and
subtracting said value from said server system clock.
6. In a computer network having a plurality of computer systems coupled by
a communication medium, one of said plurality of computer systems being a
server system, a different one of said plurality of computer systems being
a client system, a method for synchronizing data streams transferred from
said server system to said client system, said method comprising the steps
of:
capturing information into a first set of data packets, said capturing step
being performed by said server system;
capturing information into a second set of data packets, said capturing
step being performed by said server system;
storing time information into said first set of data packets, said time
information indicating when said capturing step is complete;
storing time information into said second set of data packets, said time
information indicating when said capturing step is complete;
transferring said first set of data packets to said client system across
said communication medium in a first data stream;
transferring said second set of data packets to said client system across
said communication medium in a second data stream;
initializing a virtual server system clock if a data packet is a first data
packet in a data stream, said initializing step includes computing a
difference based on the time stamped on a reference data packet in a
respective data stream and a time reading of a server clock;
retrieving said time information from data packets of said first set or
said second set of data packets;
comparing said time information with said virtual server system clock; and
processing said information from data packets of said first set or said
second set of data packets if said time information indicates a time equal
to a time indicated by said virtual server system clock.
7. The method as claimed in claim 6 further including the step of:
scheduling processing of said information from data packets of said first
set or said second set of data packets if said time information indicates
a time later than a time indicated by said virtual server system clock.
8. The method as claimed in claim 6 further including the step of:
discarding data packets of said first set or said second set of data
packets if said time information indicates a time earlier than a time
indicated by said virtual server system clock.
9. The method as claimed in claim 6 further including the steps of:
calculating an index into said information in data packets of said first
set or said second set of data packets corresponding to a value in said
virtual server system clock;
discarding a first portion of data packets of said first set or said second
set of data packets from a beginning position in said data packets to a
position in said data packets indicated by said index; and
processing a second portion of data packets of said first set or said
second set of data packets from a position in said data packets indicated
by said index to an ending position of said data packets.
10. The method as claimed in claim 6 wherein said information captured in
said first set of data packets comprises video information and said
information captured in said second set of data packets comprises audio
information.
11. The method of claim 6, wherein said step of computing of a difference
based on a first data packet and time information on a server clock
further comprises the steps of:
computing a value by subtracting the capture time period of said first data
packet from the time information stamped on said first data packet; and
subtracting said value from said server system clock.
12. In a computer network having a plurality of computer systems coupled by
a communication medium, one of said plurality of computer systems being a
server system, a different one of said plurality of computer systems being
a client system, an apparatus for synchronizing a data stream transferred
from said server system to said client system, said apparatus comprising:
means for capturing information into a data packet, said means for
capturing residing in said server system;
means for storing time information into said data packet, said time
information indicating when said capturing is finished;
means for transferring said data packet to said client system across said
communication medium;
means for initializing a virtual server system clock if said data packet is
a first data packet in said data stream by generating a difference based
on a time stamped on said first data packet and a time reading on a server
clock;
means for retrieving said time information from said data packet;
means for comparing said time information with said virtual server system
clock; and
means for processing said information in said data packet if said time
information indicates a time equal to a time indicated by said virtual
server system clock.
13. The apparatus as claimed in claim 12 further including:
means for scheduling processing of said information in said data packet if
said time information indicates a time later than a time indicated by said
virtual server system clock.
14. The apparatus as claimed in claim 12 further including:
means for discarding said data packet if said time information indicates a
time earlier than a time indicated by said virtual server system clock.
15. The apparatus as claimed in claim 12 further including:
means for calculating an index into said information in said data packet
corresponding to a value in said virtual server system clock;
means for discarding a first portion of said data packet from a beginning
position in said data packet to a position in said data packet indicated
by said index; and
means for processing a second portion of said data packet from a position
in said data packet indicated by said index to an ending position of said
data packet.
16. The apparatus as claimed in claim 12 wherein said information captured
in said data packet comprises either video or audio information.
17. In a computer network having a plurality of computer systems coupled by
a communication medium, one of said plurality of computer systems being a
server system, a different one of said plurality of computer systems being
a client system, an apparatus for synchronizing data streams transferred
from said server system to said client system, said apparatus comprising:
means for capturing information into a first set of data packets, said
means for capturing residing in said server system;
means for capturing information into a second set of data packets, said
means for capturing residing in said server system;
means for storing time information into said first set of data packets,
said time information indicating when said capturing is finished;
means for storing time information into said second set of data packets,
said time information indicating when said capturing is finished;
means for transferring said first set of data packets to said client system
across said communication medium in a first data stream;
means for transferring said second set of data packets to said client
system across said communication medium in a second data stream;
means for initializing a virtual server system clock if a data packet is a
first data packet in a data stream by generating a difference based on a
time stamped on a reference data packet and time information on a server
clock;
means for retrieving said time information from data packets of said first
set or said second set of data packets;
means for comparing said time information with said virtual server system
clock; and
means for processing said information from data packets of said first set
or said second set of data packets if said time information indicates a
time equal to a time indicated by said virtual server system clock.
18. The apparatus as claimed in claim 17 further including:
means for scheduling processing of said information from data packets of
said first set or said second set of data packets if said time information
indicates a time later than a time indicated by said virtual server system
clock.
19. The apparatus as claimed in claim 17 further including:
means for discarding data packets of said first set or said second set of
data packets if said time information indicates a time earlier than a time
indicated by said virtual server system clock.
20. The apparatus as claimed in claim 17 further including:
means for calculating an index into said information in data packets of
said first set or said second set of data packets corresponding to a value
in said virtual server system clock;
means for discarding a first portion of data packets of said first set or
said second set of data packets from a beginning position in said data
packets to a position in said data packets indicated by said index; and
means for processing a second portion of data packets of said first set or
said second set of data packets from a position in said data packets
indicated by said index to an ending position of said data packets.
21. The apparatus as claimed in claim 17 wherein said information captured
in said first set of data packets comprises video information and said
information captured in said second set of data packets comprises audio
information.
22. In a computer network having a plurality of computer systems coupled by
a communication medium, one of said plurality of computer systems being a
server system, a different one of said plurality of computer systems being
a client system, a method for synchronizing a data stream transferred from
said server system to said client system, said method comprising the steps
of:
capturing information into a data packet, said capturing step being
performed by said server system;
storing time information into said data packet, said time information
indicating when said capturing step is complete;
transferring said data packet to said client system across said
communication medium;
initializing a virtual server system clock if said data packet is a first
data packet in said data stream;
retrieving said time information from said data packet;
comparing said time information with said virtual server system clock;
processing said information in said data packet if said time information
indicates a time equal to a time indicated by said virtual server system
clock;
calculating an index into said information in said data packet
corresponding to a value in said virtual server system clock;
discarding a first portion of said data packet from a beginning position in
said data packet to a position in said data packet indicated by said
index; and
processing a second portion of said data packet from a position in said
data packet indicated by said index to an ending position of said data
packet.
23. In a computer network having a plurality of computer systems coupled by
a communication medium, one of said plurality of computer systems being a
server system, a different one of said plurality of computer systems being
a client system, a method for synchronizing a data stream transferred from
said server system to said client system, said method comprising the steps
of:
capturing information into a data packet, said capturing step being
performed by said server system;
storing time information into said data packet, said time information
indicating when said capturing step is complete;
transferring said data packet to said client system across said
communication medium;
initializing a virtual server system clock if said data packet is a first
data packet in said data stream;
retrieving said time information from said data packet;
comparing said time information with said virtual server system clock;
processing said information in said data packet if said time information
indicates a time equal to a time indicated by said virtual server system
clock; and
scheduling processing of said information in said data packet if said time
information indicates a time later than a time indicated by said virtual
server system clock. |
|
 |
|
 |
Claims |
|
|
|
Description |
|
|
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to the field of a networked computer system.
Specifically, the present invention relates to the transfer and
synchronization of multiple data streams across a computer network or
various types of communication media.
2. Description of Related Art
In a networked computer system, it is often necessary to send multiple
streams of data to a remote network site. Typically, the streams of data
must be processed in the correct order and synchronized with respect to
each data stream. For example, audio and video streams in a
teleconferencing application must be synchronized to provide the correct
relationship between the audio and video data. Conventional systems
implement data streams as a plurality of data packets that can be
independently transferred across a network. Currently, packetized data
streams may be synchronized by interleaving the data packets that make up
the streams in such a way that the packets are sent to the network in
synchronized order. This interleaving method has several drawbacks. First,
not all networks can guarantee an in-order delivery of data packets. The
synchronization between data streams may be lost by a network that
randomly delays the transfer of a data packet. Secondly, even if in-order
delivery of packets is guaranteed, the interleaving method does not allow
synchronization of data streams where the data stream cannot be sent in a
single interleaved stream. For example, where the data streams are sent on
different network channels, two different network addresses, or on
different networks, the data streams cannot be sent in a single
interleaved stream. In other situations, multiple originators or producers
of data streams (i.e. servers) may exist on a network. In this case, data
streams from a single server must be synchronized with respect to each
other; but, synchronization from one server to another may not be
required. In other situations it may be necessary to mix a set of data
streams originated by multiple servers into one unified data stream for
delivery to a client network node. For example, instead of two servers
each sending an audio and video stream to a client node for processing, it
may be more efficient to mix the audio signals into a unified stream which
then must be synchronized with both video streams. Conventional systems
are incapable of providing these multiple levels of data stream
synchronization.
Thus, a better method for synchronizing multiple independent data streams
in a networked computer system is needed.
SUMMARY OF THE INVENTION
The present invention is an apparatus and method for synchronizing multiple
independent data streams in a networked computer system. In a computer
network having a plurality of computer systems coupled by a communication
medium, one of the plurality of computer systems being a server system, a
different one of the plurality of computer systems being a client system,
a method for synchronizing a data stream transferred from the server
system to the client system, the method comprising the steps of: 1)
capturing information into a data packet, the capturing step being
performed by the server system; 2) storing time information into the data
packet, the time information indicating when the capturing step is
complete; 3) transferring the data packet to the client system across the
communication medium; 4) initializing a virtual server system clock if the
data packet is a first data packet is the data stream; 5) retrieving the
time information from the data packet; 6) comparing the time information
with the virtual server system clock; and 7) processing the information in
the data packet if the time information indicates a time equal to a time
indicated by the virtual server system clock. The present invention
further includes the steps of: 1) scheduling processing of the information
in the data packet if the time information indicates a time later than a
time indicated by the virtual server system clock; and 2) discarding the
data packet if the time information indicates a time earlier than a time
indicated by the virtual server system clock.
It is therefore an advantage of the present invention to provide a means
and method for time stamping packets of a data stream. It is a further
advantage of the present invention to provide a means and method for
synchronizing data streams arriving from one server and providing them to
a client in the same time sequence as they were captured. It is a further
advantage of the present invention to provide a data stream
synchronization means and method wherein data streams may take independent
paths across a network. It is a further advantage of the present invention
to provide a means and method for mixing a plurality of data streams for
playback in one unified stream.
These and other advantages of the present invention are fully described in
the following detailed description of the preferred embodiment.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram illustrating the computer system hardware used in
the preferred embodiment.
FIG. 2 illustrates a typical network environment in which the present
invention operates.
FIG. 3 illustrates a multiple server, multiple client network environment.
FIG. 4 illustrates a time line with associated data packets of a data
stream.
FIG. 5 illustrates the data structure of a data packet.
FIGS. 6 through 9 are flow charts illustrating the processing logic of the
present invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
The present invention is an apparatus and method for synchronizing multiple
independent data streams in a networked computer system. In the following
detailed description, numerous specific details are set forth in order to
provide a thorough understanding of the invention. However, it will be
apparent to one of ordinary skill in the art that these specific details
need not be used to practice the present invention. In other
circumstances, well known structures, materials, circuits, and interfaces
have not been shown or described in detail in order to not unnecessarily
obscure the present invention.
Referring now to FIG. 1, a block diagram of the computer system hardware
used in the preferred embodiment is illustrated. The computer system used
in the preferred embodiment comprises a Bus 100 for communicating
information between computer system components. These components coupled
to Bus 100 include Processor 101. Processor 101 may be implemented as any
of a variety of commonly available microprocessors such as the 386 brand,
i486 brand, or Pentium brand microprocessors manufactured by Intel
Corporation, Santa Clara, Calif. The Intel, 386, i486, and Pentium marks
are trademarks of Intel Corporation. It will be apparent of ordinary skill
in the art that other equivalent processors may be used with the present
invention. Other system components include Random Access Memory (RAM) 102,
Read Only Memory (ROM) 103, and mass storage device or disk drive 104. The
computer system of the preferred embodiment also includes a display device
105 coupled to the Bus 100 for displaying information to a computer user,
an alphanumeric input device 106 coupled to the Bus 100 for communicating
information and command selections to the Processor 101, a pointing device
or cursor control device 107 coupled to the Bus 100 for communicating
information and command selections to Processor 101, and a signal
generation device 112 coupled to the Bus 100 for communicating command
selections to the Processor 101. Display device 105 may be a liquid
crystal device, a cathode ray tube, or other suitable display device.
Alphanumeric input device 106 is typically an alphanumeric or function key
keyboard. Pointing device or cursor control device 107 is typically a
mouse, trackball, or stylus device allowing the computer user to
dynamically signal the two dimensional movement of a visible symbol (i.e.,
cursor) on the display screen of display device 105. Many implementations
of the cursor control device 107 are well known in the art including
trackball systems, mouse systems, joy sticks, stylus, or special keys on
the alphanumeric input device 106 capable of signaling movement in a given
direction. Signal generation device 112 is typically implemented as a
button or switch located on or near the cursor control device 107.
The computer system used in the preferred embodiment includes a
communication interface 108 coupled to Bus 100. Communication interface
108 is also coupled to a communication medium 110. Communication interface
108 provides a means for the computer system to communicate with a network
of other computer systems. A variety of networks or communication media
are well known in the art. These communication media 110 include well
known Local Area Networks (LAN) such as Ethernet, Wide Area Networks
(WAN), telephone modem links, or other well known and conventional
computer networking technology.
The computer system of the preferred embodiment also includes video
hardware subsystem 114 and audio hardware subsystem 116 both coupled to
Bus 100. A video source 118, such as a video camera or video tape player,
is coupled to video hardware subsystem 114. An audio source 120, such as a
microphone or audio tape player, is coupled to audio hardware subsystem
116. Video hardware subsystem 114 provides a means by which the computer
system of the preferred embodiment may receive video signals and convert
the video signals for transfer to other system components as a stream of
digital video information via Bus 100. Similarly, audio hardware subsystem
116 provides a means by which the present invention may receive audio
signals and convert the audio signals to a stream of digital audio
information for transfer to other system components via Bus 100. In a
similar manner, the video data streams provided by video hardware
subsystem 114 and the audio data streams provided by audio hardware
subsystem 116 may be transferred under control of Processor 101 to other
networked computer systems via communication interface 108 and
communication medium 110. The present invention provides a means and
method for synchronizing these data streams in a networked computer
system. Except as described herein, the configuration and interconnection
of computer system components as illustrated in FIG. 1 are well known to
those of ordinary skill in the art.
Referring now to FIG. 2, a typical environment in which the present
invention operates is illustrated. The present invention operates in an
environment comprising a set of computer systems linked in a network
topology by a communication medium 210. It will be apparent to those of
ordinary skill in the art that any arbitrary number of computer systems
may be so linked. In the example network environment illustrated in FIG.
2, each computer system 214, 216 and 222 is shown coupled to communication
medium 210. Each of these computer systems comprise components such as
those illustrated in FIG. 1. Computer systems 214 and 216 may be coupled
to video rendering devices, such as a display, and audio playback devices.
Computer system 222 may be coupled to a video source and an audio source.
For example, computer system 214 is coupled to video rendering device 228
and audio playback device 230. Similarly, computer system 222 is coupled
to video source 240 and audio source 242. It is not necessary for each
computer system coupled to communication medium 210 to have video
rendering devices, video sources, audio playback devices, and/or audio
sources connected to them. In a variety of configurations, computer
systems coupled to communication medium 210 may independently be coupled
to a video source or rendering device and an audio source or playback
device, a video source only, an audio source only, or neither a video
source or audio source. A single video source or audio source may also be
shared between two computer systems. In any of these configurations, the
video source provides a stream of video data which a computer system may
provide to other computer systems via communication medium 210. Similarly,
an audio source provides a stream of audio data which may be provided to
other computer systems via communication medium 210. It will be apparent
to one of ordinary skill in the art that data types other than video or
audio may equivalently be supported.
A computer system on which video or audio data steams originate is called a
server for the data streams. In the example of FIG. 2, computer system 222
is a server. When acting as a server, computer system 222 receives video
signals via video source 240 and audio signals via audio source 242 and
converts these signals to video and audio data streams, which are provided
to other computer systems via communication medium 210. These data streams
are transferred on communication medium 210 by server 222 and transported
to remote computer systems such as computer system 214 and 216. Any remote
computer system receiving data streams from a server and rendering or
playing the data to a local user is known as a client system for the data
streams. Thus, in general, a server system is an originator of data
streams and a client system is a receiver or consumer of data streams. The
client system is responsible for synchronizing the data streams arriving
from a server and for rendering the data streams to a local user in the
same time sequence as the data streams were captured by the server system.
It is not essential to the present invention that the data streams from
the server arrive on the same network media. It is only important that the
data streams are provided in some way to a client computer system via a
communication medium. The client system may then synchronize the data
streams and render them to a local user using the technique taught herein.
Although the example illustrated in FIG. 2 shows a single server system
222, the present invention is applicable to network environments wherein
multiple servers and multiple clients exist on the network at one time. A
multiple server, multiple client network configuration is illustrated in
FIG. 3.
Referring to FIG. 3, servers 310 and 312 are coupled to clients 320 and 322
via communication medium 314, 316 and 324. A portion of network medium 316
is blown up in FIG. 3 to illustrate the multiple independent data streams
318 concurrently being transferred across network medium 316 from servers
310 and 312 to clients 320 and 322. Each data stream of data streams 318
consist of a sequence of data packets. Each data packet comprises a
portion of information captured by a server system. This information may
represent a captured video image, or portion thereof, a captured audio
representation or any other collection of information that may be
represented as binary information. In a typical implementation, the video
hardware subsystem of a server system captures a video image and creates a
sequence of data packets that are transferred across network 316 as one of
data streams 318. Similarly, other data streams of data steams 318 are
generated by other server systems. In the synchronization process of the
present invention, the client system seeks to establish the same time line
or synchronization for rendering the data packets as was present during
the capturing of the data packets performed by the corresponding server
system. In this manner, the client system is able to render information in
a manner corresponding to the way the information was captured by a server
system. This synchronization can occur regardless of the manner by which
the information is transferred between the server and client systems.
Moreover, the present invention can be used to synchronize data streams
with respect to each other. In this manner, a video stream may be
synchronized with an audio stream and rendered by a client system
appropriately. Further details of the synchronization process of the
present invention follow.
Referring now to FIG. 4, data packets P.sub.1 to P.sub.n are shown on a
time line. Data packets P.sub.1 through P.sub.n represent the data packets
of a data stream such as data stream D.sub.1 illustrated in FIG. 3. Each
packet comprises information collected or captured during a time frame
defined by a beginning capture time and an ending capture time. For
example, referring to FIG. 4, packet P.sub.1 comprises information
captured between time t.sub.a and time t.sub.b. A function CT defines the
capture time for a particular packet of a data stream. As illustrated in
FIG. 4, the capture time for packet P.sub.1, CT (P.sub.1) is defined as
the time period between t.sub.a and t.sub.b. In one embodiment of the
present invention, the capture time for each data packet of a data stream
is identical and preset to some fixed value. In an alternative embodiment
of the present invention, the capture time for each data packet of a data
stream may be different. Thus, function CT is used to determine the
capture time for a particular data packet. As indicated by the equations
in FIG. 4, the ending capture time, t.sub.b, may be determined by knowing
the beginning capture time, t.sub.a, and the capture time CT (P.sub.1).
Similarly, the beginning capture time, t.sub.a, may be determined if the
ending capture time, t.sub.b, and the capture time CT (P.sub.1) is known.
Similar computations may be performed for each data packet P.sub.1 through
P.sub.n of a data stream.
In support of the synchronization process of the present invention, a time
stamp is applied to each data packet of a data stream. When a server
computer system captures data into data packets, the server stamps its
system clock time into each packet of the data stream. In the preferred
embodiment, this time stamp represents the system clock time at the data
packet ending capture time. Thus, for the example illustrated in FIG. 4,
the time stamp applied to packet P.sub.1 is t.sub.b. The computer system
clock is assumed to be a monotonically increasing count of the system
clock ticks. Thus, CT (P.sub.1) equals the number of system clock ticks
between t.sub.a and t.sub.b. Each data packet generated by a server is
stamped with the ending capture time of the packet. Thus, packet P.sub.2
is stamped with time t.sub.d, packet P.sub.3 is stamped with time t.sub.f,
and packet P.sub.n is time stamped with time t.sub.y. In this manner, each
data packet of a data stream is referenced to a unique position on a data
stream time line, such as the one illustrated in FIG. 4. Each data packet
is thereby associated with a fixed mom | | |
