 |
Description  |
|
|
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to television image processing systems and
particularly to such systems having frame capture, merge and display
capability for providing a user controllable composite video display
having a user selectable combination of full motion video, still video
images in user controllable variable windows, graphics and/or text.
2. Description of the Prior Art
Although there are prior art image processing systems capable of capturing
multiple video images and providing a composite display therefrom such as
the Magnavox Digital Stereo 4 Head VHS HQ VCR which enables the user to
break the television screen into four fixed windows to enable a display of
three still video images and one live video image, there are none known to
applicants which has the flexibility of unlimited windows having user
controllable variable size and location in the composite with full size
instantaneous image capture and storage for providing user controllable
merging of still video images in these windows with graphics, text and/or
live full motion video in an efficient and economical system, particularly
one capable of providing high resolution color images. Image manipulation
to provide variable size window computer generated images are well known,
such as available by use of the 82786 graphics coprocessor available from
Intel. However, this coprocessor has never been used before with live
video images in a system to provide instantaneous grabbing of these images
at random under user control. This is so despite it having been known that
the 82786 graphic output can be combined with output from other video
sources such as broadcast TV, video recorders, and video laser disc
players. Other prior art systems lacking the full system flexibility of
the present invention are the Truevision line of microcomputer graphics
hardware and software available from AT&T, including the VISTA and TARGA
Videographics Adapters, and the image capture board available from VuTech
which is capable of overlaying a captured image in an EGA format over a
moving image in a variable size and location but with limited color
resolution. Still another prior art image processing system having
limitation on its flexibility is disclosed in U.S. Pat. No. 4,700,181
which is a graphics display system for creating composite displays of
stored image segments line by line in an iterative process using slices to
build up the composite image. Thus, none of these prior art systems have
the desired efficiency and economics of the present invention.
DISCLOSURE OF THE INVENTION
The present invention relates to an image processing system capable of
selectively merging graphics, text, digitized video frames and/or full
motion video from any composite or RGB component video source, such as
live video, video camera, video laser disc or video cassette recorder,
into a user selectable composite television display in which a number of
windows may be overlayed with the windows having variable size and
location under user control. One of the windows or the background may
contain motion video. A graphics coproessor controls the manipulation and
retrievable storage of instantaneously grabbed full motion video images
which are digitized and stored full size in a memory for enabling
selective merger of the images with the graphics, text and full motion
video in the composite television display. In order to store the
instantaneously grabbed ful motion video, such as live video, the read
memory control signals from the graphics coprocessor are intercepted and
write memory control signals are substituted therefor, thus, fooling the
graphics coprocessor such as the Intel 82786, into writing to memory when
it thinks it is reading. In order to enhance the color resolution of the
captured and, ultimately, retrieved still video images, when the full
motion video input, such as live video, is a full color signal, the input
signal is decoded into its red, blue and green components and is digitally
processed in parallel by individual graphics coprocessors, and is
recombined upon retrieval to provide a high resolution color display,
thereby providing the same range of color resolution as in the normal
color television picture.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a functional block diagram of a prior art configuration for an
Intel 82786 graphics coprocessor for combining live video and graphics;
FIG. 2 is a functional block diagram, similar to FIG. 1, of a presently
preferred embodiment of the present invention illustrating a new
utilization for an Intel 82786 graphics coprocessor in a configuration
capable of instantaneous capture of live full motion video images;
FIG. 3 is a functional block diagram, similar to FIG. 2, of a presently
preferred embodiment of the image processing system of the present
invention for providing high resolution color image capture of live full
motion color video images;
FIG. 4 is a functional block diagram illustrating the various bus
interfaces between the image processing circuit of the present invention
and the host computer's data, address, and control buses;
FIG. 5 is a more detailed functional block diagram of the embodiment of
FIG. 2, illustrating a typical processing circuit in the system of the
present invention;
FIG. 6 is a functional block diagram, partially, in schematic, of the
decoder circuit portion of the system of FIG. 3;
FIG. 7 is a functional block diagram, partially in schematic, of a typical
analog to digital converter circuit for use with the system of FIGS. 3 and
5, illustrating processing of the red color component of the color video
signal;
FIG. 8 is a functional block diagram, partially in schematic, of a typical
digital to analog converter circuit for the red color component of the
color video signal for use with the system of FIGS. 3 and 5;
FIG. 9 is a schematic diagram, partially in block, of the video output
circuit portion of the system of the present invention;
FIG. 10 is a schematic diagram, partially in block, of a reference voltage
circuit for the system of the present invention;
FIG. 11 is a schematic diagram, partially in block, of a power filter
circuit for the system of the present invention;
FIG. 12 is a functional block diagram of the video controller portion of
the system of the presnnt invention.
FIG. 13 is a diagrammatic illustration of the line TV signal source portion
of the system of FIG. 2; and
FIG. 14 is a detailed functional block diagram similar to FIG. 5, of the
embodiment of FIG. 2, illustrating a complete R, G, B processing circuit
in the system of the present invention.
BEST MODE FOR CARRYING OUT THE INVENTION
Referring now to the drawings in detail, and initially to FIG. 1, FIG. 1
illustrates a conventional prior art configuration for the Intel 82786
graphics coprocessor 20 employed as a video interface, such as described
in Chapter 6 of the 1987 edition of the 82786 Graphics Coprocessot User's
Manual, the contents of which is hereby specifically incorporated by
reference herein in its entirety. As shown in FIG. 1, the graphics
coprocessor 20 is connected to a CRT monitor or display 22 through a video
switch 24 which also receives a live television signal input 26 which may
be combined with the graphic output from the graphics coprocessor 20, such
as described in section 6.8 of the above User's Manual, as long as the
graphics coprocessor 20 is locked in synchronization with the live video
source 26. As further shown in FIG. 1, the graphics coprocessor 20 is
normally under control of a conventional host computer 28, with the
graphics coprocessor 20 normally providing memory read control signals via
control path 30 to the memory, such as a dynamic random access memory or
DRAM 32, associated with the graphics coprocessor 20 with data
bidirectionally passing between the graphics coprocessor 20 and the DRAM
32 via data path 34. As described in section 6.5 of the above User's
Manual, combining multiple 82786s in a system can provide greater
resolution of colors than a system having one 82786, such as the prior art
system of FIG. 1. In the above prior art system, there is no provision for
instantaneous grabbing of a full motion video signal such as the live
video signal 26 so that still video images in multiple windows having
variable size and location cannot be provided from the live television
signal 26, nor can such images therefore be overlayed on the live video
signal with or without the addition of graphics and/or text in the
composite video display 22.
Referring now to FIGS. 2-12, the presently preferred system of the present
invention for providing multiple, virtually unlimited variable windows of
still video images, graphics and/or text along with full motion from any
composite or RGB component video source, such as laser disks, still frame
recorders, video cassette recorders, video cameras and live television
signals 26, such as illustrated in FIG. 13, will be described. The system
of the present invention will be described as one for preferably providing
high resolution color images in which full motion color video images, such
as live color television signals 26, may be instantly grabbed using the
graphics coprocessor 20, which normally merely generates computer images
on a television screen, at random by the user, digitized and stored as a
full size video image, and manipulated to provide a variable size and
location window for each grabbed image in the composite video display 22a
in which the grabbed images are merged with text and/or graphics and/or
full motion video, such as live television signals or television signals
from any composite video source. Preferably, in order to enhance the color
resolution as shown and preferred in FIG. 3, and as will be described in
greater detail hereinafter, parallel processing of the color video signal
in its red, blue and green components is employed, with the retrieved
parallel processed red, blue and green components ultimately being
recombined in the video output circuit 40 (FIG. 9), although if desired
the components may be luminance and color difference signals. For the sake
of clarity, the same reference numeral is used throughout for the same
functioning component.
The image processing system of the present invention may preferably be
implemented as a single slot, AT compatible image processing board which
provides free flowing, real time digitalization of composite video and RGB
analog video inputs, such as NTSC, PAL, SEACAM or other similar video
formats. Real time full motion or moving video pictures, such as via
closed circuit TV or VCR/VTR outputs or live television signals 26, can be
displayed as real time full motion video pictures and captured as
independent still frames or frozen video images, with a time base
correction circuit preferably being incorporated into the image processing
system of the present invention in order to maintain high quality for the
captured or grabbed real time full motion video images.
As shown and preferred in FIGS. 5 and 14, three graphics coprocessors 20a,
20b, 20c, such as preferably three Intel 82786 graphics coprocessors, are
utilized in the image processing system of the present invention to
provide high resolution fast throughput of digitized video images. Fast
throughput is preferably delivered by dedicating each of the three
graphics coprocessors 20a, 20b, 20c to one of the three primary colors
red, blue, green, thereby preferably providing parallel processing of
video images. Of course, if a monochrome video signal is involved, then
only one graphics coprocessor 20 would be needed, such as the typical
configuration illustrated in FIG. 2. Each of the three color processing
circuits which each preferably contain an 82786 graphics coprocessor 20a,
20b, 20c and a DRAM 32a, 32b, 32c, preferably contain at least one
megabyte of storage capacity in each DRAM 32a, 32b, 32c. This quantity of
memory may be any desired size depending on the desired storage capacity
such as, by way of example providing for the storage of up to three
maximum resolution images (668.times.480 pixels for example for an NTSC
system and 668.times.575 for a PAL system) in the image processing system
of the present invention. Of course, the images may be reduced in size,
for example, from 668.times.480 pixels for an NTSC system to 334.times.240
pixels, with a resultant increase in the number of images which can be
stored.
The resolution of compute generated images can preferably be modified to
emulate CGA, EGA, and EGA+graphics and colors if desired. Multiple
windows, such as up to 16 by way of example, can be defined and displayed
on-screen, each with a different captured image. As previously mentioned,
one of the windows may contain a full motion video image or windows can be
displayed as overlays on a full screen real time moving or full motion
video image in the composite display 22, and each window's size and
contents can be independent of others displayed. Preferably, hardware
windows allow horizontal and vertical scrolling of images and windows, and
pixel zooming of up to 64 times is preferably supported. Preferably, other
features of the image processing system of the present invention include a
line and polygon drawing capability, such as, by way of example, at a
speed of 2.5 million pixels per second, a circle drawing capability, such
as, by way of example, at a speed of 2 million pixels per second, a solid
area shading capability, such as, by way of example, at a speed of 3.75
million pixels per second, a block transfer capability, such as, by way of
example, at a speed of 15 million bits per second, and a character drawing
capability, such as, by way of example, at a speed of 1200 characters per
second. Due to the use of three 82786 coprocessors 20a, 20b, 20c, a
maximum palette of 16.7 million colors, is available for display. At any
given time the number of colors from this maximum palette is determined by
the number of pixels chosen. The video input is preferably composite video
and the aforementioned time base correction circuit provides automatic
sync correction for any composite video input source for ensuring high
quality capturing and display of video images.
The image processing system of the present invention preferably digitizes
composite or component RGB video input signals. Preferably the video
images ar digitized as bit mapped screens, and, thus, the content of the
video images, for example text, graphics or TV pictures, does not affect
the image processing or the resolution of processed images. As a result,
the image processing system of the present invention is an ideal medium
for the digitalization, capturing, and display of video images, images
which can include, by way of example, scanned documents, photographs, VCR
outputs, VTR outputs, still frame recorder outputs, closed circuit TV
outputs, live video, and any other composite video outputs available.
Preferably, there are four subsystems which comprise the image processing
system of the present invention; namely, the bus interface, the color
processors, the video interface, and the video controller.
The bus interface, which is shown by way of example in FIG. 4, provides the
interface between the image processing system of the present invention and
the data, address, and control buses of the host compute 28. The bus
interface circuit is preferably comprised of conventional bidirectional
data bus drivers 42, 44, address bus latches 46, 48, programmable logic
arrays 50, 52, 54, 56 and various associated control circuits. The
bidirectional data bus drivers 42, 44 provides the interface between the
host computer's 28 data bus and the data bus of the image processing
system of the present invention. Driver 42 drives data bits 0-7 and driver
44 drives data bits 8-15. The direction in which the drivers 42, 44 are
enabled is preferably controlled by programmable logic array 50 which
interprets signals from the host computer's 28 control bus to select the
desired data bus direction.
Programmable logic array 52 generates the four most significant bits of the
addresses used by the color processor circuits 60a, 60b, 60c. In addition,
this circuit 52 controls the type of access to be performed on the color
processor circuits 60a, 60b, 60c; i.e., memory access or I/O access.
Select signals generated by this circuit 52 also enable any combination of
the color processor circuits 60a, 60b, 60c to perform read or write
operations.
When the appropriate control signals have been generated by logic arrays 50
and 52, the data bus outputs of bus drivers 42 and 44 are enabled, thereby
providing a path for the transfer of data between the image processing
system's color processor circuits 60a, 60b, 60c and the host computer's 28
data bus. Address latches 46 and 48 provide the interface between the host
computer's 28 address bus and the image processing system's address bus.
Signal "BALE" (buffered address latch enable) from the host computer's 28
control bus is used to enable address latches 46 and 48. The remaining
four address bits (Addr 16-19) of the host computer's 28 address bus are
latched into comparator 58, which compares the state of these four address
bits to the state of Switch 1-1, 2, 3, given reference numeral 72.
Comparator 58 performs this comparison to detect when a memory access is
being requested by the host computer 28. Signal "MAD" is generated by
comparator 58 and sent to logic arrays 52 and 56. Similarly, address bits
A3-9 are latched into comparator 78, which compares the state of these
address bits to the state of Switch 1-4, 5, 6, 7, 8, given reference
numeral 76.
As previously stated, logic array 50 uses the signal "MAD" as part of the
logical inputs necessary to determine the direction in which bidirectional
data bus drivers 42 and 44 will enable data transfers. Logic array 56 uses
the signal "MAD" as part of the logical inputs to determine whether an
interrupt must be sent to the host computer 28 for the initialization of a
16-bit data transfer, for example, or to disable signal "I/O Channel
Ready," which will force the host computer 28 into a wait state during a
data transfer and thereby provides enough time for the color processor
circuits 60a, 60b, 60c to complete a data transfer.
Logic array 54 preferably has four functions. The first function is the
generation of signal "SFTRS", a software selectable reset signal which is
used to reset the graphics controllers or coprocessors 20a, 20b, 20c
contained in the color processor circuits 60a, 60b, 60c, respectively.
This software selectable reset is preferably "OR'd" with the host
computer's 28 hardware "reset" signal. The second function of logic array
54 is the generation of signal "WDA" (write pulse), which is applied to D
to A converter (DAC) 762 which generates the four control voltages used by
the decoder circuits 64a, 64b, 64c. Converter 62 receives its reference
voltage from regulator 66. The third function of logic array 54 is the
enabling of the host computer 28 to rend the "Field Indent" (FI) and
"Double Line Frequency" (D2) signals generated by the image processing
system of the present invention. The fourth function of array 54 is the
generation of signal "GRAB" which is used to enable the storage of a video
frame.
As was previously mentioned with respect to FIGS. 5 and 14, by preferably
dedicating individual color processing circuits 60a, 60b, 60c to each of
the three primary colors, red, blue and green, a high speed, high quality
image processing system is established, with each circuit 60a, 60b, 60c
being dedicated to the processing of one of the three colors. A typical
one of these color processing circuits, such as 60a by way of example, is
shown in FIG. 5. As was previously mentioned each of the processing
circuits 60a, 60b, 60c preferably contains a graphics coprocessor 20a,
20b, 20c, respectively, such as preferably the Intel 82786 graphics
coprocessor. Digitized video images are sent to the graphics coprocessors
20a, 20b, 20c over the image processing system's data bus. Bidirectional
data bus drivers 80, 82 act as selectable buffers to link the image
processing system's data bus lines to the color processor circuit's 60a
internal data bus. Logic array 84 is the control signal interceptor which,
under control of the graphics coprocessor 20a, enables/disables drivers 80
and 82 when apropriate. In addition, interceptor 84 preferably generates
signals "WE" (write enable) and "BE" (bus enable) which are used to
control the storage and retrieval of data to and from the DRAM 32a, with
each color processing circuit 60a, 60b and 60c each preferably containing
its own DRAM 32a, 32b, 32c, respectively, such as a 1 Megabyte.times.8
bit DRAM. Address lines A0-A8 of the DRAM 32a are output by the graphics
coprocessor 20a.
The control signal interceptor 84 enables video frame grabbing in the image
processing system of the present invention. When the host computer 28
signals the grab control 54 to accept the live video, for example, the
grab control enables both a live video buffer 86 (FIG.2) and the control
signal interceptor 84. The live video buffer 86 preferably receives video
information the entire time the image processing system is in operation,
but does nothing with it until the Grab Control 54 is active. When the
Grab Control 54 is active, the live video buffer 86 sends the video data
to the memory 32a via the data bus (D0-D15). The control signal
interceptor 84, as previously mentioned, changes the RD or read signal
from the graphics coprocessor 20a to a WR or write signal and sends the
write signal to the video memory 32a. The live video signal is then passed
to the video memory 32a through the enabled video buffer 86 and stored as
a digitized full size video frame of instantaneously grabbed video.
Video data outputs from the graphics coprocessor 20a (VDATO-7) are
preferably applied to selectable buffer driver 88 which receives its
enable signal from the image processing system's control bus. When
enabled, driver 88 passes the video data through to first-in-first-out
(FIFO) buffers 90 and 92 or directly to the D to A converter 62a. Since
buffers 90 and 92 are preferably 8 bit FIFO buffers, the first eight video
data bits (VDO-7) are applied to buffer 90 and the second eight video data
bits (VDO-7) are applied to buffer 92. Although the color processor
circuits's 60a internal data bus contains sixteen data lines in the above
example, the 82786 graphics coprocessor 20a outputs only eight video data
bits at a time. This is why dual FIFO buffers 90 and 92 and their
alternating accesses is preferably employed. Signals "SIL" and "SIH"
control the selection of buffers 90 and 92 and are received from the image
processing system's control bus. Preferably, in order to provide a display
of real time full motion or moving video pictures without any noticeable
delays, the image processing system of the present invention enables the
FIFO buffers 90, 92 and the DRAM 32a to be directly bypassed. However, by
nevertheless preferably passing or cycling the real time full motion or
moving video picture through the image processing system of the present
invention, these pictures are enhanced and time base corrected, thereby
improving the quality of the real time moving video pictures.
FIFO buffers 90 and 92 , when enabled via signal "OE" from the image
processing system's control bus, transfer stored video data via the Color
Processor's 60a internal data bus to either the 82786 graphics coprocessor
20a directly or to the 1 Megabyte.times.8 bit DRAM 32a. This completes the
loop of video storage and retrieval within each of the Color Processor
circuits 60a, 60b, 60c. In order to better understand the flow of video
data within each of the Color Processor 60a, 60b, 60c, the following
operation shall be described below.
In normal operation, the 82786 graphics coprocessor 20a reads data from
memory 32a using an interleaved "fast page mode" technique; data being
read at approximately twice the rate that it will be displayed. The data
is read in "bursts" and is buffered by internal FIFO buffers which can
hold one-hundred and twenty-eight pixels. The 82786 graphics coprocessor
20a starts to read video data at the beginning of the horizontal blanking
period and continues until the FIFO buffers are full, which occurs before
the end of horizontal blanking. At the start of the active line period,
pixels are taken from the FIFO buffers and transferred to the 1
Megabyte.times.8 bit DRAM 32a. When the FIFO buffers empty to a
predetermined level, another memory access burst starts which continues
until the FIFO buffers are full again. This process continues until the
end of the displayed line.
Thus, the first memory access, after the start of a line, is to a memory
location one-hundred and twenty-eight pixels onwards from the beginning of
the memory map of the line. This method is used in the "grab" process by
preferably defining a special "grab" mode memory map which consists of two
tiles (vertical strips); the first tile consists of one-hundred and
twenty-eight pixels from the right side of the picture, and the second is
the remainder of the picture starting with the left edge.
Incoming video data is preferably buffed by an external FIFO buffer with
the same depth as the Color Processor circuits's 60a internal FIFO
buffers. Incoming video data is therefore arranged such that at the
beginning of a line the external FIFO buffer is empty. As explained above,
the internal FIFO buffer is full at this time. During the "frame grab",
memory accesses are overridden to be "write" accesses so that each memory
access represents one word "taken" from the external FIFO and one word
"put into" the internal FIFO buffer. Similarly, each pixel of video
represents one byte "taken out of" the internal FIFO and one byte "put
into" the external FIFO (from the real time moving video source). Since
the Video Controller circuit arranges the data flow so that the internal
FIFO buffer does not overflow or underflow, it follows that the external
FIFO buffer, which mirrors the internal FIFO buffer, starts out empty when
the internal FIFO buffer is full, and therefore will not overflow or
underflow either.
The internal FIFO buffer does not generally perform any useful function
during the "grab" process but the control of its operation ensures that
the external FIFO buffer operates correctly. In order for the handshaking
between the external and internal FIFO buffers to work correctly, delays
within the image processing system's subsystems must be accounted for. If
the external FIFO buffer is the "fall through" type, then they cannot be
completely emptied during the active line since several data words will
always be moving through them from input to output. Since the internal
FIFO buffers are completely filled at the end of each memory access burst,
this would cause an underflow condition. This can be avoided by defining a
further tile of field color at the left edge of the picture. The time
taken to "display" this tile is chosen to be greater than the "fall
through" time of the external FIFO buffer. Thus the external FIFO buffer
runs ahead of the internal FIFO buffers by this time so that data is
always available at the outputs of the external FIFO buffer when it is
required. This extra tile, however, leads to two further considerations.
The first consideration is that it would appear that there is now a danger
of the external FIFO buffer's overflowing, but this does not occur in
practice because the internal FIFO buffers are never allowed to become
much less than half full, and therefore the external FIFO buffer never
gets much more than half full. The total FIFO buffer depth is one-hundred
and twenty-eight pixels and a "dummy tile" of a sixteen pixel width is
sufficient to overcome the "fall through" problem, thereby establishing a
large safety margin.
The second consideration is what happens to the right edge of the memory
map if an extra tile is put in at the left. The 82786 graphics coprocessor
20a continues to read data for the defined memory map, even if the memory
map is larger than can actually be displayed; until blanking time. Memory
accesses are always at least sixty-four pixels ahead so the extra tile can
be accommodated.
When a frame is to be grabbed, the normal memory accesses for reading video
data are preferably overridden to store video data instead. However, other
types of memory accesses must also be considered in order to not cause any
undesirable effects. The four types of normal memory accesses are RAM
Refresh Cycles, Host Accesses, Graphics Coprocessor Accesses, and Display
Processor Control Accesses.
The RAM Refresh Cycles generated by the 82786 graphics coprocessor 20a are
"RAS only" cycles. In order for the memory devices to recognize such
cycles it is only required that signal "CAS" be high; the state of signal
"WRITE" is unimportant so no undesirable effects will result. However
insofar as Host Accesses and Graphics Coprocessor Accesses, they must not
be | | |
