 |
Claims  |
|
|
What is claimed is:
1. A digital video processor which replaces selected digital pixels in a
data stream with modified pixels without replacing unselected digital
pixels, comprising:
an input circuit which receives a data stream of digital pixels
representative of pixels in a video picture;
a selecting circuit which receives and selects digital pixels in said data
stream according to predetermined selection criteria and generates a
selection signal to indicate the selection of a digital pixel if said
digital pixel meets said predetermined selection criteria;
a modified pixel generating circuit which receives said selection signal
and generates a respective modified digital pixel in response to each
selected digital pixel according to predetermined modification parameters;
a first combining circuit which receives each said modified digital pixel
from said modified pixel generating circuit, and receives said data stream
including selected and unselected digital pixels from said input circuit,
and replaces each said selected pixel with a respective modified pixel, to
generate a modified data stream containing said unselected pixels combined
with said modified pixels; and
an output circuit for receiving and supplying said modified data stream as
an output of said digital video processor;
wherein said input circuit receives digital high-definition pixels in said
data stream at a given data rate; and
said digital video processor further comprises a circuit which reduces said
data rate before said selecting circuit selects said pixels.
2. A digital video processor as in claim 1, wherein said reducing means
reduces said data rate in said stream by interpolation.
3. A digital video processor as in claim 1, wherein said reducing means
removes pixels from said stream of pixel data.
4. A digital video processor which replaces selected digital pixels in a
data stream with modified pixels without replacing unselected digital
pixels, comprising:
an input circuit which receives a data stream of digital pixels
representative of pixels in a video picture;
a selecting circuit which receives and selects digital pixels in said data
stream according to predetermined selection criteria and generates a
selection signal to indicate the selection of a digital pixel if said
digital pixel meets said predetermined selection criteria;
a modified pixel generating circuit which receives said selection signal
and generates a respective modified digital pixel in response to each
selected digital pixel according to predetermined modification parameters;
a first combining circuit which receives each said modified digital pixel
from said modified pixel generating circuit and receives said data stream
including selected and unselected digital pixels from said input circuit,
and replaces each said selected pixel with a respective modified pixel, to
generate a modified data stream containing said unselected pixels combined
with said modified pixels; and
an output circuit which receives and supplies said modified data stream as
an output of said digital video processor; wherein
said selecting circuit and said modified pixel generating circuit have
respective memories which store corresponding sets of selection criteria
and modification parameters; and the modified pixel generating circuit
stores respective modification parameters for each of said sets of
selection criteria; and
further comprising a priority deciding circuit which stores decision data
indicative of which set of modification parameters will be associated with
given selected pixels when said given selected pixels meet a plurality of
selection criteria corresponding to a plurality of sets of selection
criteria.
5. A digital video processor as in claim 4, wherein said modified pixel
generating circuit comprises:
a modification data supplying circuit responsive to said selecting circuit
for supplying modification data for modifying said selected pixel
according to said predetermined modification parameters; and
second combining means for combining said modification data and said
selected pixel and thereby generating said modified pixel.
6. A digital video processor as in claim 5, wherein said digital pixels
comprise data which are representative of at least hue, saturation, and
luminance of said pixels; and said selecting circuit comprises:
means for storing digital selection data which indicate particular
selection criteria including hue, saturation and luminance values for
which corresponding pixels are to be selected; and
means for comparing said digital selection data with said digital pixels to
determine whether to select said pixels.
7. A digital video processor as in claim 6, wherein said selection criteria
further include texture of said pixels in said video picture; and
said selecting means further comprises: means for evaluating a texture of
said pixels in said video picture in response to said digital pixels; and
said means for storing stores digital selection data which indicate a
particular texture, for which corresponding pixels are to be selected.
8. A digital video processor as in claim 5, wherein said digital pixels
comprise data which are representative of at least hue, saturation,
luminance, and location of said pixels; and said selecting means
comprises:
means for storing digital selection data which indicate particular
selection criteria including hue, saturation, luminance, and location
values for which corresponding pixels are to be selected; and
means for comparing said digital selection data with said digital pixels to
determine whether to select said pixels.
9. A digital video processor as in claim 8, wherein said selection criteria
further include texture of said pixels in said video picture; and
said selecting means further comprises: means for evaluating a texture of
said pixels in said video picture; and said means for storing stores
digital selection data which indicate a particular texture, for which
corresponding pixels are to be selected.
10. A digital video processor as in claim 5, wherein said second combining
means further comprises threshold means for determining whether a
difference between a modified pixel and a corresponding selected pixel
data is below a predetermined threshold, and if so, not supplying said
modified pixel so that said selected pixel is not replaced.
11. A digital video processor as in claim 10, wherein said threshold means
determines a difference between said modified pixel and said selected
pixel, and disregards said modified pixel if said difference is below said
predetermined threshold.
12. A digital video processor as in claim 5, wherein said output circuit
comprises first smoothing means for receiving said output data from said
first combining circuit, detecting whether first transition values between
respective pixels corresponding to said output data exceed predetermined
first transition limits, and if so, reducing said first transition values
in said output data.
13. A digital video processor as in claim 12, further comprising second
smoothing means for receiving said modified pixels from said second
combining means, detecting whether second transition values between
respective modified pixels exceed predetermined second transition limits,
and if so, reducing said second transition values.
14. A digital video processor as in claim 13, wherein said first and second
smoothing means each comprise a convolver.
15. A digital video processor as in claim 5, further comprising smoothing
means for receiving said modified pixels from said second combining means,
detecting whether transition values between respective modified pixels
exceed predetermined transition limits, and if so, reducing said
transition values.
16. A digital video processor as in claim 15, wherein said smoothing means
comprises a convolver.
17. A digital video processor as in claim 4, wherein said selecting circuit
tests individual pixels according to said predetermined selection
criteria.
18. A digital video processor as in claim 4, wherein said selecting circuit
tests predefined groups of pixels according to said predetermined
selection criteria.
19. A digital video processor for modifying a data stream comprising
digital pixels and outputting a modified data stream comprising modified
pixels, the digital video processor comprising:
an input circuit which receives a data stream of digital pixels which
define pixels in a video picture according to a first coordinate system;
a first converter for receiving and converting said digital pixels to a
second coordinate system having data for each digital pixel respectively
corresponding to hue, saturation and luminance of said pixel;
a selecting circuit which receives said data stream from said first
converter and selects said digital pixels in said data stream according to
predetermined selection criteria, and generates a selection signal to
indicate the selection of a digital pixel if said digital pixel meets said
predetermined selection criteria;
a modification circuit which receives said selection signal and generates a
respective modification signal in response to each selected digital pixel
according to predetermined modification parameters within said second
coordinate system;
a second converter for converting each said modification signal from said
second to said first coordinate system;
a first combining circuit which receives each said modification signal from
said modification circuit, and receives said data stream from said input
circuit, and modifies each said selected pixel in response to said
modification signal, to thereby generate a modified data stream containing
digital pixels which define a modified video picture according to said
first coordinate system; and
an output circuit for receiving and supplying said modified data stream as
an output of said digital video processor.
20. A digital video processor as in claim 19, wherein said selecting
circuit tests individual pixels according to said predetermined selection
criteria.
21. A digital video processor as in claim 19, wherein said selecting
circuit tests predefined groups of pixels according to said predetermined
selection criteria.
22. A digital video processor as in claim 19, wherein said input circuit
comprises a third converter which receives and converts said data stream
from said first coordinate system to a third coordinate system, and a
fourth converter which receives and converts said data stream from said
third coordinate system to said first coordinate system.
23. A digital video processor as in claim 22, wherein said output circuit
comprises a fifth converter which receives and converts said data stream
from said first coordinate system to a third coordinate system, and a
sixth converter which receives and converts said data stream from said
third coordinate system to said first coordinate system.
24. A digital video processor as in claim 22, wherein said first coordinate
system is RGB.
25. A digital video processor as in claim 24, wherein said third coordinate
system is YUV.
26. A digital video processor as in claim 22, wherein said third coordinate
system is YUV.
27. A digital video processor as in claim 19, wherein said output circuit
comprises a fifth converter which receives and converts said data stream
from said first coordinate system to a third coordinate system, and a
sixth converter which receives and converts said data stream from said
third coordinate system to said first coordinate system.
28. A digital video processor as in claim 27, wherein said third coordinate
system is YUV.
29. A digital video processor for modifying a data stream comprising
digital pixels and outputting a modified data stream comprising modified
pixels, the digital video processor comprising:
an input circuit which receives a data stream of digital pixels
representative of pixels in a video picture;
a selecting circuit which receives and tests digital pixels in said data
stream according to predetermined selection criteria, said selection
criteria including a plurality of predetermined sets of independent
picture attributes and generates a selection signal to indicate the
selection of a digital pixel if said digital pixel has all of said picture
attributes in one of said sets;
a modification circuit which receives said selection signal and generates a
respective modification signal in response to each selected digital pixel
according to predetermined modification parameters;
a first combining circuit which receives each said modification signal from
said modification circuit, and receives said data stream from said input
circuit, and modifies each said selected pixel in response to said
modification signal, to thereby generate a modified data stream which
defines a modified video picture; and
an output circuit for receiving and supplying said modified data stream as
an output of said digital video processor.
30. A digital video processor as in claim 29, wherein said modification
circuit further comprises a priority circuit, wherein:
said priority circuit detects when said selecting circuit generates a
plurality of said selection signals, in response to one digital pixel
having the predetermined picture attributes in a corresponding plurality
of said sets of attributes, and
said priority circuit controls an order in which said modification circuit
employs the modification parameters corresponding to said selection
signals for generating said modification signal.
31. A digital video processor as in claim 29, wherein said selecting
circuit tests individual pixels according to said predetermined selection
criteria.
32. A digital video processor as in claim 29, wherein said selecting
circuit tests predefined groups of pixels according to said predetermined
selection criteria. |
|
 |
|
 |
Claims |
|
|
|
Description |
|
|
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to a digital video processor, and more
particularly to a secondary digital color processor (DCP) which is usable
with modern standard-definition digital telecines and is also compatible
with high-definition television systems, for correcting video color and
for other purposes.
2. Background Art
Telecines and external secondary color correctors (ESCC's) are used for the
production of high-quality television pictures in a post-production
television studio environment. The highest quality pictures normally
originate from motion picture film, normally 35 mm film. These pictures
are transcribed to video on a telecine, such as the Rank Cintel MkIII, the
Rank Cintel URSA, or the BTS FDL90. Telecine machines convert the picture
to a digital format and contain processing elements to enable simple
manipulation of the color reproduction of the resulting images.
This sort of color manipulation may also be performed by an external analog
system such as the RCA Chromacomp or its derivatives. This simple RCA-type
color processing has the limited capability of, for example, intensifying
the red component, over the entire picture, in all colors that have a red
component. However, it is incapable of, for example, altering only the
dark reds and leaving the light reds unchanged.
Over the years, improved ESCC's such as the PRISM from Encore Video
Industries, Inc., of California, have emerged. The PRISM is an analog
device which is capable of more selective color adjustment than the RCA
Chromacomp. Other analog ESCC's are the Da Vinci, and the "Sunburst" made
by Corporate Communications Consultants.
Color correctors have also been used for many years for tape-to-tape
correction. The output of one videotape recorder (VTR) is processed by the
ESCC, whose output is then recorded by another VTR.
ESCC's were traditionally analog, but now some color correctors are
available that are partly or fully digital. The output of most modern
telecines is digital.
Analog color correctors have several problems. Since a logical way to
interconnect the telecine to the color corrector is to connect the output
of the telecine to the input of the color corrector, it has normally been
necessary to convert the digital signal to analog, apply the color
correction, and then convert the corrected signal back to digital for
storage or further processing. The conversion from digital to analog and
back again causes degradation, and is therefore undesirable.
This phenomenon is most easily visible when one feeds a color signal
through a color corrector with all of the controls set to zero (which in
theory, should not change the colors). When one compares the output of the
color corrector with a signal which bypasses the color corrector, quite
often, there will be a difference between these two signals, which will
become apparent when the two images are viewed "split-screen".
One reason for this discrepancy is that the signals in the electronic
signal path (particularly in analog) are unintentionally modified due to
the imperfections of real electronic circuits.
Another reason is that since the color corrector is an analog device, it
will suffer from drift, which is inevitable in analog devices. Drift
originates from many sources, one of which is temperature.
A third reason for the visible differences is noise. This noise may be
visible as a change in level (and therefore color) or a difference in
texture of a given flat color bar.
Even if, as has occasionally been done, the signals are processed in a
totally digital domain, there have been degradations.
One reason may be digital "rounding", such that a color passing through a
color corrector unintentionally becomes slightly modified, compared with
the input.
Further, in known color correction systems (particularly digital systems),
discontinuities occur. This happens where colors below a given limit are
to remain unmodified, but colors above that limit are to be modified. If
the picture to be modified contains an increasing ramp of that color, then
the portion of the picture around the limit will show an unnatural
discontinuity when it is color-corrected.
The known color-correction devices will be discussed further below, in the
context of the following discussion of their use with telecine devices.
Referring first to FIG. 1, there is seen a schematic block diagram of the
most basic type of stand-alone telecine 20, which is usable to broadcast a
film directly over the air in real time. A film transport 21 runs a film
22 past a CRT scanner 24, which incorporates photomultipliers for
generating red, green, and blue component signals designated R.sub.v,
G.sub.v, and B.sub.v. Gain, lift and gamma of the three signals are
adjusted by a processor 26, and color is adjusted by a processor 28.
Simple overall picture adjustments such as lightening and darkening are
provided by a local control panel 30. Adjustments are performed live, as
the film is broadcast. The local control panel 30 also contains controls
such as a START control for the film transport and an ON control for the
CRT scanner. A disadvantage of this basic system having only the foregoing
components is that, in order to provide interlace for producing a
conventional broadcast signal, it is necessary for the lines in each film
frame to be scanned non-sequentially, that is, lines 1, 3, 5, . . . ,
followed by lines 2, 4, 6 . . .
An improvement upon the foregoing basic system, developed in the late
1970's, is the addition of a store 32, also shown in FIG. 1. The store
improves the scanning process, by permitting the lines of each frame to be
scanned sequentially, and then read out of the store with interlace. This
feature was patented by Rank Cintel in British patents 1,542,213 and
1,535,563, and equivalent U.S. Pat. Nos. 4,184,177 and 4,127,869,
respectively.
An improvement shown in FIG. 2 is a controller/programmer 30', which as
shown is an accessory for the telecine 20', although it could conceivably
be built into the telecine 20'. The controller/programmer 30' provides a
control panel for the processors 26, 28, and the film transport 21. It
further includes a programming function, whereby an operator can rehearse,
slowly, the optimum grading and picture adjustments for each scene in the
film, which the programmer stores. A key feature is that the programmer is
driven by time code, or by film footage, so it can tell one scene from
another by the time code or by the position of the film.
One example of a controller/programmer is the POGLE telecine
controller/programmer manufactured by Pandora International Ltd., which is
designed for use with a range of telecine machines. It is capable of
providing a large number of control signals simultaneously, for example 96
control channels, either analog (a voltage within a predeterminable range)
or digital. The channels can be used to control a telecine and/or other
peripheral devices, such as noise reducers, VTR's, still stores, mixers,
and color correctors (such as the DCP disclosed herein).
Another example of a controller/programmer is the "Rainbow" system of
Corporate Communications Consultants.
A further improvement, shown in FIG. 3, is the external secondary color
corrector (ESCC) 34. Examples are the RCA Chromacomp, the PRISM, and the
Da Vinci. ESCC's can be either digital or analog. Instead of simple RGB
control, which only gives the ability to add or subtract red, green, and
blue, everywhere in the picture, this advanced generation of ESCC's
provides 6-or-more-channel color control. That is, for example, 6 separate
colors can be selected and then modified, without modifying any other
colors. The ESCC 34 is controlled by the controller/programmer 30".
The ESCC 34 takes its inputs from the color processor 28 in the telecine
20", and provides its outputs to the store 32. The reason for this signal
path is that the store is usually capable of handling lower bandwidth
(i.e., fewer bits or lower resolution) than the signals from the processor
28, so the number of bits is reduced, for example, by rounding, before the
signals are stored. For example, the URSA's output is medium-bandwidth
"4:2:2" color (D-1 format). Thus, taking the input to the ESCC 34 from the
output of the processor 28 allows the ESCC to operate on a
higher-bandwidth signal than if it operated on the output from the store.
The trend is for telecines to be made digital and to incorporate improved
color correction facilities. For example, one advanced digital telecine,
the Rank Cintel URSA telecine, has a built-in digital implementation of a
6-vector RCA-type processor, in place of the analog processor 28 shown in
FIGS. 1-3. It would be desirable to provide an ESCC which is capable of
processing the digital output from the internal digital color processor 28
in the URSA (or the like), having greater capacity and flexibility than
that RCA-type color corrector.
A further trend is to increase the bandwidth of the store 32, which will
make it practical for the digital color processor to perform its functions
on the digital output of the store, making it unnecessary to connect the
ESCC to any internal circuits of the telecine.
In either case, an ESCC is needed which will not noticeably reduce the
bandwidth of the digital color signal or degrade the picture.
The disclosures of all prior art publications mentioned herein are
expressly incorporated by reference.
SUMMARY OF THE INVENTION
The present invention is able to provide remedies for the above, and other,
disadvantages of the prior art.
According to one important feature of the invention, only pixels that are
specifically selected to be modified are processed by the digital
circuitry. The pixels that are not to be modified are passed through the
DCP without any processing that could create rounding or other errors.
In contrast, in a conventional architecture, all of the pixels in the
picture would be processed through the same signal modification path,
possibly being converted from red, green and blue (RGB) to hue, saturation
and luminance (HSL), and then back again to RGB, causing errors.
Pixel selection advantageously is carried out by using the architecture
referred to below as the "pixel identification table" or alternatively as
the "cache tag RAM". The pixel identification table stores digital bits
which define which pixels will be selected from the pixel stream for
modification. Pixels may be selected as a function of their color (hue) as
in prior systems, and/or as a function of other criteria, such as
saturation, luminance, (X,Y) pixel coordinates, sharpness, and texture.
Further, after a pixel or region to be changed has been isolated, other
parameters besides (H,S,L) color attributes can be changed. For example,
the sharpness or even the (X,Y) coordinates of a region can be changed.
Modifying the (x,y) coordinates of a region would be useful, for example,
for special effects such as moving an object in the picture. Detecting
pixels according to their (X,Y) coordinates could also be useful for
copying pixels at a given x,y from one frame to another for scratch
concealment. The latter process might be carried out simply by, for the
given X,Y, controlling the frame store of the DCP (discussed below), so
that those specific pixels are not overwritten from frame to frame.
According to another important feature, a very minute modification will be
disregarded and not applied to the input signal, since such a minute
modification may be an inadvertent mathematical error.
The present invention avoids the problem of discontinuities as well. The
known digital color correctors process one picture element (pixel) at a
time, and have no knowledge of picture elements adjacent to the element
being processed. For example, brightening an actor's lips by simply
intensifying the reds will result in bright shimmering spots on the lips,
since in practice not all of the pixels in the original lips are of equal
saturation and luminance, nor are they all red. The DCP preferably has a
first convolver which has knowledge of preceding and following pixels on
that line, and also neighboring picture elements on preceding and
following lines. By convolving the adjacent pixels in the actual picture,
including unmodified pixels, the first convolver can provide gradual
transitions. Advantageously, a second convolver receives just the R, G,
and B modification values of the pixels that are to be modified. Both
convolvers preferably perform a mathematical 3.times.3, 5.times.5, or
similar convolution on the array of picture elements.
The convolvers afford a more gradual change from "unmodified" to "modified"
signals, thus removing any discontinuity. The disclosed scheme smooths out
both the modification increments between adjacent modified pixels, and
transitions between the resulting pixels as displayed, including
transitions between modified and unmodified pixels.
Thus, there are two levels of convolution in the DCP. The first convolver
modifies the boundaries between selected and not-selected regions. The
second convolver selectively modifies parts of the picture, in response to
any or all of the selection criteria that the DCP can detect and respond
to.
For example, if an original picture contains a region where the color
gradually transitions from red to green, and if it is desired to alter the
reds but not the greens, there will be transition problems on two levels.
The first level will be referred to as the "macro" level. After a
substantial adjustment of the reds, a line will be clearly visible between
the reds and the (unmodified) greens, rather than a gradual transition.
The first convolver addresses this problem by processing both modified and
unmodified pixels to smooth out the macro transition effect. There will
also be a discontinuity on a "micro" level. Film is made up of grains, or
more precisely dye clouds, which have fixed colors. The apparent change of
color in the original picture corresponds to a decreasing density of red
grains during the transition into the green region. However, with a
high-definition color corrector, it is possible to pick out individual
film grains and change them. This type of modification is not usually
desirable. It will cause a visible lack of smoothness in the transition
from the red region into the green region, because in that transition area
the DCP will select and modify half the grains, but not the other half.
The second convolver addresses that "micro" problem. It smooths out the
color by converting a collection of red and green grains into an even
mixed color, to prevent the poor appearance which results from changing
only some of the pixels in the transition region.
According to a further aspect of the invention, which is useful
particularly for a high definition version of the DCP, the DCP has a high
definition, wide bandwidth, main signal path, but a standard bandwidth (or
at least lower than HD) signal modification path. This involves a
downsampling before the signal modification path, and an upsampling after
the signal modification path. This novel architecture achieves the
principle of not downsampling or changing in any way the unmodified
content of the HD picture, and only imposes bandwidth limitations in the
parts that have been modified. Such bandwidth limitations have been found
to make little or no difference in the perceived quality of the picture.
For example, if the changes in the picture are mainly a chrominance
modification, rather than a luminance change, then this will produce
results very nearly as good as doing the modification signal path in full
HD bandwidth, as it is known that observers are less sensitive to high
frequency color changes than high frequency luminance changes. The
foregoing method could be described as "real-time pseudo-HD."
In view of its great digital processing power, the DCP is also capable of
"non-real-time real-HD". Without subsampling, the system can perform full
high definition secondary color correction, but slower than the normal HD
video rate. Also, film-type resolutions, up to for example 8000 by 8000
pixels, can be corrected in non-real-time. | | |
