 |
Claims  |
|
|
What is claimed is:
1. A system for periodically refreshing a display frame on a display
device, which display frame is formed from a plurality of display pixels
and is adapted to display a plurality of images, each image appearing as a
separate image frame, the image frames being displayed forming said
display frame, the system comprising;
at least one source for providing image frames;
frame memory means which is divided into a plurality of memory frames, each
of said memory frames having a sufficient number of memory positions to
store pixel data for a single display frame, and each of said memory
frames being subdividable into a plurality of memory image frames, each of
which has a sufficient number of memory positions to store an image frame;
means for selectively storing image frames from said at least one source in
selected memory image frames in selected memory frames in said frame
memory means:
means operative during each periodic refreshing of the display device,
where the device is displaying a plurality of images which were stored in
memory image frames of different memory frames, for selectively reading
out the image frames from said frame memory means, said readout means
reading out image frames from at least two memory frames within the frame
memory, each read-out image frame corresponding to an image frame of the
display frame, and each image frame of the display frame, each image frame
being read out from the memory frame in which it is stored which memory
frame is independent of and may be different than the memory frame for
other readout images; and
means for utilizing each readout image frame to control refresh of at least
one image frame displayed on said display device.
2. A system as claimed in claim 1 wherein said means for selectively
storing includes means for providing selected information concerning each
image frame to be stored in said frame memory means, and means responsive
to said selected information for controlling the mapping of values stored
for the pixels of the image frame when stored in said frame memory means.
3. A system as claimed in claim 1 wherein said means for selectively
storing includes means for storing in a single memory frame of said frame
memory means the image frames to be displayed, said image frames being
stored in compressed form; and
wherein said read-out means includes means for reading out said single
frame to control refresh of the display.
4. A system as claimed in claim 3 wherein said compressed form is obtained
by selectively cropping the image.
5. A system as claimed in claim 3 wherein said compressed form is obtained
by selectively dropping pixels of the image.
6. A system as claimed in claim 1 wherein said means for selectively
storing includes means for storing in compressed form each image frame to
be displayed in an independent memory frame of said memory frame; and
wherein said reading-out means includes means for reading out appropriate
pixels from the appropriate memory frame of the frame memory to control
refresh of the display of each pixel of the display.
7. A system as claimed in claim 6 wherein said compressed form is obtained
by selectively cropping the image.
8. A system as claimed in claim 6 wherein said compressed form is obtained
by selectively dropping pixels of the image.
9. A system as claimed in claim 6 wherein the means for selectively storing
is operative to perform the compression function.
10. A system as claimed in claim 1 wherein said at least one image source
includes at least one source of digital image frames, and at least one
source of analog image frames;
wherein said means for selectively storing includes means for converting
image frames from said analog source to digital; and
wherein the simultaneously displayed images may include an image from the
digital source and an image from the analog source.
11. A system as claimed in claim 10 wherein said digital image source is an
ultrasonic image scanning system; and
wherein said source of analog images is a source of stored ultrasonic
images.
12. A system as claimed in claim 1 wherein said at least one image source
includes a source of ultrasonic scan images;
wherein an ultrasonic scan image may contain information of at least two
different types;
wherein said means for selectively storing including means for storing the
scan image;
wherein said means for reading out includes means for selectively reading
out appropriate pixels of said stored scan image a number of times equal
to a number of images of said scan image to be simultaneously displayed on
said display; and
wherein said utilizing means includes means for selectively processing the
information types in the multiple read-out scan image so that each display
image contains such information in different permutations and
combinations.
13. A system as claimed in claim 12 wherein said permutations and
combinations include summing said types of information and displaying each
of said types of information separately.
14. A system as claimed in claim 12 wherein said permutations and
combinations include thresholding of both types of data or thresholding of
one type of data while displaying the other type without thresholding.
15. A system as claimed in claim 12 wherein the two types of information
are tissue data and flow data.
16. A system as claimed in claim 1 wherein said read-out means may read-out
a single image frame from said frame memory means two or more times during
a single refresh of a display frame; and
wherein said means for utilizing includes means for selectively processing
said image frame to cause multiple images of said frame to be displayed,
said multiple images containing the content of said image frame in various
permutations and combinations.
17. A system as claimed in claim 1 including means for selectively color
mapping each of the simultaneously displayed image frames.
18. A system as claimed in claim 17 wherein each image frame may contain
two different types of information; and
wherein said color mapping means includes means for independently color
mapping each of said types of information.
19. A system as claimed in claim 18 wherein one of said types of
information is normally to be displayed in color and the other type of
information is normally to be displayed in black and white.
20. A system as claimed in claim 1 wherein said at least one image source
includes a first image source which presents pixel information for an
image frame in a first format and a second image source which presents
pixel information in a second formate different from the first format;
wherein said means for selectively storing stores pixel information in both
said first and second formats in said frame memory means;
wherein said reading out means reads out the pixel information in the
different formats; and
wherein said utilizing means utilizes the read-out pixel information in the
different formats to control the display on said display device.
21. A system as claimed in claim 20 wherein said first image source is a
digital source which presents the pixel information in a selected digital
format and wherein said second source is an analog source which presents
the pixel information in a standard composite video format.
22. A system as claimed in claim 1 wherein said means for reading out
includes means for selectively cropping stored images.
23. A system as claimed in claim 22 wherein said means for selectively
cropping includes means for storing a horizontal and/or vertical address
at which reading out of a selected memory frame is to begin, and means for
terminating readout of said selected memory frame after a selected number
of pixels and/or lines have been read out.
24. A system for periodically refreshing one or more simultaneously
displayed images on a video display device, which device is of a type
wherein each image appears as an image frame of a display frame formed
from a plurality of display pixels, the system comprising;
at least two sources of image frames, one of said sources being a digital
source presenting pixel data in a first selected digital format, and the
other source presenting pixel data in a second selected format;
frame memory means containing a plurality of memory frames each of which is
subdividable to store one or more image frames for display on said display
device;
means for selectively storing image frames from said at least two image
sources in the memory frames of said frame memory means, said frame memory
means storing image frames from said digital source in said digital format
and storing image frames from said other source in digital form in said
second selected format;
means for indicating the format for each stored image frame;
means operative during each periodic refreshing of the display device,
where the device is displaying a plurality of images which were stored in
memory image frames of different memory frames, for selectively reading
out image frames from said frame memory means, said readout means reading
out image frames from at least two memory frames within the frame memory,
each read-out image frame corresponding to an image frame of the display
frame, and each image frame being read out from the memory frame in which
it is stored, which memory frame is independent of and may be different
than the memory frame for other readout images; and
means for utilizing said read out image frames in the corresponding format
from said indicating means to control each refreshing of the at least one
image displayed on said display device.
25. A system as claimed in claim 24 wherein said other source is an analog
source, and wherein said second selected format is a standard composite
video format. |
|
 |
|
 |
Claims |
|
|
|
Description |
|
|
FIELD OF THE INVENTION
This invention relates to video display systems and more particularly to a
method and apparatus for providing a high level of flexibility in the
image frames which are available for display, either singly or in multiple
image combinations, including the capability of separately displaying
various types of information from a single image frame, for independently
mapping image levels and colors in each displayed image frame, and for
simultaneously storing and displaying images from different sources and in
different formats in a manner transparent to the user.
BACKGROUND OF THE INVENTION
In many applications, including scientific, industrial, educational,
medical and military applications, information received from a single
source, from a single source at different times, or from multiple sources
are provided on a video display. Various windowing techniques are
available for permitting two or more of such images to be simultaneously
displayed to either provide the user with additional information or to
permit various comparisons to be made.
while existing systems of this type offer many capabilities, such systems
normally have limited flexibility and lack various useful options. For
example, most such systems are adapted to only receive information from
digital sources and in particular digital sources which present the
display pixel information in a particular format. These systems do not
have the capability of mixing information received from both digital and
analog sources, which sources may present the information in different
formats, and to both store such information in a common memory and to
simultaneously display such information on a single monitor, all in a
manner transparent to the user. Such systems also frequently lack a
capability for mapping received information in a flexible manner to obtain
display levels and most such systems do not have the capability of
flexibly color mapping images to be displayed, which images are in either
color or black and white, with a capability for color mapping image frames
being simultaneously displayed using different mapping guidelines for at
least selected ones of the displays. Another limitation with most existing
systems is that they do not have the capability of separating various
types of information available in a single image frame and for
simultaneously processing such different types of information in different
ways with the results of such processing being simultaneously displayable
either alone or in conjunction with other information.
Further, for maximum flexibility in displaying multiple images, it is
desirable to be able to compress or crop images horizontally and/or
vertically as they are being stored with little or no additional steps and
to be able to easily crop stored images for display.
A need therefore exists for a method and apparatus for controlling the
display of one or more image frames on a display device which provides
enhanced capability and flexibility over existing systems. In particular,
such capability is required in conjunction with medical ultrasonic
scanning equipment which is adapted to receive both black and white tissue
data and Doppler color flow data from a single ultrasonic scan, and may
also receive information concerning prior ultrasonic scans on the same
patient, or other information useful in the diagnosis or treatment of a
patient, from a video recorder or other analog source in a standard
composite video format rather than in a selected digital black and white
or color format. In such an application, a need may exist for storing
and/or simultaneously displaying the differently formatted inputs, for
quantizing information received from scans, for separating tissue and
color flow data for display and analysis and for manipulating the color
mapping of either the color flow data or the normally black and white
tissue data to more clearly show information needed by the physician or
other medical personnel utilizing the system.
SUMMARY OF THE INVENTION
In accordance with the above, this invention provides a system for
displaying at least one selected image, and preferably two or more
selected images, on a video display device of the type which causes an
image to appear as an image frame in a display frame formed from a
plurality of display pixels. The system may include one or more sources of
image frames which sources may be digital or analog and which sources may
present pixel data in different selected formats. A frame memory is
provided which is adapted to store a plurality of memory frames each of
which may store one or more image frames to be displayed. Image frames are
selectively stored in the memory frames of the frame memory in the format
in which they are received. Selected information concerning each image
frame being stored may be utilized to control the mapping of values stored
for pixels of the image frame. The images may be stored in full size, or
may be compressed for windowing display, either by cropping, by
selectively dropping pixels or by other known compression techniques. A
display frame may be formed of two or more compressed image frames with
each image frame being read out from a selected memory frame which memory
frame may be independent of or the same as the memory frame for other
read-out images.
The two or more frames simultaneously displayed on the display device may
be a digital image and an analog image from separate sources. In an
ultrasonic scanner application, the digital image may be an ultrasonic
image obtained from a current scan while the analog image is an image,
obtained from for example a video recorder, of a prior scan on the same
individual.
Where images contain two or more different types of information, such as in
an ultrasonic application where the image may contain both tissue and
color flow information, processing means may be provided which processes
the received image in a selected way to cause the content of the multiple
information images to be displayed as multiple separate images containing
the content of the image frame in various permutations and combinations.
For example, image frames being simultaneously displayed may contain
either one or both types of information, with thresholding being provided
for one or both types of information for selected ones of the image frames
being displayed. Color mapping using the same or different criteria may
also be provided, with the color mapping being independent for each of the
image frames being simultaneously displayed.
The foregoing and other objects, features and advantages of the invention
will be apparent from the following more particular description of a
preferred embodiment of the invention as illustrated in the accompanying
drawings.
IN THE DRAWINGS
FIG. 1 is a schematic block diagram of a system in which the teachings of
this invention may be utilized.
FIG. 2 and FIG. 3 are schematic block diagrams of a black and white control
circuit and a color control circuit respectively suitable for use in the
frame memory circuits shown in FIG. 1.
FIG. 4 is a schematic block diagram of a video I/O circuit suitable for use
as the video I/O circuit in FIG. 1.
FIG. 5 is a diagram of an exemplary four quad display obtained utilizing
the teachings of this invention.
FIGS. 6 and 7 are schematic block diagrams of an input address circuit and
an output address circuit respectively suitable for use in the circuit of
FIG. 2.
DETAILED DESCRIPTION
Referring first to FIG. 1, the major components of a video display system
in which the teachings of this invention might be employed are shown. In
this drawing, and in the remaining circuit diagrams, lines which carry
digital signals are illustrated with a diagonal line which line may also
contain an indication of the number of wires or lines which form the
digital bus or cable. Lines which contain analog signals have a circle in
them.
Thus, in FIG. 1, the circuit 10 has one or more digital image sources 12
each of which sources may be connected by a black and white bus 14A (the
SBW bus) and a color bus 14B (the SCL bus) to frame memory circuits 16 and
video I/O circuit 18. For a preferred embodiment, image sources 12 are
various types of information available from an ultrasonic scanning system
such as tissue and color flow data or M mode data. The former will be
discussed in greater detail later while the latter is the image which
results from a sequence of scan lines in a single scan direction. The
number of lines in buses 14 will vary with the image sources 12 and with
the degree of resolution required from the system. In a typical
application, such buses might contain 8 or 9 lines each. The digital
format in which pixel information is presented (i.e., the coding) for the
black and white and/or the color bus may also vary with application.
In addition to digital images obtained from sources 12, images may also be
obtained from an analog source or sources such as video tape recorder
(VTR) 20 over analog lines 22, such inputs being applied to video I/O
circuit 18. The format for the analog pixel data would typically be a
standard composite video format and would normally be a different format
than the digital format on lines 14. Video I/O circuit 18 may also apply
analog signals over analog lines 24 to VTR 20, may apply blue, green and
red video data signals over lines 26 to control video display 28 and may
apply analog signals derived from VTR 20 over lines 30 to frame memory
circuit 16. As will be discussed in greater detail later, in order to
reduce memory utilization requirements, the analog information to circuit
16 is converted into a luminance level (Y) on line 30A, a
red-minus-luminance (RMY) signal on line 30B and a blue-minus-luminance
(BMY) signal on line 30C and is digitized and stored in this format. Thus,
the pixel data obtained from VTR 20 may be stored in a different format
from the digital data received in lines 14.
Digital signals from frame memory circuit 16 are applied over lines 32 to
video I/O circuit 18. The lines 32A, 32B and 32C carry the red, green and
blue component respectively if data originally stored from an analog
source such as VTR 20 is being output from frame memory circuits 16. When
information originally stored from a system bus 14 is outputted from
memory circuits 16, black and white data is outputted on line 32B and
color data is outputted on line 32C. As will be discussed in greater
detail later, for some types of image frames, both black and white and
color data are outputted.
In addition to the digital image source(s) 12, and the analog image
source(s) 20, image information may also be stored in frame memory
circuits 16 directly from a control processor 34 over digital lines or bus
36. Control processor 34 also generates various control signals over lines
38 to frame memory circuits 16 and over lines 40 to video I/O circuit 18.
The signals on lines 38 and 40 may perform a variety of functions which
will be described in greater detail later, including indicating the
current state or mode of the system for the image currently being read
out, setting threshold levels, providing mapping information to various
control RAMs and controlling the storing and reading out of information
from the frame memory circuits 16. The system user may provide control
inputs to processor 34 from a number of standard I/O devices including
keyboard 42 and roller ball 44.
The video display 28 shown in FIG. 1 may, for example, be a standard
cathode ray tube monitor which has a matrix of display pixels, for example
512.times.512, each of which may provide a selected color or grey scale
display. The system shown in FIG. 1 is adapted to generate a variety of
image displays on video display 28. The display on video display 28 may,
for example, be a single video image from an image source 12, an image
source 20, or control processor 34 which image fills the entire screen of
display 28. Alternatively, as shown in FIG. 5, the screen may be divided
into a selected number of areas or windows, four such areas being shown in
FIG. 5. Under these circumstances, one image frame of the display, for
example upper left image frame 50A, may display an image of the current
system information being outputted on buses 14. A second image frame may
contain old information on the individual currently being examined which
image data is obtained from VTR 20. This image may for example be
displayed in upper right image frame 50B of the display. Either the
current information or the old information may be applied to video I/O
circuit 18 directly from the source or may be initially stored in circuits
16 and applied to the video I/O from these circuits. While in some
applications the current and old information may each occupy half of the
display screen, in FIG. 5, for illustrative purposes, two additional types
of information are shown. Thus, bottom left frame 50C contains an image of
only the tissue portion of the composite current image being shown in
frame 50A while bottom right frame 50D shows only the color flow portion
of the image in frame 50A. As will be discussed in greater detail later,
each of the images in areas 50A-50D may be independently processed to, for
example, threshold selected information in the image, color map the image
in a selected way, or perform some other desired processing function.
The capability for performing the multiple image display function with
flexible controls is provided by frame memory circuits 16 and video I/O
circuit 18 under control of processor 34.
FIGS. 2 and 3 respectively show the black and white circuit 56 and the
color circuit 58 which form frame memory circuits 16. Referring first to
FIG. 2, the luminance input on line 30A is applied through a buffer
amplifier 60 and an A to D converter 62 to a line 63 which is one input to
a multiplexer 64. The output of multiplexer 64 is connected as the
addressing input to a mapping look-up table RAM 66. The other input to
multiplexer 64 is system black and white bus 14A. MUX 64 and the other
MUX's in the system may be controlled by appropriate signals from
processor 34 or by other suitable control signals. RAM 66 permits
manipulation or mapping of the received input based on the type of input,
the source of input, or other information relevant thereto. For example,
differences in the digitized input values to MUX 64 for a desired gray
scale output may be compensated for in mapping RAM 66. Contrast in the
displayed image may also be enhanced by remapping received black and white
gray scale values over a wider range or with greater separation between
levels. If for a certain type of data most of the information is grouped
in a particular area of the gray scale, increased spacing between levels
may be provided in this area alone to enhance contrast. Mapping of
gray/scale levels may also be performed for other purposes.
RAM 66 contains a plurality of gray/scale tables, each of which tables may
be addressed by the digital value on the output lines from multiplexer 64.
The table which is accessed to control mapping at any given time is
determined by the value on lines 68 from programmable logic array 70.
Array 70 may receive an input over line 72 from the appropriate one of the
digital sources 12 when the input is from such source or may receive a
control from processor interface 74 over line 76 under other conditions.
Line or lines 72 contain flag bits from the sources which identify the
type of data. Such lines may form part of the data bases 14 or be in
addition to such bases. The number of lines 68 will depend on the number
of different types of inputs or mapping options which are to be provided.
For example, if there are eight types of data or mapping options, three
lines 68 would be provided.
The data outputted from RAM 66 is applied through merge point 78 and lines
80 as the data input to frame memory 82. The other input to merge point 78
is the data input from the control processor on lines 36 (FIG. 1) which
passes through interface 74 and lines 84. The image frame information
inputted on lines 80 is stored at address locations in frame memory 82
which are determined by an input address on output lines 85 from input
address circuit 86. The input address circuit operates under control of
the processor and processor interface. Input address circuit 86 permits an
input image frame to be compressed or clipped so as to be displayable in a
half screen, quarter screen, or other selected size. The input address
circuit also controls the memory frames in which received image frames are
stored and, for compressed images, in which quad or other division of the
frames the compressed image is stored. The manner in which the input
address circuit 86 functions to perform these operations will be described
in greater detail hereinafter.
An output address circuit 88 is also provided which generates output
addresses on lines 87 under processor control to read out either a single
display frame on CBW line 32B to video I/O circuit 18 or to read out
selected compressed (or clipped) frames in selected pixel order over line
32B. As will be discussed in greater detail later, the output address
circuit may also clip a stored image or may read out a given compressed or
clipped frame a number of times in selected pixel sequence with the read
out information being processed differently in the video I/O circuit
during each read out. The output address circuits 88 will also be
discussed in greater detail hereinafter.
As was previously discussed, information is originally derived from
different sources, for example digital sources 12 and VTR analog source
20, which sources present the information in different formats. For black
and white information, these different formats are not of concern since
the inputs in different formats are mapped to be in the same format in RAM
66 before being stored in frame memory 82. However, as will be discussed
shortly, color data is in fact stored in different formats. Thus, the
pixel information outputted from frame memory 82 appears on CBW line 32B
if the system is in the black and white mode for the particular image
frame being displayed. When the system is in the system color mode, both
black and white tissue data and color flow data may be outputted
simultaneously from their respective memories. However, when the system is
in the VCR color mode, an output is obtained, as will be described later,
only from the color card, black and white information being obtained by
setting the values of all three color outputs (red, green and blue) to
equal values. Thus, program array logic (PAL) 89 is provided which
receives as inputs a signal indicating whether the image to be displayed
in a given quad is VCR color data over a plurality of lines 91A-91D. Thus,
there is a signal on line 91A if the image being displayed in the top left
quad 50A (FIG. 5) is VCR color data, a signal appears on line 91B if the
image appearing in top right quadrant 50B is VCR color data etc. The
remaining inputs to PAL 89 are a horizontal switch signal on line 93 and a
vertical switch signal on line 95. The horizontal and vertical switch
signals occur at the points in the image generating scan where transitions
occur from one quad to the other. PAL 89 generates an output on VCRCL line
97 if, for the quad being scanned at a given time, there is no signal on
the corresponding line 91 while PAL 89 generates an output on VCRCL line
99 if there is a signal on line 91 for the quad being scanned at the given
time.
The VCRCL signal on line 97 is applied as the enable input to gate 101, the
information input to gate 101 being output line 103 from frame memory 82.
The output from gate 101 is a CBW line 32B.
Referring to FIG. 3, in the color circuit 58 the RMY line 30B and the BMY
line 30C are each applied through a corresponding buffer amplifier 90 and
A to D convertor 92 to separate inputs of 3:1 multiplexing circuit 94. The
third input to multiplexing circuit 94 is system color line 14B.
The output from multiplexer 94 is applied as one address input to mapping
look up table RAM 96. The other input to RAM 96 is output line 98 from
programmable logic array 100. As with programmable logic array 70 (FIG.
2), array 100 may receive an input over a line 102 from one of the input
sources 12 or 20 indicating the type of data being applied to the color
board circuit or the input to PAL 100 may be obtained over line 104 from
processor interface 106. RAM 96 permits the color inputs to be remapped in
much the same way that RAM 66 functions to perform this operation for
black and white inputs. Thus, depending on the type of data received or
other selected criteria, the mapping of the color input information from
multiplexer 94 may be selectively altered to achieve a desired color
effect for any of the stored images. Note however that RAM 96 cannot map
the systems originated inputs and the VCR originated inputs to be
identical since these inputs are presented and stored in completely
different formats.
A frame memory 110 is provided which stores frame images in substantially
the same manner as frame memory 82. The frame image information to be
stored in memory 110 is received over line 112 from either the output of
RAM 96 or over a data line 113 from processor interface 106. Input
addressing is controlled by input address circuit 86 (FIG. 2) over lines
85 and output addressing is controlled by output address circuit 88 over
lines 87. It should be noted that since a single image frame may contain
both black and white and color data, for example tissue data and color
flow data in ultrasonic scan images, frame images may be read out
simultaneously from both frame memory 82 and frame memory 110. For VCR
data, luminance data is always stored in memory 82 and is used with image
data stored in memory 110 to obtain images for black and white and for
color. Information concerning a given image frame may also be stored
simultaneously in both memories.
Since the green-minus-luminance (GMY) signal for a given color display is
defined by the following equation:
GMY=.alpha.BMY+RMY+K (1)
(where: .alpha., and K are constants) the storage required to store a color
image frame is reduced by storing only the BMY and RMY signals and
deriving the GMY signal therefrom in accordance with Equation (1) above.
This derivation is obtained by utilizing the BMY and RMY outputs from
frame memory 110 as address inputs to a GMY look-up table ROM 120. The GMY
output from ROM 120 is applied as one input to adder circuit 122. The BMY
and RMY outputs from frame memory 110 are also applied as one input to
adders 124 and 126 respectively. The other input to each of adders 122,
124 and 126 is the Y value on line 103 from black and white circuit 56
(FIG. 2). The | | |
