 |
Description  |
|
|
BACKGROUND OF THE INVENTION
The present invention relates to a video data display control technique.
More particularly, the invention relates to a video data display
controlling method and a video data display processing system, suitable
for systems such as a video data monitoring system using a computer, for
displaying a plurality of images on a display screen by changing the
display quality of each image or for displaying one image by changing the
display quality of an image in a desired partial area.
It is common to display video data supplied from a camera or a VTR on a
display screen of a computer system. There are two main systems for
displaying video data on a display screen of a computer system. According
to one system, supplied video signals are made compatible with computer
video signals in an analog level. According to the other system, video
signals are sampled and converted into digital video signals and displayed
on a display screen of a computer system.
In a communications network interconnecting a plurality of remote computers
and transferring data therebetween, it is convenient that video signals
can be transferred over the communications network in the same manner as
general digital data. The method of converting video signals into digital
video signals is therefore becoming a main trend as compared to the method
of synthesizing video signals in an analog level.
Real time and data amounts are important considerations when video data is
processed in the form of digital data. In a general television, one frame
has 525 scan lines in the vertical direction, and a frame rate of 30
frames per second are used. An image in one frame is sampled at
525.times.525 points in the vertical and horizontal directions so that if
one pixel is converted into digital data of 8 bits, the image in one frame
has a capacity of 525.times.525=275625 bytes (about 300 Kbyte). Therefore,
at a frame rate of 30 frames per second, a data capacity of 300
Kbyte.times.30 frames=9 MB per second is required. Each image in one frame
is required to be displayed during one frame period. If one computer
processes digital data of a plurality of images, the total data capacity
increases further. As a result, the load of display processing on a
computer becomes large, and the transmission load on a communications
network becomes large if digital video data is transferred via the network
to a video data display unit.
As proposed in JP-A-3-205999, the load of display processing on a computer
and a transmission load on a communications path, i.e., a system load, has
been reduced conventionally in the following manner. In displaying a
plurality of images of subjects to be monitored on a single monitor
screen, the frequency of displaying each image is changed in accordance
with the running condition of the subject, the importance factor of the
subject, or the danger degree of the subject, to thereby reduce the amount
of data to be processed and transferred and provide efficient monitoring.
The above-described conventional technique does not consider the degree of
display quality of each of a plurality of images displayed on a display
screen of a computer. The frequency of displaying video data is changed
depending upon only the importance factor of a subject, so that there
arises a problem that an image a user really desires to observe cannot be
displayed finely.
SUMMARY OF THE INVENTION
The present invention provides a digital video data display controlling
method and a digital video data display processing system, capable of
displaying video data requested by a user efficiently and with a different
display quality and reducing a system load.
According to one aspect of the present invention, a video data display
controlling method is provided in which basically a user interest degree
of an image in each image area or in each partial image area of each image
area to be displayed on a display screen is estimated, an image in each
image area or an image in each partial image area having a high interest
degree is displayed with a high display quality whereas an image in each
image area or an image in each partial image area having a low interest
degree is displayed with a low display quality. Specifically, in this
method, the display quality priority order of images in image areas (or in
partial image areas) is determined in accordance with the display state of
each image in the image area (or in the partial image area), a preset
importance factor, and the like. In accordance with this display quality
priority order, the display specification or attribute of each image in
the image area (or in the partial image area) is changed. As a result,
according to another aspect of the present invention, the amount of
digital video data is totally reduced so as to match the system load.
According to another aspect of the present invention, there is provided a
system for realizing a video data display controlling method that
includes:
a video data input device;
a video data display; and
a signal transmission path interconnecting the video data input device and
the video data display means,
The video data input device includes:
a video data sampler for sampling inputted analog video data at a
predetermined sampling timing and converting the inputted analog video
data into digital video data;
a display specification management table for storing a display
specification of each image transferred from the video data display;
a video data compressor for compressing the digital video data in
accordance with the display specification; and
a video data input controller for controlling the video data sampler and
the video data compressor; and
The video data display includes:
a display for displaying each image;
a display driver for driving the display;
a video data display controller for controlling the display driver means;
a display state management table for storing data representing at least the
display state of each image displayed on the display; and
priority order determing for determining a display quality priority order
of images in accordance with the data stored in the display state
management.
The video data display controller controls the display driver means to
receive from the video data input device the digital video data of a
plurality of images corresponding to a display request command and to
display a plurality of images on the display, and operates to store the
display state of each image displayed on the display in the display state
management table, to change the display specification of each image in
accordance with the priority order determined by the priority order
determiner, and to transfer the changed display specification to the
display specification management table of the video data input unit.
The teaching of the above-described constituent elements of this invention
relies upon the following recognition by the inventor. Generally, a user
does not monitor all of a plurality of images in image areas displayed on
a display screen at the same interest degree. For example, if images are
displayed in an overlap manner, a user pays attention to the front image,
and other background images are observed by the user only supplementarily.
An image frequently displayed has a high interest degree or importance
factor, whereas an image less frequently displayed has a low interest
degree and can be assumed to be a supplementary image. An image with a
high interest degree is displayed with a good display quality having a
high resolution, whereas a supplementary image with a low interest degree
is not necessary to be displayed always with a good display quality.
From the above recognition, according to the present invention, the display
specification of each image is changed in accordance with the display
quality priority order of images determined from a difference between
image display states. It is possible therefore to reduce the transfer
amount of digital video data and the amount of display data to be
processed.
The transfer capacity of a communications path such as a communications
network and the display processing performance of a computer can be
efficiently used if the display specification defining the frame rate, the
number of pixels in each image area, and the data depth (number of bits)
per pixel, respectively of each image area is changed so as to match the
transfer capacity and the display processing performance.
The priority order of images can be determined by correlating it to the
display area of each image, to the overlapped display area of each image,
to a total display time of each image, to an importance factor preset to
each image, and to a combination of these parameters.
The display specification may be automatically changed or may be changed by
a user by inputting a change instruction.
Other objects and advantages of the invention will become apparent from the
following detailed description when read in conjunction with the
accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a schematic block diagram showing the overall structure of a
video data processing system according to a first embodiment of the
present invention.
FIG. 2 is a block diagram showing the detailed structure of the video data
input unit of the first embodiment.
FIG. 3 is a diagram showing an example of the display specification
management table.
FIG. 4A is a diagram showing an example of a plurality of image areas
opened on the display screen.
FIG. 4B is a diagram showing another example of a plurality of image areas
opened on the display screen.
FIG. 5 is a diagram showing an example of the display state management
table.
FIG. 6 is a schematic diagram explaining how a user can change a display
specification.
FIG. 7 is a schematic block diagram showing the overall structure of a
video data processing system according to a second embodiment of the
present invention.
FIG. 8 is a schematic block diagram showing the overall structure of a
video data processing system according to a third embodiment of the
present invention.
FIG. 9 is a diagram explaining a video data processing method and system
according to a fourth embodiment of the present invention.
DETAILED DESCRIPTION
A first embodiment of the present invention will be described with
reference to FIGS. 1 to 6. As shown in FIG. 1, a video data processing
system has three main units including a video data input unit 100 for
inputting video data to the system, a video data display unit 200 for
displaying video data of text, graphics, or both, and a communications
network 300 for interconnecting the video data input unit 100 and video
data display unit 200 and transferring various data including video data
and control data.
The video data input unit 100 includes a camera 101 for taking an image and
inputting analog video data representing the image, a video data sampling
unit 102, a video data compressing unit 103, a transceiver unit 104, a
video data input control unit 105, a display specification management
table 106, and a bus 107. The video data sampling unit 102 converts analog
digital data inputted from the camera 101 into digital video data. The
video data compressing unit 103 compresses digital video data and reduces
its capacity. The transceiver unit 104 transmits various data including
compressed digital video data to the video data display unit 200 via the
communications network 300 and receives data from the video data display
unit 200 via the communications network 300. The video data input control
unit 105 controls the operation of the video data input unit 100. The
display specification management table stores attribute data of each image
area (each picture area or each window opened to the display screen).
Video data of an image in each image area is transmitted to the video data
display unit 200. The attribute data includes an image area position, an
image area size, the number of frames per unit time, the number of pixels,
a pixel data length, and the like. The bus 107 interconnects these units
and table of the video data input unit 100.
The video data display unit 200 includes a transceiver unit 201, a video
data expanding unit 202, a display driver unit 204 for driving a display
203 and displaying video data on the screen, a video data display control
unit 205, a display state management table 206, an interest degree
determining unit 207, and a bus 208. The transceiver unit 201 receives
data from the video data input unit 100 via the communications network 300
and transmits data to the data input unit 100 via the communications
network 300. The video data expanding unit 202 expands compressed digital
video data inputted from the video data input unit. The video data display
control unit 205 controls the display driver unit 204 in accordance with
information of display images designated by an operator display
instruction and information of a command representing a display state,
thereby displaying the designated images on the display at the requested
display state. The display state management table 206 stores display state
data and preset image area importance factors of image areas displayed on
the display 203. The display state data includes the size of each image
area, the order of overlapping image areas, and the like. The interest
degree determining unit 207 determines a user interest degree of each
image area in accordance with the display state data and preset image area
superiority rank data. The bus 208 interconnects these units and table of
the video data display unit 200. The user interest degree signifies the
order of image areas in which order a user is assumed to see the images
displayed on the screen in respective image areas. The quality of each
displayed image is controlled in accordance with this order. Therefore,
the user interest degree corresponds to the priority order of qualities of
images displayed in image areas. The video data display control unit 205
manages the display state of each image area, stores preset display state
data to be described later in the display state management table 206,
determines the display specification of each image in accordance with the
interest degree determined by the interest degree determining unit 207,
and transfers the display specification to the video input unit 100 to
change the contents of the display specification management table 106.
In the video data processing system shown in FIG. 1, a single set of the
video data input unit 100 and the video data display unit 200 is shown.
However, this system may have a plurality of video data input units 100
and video data display units 200. In this case, a single image may be
displayed on a plurality of video data display units 200 or images from a
plurality of video data input units 100 may be displayed at one video data
display unit 200.
The details of the structure of each unit and table will be explained.
FIG. 2 shows the detailed structure of the video data input unit 100. The
video data input unit 100 is provided with a plurality of cameras 101 (in
this example, two cameras). Each camera 101 has a photosensor 111 for
converting a light intensity of a subject into electric signals. For the
simplicity of description, the effective number of pixels of each
photosensor is assumed to be about 500.times.500 pixels for a
monochromatic image. A video signal may be inputted directly to the video
data input unit, without using the camera 101.
Video data outputted from the camera 101 is analog signals. In order to
process these analog signals by a computer system, they are first
converted into digital signals by an A/D converter 112 of the video data
sampling unit 102. The number of bits used for the A/D conversion is
assumed to be 8 bits per pixel. The A/D conversion is performed
synchronously with timing signals outputted from a timing generator unit
115. The converted digital video data outputted from the A/D converter 112
is stored in a video memory 113 synchronously with timing signals
outputted from the timing generator unit 115.
The timing generator unit 115 generates timing signals for synchronizing
the operation of the A/D converter 112 which converts analog video data
from the photosensor 111 into digital video data, and also generates
timing signals for synchronizing the operation of storing converted
digital video data in the video memory 113. In this embodiment, these
timing signals are generated so as to sample analog digital data and
convert it into digital video data having 8 bits per pixel, and so as to
satisfy the conditions that the maximum number of frames per second is 30
frames and the number of pixels of each photosensor is 500.times.500
pixels.
Each image converted into digital data has a large amount of data. From
this reason, in this embodiment digital video data is compressed by the
video data compressing unit 103 before it is transmitted to the
communications network 300. As a digital video data compressing method,
there is known a moving picture compressing method by MPEG (Moving Picture
Coding Experts Group). By using this method, moving pictures or images are
compressed with respect to their line density and pixel density, and still
images having no difference between two consecutive frames are compressed.
A compression factor obtainable by a method by MPEG is generally 10 to
100. Therefore, compressed digital video data is several megabits (Mb) at
a maximum. The transceiver unit 104 transmits compressed digital data to a
designated one of video data display units 200.
In accordance with the display specifications set to the display
specification management table 106, the video data input control unit 105
instructs the video compressing unit 103 which digital video data is to be
compressed at what timing, and instructs the transceiver 104 to transmit
the compressed digital video data to a predetermined video display unit
200.
The display specification management table 106 stores data such as shown in
FIG. 3. Specifically, the display specification management table 106
stores the name of each video data display unit 200 to which an image is
transmitted, the serial number of each input apparatus for inputting video
data of an image, and a display specification of each image to be
transmitted to a video data display unit 200. The example shown in FIG. 3
indicates that an image taken by a camera 101 identified by "Input 1" is
transmitted to a video data display unit 200 identified by "Display 1".
The example of a display specification (attribute data) of an image area
to be transmitted shown in FIG. 3 indicates that:
the display origin of input video data to be transmitted and displayed at
the video data display unit 200, corresponding to the lower left corner of
the image area of the input video data to be transmitted and displayed, is
(0, 0) in the horizontal and vertical directions;
the image area size corresponds to (300, 300) dots of the input video data
to be transmitted and displayed at the video data display unit 200, in the
horizontal and vertical directions;
the number of frames per second of an image to be displayed at the video
data display unit 200 is 10 frames; and
the number of pixels of an image to be displayed at the video data display
unit is (150, 150) in the horizontal and vertical directions, i.e., a
resolution is 1/2.
The contents of the display specification management table 106 are set in
accordance with display specification setting data sent from the video
data display unit 200, and are supervised by the video data input control
unit 105 independently for each video data display unit 200. That is to
say, the display specification setting data is generated by the video data
display control unit 205 of the video data display unit 200 as will be
later detailed, and transmitted to the transceiver 104 of the video data
input unit 100 in the form of a display specification setting command.
In accordance with a display specification setting command received by the
transceiver 104, the video data input control unit 105 sets a display
specification of each image area to the display specification management
table 106.
The format of a display specification setting command is as follows:
"begin": a start of a command train;
"setDisplay": setting a name of a video data display unit 200;
"setInput": setting a serial number of an input apparatus of the video data
input unit 100;
"setOrigin": setting a display origin of input video data to be transmitted
and displayed at the video data display unit 200;
"setArea": setting an image area size of the input video data to be
transmitted and displayed at the video data display unit 200;
"setSamplingTime": setting the number of frames per second of an image to
be displayed at the video data display unit 200;
"setResolution": setting the number of pixels of an image to be displayed
at the video data display unit 200;
"setBit": setting a data length (the number of bits) of a pixel; and
"end": an end of a command train.
In this embodiment, a network like Ethernet or a network complying with
IEEE 802.3 may be used as the communications network 300. Ethernet has a
maximum data transfer rate of 10 Mb/sec. As described previously, one
image of compressed digital video data to be transmitted from the video
data input unit 100 has a capacity of several Mb/sec. Therefore, if one
Ethernet cable is used, only several images can be transmitted in one
second. As a result, the number of images to be transmitted in one second
is limited, or the transfer state of each image is required to be
controlled.
A plurality of images displayed on one display 203 may be overlapped or may
each have a different display image area without any overlap. It can be
hypothesized that a user does not look at a plurality of overlapped images
uniformly but rather looks the front image. It can also be hypothesized
that a user pays much attention to an image having a larger display image
area and pays less attention to an image having a smaller display image
area.
The present invention has relied upon the fact that a user interest degree
of each image can be estimated from an image display state. In this
embodiment, if the transfer capacity of the communications network 300 is
lower than a transfer capacity required for inputting, transmitting, and
displaying a plurality of images, the display quality of an image, i.e,
the display specification of an image, is changed in accordance with a
user interest degree. An image with a high user interest degree is
displayed finely, whereas an image with a low user interest degree is
displayed roughly by degrading a resolution, the number of frames per
second, or other parameters. The transfer of digital video data is
controlled in accordance with a changed display specification, thereby
reducing the transfer load of the communications network 300.
The detailed structure of the main part of the video data display unit 200
will be described, the video data display unit 200 controlling to change
the display specification of each image area in accordance with a user
interest degree. Display state data of image areas (windows) Win-1 to
Win-3 shown in FIG. 4 is stored in the display state management table 206
as shown in FIG. 5. The contents of the display state management table 206
are updated by the video data display control unit 205 each time the
display states of the image areas Win 1 to Win-3 opened to a display
screen 203a are changed. As illustratively shown in FIG. 5, the display
state management table 206 stores a display image area (corresponding to
the number of dots on the screen), the order of overlapped images (1, 2, 3
from the front image area), and an image area system importance factor
(%), respectively for each image area. The importance factor changes with
the running conditions of a system, and is stored in advance in the
system. For example, in the case of a plant monitoring system, a
combination of image areas to be displayed and monitored on a display
screen, as well as how each image area is displayed, is preset in advance
for each running stage of the system. The user interest degree is
therefore determined by considering both the image area importance or
materiality preset in the system and the current display state.
The interest degree determining unit 207 evaluates a user interest degree,
i.e., the priority order of displaying images at a high display quality,
by using the display state data and importance factor set in the display
state management table 206. In this embodiment, a user interest degree
evaluating method uses the following evaluation equation by way of
example.
User interest degree=display image
area.times.(100/overlapping-order).times.importance factor
The user interest degree is proportional to the display image area and
importance factor, and inversely proportional to the overlapping order.
The user interest degree of each window having the display state shown in
FIG. 4 can be expressed therefore as:
##EQU1##
The video data display control unit 205 determines the display
specification of each image based upon the interest degree (priority
order) determined by the interest degree determining unit 207. While
taking into consideration that the transfer capacity of the communications
network 300 of the video data processing system of this embodiment is 10
Mb/sec, a usable transfer capacity of the communications network 300 by
each video data display unit 200 is first determined. This usable transfer
capacity can be changed when the total running conditions of the system
change.
It is assumed here that the usable transfer capacity allocated to each
video data display control unit 200 is 2 Mb/sec. The video data display
control unit 205 distributes this 2 Mb/sec usable transfer capacity to
respective image areas in proportion to the user interests. First, the
(total) interest degree of all images displayed at the same time is first
calculated. In the example shown in FIG. 5, the total interest degree is
8167. The image area Win-1 is assigned a usable transfer capacity in
proportion to a ratio of its interest degree to the total interest degree,
i.e., a usable transfer capacity of 2M.times.7000/8167=1.7 Mb/sec is
assigned. The usable transfer capacity N can be expressed by:
N=r.times.r.times.t.times.p/c
where c is a compression factor of video data of an image in the image area
Win-1 to be sent from the video data input unit 100, r.times.r represents
the number of pixels of an image to be displayed, t represent the number
of frames per second, and p represents the number of bits per pixel.
In this embodiment, a compression method by MPEG is used which has a
compression ratio of 10 to 100. It is assumed that a compression ratio c
is 20. The video data display control unit 205 is required to calculate
the parameters of the display specification including the pixel number
r.times.r, frame number t, and bit number p of one pixel data length, by
using the assigned usable transfer capacity N. However, the above equation
has many unknown parameters. In view of this, it is preferable to assume
the values of several parameters and change a parameter most effective in
reducing the data transfer amount in accordance with the usable transfer
capacity N. Therefore, for example, the bit number of one pixel data
length is assumed to p=8 and the frame number t is defined to have a value
of 30/overlapping-order.
With such assumption, the number r of pixels | | |
