 |
Description  |
|
|
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to an image processing apparatus, into which
image data is input and in which the input data is processed to produce a
high resolution image.
2. Description of the Related Art
The following methods are conventionally known to input a high resolution
image or a panorama image:
(1) Method to use a high resolution image pickup device:
A high resolution image may be input by using a high resolution image
pickup device as input means of an image processing apparatus.
(2) Method for a line sensor to scan:
This method is a technique in popular use in image scanners, facsimile
devices, copiers, or the like mainly as a contact type image scanner, in
which each reading operation of a linear CCD sensor obtains a linear image
in a horizontal scanning direction and in which the vertical scanning is
carried out in a mechanical and optical manner thereby to input image
information.
The below Reference 1 describes a trial to obtain a panorama image by the
vertical scanning of a noncontact line image sensor in a mechanical and
optical manner.
(Ref. 1) Tsuji (Osaka Univ., Faculty of Fund. Engi.) "Panoramic expression
of environment", J. of Institute of Electronics, Information and
Communication Engineers of Japan (IECEJ), Vol. 74, No. 4, pp 354-359,
(1991).
(3) Method using a plurality of image pickup devices
This method is described in the following Reference 2 to Reference 5, in
which image information is obtained from a plurality of image pickup
devices the relative positional relations of which are known and in which
the obtained image information pieces are joined with one another to
obtain a high resolution image.
(Ref. 2) Aizawa (Tokyo Univ., Fac. Engi.), Saito and Komatsu (Kanagawa
Univ., Fac. Engi.), "Fundamental studies for obtaining ultra high
definition image-high definition image acquisition through stereo image
processing", IE 90-54 (1990).
(Ref. 3) Uehira (NTT HI Lab.) "High speed still image input method by image
pickup region combining method", Briefs 1991 Spring Nat. Conf., IECEJ, pp
7-103.
(Ref. 4) Uehira and Matsuki (NTT) "High speed high definition document
reading technique by image pickup region combining method", Briefs 18 1989
Nat. Conf., Inst. Img Electro. Jap. (IIEJ), pp 75-78.
(Ref. 5) Uehira (NTT) "Optical connection type still image camera", J.
IIEJ., Vol. 20, No. 3, pp 203-208, (1991).
(4) Method to put images together:
This method is described in the following Reference 6 to Reference 8, in
which a plurality of images input in time series are joined together to
input a panorama image or a wide range image.
(Ref. 6) Okada, Ohta, and Sakai (Kyoto Univ., Fac. Engi.) "Hand scan type
document image input apparatus having real time image joining function",
IE 81-17 (1981).
(Ref. 7) Yoshizawa, Hanamura, and Tominaga (Waseda Univ.) "Composition of
background image considering panning of dynamic image", Briefs 1990 Spring
Nat. Conf., IECEJ, pp 7-51.
(Ref. 8) Nakamura and Kaneko (Tokyo Rika Univ.), and Hayashi (NHK
Broadcasting Tech. Lab.) "Production method of panorama image by division
image pickup" Briefs 1991 Spring Nat. Conf., IECEJ, pp 7-165.
(5) Method to vibrate an image pickup device:
This method is described in the following Reference 9, in which a CCD image
pickup device is vibrated to obtain a doubled resolution image in the
vibration direction.
(Ref. 9) Yoshida, Endo, and Harada (Toshiba Inc.) "High resolution process
of CCD by swing image pickup", IE 83-65 (1983).
There are, however, the following problems present in the above image input
methods.
(a) In the "method to use a high resolution image pickup device" as
described in (1) in the above conventional techniques, the number of
pixels in an input image depends upon the image pickup device, and,
therefore, it is impossible to input an image with the number of pixels
more than the capacity of the image pickup device.
(b) In the "method for a line sensor to scan" as described in (2) in the
above conventional techniques, the number of pixels in the horizontal
scanning direction is limited to that of the line sensor in the same
reason as in above (a), and the image alignment cannot be fine without
precise mechanical and optical scan in the vertical scanning direction so
as to fail to input a fine image.
(c) In the "method to use a plurality of image pickup devices" as described
in (3) in the above conventional techniques, the number of pixels is
limited to "(pixel number).times.(number of image pickup devices)" in each
image pickup device, and the use of the plural image pickup devices
inevitably increases the production cost and the scale of image pickup
system.
(d) In the "method to join images" as described in (4) in the above
conventional techniques, the following problems exist.
In the method as shown in Reference 6, the object image is limited to a
binary image, there is a restriction on a direction of scanning performed
by an operator, and it is readily influenced by the camera vibration in a
direction perpendicular to the scanning direction.
In the method as shown in Reference 7, there is a defect of resolution
reduction if two frame images are relatively rotated, magnified, or
contracted, because the position alignment is carried out between two
frame images only by parallel displacement components between the frames.
In the method as shown in Reference 8, there is a restriction on a
positional relation of plural frames, that is, on the displacement amount,
the angle, and the like of panning and tilting of camera, and the method
is not applicable to parallel displacement of camera, though it is
possible to obtain a panorama image as seen from a point.
(e) In the "method to vibrate the image pickup device" as described in (5)
in the above conventional techniques, the vibrations are allowed only in
one direction, the resolution is at most a double of pixel number of the
image pickup device, and a special image pickup device is necessary for
accurately synchronizing the amplitude of vibrations with taking-in of
image pickup signals.
SUMMARY OF THE INVENTION
The present invention has been accomplished to solve the above various
problems. It is an object of the present invention to provide an image
processing apparatus which can produce a high definition and wide range
image from a plurality of images each having the relatively small number
of pixels, and to provide an image processing apparatus which uses a
single common image pickup device without specific and strict restrictions
on the scanning of image pickup device in input, which can input a
multi-valued image exceeding the resolution of the image pickup device in
a noncontact manner, which can execute photography of panorama
multi-valued image, and which can produce to output a high resolution
image even with low resolution images being input.
The above object of the present invention can be achieved by an image
processing apparatus comprising:
image storing means for storing an image;
image inputting means for inputting an image;
image combining means for conducting a combination process of the image
stored in the image storing means and the image input through the image
inputting means; and
image comparing means for comparing the image stored in the image storing
means with the image input through the image inputting means;
wherein the image combining means conducts the image combination, based
upon a difference between the two images obtained by the image comparing
means.
The image comparing means compares the two images when the image combining
means combines the image input through the image inputting means and the
image stored in the image memory. The combination process is carried out
with parameters obtained from the difference between the images obtained
upon the comparison.
Further objects and advantages of the present invention will be apparent
from the following description, reference being had to the accompanying
drawings wherein preferred embodiments of the present invention are
clearly shown.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a constitutional drawing of the first embodiment;
FIG. 2 is a constitutional drawing of an image producing apparatus
according to the present invention;
FIG. 3 is a constitutional drawing of an image combining circuit in the
first embodiment;
FIG. 4 is a flowchart to show a process of a coordinate generating portion;
FIG. 5 is a constitutional drawing of an input image correcting circuit in
the first embodiment;
FIG. 6 is a constitutional drawing of a main image correcting circuit in
the first embodiment;
FIG. 7 is a constitutional drawing of a parameter determining circuit in
the first embodiment;
FIG. 8 is a drawing to illustrate a relation between a main image and an
input image in the first embodiment;
FIG. 9a and 9b are drawings to show an example of representative points in
the parameter determining circuit;
FIG. 10a and 10b are drawings to illustrate a selection method of the
representative points;
FIG. 11 is a drawing to illustrate block matching;
FIG. 12 is a drawing to illustrate a relation among synchronizing signals;
FIG. 13 is a drawing to illustrate a process for each pixel;
FIG. 14 is a drawing to exemplify a process result in the first embodiment;
FIG. 15 is a constitutional drawing of the second embodiment;
FIG. 16 is a constitutional drawing of an image enlarging circuit in the
second embodiment;
FIG. 17 is a constitutional drawing of a main image correcting circuit in
the second embodiment;
FIG. 18 is a constitutional drawing of a parameter determining circuit in
the second embodiment;
FIG. 19 is a drawing to illustrate a process for each pixel in the second
embodiment;
FIG. 20 is a drawing to schematically illustrate a process in the second
embodiment;
FIG. 21 is a constitutional drawing of the third embodiment;
FIG. 22 is a constitutional drawing of an enlarged image correcting circuit
in the third embodiment;
FIG. 23 is a drawing to schematically illustrate a process in the third
embodiment;
FIG. 24 is a constitutional drawing of the fourth embodiment;
FIG. 25 is a constitutional drawing of the fourth embodiment;
FIG. 26 is a constitutional drawing of an image contracting circuit in the
fourth embodiment;
FIG. 27 is a constitutional drawing of the fifth embodiment;
FIG. 28a and 28b are constitutional drawings of image inputting means in
the sixth embodiment;
FIG. 29 is a constitutional drawing of the seventh embodiment; and
FIG. 30 is a drawing to show an example of representative points.
DESCRIPTION OF THE PREFERRED EMBODIMENTS
First Embodiment
The following describes an embodiment of an image processing apparatus
according to the present invention, which enables production of a high
resolution and wide range image, referring to the accompanying drawings.
The hardware of the image processing apparatus in the present embodiment
comprises, as shown in FIG. 2, image inputting means 11, input processing
means 12, image storing means 13, output processing means 14, and image
outputting means 15.
The image inputting means 11 is a device to carry out the photoelectric
conversion of an image into digital data, which for example comprises a
CCD camera, an A/D converter, and peripheral circuits. The input
processing means 12 is a device to process the image data converted by the
image inputting means into digital signal and image data in the image
storing means 13, which for example comprises a DSP, an LSI, a
semiconductor memory, and peripheral circuits.
The image storing means 13 is means to store the image data output from the
input processing means 12, which for example comprises a semiconductor
memory and peripheral circuits. The output processing means 14 is a device
to convert the image data in the image storing means 18 into a form
capable of being output to the outputting means 15, which for example
comprises an IC or LSI, a semiconductor memory and peripheral circuits.
The outputting means 15 is a device to display or to print out the image
data converted by the output processing means into the appropriate form,
which for example comprises a printing apparatus or a display device.
FIG. 1 is a drawing to show an example of arrangement of the image
inputting means 11, the input processing means 12, and the image storing
means 18 in the present embodiment.
The present embodiment shows an example of an apparatus in which a
plurality of frame images are continuously picked up while an operator is
carrying out panning of camera carried by its hands, whereby producing a
panorama image in a range in which the images are picked up.
In FIG. 1, synchronizing means 20 comprises a digital circuit, which
outputs synchronizing pulse signals to various means for synchronizing the
processes of the image inputting means 11, the input processing means 12,
and the image storing means 13. There are three kinds of synchronizing
pulse signals generated by the synchronizing means 20. In detail, as shown
in FIG. 12, the signals are as follows:
Reset signal: A binary signal to reset the system, which holds an active
state before the first frame switch signal, which first appears after the
reset signal became active, turns into an non-active state.
Frame switch signal: A binary signal which becomes active for each frame to
be processed and holds the active state while the process for the frame is
carrying out.
Pixel switch signal: A binary pulse signal generated for each process of a
pixel in the main memory.
Functions of these signals in FIG. 12 will be detailed later.
The image inputting means 11 of FIG. 1 comprises a CCD image pickup portion
21 and an A/D converting circuit 22. The CCD image pickup portion 21
comprises a CCD element and peripheral circuits, which executes the
photoelectric conversion of an image in synchronism with the frame switch
signal in the synchronizing signals input from the synchronizing means 20
to output an electric signal to the A/D converting circuit 22.
The A/D converting circuit 22 comprises an A/D converter and peripheral
circuits, which executes the A/D conversion of the electric signal
supplied from the CCD image pickup portion 21 in synchronism with the
signal input from the synchronizing means 20, that is, in a period between
a timing when the frame switch signal turns into non-active state or when
the reset signal turns into an active state and a timing when the frame
switch signal again turns into an active state, to output digital image
data for one screen to an input image memory 23. This image data for one
screen will be hereinafter referred to as a frame.
The input processing means 12 comprises an input image memory 23, an image
combining circuit 24, an input image correcting circuit 25, a main image
correcting circuit 26, and a parameter determining circuit 27.
The input image memory 23 comprises a semiconductor memory and peripheral
circuits, which stores the digital image data input from the A/D
converting circuit before next data is input and which outputs respective
pixel values corresponding to coordinates input from the parameter
determining circuit 27 and the input image correcting circuit 25, to the
respective circuits. In case that a coordinate at which no pixel is
present in the memory is input, 0 is output.
The number of pixels in the input image memory is determined by the number
of pixels in one frame in the image inputting means, which is smaller than
that in the main image memory.
In the present embodiment, the input image memory 23 has pixels of XI in
the horizontal direction and YI in the vertical direction, each of the
pixels storing lightness information of one of GI gradations. Let a pixel
value be I (i, j) at a coordinate (i, j) in the input image memory, where
0.ltoreq.i<XI, 0.ltoreq.j<YI, 0.ltoreq.I(i, j)<GI, and i, j, and I (i, j)
are integers.
The image combining circuit 24 comprises a digital calculating circuit,
which outputs a coordinate value to the input image correcting circuit 25
and to the main image correcting circuit 26 and which calculates pixel
data obtained as a result of the coordinate output from the respective
circuits to output a value of calculation result together with the
coordinate value to the main image memory 28. The details of the operation
of the image combining circuit 24 will be described later.
The input image correcting circuit 25 comprises a digital calculating
circuit, a semiconductor memory, and peripheral circuits, which effects
the transformation correction on the digital image data stored in the
input image memory 23 with parameters input from the parameter determining
circuit 27 and which outputs to the image combining circuit 24 pixel data
corresponding to the coordinate value input from the image combining
circuit 24. This operation of the input image correcting circuit 25 will
be detailed later.
In the present embodiment, the correction process in the input image
correcting circuit 25 is described by a function F.sub.i for brevity of
description, so that the corrected pixel value at coordinate (k, l) may be
expressed as F.sub.i (k, l). In the expression, 0.ltoreq.k<XM,
0.ltoreq.l<YM, 0.ltoreq.F.sub.i (k, l)<GM, and k, l, and F.sub.i (k, l)
are integers.
The main image correcting circuit 26 comprises a digital calculating
circuit, a semiconductor memory, and peripheral circuits, which executes
the coordinate transformation of the image data in the main image memory
28 with the parameters input from the parameter determining circuit 27 and
which outputs to the image combining circuit 24 the pixel data
corresponding to the coordinate value input from the image combining
circuit 24. This operation of the main image correcting circuit 26 will be
detailed later.
In the present embodiment, the correction process in the main image
correcting circuit 28 is expressed by a function F.sub.m for brevity of
description, so that the corrected pixel value at a coordinate (k, l) may
be expressed by F.sub.m (k, l). In the expression, 0.ltoreq.k<XM,
0.ltoreq.l<YM, 0.ltoreq.F.sub.m (k, l)<GM, and k, l and F.sub.m (k, l) are
integers.
The parameter determining circuit 27 comprises a digital calculating
circuit, a semiconductor memory, and peripheral circuits, which compares
image data values respectively stored in the input image memory 23 and in
the main image memory 28 with each other and which outputs the correction
parameters to the input image correcting circuit 25 and to the main image
correcting circuit 26. The details of this operation of the parameter
determining circuit 27 will be described later.
The image storing means 13 comprises a main image memory 28. The main image
memory 28 comprises a semiconductor memory and peripheral circuits, into
which a coordinate and a pixel value are input from the image combining
circuit 24 and in which the pixel value is written in a memory region
corresponding to the input coordinate. With a coordinate input from the
parameter determining circuit 27 and the image correcting circuit 26, the
main image memory 28 outputs a pixel value corresponding to the input
coordinate to the respective circuits. In case that a coordinate at which
no pixel exists in the memory is input, 0 is output.
The main image memory 28 clears all pixel values into zero after the reset
signal from the synchronizing means 20 turns into an active state and
before the frame switch signal turns into an active state.
The number of pixels in the main image memory 28 is greater than that in
the input image memory. In the present embodiment, the main image memory
28 has pixels of XM.times.YM, each pixel storing lightness information of
one of GM gradations. Let a pixel value at a coordinate (m, n) in the main
image memory be M(m, n), where 0.ltoreq.m<XM, 0.ltoreq.n<YM, 0.ltoreq.M(m,
n)<GM, and m, n, and M(m, n) are integers.
Details of the process in FIG. 1
There are below described in detail the operations of the image combining
circuit 24, the input image correcting circuit 25, the main image
correcting circuit 26, and the parameter determining circuit 27.
Description of the image combining circuit 24
The image combining circuit 24 comprises a coordinate generating portion 31
and a pixel calculating portion 32 as shown in FIG. 3, into which pixel
values corresponding to a coordinate value of the main image memory
produced by the coordinate generating portion 31 are input from the input
image correcting circuit 25 and from the main image correcting circuit 26
and which writes the calculation result in a region corresponding to the
above coordinate value in the main image memory.
The coordinate generating portion 31 generates all the coordinate values
present in the main image memory in the order of raster in synchronism
with the pixel switch pulse signal in the synchronizing signals 30 from
the synchronizing means 20. In detail, counters of X and Y directions
inside the portion 31 are given an increment by the synchronizing signals
as shown in the flowchart of FIG. 4.
At S41 a value y of the Y counter is reset to 0.
At S42 a value x of the X counter is reset to 0.
At S43 the process is suspended until the pixel switch pulse starts rising.
At S44 a set (x, y) of current values of the X, Y counters are output
through signal lines 33 and 34.
At S45 the value x of the X counter is given an increment of 1.
At S46 the value x of the X counter is compared with an X directional pixel
number XM of the main image memory. If x.gtoreq.XM, the flow proceeds to
S47, while if x<XM, the process starting from S43 is again carried out.
At S47 the value y of the Y counter is given an increment of 1.
At S48 the value y of the Y counter is compared with a Y directional pixel
number YM of the main image memory. If y.gtoreq.YM, the process from S41
is carried out, while if y<YM, the process from S42 is again carried out.
The pixel calculating portion 32 of FIG. 3 takes in pixel values through
respective signal lines 35 and 36 from the input image correcting circuit
25 and from the main image correcting circuit 26 in synchronism with the
synchronizing signals 30 from the synchronizing means 20, that is, at a
timing when the pixel switch signal is changed from active to non-active.
The pixel calculating portion 32 executes the calculation of below
Equation 1 with the pixel values and outputs the calculation result
through a signal line 37 to the main image memory 28 before the pixel
switch signal is again turned into active.
______________________________________
Equation 1
______________________________________
M=.alpha.xF.sub.i +(1-.alpha.)xF.sub.m,
if F.sub.i >0 and F.sub.m >0;
M=F.sub.m, if F.sub.i =0;
M=F.sub.i, if F.sub.m =0;
M=0, if F.sub.i =F.sub.m =0;
______________________________________
If M.gtoreq.GM then M=GM;
where .alpha. is a constant preliminarily determined as 0<.alpha.<1,
M is a pixel value output through the signal line 37,
F.sub.i is a pixel value input through the signal line 35, and
F.sub.m is a pixel value input through the signal line 36,
under the condition that while the reset signal is in active the following
calculation of Equation 2, that is, a process to write only the input data
from the input image correcting circuit 25 into the main image memory 28,
is carried out.
M=F.sub.i : Equation 2
If M.gtoreq.GM then M=GM.
Description of the input image correcting circuit 25
The input image correcting circuit 25 comprises an affine transformation
portion 51 and an interpolation calculating portion 52 as shown in FIG. 5.
The affine transformation portion 51 executes the following process.
(1) Affine transformation coefficients are input through a signal line 53
from the parameter determining circuit 27 after the frame switch signal of
the synchronizing signal 30 has risen and before the pixel switch signal
starts rising.
(2) Every time when the pixel switch signal rises, a coordinate value input
through the signal line 33 from the image combining circuit 24 is
transformed by the following Equation 3 and the result is output through a
signal line 56.
##EQU1##
where matrix elements p.sub.ij are affine transformation coefficients
input from the parameter determining circuit 27,
(x, y) is a coordinate value input through the signal line 33, and
(x', y') is a coordinate value output through the signal line 56.
The interpolation calculating portion 52 executes the following process.
(3) The coordinate value, which is the result of the process in above (2)
in the above affine transformation portion 51, is input through the signal
line 56 from the affine transformation portion 51.
(4) A pixel interpolation calculation process by the cubic interpolation
method as shown in below Equation 4 is made on the input coordinate value
while the pixel switch signal is in active, and the obtained result is
output through the signal line 35 to the image combining circuit 24.
Interpolation method 1: cubic interpolation method
##EQU2##
In the above equation, the summation range is between sixteen pixels as
defined below in the proximity of a coordinate (x', y') input through the
signal line 56.
x'-2.ltoreq.m<x'+2; y'-2.ltoreq.n<y'+2,
where m and n are integers.
In Equation 4, M designates a pixel value output through the signal line
35, and I (m, n) does a process to output a coordinate value (m, n)
through the signal line 54 to the input image memory 23 and to obtain a
pixel value corresponding to the coordinate value through the signal line
55 from the input image memory 23.
The pixel interpolation process may be done not only by the above cubic
interpolation method but also by the bilinear interpolation method or by
the nearest proximity method as described below.
Interpolation method 2: bilinear interpolation method
Equation 5
M=I(x", y").times.(1-.beta.).times.(1-.gamma.)+I(x"+1,
y").times..beta..times.(1-.gamma.)+I(x",
y"+1).times.(1-.beta.).times..gamma.+I(x"+1,
y"+1).times..beta..times..beta.
Interpolation method 3: nearest proximity method
______________________________________
Equation 6
______________________________________
M=M (x", y") if .beta..ltoreq.0.5 and .gamma. .ltoreq.0.5;
M=M (x"+1, y") if .beta.>0.5 and .gamma. .ltoreq.0.5;
M=M (x", y"+1) if .beta..ltoreq.0.5 and .gamma. >0.5;
M=M (x"+1, y"+1) if .beta.>0.5 and .gamma. >0.5.
______________________________________
The above pixel interpolation methods are described in the following
Reference 10.
(Ref. 10) "Handbook of image processing" published by Shokodo, pp 274-275.
Description of the main image correcting circuit 26
The main image correcting circuit 26 comprises an image buffer 61 and a
coordinate calculating portion 62 as shown in FIG. 6. The image buffer 61
is an image memory having the same capacity as the main image memory 28,
which copies the entire contents of the main image memory 28 during a
period between a timing when the frame switch signal in the synchronizing
signals 30 from the synchronizing means 20 turns into non-active and a
timing when the frame switch signal again turns into active. This copying
process is not carried out if the reset signal is in active.
The image buffer 61 outputs a pixel value corresponding to a coordinate
value input from the coordinate calculating portion 62, through the signal
line 36 to the image combining circuit 24.
The coordinate calculating portion 62 executes the following process.
(1) Coefficients (q.sub.x, q.sub.y) for coordinate transformation are input
through a signal line 63 from the parameter determining circuit 27 in a
period between a timing when the frame switch signal in the synchronizing
signals 30 from the synchronizing means 20 is turned into active and a
timing when the pixel switch signal is turned into active.
(2) A coordinate value input from the image combining circuit 24 is
transformed by the following Equation 7 when the pixel switch signal is
turned into active, and the transformed coordinate value is output to the
image buffer 61.
Equation 7
(x', y')=(x+q.sub.x, y+q.sub.y),
where coefficients q.sub.x and q.sub.y are integers input from the
parameter determining circuit 27, (x, y) is a coordinate value input
through the signal line 33 from the image combining circuit 24, and (x',
y') is a coordinate value output to the image buffer 61.
(3) After above, a pixel value corresponding to the coordinate (x', y') is
output from the image buffer 61 to the image combining circuit.
The above process of (2) and (3) is carried out while the pixel switch
signal is in active, but are not carried out while the reset signal is in
active.
In the above process, the main image correcting circuit 26 conducts the
parallel displacement of the image data in the main image memory by the
integer values (q.sub.x, q.sub.y) and carries out the output process of
the processed data to the image combining circuit 24.
Description of the parameter determining circuit 27
The parameter determining circuit 27 comprises a vector detecting portion
71, a vector buffer 76, a coefficient calculating portion 72, a parameter
dividing portion 73, and an origin buffer 75 as shown in FIG. 7, which
executes processes in synchronism with the synchronizing signals 30 from
the synchronizing means 20.
The origin buffer 75 stores a matching origin coordinate value (X.sub.0,
Y.sub.0) where X.sub.0 and Y.sub.0 are integers, which is for use in the
vector detecting portion 71, the coefficient calculating portion 72, and
the parameter dividing portion 73. The matching origin coordinate value is
rewritten at each process end of one screen by the parameter dividing
portion 73 in synchronism with the synchronizing signals 30 from the
synchronizing means 20.
An initial value is set as (X.sub.0, Y.sub.0)=(D.sub.x, D.sub.y),
where (D.sub.x, D.sub.y) are constants calculated by the following
equations since the input image should be disposed at the center of the
main image memory 28 as shown in FIG. 8, which is written by the parameter
dividing portion 73 when the reset signal in the synchronizing signals 30
is turned into active:
D.sub.x =int{(XM-XI)/2};
D.sub.y =int{(YM-YI)/2},
where int {a} is a maximum integer not exceeding a,
XM and YM are numbers of pixels in the x- and the y-directions of the main
image memory 28, and
XI and YI are numbers of pixels in the x- and the y-directions of the input
image memory 23.
A vector buffer 76 is temporary memory means for storing in the form of
table as shown in FIG. 30 coordinate values of representative points and
motion vector values at the respective representative points, which
comprises a semiconductor memory and peripheral circuits. Writing in and
reading out of the vector buffer 76 are carried out by the vector
detecting portion 71, and the coefficient calculating portion 72 reads
data out of the vector buffer 76.
The vector detecting portion 71 compares image data in the main image
memory 28 with image data in the input image memory 23 to obtain motion
vectors at a plurality of representative points set in an input image.
The following is the description of a representative point determining
portion. In the present embodiment, K of representative points are taken
in an input image, and coordinates of the representative points are
defined as
(x.sub.i, y.sub.i),
where i=0 to (K-1),
0.ltoreq.x.sub.i <XI; 0.ltoreq.y.sub.i <YI.
The setting of the plural representative points may be conducted as follows
in the input image: as shown in FIG. 9a, they are determined at a period
preliminarily determined in the horizontal and the vertical directions; or
as shown in FIG. 9b, they may be determined in each input image as to be
concentrated in the edge portion and in regions having locally large
statistical pixel values.
In case that the representative points are periodically set as shown in
FIG. 9a, coordinates of the representative points are determined for
example by the following equations. Supposing there are representative
points of Kx in the x-direction and of Ky in the y-direction at intervals
of I.sub.dx and I.sub.dy pixels, a coordinate of i-th representative point
(x.sub.i, y.sup.i) is obtained by the following equations.
x.sub.i =I.sub.dx x{1+(i mod K.sub.x)};
y.sub.i =I.sub.dy x{1+int(i/K.sub.y)},
where
I.sub.dx =int{XI/(K.sub.x +1)},
I.sub.dy =int{YI/(K.sub.y +1)},
K.sub.x and K.sub.y are numbers of representative points in the x-direction
and in the y-direction, which are integers satisfying K.sub.x
.times.K.sub.y =K, and
(a mod b) shows a residue when a is divided by b.
These coordinates of the representative points are independent of the input
image, and, therefore, the coordinates of the K representative points are
preliminarily written in a section of "representative point numbers" in
the above vector buffer.
In case that the coordinates of the representative points vary depending
upon the input image as shown in FIG. 9b, the coordinates of the
representative points have to be determined for example by the following
process.
(1) An image in the input image memory is divided into blocks of (K.sub.x
.times.K.sub.y) as shown in FIG. 10a. In this division, the number
(K.sub.x .times.K.sub.y) of blocks is greater than the number K of
necessary representative points.
(2) Statistical values of pixel data (for example either a difference
between the maximum and the minimum or the dispersion) are calculated in
all blocks.
(3) K blocks are selected from the largest statistical value calculated in
above (2), and a coordinate of the center of a selected block is set as a
coordinate of i-th representative point.
The following means is used to achieve the above processing method. In
detail, a left upper coordinate (B.sub.xj, B.sub.yj) of the j-th block is
obtained by the following equations in the block division in above (1).
B.sub.xj =(j mod K.sub.x).times.I.sub.dx ;
B.sub.yj =int(j/K.sub.y).times.I.sub.dy,
where
I.sub.dx =int(XI/K.sub.x),
I.sub.dy =int(YI/K.sub.y),
K.sub.x and K.sub.y are numbers of blocks in the x-direction and in the
y-direction, which are integers satisfying K.sub.x .times.K.sub.y >K.
Since these coordinates are independent of an input image, the vector
detecting portion 71 may be arranged to have a circuit for outputting the
block left upper coordinate (B.sub.xj, B.sub.yj) when the block number j
is input, such as a lookup table using a semiconductor memory or as a
circuit for conducting the above calculation.
The following means is used for the calculation of statistical values in a
block in above (2). In detail, the left upper coordinate (B.sub.xj,
B.sub.yj) is obtained for the j-th block in above (1), and a statistical
value S.sub.j for each block is obtained by the following process as
described in C language. As an example, a process is shown to obtain a
difference between the maximum value and the minimum value in a block as
the statistical value S.sub.j.
______________________________________
for (j=0; j<(K.sub.x * K.sub.y) ; j ++)
{ x=B.sub.xj ;
y=B.sub.yj ;
max=0;
min=GI;
for (n=0; n<K.sub.y ; n++)
{ for (m=0; m<K.sub.x ; m++)
{ a=I (x+m, y+n) ;
if (a<max) max=a;
if (a<min) min=a;
}
}
S[j]=max-min;
}
______________________________________
provided that x, y, max, min, and S[j] are integer type variable buffers.
In the above program, "a=b" represents a writing process in which the right
side value b is written in the left side buffer a.
Also, B.sub.xj and B.sub.yj represent processes to obtain a left upper
coordinate of the j-th block by the means as shown in above (1).
Further, I (m, n) means a process to output a coordinate value (m, n)
through a signal line 713 to the input image memory 23 and then to obtain
a pixel value corresponding to the coordinate in the input image memory 23
through a signal line 714.
The following means is used for selection of the representative points in
above (3).
[I] An array buffer SS [i] is made by sorting S [j] from the largest. As a
result, a value of j having the i-th largest statistical value S [j] is
stored in SS [i].
[II] Selected numbers of representative points are from i=0 to | | |
