 |
Claims  |
|
|
What is claimed is:
1. An image combining apparatus for combining a first image and a second image having an overlapping region which overlaps with the first image, comprising:
identifying means for discriminating the overlapping region within the first and second images, and identifying a first partial image corresponding to the overlapping region of the first image and a second partial image corresponding to the
overlapping region of the second image;
detecting means for detecting corresponding pixels between the first partial image and the second partial image, and detecting tone densities of the first and second partial images on the basis of detected corresponding pixels;
tone correction means for performing tone correction on at least one of the corresponding pixels of the first partial image and those of the second partial image on the basis of a difference in tone density of the first partial image and the
second partial image so that the tone density of the at least one partial image approximates to the other; and
combining means for combining the first image and the second image, substituting image data in the overlapping region with image data indicative of the corrected first partial image and the second partial image.
2. The image combining apparatus according to claim 1, wherein said detecting means comprises calculating means for calculating a correction function which prescribes a correlation between the tone density of the first partial image and the tone
density of the second partial image, and said tone correction means performs tone correction on either the first image or the second image utilizing the calculated correction function.
3. The image combining apparatus according to claim 2, wherein the correlation is approximated by a quadratic function.
4. The image combining apparatus according to claim 1, wherein said combining means adds weights to at least one of the first and second images corrected by said tone correction means, adds the weighted image to the original image, and combines
the added image with the other image.
5. The image combining apparatus according to claim 1, further comprising:
setting means for setting a boundary at a substantial center of the overlapping region, wherein said combining means performs correction on pixels of the second partial image in correspondence with a distance from the boundary to the pixel.
6. The image combining apparatus according to claim 1, wherein said combining means sets a tone conversion area in the overlapping region.
7. The image combining apparatus according to claim 5, wherein said combining means performs tone conversion only on the second partial image.
8. The image combining apparatus according to claim 5, wherein said combining means performs tone conversion on the first partial image and the second partial image.
9. The image combining apparatus according to claim 5, wherein said combining means changes a size of the tone conversion area in accordance with a difference in density of the first partial image and density of the second partial image.
10. The image combining apparatus according to claim 5, wherein said combining means sets the tone conversion area in accordance with a difference in each average value of the first image and the second image within the overlapping region.
11. The image combining apparatus according to claim 1, wherein said identifying means determines mapping of coordinates conversion from a space of the first partial image to a space of the second partial image, and
said combining means utilizes inverse transformation of said mapping to map the second partial image into the space of the first partial image, thereby combining the first image and the second image.
12. An image combine method of combining a first image and a second image having an overlapping region which overlaps with the first image, comprising the steps of:
discriminating the overlapping region within the first and second images to identify a first partial image corresponding to the overlapping region of the first image and a second partial image corresponding to the overlapping region of the second
image;
detecting means for detecting corresponding pixels between the first partial image and the second partial image, and detecting tone densities of the first and second partial images on the basis of detected corresponding pixels;
performing tone correction on at least one of the corresponding pixels of the first partial image and those of the second partial image on the basis of a difference in tone density of the first partial image and the second partial image so that
the tone density of the at least one partial image approximates to the other; and
combining the first image and the second image, with substitution of image data in the overlapping region with image data indicative of the corrected first partial image and the second partial image.
13. The apparatus according to claim 1, wherein said identifying means comprises:
corresponding-point-extracting means for extracting a set of corresponding points from the first image and the second image; and
removing means for removing from the set of corresponding points, data indicative of corresponding points which are erroneously detected as corresponding points by said corresponding-point-extracting means, on the basis of coordinates conversion
parameters estimated by utilizing the set of corresponding points extracted by said corresponding-point-extracting means,
wherein said combining means estimates image combine parameters on the basis of the set of corresponding points from which erroneous corresponding points have been removed.
14. The image combining apparatus according to claim 13, wherein said tone correction means extracts pixel values of the set of corresponding points, from which the erroneous corresponding points have been removed, respectively from the first
image and the second image on the basis of the estimated image combine parameters, and performs tone correction respectively on the first and second images utilizing the extracted pixel values.
15. The image combining apparatus according to claim 13,
wherein said tone correction means generates a first correction function for performing tone correction on the first image on the basis of a difference between an average value of the pixel values extracted from the first image and an average
value of the pixel values extracted from the second image,
performs tone correction on the first image utilizing the fist correction function,
generates a second correction function to coincide the pixel values of the corresponding points in the second image from which the erroneous corresponding points have been removed by said removing means, with the pixel values of the corresponding
points in the corrected first image from which the erroneous corresponding points have been removed, and
performs tone correction on the second image utilizing the second correction function.
16. The image combine method according to claim 12, wherein said identifying step comprises the steps of:
extracting a set of corresponding points from the first image and the second image; and
removing, from the set of corresponding points, data indicative of corresponding points which are erroneously detected as corresponding points in said extracting step, on the basis of coordinates conversion parameters estimated by utilizing the
set of corresponding points extracted in said extracting step,
wherein in said combining step, image combine parameters are estimated on the basis of the set of corresponding points from which erroneous corresponding points have been removed.
17. The image combine method according to claim 16, wherein said step of performing tone correction comprises the steps of:
extracting pixel values of the set of corresponding points, from which the erroneous corresponding points have been removed, respectively from the first image and the second image on the basis of the estimated image combine parameters; and
performing tone correction respectively on the first and second images utilizing the extracted pixel values.
18. The image combine method according to claim 16, wherein said step of performing tone correction comprises the steps of:
generating a first correction function for performing tone correction on the first image on the basis of a difference between an average value of the pixel values extracted from the first image and an average value of the pixel values extracted
from the second image;
performing tone correction on the first image utilizing the fist correction function,
generating a second correction function to coincide the pixel values of the corresponding points in the second image from which the erroneous corresponding points have been removed in said removing step, with the pixel values of the corresponding
points in the corrected first image from which the erroneous corresponding points have been removed, and
performing tone correction on the second image utilizing the second correction function. |
|
 |
|
 |
Claims |
|
|
|
Description |
|
|
BACKGROUND OF THE INVENTION
The present invention relates to an apparatus and a method for combining images, and more particularly, to an image combining apparatus and method thereof for combining a plurality of images partially including an overlapping region having the
same image, to generate one panoramic image having a wide angle of view.
According to the conventional method of generating a panoramic image having a wide angle of view by combining a plurality of images partially having an overlapping region of the same image, two images are combined on a plane by geometrical
transformation such as Affin transformation or the like to coincide coordinates values of two corresponding points, which are extracted from the overlapping regions having the same image but having different coordinates values.
However, in a case where conditions, particularly exposure conditions, of photographing the plurality of images are different for each image due to some factors of a photographing subject, even if the two images are combined precisely with the
corresponding points being coincided according to the conventional method, the boundary of the images would have a conspicuous line due to the difference in lightness of the images.
The disadvantage of the conventional image combine technique is explained with reference to FIGS. 1, 2A, 2B and 3.
It is assumed herein that an image of a subject shown in FIG. 1 is picked up by an electronic still camera or the like, taking two frames (frame fl and frame f2) of photographs. Since the subject picked up by the frame fl has a dark atmosphere
as a whole, the camera corrects an exposure amount at the time of image pick-up such that the dark portion would be lighter. As a result, an image shown in FIG. 2A is obtained. Since the subject picked up by the frame f2 has a light atmosphere as a
whole, the camera corrects an exposure amount at the time of image pick-up such that the light portion would be darker. As a result, an image shown in FIG. 2B is obtained. Accordingly, even if the two inputted images (images in FIGS. 1A and 1B) are
combined precisely, the combined image shown in FIG. 3 would have a conspicuous line due to the difference in lightness.
SUMMARY OF THE INVENTION
The present invention has been made in consideration of the above situation, and has as its object to provide an image combining apparatus and method capable of combining images in a manner such that the boundary of the images is inconspicuous
even in a case where exposure conditions are different for each inputted image.
Another object of the present invention is to provide an image combining apparatus and method which can generate a combined image whose boundary of the images is inconspicuous, by discriminating and identifying an overlapping region having the
same image in two inputted images, and correcting tones of either or both of the inputted images in accordance with difference in tone density between the two inputted images.
Still another object of the present invention is to provide an image combining apparatus and method which can generate a combined image whose boundary of the images is inconspicuous, by performing tone correction particularly in the neighbor of
the boundary.
Still another object of the present invention is to provide an image combining apparatus and method which can generate a combined image whose boundary of the images is inconspicuous, by performing tone correction on pixels in the neighbor of the
boundary in the overlapping region, in accordance with how far the pixel of interest is from the boundary.
Still another object of the present invention is to provide an image combining apparatus and method which can generate a combined image whose boundary of the images is inconspicuous, by removing erroneously recognized corresponding points from a
set of corresponding points which have been detected to identify the overlapping region of the images, and by determining parameters of tone correction on the basis of the set of corresponding points from which the erroneous corresponding points are
removed, thereby increasing precision of tone correction.
According to a preferred embodiment of the present invention, since a coefficient of weighting is set in accordance with how far the pixel of interest is from the boundary of the images, it is possible to obtain a combined image on which smooth
tone conversion has been performed.
According to a preferred embodiment of the present invention, since the tone conversion is performed only within a predetermined area, the processing time can be reduced, moreover it is possible to obtain a combined image which is consistent with
the image before conversion.
According to a preferred embodiment of the present invention, since an area to be subjected to tone conversion is determined in accordance with the image in the overlapping region of the inputted images, tone correction and combine processing
appropriate for the image are performed, thus possible to obtain a high-quality combined image.
According to a preferred embodiment of the present invention, since an area to be subjected to tone conversion is determined in accordance with difference in average values of the images in the overlapping region of the inputted images, it is
possible to obtain a combined image on which appropriate and smooth tone conversion have been performed.
Other features and advantages of the present invention will be apparent from the following description taken in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts throughout the
figures thereof.
BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention, and together with the description, serve to explain the principles of the invention.
FIG. 1 is a view showing image pick-up conditions of inputted images according to the conventional example and the present embodiment;
FIGS. 2A and 2B show examples of the input image;
FIG. 3 shows an image combined in the conventional method;
FIG. 4 is a block diagram showing an arrangement of an image combining apparatus according to the first embodiment of the present invention;
FIG. 5 is a flowchart showing a process algorithm performed by a corresponding-point-extracting unit according to the first embodiment;
FIG. 6A is a view illustrating a method of extracting a template utilized to extract corresponding points from inputted images;
FIG. 6B is a view illustrating a method of extracting a template utilized to extract corresponding points in a case where inputted images are arranged one on top of the other;
FIGS. 7A and 7B are views illustrating a method of setting a searching objective area for extracting corresponding points;
FIG. 8 is a flowchart showing a process algorithm of tone correction;
FIG. 9 is a graph showing characteristics of the relationship of sample data;
FIG. 10 shows a table which stores values of conversion functions for a tone converting unit;
FIG. 11 is a flowchart showing a process algorithm performed by an image combining unit;
FIG. 12 is an explanatory view of the image combining method;
FIG. 13 is an explanatory view of the image combining method;
FIG. 14 shows a combined image;
FIG. 15 is a graph showing variance in density in the combined image shown in FIG. 14;
FIG. 16 is an explanatory view for explaining extraction of sample data according to a modified embodiment of the first embodiment;
FIG. 17 is a graph showing characteristics of tone conversion according to a second modified embodiment of the first embodiment;
FIG. 18 is a graph showing characteristics of image combining process according to the second modified embodiment;
FIG. 19 is a block diagram showing the arrangement of the image combining apparatus and the process flow of an inputted image according to the second embodiment;
FIG. 20 is an explanatory view showing a point Pi in an image a and a corresponding point Qi in an image b, according to the second embodiment;
FIG. 21 is a flowchart showing the steps of tone correction processing performed by a correction-coefficient-determining unit and tone converting units according to the second embodiment;
FIG. 22 is an explanatory view showing sample data obtained to determine tone correction coefficients;
FIGS. 23A and 23B are graphs explaining the technique of tone correction according to the second embodiment;
FIG. 24 shows a table utilized for converting pixel values in the image b to pixel values in the image a, according to the second embodiment;
FIGS. 25A-25C are explanatory views explaining the steps of removing erroneous corresponding components by a corresponding-point-selecting unit, according to the third embodiment;
FIGS. 26A and 26B are graphs for explaining the steps of determining tone correction coefficients by a correction-coefficient-determining unit according to a third embodiment; and
FIG. 27 is an explanatory view of obtaining sample data in the fourth embodiment.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
Preferred embodiments of the present invention will be described in detail in accordance with the accompanying drawings.
<First Embodiment>
FIG. 4 shows the arrangement of an image combining apparatus 1000 according to the first embodiment of the present invention.
The combine apparatus 1000 may be a single apparatus, or may be built in a camera or the like. The apparatus serves to input two input images a and b, and to output a combined image c.
Referring to FIG. 4, input images a and b are picked up by an electronic still camera or a video camera or the like in a manner such that both images partially include an overlapping region having the same image. Even if the input images a and b
are picked up with different exposure conditions such as those shown in FIGS. 2A and 2B (e.g. one image picked up in a dark condition, and the other in a light condition), the combine apparatus 1000 outputs a combined image c whose boundary of the images
is inconspicuous.
The combine apparatus 1000 includes a corresponding-point-extracting unit 1, parameter estimating unit 2, correction-coefficient-determining unit 3, two tone converting units 4 and 5, and image combining unit 6.
The corresponding-point-extracting unit 1 extracts corresponding points from the two input images a and b. Herein, corresponding points are each of the respective points in two images including an identical subject, which are picked up in
separate image pick-up operation by the same light source. The corresponding points are outputted in the form of vectors. The corresponding-point-extracting unit 1 identifies an overlapping region of images a and b.
A parameter estimating unit 2 estimates parameters used for image conversion for the purpose of combining the two images, on the basis of the corresponding-points vectors extracted by the extracting unit 1.
A correction-coefficient-determining unit 3 determines coefficients used for tone correction performed on each of the input images a and b. The correction coefficients are determined on the basis of image data in the overlapping region of the
input images a and b.
Tone converting units 4 and 5 respectively perform tone correction on the input images a and b such that lightness and tonality of every colors are equal in the overlapping region of the two images.
An image combining unit 6 converts the input images a and b, whose lightness has been corrected, utilizing the conversion parameters and combines the two images into a single image.
A control unit 7 controls the entire image combining apparatus.
Corresponding Point Extraction by Unit 1
Description will now be provided on the operation for generating a combined image c based on the input images a and b. Herein, description will be provided in a case where the input images a and b have N tones of density image data.
The corresponding-point-extracting unit 1 extracts corresponding points from the input images a and b to identify an overlapping region. The process algorithm performed by the corresponding-point-extracting unit 1 is shown in FIG. 5. The
algorithm shown in FIG. 5 is executed to perform template matching and extract corresponding points (i.e. overlapping region).
In step S1, the corresponding-point-extracting unit 1 determines a region which includes a template for performing template matching (hereinafter referred to as a template region). Since the frames of the input images a and b are arbitrarily
set, the overlapping region cannot be determined in advance. Thus, the extracting unit 1 sets a predetermined region as a template region.
The predetermined region can be determined in various ways.
It is assumed in the first embodiment that the image of the frame fl is photographed first and the image of the frame f2 is photographed next. Therefore, the input image a is arranged in the left side and the input image b, in the right side.
In other words, an overlapping region of the two images should be the right portion of the input image a and the left portion of the input image b. Accordingly, a predetermined region T is set as shown in FIG. 6A. The region T is located in the right
portion of the image a. The horizontal length of the region T is 30% of the horizontal length of the image a, and a vertical length of the region T is 80% of the vertical length of the image a. Note that the region T is located in the center of the
vertical length of the image a, with the right end thereof adjoining to the right end of the image a.
Note that in a case where the input images a and b are arranged vertically, one on top of the other, the template region T is set as shown in FIG. 6B. More specifically, the region T is located at the lower portion of the image a. The vertical
length of the region T is 30% of the vertical length of the image a, and the horizontal length of the region T is 80% of the horizontal length of the image a. Note that the region T is located in the center of the horizontal length of the image a, with
the lower end adjoining to the lower end of the image a.
In the process of extracting corresponding points as shown in FIG. 5, matching process is repeated for a plurality of times with respect to the region T. A region (hereinafter referred to as a searching template) subjected to single matching
processing is 1/24 (=1/3.times.1/8) of the region T. A single searching template is indicated by "t" in FIG. 6A.
Steps S2 to S4 which will be described below are performed with respect to the region T extracted in step Si.
In step S2, the corresponding-point-extracting unit 1 determines a searching objective area S in the input image b, and within the area S, sets a searched area s which corresponds to the searching template t designated in the image a. The
searching objective area S is set in the similar manner to the setting of the region T. That is, the searching objective area S as well as the searching template t are determined as shown in FIGS. 7A and 7B based on an assumption that, when a user takes
photographs for generating a panoramic image, the user would set the frames fl and f2 such that no more than 50% of the input image a overlaps the input image b in the horizontal direction, and no more than .+-.10% of the image a deviates in the vertical
direction with the image b. More specifically, the template t which is a searched area s is set to have a size of 10 % in the vertical direction and less than 50 % (preferably, 40 %) in the horizontal direction, of the image a.
Note that if an estimated overlapping condition of the input images a and b is different, the setting of the searching objective area for extracting corresponding points may be changed.
The area S shown in FIG. 7B is the searching objective area S set in correspondence with the template t shown in FIG. 7A. The extracting unit 1 identifies a corresponding point in step S3. More specifically, the extracting unit 1 parallelly
moves the template t set in the image a within the searching objective area S set in the area b, i.e. the area s is moved within the area S. The extracting unit 1 then calculates a summation of absolute values of difference between all the image data
included in the searching template t and all the image data included in the area s. The extracting unit 1 determines a position in the image b having the minimum summation .SIGMA.D of the absolute values of the differences, as a corresponding point of
the template t.
In step S4, the corresponding-point-extracting unit 1 determines reliability of the result of the corresponding points detected in step S3. The determination of reliability is made on the basis of the minimum value .SIGMA.D.sup.1.sub.min and the
second minimum value .SIGMA.D.sup.2.sub.min of the summation .SIGMA.D of the absolute values of the differences. That is, assuming that Th.sub.1 and Th.sub.2 are the respective predetermined threshold values, the detected corresponding point is
determined to be reliable when the following equations are satisfied:
Then the corresponding-point-extracting unit 1 stores coordinates of the corresponding points for the images a and b respectively in a memory (not shown).
Note that in order to detect positions of the corresponding points, besides from the above technique where the position having the minimum summation .SIGMA.D of absolute values of the differences is determined as a corresponding point, the
extracting unit 1 may calculate a correlation and determine a position having the largest correlation value as the corresponding point.
In addition, the corresponding position may be determined by a user. For instance, a user may designate an identical point in both images with a cursor or the like to be extracted, by referring to the two images a and b displayed on a display.
In the foregoing manner, the overlapping region of the images a and b are determined by repeatedly performing the control steps described in FIG. 5.
Determining Coordinates Conversion Parameter by Estimating Unit 2
The parameter estimating unit 2 estimates parameters for coordinates conversion on the basis of the extracted corresponding points from the overlapping region. Herein, coordinates conversion is the operation to coincide the overlapping region of
the image a with the overlapping region of the image b. In the first embodiment, Affin transformation is employed.
Assuming that the image b is rotated for .theta..degree. with respect to the position of the image a, parallelly moved for a distance dx in the direction of X and a distance dy in the direction of Y, and is enlarged m times as the image a, an
arbitrary point (x.sub.a, y.sub.a) in the image a corresponds to the point (x.sub.b, y.sub.b) in the image b defined by the following equation (3):
The parameter estimating unit 2 estimates the parameters A, B, C and D employing the least squares method. In order to estimate the parameters using the least squares method, at least two pairs of corresponding points are necessary.
Where only one pair of corresponding points is searched, the estimating unit 2 cannot perform matching processing since the rotation component .theta. cannot be determined. In such case, the estimating unit 2 assumes .theta.=0 and m=1, which
implies that the image is not rotated. In other words, A and B are set to 1 and 0, respectively. Thus, assuming that the corresponding points vectors is denoted as (a.sub.x, a.sub.y), the unit 2 outputs the following parameters:
In a case where no corresponding points is obtained, the subsequent processing will not be performed; instead, for instance, a message or the like is outputted to a CRT and the processing ends. The parameters obtained in the above manner are
used when an overlapping region is estimated.
Tone Conversion
In order to generate a combined image whose boundary of the images is inconspicuous, the tone converting units 4 and 5 perform tone conversion on the inputted images a and b respectively so that the lightness and tonality of colors (R, G and B)
are equal in the overlapping region of the two images. The correction-coefficient-determining unit 3 determines correction coefficients for the tone conversion processing.
Meanwhile, the tone conversion on lightness components is made in the embodiment. Where the embodiment adopts color images, tone correction on each color (R, G and B) may be performed. Specifically, a tone conversion using individual tone
correction function is made on each of R, G and B images.
FIG. 8 shows the process algorithm performed by the correction-coefficient-determining unit 3 and the tone converting units 4 and 5. The correction-coefficient-determining unit 3 inputs parameters A, B, C and D estimated by the estimating unit 2
for coordinates conversion, and the images a and b. The tone converting units 4 and 5 respectively input the images a and b.
In step S11 of FIG. 8, the correction-coefficient-determining unit 3 determines whether or not each pixel of the input images a and b is within the overlapping region. For determination, coordinates values of each pixel in the input images are
subjected to Affin transformation according to the equation (3), utilizing the parameters A, B, C and D. Then determination is made as to whether or not the coordinates values on which Affin transformation has been performed are within the area
determined to be the overlapping region of the image b.
Next, in step S12 of FIG. 8, the correction-coefficient-determining unit 3 takes samples of image data for all or a predetermined number of pixels which are determined to be included in the overlapping region. Herein, it is assumed that N number
of sample pixels are obtained. In other words, pixel values P.sub.a (k) and P.sub.b (k) (k=1 to N) are obtained. The following equation (5) can be obtained if Affin transformation in equation (3) is expressed simply by matrix H:
P.sub.a (x, y) represents a pixels value at (x, y) coordinate position which is not subjected to the Affin transformation. P.sub.b (x', y') represents a pixels value at (x, y) coordinate position which corresponds to the location (x', y') that
has been subjected to the Affin transformation. S of the equation (5) represents a function which performs tone conversion processing in pixels which is accompanied with the above coordinate transformation. The tone conversion processing will be
described in detail later.
Note that the correction-coefficient-determining unit 3 may perform the sampling of pixel values pixel by pixel, or for every arbitrary number of pixels. Moreover, the correction-coefficient-determining unit 3 may utilize, as sample data, an
average value of the neighboring pixel values based on the coordinates of the corresponding points obtained by utilizing the parameters.
The correction-coefficient-determining unit 3 then obtains tone correction coefficients in step S13 of FIG. 8 on the basis of the sample data P.sub.a (k) and P.sub.b (k).
FIG. 9 shows the brief method of calculating tone correction coefficients.
Referring to FIG. 9, the abscissa indicates a density value of the sample pixel data P.sub.b (k) of the image b; and the ordinate, a density value of the sample pixel data P.sub.a (k) of the image a. In step S13 of FIG. 8, the
correction-coefficient-determining unit 3 generates a conversion function (function f900 in FIG. 9) on the basis of the above sample data, to coincide the pixel density value of one image (e.g. image a) to the pixel density value of the other image (e.g.
image b).
Experimentally speaking, density values of the images a and b have a distribution similar to a quadratic function. Thus, to convert a pixel value in the overlapping region of the image b to a pixel value in the overlapping region of the image a,
the following quadratic function will be employed.
where T.sub.b1, T.sub.b2 and T.sub.b3 are coefficients.
In step S13 of FIG. 8, the correction-coefficient-determining unit 3 obtains the coefficients T.sub.b1, T.sub.b2 and T.sub.b3 to generate the f(P.sub.b) in equation (6). To obtain these values, the correction-coefficient-determining unit 3
calculates T.sub.b1, T.sub.b2 and T.sub.b3 which minimizes an evaluation function .epsilon. expressed by the following equation:
The correction-coefficient-determining unit 3 supplies the tone converting units 4 and 5 with the calculated coefficients T.sub.b1, T.sub.b2 and T.sub.b3. Note that since the correction-coefficient-determining unit 3 calculates the coefficients
to coincide the pixel values of the image b with the pixel values of the image a in the first embodiment, the tone correction coefficients T.sub.a1, T.sub.a2 and T.sub.a3 for the image a are respectively, T.sub.a1 =0, T.sub.a2 =1 and T.sub.a3 =0.
Next, the tone converting units 4 and 5 convert pixel values of each of the images a and b in accordance with the tone correction coefficients T.sub.b1, T.sub.b2 and T.sub.b3 in step S14 of FIG. 8. Hereinafter, the operation performed by the
tone converting unit 5 will be described.
The tone converting unit 5 converts tones of the image b into tones of the image a. The tone converting unit 5 generates a table for converting the tones of the image b to the tones of the image a on the basis of the tone correction coefficients
T.sub.b1, T.sub.b2 and T.sub.b3. When the dynamic range of an image is 8 bits, pixel values 0 to 255 of the image b have the values f(0) to f(255) in the space of the image a, according to the quadratic function f(P.sub.b) in equation (6). Examples of
f(0) to f(255) in a conversion table 910 are shown in FIG. 10.
Although the tone converting unit 4 is capable of converting density values of the image a into density values in the space of image b, pixel values of the image a do not need to be converted in the first embodiment; thus f(P.sub.a)=P.sub.a.
Therefore, the conversion table in the tone converting unit 4 converts the pixel values 0 to 255 into pixel values 0 to 255, in other words, no conversion is performed.
Note that in a case of a color image, it is preferable to perform tone conversion by generating a conversion function commonly utilized by R, G and B. Tone conversion functions for respective colors may be provided so that color matching may be
improved.
Although the quadratic function is utilized as a conversion function in the first embodiment, it is possible to utilize another form of function. Needless to say, it is also possible to perform tone conversion by utilizing a non-linear table.
Combining Images
Image combining unit 6 generates a single combined image, on the basis of the input images and the images on which the tone correction has been respectively performed by the tone converting unit 4 and 5.
The image combining unit 6 generates a combined image c in accordance with an algorithm shown in FIG. 11.
In step S21, the image combining unit 6 sets an image area for the combined image c.
The area indicated by broken lines in FIG. 12 denotes the area for the combined image which has been set on the basis of a coordinates system of the input image a. Since the description of the first embodiment bases upon the coordinates of the
input image a, for the purpose of a simple explanation, the upper side and the lower side of the image a are assumed to be parallel to coordinates axis X.
The image combining unit 6 first coincides the left end of the combined image c with the left end of the image a. The image combining unit 6 converts a position of the pixel (100) at the upper right end of the image b and a position of the pixel
(101) at the lower right end of the image b respectively to positions in the coordinates system of the image a. Then, the image combining unit 6 coincides the right end of the combined image c with a position having a larger coordinates value between the
converted two positions. In FIG. 12, since the lower right end 101 has a larger coordinates value, the image combining unit 6 coincides the right end of the combined image c with the lower right end 101.
Note that for the process of converting a coordinates value of image b into a coordinates value of image a to obtain a position having a larger coordinates, the image combining unit employs the inverse transformation of Affin transformation shown
in FIG. (8).
Herein, A', B', C' and D' are parameters of the inverse transformation.
Similarly (see FIG. 12), the upper end of the combined image c is determined in the following manner. More specifically, the position of the pixel (100) at the upper right end of the image b and the position of the pixel (103) at the upper left
end are converted respectively to positions in the coordinates system of the image a. The image combining unit 6 coincides the upper end of the combined image c with a position having the smallest coordinates value among the two converted positions, and
the position of the pixel (102) at the upper left end of the image a. Furthermore, the image combining unit 6 coincides the lower end of the combined image c with a position having a larger coordinates value between the position of the pixel (101) at the
lower right end and the position of the pixel (105) at the lower left end of the image b.
The combined image area c shown in FIG. 12 is determined in the above-described manner.
In step S22 in FIG. 11, the image combining unit 6 sets a boundary of the images so that the boundary is the center of the overlapping region. Since the images a and b are arranged side by side in the first embodiment, the boundary L which has
been set in the center is indicated by broken lines L in FIG. 12. To explain more in detail, the boundary L is set parallel to vertical coordinates axis Y. In the horizontal direction, the boundary L is set so that the two images are combined at a
barycenter between the coordinates of the pixel 106 located at the lower right end of the image a and a smaller coordinates value of the two pixels 103 or 105, respectively located at the upper left end and lower left end of the image b, which have been
converted respectively to pixel positions in the coordinates system of image a.
In step S23, the image combining unit 6 repeatedly performs the processing in steps S24 and S25 on the area for the combined image c set in step S21.
The present invention has an object to generate a combined image whose density difference at the boundary of the images is inconspicuous. This object is partially attained by the tone correction in step S14 in FIG. 8. However, the tone
conversion performed in step S14 is performed independently for each of the input images a and b, therefore does not always minimize the density difference near the boundary of the images in the overlapping region. The processing performed in steps S24
and S25 further smoothes the density difference near the boundary of the images in the overlapping region.
As shown in FIG. 13, a tone conversion area 150 having a predetermined width 2W is set in steps S24 and S25 of FIG. 11, with the boundary L of the images as its center. In other words, the tone conversion area 150 is a rectangular area having a
width W respectively to the left and right of the boundary L.
In step S24, the image combining unit 6 writes pixel values of the pixels of the image a in the corresponding area of the combined image area c. With respect to those pixels included in an area 140a of the image a shown in FIG. 13 but not
included in the tone conversion area 150, the image combining unit 6 writes the pixel values of the original image a. With respect to a pixel P.sub.a in the image a included in the tone conversion area 150, the image combining unit 6 determines a tone
conversion value P'.sub.a in accordance with how far (dx.sub.a) the pixel P.sub.a is from the boundary L. The tone conversion value P'.sub.a is determined by the following equation, | | |
